2024-04-02T13:14:19.451 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-04-02T13:14:19.451 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588 --owner scheduled_pdonnell@teuthology --name pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} -- /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/orig.config.yaml 2024-04-02T13:14:19.508 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T13:14:19.606 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588 branch: wip-batrick-testing-20240402.004512 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: pdonnell@redhat.com first_in_suite: false job_id: '7635588' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240402.004512 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240402.004512 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2763 sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 sleep_before_teardown: 0 subset: 318/512 suite: fs suite_branch: wip-batrick-testing-20240402.004512 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNskywgo/DeubM9W/8LlYkUwcBurw16cuyUumgZiyDTe4tKKKoPWITtOwy6JLTYuyUBBA3+r6L3IhIgt1mbGoMY= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCEPZCnQtkekO28xEN8unj+D/VAtm8pS8j2A1KvqbTiEUQTZ2OV726p29G0WP0FPq0/zCsUr/8MvXZ2hMTUcvMc= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMP/0mwIDEySyDMRr37bT2RE0AEaGbpH6ontMNdo44iFCc0s3vvjzsh0qaea0YZuGciJKQFsQk2Cu1ROpTGvnU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-04-02_11:52:43 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-02T13:14:19.606 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa; will attempt to use it 2024-04-02T13:14:19.607 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks 2024-04-02T13:14:19.607 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-02T13:14:19.609 INFO:teuthology.task.internal:Checking packages... 2024-04-02T13:14:19.631 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '191d963ac9de7322da8aff20b567b3fa458e2ef6' 2024-04-02T13:14:19.631 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-02T13:14:19.631 INFO:teuthology.packaging:ref: None 2024-04-02T13:14:19.631 INFO:teuthology.packaging:tag: None 2024-04-02T13:14:19.631 INFO:teuthology.packaging:branch: wip-batrick-testing-20240402.004512 2024-04-02T13:14:19.631 INFO:teuthology.packaging:sha1: 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:14:19.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240402.004512 2024-04-02T13:14:19.770 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2609.g191d963a 2024-04-02T13:14:19.771 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-02T13:14:19.778 INFO:teuthology.task.internal:no buildpackages task found 2024-04-02T13:14:19.778 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-02T13:14:19.867 INFO:teuthology.task.internal:Saving configuration 2024-04-02T13:14:19.889 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-02T13:14:19.896 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T13:14:19.934 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.157021', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNskywgo/DeubM9W/8LlYkUwcBurw16cuyUumgZiyDTe4tKKKoPWITtOwy6JLTYuyUBBA3+r6L3IhIgt1mbGoMY='} 2024-04-02T13:14:19.955 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.158497', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCEPZCnQtkekO28xEN8unj+D/VAtm8pS8j2A1KvqbTiEUQTZ2OV726p29G0WP0FPq0/zCsUr/8MvXZ2hMTUcvMc='} 2024-04-02T13:14:19.977 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.155260', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMP/0mwIDEySyDMRr37bT2RE0AEaGbpH6ontMNdo44iFCc0s3vvjzsh0qaea0YZuGciJKQFsQk2Cu1ROpTGvnU='} 2024-04-02T13:14:19.977 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-02T13:14:19.984 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-02T13:14:19.984 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-02T13:14:19.984 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-02T13:14:19.984 INFO:teuthology.run_tasks:Running task console_log... 2024-04-02T13:14:20.087 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f965fc0c160>, signals=[15]) 2024-04-02T13:14:20.088 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-02T13:14:20.094 INFO:teuthology.task.internal:Opening connections... 2024-04-02T13:14:20.094 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:14:20.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:14:20.187 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-02T13:14:20.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:14:20.275 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T13:14:20.275 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:14:20.365 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-02T13:14:20.372 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-04-02T13:14:20.390 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-04-02T13:14:20.390 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-04-02T13:14:20.448 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:14:20.449 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:14:20.450 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-04-02T13:14:20.481 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-02T13:14:20.499 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-02T13:14:20.499 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-02T13:14:20.554 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-04-02T13:14:20.554 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-04-02T13:14:20.554 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:14:20.555 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:14:20.555 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-02T13:14:20.573 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-04-02T13:14:20.590 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-04-02T13:14:20.591 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-04-02T13:14:20.648 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-04-02T13:14:20.648 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-02T13:14:20.649 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-02T13:14:20.649 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-04-02T13:14:20.672 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-02T13:14:20.683 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-02T13:14:20.689 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-02T13:14:20.689 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:14:20.692 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:14:20.694 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-04-02T13:14:20.711 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-02T13:14:20.717 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-02T13:14:20.717 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:14:20.750 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:14:20.753 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-04-02T13:14:20.870 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-02T13:14:20.947 INFO:teuthology.run_tasks:Running task kernel... 2024-04-02T13:14:20.961 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-02T13:14:20.962 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-02T13:14:20.962 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-02T13:14:20.962 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:14:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:14:20.978 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-02T13:14:21.036 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-361.el9.x86_64 2024-04-02T13:14:21.036 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.14.0-361.el9.x86_64 2024-04-02T13:14:21.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum install -y kernel 2024-04-02T13:14:23.162 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-04-02T13:14:24.554 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2024-04-02T13:14:26.095 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-02T13:14:27.921 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-02T13:14:32.822 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.1 MB 00:01 2024-04-02T13:14:34.415 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-02T13:14:36.543 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-02T13:14:41.516 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-02T13:14:43.806 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-02T13:14:43.807 INFO:teuthology.orchestra.run.smithi122.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-04-02T13:14:43.900 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-02T13:14:43.905 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: kernel x86_64 5.14.0-434.el9 CentOS-BaseOS 192 k 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core x86_64 5.14.0-434.el9 CentOS-BaseOS 15 M 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules x86_64 5.14.0-434.el9 CentOS-BaseOS 34 M 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core x86_64 5.14.0-434.el9 CentOS-BaseOS 28 M 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-02T13:14:43.906 INFO:teuthology.orchestra.run.smithi122.stdout:================================================================================ 2024-04-02T13:14:43.907 INFO:teuthology.orchestra.run.smithi122.stdout:Install 4 Packages 2024-04-02T13:14:43.907 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:14:43.907 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 76 M 2024-04-02T13:14:43.907 INFO:teuthology.orchestra.run.smithi122.stdout:Installed size: 123 M 2024-04-02T13:14:43.907 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-02T13:14:44.068 INFO:teuthology.orchestra.run.smithi122.stdout:(1/4): kernel-5.14.0-434.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T13:14:45.043 INFO:teuthology.orchestra.run.smithi122.stdout:(2/4): kernel-modules-5.14.0-434.el9.x86_64.rpm 30 MB/s | 34 MB 00:01 2024-04-02T13:14:45.268 INFO:teuthology.orchestra.run.smithi122.stdout:(3/4): kernel-core-5.14.0-434.el9.x86_64.rpm 11 MB/s | 15 MB 00:01 2024-04-02T13:14:45.576 INFO:teuthology.orchestra.run.smithi122.stdout:(4/4): kernel-modules-core-5.14.0-434.el9.x86_6 18 MB/s | 28 MB 00:01 2024-04-02T13:14:45.580 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:14:45.580 INFO:teuthology.orchestra.run.smithi122.stdout:Total 46 MB/s | 76 MB 00:01 2024-04-02T13:14:45.580 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-02T13:14:45.953 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-02T13:14:45.953 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-02T13:14:46.768 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-02T13:14:46.769 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-02T13:14:49.080 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-02T13:14:49.692 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-core-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:14:49.788 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:14:52.558 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:14:52.703 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:14:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:14:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:02.153 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:31.334 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:31.334 INFO:teuthology.orchestra.run.smithi122.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-04-02T13:15:31.334 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:15:31.376 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-modules-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:31.695 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: kernel-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:31.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-5.14.0-434.el9.x86_64 1/4 2024-04-02T13:15:31.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-core-5.14.0-434.el9.x86_64 2/4 2024-04-02T13:15:31.696 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-5.14.0-434.el9.x86_64 3/4 2024-04-02T13:15:32.125 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : kernel-modules-core-5.14.0-434.el9.x86_64 4/4 2024-04-02T13:15:32.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:15:32.125 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-core-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout: kernel-modules-core-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:15:32.126 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-02T13:15:32.471 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:15:32.590 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.590 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:15:32.590 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-434.el9.x86_64 2024-04-02T13:15:32.590 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-02T13:15:32.591 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-02T13:15:32.591 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-04-02T13:15:32.591 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-04-02T13:15:32.591 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q kernel | sort -rV | head -n 1 2024-04-02T13:15:32.671 INFO:teuthology.orchestra.run.smithi122.stdout:kernel-5.14.0-434.el9.x86_64 2024-04-02T13:15:32.671 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-434.el9.x86_64 2024-04-02T13:15:32.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo rpm -qi grub2-tools 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Name : grub2-tools 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Epoch : 1 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Version : 2.06 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Release : 68.el9 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: x86_64 2024-04-02T13:15:32.714 INFO:teuthology.orchestra.run.smithi122.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Group : Unspecified 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Size : 8264254 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:License : GPLv3+ 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Packager : builder@centos.org 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Vendor : CentOS 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Summary : Support tools for GRUB. 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:Description : 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:15:32.715 INFO:teuthology.orchestra.run.smithi122.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-02T13:15:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-02T13:15:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-02T13:15:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:hardware devices. 2024-04-02T13:15:32.716 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:15:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:This subpackage provides tools for support of all platforms. 2024-04-02T13:15:32.717 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-02T13:15:32.717 INFO:teuthology.task.kernel:Updating grub on smithi122 to boot 5.14.0-434.el9.x86_64 2024-04-02T13:15:32.717 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-02T13:15:33.308 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-04-02T13:15:34.683 INFO:teuthology.orchestra.run.smithi122.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-02T13:15:34.712 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-04-02T13:15:34.715 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-04-02T13:15:34.731 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.Edri33LrvG 2024-04-02T13:15:34.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Edri33LrvG 2024-04-02T13:15:34.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.Edri33LrvG 2024-04-02T13:15:34.943 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.Edri33LrvG is 6KB 2024-04-02T13:15:34.994 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.Edri33LrvG 2024-04-02T13:15:35.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo /bin/ls /boot/loader/entries || true 2024-04-02T13:15:35.073 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-04-02T13:15:35.073 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-04-02T13:15:35.073 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-04-02T13:15:35.073 INFO:teuthology.orchestra.run.smithi122.stdout:15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64.conf 2024-04-02T13:15:35.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-434.el9.x86_64 2024-04-02T13:15:35.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-04-02T13:16:05.210 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-02T13:16:05.211 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:16:05.212 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:16:23.711 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-02T13:16:32.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:16:32.713 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:16:35.778 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-02T13:16:47.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:16:47.780 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:17:06.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-04-02T13:17:21.207 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:17:21.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:17:21.863 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-02T13:17:22.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:17:22.367 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-434.el9.x86_64"... 2024-04-02T13:17:22.368 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-04-02T13:17:22.394 INFO:teuthology.orchestra.run.smithi122.stdout:5.14.0-434.el9.x86_64 2024-04-02T13:17:22.394 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-434.el9.x86_64 vs 5.14.0-434.el9.x86_64 2024-04-02T13:17:22.394 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-02T13:17:22.394 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-02T13:17:23.395 DEBUG:teuthology.parallel:result is None 2024-04-02T13:17:23.395 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-02T13:17:23.403 INFO:teuthology.task.internal:Creating test directory... 2024-04-02T13:17:23.404 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:17:23.406 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:17:23.409 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-02T13:17:23.427 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-02T13:17:23.433 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-02T13:17:23.440 INFO:teuthology.task.internal:Creating archive directory... 2024-04-02T13:17:23.441 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:17:23.464 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:17:23.466 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-02T13:17:23.503 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-02T13:17:23.510 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-02T13:17:23.510 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:17:23.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:23.548 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-02T13:17:23.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:23.564 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:17:23.566 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:17:23.591 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-02T13:17:23.622 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.632 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.640 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.663 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.671 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.693 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-02T13:17:23.695 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-02T13:17:23.703 INFO:teuthology.task.internal:Configuring sudo... 2024-04-02T13:17:23.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:17:23.705 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:17:23.716 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-02T13:17:23.745 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-02T13:17:23.753 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-02T13:17:23.753 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:17:23.772 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:17:23.786 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-02T13:17:23.802 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:23.866 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:23.923 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.046 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:17:24.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:17:24.113 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:24.154 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:24.208 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.291 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.335 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:17:24.335 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:17:24.403 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:24.445 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-02T13:17:24.504 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.588 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-02T13:17:24.632 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:17:24.632 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-02T13:17:24.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-04-02T13:17:24.703 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-02T13:17:24.705 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-04-02T13:17:24.742 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:17:24.773 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:17:24.776 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T13:17:25.290 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-02T13:17:25.297 INFO:teuthology.task.internal:Starting timer... 2024-04-02T13:17:25.298 INFO:teuthology.run_tasks:Running task pcp... 2024-04-02T13:17:25.308 INFO:teuthology.run_tasks:Running task selinux... 2024-04-02T13:17:25.316 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-02T13:17:25.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-04-02T13:17:25.379 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-04-02T13:17:25.380 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-04-02T13:17:25.445 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-04-02T13:17:25.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-04-02T13:17:25.498 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-04-02T13:17:25.499 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-02T13:17:25.500 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-04-02T13:17:25.523 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-04-02T13:17:25.523 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-04-02T13:17:25.544 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-04-02T13:17:25.544 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-04-02T13:17:25.562 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-04-02T13:17:25.562 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi122.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-04-02T13:17:25.562 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:17:25.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:25.591 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:17:25.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:25.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T13:17:25.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:17:25.648 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-02T13:17:25.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:17:25.683 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:17:25.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-04-02T13:17:25.746 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-02T13:17:25.757 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-02T13:17:25.762 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-02T13:17:25.898 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-02T13:17:25.915 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-02T13:17:25.916 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi187.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-02T13:21:56.474 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-04-02T13:21:56.476 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:21:56.477 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:21:56.571 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-04-02T13:21:56.645 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-04-02T13:21:56.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-02T13:21:56.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:21:56.738 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-02T13:21:56.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-02T13:21:56.763 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T13:21:56.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-02T13:21:56.858 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-04-02T13:21:56.929 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-04-02T13:21:56.929 INFO:teuthology.run_tasks:Running task clock... 2024-04-02T13:21:56.939 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-02T13:21:56.939 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:21:56.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:21:56.942 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:21:56.942 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:21:56.944 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-02T13:21:56.944 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T13:21:56.971 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:21:56.973 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:21:56.987 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:21:56.990 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:21:57.001 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-02T13:21:57.016 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-04-02T13:21:57.018 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-02T13:21:57.019 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-04-02T13:21:57.030 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-04-02T13:21:57.032 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-04-02T13:21:57.047 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:21:57.048 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-04-02T13:21:57.049 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:21:57.062 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-04-02T13:21:57.063 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:21:57.065 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:21:57.078 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-02T13:21:57.095 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-02T13:21:57.108 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.111 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.113 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.117 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.139 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.143 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T13:21:57.144 INFO:teuthology.run_tasks:Running task install... 2024-04-02T13:21:57.151 DEBUG:teuthology.task.install:project ceph 2024-04-02T13:21:57.152 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'}} 2024-04-02T13:21:57.152 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6'} 2024-04-02T13:21:57.152 INFO:teuthology.task.install:Using flavor: default 2024-04-02T13:21:57.158 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-02T13:21:57.158 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-02T13:21:57.158 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:21:57.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:21:57.161 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:21:57.161 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:21:57.162 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'tag': None, 'wait_for_package': False} 2024-04-02T13:21:57.162 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:21:57.316 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:21:57.317 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:21:57.405 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:21:57.406 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:21:57.409 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/ 2024-04-02T13:21:57.409 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2609.g191d963a 2024-04-02T13:21:57.430 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:21:57.431 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:21:57.431 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:21:57.467 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:21:57.467 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:21:57.519 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:21:57.519 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:21:57.519 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:21:57.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:21:57.547 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:21:57.547 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:21:57.587 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-02T13:21:57.587 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:21:57.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-02T13:21:57.614 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-02T13:21:57.615 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/191d963ac9de7322da8aff20b567b3fa458e2ef6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-02T13:21:57.617 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:21:57.617 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:21:57.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-02T13:21:57.684 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-04-02T13:21:57.685 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-04-02T13:21:57.697 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:21:57.762 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-02T13:21:57.764 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-04-02T13:21:57.764 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-04-02T13:21:57.829 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-04-02T13:21:57.831 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-04-02T13:21:57.920 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-04-02T13:21:57.947 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:21:57.998 INFO:teuthology.orchestra.run.smithi187.stdout:74 files removed 2024-04-02T13:21:58.025 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:21:58.071 INFO:teuthology.orchestra.run.smithi122.stdout:74 files removed 2024-04-02T13:21:58.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-02T13:21:58.707 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-04-02T13:21:58.766 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 185 kB/s | 83 kB 00:00 2024-04-02T13:21:58.925 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 157 kB/s | 83 kB 00:00 2024-04-02T13:21:59.249 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-02T13:21:59.283 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-02T13:21:59.426 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-02T13:21:59.600 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 6.6 kB/s | 1.9 kB 00:00 2024-04-02T13:21:59.616 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-02T13:21:59.843 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-02T13:22:00.052 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-02T13:22:00.086 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-04-02T13:22:00.304 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-04-02T13:22:01.487 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-04-02T13:22:01.536 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-04-02T13:22:01.755 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-04-02T13:22:02.577 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-02T13:22:03.286 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-04-02T13:22:03.534 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-04-02T13:22:04.478 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-02T13:22:05.262 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-02T13:22:06.452 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-04-02T13:22:09.189 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-04-02T13:22:09.287 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.1 MB 00:01 2024-04-02T13:22:10.577 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.1 MB 00:00 2024-04-02T13:22:10.873 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-02T13:22:10.880 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-02T13:22:11.798 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 26 MB/s | 21 MB 00:00 2024-04-02T13:22:12.128 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-04-02T13:22:13.658 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-02T13:22:13.790 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-02T13:22:16.956 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-02T13:22:18.516 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-02T13:22:18.862 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-02T13:22:19.505 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:19.506 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:19.520 INFO:teuthology.orchestra.run.smithi187.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:22:19.521 INFO:teuthology.orchestra.run.smithi187.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:22:19.522 INFO:teuthology.orchestra.run.smithi187.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:22:19.523 INFO:teuthology.orchestra.run.smithi187.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:19.525 INFO:teuthology.orchestra.run.smithi187.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:19.528 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:22:19.529 INFO:teuthology.orchestra.run.smithi187.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:22:19.610 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-04-02T13:22:19.622 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-02T13:22:19.622 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-04-02T13:22:19.622 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-02T13:22:19.622 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:22:19.623 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:22:19.624 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:19.625 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:22:19.626 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:22:19.627 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:22:19.628 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:22:19.629 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:22:19.630 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:22:19.631 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:22:19.632 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:22:19.633 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:22:19.634 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:22:19.635 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================================ 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout:Install 150 Packages 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 24 Packages 2024-04-02T13:22:19.636 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:22:19.637 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 226 M 2024-04-02T13:22:19.637 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-04-02T13:22:20.399 INFO:teuthology.orchestra.run.smithi187.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 28 kB/s | 6.6 kB 00:00 2024-04-02T13:22:20.724 INFO:teuthology.orchestra.run.smithi187.stdout:(2/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 9.6 MB/s | 5.4 MB 00:00 2024-04-02T13:22:20.774 INFO:teuthology.orchestra.run.smithi187.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 2.8 MB/s | 145 kB 00:00 2024-04-02T13:22:20.799 INFO:teuthology.orchestra.run.smithi187.stdout:(4/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 2.2 MB/s | 890 kB 00:00 2024-04-02T13:22:20.925 INFO:teuthology.orchestra.run.smithi187.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 12 MB/s | 1.5 MB 00:00 2024-04-02T13:22:21.083 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.084 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.096 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:22:21.097 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.098 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:22:21.099 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.100 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.103 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.104 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:22:21.185 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:22:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:22:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:22:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:22:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:22:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:22:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:22:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:22:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:22:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:22:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:22:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:22:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:22:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================================ 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:Install 150 Packages 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 24 Packages 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:22:21.217 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 226 M 2024-04-02T13:22:21.218 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-04-02T13:22:21.283 INFO:teuthology.orchestra.run.smithi187.stdout:(6/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 13 MB/s | 4.7 MB 00:00 2024-04-02T13:22:21.364 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.365 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.380 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-04-02T13:22:21.381 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-02T13:22:21.382 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-04-02T13:22:21.383 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.385 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.389 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-04-02T13:22:21.390 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-04-02T13:22:21.450 INFO:teuthology.orchestra.run.smithi187.stdout:(7/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 3.5 MB/s | 2.4 MB 00:00 2024-04-02T13:22:21.470 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2609.g191d963a.el9 ceph 6.6 k 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.4 M 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 890 k 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2609.g191d963a.el9 ceph 145 k 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2609.g191d963a.el9 ceph 1.5 M 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 141 k 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 3.5 M 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 7.4 M 2024-04-02T13:22:21.483 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 49 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 9.8 M 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2609.g191d963a.el9 ceph 48 M 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 264 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 762 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 225 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2609.g191d963a.el9 ceph 27 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 18 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 32 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 747 k 2024-04-02T13:22:21.484 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2609.g191d963a.el9 ceph 127 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 CentOS-AppStream 1.5 M 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 CentOS-AppStream 817 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 CentOS-AppStream 5.7 M 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2609.g191d963a.el9 ceph 161 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2609.g191d963a.el9 ceph 320 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 299 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2609.g191d963a.el9 ceph 99 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 85 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 170 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-04-02T13:22:21.485 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.2-1.el9 CentOS-BaseOS 2.3 M 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 CentOS-BaseOS 1.8 M 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 CentOS-BaseOS 41 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 CentOS-BaseOS 247 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.4 M 2024-04-02T13:22:21.486 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 3.1 M 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-04-02T13:22:21.487 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 21 M 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 34 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2609.g191d963a.el9 ceph 2.4 M 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 249 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2609.g191d963a.el9 ceph 4.7 M 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2609.g191d963a.el9 ceph 17 M 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2609.g191d963a.el9 ceph-noarch 16 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2609.g191d963a.el9 ceph 25 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-02T13:22:21.488 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2609.g191d963a.el9 ceph 164 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 500 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2609.g191d963a.el9 ceph 5.1 M 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-02T13:22:21.489 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 CentOS-AppStream 132 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-04-02T13:22:21.490 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2609.g191d963a.el9 ceph 45 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2609.g191d963a.el9 ceph 134 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-02T13:22:21.491 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-04-02T13:22:21.492 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-04-02T13:22:21.493 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-04-02T13:22:21.494 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 CentOS-AppStream 220 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 CentOS-AppStream 46 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 CentOS-AppStream 35 k 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:22:21.495 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout:Install 150 Packages 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 24 Packages 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 226 M 2024-04-02T13:22:21.496 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-04-02T13:22:21.826 INFO:teuthology.orchestra.run.smithi122.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-02T13:22:22.321 INFO:teuthology.orchestra.run.smithi188.stdout:(1/174): ceph-19.0.0-2609.g191d963a.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-02T13:22:22.584 INFO:teuthology.orchestra.run.smithi122.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-02T13:22:22.696 INFO:teuthology.orchestra.run.smithi188.stdout:(2/174): ceph-fuse-19.0.0-2609.g191d963a.el9.x8 2.3 MB/s | 890 kB 00:00 2024-04-02T13:22:22.751 INFO:teuthology.orchestra.run.smithi122.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 870 kB/s | 145 kB 00:00 2024-04-02T13:22:22.780 INFO:teuthology.orchestra.run.smithi188.stdout:(3/174): ceph-immutable-object-cache-19.0.0-260 1.7 MB/s | 145 kB 00:00 2024-04-02T13:22:22.942 INFO:teuthology.orchestra.run.smithi187.stdout:(8/174): ceph-osd-19.0.0-2609.g191d963a.el9.x86 10 MB/s | 17 MB 00:01 2024-04-02T13:22:23.043 INFO:teuthology.orchestra.run.smithi187.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 252 kB/s | 25 kB 00:00 2024-04-02T13:22:23.347 INFO:teuthology.orchestra.run.smithi188.stdout:(4/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 4.2 MB/s | 2.4 MB 00:00 2024-04-02T13:22:23.797 INFO:teuthology.orchestra.run.smithi188.stdout:(5/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-02T13:22:24.093 INFO:teuthology.orchestra.run.smithi187.stdout:(10/174): ceph-common-19.0.0-2609.g191d963a.el9 5.3 MB/s | 21 MB 00:03 2024-04-02T13:22:24.293 INFO:teuthology.orchestra.run.smithi187.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 1.1 MB/s | 225 kB 00:00 2024-04-02T13:22:24.401 INFO:teuthology.orchestra.run.smithi187.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 250 kB/s | 27 kB 00:00 2024-04-02T13:22:24.464 INFO:teuthology.orchestra.run.smithi188.stdout:(6/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 2.1 MB/s | 5.4 MB 00:02 2024-04-02T13:22:24.493 INFO:teuthology.orchestra.run.smithi187.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 346 kB/s | 32 kB 00:00 2024-04-02T13:22:24.593 INFO:teuthology.orchestra.run.smithi122.stdout:(4/174): ceph-base-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-02T13:22:24.668 INFO:teuthology.orchestra.run.smithi122.stdout:(5/174): ceph-mds-19.0.0-2609.g191d963a.el9.x86 1.2 MB/s | 2.4 MB 00:01 2024-04-02T13:22:24.802 INFO:teuthology.orchestra.run.smithi187.stdout:(14/174): ceph-radosgw-19.0.0-2609.g191d963a.el 2.9 MB/s | 9.8 MB 00:03 2024-04-02T13:22:24.860 INFO:teuthology.orchestra.run.smithi187.stdout:(15/174): libcephfs2-19.0.0-2609.g191d963a.el9. 2.0 MB/s | 747 kB 00:00 2024-04-02T13:22:24.911 INFO:teuthology.orchestra.run.smithi187.stdout:(16/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.5 MB/s | 164 kB 00:00 2024-04-02T13:22:24.961 INFO:teuthology.orchestra.run.smithi187.stdout:(17/174): librados-devel-19.0.0-2609.g191d963a. 1.2 MB/s | 127 kB 00:00 2024-04-02T13:22:25.089 INFO:teuthology.orchestra.run.smithi188.stdout:(7/174): ceph-common-19.0.0-2609.g191d963a.el9. 6.5 MB/s | 21 MB 00:03 2024-04-02T13:22:25.120 INFO:teuthology.orchestra.run.smithi187.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 2.3 MB/s | 500 kB 00:00 2024-04-02T13:22:25.164 INFO:teuthology.orchestra.run.smithi188.stdout:(8/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-04-02T13:22:25.185 INFO:teuthology.orchestra.run.smithi122.stdout:(6/174): ceph-mgr-19.0.0-2609.g191d963a.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-04-02T13:22:25.220 INFO:teuthology.orchestra.run.smithi187.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 452 kB/s | 45 kB 00:00 2024-04-02T13:22:25.231 INFO:teuthology.orchestra.run.smithi188.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 377 kB/s | 25 kB 00:00 2024-04-02T13:22:25.320 INFO:teuthology.orchestra.run.smithi187.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.3 MB/s | 134 kB 00:00 2024-04-02T13:22:25.437 INFO:teuthology.orchestra.run.smithi187.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.3 MB/s | 161 kB 00:00 2024-04-02T13:22:25.662 INFO:teuthology.orchestra.run.smithi187.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 1.4 MB/s | 320 kB 00:00 2024-04-02T13:22:25.863 INFO:teuthology.orchestra.run.smithi187.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 1.5 MB/s | 299 kB 00:00 2024-04-02T13:22:25.980 INFO:teuthology.orchestra.run.smithi187.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 853 kB/s | 99 kB 00:00 2024-04-02T13:22:26.105 INFO:teuthology.orchestra.run.smithi187.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 684 kB/s | 85 kB 00:00 2024-04-02T13:22:26.265 INFO:teuthology.orchestra.run.smithi188.stdout:(10/174): ceph-radosgw-19.0.0-2609.g191d963a.el 8.4 MB/s | 9.8 MB 00:01 2024-04-02T13:22:26.348 INFO:teuthology.orchestra.run.smithi188.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 2.6 MB/s | 225 kB 00:00 2024-04-02T13:22:26.424 INFO:teuthology.orchestra.run.smithi188.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 360 kB/s | 27 kB 00:00 2024-04-02T13:22:26.499 INFO:teuthology.orchestra.run.smithi188.stdout:(13/174): libcephfs-devel-19.0.0-2609.g191d963a 423 kB/s | 32 kB 00:00 2024-04-02T13:22:26.658 INFO:teuthology.orchestra.run.smithi188.stdout:(14/174): libcephfs2-19.0.0-2609.g191d963a.el9. 4.6 MB/s | 747 kB 00:00 2024-04-02T13:22:27.518 INFO:teuthology.orchestra.run.smithi122.stdout:(7/174): ceph-mon-19.0.0-2609.g191d963a.el9.x86 1.7 MB/s | 4.7 MB 00:02 2024-04-02T13:22:27.519 INFO:teuthology.orchestra.run.smithi188.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 1.9 MB/s | 164 kB 00:00 2024-04-02T13:22:27.519 INFO:teuthology.orchestra.run.smithi188.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.7 MB/s | 127 kB 00:00 2024-04-02T13:22:27.519 INFO:teuthology.orchestra.run.smithi188.stdout:(17/174): libradosstriper1-19.0.0-2609.g191d963 4.9 MB/s | 500 kB 00:00 2024-04-02T13:22:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:(26/174): librgw2-19.0.0-2609.g191d963a.el9.x86 2.5 MB/s | 5.1 MB 00:02 2024-04-02T13:22:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.3 MB/s | 170 kB 00:00 2024-04-02T13:22:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 291 kB/s | 34 kB 00:00 2024-04-02T13:22:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.1 MB/s | 141 kB 00:00 2024-04-02T13:22:27.520 INFO:teuthology.orchestra.run.smithi187.stdout:(30/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 2.3 MB/s | 3.1 MB 00:01 2024-04-02T13:22:27.559 INFO:teuthology.orchestra.run.smithi188.stdout:(18/174): librgw2-19.0.0-2609.g191d963a.el9.x86 8.0 MB/s | 5.1 MB 00:00 2024-04-02T13:22:27.643 INFO:teuthology.orchestra.run.smithi188.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 543 kB/s | 45 kB 00:00 2024-04-02T13:22:27.725 INFO:teuthology.orchestra.run.smithi188.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 1.6 MB/s | 134 kB 00:00 2024-04-02T13:22:27.809 INFO:teuthology.orchestra.run.smithi188.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 1.9 MB/s | 161 kB 00:00 2024-04-02T13:22:27.901 INFO:teuthology.orchestra.run.smithi188.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 3.4 MB/s | 320 kB 00:00 2024-04-02T13:22:27.993 INFO:teuthology.orchestra.run.smithi188.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.2 MB/s | 299 kB 00:00 2024-04-02T13:22:28.076 INFO:teuthology.orchestra.run.smithi188.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.2 MB/s | 99 kB 00:00 2024-04-02T13:22:28.160 INFO:teuthology.orchestra.run.smithi188.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-02T13:22:28.660 INFO:teuthology.orchestra.run.smithi188.stdout:(26/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 6.2 MB/s | 3.1 MB 00:00 2024-04-02T13:22:28.744 INFO:teuthology.orchestra.run.smithi188.stdout:(27/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-02T13:22:28.819 INFO:teuthology.orchestra.run.smithi188.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2609.g 453 kB/s | 34 kB 00:00 2024-04-02T13:22:28.840 INFO:teuthology.orchestra.run.smithi187.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 2.5 MB/s | 3.5 MB 00:01 2024-04-02T13:22:28.895 INFO:teuthology.orchestra.run.smithi188.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 1.8 MB/s | 141 kB 00:00 2024-04-02T13:22:29.032 INFO:teuthology.orchestra.run.smithi187.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 1.3 MB/s | 249 kB 00:00 2024-04-02T13:22:29.148 INFO:teuthology.orchestra.run.smithi187.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 423 kB/s | 49 kB 00:00 2024-04-02T13:22:29.282 INFO:teuthology.orchestra.run.smithi187.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 123 kB/s | 16 kB 00:00 2024-04-02T13:22:29.474 INFO:teuthology.orchestra.run.smithi187.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 1.3 MB/s | 264 kB 00:00 2024-04-02T13:22:29.486 INFO:teuthology.orchestra.run.smithi188.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 6.0 MB/s | 3.5 MB 00:00 2024-04-02T13:22:29.874 INFO:teuthology.orchestra.run.smithi187.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 1.9 MB/s | 762 kB 00:00 2024-04-02T13:22:30.025 INFO:teuthology.orchestra.run.smithi187.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 122 kB/s | 18 kB 00:00 2024-04-02T13:22:30.225 INFO:teuthology.orchestra.run.smithi187.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 662 kB 00:00 2024-04-02T13:22:30.283 INFO:teuthology.orchestra.run.smithi187.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-04-02T13:22:30.317 INFO:teuthology.orchestra.run.smithi187.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-04-02T13:22:30.359 INFO:teuthology.orchestra.run.smithi187.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 307 kB 00:00 2024-04-02T13:22:30.393 INFO:teuthology.orchestra.run.smithi187.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 947 kB/s | 32 kB 00:00 2024-04-02T13:22:30.526 INFO:teuthology.orchestra.run.smithi187.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-02T13:22:30.614 INFO:teuthology.orchestra.run.smithi187.stdout:(44/174): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-04-02T13:22:30.643 INFO:teuthology.orchestra.run.smithi187.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 146 kB/s | 17 kB 00:00 2024-04-02T13:22:30.688 INFO:teuthology.orchestra.run.smithi187.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 138 kB/s | 10 kB 00:00 2024-04-02T13:22:30.719 INFO:teuthology.orchestra.run.smithi187.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-04-02T13:22:30.745 INFO:teuthology.orchestra.run.smithi187.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 782 kB/s | 44 kB 00:00 2024-04-02T13:22:30.770 INFO:teuthology.orchestra.run.smithi187.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-02T13:22:30.795 INFO:teuthology.orchestra.run.smithi187.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-02T13:22:30.812 INFO:teuthology.orchestra.run.smithi187.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-02T13:22:30.826 INFO:teuthology.orchestra.run.smithi122.stdout:(8/174): ceph-common-19.0.0-2609.g191d963a.el9. 2.2 MB/s | 21 MB 00:09 2024-04-02T13:22:30.838 INFO:teuthology.orchestra.run.smithi187.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-04-02T13:22:30.863 INFO:teuthology.orchestra.run.smithi187.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T13:22:30.868 INFO:teuthology.orchestra.run.smithi122.stdout:(9/174): ceph-selinux-19.0.0-2609.g191d963a.el9 606 kB/s | 25 kB 00:00 2024-04-02T13:22:30.878 INFO:teuthology.orchestra.run.smithi188.stdout:(31/174): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-04-02T13:22:30.888 INFO:teuthology.orchestra.run.smithi187.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-04-02T13:22:30.905 INFO:teuthology.orchestra.run.smithi187.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 826 kB/s | 34 kB 00:00 2024-04-02T13:22:30.945 INFO:teuthology.orchestra.run.smithi188.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 3.7 MB/s | 249 kB 00:00 2024-04-02T13:22:31.004 INFO:teuthology.orchestra.run.smithi188.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 847 kB/s | 49 kB 00:00 2024-04-02T13:22:31.063 INFO:teuthology.orchestra.run.smithi188.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 282 kB/s | 16 kB 00:00 2024-04-02T13:22:31.129 INFO:teuthology.orchestra.run.smithi188.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.9 MB/s | 264 kB 00:00 2024-04-02T13:22:31.252 INFO:teuthology.orchestra.run.smithi122.stdout:(10/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 2.8 MB/s | 17 MB 00:06 2024-04-02T13:22:31.255 INFO:teuthology.orchestra.run.smithi188.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 6.0 MB/s | 762 kB 00:00 2024-04-02T13:22:31.313 INFO:teuthology.orchestra.run.smithi188.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 313 kB/s | 18 kB 00:00 2024-04-02T13:22:31.336 INFO:teuthology.orchestra.run.smithi122.stdout:(11/174): cephfs-mirror-19.0.0-2609.g191d963a.e 2.6 MB/s | 225 kB 00:00 2024-04-02T13:22:31.339 INFO:teuthology.orchestra.run.smithi187.stdout:(56/174): ceph-test-19.0.0-2609.g191d963a.el9.x 5.7 MB/s | 48 MB 00:08 2024-04-02T13:22:31.386 INFO:teuthology.orchestra.run.smithi122.stdout:(12/174): cephfs-shell-19.0.0-2609.g191d963a.el 542 kB/s | 27 kB 00:00 2024-04-02T13:22:31.423 INFO:teuthology.orchestra.run.smithi187.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 8.9 MB/s | 4.7 MB 00:00 2024-04-02T13:22:31.465 INFO:teuthology.orchestra.run.smithi187.stdout:(58/174): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.5 MB 00:00 2024-04-02T13:22:31.495 INFO:teuthology.orchestra.run.smithi122.stdout:(13/174): ceph-radosgw-19.0.0-2609.g191d963a.el 2.4 MB/s | 9.8 MB 00:04 2024-04-02T13:22:31.498 INFO:teuthology.orchestra.run.smithi187.stdout:(59/174): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 817 kB 00:00 2024-04-02T13:22:31.511 INFO:teuthology.orchestra.run.smithi122.stdout:(14/174): libcephfs-devel-19.0.0-2609.g191d963a 253 kB/s | 32 kB 00:00 2024-04-02T13:22:31.523 INFO:teuthology.orchestra.run.smithi187.stdout:(60/174): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 132 kB 00:00 2024-04-02T13:22:31.562 INFO:teuthology.orchestra.run.smithi122.stdout:(15/174): libcephsqlite-19.0.0-2609.g191d963a.e 3.2 MB/s | 164 kB 00:00 2024-04-02T13:22:31.634 INFO:teuthology.orchestra.run.smithi122.stdout:(16/174): librados-devel-19.0.0-2609.g191d963a. 1.7 MB/s | 127 kB 00:00 2024-04-02T13:22:31.687 INFO:teuthology.orchestra.run.smithi122.stdout:(17/174): libcephfs2-19.0.0-2609.g191d963a.el9. 3.8 MB/s | 747 kB 00:00 2024-04-02T13:22:31.746 INFO:teuthology.orchestra.run.smithi122.stdout:(18/174): libradosstriper1-19.0.0-2609.g191d963 4.4 MB/s | 500 kB 00:00 2024-04-02T13:22:31.747 INFO:teuthology.orchestra.run.smithi188.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2024-04-02T13:22:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.7 MB 00:00 2024-04-02T13:22:31.796 INFO:teuthology.orchestra.run.smithi122.stdout:(19/174): python3-ceph-argparse-19.0.0-2609.g19 907 kB/s | 45 kB 00:00 2024-04-02T13:22:31.825 INFO:teuthology.orchestra.run.smithi187.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-04-02T13:22:31.850 INFO:teuthology.orchestra.run.smithi187.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 240 kB/s | 78 kB 00:00 2024-04-02T13:22:31.855 INFO:teuthology.orchestra.run.smithi122.stdout:(20/174): python3-ceph-common-19.0.0-2609.g191d 2.3 MB/s | 134 kB 00:00 2024-04-02T13:22:31.889 INFO:teuthology.orchestra.run.smithi188.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 6.4 MB/s | 920 kB 00:00 2024-04-02T13:22:31.913 INFO:teuthology.orchestra.run.smithi122.stdout:(21/174): python3-cephfs-19.0.0-2609.g191d963a. 2.7 MB/s | 161 kB 00:00 2024-04-02T13:22:31.964 INFO:teuthology.orchestra.run.smithi188.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 106 kB 00:00 2024-04-02T13:22:32.005 INFO:teuthology.orchestra.run.smithi122.stdout:(22/174): python3-rados-19.0.0-2609.g191d963a.e 3.4 MB/s | 320 kB 00:00 2024-04-02T13:22:32.058 INFO:teuthology.orchestra.run.smithi187.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2024-04-02T13:22:32.084 INFO:teuthology.orchestra.run.smithi187.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 105 kB/s | 24 kB 00:00 2024-04-02T13:22:32.089 INFO:teuthology.orchestra.run.smithi122.stdout:(23/174): python3-rbd-19.0.0-2609.g191d963a.el9 3.5 MB/s | 299 kB 00:00 2024-04-02T13:22:32.109 INFO:teuthology.orchestra.run.smithi187.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 70 kB/s | 18 kB 00:00 2024-04-02T13:22:32.126 INFO:teuthology.orchestra.run.smithi187.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-04-02T13:22:32.131 INFO:teuthology.orchestra.run.smithi188.stdout:(41/174): ceph-osd-19.0.0-2609.g191d963a.el9.x8 2.2 MB/s | 17 MB 00:07 2024-04-02T13:22:32.147 INFO:teuthology.orchestra.run.smithi122.stdout:(24/174): python3-rgw-19.0.0-2609.g191d963a.el9 1.7 MB/s | 99 kB 00:00 2024-04-02T13:22:32.159 INFO:teuthology.orchestra.run.smithi187.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 636 kB/s | 32 kB 00:00 2024-04-02T13:22:32.165 INFO:teuthology.orchestra.run.smithi188.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 1.5 MB/s | 307 kB 00:00 2024-04-02T13:22:32.190 INFO:teuthology.orchestra.run.smithi188.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 548 kB/s | 32 kB 00:00 2024-04-02T13:22:32.197 INFO:teuthology.orchestra.run.smithi122.stdout:(25/174): rbd-fuse-19.0.0-2609.g191d963a.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-02T13:22:32.398 INFO:teuthology.orchestra.run.smithi188.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-02T13:22:32.410 INFO:teuthology.orchestra.run.smithi187.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 4.8 MB 00:00 2024-04-02T13:22:32.423 INFO:teuthology.orchestra.run.smithi188.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 73 kB/s | 17 kB 00:00 2024-04-02T13:22:32.435 INFO:teuthology.orchestra.run.smithi187.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 704 kB/s | 217 kB 00:00 2024-04-02T13:22:32.460 INFO:teuthology.orchestra.run.smithi187.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 583 kB/s | 174 kB 00:00 2024-04-02T13:22:32.466 INFO:teuthology.orchestra.run.smithi188.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 243 kB/s | 10 kB 00:00 2024-04-02T13:22:32.477 INFO:teuthology.orchestra.run.smithi187.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-04-02T13:22:32.491 INFO:teuthology.orchestra.run.smithi188.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 159 kB 00:00 2024-04-02T13:22:32.502 INFO:teuthology.orchestra.run.smithi187.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 988 kB/s | 41 kB 00:00 2024-04-02T13:22:32.516 INFO:teuthology.orchestra.run.smithi188.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 890 kB/s | 44 kB 00:00 2024-04-02T13:22:32.527 INFO:teuthology.orchestra.run.smithi187.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-04-02T13:22:32.550 INFO:teuthology.orchestra.run.smithi188.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-04-02T13:22:32.561 INFO:teuthology.orchestra.run.smithi187.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-04-02T13:22:32.575 INFO:teuthology.orchestra.run.smithi188.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 9.8 MB/s | 578 kB 00:00 2024-04-02T13:22:32.586 INFO:teuthology.orchestra.run.smithi187.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-04-02T13:22:32.603 INFO:teuthology.orchestra.run.smithi188.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 901 kB/s | 46 kB 00:00 2024-04-02T13:22:32.612 INFO:teuthology.orchestra.run.smithi187.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 863 kB/s | 43 kB 00:00 2024-04-02T13:22:32.628 INFO:teuthology.orchestra.run.smithi188.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.3 MB/s | 282 kB 00:00 2024-04-02T13:22:32.637 INFO:teuthology.orchestra.run.smithi187.stdout:(78/174): python3-pytz-2021.1-5.el9.noarch.rpm 632 kB/s | 47 kB 00:00 2024-04-02T13:22:32.653 INFO:teuthology.orchestra.run.smithi188.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T13:22:32.664 INFO:teuthology.orchestra.run.smithi122.stdout:(26/174): librgw2-19.0.0-2609.g191d963a.el9.x86 5.3 MB/s | 5.1 MB 00:00 2024-04-02T13:22:32.687 INFO:teuthology.orchestra.run.smithi188.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-04-02T13:22:32.712 INFO:teuthology.orchestra.run.smithi188.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 581 kB/s | 34 kB 00:00 2024-04-02T13:22:32.748 INFO:teuthology.orchestra.run.smithi122.stdout:(27/174): rbd-mirror-19.0.0-2609.g191d963a.el9. 5.6 MB/s | 3.1 MB 00:00 2024-04-02T13:22:32.773 INFO:teuthology.orchestra.run.smithi122.stdout:(28/174): rbd-nbd-19.0.0-2609.g191d963a.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-02T13:22:32.790 INFO:teuthology.orchestra.run.smithi122.stdout:(29/174): ceph-grafana-dashboards-19.0.0-2609.g 814 kB/s | 34 kB 00:00 2024-04-02T13:22:32.824 INFO:teuthology.orchestra.run.smithi122.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2609.g191d963 2.8 MB/s | 141 kB 00:00 2024-04-02T13:22:32.955 INFO:teuthology.orchestra.run.smithi188.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 18 MB/s | 4.7 MB 00:00 2024-04-02T13:22:32.996 INFO:teuthology.orchestra.run.smithi188.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 5.4 MB/s | 1.5 MB 00:00 2024-04-02T13:22:33.030 INFO:teuthology.orchestra.run.smithi188.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 817 kB 00:00 2024-04-02T13:22:33.054 INFO:teuthology.orchestra.run.smithi187.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 87 kB/s | 39 kB 00:00 2024-04-02T13:22:33.055 INFO:teuthology.orchestra.run.smithi188.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-02T13:22:33.079 INFO:teuthology.orchestra.run.smithi187.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 93 kB/s | 41 kB 00:00 2024-04-02T13:22:33.316 INFO:teuthology.orchestra.run.smithi122.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2609.g191d9 6.7 MB/s | 3.5 MB 00:00 2024-04-02T13:22:33.338 INFO:teuthology.orchestra.run.smithi187.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-02T13:22:33.363 INFO:teuthology.orchestra.run.smithi187.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 712 kB/s | 220 kB 00:00 2024-04-02T13:22:33.374 INFO:teuthology.orchestra.run.smithi122.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2609.g19 4.2 MB/s | 249 kB 00:00 2024-04-02T13:22:33.388 INFO:teuthology.orchestra.run.smithi187.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 113 kB/s | 35 kB 00:00 2024-04-02T13:22:33.413 INFO:teuthology.orchestra.run.smithi187.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 622 kB/s | 46 kB 00:00 2024-04-02T13:22:33.416 INFO:teuthology.orchestra.run.smithi122.stdout:(33/174): ceph-mgr-rook-19.0.0-2609.g191d963a.e 1.2 MB/s | 49 kB 00:00 2024-04-02T13:22:33.438 INFO:teuthology.orchestra.run.smithi187.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-04-02T13:22:33.450 INFO:teuthology.orchestra.run.smithi122.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2609.g1 492 kB/s | 16 kB 00:00 2024-04-02T13:22:33.464 INFO:teuthology.orchestra.run.smithi187.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 751 kB/s | 56 kB 00:00 2024-04-02T13:22:33.489 INFO:teuthology.orchestra.run.smithi187.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 836 kB/s | 63 kB 00:00 2024-04-02T13:22:33.514 INFO:teuthology.orchestra.run.smithi187.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-04-02T13:22:33.517 INFO:teuthology.orchestra.run.smithi122.stdout:(35/174): ceph-volume-19.0.0-2609.g191d963a.el9 3.9 MB/s | 264 kB 00:00 2024-04-02T13:22:33.540 INFO:teuthology.orchestra.run.smithi187.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T13:22:33.565 INFO:teuthology.orchestra.run.smithi187.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-04-02T13:22:33.607 INFO:teuthology.orchestra.run.smithi187.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-04-02T13:22:33.632 INFO:teuthology.orchestra.run.smithi187.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-04-02T13:22:33.649 INFO:teuthology.orchestra.run.smithi187.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T13:22:33.650 INFO:teuthology.orchestra.run.smithi122.stdout:(36/174): cephadm-19.0.0-2609.g191d963a.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-02T13:22:33.663 INFO:teuthology.orchestra.run.smithi188.stdout:(60/174): ceph-test-19.0.0-2609.g191d963a.el9.x 5.7 MB/s | 48 MB 00:08 2024-04-02T13:22:33.748 INFO:teuthology.orchestra.run.smithi188.stdout:(61/174): postgresql-server-13.14-1.el9.x86_64. 8.0 MB/s | 5.7 MB 00:00 2024-04-02T13:22:33.768 INFO:teuthology.orchestra.run.smithi187.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 1.1 MB/s | 187 kB 00:00 2024-04-02T13:22:33.773 INFO:teuthology.orchestra.run.smithi188.stdout:(62/174): protobuf-3.14.0-13.el9.x86_64.rpm 1.4 MB/s | 1.0 MB 00:00 2024-04-02T13:22:33.784 INFO:teuthology.orchestra.run.smithi122.stdout:(37/174): cephfs-top-19.0.0-2609.g191d963a.el9. 137 kB/s | 18 kB 00:00 2024-04-02T13:22:33.798 INFO:teuthology.orchestra.run.smithi188.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 598 kB/s | 78 kB 00:00 2024-04-02T13:22:33.856 INFO:teuthology.orchestra.run.smithi188.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 295 kB/s | 24 kB 00:00 2024-04-02T13:22:33.949 INFO:teuthology.orchestra.run.smithi188.stdout:(65/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-04-02T13:22:33.974 INFO:teuthology.orchestra.run.smithi188.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 104 kB/s | 18 kB 00:00 2024-04-02T13:22:33.993 INFO:teuthology.orchestra.run.smithi122.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 6.3 MB/s | 7.4 MB 00:01 2024-04-02T13:22:33.999 INFO:teuthology.orchestra.run.smithi188.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 144 kB 00:00 2024-04-02T13:22:34.018 INFO:teuthology.orchestra.run.smithi122.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-04-02T13:22:34.024 INFO:teuthology.orchestra.run.smithi188.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 421 kB/s | 32 kB 00:00 2024-04-02T13:22:34.060 INFO:teuthology.orchestra.run.smithi122.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-04-02T13:22:34.075 INFO:teuthology.orchestra.run.smithi188.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.8 MB/s | 217 kB 00:00 2024-04-02T13:22:34.085 INFO:teuthology.orchestra.run.smithi122.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-04-02T13:22:34.111 INFO:teuthology.orchestra.run.smithi122.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-04-02T13:22:34.128 INFO:teuthology.orchestra.run.smithi122.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 766 kB/s | 32 kB 00:00 2024-04-02T13:22:34.142 INFO:teuthology.orchestra.run.smithi188.stdout:(70/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-04-02T13:22:34.167 INFO:teuthology.orchestra.run.smithi188.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2024-04-02T13:22:34.192 INFO:teuthology.orchestra.run.smithi188.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 601 kB/s | 69 kB 00:00 2024-04-02T13:22:34.200 INFO:teuthology.orchestra.run.smithi187.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 452 kB/s | 195 kB 00:00 2024-04-02T13:22:34.203 INFO:teuthology.orchestra.run.smithi122.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-04-02T13:22:34.217 INFO:teuthology.orchestra.run.smithi188.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 825 kB/s | 41 kB 00:00 2024-04-02T13:22:34.220 INFO:teuthology.orchestra.run.smithi122.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 190 kB/s | 17 kB 00:00 2024-04-02T13:22:34.225 INFO:teuthology.orchestra.run.smithi187.stdout:(96/174): jq-1.6-16.el9.x86_64.rpm 325 kB/s | 187 kB 00:00 2024-04-02T13:22:34.237 INFO:teuthology.orchestra.run.smithi122.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-04-02T13:22:34.242 INFO:teuthology.orchestra.run.smithi188.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-04-02T13:22:34.262 INFO:teuthology.orchestra.run.smithi122.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-04-02T13:22:34.267 INFO:teuthology.orchestra.run.smithi188.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-04-02T13:22:34.288 INFO:teuthology.orchestra.run.smithi122.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 888 kB/s | 44 kB 00:00 2024-04-02T13:22:34.293 INFO:teuthology.orchestra.run.smithi188.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-04-02T13:22:34.310 INFO:teuthology.orchestra.run.smithi188.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 710 kB/s | 47 kB 00:00 2024-04-02T13:22:34.313 INFO:teuthology.orchestra.run.smithi122.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-04-02T13:22:34.325 INFO:teuthology.orchestra.run.smithi187.stdout:(97/174): elfutils-libelf-devel-0.190-2.el9.x86 215 kB/s | 21 kB 00:00 2024-04-02T13:22:34.335 INFO:teuthology.orchestra.run.smithi188.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 43 kB 00:00 2024-04-02T13:22:34.338 INFO:teuthology.orchestra.run.smithi122.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 12 MB/s | 578 kB 00:00 2024-04-02T13:22:34.350 INFO:teuthology.orchestra.run.smithi187.stdout:(98/174): python3-requests-2.25.1-8.el9.noarch. 830 kB/s | 125 kB 00:00 2024-04-02T13:22:34.363 INFO:teuthology.orchestra.run.smithi122.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 46 kB 00:00 2024-04-02T13:22:34.376 INFO:teuthology.orchestra.run.smithi187.stdout:(99/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-04-02T13:22:34.385 INFO:teuthology.orchestra.run.smithi188.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 516 kB/s | 39 kB 00:00 2024-04-02T13:22:34.389 INFO:teuthology.orchestra.run.smithi122.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-04-02T13:22:34.409 INFO:teuthology.orchestra.run.smithi187.stdout:(100/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-02T13:22:34.414 INFO:teuthology.orchestra.run.smithi122.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-04-02T13:22:34.431 INFO:teuthology.orchestra.run.smithi122.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-04-02T13:22:34.456 INFO:teuthology.orchestra.run.smithi122.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 829 kB/s | 34 kB 00:00 2024-04-02T13:22:34.468 INFO:teuthology.orchestra.run.smithi187.stdout:(101/174): python3-devel-3.9.17-2.el9.x86_64.rp 2.6 MB/s | 245 kB 00:00 2024-04-02T13:22:34.510 INFO:teuthology.orchestra.run.smithi187.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-04-02T13:22:34.535 INFO:teuthology.orchestra.run.smithi188.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 207 kB/s | 41 kB 00:00 2024-04-02T13:22:34.543 INFO:teuthology.orchestra.run.smithi187.stdout:(103/174): python3-jinja2-2.11.3-5.el9.noarch.r 1.8 MB/s | 248 kB 00:00 2024-04-02T13:22:34.574 INFO:teuthology.orchestra.run.smithi122.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-04-02T13:22:34.607 INFO:teuthology.orchestra.run.smithi122.stdout:(57/174): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2024-04-02T13:22:34.627 INFO:teuthology.orchestra.run.smithi187.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 15 MB/s | 4.1 MB 00:00 2024-04-02T13:22:34.640 INFO:teuthology.orchestra.run.smithi122.stdout:(58/174): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 817 kB 00:00 2024-04-02T13:22:34.653 INFO:teuthology.orchestra.run.smithi187.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-04-02T13:22:34.666 INFO:teuthology.orchestra.run.smithi122.stdout:(59/174): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 132 kB 00:00 2024-04-02T13:22:34.719 INFO:teuthology.orchestra.run.smithi188.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-04-02T13:22:34.744 INFO:teuthology.orchestra.run.smithi188.stdout:(82/174): qatlib-24.02.0-1.el9.x86_64.rpm 618 kB/s | 220 kB 00:00 2024-04-02T13:22:34.769 INFO:teuthology.orchestra.run.smithi188.stdout:(83/174): qatlib-service-24.02.0-1.el9.x86_64.r 150 kB/s | 35 kB 00:00 2024-04-02T13:22:34.795 INFO:teuthology.orchestra.run.smithi188.stdout:(84/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 926 kB/s | 46 kB 00:00 2024-04-02T13:22:34.799 INFO:teuthology.orchestra.run.smithi122.stdout:(60/174): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.7 MB 00:00 2024-04-02T13:22:34.820 INFO:teuthology.orchestra.run.smithi188.stdout:(85/174): socat-1.7.4.1-5.el9.x86_64.rpm 3.9 MB/s | 300 kB 00:00 2024-04-02T13:22:34.841 INFO:teuthology.orchestra.run.smithi122.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-02T13:22:34.845 INFO:teuthology.orchestra.run.smithi188.stdout:(86/174): uuid-1.6.2-55.el9.x86_64.rpm 753 kB/s | 56 kB 00:00 2024-04-02T13:22:34.866 INFO:teuthology.orchestra.run.smithi122.stdout:(62/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 78 kB 00:00 2024-04-02T13:22:34.870 INFO:teuthology.orchestra.run.smithi188.stdout:(87/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T13:22:34.895 INFO:teuthology.orchestra.run.smithi188.stdout:(88/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-04-02T13:22:34.919 INFO:teuthology.orchestra.run.smithi187.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-02T13:22:34.921 INFO:teuthology.orchestra.run.smithi188.stdout:(89/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-04-02T13:22:34.946 INFO:teuthology.orchestra.run.smithi188.stdout:(90/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T13:22:34.980 INFO:teuthology.orchestra.run.smithi188.stdout:(91/174): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-04-02T13:22:35.005 INFO:teuthology.orchestra.run.smithi188.stdout:(92/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-04-02T13:22:35.017 INFO:teuthology.orchestra.run.smithi122.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 5.8 MB 00:00 2024-04-02T13:22:35.030 INFO:teuthology.orchestra.run.smithi188.stdout:(93/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-04-02T13:22:35.034 INFO:teuthology.orchestra.run.smithi122.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 148 kB/s | 24 kB 00:00 2024-04-02T13:22:35.050 INFO:teuthology.orchestra.run.smithi122.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 545 kB/s | 18 kB 00:00 2024-04-02T13:22:35.055 INFO:teuthology.orchestra.run.smithi188.stdout:(94/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T13:22:35.069 INFO:teuthology.orchestra.run.smithi187.stdout:(107/174): gperftools-libs-2.9.1-2.el9.x86_64.r 742 kB/s | 309 kB 00:00 2024-04-02T13:22:35.076 INFO:teuthology.orchestra.run.smithi122.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-04-02T13:22:35.093 INFO:teuthology.orchestra.run.smithi122.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 759 kB/s | 32 kB 00:00 2024-04-02T13:22:35.095 INFO:teuthology.orchestra.run.smithi187.stdout:(108/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-02T13:22:35.170 INFO:teuthology.orchestra.run.smithi187.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-02T13:22:35.195 INFO:teuthology.orchestra.run.smithi187.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-02T13:22:35.210 INFO:teuthology.orchestra.run.smithi122.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-04-02T13:22:35.236 INFO:teuthology.orchestra.run.smithi122.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-04-02T13:22:35.261 INFO:teuthology.orchestra.run.smithi122.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-04-02T13:22:35.271 INFO:teuthology.orchestra.run.smithi187.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 648 kB/s | 49 kB 00:00 2024-04-02T13:22:35.286 INFO:teuthology.orchestra.run.smithi122.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-04-02T13:22:35.303 INFO:teuthology.orchestra.run.smithi122.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 985 kB/s | 41 kB 00:00 2024-04-02T13:22:35.321 INFO:teuthology.orchestra.run.smithi122.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 7.0 MB/s | 238 kB 00:00 2024-04-02T13:22:35.337 INFO:teuthology.orchestra.run.smithi187.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 475 kB/s | 67 kB 00:00 2024-04-02T13:22:35.353 INFO:teuthology.orchestra.run.smithi122.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-04-02T13:22:35.363 INFO:teuthology.orchestra.run.smithi187.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-02T13:22:35.395 INFO:teuthology.orchestra.run.smithi122.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-04-02T13:22:35.412 INFO:teuthology.orchestra.run.smithi122.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 814 kB/s | 47 kB 00:00 2024-04-02T13:22:35.429 INFO:teuthology.orchestra.run.smithi122.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-04-02T13:22:35.430 INFO:teuthology.orchestra.run.smithi187.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-02T13:22:35.483 INFO:teuthology.orchestra.run.smithi187.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.5 MB/s | 548 kB 00:00 2024-04-02T13:22:35.500 INFO:teuthology.orchestra.run.smithi187.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 624 kB/s | 43 kB 00:00 2024-04-02T13:22:35.539 INFO:teuthology.orchestra.run.smithi188.stdout:(95/174): jq-1.6-16.el9.x86_64.rpm 351 kB/s | 187 kB 00:00 2024-04-02T13:22:35.546 INFO:teuthology.orchestra.run.smithi122.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 332 kB/s | 39 kB 00:00 2024-04-02T13:22:35.550 INFO:teuthology.orchestra.run.smithi187.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-02T13:22:35.567 INFO:teuthology.orchestra.run.smithi187.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 223 kB/s | 15 kB 00:00 2024-04-02T13:22:35.581 INFO:teuthology.orchestra.run.smithi188.stdout:(96/174): libquadmath-11.4.1-3.el9.x86_64.rpm 372 kB/s | 195 kB 00:00 2024-04-02T13:22:35.617 INFO:teuthology.orchestra.run.smithi187.stdout:(119/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2024-04-02T13:22:35.643 INFO:teuthology.orchestra.run.smithi187.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-04-02T13:22:35.648 INFO:teuthology.orchestra.run.smithi188.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-02T13:22:35.668 INFO:teuthology.orchestra.run.smithi187.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-02T13:22:35.673 INFO:teuthology.orchestra.run.smithi188.stdout:(98/174): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-02T13:22:35.702 INFO:teuthology.orchestra.run.smithi187.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-04-02T13:22:35.727 INFO:teuthology.orchestra.run.smithi187.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-04-02T13:22:35.733 INFO:teuthology.orchestra.run.smithi188.stdout:(99/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-02T13:22:35.748 INFO:teuthology.orchestra.run.smithi188.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 253 kB/s | 25 kB 00:00 2024-04-02T13:22:35.752 INFO:teuthology.orchestra.run.smithi187.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-02T13:22:35.780 INFO:teuthology.orchestra.run.smithi122.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-04-02T13:22:35.797 INFO:teuthology.orchestra.run.smithi122.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 165 kB/s | 41 kB 00:00 2024-04-02T13:22:35.802 INFO:teuthology.orchestra.run.smithi187.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-04-02T13:22:35.822 INFO:teuthology.orchestra.run.smithi122.stdout:(81/174): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2024-04-02T13:22:35.828 INFO:teuthology.orchestra.run.smithi187.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-02T13:22:35.839 INFO:teuthology.orchestra.run.smithi122.stdout:(82/174): qatlib-service-24.02.0-1.el9.x86_64.r 835 kB/s | 35 kB 00:00 2024-04-02T13:22:35.849 INFO:teuthology.orchestra.run.smithi188.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 2.4 MB/s | 248 kB 00:00 2024-04-02T13:22:35.864 INFO:teuthology.orchestra.run.smithi122.stdout:(83/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-04-02T13:22:35.878 INFO:teuthology.orchestra.run.smithi187.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-02T13:22:35.889 INFO:teuthology.orchestra.run.smithi122.stdout:(84/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.8 MB/s | 300 kB 00:00 2024-04-02T13:22:35.899 INFO:teuthology.orchestra.run.smithi188.stdout:(102/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-04-02T13:22:35.903 INFO:teuthology.orchestra.run.smithi187.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-04-02T13:22:35.915 INFO:teuthology.orchestra.run.smithi122.stdout:(85/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-04-02T13:22:35.928 INFO:teuthology.orchestra.run.smithi187.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-04-02T13:22:35.933 INFO:teuthology.orchestra.run.smithi188.stdout:(103/174): python3-devel-3.9.17-2.el9.x86_64.rp 1.2 MB/s | 245 kB 00:00 2024-04-02T13:22:35.940 INFO:teuthology.orchestra.run.smithi122.stdout:(86/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-04-02T13:22:35.962 INFO:teuthology.orchestra.run.smithi187.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-02T13:22:35.965 INFO:teuthology.orchestra.run.smithi122.stdout:(87/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 877 kB/s | 44 kB 00:00 2024-04-02T13:22:35.987 INFO:teuthology.orchestra.run.smithi187.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-02T13:22:35.991 INFO:teuthology.orchestra.run.smithi122.stdout:(88/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-04-02T13:22:35.991 INFO:teuthology.orchestra.run.smithi188.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 13 MB/s | 4.1 MB 00:00 2024-04-02T13:22:36.016 INFO:teuthology.orchestra.run.smithi122.stdout:(89/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-04-02T13:22:36.029 INFO:teuthology.orchestra.run.smithi187.stdout:(132/174): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-02T13:22:36.055 INFO:teuthology.orchestra.run.smithi187.stdout:(133/174): python3-logutils-0.3.5-21.el9.noarch 503 kB/s | 46 kB 00:00 2024-04-02T13:22:36.058 INFO:teuthology.orchestra.run.smithi122.stdout:(90/174): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.1 MB 00:00 2024-04-02T13:22:36.080 INFO:teuthology.orchestra.run.smithi187.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 855 kB/s | 79 kB 00:00 2024-04-02T13:22:36.083 INFO:teuthology.orchestra.run.smithi122.stdout:(91/174): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2024-04-02T13:22:36.105 INFO:teuthology.orchestra.run.smithi187.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 764 kB/s | 58 kB 00:00 2024-04-02T13:22:36.108 INFO:teuthology.orchestra.run.smithi122.stdout:(92/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-04-02T13:22:36.130 INFO:teuthology.orchestra.run.smithi187.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-02T13:22:36.134 INFO:teuthology.orchestra.run.smithi122.stdout:(93/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-04-02T13:22:36.156 INFO:teuthology.orchestra.run.smithi187.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-02T13:22:36.181 INFO:teuthology.orchestra.run.smithi187.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-02T13:22:36.206 INFO:teuthology.orchestra.run.smithi187.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-02T13:22:36.232 INFO:teuthology.orchestra.run.smithi187.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-02T13:22:36.257 INFO:teuthology.orchestra.run.smithi187.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-02T13:22:36.276 INFO:teuthology.orchestra.run.smithi122.stdout:(94/174): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2024-04-02T13:22:36.282 INFO:teuthology.orchestra.run.smithi187.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-04-02T13:22:36.307 INFO:teuthology.orchestra.run.smithi187.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-02T13:22:36.326 INFO:teuthology.orchestra.run.smithi122.stdout:(95/174): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-02T13:22:36.341 INFO:teuthology.orchestra.run.smithi187.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 607 kB/s | 51 kB 00:00 2024-04-02T13:22:36.351 INFO:teuthology.orchestra.run.smithi122.stdout:(96/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-02T13:22:36.366 INFO:teuthology.orchestra.run.smithi187.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-02T13:22:36.377 INFO:teuthology.orchestra.run.smithi122.stdout:(97/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-02T13:22:36.391 INFO:teuthology.orchestra.run.smithi187.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-02T13:22:36.417 INFO:teuthology.orchestra.run.smithi187.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-04-02T13:22:36.442 INFO:teuthology.orchestra.run.smithi187.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-02T13:22:36.476 INFO:teuthology.orchestra.run.smithi187.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-02T13:22:36.500 INFO:teuthology.orchestra.run.smithi188.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-04-02T13:22:36.525 INFO:teuthology.orchestra.run.smithi188.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-02T13:22:36.543 INFO:teuthology.orchestra.run.smithi187.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-02T13:22:36.619 INFO:teuthology.orchestra.run.smithi188.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 207 kB/s | 19 kB 00:00 2024-04-02T13:22:36.735 INFO:teuthology.orchestra.run.smithi187.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 14 MB/s | 2.3 MB 00:00 2024-04-02T13:22:36.810 INFO:teuthology.orchestra.run.smithi187.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-04-02T13:22:36.844 INFO:teuthology.orchestra.run.smithi187.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 1.2 MB/s | 41 kB 00:00 2024-04-02T13:22:36.885 INFO:teuthology.orchestra.run.smithi187.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T13:22:36.928 INFO:teuthology.orchestra.run.smithi187.stdout:(155/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-04-02T13:22:36.961 INFO:teuthology.orchestra.run.smithi188.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 670 kB/s | 309 kB 00:00 2024-04-02T13:22:36.978 INFO:teuthology.orchestra.run.smithi187.stdout:(156/174): librados2-19.0.0-2609.g191d963a.el9. 6.3 MB/s | 3.4 MB 00:00 2024-04-02T13:22:37.062 INFO:teuthology.orchestra.run.smithi188.stdout:(109/174): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-02T13:22:37.070 INFO:teuthology.orchestra.run.smithi187.stdout:(157/174): librbd1-19.0.0-2609.g191d963a.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-02T13:22:37.087 INFO:teuthology.orchestra.run.smithi188.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-04-02T13:22:37.154 INFO:teuthology.orchestra.run.smithi188.stdout:(111/174): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-02T13:22:37.179 INFO:teuthology.orchestra.run.smithi188.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-04-02T13:22:37.195 INFO:teuthology.orchestra.run.smithi187.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 267 kB/s | 71 kB 00:00 2024-04-02T13:22:37.254 INFO:teuthology.orchestra.run.smithi187.stdout:(159/174): elfutils-debuginfod-client-0.190-2.e 132 kB/s | 36 kB 00:00 2024-04-02T13:22:37.254 INFO:teuthology.orchestra.run.smithi188.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-02T13:22:37.269 INFO:teuthology.orchestra.run.smithi122.stdout:(98/174): ceph-test-19.0.0-2609.g191d963a.el9.x 7.4 MB/s | 48 MB 00:06 2024-04-02T13:22:37.294 INFO:teuthology.orchestra.run.smithi122.stdout:(99/174): elfutils-libelf-devel-0.190-2.el9.x86 23 kB/s | 21 kB 00:00 2024-04-02T13:22:37.319 INFO:teuthology.orchestra.run.smithi122.stdout:(100/174): libuuid-devel-2.37.4-18.el9.x86_64.r 27 kB/s | 25 kB 00:00 2024-04-02T13:22:37.354 INFO:teuthology.orchestra.run.smithi187.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 230 kB/s | 23 kB 00:00 2024-04-02T13:22:37.355 INFO:teuthology.orchestra.run.smithi188.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-02T13:22:37.380 INFO:teuthology.orchestra.run.smithi188.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-02T13:22:37.455 INFO:teuthology.orchestra.run.smithi188.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 434 kB/s | 43 kB 00:00 2024-04-02T13:22:37.471 INFO:teuthology.orchestra.run.smithi187.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 927 kB/s | 256 kB 00:00 2024-04-02T13:22:37.480 INFO:teuthology.orchestra.run.smithi188.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-04-02T13:22:37.538 INFO:teuthology.orchestra.run.smithi187.stdout:(162/174): elfutils-libelf-0.190-2.el9.x86_64.r 411 kB/s | 192 kB 00:00 2024-04-02T13:22:37.547 INFO:teuthology.orchestra.run.smithi188.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-02T13:22:37.563 INFO:teuthology.orchestra.run.smithi187.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 519 kB/s | 108 kB 00:00 2024-04-02T13:22:37.580 INFO:teuthology.orchestra.run.smithi187.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 155 kB 00:00 2024-04-02T13:22:37.581 INFO:teuthology.orchestra.run.smithi188.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-02T13:22:37.614 INFO:teuthology.orchestra.run.smithi188.stdout:(120/174): protobuf-compiler-3.14.0-13.el9.x86_ 513 kB/s | 863 kB 00:01 2024-04-02T13:22:37.647 INFO:teuthology.orchestra.run.smithi187.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 136 kB 00:00 2024-04-02T13:22:37.648 INFO:teuthology.orchestra.run.smithi188.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-02T13:22:37.672 INFO:teuthology.orchestra.run.smithi187.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 586 kB/s | 64 kB 00:00 2024-04-02T13:22:37.690 INFO:teuthology.orchestra.run.smithi188.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-02T13:22:37.697 INFO:teuthology.orchestra.run.smithi187.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 245 kB/s | 29 kB 00:00 2024-04-02T13:22:37.749 INFO:teuthology.orchestra.run.smithi188.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-02T13:22:37.857 INFO:teuthology.orchestra.run.smithi188.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-02T13:22:37.874 INFO:teuthology.orchestra.run.smithi188.stdout:(125/174): python3-colorama-0.4.4-12.el9.noarch 136 kB/s | 35 kB 00:00 2024-04-02T13:22:37.940 INFO:teuthology.orchestra.run.smithi187.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.1 MB/s | 1.2 MB 00:00 2024-04-02T13:22:37.966 INFO:teuthology.orchestra.run.smithi188.stdout:(126/174): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-04-02T13:22:37.983 INFO:teuthology.orchestra.run.smithi188.stdout:(127/174): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-02T13:22:38.000 INFO:teuthology.orchestra.run.smithi188.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-02T13:22:38.048 INFO:teuthology.orchestra.run.smithi187.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 4.2 MB/s | 465 kB 00:00 2024-04-02T13:22:38.050 INFO:teuthology.orchestra.run.smithi188.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 276 kB/s | 23 kB 00:00 2024-04-02T13:22:38.067 INFO:teuthology.orchestra.run.smithi188.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 231 kB/s | 19 kB 00:00 2024-04-02T13:22:38.084 INFO:teuthology.orchestra.run.smithi188.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 234 kB/s | 20 kB 00:00 2024-04-02T13:22:38.090 INFO:teuthology.orchestra.run.smithi187.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.8 MB/s | 2.3 MB 00:00 2024-04-02T13:22:38.132 INFO:teuthology.orchestra.run.smithi187.stdout:(171/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.7 MB/s | 2.2 MB 00:00 2024-04-02T13:22:38.158 INFO:teuthology.orchestra.run.smithi187.stdout:(172/174): util-linux-user-2.37.4-18.el9.x86_64 285 kB/s | 31 kB 00:00 2024-04-02T13:22:38.176 INFO:teuthology.orchestra.run.smithi188.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2024-04-02T13:22:38.216 INFO:teuthology.orchestra.run.smithi187.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-04-02T13:22:38.219 INFO:teuthology.orchestra.run.smithi122.stdout:(101/174): python3-jinja2-2.11.3-5.el9.noarch.r 276 kB/s | 248 kB 00:00 2024-04-02T13:22:38.226 INFO:teuthology.orchestra.run.smithi188.stdout:(133/174): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-02T13:22:38.249 INFO:teuthology.orchestra.run.smithi187.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 634 kB/s | 74 kB 00:00 2024-04-02T13:22:38.251 INFO:teuthology.orchestra.run.smithi188.stdout:(134/174): python3-more-itertools-8.12.0-2.el9. 472 kB/s | 79 kB 00:00 2024-04-02T13:22:38.256 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:22:38.256 INFO:teuthology.orchestra.run.smithi187.stdout:Total 12 MB/s | 226 MB 00:18 2024-04-02T13:22:38.277 INFO:teuthology.orchestra.run.smithi188.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 573 kB/s | 58 kB 00:00 2024-04-02T13:22:38.303 INFO:teuthology.orchestra.run.smithi122.stdout:(102/174): python3-devel-3.9.17-2.el9.x86_64.rp 249 kB/s | 245 kB 00:00 2024-04-02T13:22:38.327 INFO:teuthology.orchestra.run.smithi188.stdout:(136/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-02T13:22:38.344 INFO:teuthology.orchestra.run.smithi188.stdout:(137/174): python3-portend-3.1.0-2.el9.noarch.r 177 kB/s | 16 kB 00:00 2024-04-02T13:22:38.370 INFO:teuthology.orchestra.run.smithi188.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 975 kB/s | 90 kB 00:00 2024-04-02T13:22:38.412 INFO:teuthology.orchestra.run.smithi188.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-04-02T13:22:38.428 INFO:teuthology.orchestra.run.smithi188.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-02T13:22:38.470 INFO:teuthology.orchestra.run.smithi188.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-02T13:22:38.496 INFO:teuthology.orchestra.run.smithi188.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-04-02T13:22:38.513 INFO:teuthology.orchestra.run.smithi188.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-02T13:22:38.555 INFO:teuthology.orchestra.run.smithi188.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 608 kB/s | 51 kB 00:00 2024-04-02T13:22:38.570 INFO:teuthology.orchestra.run.smithi122.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 84 kB/s | 22 kB 00:00 2024-04-02T13:22:38.597 INFO:teuthology.orchestra.run.smithi188.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-02T13:22:38.622 INFO:teuthology.orchestra.run.smithi188.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2024-04-02T13:22:38.640 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-04-02T13:22:38.655 INFO:teuthology.orchestra.run.smithi188.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-02T13:22:38.681 INFO:teuthology.orchestra.run.smithi188.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-02T13:22:38.781 INFO:teuthology.orchestra.run.smithi188.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-02T13:22:38.840 INFO:teuthology.orchestra.run.smithi188.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-02T13:22:38.914 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-04-02T13:22:38.915 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-04-02T13:22:38.932 INFO:teuthology.orchestra.run.smithi188.stdout:(151/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-02T13:22:38.979 INFO:teuthology.orchestra.run.smithi122.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 123 kB/s | 94 kB 00:00 2024-04-02T13:22:39.012 INFO:teuthology.orchestra.run.smithi122.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-04-02T13:22:39.015 INFO:teuthology.orchestra.run.smithi188.stdout:(152/174): NetworkManager-libnm-1.47.2-1.el9.x8 22 MB/s | 1.8 MB 00:00 2024-04-02T13:22:39.132 INFO:teuthology.orchestra.run.smithi188.stdout:(153/174): NetworkManager-team-1.47.2-1.el9.x86 351 kB/s | 41 kB 00:00 2024-04-02T13:22:39.174 INFO:teuthology.orchestra.run.smithi188.stdout:(154/174): NetworkManager-tui-1.47.2-1.el9.x86_ 5.8 MB/s | 247 kB 00:00 2024-04-02T13:22:39.233 INFO:teuthology.orchestra.run.smithi188.stdout:(155/174): librados2-19.0.0-2609.g191d963a.el9. 6.1 MB/s | 3.4 MB 00:00 2024-04-02T13:22:39.258 INFO:teuthology.orchestra.run.smithi188.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 3.8 MB/s | 321 kB 00:00 2024-04-02T13:22:39.262 INFO:teuthology.orchestra.run.smithi122.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-02T13:22:39.288 INFO:teuthology.orchestra.run.smithi122.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-02T13:22:39.313 INFO:teuthology.orchestra.run.smithi122.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 387 kB/s | 19 kB 00:00 2024-04-02T13:22:39.363 INFO:teuthology.orchestra.run.smithi122.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-04-02T13:22:39.414 INFO:teuthology.orchestra.run.smithi122.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 974 kB/s | 49 kB 00:00 2024-04-02T13:22:39.467 INFO:teuthology.orchestra.run.smithi188.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 175 kB/s | 36 kB 00:00 2024-04-02T13:22:39.500 INFO:teuthology.orchestra.run.smithi188.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 268 kB/s | 71 kB 00:00 2024-04-02T13:22:39.514 INFO:teuthology.orchestra.run.smithi122.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-02T13:22:39.539 INFO:teuthology.orchestra.run.smithi122.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2024-04-02T13:22:39.572 INFO:teuthology.orchestra.run.smithi122.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-02T13:22:39.631 INFO:teuthology.orchestra.run.smithi122.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-04-02T13:22:39.656 INFO:teuthology.orchestra.run.smithi122.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.0 MB/s | 838 kB 00:00 2024-04-02T13:22:39.682 INFO:teuthology.orchestra.run.smithi122.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-04-02T13:22:39.684 INFO:teuthology.orchestra.run.smithi188.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.4 MB/s | 256 kB 00:00 2024-04-02T13:22:39.707 INFO:teuthology.orchestra.run.smithi122.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-04-02T13:22:39.726 INFO:teuthology.orchestra.run.smithi188.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 742 kB/s | 192 kB 00:00 2024-04-02T13:22:39.732 INFO:teuthology.orchestra.run.smithi122.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-02T13:22:39.791 INFO:teuthology.orchestra.run.smithi122.stdout:(119/174): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-02T13:22:39.793 INFO:teuthology.orchestra.run.smithi188.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 212 kB/s | 23 kB 00:00 2024-04-02T13:22:39.816 INFO:teuthology.orchestra.run.smithi122.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-02T13:22:39.850 INFO:teuthology.orchestra.run.smithi122.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-02T13:22:39.851 INFO:teuthology.orchestra.run.smithi188.stdout:(162/174): librbd1-19.0.0-2609.g191d963a.el9.x8 2.9 MB/s | 3.1 MB 00:01 2024-04-02T13:22:39.866 INFO:teuthology.orchestra.run.smithi122.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 701 kB/s | 35 kB 00:00 2024-04-02T13:22:39.885 INFO:teuthology.orchestra.run.smithi188.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 682 kB/s | 108 kB 00:00 2024-04-02T13:22:39.909 INFO:teuthology.orchestra.run.smithi122.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-04-02T13:22:39.919 INFO:teuthology.orchestra.run.smithi188.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 155 kB 00:00 2024-04-02T13:22:39.967 INFO:teuthology.orchestra.run.smithi122.stdout:(124/174): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-02T13:22:39.986 INFO:teuthology.orchestra.run.smithi188.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 636 kB/s | 64 kB 00:00 2024-04-02T13:22:40.018 INFO:teuthology.orchestra.run.smithi122.stdout:(125/174): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-02T13:22:40.019 INFO:teuthology.orchestra.run.smithi188.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 286 kB/s | 29 kB 00:00 2024-04-02T13:22:40.034 INFO:teuthology.orchestra.run.smithi122.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-04-02T13:22:40.068 INFO:teuthology.orchestra.run.smithi122.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-04-02T13:22:40.085 INFO:teuthology.orchestra.run.smithi122.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 458 kB/s | 23 kB 00:00 2024-04-02T13:22:40.127 INFO:teuthology.orchestra.run.smithi122.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-02T13:22:40.152 INFO:teuthology.orchestra.run.smithi122.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-04-02T13:22:40.203 INFO:teuthology.orchestra.run.smithi122.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-02T13:22:40.203 INFO:teuthology.orchestra.run.smithi188.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 386 kB/s | 136 kB 00:00 2024-04-02T13:22:40.229 INFO:teuthology.orchestra.run.smithi122.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-04-02T13:22:40.253 INFO:teuthology.orchestra.run.smithi122.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-02T13:22:40.262 INFO:teuthology.orchestra.run.smithi188.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 4.4 MB/s | 1.2 MB 00:00 2024-04-02T13:22:40.287 INFO:teuthology.orchestra.run.smithi122.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-04-02T13:22:40.321 INFO:teuthology.orchestra.run.smithi122.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-02T13:22:40.337 INFO:teuthology.orchestra.run.smithi188.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.8 MB/s | 2.2 MB 00:00 2024-04-02T13:22:40.346 INFO:teuthology.orchestra.run.smithi122.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-04-02T13:22:40.379 INFO:teuthology.orchestra.run.smithi188.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 3.9 MB/s | 465 kB 00:00 2024-04-02T13:22:40.379 INFO:teuthology.orchestra.run.smithi122.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-02T13:22:40.404 INFO:teuthology.orchestra.run.smithi122.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-04-02T13:22:40.438 INFO:teuthology.orchestra.run.smithi122.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-04-02T13:22:40.446 INFO:teuthology.orchestra.run.smithi188.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 287 kB/s | 31 kB 00:00 2024-04-02T13:22:40.463 INFO:teuthology.orchestra.run.smithi122.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-02T13:22:40.504 INFO:teuthology.orchestra.run.smithi188.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 8.6 MB/s | 1.1 MB 00:00 2024-04-02T13:22:40.505 INFO:teuthology.orchestra.run.smithi122.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-04-02T13:22:40.531 INFO:teuthology.orchestra.run.smithi122.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 536 kB/s | 36 kB 00:00 2024-04-02T13:22:40.546 INFO:teuthology.orchestra.run.smithi188.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 743 kB/s | 74 kB 00:00 2024-04-02T13:22:40.572 INFO:teuthology.orchestra.run.smithi122.stdout:(143/174): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-04-02T13:22:40.606 INFO:teuthology.orchestra.run.smithi122.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-02T13:22:40.628 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-04-02T13:22:40.628 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-04-02T13:22:40.631 INFO:teuthology.orchestra.run.smithi122.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-02T13:22:40.663 INFO:teuthology.orchestra.run.smithi188.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 5.0 MB/s | 2.3 MB 00:00 2024-04-02T13:22:40.665 INFO:teuthology.orchestra.run.smithi122.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-04-02T13:22:40.667 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:22:40.667 INFO:teuthology.orchestra.run.smithi188.stdout:Total 12 MB/s | 226 MB 00:19 2024-04-02T13:22:40.690 INFO:teuthology.orchestra.run.smithi122.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-04-02T13:22:40.724 INFO:teuthology.orchestra.run.smithi122.stdout:(148/174): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-04-02T13:22:40.783 INFO:teuthology.orchestra.run.smithi122.stdout:(149/174): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-02T13:22:41.046 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-04-02T13:22:41.166 INFO:teuthology.orchestra.run.smithi122.stdout:(150/174): librados2-19.0.0-2609.g191d963a.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-02T13:22:41.217 INFO:teuthology.orchestra.run.smithi122.stdout:(151/174): librbd1-19.0.0-2609.g191d963a.el9.x8 7.3 MB/s | 3.1 MB 00:00 2024-04-02T13:22:41.258 INFO:teuthology.orchestra.run.smithi122.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 25 MB/s | 2.3 MB 00:00 2024-04-02T13:22:41.300 INFO:teuthology.orchestra.run.smithi122.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 23 MB/s | 1.8 MB 00:00 2024-04-02T13:22:41.317 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-04-02T13:22:41.317 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-04-02T13:22:41.326 INFO:teuthology.orchestra.run.smithi122.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 637 kB/s | 41 kB 00:00 2024-04-02T13:22:41.351 INFO:teuthology.orchestra.run.smithi122.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 4.8 MB/s | 247 kB 00:00 2024-04-02T13:22:41.376 INFO:teuthology.orchestra.run.smithi122.stdout:(156/174): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-02T13:22:41.468 INFO:teuthology.orchestra.run.smithi122.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 397 kB/s | 36 kB 00:00 2024-04-02T13:22:41.493 INFO:teuthology.orchestra.run.smithi122.stdout:(158/174): acl-2.3.1-4.el9.x86_64.rpm 504 kB/s | 71 kB 00:00 2024-04-02T13:22:41.577 INFO:teuthology.orchestra.run.smithi122.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2024-04-02T13:22:41.603 INFO:teuthology.orchestra.run.smithi122.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.3 MB/s | 256 kB 00:00 2024-04-02T13:22:41.628 INFO:teuthology.orchestra.run.smithi122.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 457 kB/s | 23 kB 00:00 2024-04-02T13:22:41.653 INFO:teuthology.orchestra.run.smithi122.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 108 kB 00:00 2024-04-02T13:22:41.678 INFO:teuthology.orchestra.run.smithi122.stdout:(163/174): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-02T13:22:41.704 INFO:teuthology.orchestra.run.smithi122.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 136 kB 00:00 2024-04-02T13:22:41.729 INFO:teuthology.orchestra.run.smithi122.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-04-02T13:22:41.754 INFO:teuthology.orchestra.run.smithi122.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 573 kB/s | 29 kB 00:00 2024-04-02T13:22:41.846 INFO:teuthology.orchestra.run.smithi122.stdout:(167/174): openssl-3.0.7-27.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2024-04-02T13:22:41.896 INFO:teuthology.orchestra.run.smithi122.stdout:(168/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-04-02T13:22:41.947 INFO:teuthology.orchestra.run.smithi122.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 9.1 MB/s | 465 kB 00:00 2024-04-02T13:22:41.989 INFO:teuthology.orchestra.run.smithi122.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 16 MB/s | 2.3 MB 00:00 2024-04-02T13:22:42.014 INFO:teuthology.orchestra.run.smithi122.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 466 kB/s | 31 kB 00:00 2024-04-02T13:22:42.056 INFO:teuthology.orchestra.run.smithi122.stdout:(172/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-04-02T13:22:42.756 INFO:teuthology.orchestra.run.smithi122.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 100 kB/s | 74 kB 00:00 2024-04-02T13:22:43.018 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-04-02T13:22:43.018 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-04-02T13:22:43.281 INFO:teuthology.orchestra.run.smithi122.stdout:(174/174): openssl-devel-3.0.7-27.el9.x86_64.rp 704 kB/s | 4.1 MB 00:06 2024-04-02T13:22:43.286 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:22:43.287 INFO:teuthology.orchestra.run.smithi122.stdout:Total 10 MB/s | 226 MB 00:22 2024-04-02T13:22:43.669 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-04-02T13:22:43.943 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-04-02T13:22:43.943 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-04-02T13:22:45.959 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-04-02T13:22:45.960 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-04-02T13:22:47.465 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-04-02T13:22:47.586 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:22:47.784 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:22:47.952 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:47.964 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:48.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:22:48.073 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:22:48.114 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:22:48.139 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:22:48.183 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:22:48.513 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:22:48.573 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:48.686 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:48.716 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:48.798 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:48.828 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:22:48.866 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:22:49.035 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:22:49.093 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:22:49.121 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:49.225 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:49.247 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:49.311 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:49.351 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:22:49.374 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:49.390 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:49.582 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:49.613 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:49.645 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:22:49.749 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:22:49.847 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:49.907 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:49.982 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:50.044 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-04-02T13:22:50.051 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:50.150 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:22:50.184 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:22:50.214 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:50.224 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:22:50.268 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:22:50.352 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:22:50.372 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:22:50.377 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:22:50.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:22:50.458 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:22:50.505 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:22:50.532 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:22:50.567 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:22:50.641 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:22:50.695 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:22:50.734 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:22:50.748 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:22:50.799 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:22:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:50.835 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:22:50.879 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:22:50.911 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:22:50.939 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:22:50.941 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:50.971 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:51.011 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:22:51.045 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:22:51.045 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:51.075 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:22:51.080 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:22:51.108 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:22:51.115 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:22:51.178 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:22:51.224 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:22:51.294 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:22:51.297 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:22:51.348 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:22:51.376 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:51.434 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:22:51.471 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:51.485 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:22:51.494 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:51.558 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:51.597 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:22:51.620 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:51.636 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:51.778 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:51.809 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:51.850 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:22:51.949 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:22:52.020 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:22:52.046 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:52.088 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:52.106 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:22:52.160 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:52.174 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:22:52.214 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:52.215 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:52.227 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:52.255 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:22:52.305 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:22:52.344 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:22:52.375 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:22:52.439 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:52.547 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:22:52.602 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:22:52.639 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:22:52.677 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:22:52.704 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:22:52.740 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:22:52.771 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:22:52.814 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:22:52.825 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:22:52.868 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:22:52.868 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:22:52.891 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-04-02T13:22:52.994 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-04-02T13:22:53.005 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:22:53.037 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-04-02T13:22:53.052 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:22:53.063 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:22:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:53.123 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:22:53.190 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-04-02T13:22:53.200 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:22:53.230 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:22:53.266 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:22:53.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:22:53.314 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:22:53.338 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:22:53.349 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:22:53.373 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-04-02T13:22:53.381 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:22:53.400 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:22:53.414 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:22:53.431 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-04-02T13:22:53.442 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:22:53.443 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:22:53.511 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:22:53.538 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-04-02T13:22:53.547 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:22:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-04-02T13:22:53.587 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:22:53.590 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:22:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-04-02T13:22:53.619 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:22:53.640 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:22:53.670 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:22:53.688 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:22:53.712 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:22:53.734 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:22:53.807 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:22:53.807 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:22:53.836 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:22:53.868 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:22:53.894 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:22:53.918 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:22:53.943 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:22:53.946 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-04-02T13:22:53.992 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:22:53.994 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:22:54.009 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:54.118 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:22:54.119 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:22:54.151 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:54.219 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:22:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:22:54.260 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:22:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:22:54.303 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-04-02T13:22:54.309 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:22:54.365 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:22:54.488 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-04-02T13:22:54.509 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:22:54.545 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-04-02T13:22:54.548 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:22:54.574 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:54.589 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:22:54.597 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:22:54.671 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:22:54.677 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:22:54.708 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:54.719 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:54.729 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:54.743 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:22:54.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:22:54.773 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 17/198 2024-04-02T13:22:54.787 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:22:54.799 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:22:54.819 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:22:54.821 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-04-02T13:22:54.837 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:22:54.850 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:22:54.851 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:54.866 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:22:54.867 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-04-02T13:22:54.878 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:22:54.912 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:22:54.951 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:22:54.983 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:22:55.013 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:22:55.068 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:55.087 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:22:55.090 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:22:55.100 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-04-02T13:22:55.123 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:22:55.139 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-04-02T13:22:55.156 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:22:55.215 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:22:55.246 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:22:55.249 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:22:55.311 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:22:55.339 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:22:55.354 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:22:55.429 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:22:55.472 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:22:55.497 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-04-02T13:22:55.503 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:22:55.551 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:22:55.648 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:22:55.696 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:22:55.704 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:22:55.751 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:55.775 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:22:55.810 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-04-02T13:22:55.830 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:22:55.902 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:22:55.930 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:55.933 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:22:55.961 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:22:55.986 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:56.007 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:22:56.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:22:56.087 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:22:56.151 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:22:56.189 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-04-02T13:22:56.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:22:56.290 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-04-02T13:22:56.331 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:22:56.362 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:22:56.378 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-04-02T13:22:56.402 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:22:56.416 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-04-02T13:22:56.421 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:22:56.446 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-04-02T13:22:56.469 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:22:56.489 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-04-02T13:22:56.496 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:22:56.502 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:22:56.536 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:22:56.551 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:22:56.563 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-04-02T13:22:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:22:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:22:56.617 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-04-02T13:22:56.627 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:22:56.638 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:22:56.656 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-04-02T13:22:56.686 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:22:56.699 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:22:56.720 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-04-02T13:22:56.763 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-04-02T13:22:56.806 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-04-02T13:22:56.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:22:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-04-02T13:22:56.863 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:22:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-04-02T13:22:56.892 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:22:56.926 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:22:56.936 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-04-02T13:22:56.970 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:22:56.971 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 39/198 2024-04-02T13:22:56.988 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:22:57.015 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 40/198 2024-04-02T13:22:57.042 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:22:57.043 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 41/198 2024-04-02T13:22:57.098 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:22:57.113 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-04-02T13:22:57.175 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-04-02T13:22:57.249 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-04-02T13:22:57.282 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:22:57.323 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:22:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-04-02T13:22:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-04-02T13:22:57.450 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:22:57.497 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:22:57.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:22:57.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:22:57.588 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:22:57.611 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:22:57.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:22:57.662 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:22:57.701 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:22:57.728 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:22:57.731 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:22:57.807 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:22:57.850 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:22:57.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:22:57.933 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:22:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-04-02T13:22:57.968 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:22:58.040 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-04-02T13:22:58.082 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:22:58.137 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-04-02T13:22:58.172 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:22:58.184 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:58.194 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-04-02T13:22:58.216 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:22:58.230 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-04-02T13:22:58.247 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:22:58.281 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-04-02T13:22:58.285 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:22:58.320 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 53/198 2024-04-02T13:22:58.350 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-04-02T13:22:58.382 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:22:58.432 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:22:58.644 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-04-02T13:22:58.832 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-04-02T13:22:58.919 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-04-02T13:22:58.941 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:22:58.963 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-04-02T13:22:59.043 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:22:59.073 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:22:59.102 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-04-02T13:22:59.104 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:22:59.132 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:22:59.157 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-04-02T13:22:59.161 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:22:59.212 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-04-02T13:22:59.213 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:22:59.246 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:22:59.294 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-04-02T13:22:59.294 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:22:59.333 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-04-02T13:22:59.369 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-04-02T13:22:59.373 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:22:59.406 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:22:59.424 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-04-02T13:22:59.446 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:22:59.460 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-04-02T13:22:59.471 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:22:59.504 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-04-02T13:22:59.546 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-04-02T13:22:59.561 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:22:59.581 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:22:59.592 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:22:59.620 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:22:59.627 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:22:59.679 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-04-02T13:22:59.697 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:22:59.726 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-04-02T13:22:59.756 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-04-02T13:22:59.823 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-04-02T13:22:59.917 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-04-02T13:22:59.957 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-04-02T13:22:59.996 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-04-02T13:23:00.022 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-04-02T13:23:00.047 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-04-02T13:23:00.090 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:23:00.123 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:00.136 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-04-02T13:23:00.194 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:00.216 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:23:00.256 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-04-02T13:23:00.315 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:23:00.360 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-04-02T13:23:00.405 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-04-02T13:23:00.416 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:23:00.454 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-04-02T13:23:00.510 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-04-02T13:23:00.693 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-04-02T13:23:00.734 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-04-02T13:23:00.864 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-04-02T13:23:00.907 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-04-02T13:23:00.940 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-04-02T13:23:00.970 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:23:01.002 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-04-02T13:23:01.033 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-04-02T13:23:01.072 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-04-02T13:23:01.104 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-04-02T13:23:01.134 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-04-02T13:23:01.203 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-04-02T13:23:01.236 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-04-02T13:23:01.269 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-04-02T13:23:01.310 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-04-02T13:23:01.345 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-04-02T13:23:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-04-02T13:23:01.524 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-04-02T13:23:01.584 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-04-02T13:23:01.608 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:23:01.624 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-04-02T13:23:01.671 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-04-02T13:23:01.680 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:23:01.744 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:23:01.769 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-04-02T13:23:01.776 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:23:01.803 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:23:01.817 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-04-02T13:23:02.113 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:23:02.134 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:23:02.214 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:23:02.246 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:23:02.274 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:23:02.418 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:02.455 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:02.545 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-04-02T13:23:02.598 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-04-02T13:23:02.631 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-04-02T13:23:02.681 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-04-02T13:23:02.738 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:23:02.770 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:02.777 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-04-02T13:23:02.817 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-04-02T13:23:02.841 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:02.863 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:23:02.994 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-04-02T13:23:02.995 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:03.031 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-04-02T13:23:03.032 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:03.106 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:03.106 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:23:03.106 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:23:03.106 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:03.109 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-04-02T13:23:03.134 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:03.754 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-04-02T13:23:03.871 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-04-02T13:23:05.078 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:05.096 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-04-02T13:23:05.109 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:05.174 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-04-02T13:23:05.205 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-04-02T13:23:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-04-02T13:23:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-04-02T13:23:05.582 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-04-02T13:23:05.610 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:23:05.643 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:05.680 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:05.701 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 124/198 2024-04-02T13:23:05.732 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 125/198 2024-04-02T13:23:05.761 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:05.761 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:23:05.761 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:23:05.761 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-04-02T13:23:05.789 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-04-02T13:23:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:06.326 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 128/198 2024-04-02T13:23:06.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 129/198 2024-04-02T13:23:08.582 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:08.620 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:09.162 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 130/198 2024-04-02T13:23:09.198 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:09.289 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 131/198 2024-04-02T13:23:09.289 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-02T13:23:09.289 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-02T13:23:09.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:09.324 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-04-02T13:23:15.148 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:15.321 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:15.339 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:15.339 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:15.339 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:23:15.339 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:16.799 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:16.824 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:16.825 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:16.825 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:23:16.825 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:16.825 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:16.825 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:17.049 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:17.072 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:17.072 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:17.072 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:23:17.072 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:17.073 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:17.073 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:17.517 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:17.539 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:17.652 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:23:17.658 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-04-02T13:23:17.659 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:17.830 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:17.846 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:17.846 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:17.846 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:23:17.846 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:18.087 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:18.117 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:18.149 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:23:18.209 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:23:18.240 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:19.274 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:19.308 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:19.323 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:19.349 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:19.566 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:19.588 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:19.588 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:19.588 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:23:19.588 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:19.589 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:19.589 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:20.025 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:20.047 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:20.047 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:20.047 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:23:20.048 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:20.048 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:20.048 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:20.145 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:23:20.275 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:20.300 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:20.401 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:20.530 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:20.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:20.572 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:23:20.592 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:23:20.611 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:20.634 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:20.653 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:23:20.678 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:20.683 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:20.689 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:20.756 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:23:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 132/198 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-04-02T13:23:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:21.631 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:23:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 133/198 2024-04-02T13:23:21.639 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:21.639 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-02T13:23:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:21.663 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:21.686 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:21.687 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:21.687 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:23:21.687 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:21.687 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:21.687 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:21.708 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:21.731 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:21.752 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:21.753 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:21.753 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:21.753 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:23:21.753 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:21.798 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:21.821 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:21.821 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:21.821 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:23:21.821 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:21.822 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:21.822 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:22.124 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:22.146 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:22.146 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:22.146 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:23:22.146 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:22.147 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:22.147 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:22.708 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:22.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:22.843 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:23.006 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:23:23.038 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:23.061 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:23.061 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:23.062 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:23:23.062 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:23.062 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:23.062 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:23.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:23.116 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:23.125 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:23.151 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 134/198 2024-04-02T13:23:23.151 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:23.151 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-02T13:23:23.151 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:23.151 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-02T13:23:23.152 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:23.161 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:23:23.376 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 135/198 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-02T13:23:23.399 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:23.843 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:23.865 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 136/198 2024-04-02T13:23:23.865 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:23.866 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-02T13:23:23.866 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:23.866 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-02T13:23:23.866 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:23.953 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-04-02T13:23:24.032 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:23:24.066 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:24.089 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:24.133 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:24.155 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:24.155 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:24.155 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:23:24.156 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:24.201 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:24.223 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:24.223 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:23:24.223 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:24.224 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:24.397 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:24.426 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 138/198 2024-04-02T13:23:24.459 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 139/198 2024-04-02T13:23:24.520 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 140/198 2024-04-02T13:23:24.543 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:24.550 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:24.565 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:24.566 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:24.566 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:23:24.566 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:24.566 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:24.566 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:25.579 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 141/198 2024-04-02T13:23:25.618 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:26.582 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 142/198 2024-04-02T13:23:26.587 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:26.610 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:26.627 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:23:26.658 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:23:26.685 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:23:26.713 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 143/198 2024-04-02T13:23:26.864 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:23:26.872 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 144/198 2024-04-02T13:23:26.896 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:26.919 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 145/198 2024-04-02T13:23:26.919 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:26.920 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-02T13:23:26.920 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:26.920 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-02T13:23:26.920 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:26.972 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:26.983 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 146/198 2024-04-02T13:23:26.984 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:23:27.024 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:23:27.030 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 147/198 2024-04-02T13:23:27.064 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:23:27.116 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:23:27.151 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:23:27.184 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:23:27.216 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:23:27.245 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:23:27.291 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:23:27.335 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:23:27.360 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:23:27.909 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-04-02T13:23:27.940 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:27.963 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 149/198 2024-04-02T13:23:27.963 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:27.963 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-02T13:23:27.963 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:27.964 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-02T13:23:27.964 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:27.980 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:28.008 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:28.016 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:28.029 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2609.g191d9 150/198 2024-04-02T13:23:28.029 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:28.030 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-02T13:23:28.030 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:28.036 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:28.075 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:28.097 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:28.097 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:28.097 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-02T13:23:28.098 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:28.098 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-02T13:23:28.098 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:28.417 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 152/198 2024-04-02T13:23:28.650 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-02T13:23:28.650 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-02T13:23:28.650 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:28.650 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-02T13:23:28.651 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:28.858 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:23:28.947 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:23:28.987 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:23:29.015 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:29.055 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:23:29.076 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:23:29.086 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:23:29.113 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:23:29.133 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:23:29.158 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:23:29.184 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:23:29.211 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:23:29.211 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:29.281 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:29.294 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:23:29.311 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:23:29.312 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:29.379 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:29.405 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:23:29.414 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:23:29.439 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:23:29.453 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:23:29.464 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:23:29.489 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:23:29.492 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:23:29.522 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:23:29.545 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:23:29.570 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:23:29.588 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:23:29.610 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:23:29.621 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:23:29.635 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:23:29.653 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:23:29.661 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:23:29.686 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:29.711 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:23:29.718 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:29.718 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:29.737 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:23:29.795 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:23:29.832 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:23:29.889 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:29.889 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:29.956 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:29.987 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:30.014 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:23:30.040 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:23:30.065 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:23:30.129 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:30.417 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:30.455 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:30.474 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:31.286 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:23:31.375 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:23:31.416 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:23:31.497 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:23:31.529 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:23:31.563 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:23:31.588 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:23:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:23:31.614 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:31.694 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:31.723 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:23:31.724 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:31.817 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:23:31.884 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:23:31.910 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:23:31.935 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:23:31.967 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:23:32.007 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:23:32.040 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:23:32.065 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:23:32.090 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:23:32.116 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:32.141 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:23:32.147 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:32.147 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:32.327 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:32.327 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:32.410 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:32.441 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:32.468 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:23:32.494 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:23:32.528 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:23:32.592 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:32.897 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:32.938 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 154/198 2024-04-02T13:23:32.969 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 155/198 2024-04-02T13:23:32.996 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-04-02T13:23:33.192 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 157/198 2024-04-02T13:23:33.312 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-04-02T13:23:33.352 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 159/198 2024-04-02T13:23:33.407 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-04-02T13:23:33.469 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-04-02T13:23:33.512 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-04-02T13:23:33.537 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 163/198 2024-04-02T13:23:33.569 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-04-02T13:23:33.598 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-04-02T13:23:33.663 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-04-02T13:23:33.722 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-04-02T13:23:33.747 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-04-02T13:23:34.368 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:34.403 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:34.423 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-04-02T13:23:35.229 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 170/198 2024-04-02T13:23:35.309 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-04-02T13:23:35.349 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-04-02T13:23:35.439 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-04-02T13:23:35.470 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-04-02T13:23:35.495 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-04-02T13:23:35.521 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-04-02T13:23:35.547 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-04-02T13:23:35.547 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:35.628 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:35.628 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:23:35.628 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:23:35.628 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:23:35.628 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:23:35.629 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:23:35.630 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:23:35.632 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:23:35.633 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:23:35.634 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:23:35.635 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:23:35.636 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:23:35.637 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:23:35.638 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:23:35.639 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:23:35.640 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:23:35.641 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:23:35.642 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:23:35.643 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:23:35.644 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:23:35.645 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-04-02T13:23:35.656 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-04-02T13:23:35.657 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:35.733 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-04-02T13:23:35.759 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-04-02T13:23:35.801 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-04-02T13:23:35.826 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-04-02T13:23:35.851 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-04-02T13:23:35.884 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-04-02T13:23:35.924 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-04-02T13:23:35.956 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-04-02T13:23:35.981 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-04-02T13:23:36.006 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-04-02T13:23:36.032 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:36.057 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-04-02T13:23:36.063 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:36.063 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-04-02T13:23:36.243 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:36.326 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-04-02T13:23:36.357 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:36.376 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-04-02T13:23:36.402 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-04-02T13:23:36.427 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-04-02T13:23:36.492 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:38.176 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:23:38.177 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:23:38.178 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:23:38.179 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:23:38.179 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:23:38.193 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:23:38.193 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:23:38.194 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:23:38.195 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:23:38.196 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:23:38.197 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:23:38.198 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:23:38.199 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:23:38.200 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:23:38.201 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:23:38.202 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:23:38.203 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:23:38.204 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:23:38.797 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.798 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.799 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.800 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:23:38.801 INFO:teuthology.orchestra.run.smithi187.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.802 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:38.803 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.804 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:23:38.805 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:38.806 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.807 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:23:38.808 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:23:38.809 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:23:38.810 INFO:teuthology.orchestra.run.smithi187.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:23:38.810 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-02T13:23:38.810 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-04-02T13:23:39.259 DEBUG:teuthology.parallel:result is None 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:41.132 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:41.133 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.134 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.135 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:23:41.136 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:23:41.137 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:23:41.138 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:23:41.139 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:23:41.140 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:23:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:23:41.142 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:23:41.143 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:23:41.144 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:23:41.145 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-04-02T13:23:41.588 DEBUG:teuthology.parallel:result is None 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.0.0-2609.g191d963a.el9.x86_64 1/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 3/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 4/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2609.g191d9 5/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 6/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 7/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 8/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 9/198 2024-04-02T13:23:42.064 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 10/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 11/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 12/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 13/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 14/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_ 15/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 16/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 17/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.0.0-2609.g191d963a.el9.x86_6 18/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86 19/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 20/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2609.g191d963a.el 21/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.0.0-2609.g191d963a.el9. 22/198 2024-04-02T13:23:42.065 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_6 23/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 24/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 25/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 26/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 27/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 28/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 29/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2609.g191d963a. 30/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noa 31/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.n 32/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2609.g191 33/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el 34/198 2024-04-02T13:23:42.066 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 35/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.e 36/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 37/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.0.0-2609.g191d963a.el9.noarch 38/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 39/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-04-02T13:23:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.14-1.el9.x86_64 58/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 59/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 60/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 61/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-04-02T13:23:42.068 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-04-02T13:23:42.069 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 82/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 83/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 84/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 85/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 86/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 87/198 2024-04-02T13:23:42.070 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 88/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 89/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 94/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : jq-1.6-16.el9.x86_64 95/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 96/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 97/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 98/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 99/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 100/198 2024-04-02T13:23:42.071 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 101/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 102/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 103/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-04-02T13:23:42.072 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-04-02T13:23:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-04-02T13:23:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-04-02T13:23:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-04-02T13:23:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-04-02T13:23:42.073 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-04-02T13:23:42.074 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-04-02T13:23:42.075 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.0.0-2609.g191d963a.el9.x86_64 151/198 2024-04-02T13:23:42.076 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 153/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 155/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 156/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 157/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 159/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 160/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 161/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 162/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 163/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 164/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-04-02T13:23:42.077 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-04-02T13:23:42.078 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-04-02T13:23:42.079 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-04-02T13:23:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-04-02T13:23:42.080 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-04-02T13:23:44.950 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: acl-2.3.1-4.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-2.3.1-4.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-04-02T13:23:44.951 INFO:teuthology.orchestra.run.smithi122.stdout: libblkid-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: libmount-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-04-02T13:23:44.952 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.953 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.954 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.0.0-2609.g191d963a.el9.noarch 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: jq-1.6-16.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-02T13:23:44.955 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-02T13:23:44.956 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-02T13:23:44.957 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.14-1.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-02T13:23:44.958 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-04-02T13:23:44.959 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-02T13:23:44.960 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-02T13:23:44.961 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-02T13:23:44.962 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.0.0-2609.g191d963a.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-3.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-02T13:23:44.963 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:23:44.964 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-04-02T13:23:45.450 DEBUG:teuthology.parallel:result is None 2024-04-02T13:23:45.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:45.630 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:23:45.651 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:23:45.652 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:23:45.652 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:23:45.653 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:45.826 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:23:45.851 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:23:45.852 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:23:45.852 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:23:45.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:46.162 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-02T13:23:46.187 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2609.g191d963a.el9 2024-04-02T13:23:46.187 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2609.g191d963a.el9 2024-04-02T13:23:46.187 INFO:teuthology.task.install:The correct ceph version 19.0.0-2609.g191d963a is installed. 2024-04-02T13:23:46.189 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-02T13:23:46.189 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:23:46.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:23:46.221 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:23:46.221 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:23:46.256 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:23:46.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-02T13:23:46.289 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-02T13:23:46.289 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:23:46.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:23:46.318 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:23:46.386 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:23:46.386 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:23:46.417 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:23:46.488 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:23:46.488 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-04-02T13:23:46.519 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-02T13:23:46.589 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-02T13:23:46.589 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:23:46.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:23:46.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:23:46.686 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:23:46.686 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:23:46.717 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:23:46.788 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:23:46.788 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-02T13:23:46.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-02T13:23:46.887 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-02T13:23:46.888 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:23:46.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:23:46.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:23:46.983 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:23:46.983 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:23:47.014 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:23:47.084 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:23:47.084 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-04-02T13:23:47.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-02T13:23:47.182 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-02T13:23:47.286 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-02T13:23:47.286 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:47.287 INFO:tasks.cephadm:Cluster fsid is 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:23:47.287 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-02T13:23:47.287 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.122', 'mon.b': '172.21.15.187', 'mon.c': '172.21.15.188'} 2024-04-02T13:23:47.287 INFO:tasks.cephadm:First mon is mon.a on smithi122 2024-04-02T13:23:47.287 INFO:tasks.cephadm:First mgr is x 2024-04-02T13:23:47.287 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-02T13:23:47.288 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-04-02T13:23:47.315 DEBUG:teuthology.orchestra.run.smithi187:> sudo hostname $(hostname -s) 2024-04-02T13:23:47.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo hostname $(hostname -s) 2024-04-02T13:23:47.376 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-02T13:23:47.377 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:47.539 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '191d963ac9de7322da8aff20b567b3fa458e2ef6', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78154/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2609-g191d963a', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2609.g191d963a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-02 02:04:04.615852', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240402.004512', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-02T13:23:47.703 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-batrick-testing-20240402.004512, sha1 191d963ac9de7322da8aff20b567b3fa458e2ef6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:23:47.705 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm 2024-04-02T13:23:47.705 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:48.234 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:23 /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:48.234 DEBUG:teuthology.orchestra.run.smithi187:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:48.747 INFO:teuthology.orchestra.run.smithi187.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:23 /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:48.748 DEBUG:teuthology.orchestra.run.smithi188:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-batrick-testing-20240402.004512/191d963ac9de7322da8aff20b567b3fa458e2ef6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:49.456 INFO:teuthology.orchestra.run.smithi188.stdout:-rw-r--r--. 1 ubuntu ubuntu 781237 Apr 2 13:23 /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:49.457 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:49.475 DEBUG:teuthology.orchestra.run.smithi187:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:49.494 DEBUG:teuthology.orchestra.run.smithi188:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-02T13:23:49.529 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 on all hosts... 2024-04-02T13:23:49.530 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:23:49.532 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:23:49.536 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 pull 2024-04-02T13:23:50.408 INFO:teuthology.orchestra.run.smithi187.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:23:50.446 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:23:50.447 INFO:teuthology.orchestra.run.smithi188.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:24:21.908 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:24:21.909 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:24:42.393 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-02T13:24:42.393 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:24:42.394 INFO:teuthology.orchestra.run.smithi187.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:24:42.394 INFO:teuthology.orchestra.run.smithi187.stdout: "repo_digests": [ 2024-04-02T13:24:42.394 INFO:teuthology.orchestra.run.smithi187.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:24:42.394 INFO:teuthology.orchestra.run.smithi187.stdout: ] 2024-04-02T13:24:42.394 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout: "image_id": "f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f", 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout: "repo_digests": [ 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534" 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout: ] 2024-04-02T13:24:44.814 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-02T13:24:44.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-04-02T13:24:44.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph 2024-04-02T13:24:44.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph 2024-04-02T13:24:44.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-04-02T13:24:44.969 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /etc/ceph 2024-04-02T13:24:44.999 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /etc/ceph 2024-04-02T13:24:45.025 INFO:tasks.cephadm:Writing seed config... 2024-04-02T13:24:45.026 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-02T13:24:45.026 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-02T13:24:45.026 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-02T13:24:45.027 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-02T13:24:45.028 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-02T13:24:45.029 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-02T13:24:45.030 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-02T13:24:45.030 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:24:45.030 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-02T13:24:45.048 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3c818384-f0f4-11ee-b647-cb9ed24678a4 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-02T13:24:45.048 DEBUG:teuthology.orchestra.run.smithi122:mon.a> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service 2024-04-02T13:24:45.091 DEBUG:teuthology.orchestra.run.smithi122:mgr.x> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service 2024-04-02T13:24:45.133 INFO:tasks.cephadm:Bootstrapping... 2024-04-02T13:24:45.133 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 -v bootstrap --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.122 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:24:45.329 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-04-02T13:24:45.330 INFO:teuthology.orchestra.run.smithi122.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6', '-v', 'bootstrap', '--fsid', '3c818384-f0f4-11ee-b647-cb9ed24678a4', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.122', '--skip-admin-label'] 2024-04-02T13:24:45.330 INFO:teuthology.orchestra.run.smithi122.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-02T13:24:45.330 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying podman|docker is present... 2024-04-02T13:24:45.355 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:24:45.355 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying lvm2 is present... 2024-04-02T13:24:45.355 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying time synchronization is in place... 2024-04-02T13:24:45.363 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:24:45.363 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:24:45.369 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:24:45.369 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:24:45.375 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-04-02T13:24:45.380 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-04-02T13:24:45.380 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:24:45.381 INFO:teuthology.orchestra.run.smithi122.stdout:Repeating the final host check... 2024-04-02T13:24:45.403 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-02T13:24:45.404 INFO:teuthology.orchestra.run.smithi122.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-02T13:24:45.404 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl is present 2024-04-02T13:24:45.404 INFO:teuthology.orchestra.run.smithi122.stdout:lvcreate is present 2024-04-02T13:24:45.409 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-02T13:24:45.410 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-02T13:24:45.415 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-02T13:24:45.415 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:24:45.421 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout enabled 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout active 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Unit chronyd.service is enabled and running 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Host looks OK 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Cluster fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Acquiring lock 140280134509232 on /run/cephadm/3c818384-f0f4-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Lock 140280134509232 acquired on /run/cephadm/3c818384-f0f4-11ee-b647-cb9ed24678a4.lock 2024-04-02T13:24:45.428 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 3300 ... 2024-04-02T13:24:45.429 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying IP 172.21.15.122 port 6789 ... 2024-04-02T13:24:45.430 INFO:teuthology.orchestra.run.smithi122.stdout:Base mon IP(s) is [172.21.15.122:3300, 172.21.15.122:6789], mon addrv is [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-04-02T13:24:45.432 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.122 metric 100 2024-04-02T13:24:45.432 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.122 metric 100 2024-04-02T13:24:45.434 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-02T13:24:45.434 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-04-02T13:24:45.434 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1777sec hoplimit 64 pref medium 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-04-02T13:24:45.437 INFO:teuthology.orchestra.run.smithi122.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-02T13:24:45.438 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-04-02T13:24:45.438 INFO:teuthology.orchestra.run.smithi122.stdout:Mon IP `172.21.15.122` is in CIDR network `172.21.0.0/20` 2024-04-02T13:24:45.438 INFO:teuthology.orchestra.run.smithi122.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-02T13:24:45.438 INFO:teuthology.orchestra.run.smithi122.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-02T13:24:45.439 INFO:teuthology.orchestra.run.smithi122.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stdout f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6... 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Getting image source signatures 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:db8c11fadded9ebce76f2e720ef87f32e8913958c61c526c89a5e4e2c41a8200 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying blob sha256:11da15500eca1f826c328eb3265caa30b0a3730510e2a32023084d6fe9cee20c 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Copying config sha256:f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f 2024-04-02T13:24:48.851 INFO:teuthology.orchestra.run.smithi122.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-02T13:24:50.063 INFO:teuthology.orchestra.run.smithi122.stdout:ceph: stdout ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:24:50.063 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph version: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev) 2024-04-02T13:24:50.063 INFO:teuthology.orchestra.run.smithi122.stdout:Extracting ceph user uid/gid from container image... 2024-04-02T13:24:50.919 INFO:teuthology.orchestra.run.smithi122.stdout:stat: stdout 167 167 2024-04-02T13:24:50.919 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial keys... 2024-04-02T13:24:51.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQAjBwxmY7N9DRAAkBO7ABgYGTnTZwe/knsy6g== 2024-04-02T13:24:52.548 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQAkBwxm78sHBBAAk3ifizPx8CEWpUiyMOqSfg== 2024-04-02T13:24:53.546 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph-authtool: stdout AQAkBwxmly59MRAA60VXKnBo/7mttz4thlDEwg== 2024-04-02T13:24:53.547 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial monmap... 2024-04-02T13:24:54.502 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool for a [v2:172.21.15.122:3300,v1:172.21.15.122:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: set fsid to 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:24:54.503 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mon... 2024-04-02T13:24:59.458 INFO:teuthology.orchestra.run.smithi122.stdout:create mon.a on 2024-04-02T13:24:59.691 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-02T13:24:59.829 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-02T13:25:00.011 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:25:00.011 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target → /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target. 2024-04-02T13:25:00.229 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T13:25:00.229 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service: Unit ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service not loaded. 2024-04-02T13:25:00.399 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target.wants/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service → /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:25:00.964 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:25:00.964 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-04-02T13:25:00.970 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:25:00.970 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:25:00.970 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-04-02T13:25:00.971 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:25:00.971 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon to start... 2024-04-02T13:25:00.971 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mon... 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout cluster: 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout id: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout services: 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.569798s) 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout data: 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-02T13:25:02.201 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout pgs: 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.398+0000 7f9e53099700 1 Processor -- start 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.398+0000 7f9e53099700 1 -- start start 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.398+0000 7f9e53099700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.398+0000 7f9e53099700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c07c640 con 0x7f9e4c108a90 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e50e35700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e50e35700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60776/0 (socket says 172.21.15.122:60776) 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e50e35700 1 -- 172.21.15.122:0/1056566616 learned_addr learned my addr 172.21.15.122:0/1056566616 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e50e35700 1 -- 172.21.15.122:0/1056566616 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c10b3c0 con 0x7f9e4c108a90 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e50e35700 1 --2- 172.21.15.122:0/1056566616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009e80 tx=0x7f9e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6061a8d3d5a0b2be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:02.202 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e4b7fe700 1 -- 172.21.15.122:0/1056566616 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e44017070 con 0x7f9e4c108a90 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e4b7fe700 1 -- 172.21.15.122:0/1056566616 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44003a20 con 0x7f9e4c108a90 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.399+0000 7f9e4b7fe700 1 -- 172.21.15.122:0/1056566616 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e44003d20 con 0x7f9e4c108a90 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 -- 172.21.15.122:0/1056566616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 msgr2=0x7f9e4c10ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 --2- 172.21.15.122:0/1056566616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009e80 tx=0x7f9e4400b330 comp rx=0 tx=0).stop 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 -- 172.21.15.122:0/1056566616 shutdown_connections 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 --2- 172.21.15.122:0/1056566616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c10ae80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 -- 172.21.15.122:0/1056566616 >> 172.21.15.122:0/1056566616 conn(0x7f9e4c1024c0 msgr2=0x7f9e4c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 -- 172.21.15.122:0/1056566616 shutdown_connections 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.400+0000 7f9e53099700 1 -- 172.21.15.122:0/1056566616 wait complete. 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e53099700 1 Processor -- start 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e53099700 1 -- start start 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e53099700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:02.203 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e53099700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10b7d0 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e50e35700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e50e35700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60792/0 (socket says 172.21.15.122:60792) 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e50e35700 1 -- 172.21.15.122:0/3800109793 learned_addr learned my addr 172.21.15.122:0/3800109793 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e50e35700 1 -- 172.21.15.122:0/3800109793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c108850 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e50e35700 1 --2- 172.21.15.122:0/3800109793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c07bad0 tx=0x7f9e44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e44022020 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44018040 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e44017070 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.401+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c106a40 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.402+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c106f80 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.402+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e44025020 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.402+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e4400f050 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.403+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c063c20 con 0x7f9e4c108a90 2024-04-02T13:25:02.204 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.405+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9e4402c030 con 0x7f9e4c108a90 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.477+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f9e4c195050 con 0x7f9e4c108a90 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.478+0000 7f9e49ffb700 1 -- 172.21.15.122:0/3800109793 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9e4401e480 con 0x7f9e4c108a90 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 msgr2=0x7f9e4c108310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 --2- 172.21.15.122:0/3800109793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c07bad0 tx=0x7f9e44004ad0 comp rx=0 tx=0).stop 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 shutdown_connections 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 --2- 172.21.15.122:0/3800109793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9e4c108a90 0x7f9e4c108310 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 >> 172.21.15.122:0/3800109793 conn(0x7f9e4c1024c0 msgr2=0x7f9e4c1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 shutdown_connections 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:01.479+0000 7f9e53099700 1 -- 172.21.15.122:0/3800109793 wait complete. 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:mon is available 2024-04-02T13:25:02.205 INFO:teuthology.orchestra.run.smithi122.stdout:Assimilating anything we can from ceph.conf... 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.122:3300,v1:172.21.15.122:6789] 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:25:03.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff602b29700 1 Processor -- start 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff602b29700 1 -- start start 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff602b29700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff602b29700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc07d0f0 con 0x7ff5fc07e670 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff6008c5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff6008c5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60794/0 (socket says 172.21.15.122:60794) 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.600+0000 7ff6008c5700 1 -- 172.21.15.122:0/3737538887 learned_addr learned my addr 172.21.15.122:0/3737538887 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.601+0000 7ff6008c5700 1 -- 172.21.15.122:0/3737538887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc07d270 con 0x7ff5fc07e670 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.601+0000 7ff6008c5700 1 --2- 172.21.15.122:0/3737538887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00098a0 tx=0x7ff5f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9739bafa432b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.601+0000 7ff5fb7fe700 1 -- 172.21.15.122:0/3737538887 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f0004160 con 0x7ff5fc07e670 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.601+0000 7ff5fb7fe700 1 -- 172.21.15.122:0/3737538887 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f0004300 con 0x7ff5fc07e670 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.601+0000 7ff5fb7fe700 1 -- 172.21.15.122:0/3737538887 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f00036a0 con 0x7ff5fc07e670 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- 172.21.15.122:0/3737538887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 msgr2=0x7ff5fc07cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 --2- 172.21.15.122:0/3737538887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00098a0 tx=0x7ff5f0009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- 172.21.15.122:0/3737538887 shutdown_connections 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 --2- 172.21.15.122:0/3737538887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc07cb20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- 172.21.15.122:0/3737538887 >> 172.21.15.122:0/3737538887 conn(0x7ff5fc1035f0 msgr2=0x7ff5fc105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- 172.21.15.122:0/3737538887 shutdown_connections 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- 172.21.15.122:0/3737538887 wait complete. 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 Processor -- start 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- start start 2024-04-02T13:25:03.233 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.602+0000 7ff602b29700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc07d7a0 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff6008c5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff6008c5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60804/0 (socket says 172.21.15.122:60804) 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff6008c5700 1 -- 172.21.15.122:0/3423134150 learned_addr learned my addr 172.21.15.122:0/3423134150 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff6008c5700 1 -- 172.21.15.122:0/3423134150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19de40 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff6008c5700 1 --2- 172.21.15.122:0/3423134150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0000c00 tx=0x7ff5f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f0003db0 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f0011630 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc19e0d0 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5f0011950 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.603+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a0d70 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.604+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5f0011af0 con 0x7ff5fc07e670 2024-04-02T13:25:03.234 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.604+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5f001c450 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.604+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc063c20 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.607+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff5f0032020 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.673+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff5fc02df80 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.676+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7ff5f0004260 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.676+0000 7ff5f9ffb700 1 -- 172.21.15.122:0/3423134150 <== mon.0 v2:172.21.15.122:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff5f001cb60 con 0x7ff5fc07e670 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 msgr2=0x7ff5fc19d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 --2- 172.21.15.122:0/3423134150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0000c00 tx=0x7ff5f0004750 comp rx=0 tx=0).stop 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 shutdown_connections 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 --2- 172.21.15.122:0/3423134150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5fc07e670 0x7ff5fc19d900 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 >> 172.21.15.122:0/3423134150 conn(0x7ff5fc1035f0 msgr2=0x7ff5fc1058d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 shutdown_connections 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:02.677+0000 7ff602b29700 1 -- 172.21.15.122:0/3423134150 wait complete. 2024-04-02T13:25:03.235 INFO:teuthology.orchestra.run.smithi122.stdout:Generating new minimal ceph.conf... 2024-04-02T13:25:04.669 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f439b721700 1 Processor -- start 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f439b721700 1 -- start start 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f439b721700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f439b721700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43941089d0 con 0x7f4394108070 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f43994bd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f43994bd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60806/0 (socket says 172.21.15.122:60806) 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.627+0000 7f43994bd700 1 -- 172.21.15.122:0/3137544889 learned_addr learned my addr 172.21.15.122:0/3137544889 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f43994bd700 1 -- 172.21.15.122:0/3137544889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4394108b50 con 0x7f4394108070 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f43994bd700 1 --2- 172.21.15.122:0/3137544889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43840098a0 tx=0x7f4384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672f358bfc0dd08e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f438bfff700 1 -- 172.21.15.122:0/3137544889 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4384004160 con 0x7f4394108070 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f438bfff700 1 -- 172.21.15.122:0/3137544889 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4384004300 con 0x7f4394108070 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f438bfff700 1 -- 172.21.15.122:0/3137544889 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4384011b50 con 0x7f4394108070 2024-04-02T13:25:04.670 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f439b721700 1 -- 172.21.15.122:0/3137544889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 msgr2=0x7f4394108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f439b721700 1 --2- 172.21.15.122:0/3137544889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43840098a0 tx=0x7f4384009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f439b721700 1 -- 172.21.15.122:0/3137544889 shutdown_connections 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.628+0000 7f439b721700 1 --2- 172.21.15.122:0/3137544889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f4394108490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 -- 172.21.15.122:0/3137544889 >> 172.21.15.122:0/3137544889 conn(0x7f43941036d0 msgr2=0x7f4394105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 -- 172.21.15.122:0/3137544889 shutdown_connections 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 -- 172.21.15.122:0/3137544889 wait complete. 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 Processor -- start 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 -- start start 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f439b721700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f439410c490 con 0x7f4394108070 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f43994bd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f43994bd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60820/0 (socket says 172.21.15.122:60820) 2024-04-02T13:25:04.671 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.629+0000 7f43994bd700 1 -- 172.21.15.122:0/1046702425 learned_addr learned my addr 172.21.15.122:0/1046702425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f43994bd700 1 -- 172.21.15.122:0/1046702425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439419c4d0 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f43994bd700 1 --2- 172.21.15.122:0/1046702425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4384011670 tx=0x7f4384011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43840039c0 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4384011e50 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439419c760 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f438401a5c0 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439419f450 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f438401ac50 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.630+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4384019700 con 0x7f4394108070 2024-04-02T13:25:04.672 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.631+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4394063c20 con 0x7f4394108070 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.633+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4384003b60 con 0x7f4394108070 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.700+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f439419f620 con 0x7f4394108070 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.700+0000 7f438a7fc700 1 -- 172.21.15.122:0/1046702425 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7f4384019900 con 0x7f4394108070 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 msgr2=0x7f439419bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 --2- 172.21.15.122:0/1046702425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4384011670 tx=0x7f4384011b60 comp rx=0 tx=0).stop 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 shutdown_connections 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 --2- 172.21.15.122:0/1046702425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4394108070 0x7f439419bf90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 >> 172.21.15.122:0/1046702425 conn(0x7f43941036d0 msgr2=0x7f4394104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:04.674 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 shutdown_connections 2024-04-02T13:25:04.674 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:03.701+0000 7f439b721700 1 -- 172.21.15.122:0/1046702425 wait complete. 2024-04-02T13:25:04.674 INFO:teuthology.orchestra.run.smithi122.stdout:Restarting the monitor... 2024-04-02T13:25:04.932 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:04 smithi122 systemd[1]: Stopping Ceph mon.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:25:04.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a[31738]: 2024-04-02T13:25:04.928+0000 7f8c988a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:25:04.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a[31738]: 2024-04-02T13:25:04.928+0000 7f8c988a2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-02T13:25:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:04 smithi122 podman[31987]: 2024-04-02 13:25:04.940003085 +0000 UTC m=+0.107668293 container died f6816d089f0d92c656134ca0208f4758c9cad782d4e6b649102187d166b9754e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-02T13:25:05.239 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 podman[31987]: 2024-04-02 13:25:05.237724764 +0000 UTC m=+0.405389969 container cleanup f6816d089f0d92c656134ca0208f4758c9cad782d4e6b649102187d166b9754e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2) 2024-04-02T13:25:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 bash[31987]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T13:25:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 podman[32000]: 2024-04-02 13:25:05.480500381 +0000 UTC m=+0.537581354 container remove f6816d089f0d92c656134ca0208f4758c9cad782d4e6b649102187d166b9754e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-02T13:25:05.922 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service: Deactivated successfully. 2024-04-02T13:25:05.922 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 systemd[1]: Stopped Ceph mon.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:25:05.922 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 systemd[1]: Starting Ceph mon.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:25:05.922 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 podman[32095]: 2024-04-02 13:25:05.714604404 +0000 UTC m=+0.017487048 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:25:05.922 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:05 smithi122 podman[32095]: 2024-04-02 13:25:05.814536121 +0000 UTC m=+0.117418744 container create ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:25:06.097 INFO:teuthology.orchestra.run.smithi122.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-02T13:25:06.311 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 podman[32095]: 2024-04-02 13:25:06.017123693 +0000 UTC m=+0.320006331 container init ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 podman[32095]: 2024-04-02 13:25:06.02020417 +0000 UTC m=+0.323086797 container start ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: set uid:gid to 167:167 (ceph:ceph) 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev), process ceph-mon, pid 2 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: pidfile_write: ignore empty --pid-file 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: load: jerasure load: lrc 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Git sha 0 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: DB SUMMARY 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: DB Session ID: ZACRJU0X9RNGPQFMXU6N 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:25:06.312 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105106 ; 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.env: 0x56542fd70c60 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.info_log: 0x565430f55ae0 2024-04-02T13:25:06.313 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.statistics: (nil) 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.use_fsync: 0 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:25:06.315 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.db_log_dir: 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.wal_dir: 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:25:06.316 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.write_buffer_manager: 0x565430f261e0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.unordered_write: 0 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:25:06.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.row_cache: None 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.wal_filter: None 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.wal_compression: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:25:06.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:25:06.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_open_files: -1 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Compression algorithms supported: 2024-04-02T13:25:06.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kZSTD supported: 0 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.merge_operator: 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_filter: None 2024-04-02T13:25:06.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x565430f55cc0) 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_type: 0 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_index_type: 0 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_shortening: 1 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: checksum: 4 2024-04-02T13:25:06.322 INFO:journalctl@ceph.mon.a.smithi122.stdout: no_block_cache: 0 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache: 0x565430f59090 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_options: 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: capacity : 536870912 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_shard_bits : 4 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: strict_capacity_limit : 0 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_cache_compressed: (nil) 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: persistent_cache: (nil) 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size: 4096 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_size_deviation: 10 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_restart_interval: 16 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: index_block_restart_interval: 1 2024-04-02T13:25:06.323 INFO:journalctl@ceph.mon.a.smithi122.stdout: metadata_block_size: 4096 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: partition_filters: 0 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: use_delta_encoding: 1 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: filter_policy: bloomfilter 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: whole_key_filtering: 1 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: verify_compression: 0 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: format_version: 5 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_index_compression: 1 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: block_align: 0 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: max_auto_readahead_size: 262144 2024-04-02T13:25:06.324 INFO:journalctl@ceph.mon.a.smithi122.stdout: prepopulate_block_cache: 0 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression: NoCompression 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.num_levels: 7 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:25:06.325 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:25:06.326 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:25:06.327 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:25:06.328 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:25:06.329 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.ttl: 2592000 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:25:06.330 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 77caec49-3336-4357-a417-1be209b811d4 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064306049375, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-02T13:25:06.331 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064306050347, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100239, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98127, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14167, "raw_average_key_size": 49, "raw_value_size": 90444, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712064306, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "77caec49-3336-4357-a417-1be209b811d4", "db_session_id": "ZACRJU0X9RNGPQFMXU6N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064306050436, "job": 1, "event": "recovery_finished"} 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565431048000 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: DB pointer 0x565431032000 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** DB Stats ** 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:25:06.332 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: L0 2/0 99.75 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Sum 2/0 99.75 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** Compaction Stats [default] ** 2024-04-02T13:25:06.333 INFO:journalctl@ceph.mon.a.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Cumulative compaction: 0.00 GB write, 32.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Interval compaction: 0.00 GB write, 32.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:25:06.334 INFO:journalctl@ceph.mon.a.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache BinnedLRUCache@0x565430f59090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: starting mon.a rank 0 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???) e1 preinit fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).mds e1 new map 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).mds e1 print_map 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: e1 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: legacy client fscid: -1 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: 2024-04-02T13:25:06.335 INFO:journalctl@ceph.mon.a.smithi122.stdout: No filesystems configured 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 bash[32095]: ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 systemd[1]: Started Ceph mon.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: monmap epoch 1 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: last_changed 2024-04-02T13:24:53.926501+0000 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: min_mon_release 19 (squid) 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: election_strategy: 1 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:25:06.336 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: fsmap 2024-04-02T13:25:06.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: osdmap e1: 0 total, 0 up, 0 in 2024-04-02T13:25:06.337 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:06 smithi122 ceph-mon[32114]: mgrmap e1: no daemons active 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7437f33700 1 Processor -- start 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7437f33700 1 -- start start 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7437f33700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7437f33700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f743007c680 con 0x7f7430108a90 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7435ccf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7435ccf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60832/0 (socket says 172.21.15.122:60832) 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.513+0000 7f7435ccf700 1 -- 172.21.15.122:0/91923593 learned_addr learned my addr 172.21.15.122:0/91923593 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.514+0000 7f7435ccf700 1 -- 172.21.15.122:0/91923593 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743010b3c0 con 0x7f7430108a90 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.514+0000 7f7435ccf700 1 --2- 172.21.15.122:0/91923593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f742c0098a0 tx=0x7f742c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dbbe7e2400d8ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.514+0000 7f7434ccd700 1 -- 172.21.15.122:0/91923593 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742c004160 con 0x7f7430108a90 2024-04-02T13:25:07.141 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.514+0000 7f7434ccd700 1 -- 172.21.15.122:0/91923593 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f742c004300 con 0x7f7430108a90 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.514+0000 7f7434ccd700 1 -- 172.21.15.122:0/91923593 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742c011b50 con 0x7f7430108a90 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- 172.21.15.122:0/91923593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 msgr2=0x7f743010ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 --2- 172.21.15.122:0/91923593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f742c0098a0 tx=0x7f742c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- 172.21.15.122:0/91923593 shutdown_connections 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 --2- 172.21.15.122:0/91923593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743010ae80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- 172.21.15.122:0/91923593 >> 172.21.15.122:0/91923593 conn(0x7f7430102500 msgr2=0x7f7430104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- 172.21.15.122:0/91923593 shutdown_connections 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- 172.21.15.122:0/91923593 wait complete. 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 Processor -- start 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- start start 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.515+0000 7f7437f33700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f743010c3e0 con 0x7f7430108a90 2024-04-02T13:25:07.142 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7435ccf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7435ccf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60842/0 (socket says 172.21.15.122:60842) 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7435ccf700 1 -- 172.21.15.122:0/3376828209 learned_addr learned my addr 172.21.15.122:0/3376828209 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7435ccf700 1 -- 172.21.15.122:0/3376828209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743019c540 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7435ccf700 1 --2- 172.21.15.122:0/3376828209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f742c009fb0 tx=0x7f742c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742c0038b0 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743019c7d0 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f742c011e50 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f742c019420 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743019f4c0 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f742c019ad0 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.516+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f742c024730 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.517+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7430063c20 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.519+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f742c043dd0 con 0x7f7430108a90 2024-04-02T13:25:07.143 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.589+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f743019f690 con 0x7f7430108a90 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.591+0000 7f7426ffd700 1 -- 172.21.15.122:0/3376828209 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f742c032030 con 0x7f7430108a90 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 msgr2=0x7f743019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 --2- 172.21.15.122:0/3376828209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f742c009fb0 tx=0x7f742c011bf0 comp rx=0 tx=0).stop 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 shutdown_connections 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 --2- 172.21.15.122:0/3376828209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7430108a90 0x7f743019c000 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 >> 172.21.15.122:0/3376828209 conn(0x7f7430102500 msgr2=0x7f7430103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 shutdown_connections 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:06.592+0000 7f7437f33700 1 -- 172.21.15.122:0/3376828209 wait complete. 2024-04-02T13:25:07.144 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-02T13:25:07.158 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:25:07.158 INFO:teuthology.orchestra.run.smithi122.stdout:Creating mgr... 2024-04-02T13:25:07.158 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-02T13:25:07.159 INFO:teuthology.orchestra.run.smithi122.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-02T13:25:07.433 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T13:25:07.433 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service: Unit ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service not loaded. 2024-04-02T13:25:07.586 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4.target.wants/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service → /etc/systemd/system/ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@.service. 2024-04-02T13:25:07.907 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3376828209' entity='client.admin' 2024-04-02T13:25:08.097 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:25:08.097 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-04-02T13:25:08.104 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:25:08.105 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:25:08.105 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-04-02T13:25:08.105 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to enable service . firewalld.service is not available 2024-04-02T13:25:08.113 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:25:08.113 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-04-02T13:25:08.121 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:25:08.121 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:25:08.121 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-04-02T13:25:08.121 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-02T13:25:08.121 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr to start... 2024-04-02T13:25:08.122 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr... 2024-04-02T13:25:08.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3506492052' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:25:08.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:08.678+0000 7f1bb6943200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:25:09.165 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:25:09.166 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:25:09.167 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:25:09.168 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:25:00.910782+0000", 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:25:09.169 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.506+0000 7fe3b3517700 1 Processor -- start 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.506+0000 7fe3b3517700 1 -- start start 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.507+0000 7fe3b3517700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.507+0000 7fe3b3517700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac108a20 con 0x7fe3ac108030 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3b12b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3b12b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33810/0 (socket says 172.21.15.122:33810) 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3b12b3700 1 -- 172.21.15.122:0/4246999633 learned_addr learned my addr 172.21.15.122:0/4246999633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3b12b3700 1 -- 172.21.15.122:0/4246999633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac109190 con 0x7fe3ac108030 2024-04-02T13:25:09.170 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3b12b3700 1 --2- 172.21.15.122:0/4246999633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aef047b1870b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3a3fff700 1 -- 172.21.15.122:0/4246999633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe39c004160 con 0x7fe3ac108030 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3a3fff700 1 -- 172.21.15.122:0/4246999633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe39c004300 con 0x7fe3ac108030 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.508+0000 7fe3a3fff700 1 -- 172.21.15.122:0/4246999633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe39c011b50 con 0x7fe3ac108030 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 -- 172.21.15.122:0/4246999633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 msgr2=0x7fe3ac108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 --2- 172.21.15.122:0/4246999633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe39c0098a0 tx=0x7fe39c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 -- 172.21.15.122:0/4246999633 shutdown_connections 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 --2- 172.21.15.122:0/4246999633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac108450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 -- 172.21.15.122:0/4246999633 >> 172.21.15.122:0/4246999633 conn(0x7fe3ac1035f0 msgr2=0x7fe3ac105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 -- 172.21.15.122:0/4246999633 shutdown_connections 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.509+0000 7fe3b3517700 1 -- 172.21.15.122:0/4246999633 wait complete. 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b3517700 1 Processor -- start 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b3517700 1 -- start start 2024-04-02T13:25:09.171 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b3517700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b3517700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3ac10c540 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b12b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b12b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33820/0 (socket says 172.21.15.122:33820) 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.510+0000 7fe3b12b3700 1 -- 172.21.15.122:0/3506492052 learned_addr learned my addr 172.21.15.122:0/3506492052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3b12b3700 1 -- 172.21.15.122:0/3506492052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3ac07e8d0 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3b12b3700 1 --2- 172.21.15.122:0/3506492052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000c00 tx=0x7fe39c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe39c003b20 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe39c003cc0 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3ac07cac0 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe39c024910 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3ac07d000 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe39c022020 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.511+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe39c011e00 con 0x7fe3ac108030 2024-04-02T13:25:09.172 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.512+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3ac063c20 con 0x7fe3ac108030 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.514+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe39c042dd0 con 0x7fe3ac108030 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.587+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe3ac07d530 con 0x7fe3ac108030 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.588+0000 7fe3a27fc700 1 -- 172.21.15.122:0/3506492052 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe39c031020 con 0x7fe3ac108030 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 msgr2=0x7fe3ac07e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 --2- 172.21.15.122:0/3506492052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fe39c000c00 tx=0x7fe39c011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 shutdown_connections 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 --2- 172.21.15.122:0/3506492052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3ac108030 0x7fe3ac07e390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 >> 172.21.15.122:0/3506492052 conn(0x7fe3ac1035f0 msgr2=0x7fe3ac104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 shutdown_connections 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:08.589+0000 7fe3b3517700 1 -- 172.21.15.122:0/3506492052 wait complete. 2024-04-02T13:25:09.173 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (1/15)... 2024-04-02T13:25:09.654 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.328+0000 7f1bb6943200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:25:09.654 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.419+0000 7f1bb6943200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:25:09.654 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.558+0000 7f1bb6943200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:25:09.995 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.651+0000 7f1bb6943200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:25:09.995 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.734+0000 7f1bb6943200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:25:09.995 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:09.993+0000 7f1bb6943200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:25:10.283 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:10.102+0000 7f1bb6943200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:10.283 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:10.187+0000 7f1bb6943200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:25:10.283 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:10.281+0000 7f1bb6943200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:25:10.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:10.454+0000 7f1bb6943200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:25:11.512 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:11.432+0000 7f1bb6943200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:25:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1746663415' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:25:11.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:11.570+0000 7f1bb6943200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:25:12.201 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:11.911+0000 7f1bb6943200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:25:12.201 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:12.001+0000 7f1bb6943200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:25:12.202 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:12.199+0000 7f1bb6943200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:25:12.221 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:25:12.222 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:25:12.223 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:25:12.224 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:25:00.910782+0000", 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:12.225 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.510+0000 7fb9cae15700 1 Processor -- start 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.510+0000 7fb9cae15700 1 -- start start 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.510+0000 7fb9cae15700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.510+0000 7fb9cae15700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4106c80 con 0x7fb9c4106290 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c8bb1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c8bb1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33834/0 (socket says 172.21.15.122:33834) 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c8bb1700 1 -- 172.21.15.122:0/1151143573 learned_addr learned my addr 172.21.15.122:0/1151143573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c8bb1700 1 -- 172.21.15.122:0/1151143573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c4109220 con 0x7fb9c4106290 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c8bb1700 1 --2- 172.21.15.122:0/1151143573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80098a0 tx=0x7fb9b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3baea0449c7e6377 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c37fe700 1 -- 172.21.15.122:0/1151143573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8004160 con 0x7fb9c4106290 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c37fe700 1 -- 172.21.15.122:0/1151143573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9b8004300 con 0x7fb9c4106290 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.511+0000 7fb9c37fe700 1 -- 172.21.15.122:0/1151143573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b8011b50 con 0x7fb9c4106290 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 -- 172.21.15.122:0/1151143573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 msgr2=0x7fb9c41066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 --2- 172.21.15.122:0/1151143573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9b80098a0 tx=0x7fb9b8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 -- 172.21.15.122:0/1151143573 shutdown_connections 2024-04-02T13:25:12.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 --2- 172.21.15.122:0/1151143573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c41066b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 -- 172.21.15.122:0/1151143573 >> 172.21.15.122:0/1151143573 conn(0x7fb9c4101be0 msgr2=0x7fb9c4104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 -- 172.21.15.122:0/1151143573 shutdown_connections 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.512+0000 7fb9cae15700 1 -- 172.21.15.122:0/1151143573 wait complete. 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9cae15700 1 Processor -- start 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9cae15700 1 -- start start 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9cae15700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9cae15700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c410c5d0 con 0x7fb9c4106290 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c8bb1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c8bb1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33846/0 (socket says 172.21.15.122:33846) 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c8bb1700 1 -- 172.21.15.122:0/1746663415 learned_addr learned my addr 172.21.15.122:0/1746663415 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c8bb1700 1 -- 172.21.15.122:0/1746663415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419c4f0 con 0x7fb9c4106290 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c8bb1700 1 --2- 172.21.15.122:0/1746663415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8011670 tx=0x7fb9b8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.513+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b80039c0 con 0x7fb9c4106290 2024-04-02T13:25:12.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419c780 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb9b8011e50 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9b801a5c0 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c419f470 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb9b801ac50 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.514+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb9b8019700 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.515+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c4063c20 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.519+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb9b8003b60 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.591+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb9c419f9b0 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.591+0000 7fb9c1ffb700 1 -- 172.21.15.122:0/1746663415 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb9b80199a0 con 0x7fb9c4106290 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 msgr2=0x7fb9c419bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 --2- 172.21.15.122:0/1746663415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8011670 tx=0x7fb9b8011b60 comp rx=0 tx=0).stop 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 shutdown_connections 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 --2- 172.21.15.122:0/1746663415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9c4106290 0x7fb9c419bfb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 >> 172.21.15.122:0/1746663415 conn(0x7fb9c4101be0 msgr2=0x7fb9c4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:12.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 shutdown_connections 2024-04-02T13:25:12.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:11.592+0000 7fb9cae15700 1 -- 172.21.15.122:0/1746663415 wait complete. 2024-04-02T13:25:12.229 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (2/15)... 2024-04-02T13:25:12.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:12.387+0000 7f1bb6943200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:25:12.925 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:12.669+0000 7f1bb6943200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:13.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:12.923+0000 7f1bb6943200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:13.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:13.159+0000 7f1bb6943200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:25:14.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:13.951+0000 7f1bb6943200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:25:14.611 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:14.525+0000 7f1bb6943200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:25:14.965 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1921929557' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:25:14.965 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:14.629+0000 7f1bb6943200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:25:14.965 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:14.851+0000 7f1bb6943200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:25:15.287 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:15.288 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:25:15.289 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:25:15.290 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:15.291 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:25:00.910782+0000", 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.617+0000 7fae0d1c9700 1 Processor -- start 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.617+0000 7fae0d1c9700 1 -- start start 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae0d1c9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae0d1c9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae081077f0 con 0x7fae08106e90 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33848/0 (socket says 172.21.15.122:33848) 2024-04-02T13:25:15.292 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae06d9d700 1 -- 172.21.15.122:0/1419178845 learned_addr learned my addr 172.21.15.122:0/1419178845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae06d9d700 1 -- 172.21.15.122:0/1419178845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae08108080 con 0x7fae08106e90 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae06d9d700 1 --2- 172.21.15.122:0/1419178845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadf80098a0 tx=0x7fadf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9581ab5e9264a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.618+0000 7fae05d9b700 1 -- 172.21.15.122:0/1419178845 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8004160 con 0x7fae08106e90 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae05d9b700 1 -- 172.21.15.122:0/1419178845 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fadf8004300 con 0x7fae08106e90 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae05d9b700 1 -- 172.21.15.122:0/1419178845 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8011b50 con 0x7fae08106e90 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1419178845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 msgr2=0x7fae081072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 --2- 172.21.15.122:0/1419178845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fadf80098a0 tx=0x7fadf8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1419178845 shutdown_connections 2024-04-02T13:25:15.293 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 --2- 172.21.15.122:0/1419178845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae081072b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1419178845 >> 172.21.15.122:0/1419178845 conn(0x7fae08102500 msgr2=0x7fae08104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1419178845 shutdown_connections 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.619+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1419178845 wait complete. 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae0d1c9700 1 Processor -- start 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae0d1c9700 1 -- start start 2024-04-02T13:25:15.294 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae0d1c9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae0d1c9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae0810b340 con 0x7fae08106e90 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae06d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33852/0 (socket says 172.21.15.122:33852) 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.620+0000 7fae06d9d700 1 -- 172.21.15.122:0/1921929557 learned_addr learned my addr 172.21.15.122:0/1921929557 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fae06d9d700 1 -- 172.21.15.122:0/1921929557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae0819c5c0 con 0x7fae08106e90 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fae06d9d700 1 --2- 172.21.15.122:0/1921929557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009fb0 tx=0x7fadf8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:15.295 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf80038b0 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fadf8011e50 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae0819c850 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8019550 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0819f540 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fadf8019be0 con 0x7fae08106e90 2024-04-02T13:25:15.296 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.621+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fadf8024730 con 0x7fae08106e90 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.622+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0810b4c0 con 0x7fae08106e90 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.624+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fadf8003a50 con 0x7fae08106e90 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.697+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fae0802dff0 con 0x7fae08106e90 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.697+0000 7fadeffff700 1 -- 172.21.15.122:0/1921929557 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fadf8024930 con 0x7fae08106e90 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 msgr2=0x7fae0819c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 --2- 172.21.15.122:0/1921929557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009fb0 tx=0x7fadf8011bf0 comp rx=0 tx=0).stop 2024-04-02T13:25:15.297 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 shutdown_connections 2024-04-02T13:25:15.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 --2- 172.21.15.122:0/1921929557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae08106e90 0x7fae0819c080 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:15.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 >> 172.21.15.122:0/1921929557 conn(0x7fae08102500 msgr2=0x7fae08103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:15.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 shutdown_connections 2024-04-02T13:25:15.298 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:14.698+0000 7fae0d1c9700 1 -- 172.21.15.122:0/1921929557 wait complete. 2024-04-02T13:25:15.298 INFO:teuthology.orchestra.run.smithi122.stdout:mgr not available, waiting (3/15)... 2024-04-02T13:25:16.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: Activating manager daemon x 2024-04-02T13:25:16.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: mgrmap e2: x(active, starting, since 0.00315613s) 2024-04-02T13:25:16.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: Manager daemon x is now available 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' 2024-04-02T13:25:16.032 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:15 smithi122 ceph-mon[32114]: from='mgr.14100 172.21.15.122:0/2666729161' entity='mgr.x' 2024-04-02T13:25:17.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:16 smithi122 ceph-mon[32114]: mgrmap e3: x(active, since 1.00658s) 2024-04-02T13:25:18.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:17 smithi122 ceph-mon[32114]: mgrmap e4: x(active, since 2s) 2024-04-02T13:25:18.516 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:18.519 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:18.521 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:25:18.521 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "health": { 2024-04-02T13:25:18.521 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 0 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "a" 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-02T13:25:18.522 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-02T13:25:18.523 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.523 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-02T13:25:18.523 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:18.523 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-02T13:25:18.523 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-02T13:25:18.531 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-02T13:25:18.531 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.532 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "restful" 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ], 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-02T13:25:18.533 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "modified": "2024-04-02T13:25:00.910782+0000", 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout }, 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.713+0000 7f3a18452700 1 Processor -- start 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.713+0000 7f3a18452700 1 -- start start 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.713+0000 7f3a18452700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.713+0000 7f3a18452700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a101089d0 con 0x7f3a10108070 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a161ee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a161ee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33914/0 (socket says 172.21.15.122:33914) 2024-04-02T13:25:18.534 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a161ee700 1 -- 172.21.15.122:0/2083287594 learned_addr learned my addr 172.21.15.122:0/2083287594 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a161ee700 1 -- 172.21.15.122:0/2083287594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10108b50 con 0x7f3a10108070 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a161ee700 1 --2- 172.21.15.122:0/2083287594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba712807cc378a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a151ec700 1 -- 172.21.15.122:0/2083287594 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c004160 con 0x7f3a10108070 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a151ec700 1 -- 172.21.15.122:0/2083287594 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a0c004300 con 0x7f3a10108070 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.714+0000 7f3a151ec700 1 -- 172.21.15.122:0/2083287594 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c011b50 con 0x7f3a10108070 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 -- 172.21.15.122:0/2083287594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 msgr2=0x7f3a10108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 --2- 172.21.15.122:0/2083287594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 -- 172.21.15.122:0/2083287594 shutdown_connections 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 --2- 172.21.15.122:0/2083287594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a10108490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 -- 172.21.15.122:0/2083287594 >> 172.21.15.122:0/2083287594 conn(0x7f3a101036d0 msgr2=0x7f3a10105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 -- 172.21.15.122:0/2083287594 shutdown_connections 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 -- 172.21.15.122:0/2083287594 wait complete. 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.715+0000 7f3a18452700 1 Processor -- start 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a18452700 1 -- start start 2024-04-02T13:25:18.535 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a18452700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a18452700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010c490 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a161ee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a161ee700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33930/0 (socket says 172.21.15.122:33930) 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a161ee700 1 -- 172.21.15.122:0/3054730220 learned_addr learned my addr 172.21.15.122:0/3054730220 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a161ee700 1 -- 172.21.15.122:0/3054730220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019c4d0 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a161ee700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011b60 tx=0x7f3a0c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c003b20 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3a0c003cc0 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c024840 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019c760 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.716+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1019f450 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.717+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f3a0c022020 con 0x7f3a10108070 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.717+0000 7f3a077fe700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 0x7f39fc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:18.536 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.717+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a0c011e00 con 0x7f3a10108070 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.717+0000 7f3a159ed700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 0x7f39fc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.717+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10063c20 con 0x7f3a10108070 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.718+0000 7f3a159ed700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 0x7f39fc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006fd0 tx=0x7f3a00006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:17.723+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a0c0172b0 con 0x7f3a10108070 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.008+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a1019f930 con 0x7f3a10108070 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.008+0000 7f3a077fe700 1 -- 172.21.15.122:0/3054730220 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3a0c00ba30 con 0x7f3a10108070 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 msgr2=0x7f39fc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 0x7f39fc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a00006fd0 tx=0x7f3a00006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 msgr2=0x7f3a1019bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c011b60 tx=0x7f3a0c011c40 comp rx=0 tx=0).stop 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 shutdown_connections 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f39fc0395b0 0x7f39fc03ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 --2- 172.21.15.122:0/3054730220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a10108070 0x7f3a1019bf90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.010+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 >> 172.21.15.122:0/3054730220 conn(0x7f3a101036d0 msgr2=0x7f3a10104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.011+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 shutdown_connections 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.011+0000 7f3a18452700 1 -- 172.21.15.122:0/3054730220 wait complete. 2024-04-02T13:25:18.537 INFO:teuthology.orchestra.run.smithi122.stdout:mgr is available 2024-04-02T13:25:19.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:18 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3054730220' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [global] 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout fsid = 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-02T13:25:19.587 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout [osd] 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f233185c700 1 Processor -- start 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f233185c700 1 -- start start 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f233185c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f233185c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f232c1089a0 con 0x7f232c107fb0 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f232affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f232affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51590/0 (socket says 172.21.15.122:51590) 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.921+0000 7f232affd700 1 -- 172.21.15.122:0/1290803474 learned_addr learned my addr 172.21.15.122:0/1290803474 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.922+0000 7f232affd700 1 -- 172.21.15.122:0/1290803474 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c1091b0 con 0x7f232c107fb0 2024-04-02T13:25:19.588 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.922+0000 7f232affd700 1 --2- 172.21.15.122:0/1290803474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee93428051da4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.922+0000 7f232a7fc700 1 -- 172.21.15.122:0/1290803474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c004160 con 0x7f232c107fb0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.922+0000 7f232a7fc700 1 -- 172.21.15.122:0/1290803474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f231c004300 con 0x7f232c107fb0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.922+0000 7f232a7fc700 1 -- 172.21.15.122:0/1290803474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c011b50 con 0x7f232c107fb0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- 172.21.15.122:0/1290803474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 msgr2=0x7f232c1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 --2- 172.21.15.122:0/1290803474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f231c0098a0 tx=0x7f231c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- 172.21.15.122:0/1290803474 shutdown_connections 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 --2- 172.21.15.122:0/1290803474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c1083d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- 172.21.15.122:0/1290803474 >> 172.21.15.122:0/1290803474 conn(0x7f232c103590 msgr2=0x7f232c1059b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- 172.21.15.122:0/1290803474 shutdown_connections 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- 172.21.15.122:0/1290803474 wait complete. 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 Processor -- start 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.923+0000 7f233185c700 1 -- start start 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233185c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233185c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f232c10c560 con 0x7f232c107fb0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f232affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:19.589 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f232affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51596/0 (socket says 172.21.15.122:51596) 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f232affd700 1 -- 172.21.15.122:0/2232027948 learned_addr learned my addr 172.21.15.122:0/2232027948 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f232affd700 1 -- 172.21.15.122:0/2232027948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232c19c510 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f232affd700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f231c009fb0 tx=0x7f231c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c0038b0 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f231c011e50 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f231c019550 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232c19c7a0 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.924+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232c19f490 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.925+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f231c01a430 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.925+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232c063c20 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.925+0000 7f233085a700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 0x7f231803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.925+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f231c04d890 con 0x7f232c107fb0 2024-04-02T13:25:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.925+0000 7f23227fc700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 0x7f231803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.926+0000 7f23227fc700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 0x7f231803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:18.930+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f231c016df0 con 0x7f232c107fb0 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.126+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f232c19f660 con 0x7f232c107fb0 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.127+0000 7f233085a700 1 -- 172.21.15.122:0/2232027948 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f232c19f660 con 0x7f232c107fb0 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 msgr2=0x7f231803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 0x7f231803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2314006fd0 tx=0x7f2314006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 msgr2=0x7f232c19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f231c009fb0 tx=0x7f231c011bf0 comp rx=0 tx=0).stop 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 shutdown_connections 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2318039560 0x7f231803ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 --2- 172.21.15.122:0/2232027948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f232c107fb0 0x7f232c19bfd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 >> 172.21.15.122:0/2232027948 conn(0x7f232c103590 msgr2=0x7f232c104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 shutdown_connections 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.129+0000 7f233185c700 1 -- 172.21.15.122:0/2232027948 wait complete. 2024-04-02T13:25:19.591 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling cephadm module... 2024-04-02T13:25:19.956 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2232027948' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-02T13:25:21.176 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:20 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/249232643' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-02T13:25:21.176 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setuser ceph since I am not root 2024-04-02T13:25:21.176 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setgroup ceph since I am not root 2024-04-02T13:25:21.435 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:21.422+0000 7f41b7628200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.954+0000 7f2a90761700 1 Processor -- start 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.954+0000 7f2a90761700 1 -- start start 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a90761700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a90761700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88108a30 con 0x7f2a88108040 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8e4fd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8e4fd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51604/0 (socket says 172.21.15.122:51604) 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8e4fd700 1 -- 172.21.15.122:0/3265385161 learned_addr learned my addr 172.21.15.122:0/3265385161 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8e4fd700 1 -- 172.21.15.122:0/3265385161 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a881091a0 con 0x7f2a88108040 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8e4fd700 1 --2- 172.21.15.122:0/3265385161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a840098a0 tx=0x7f2a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b64d34ba3466cba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8d4fb700 1 -- 172.21.15.122:0/3265385161 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84004160 con 0x7f2a88108040 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8d4fb700 1 -- 172.21.15.122:0/3265385161 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a84004300 con 0x7f2a88108040 2024-04-02T13:25:21.443 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.955+0000 7f2a8d4fb700 1 -- 172.21.15.122:0/3265385161 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84011b50 con 0x7f2a88108040 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 -- 172.21.15.122:0/3265385161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 msgr2=0x7f2a88108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 --2- 172.21.15.122:0/3265385161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a840098a0 tx=0x7f2a84009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 -- 172.21.15.122:0/3265385161 shutdown_connections 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 --2- 172.21.15.122:0/3265385161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a88108460 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 -- 172.21.15.122:0/3265385161 >> 172.21.15.122:0/3265385161 conn(0x7f2a881035e0 msgr2=0x7f2a88105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 -- 172.21.15.122:0/3265385161 shutdown_connections 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.956+0000 7f2a90761700 1 -- 172.21.15.122:0/3265385161 wait complete. 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 Processor -- start 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 -- start start 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8810c550 con 0x7f2a88108040 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a8e4fd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a8e4fd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51616/0 (socket says 172.21.15.122:51616) 2024-04-02T13:25:21.444 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a8e4fd700 1 -- 172.21.15.122:0/249232643 learned_addr learned my addr 172.21.15.122:0/249232643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a8e4fd700 1 -- 172.21.15.122:0/249232643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8819c540 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a8e4fd700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fb0 tx=0x7f2a84011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a840038b0 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2a84011e50 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8819c7d0 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a84019550 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.957+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8819f4c0 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.958+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f2a84019850 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.958+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a88063c20 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.958+0000 7f2a7f7fe700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 0x7f2a7403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.958+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2a8404d950 con 0x7f2a88108040 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.959+0000 7f2a8dcfc700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 0x7f2a7403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.959+0000 7f2a8dcfc700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 0x7f2a7403b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a78006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:21.445 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:19.963+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2a8404d370 con 0x7f2a88108040 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.215+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2a8819f690 con 0x7f2a88108040 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.864+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2a8819f690 con 0x7f2a88108040 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.865+0000 7f2a7f7fe700 1 -- 172.21.15.122:0/249232643 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f2a8402d960 con 0x7f2a88108040 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 msgr2=0x7f2a7403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 0x7f2a7403b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a78006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 msgr2=0x7f2a8819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fb0 tx=0x7f2a84011bf0 comp rx=0 tx=0).stop 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 shutdown_connections 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f2a74039510 0x7f2a7403b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 --2- 172.21.15.122:0/249232643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a88108040 0x7f2a8819c000 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 >> 172.21.15.122:0/249232643 conn(0x7f2a881035e0 msgr2=0x7f2a881041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 shutdown_connections 2024-04-02T13:25:21.446 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:20.867+0000 7f2a90761700 1 -- 172.21.15.122:0/249232643 wait complete. 2024-04-02T13:25:21.745 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:21.521+0000 7f41b7628200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:25:22.049 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:21 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/249232643' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-02T13:25:22.049 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:21 smithi122 ceph-mon[32114]: mgrmap e5: x(active, since 6s) 2024-04-02T13:25:22.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.185+0000 7f41b7628200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:25:22.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.277+0000 7f41b7628200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:25:22.724 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.419+0000 7f41b7628200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:25:22.724 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.515+0000 7f41b7628200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:25:22.724 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.596+0000 7f41b7628200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb62309b700 1 Processor -- start 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb62309b700 1 -- start start 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb62309b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb62309b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c108a30 con 0x7fb61c108040 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb620e37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:22.734 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb620e37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51642/0 (socket says 172.21.15.122:51642) 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.842+0000 7fb620e37700 1 -- 172.21.15.122:0/1415743630 learned_addr learned my addr 172.21.15.122:0/1415743630 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb620e37700 1 -- 172.21.15.122:0/1415743630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c1091a0 con 0x7fb61c108040 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb620e37700 1 --2- 172.21.15.122:0/1415743630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098a0 tx=0x7fb610009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bf5294849098e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb61b7fe700 1 -- 172.21.15.122:0/1415743630 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610004160 con 0x7fb61c108040 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb61b7fe700 1 -- 172.21.15.122:0/1415743630 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb610004300 con 0x7fb61c108040 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb61b7fe700 1 -- 172.21.15.122:0/1415743630 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610011b50 con 0x7fb61c108040 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb62309b700 1 -- 172.21.15.122:0/1415743630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 msgr2=0x7fb61c108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.843+0000 7fb62309b700 1 --2- 172.21.15.122:0/1415743630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb6100098a0 tx=0x7fb610009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- 172.21.15.122:0/1415743630 shutdown_connections 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 --2- 172.21.15.122:0/1415743630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c108460 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- 172.21.15.122:0/1415743630 >> 172.21.15.122:0/1415743630 conn(0x7fb61c1035e0 msgr2=0x7fb61c105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- 172.21.15.122:0/1415743630 shutdown_connections 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- 172.21.15.122:0/1415743630 wait complete. 2024-04-02T13:25:22.735 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 Processor -- start 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- start start 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb62309b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb61c10c550 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb620e37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb620e37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51646/0 (socket says 172.21.15.122:51646) 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.844+0000 7fb620e37700 1 -- 172.21.15.122:0/816166990 learned_addr learned my addr 172.21.15.122:0/816166990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb620e37700 1 -- 172.21.15.122:0/816166990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61c07e040 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb620e37700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb610011850 tx=0x7fb610011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb610003af0 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb610020070 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61c07c230 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6100195a0 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.845+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61c07c770 con 0x7fb61c108040 2024-04-02T13:25:22.736 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb6100198a0 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb619ffb700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 0x7fb60c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb610011e00 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb61bfff700 1 -- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 msgr2=0x7fb60c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb61bfff700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 0x7fb60c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.846+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61c063c20 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:21.851+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb61000e260 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.046+0000 7fb61bfff700 1 -- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 msgr2=0x7fb60c03ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.046+0000 7fb61bfff700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 0x7fb60c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.112+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb61c07ce20 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.113+0000 7fb619ffb700 1 -- 172.21.15.122:0/816166990 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb610003c90 con 0x7fb61c108040 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 msgr2=0x7fb60c03ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 0x7fb60c03ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:22.737 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 msgr2=0x7fb61c07db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb610011850 tx=0x7fb610011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 shutdown_connections 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7fb60c0395b0 0x7fb60c03ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 --2- 172.21.15.122:0/816166990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb61c108040 0x7fb61c07db00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 >> 172.21.15.122:0/816166990 conn(0x7fb61c1035e0 msgr2=0x7fb61c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 shutdown_connections 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:22.116+0000 7fb62309b700 1 -- 172.21.15.122:0/816166990 wait complete. 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-04-02T13:25:22.738 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 5... 2024-04-02T13:25:22.980 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/816166990' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:25:22.980 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.865+0000 7f41b7628200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:25:22.980 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:22.976+0000 7f41b7628200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:23.316 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:23.064+0000 7f41b7628200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:25:23.316 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:23.163+0000 7f41b7628200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:25:23.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:23.337+0000 7f41b7628200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:25:24.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:24.293+0000 7f41b7628200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:25:24.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:24.423+0000 7f41b7628200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:25:25.039 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:24.751+0000 7f41b7628200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:25:25.039 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:24.840+0000 7f41b7628200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:25:25.039 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:25.036+0000 7f41b7628200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:25.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:25.221+0000 7f41b7628200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:25:25.766 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:25.507+0000 7f41b7628200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:25.766 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:25.763+0000 7f41b7628200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:26.070 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:25.999+0000 7f41b7628200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:25:27.070 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:26.803+0000 7f41b7628200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:25:27.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:27.373+0000 7f41b7628200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:25:27.570 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:27.470+0000 7f41b7628200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:25:28.070 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:27.688+0000 7f41b7628200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:25:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: Active manager daemon x restarted 2024-04-02T13:25:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: Activating manager daemon x 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: osdmap e2: 0 total, 0 up, 0 in 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: mgrmap e6: x(active, starting, since 0.00335988s) 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: Manager daemon x is now available 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:25:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:27 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:25:29.005 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:28 smithi122 ceph-mon[32114]: Found migration_current of "None". Setting to last migration. 2024-04-02T13:25:29.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:28 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:29.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:28 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:29.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:28 smithi122 ceph-mon[32114]: mgrmap e7: x(active, since 1.00617s) 2024-04-02T13:25:29.006 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:28 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7e0559b700 1 Processor -- start 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7e0559b700 1 -- start start 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7e0559b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7e0559b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0007bde0 con 0x7f7e00106860 2024-04-02T13:25:29.243 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7dfeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7dfeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51656/0 (socket says 172.21.15.122:51656) 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.107+0000 7f7dfeffd700 1 -- 172.21.15.122:0/3136577622 learned_addr learned my addr 172.21.15.122:0/3136577622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.108+0000 7f7dfeffd700 1 -- 172.21.15.122:0/3136577622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e001071c0 con 0x7f7e00106860 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.108+0000 7f7dfeffd700 1 --2- 172.21.15.122:0/3136577622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7df00098a0 tx=0x7f7df0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a726efff3e1d540 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.108+0000 7f7dfdffb700 1 -- 172.21.15.122:0/3136577622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7df0004160 con 0x7f7e00106860 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.108+0000 7f7dfdffb700 1 -- 172.21.15.122:0/3136577622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7df0004300 con 0x7f7e00106860 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.108+0000 7f7dfdffb700 1 -- 172.21.15.122:0/3136577622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7df0011b50 con 0x7f7e00106860 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 -- 172.21.15.122:0/3136577622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 msgr2=0x7f7e00106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 --2- 172.21.15.122:0/3136577622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7df00098a0 tx=0x7f7df0009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 -- 172.21.15.122:0/3136577622 shutdown_connections 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 --2- 172.21.15.122:0/3136577622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e00106c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.244 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 -- 172.21.15.122:0/3136577622 >> 172.21.15.122:0/3136577622 conn(0x7f7e00102460 msgr2=0x7f7e00104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 -- 172.21.15.122:0/3136577622 shutdown_connections 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 -- 172.21.15.122:0/3136577622 wait complete. 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.109+0000 7f7e0559b700 1 Processor -- start 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7e0559b700 1 -- start start 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7e0559b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7e0559b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e00108250 con 0x7f7e00106860 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7dfeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7dfeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51662/0 (socket says 172.21.15.122:51662) 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7dfeffd700 1 -- 172.21.15.122:0/2154934257 learned_addr learned my addr 172.21.15.122:0/2154934257 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7dfeffd700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0019e700 con 0x7f7e00106860 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7dfeffd700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7df0011b60 tx=0x7f7df0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.110+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7df0003b20 con 0x7f7e00106860 2024-04-02T13:25:29.245 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.111+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7df0003cc0 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.111+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0019e990 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.111+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7df0024840 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.111+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0019edb0 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.111+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f7df0022020 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.112+0000 7f7de7fff700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.112+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7e00063c20 con 0x7f7de80395b0 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.112+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7df0011e00 con 0x7f7e00106860 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.112+0000 7f7dfe7fc700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.112+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.313+0000 7f7dfe7fc700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.313+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:25:29.246 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.714+0000 7f7dfe7fc700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:23.714+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:24.515+0000 7f7dfe7fc700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:24.515+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:26.116+0000 7f7dfe7fc700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1645681773 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:26.116+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:27.691+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f7df002d910 con 0x7f7e00106860 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:27.691+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 msgr2=0x7f7de803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:27.691+0000 7f7de7fff700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.693+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f7df004f700 con 0x7f7e00106860 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.693+0000 7f7de7fff700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 0x7f7de803f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.693+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7e00063c20 con 0x7f7de803ccb0 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.693+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 0x7f7de803f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.694+0000 7f7dfe7fc700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 0x7f7de803f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7df4003a10 tx=0x7f7df4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:29.247 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.695+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7e00063c20 con 0x7f7de803ccb0 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7e00063c20 con 0x7f7de803ccb0 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7de7fff700 1 -- 172.21.15.122:0/2154934257 <== mgr.14120 v2:172.21.15.122:6800/1783168866 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7e00063c20 con 0x7f7de803ccb0 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 msgr2=0x7f7de803f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 0x7f7de803f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7df4003a10 tx=0x7f7df4009290 comp rx=0 tx=0).stop 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 msgr2=0x7f7e0019e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7df0011b60 tx=0x7f7df0011c40 comp rx=0 tx=0).stop 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 shutdown_connections 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f7de803ccb0 0x7f7de803f0a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e00106860 0x7f7e0019e1c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 --2- 172.21.15.122:0/2154934257 >> [v2:172.21.15.122:6800/1645681773,v1:172.21.15.122:6801/1645681773] conn(0x7f7de80395b0 0x7f7de803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 >> 172.21.15.122:0/2154934257 conn(0x7f7e00102460 msgr2=0x7f7e00194e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 shutdown_connections 2024-04-02T13:25:29.248 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:28.702+0000 7f7e0559b700 1 -- 172.21.15.122:0/2154934257 wait complete. 2024-04-02T13:25:29.249 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 5 is available 2024-04-02T13:25:29.249 INFO:teuthology.orchestra.run.smithi122.stdout:Setting orchestrator backend to cephadm... 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:28] ENGINE Bus STARTING 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:28] ENGINE Serving on https://172.21.15.122:7150 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:28] ENGINE Serving on http://172.21.15.122:8765 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:28] ENGINE Bus STARTED 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-02T13:25:30.187 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:30.188 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:30.188 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:29 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e8921700 1 Processor -- start 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e8921700 1 -- start start 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e8921700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e8921700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69e4106c50 con 0x7f69e4106260 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:30.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51034/0 (socket says 172.21.15.122:51034) 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.656+0000 7f69e259c700 1 -- 172.21.15.122:0/4140930837 learned_addr learned my addr 172.21.15.122:0/4140930837 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.657+0000 7f69e259c700 1 -- 172.21.15.122:0/4140930837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e4106dd0 con 0x7f69e4106260 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.657+0000 7f69e259c700 1 --2- 172.21.15.122:0/4140930837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69d40098a0 tx=0x7f69d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49508d4ce998910f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.657+0000 7f69e1d9b700 1 -- 172.21.15.122:0/4140930837 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d4004160 con 0x7f69e4106260 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.657+0000 7f69e1d9b700 1 -- 172.21.15.122:0/4140930837 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69d4004300 con 0x7f69e4106260 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.657+0000 7f69e1d9b700 1 -- 172.21.15.122:0/4140930837 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d4011b50 con 0x7f69e4106260 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 -- 172.21.15.122:0/4140930837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 msgr2=0x7f69e4106680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 --2- 172.21.15.122:0/4140930837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f69d40098a0 tx=0x7f69d4009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 -- 172.21.15.122:0/4140930837 shutdown_connections 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 --2- 172.21.15.122:0/4140930837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e4106680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 -- 172.21.15.122:0/4140930837 >> 172.21.15.122:0/4140930837 conn(0x7f69e4101be0 msgr2=0x7f69e4104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 -- 172.21.15.122:0/4140930837 shutdown_connections 2024-04-02T13:25:30.471 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 -- 172.21.15.122:0/4140930837 wait complete. 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.658+0000 7f69e8921700 1 Processor -- start 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e8921700 1 -- start start 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e8921700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e8921700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69e410c510 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51044/0 (socket says 172.21.15.122:51044) 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e259c700 1 -- 172.21.15.122:0/25592965 learned_addr learned my addr 172.21.15.122:0/25592965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e259c700 1 -- 172.21.15.122:0/25592965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e419c530 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e259c700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d4003b20 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f69d4003cc0 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e419c7c0 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.659+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69d4024910 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.660+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e419f4b0 con 0x7f69e4106260 2024-04-02T13:25:30.472 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.660+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f69d4022020 con 0x7f69e4106260 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.660+0000 7f69dbfff700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 0x7f69d003b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.660+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e4063c20 con 0x7f69e4106260 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.661+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f69d4011e00 con 0x7f69e4106260 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.661+0000 7f69d9d9b700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 0x7f69d003b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.661+0000 7f69d9d9b700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 0x7f69d003b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f69cc006fd0 tx=0x7f69cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.666+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f69d4016cd0 con 0x7f69e4106260 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.877+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f69e4102e70 con 0x7f69d0039490 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.883+0000 7f69dbfff700 1 -- 172.21.15.122:0/25592965 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f69e4102e70 con 0x7f69d0039490 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 msgr2=0x7f69d003b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 0x7f69d003b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f69cc006fd0 tx=0x7f69cc006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 msgr2=0x7f69e419bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f69d4000c00 tx=0x7f69d4011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 shutdown_connections 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f69d0039490 0x7f69d003b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:30.473 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 --2- 172.21.15.122:0/25592965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e4106260 0x7f69e419bff0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:30.474 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.885+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 >> 172.21.15.122:0/25592965 conn(0x7f69e4101be0 msgr2=0x7f69e4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:30.474 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.886+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 shutdown_connections 2024-04-02T13:25:30.474 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:29.886+0000 7f69e8921700 1 -- 172.21.15.122:0/25592965 wait complete. 2024-04-02T13:25:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:30 smithi122 ceph-mon[32114]: mgrmap e8: x(active, since 2s) 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c62b67700 1 Processor -- start 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c62b67700 1 -- start start 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c62b67700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c62b67700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c108a20 con 0x7f5c5c108030 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c60903700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c60903700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51058/0 (socket says 172.21.15.122:51058) 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c60903700 1 -- 172.21.15.122:0/1536171034 learned_addr learned my addr 172.21.15.122:0/1536171034 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:31.577 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c60903700 1 -- 172.21.15.122:0/1536171034 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c1091e0 con 0x7f5c5c108030 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.905+0000 7f5c60903700 1 --2- 172.21.15.122:0/1536171034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098a0 tx=0x7f5c50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c69215ba27627cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c5b7fe700 1 -- 172.21.15.122:0/1536171034 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50004160 con 0x7f5c5c108030 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c5b7fe700 1 -- 172.21.15.122:0/1536171034 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c50004300 con 0x7f5c5c108030 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c5b7fe700 1 -- 172.21.15.122:0/1536171034 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50011b50 con 0x7f5c5c108030 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 -- 172.21.15.122:0/1536171034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 msgr2=0x7f5c5c108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 --2- 172.21.15.122:0/1536171034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c500098a0 tx=0x7f5c50009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 -- 172.21.15.122:0/1536171034 shutdown_connections 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 --2- 172.21.15.122:0/1536171034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c108450 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 -- 172.21.15.122:0/1536171034 >> 172.21.15.122:0/1536171034 conn(0x7f5c5c1035f0 msgr2=0x7f5c5c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 -- 172.21.15.122:0/1536171034 shutdown_connections 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.906+0000 7f5c62b67700 1 -- 172.21.15.122:0/1536171034 wait complete. 2024-04-02T13:25:31.578 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c62b67700 1 Processor -- start 2024-04-02T13:25:31.579 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c62b67700 1 -- start start 2024-04-02T13:25:31.579 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c62b67700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:31.579 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c62b67700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c10c590 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c60903700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c60903700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51070/0 (socket says 172.21.15.122:51070) 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c60903700 1 -- 172.21.15.122:0/3250802403 learned_addr learned my addr 172.21.15.122:0/3250802403 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.907+0000 7f5c60903700 1 -- 172.21.15.122:0/3250802403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c07e8d0 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c60903700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009fb0 tx=0x7f5c50011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c500038b0 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5c50011e50 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c50019550 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c07cac0 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.908+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c07d000 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.909+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5c50003a50 con 0x7f5c5c108030 2024-04-02T13:25:31.580 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.909+0000 7f5c59ffb700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 0x7f5c4c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.909+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c5c063c20 con 0x7f5c5c108030 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.909+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5c5004d970 con 0x7f5c5c108030 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.909+0000 7f5c5bfff700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 0x7f5c4c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.910+0000 7f5c5bfff700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 0x7f5c4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:30.914+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5c5000fda0 con 0x7f5c5c108030 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.115+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5c5c104900 con 0x7f5c4c0395b0 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.116+0000 7f5c59ffb700 1 -- 172.21.15.122:0/3250802403 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5c5c104900 con 0x7f5c4c0395b0 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 msgr2=0x7f5c4c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 0x7f5c4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006fd0 tx=0x7f5c48006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 msgr2=0x7f5c5c07e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5c50009fb0 tx=0x7f5c50011bf0 comp rx=0 tx=0).stop 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 shutdown_connections 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5c4c0395b0 0x7f5c4c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 --2- 172.21.15.122:0/3250802403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c5c108030 0x7f5c5c07e390 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 >> 172.21.15.122:0/3250802403 conn(0x7f5c5c1035f0 msgr2=0x7f5c5c1041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:31.581 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 shutdown_connections 2024-04-02T13:25:31.582 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:31.118+0000 7f5c62b67700 1 -- 172.21.15.122:0/3250802403 wait complete. 2024-04-02T13:25:31.582 INFO:teuthology.orchestra.run.smithi122.stdout:Generating ssh key... 2024-04-02T13:25:32.242 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:31 smithi122 ceph-mon[32114]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: Generating public/private rsa key pair. 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: Your identification has been saved in /tmp/tmphyb8xl66/key. 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: Your public key has been saved in /tmp/tmphyb8xl66/key.pub. 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: The key fingerprint is: 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: SHA256:WCKNifdKvNbvXIMtB6zPZb0iQWW5ZOXJ4XPvTHypnJE ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: The key's randomart image is: 2024-04-02T13:25:33.184 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: +---[RSA 3072]----+ 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | oo | 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | . + *+ o | 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | . = o .= .* . | 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | o o =. . o.o.| 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | o o.S E .=| 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | . + ..+ .. ++.| 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | + o o.B .+ o| 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | . =.*.. . | 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: | .*. .. | 2024-04-02T13:25:33.185 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: +----[SHA256]-----+ 2024-04-02T13:25:33.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.033+0000 7fa87c006700 1 Processor -- start 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa87c006700 1 -- start start 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa87c006700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa87c006700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa87407d040 con 0x7fa87407e600 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa879da2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa879da2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51084/0 (socket says 172.21.15.122:51084) 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa879da2700 1 -- 172.21.15.122:0/699101897 learned_addr learned my addr 172.21.15.122:0/699101897 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa879da2700 1 -- 172.21.15.122:0/699101897 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87407d1c0 con 0x7fa87407e600 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.034+0000 7fa879da2700 1 --2- 172.21.15.122:0/699101897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8640098a0 tx=0x7fa864009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebcaa010d8a67b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa878da0700 1 -- 172.21.15.122:0/699101897 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa864004160 con 0x7fa87407e600 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa878da0700 1 -- 172.21.15.122:0/699101897 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa864004300 con 0x7fa87407e600 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa878da0700 1 -- 172.21.15.122:0/699101897 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa864011b50 con 0x7fa87407e600 2024-04-02T13:25:33.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 -- 172.21.15.122:0/699101897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 msgr2=0x7fa87407cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 --2- 172.21.15.122:0/699101897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8640098a0 tx=0x7fa864009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 -- 172.21.15.122:0/699101897 shutdown_connections 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 --2- 172.21.15.122:0/699101897 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87407cb00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 -- 172.21.15.122:0/699101897 >> 172.21.15.122:0/699101897 conn(0x7fa8741035d0 msgr2=0x7fa874105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 -- 172.21.15.122:0/699101897 shutdown_connections 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.035+0000 7fa87c006700 1 -- 172.21.15.122:0/699101897 wait complete. 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa87c006700 1 Processor -- start 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa87c006700 1 -- start start 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa87c006700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa87c006700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa87410e5c0 con 0x7fa87407e600 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa879da2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa879da2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51090/0 (socket says 172.21.15.122:51090) 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa879da2700 1 -- 172.21.15.122:0/1280355530 learned_addr learned my addr 172.21.15.122:0/1280355530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.036+0000 7fa879da2700 1 -- 172.21.15.122:0/1280355530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa87419e6b0 con 0x7fa87407e600 2024-04-02T13:25:33.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa879da2700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa864011b60 tx=0x7fa864011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa864003b20 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa87419e940 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa864003cc0 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa864024840 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.037+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8741a1630 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa874063c20 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa864022020 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa86affd700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 0x7fa86003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa864011e00 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa8795a1700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 0x7fa86003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.038+0000 7fa8795a1700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 0x7fa86003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa870006fd0 tx=0x7fa870006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.043+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa8640172b0 con 0x7fa87407e600 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.239+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa874104840 con 0x7fa8600395b0 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.885+0000 7fa86affd700 1 -- 172.21.15.122:0/1280355530 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa874104840 con 0x7fa8600395b0 2024-04-02T13:25:33.469 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.887+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 msgr2=0x7fa86003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.887+0000 7fa87c006700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 0x7fa86003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa870006fd0 tx=0x7fa870006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.887+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 msgr2=0x7fa87419e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.887+0000 7fa87c006700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa864011b60 tx=0x7fa864011c40 comp rx=0 tx=0).stop 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 shutdown_connections 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fa8600395b0 0x7fa86003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 --2- 172.21.15.122:0/1280355530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa87407e600 0x7fa87419e170 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 >> 172.21.15.122:0/1280355530 conn(0x7fa8741035d0 msgr2=0x7fa874104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 shutdown_connections 2024-04-02T13:25:33.470 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:32.888+0000 7fa87c006700 1 -- 172.21.15.122:0/1280355530 wait complete. 2024-04-02T13:25:34.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:33 smithi122 ceph-mon[32114]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:34.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:33 smithi122 ceph-mon[32114]: Generating ssh key... 2024-04-02T13:25:34.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:33 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:34.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:33 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:34.610 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.894+0000 7fae9a794700 1 Processor -- start 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.894+0000 7fae9a794700 1 -- start start 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.894+0000 7fae9a794700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.894+0000 7fae9a794700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae9407be00 con 0x7fae94108a50 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51094/0 (socket says 172.21.15.122:51094) 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae93fff700 1 -- 172.21.15.122:0/134067295 learned_addr learned my addr 172.21.15.122:0/134067295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae93fff700 1 -- 172.21.15.122:0/134067295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9410b380 con 0x7fae94108a50 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae93fff700 1 --2- 172.21.15.122:0/134067295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae880098a0 tx=0x7fae88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f61fbac961d5d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae92ffd700 1 -- 172.21.15.122:0/134067295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae88004160 con 0x7fae94108a50 2024-04-02T13:25:34.611 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae92ffd700 1 -- 172.21.15.122:0/134067295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae88004300 con 0x7fae94108a50 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.895+0000 7fae92ffd700 1 -- 172.21.15.122:0/134067295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae88011b50 con 0x7fae94108a50 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 -- 172.21.15.122:0/134067295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 msgr2=0x7fae9410ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 --2- 172.21.15.122:0/134067295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae880098a0 tx=0x7fae88009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 -- 172.21.15.122:0/134067295 shutdown_connections 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 --2- 172.21.15.122:0/134067295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9410ae40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 -- 172.21.15.122:0/134067295 >> 172.21.15.122:0/134067295 conn(0x7fae94102460 msgr2=0x7fae94104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 -- 172.21.15.122:0/134067295 shutdown_connections 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.896+0000 7fae9a794700 1 -- 172.21.15.122:0/134067295 wait complete. 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae9a794700 1 Processor -- start 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae9a794700 1 -- start start 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae9a794700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae9a794700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae9410c3a0 con 0x7fae94108a50 2024-04-02T13:25:34.612 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae93fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51096/0 (socket says 172.21.15.122:51096) 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae93fff700 1 -- 172.21.15.122:0/1547554342 learned_addr learned my addr 172.21.15.122:0/1547554342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.897+0000 7fae93fff700 1 -- 172.21.15.122:0/1547554342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9419c4b0 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae93fff700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae88011b60 tx=0x7fae88011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae88003b20 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fae88003cc0 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fae88024840 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9419c740 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.898+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9419f430 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fae88022020 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae917fa700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 0x7fae7c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fae88011e00 con 0x7fae94108a50 2024-04-02T13:25:34.613 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94063c20 con 0x7fae94108a50 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae937fe700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 0x7fae7c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.899+0000 7fae937fe700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 0x7fae7c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae84006fd0 tx=0x7fae84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:33.904+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae88019c70 con 0x7fae94108a50 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.101+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fae94103720 con 0x7fae7c0395b0 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.102+0000 7fae917fa700 1 -- 172.21.15.122:0/1547554342 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fae94103720 con 0x7fae7c0395b0 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 msgr2=0x7fae7c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 0x7fae7c03ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fae84006fd0 tx=0x7fae84006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 msgr2=0x7fae9419bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae88011b60 tx=0x7fae88011c40 comp rx=0 tx=0).stop 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 shutdown_connections 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fae7c0395b0 0x7fae7c03ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 --2- 172.21.15.122:0/1547554342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae94108a50 0x7fae9419bf70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:34.614 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 >> 172.21.15.122:0/1547554342 conn(0x7fae94102460 msgr2=0x7fae94103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 shutdown_connections 2024-04-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:34.104+0000 7fae9a794700 1 -- 172.21.15.122:0/1547554342 wait complete. 2024-04-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi122.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi122.stdout:Adding key to root@localhost authorized_keys... 2024-04-02T13:25:34.615 INFO:teuthology.orchestra.run.smithi122.stdout:Adding host smithi122... 2024-04-02T13:25:35.212 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:34 smithi122 ceph-mon[32114]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:36.153 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:35 smithi122 ceph-mon[32114]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:36.153 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:35 smithi122 ceph-mon[32114]: Deploying cephadm binary to smithi122 2024-04-02T13:25:40.006 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Added host 'smithi122' with addr '172.21.15.122' 2024-04-02T13:25:40.006 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc7110bf700 1 Processor -- start 2024-04-02T13:25:40.006 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc7110bf700 1 -- start start 2024-04-02T13:25:40.006 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc7110bf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:40.006 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc7110bf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1089d0 con 0x7fc70c108070 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc70ad9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc70ad9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51102/0 (socket says 172.21.15.122:51102) 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc70ad9d700 1 -- 172.21.15.122:0/4008236849 learned_addr learned my addr 172.21.15.122:0/4008236849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc70ad9d700 1 -- 172.21.15.122:0/4008236849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c108b50 con 0x7fc70c108070 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.001+0000 7fc70ad9d700 1 --2- 172.21.15.122:0/4008236849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098a0 tx=0x7fc6fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553e024f5f1cb784 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:40.007 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc709d9b700 1 -- 172.21.15.122:0/4008236849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc004160 con 0x7fc70c108070 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc709d9b700 1 -- 172.21.15.122:0/4008236849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6fc004300 con 0x7fc70c108070 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc709d9b700 1 -- 172.21.15.122:0/4008236849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc011b50 con 0x7fc70c108070 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 -- 172.21.15.122:0/4008236849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 msgr2=0x7fc70c108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 --2- 172.21.15.122:0/4008236849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098a0 tx=0x7fc6fc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 -- 172.21.15.122:0/4008236849 shutdown_connections 2024-04-02T13:25:40.008 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 --2- 172.21.15.122:0/4008236849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c108490 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 -- 172.21.15.122:0/4008236849 >> 172.21.15.122:0/4008236849 conn(0x7fc70c1036d0 msgr2=0x7fc70c105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 -- 172.21.15.122:0/4008236849 shutdown_connections 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.002+0000 7fc7110bf700 1 -- 172.21.15.122:0/4008236849 wait complete. 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc7110bf700 1 Processor -- start 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc7110bf700 1 -- start start 2024-04-02T13:25:40.009 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc7110bf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc7110bf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c10c490 con 0x7fc70c108070 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc70ad9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc70ad9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51116/0 (socket says 172.21.15.122:51116) 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.003+0000 7fc70ad9d700 1 -- 172.21.15.122:0/723887449 learned_addr learned my addr 172.21.15.122:0/723887449 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc70ad9d700 1 -- 172.21.15.122:0/723887449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c19c4d0 con 0x7fc70c108070 2024-04-02T13:25:40.010 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc70ad9d700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc011b60 tx=0x7fc6fc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc003b20 con 0x7fc70c108070 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc6fc003cc0 con 0x7fc70c108070 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c19c760 con 0x7fc70c108070 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6fc024840 con 0x7fc70c108070 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.004+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c07c900 con 0x7fc70c108070 2024-04-02T13:25:40.011 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc6fc022020 con 0x7fc70c108070 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc6f3fff700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 0x7fc6f403fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc6fc011e00 con 0x7fc70c108070 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c063c20 con 0x7fc70c108070 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc70a59c700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 0x7fc6f403fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.005+0000 7fc70a59c700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 0x7fc6f403fed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:40.012 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.010+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc6fc00b9e0 con 0x7fc70c108070 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:35.209+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "addr": "172.21.15.122", "target": ["mon-mgr", ""]}) -- 0x7fc70c1048a0 con 0x7fc6f403da10 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.382+0000 7fc6f3fff700 1 -- 172.21.15.122:0/723887449 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fc70c1048a0 con 0x7fc6f403da10 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 msgr2=0x7fc6f403fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 0x7fc6f403fed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc700006fd0 tx=0x7fc700006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 msgr2=0x7fc70c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:40.013 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc011b60 tx=0x7fc6fc011c40 comp rx=0 tx=0).stop 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 shutdown_connections 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc6f403da10 0x7fc6f403fed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 --2- 172.21.15.122:0/723887449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc70c108070 0x7fc70c19bf90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 >> 172.21.15.122:0/723887449 conn(0x7fc70c1036d0 msgr2=0x7fc70c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 shutdown_connections 2024-04-02T13:25:40.014 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:39.384+0000 7fc7110bf700 1 -- 172.21.15.122:0/723887449 wait complete. 2024-04-02T13:25:40.015 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mon service... 2024-04-02T13:25:40.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:40 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:40.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:40 smithi122 ceph-mon[32114]: Added host smithi122 2024-04-02T13:25:40.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:40 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0329066700 1 Processor -- start 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0329066700 1 -- start start 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0329066700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0329066700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0324108a20 con 0x7f0324108030 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60642/0 (socket says 172.21.15.122:60642) 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0322d9d700 1 -- 172.21.15.122:0/1476219347 learned_addr learned my addr 172.21.15.122:0/1476219347 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:41.407 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0322d9d700 1 -- 172.21.15.122:0/1476219347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03241091e0 con 0x7f0324108030 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.434+0000 7f0322d9d700 1 --2- 172.21.15.122:0/1476219347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f030c0098a0 tx=0x7f030c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1336587c60d4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0321d9b700 1 -- 172.21.15.122:0/1476219347 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f030c004160 con 0x7f0324108030 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0321d9b700 1 -- 172.21.15.122:0/1476219347 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f030c004300 con 0x7f0324108030 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0321d9b700 1 -- 172.21.15.122:0/1476219347 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f030c011b50 con 0x7f0324108030 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 -- 172.21.15.122:0/1476219347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 msgr2=0x7f0324108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 --2- 172.21.15.122:0/1476219347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f030c0098a0 tx=0x7f030c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 -- 172.21.15.122:0/1476219347 shutdown_connections 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 --2- 172.21.15.122:0/1476219347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f0324108450 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 -- 172.21.15.122:0/1476219347 >> 172.21.15.122:0/1476219347 conn(0x7f03241035f0 msgr2=0x7f0324105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 -- 172.21.15.122:0/1476219347 shutdown_connections 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.435+0000 7f0329066700 1 -- 172.21.15.122:0/1476219347 wait complete. 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.436+0000 7f0329066700 1 Processor -- start 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.436+0000 7f0329066700 1 -- start start 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.436+0000 7f0329066700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:41.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.436+0000 7f0329066700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f032410c590 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0322d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60656/0 (socket says 172.21.15.122:60656) 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0322d9d700 1 -- 172.21.15.122:0/1641586622 learned_addr learned my addr 172.21.15.122:0/1641586622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0322d9d700 1 -- 172.21.15.122:0/1641586622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032419c510 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0322d9d700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f030c011670 tx=0x7f030c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f030c0039c0 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032419c7a0 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f030c011e50 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f030c01a5c0 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.437+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032419f490 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.438+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f030c01a8c0 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.438+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0324063c20 con 0x7f0324108030 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.438+0000 7f031bfff700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 0x7f031003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:41.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.438+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f030c04dad0 con 0x7f0324108030 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.438+0000 7f032259c700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 0x7f031003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.439+0000 7f032259c700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 0x7f031003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.443+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f030c019c70 con 0x7f0324108030 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.676+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f03241048b0 con 0x7f0310039560 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.683+0000 7f031bfff700 1 -- 172.21.15.122:0/1641586622 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f03241048b0 con 0x7f0310039560 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 msgr2=0x7f031003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 0x7f031003ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0314006fd0 tx=0x7f0314006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 msgr2=0x7f032419bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f030c011670 tx=0x7f030c011b60 comp rx=0 tx=0).stop 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 shutdown_connections 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f0310039560 0x7f031003ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 --2- 172.21.15.122:0/1641586622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324108030 0x7f032419bfd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:41.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.685+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 >> 172.21.15.122:0/1641586622 conn(0x7f03241035f0 msgr2=0x7f03241041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.686+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 shutdown_connections 2024-04-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:40.686+0000 7f0329066700 1 -- 172.21.15.122:0/1641586622 wait complete. 2024-04-02T13:25:41.411 INFO:teuthology.orchestra.run.smithi122.stdout:Deploying unmanaged mgr service... 2024-04-02T13:25:41.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:41 smithi122 ceph-mon[32114]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:41.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:41 smithi122 ceph-mon[32114]: Saving service mon spec with placement count:5 2024-04-02T13:25:41.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:41 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c3edc700 1 Processor -- start 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c3edc700 1 -- start start 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c3edc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c3edc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc1089b0 con 0x7f06bc108050 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c1c78700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c1c78700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60666/0 (socket says 172.21.15.122:60666) 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.824+0000 7f06c1c78700 1 -- 172.21.15.122:0/3011648471 learned_addr learned my addr 172.21.15.122:0/3011648471 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c1c78700 1 -- 172.21.15.122:0/3011648471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc108b30 con 0x7f06bc108050 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c1c78700 1 --2- 172.21.15.122:0/3011648471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06b80098a0 tx=0x7f06b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f15ad99141c2999 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:42.825 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c0c76700 1 -- 172.21.15.122:0/3011648471 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06b8004160 con 0x7f06bc108050 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c0c76700 1 -- 172.21.15.122:0/3011648471 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f06b8004300 con 0x7f06bc108050 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c0c76700 1 -- 172.21.15.122:0/3011648471 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06b8011b50 con 0x7f06bc108050 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c3edc700 1 -- 172.21.15.122:0/3011648471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 msgr2=0x7f06bc108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.825+0000 7f06c3edc700 1 --2- 172.21.15.122:0/3011648471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f06b80098a0 tx=0x7f06b8009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- 172.21.15.122:0/3011648471 shutdown_connections 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 --2- 172.21.15.122:0/3011648471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc108470 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- 172.21.15.122:0/3011648471 >> 172.21.15.122:0/3011648471 conn(0x7f06bc103690 msgr2=0x7f06bc105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- 172.21.15.122:0/3011648471 shutdown_connections 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- 172.21.15.122:0/3011648471 wait complete. 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 Processor -- start 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- start start 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c3edc700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06bc10c470 con 0x7f06bc108050 2024-04-02T13:25:42.826 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c1c78700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c1c78700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60678/0 (socket says 172.21.15.122:60678) 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.826+0000 7f06c1c78700 1 -- 172.21.15.122:0/2830856 learned_addr learned my addr 172.21.15.122:0/2830856 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06c1c78700 1 -- 172.21.15.122:0/2830856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06bc07e8d0 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06c1c78700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011670 tx=0x7f06b8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06b80039c0 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f06b8011e50 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06bc07cac0 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06b801a5c0 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.827+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06bc07d000 con 0x7f06bc108050 2024-04-02T13:25:42.827 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f06b801a8c0 con 0x7f06bc108050 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06bc063c20 con 0x7f06bc108050 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06b2ffd700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 0x7f06a803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f06b804dd60 con 0x7f06bc108050 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06c1477700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 0x7f06a803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.828+0000 7f06c1477700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 0x7f06a803ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:41.833+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f06b8019c70 con 0x7f06bc108050 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.059+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f06bc1048e0 con 0x7f06a8039560 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.066+0000 7f06b2ffd700 1 -- 172.21.15.122:0/2830856 <== mgr.14120 v2:172.21.15.122:6800/1783168866 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f06bc1048e0 con 0x7f06a8039560 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 msgr2=0x7f06a803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 0x7f06a803ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f06ac006fd0 tx=0x7f06ac006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 msgr2=0x7f06bc07e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:42.829 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011670 tx=0x7f06b8011b60 comp rx=0 tx=0).stop 2024-04-02T13:25:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 shutdown_connections 2024-04-02T13:25:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f06a8039560 0x7f06a803ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 --2- 172.21.15.122:0/2830856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06bc108050 0x7f06bc07e390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 >> 172.21.15.122:0/2830856 conn(0x7f06bc103690 msgr2=0x7f06bc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:42.830 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 shutdown_connections 2024-04-02T13:25:42.831 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:42.068+0000 7f06c3edc700 1 -- 172.21.15.122:0/2830856 wait complete. 2024-04-02T13:25:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:43 smithi122 ceph-mon[32114]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:43 smithi122 ceph-mon[32114]: Saving service mgr spec with placement count:2 2024-04-02T13:25:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:43 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:44.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f11121bb700 1 Processor -- start 2024-04-02T13:25:44.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f11121bb700 1 -- start start 2024-04-02T13:25:44.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f11121bb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:44.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f11121bb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f110c1089d0 con 0x7f110c108070 2024-04-02T13:25:44.320 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f110b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f110b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60694/0 (socket says 172.21.15.122:60694) 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.395+0000 7f110b7fe700 1 -- 172.21.15.122:0/2988757761 learned_addr learned my addr 172.21.15.122:0/2988757761 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f110b7fe700 1 -- 172.21.15.122:0/2988757761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c108b50 con 0x7f110c108070 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f110b7fe700 1 --2- 172.21.15.122:0/2988757761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098a0 tx=0x7f10fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583466f94fd8d1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f110a7fc700 1 -- 172.21.15.122:0/2988757761 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc004160 con 0x7f110c108070 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f110a7fc700 1 -- 172.21.15.122:0/2988757761 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f10fc004300 con 0x7f110c108070 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f110a7fc700 1 -- 172.21.15.122:0/2988757761 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc011b50 con 0x7f110c108070 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f11121bb700 1 -- 172.21.15.122:0/2988757761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 msgr2=0x7f110c108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f11121bb700 1 --2- 172.21.15.122:0/2988757761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098a0 tx=0x7f10fc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f11121bb700 1 -- 172.21.15.122:0/2988757761 shutdown_connections 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f11121bb700 1 --2- 172.21.15.122:0/2988757761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c108490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.396+0000 7f11121bb700 1 -- 172.21.15.122:0/2988757761 >> 172.21.15.122:0/2988757761 conn(0x7f110c1036d0 msgr2=0x7f110c105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:44.321 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 -- 172.21.15.122:0/2988757761 shutdown_connections 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 -- 172.21.15.122:0/2988757761 wait complete. 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 Processor -- start 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 -- start start 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f11121bb700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f110c10c490 con 0x7f110c108070 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.397+0000 7f110b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f110b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60702/0 (socket says 172.21.15.122:60702) 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f110b7fe700 1 -- 172.21.15.122:0/1495711949 learned_addr learned my addr 172.21.15.122:0/1495711949 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f110b7fe700 1 -- 172.21.15.122:0/1495711949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c19c4d0 con 0x7f110c108070 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f110b7fe700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10fc011670 tx=0x7f10fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc0039c0 con 0x7f110c108070 2024-04-02T13:25:44.322 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c19c760 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f10fc011e50 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10fc01a5c0 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.398+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c19f450 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.399+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f10fc01a8c0 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.399+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c063c20 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.399+0000 7f1108ff9700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 0x7f10f403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.399+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f10fc04dc30 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.399+0000 7f110affd700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 0x7f10f403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.400+0000 7f110affd700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 0x7f10f403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1100006fd0 tx=0x7f1100006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.404+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f10fc019c70 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.605+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f110c19fb40 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.607+0000 7f1108ff9700 1 -- 172.21.15.122:0/1495711949 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f10fc02c3a0 con 0x7f110c108070 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 msgr2=0x7f10f403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:44.323 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 0x7f10f403ba20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1100006fd0 tx=0x7f1100006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 msgr2=0x7f110c19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f10fc011670 tx=0x7f10fc011b60 comp rx=0 tx=0).stop 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 shutdown_connections 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f10f4039560 0x7f10f403ba20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 --2- 172.21.15.122:0/1495711949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f110c108070 0x7f110c19bf90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 >> 172.21.15.122:0/1495711949 conn(0x7f110c1036d0 msgr2=0x7f110c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 shutdown_connections 2024-04-02T13:25:44.324 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:43.610+0000 7f11121bb700 1 -- 172.21.15.122:0/1495711949 wait complete. 2024-04-02T13:25:44.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:44 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1495711949' entity='client.admin' 2024-04-02T13:25:44.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:44 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:44.696 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:44 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:45.524 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.721+0000 7f9f28ec6700 1 Processor -- start 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.721+0000 7f9f28ec6700 1 -- start start 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.721+0000 7f9f28ec6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.721+0000 7f9f28ec6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2407d0a0 con 0x7f9f2407e660 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60712/0 (socket says 172.21.15.122:60712) 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2259c700 1 -- 172.21.15.122:0/660656015 learned_addr learned my addr 172.21.15.122:0/660656015 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2259c700 1 -- 172.21.15.122:0/660656015 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2407d220 con 0x7f9f2407e660 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2259c700 1 --2- 172.21.15.122:0/660656015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f180098a0 tx=0x7f9f18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4677d06fb113b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2159a700 1 -- 172.21.15.122:0/660656015 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f18004160 con 0x7f9f2407e660 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2159a700 1 -- 172.21.15.122:0/660656015 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f18004300 con 0x7f9f2407e660 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.722+0000 7f9f2159a700 1 -- 172.21.15.122:0/660656015 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f18011b20 con 0x7f9f2407e660 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 -- 172.21.15.122:0/660656015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 msgr2=0x7f9f2407cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:45.525 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/660656015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9f180098a0 tx=0x7f9f18009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 -- 172.21.15.122:0/660656015 shutdown_connections 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/660656015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2407cb60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 -- 172.21.15.122:0/660656015 >> 172.21.15.122:0/660656015 conn(0x7f9f241035d0 msgr2=0x7f9f24105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 -- 172.21.15.122:0/660656015 shutdown_connections 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.723+0000 7f9f28ec6700 1 -- 172.21.15.122:0/660656015 wait complete. 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f28ec6700 1 Processor -- start 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f28ec6700 1 -- start start 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f28ec6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f28ec6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2410e5b0 con 0x7f9f2407e660 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f2259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60724/0 (socket says 172.21.15.122:60724) 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f2259c700 1 -- 172.21.15.122:0/3260113155 learned_addr learned my addr 172.21.15.122:0/3260113155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.724+0000 7f9f2259c700 1 -- 172.21.15.122:0/3260113155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2419e650 con 0x7f9f2407e660 2024-04-02T13:25:45.526 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f2259c700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f180117e0 tx=0x7f9f18011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f18003bf0 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f1801a910 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f18024970 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2419e8e0 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.725+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2419ed40 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.726+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f9f18022020 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.726+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24063c20 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.726+0000 7f9f137fe700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 0x7f9f0c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.726+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9f1804d9e0 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.726+0000 7f9f21d9b700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 0x7f9f0c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.727+0000 7f9f21d9b700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 0x7f9f0c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006fd0 tx=0x7f9f14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.731+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f18011e20 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.944+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f9f241a1710 con 0x7f9f2407e660 2024-04-02T13:25:45.527 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.947+0000 7f9f137fe700 1 -- 172.21.15.122:0/3260113155 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f9f180172b0 con 0x7f9f2407e660 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 msgr2=0x7f9f0c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 0x7f9f0c03ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9f14006fd0 tx=0x7f9f14006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 msgr2=0x7f9f2419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f180117e0 tx=0x7f9f18011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 shutdown_connections 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f9f0c0395b0 0x7f9f0c03ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 --2- 172.21.15.122:0/3260113155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f2407e660 0x7f9f2419e110 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 >> 172.21.15.122:0/3260113155 conn(0x7f9f241035d0 msgr2=0x7f9f241040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 shutdown_connections 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:44.950+0000 7f9f28ec6700 1 -- 172.21.15.122:0/3260113155 wait complete. 2024-04-02T13:25:45.528 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling the dashboard module... 2024-04-02T13:25:46.173 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:45 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3260113155' entity='client.admin' 2024-04-02T13:25:47.815 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:47 smithi122 ceph-mon[32114]: from='mgr.14120 172.21.15.122:0/3501964371' entity='mgr.x' 2024-04-02T13:25:47.815 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:47 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/246353613' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-02T13:25:47.815 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:47 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setuser ceph since I am not root 2024-04-02T13:25:47.815 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:47 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setgroup ceph since I am not root 2024-04-02T13:25:48.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.421+0000 7fc7335a4700 1 Processor -- start 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc7335a4700 1 -- start start 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc7335a4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc7335a4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c108dd0 con 0x7fc72c108470 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc731340700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc731340700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60728/0 (socket says 172.21.15.122:60728) 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.422+0000 7fc731340700 1 -- 172.21.15.122:0/3270323817 learned_addr learned my addr 172.21.15.122:0/3270323817 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.423+0000 7fc731340700 1 -- 172.21.15.122:0/3270323817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c109300 con 0x7fc72c108470 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.423+0000 7fc731340700 1 --2- 172.21.15.122:0/3270323817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098a0 tx=0x7fc728009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6f99a351e4584 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.423+0000 7fc723fff700 1 -- 172.21.15.122:0/3270323817 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc728004160 con 0x7fc72c108470 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.423+0000 7fc723fff700 1 -- 172.21.15.122:0/3270323817 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc728004300 con 0x7fc72c108470 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.423+0000 7fc723fff700 1 -- 172.21.15.122:0/3270323817 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc728011b20 con 0x7fc72c108470 2024-04-02T13:25:48.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 -- 172.21.15.122:0/3270323817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 msgr2=0x7fc72c108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 --2- 172.21.15.122:0/3270323817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7280098a0 tx=0x7fc728009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 -- 172.21.15.122:0/3270323817 shutdown_connections 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 --2- 172.21.15.122:0/3270323817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c108890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 -- 172.21.15.122:0/3270323817 >> 172.21.15.122:0/3270323817 conn(0x7fc72c101be0 msgr2=0x7fc72c104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 -- 172.21.15.122:0/3270323817 shutdown_connections 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.424+0000 7fc7335a4700 1 -- 172.21.15.122:0/3270323817 wait complete. 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc7335a4700 1 Processor -- start 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc7335a4700 1 -- start start 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc7335a4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc7335a4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc72c10a3c0 con 0x7fc72c108470 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc731340700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc731340700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60742/0 (socket says 172.21.15.122:60742) 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc731340700 1 -- 172.21.15.122:0/246353613 learned_addr learned my addr 172.21.15.122:0/246353613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:48.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc731340700 1 -- 172.21.15.122:0/246353613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc72c19e6b0 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc731340700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7280117e0 tx=0x7fc728011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.425+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc728003bf0 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.426+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc72c19e940 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.426+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72801a910 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.426+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc728024970 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.426+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc72c19eda0 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.427+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc72c063c20 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.427+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fc728022020 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.427+0000 7fc7227fc700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 0x7fc71803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.427+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc72804df40 con 0x7fc72c108470 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.427+0000 7fc730b3f700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 0x7fc71803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.428+0000 7fc730b3f700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 0x7fc71803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc72400ad30 tx=0x7fc7240093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.432+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc72804d8d0 con 0x7fc72c108470 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:46.705+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc72c063200 con 0x7fc72c108470 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.494+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc72802cdf0 con 0x7fc72c108470 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.496+0000 7fc7227fc700 1 -- 172.21.15.122:0/246353613 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fc72802dc90 con 0x7fc72c108470 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 msgr2=0x7fc71803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 0x7fc71803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc72400ad30 tx=0x7fc7240093f0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 msgr2=0x7fc72c19e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc7280117e0 tx=0x7fc728011ba0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 shutdown_connections 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fc7180395b0 0x7fc71803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 --2- 172.21.15.122:0/246353613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc72c108470 0x7fc72c19e170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 >> 172.21.15.122:0/246353613 conn(0x7fc72c101be0 msgr2=0x7fc72c102640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 shutdown_connections 2024-04-02T13:25:48.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:47.499+0000 7fc7335a4700 1 -- 172.21.15.122:0/246353613 wait complete. 2024-04-02T13:25:48.325 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:48.068+0000 7fc3e41f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:25:48.325 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:48.172+0000 7fc3e41f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:25:48.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/246353613' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-02T13:25:48.735 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-mon[32114]: mgrmap e9: x(active, since 19s) 2024-04-02T13:25:49.065 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:48.829+0000 7fc3e41f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:25:49.065 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:48.922+0000 7fc3e41f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:25:49.065 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.062+0000 7fc3e41f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:25:49.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.158+0000 7fc3e41f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:25:49.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.240+0000 7fc3e41f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:25:49.408 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f512d506700 1 Processor -- start 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f512d506700 1 -- start start 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f512d506700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f512d506700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f512807d080 con 0x7f512807e580 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f5126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:49.409 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f5126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35702/0 (socket says 172.21.15.122:35702) 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f5126ffd700 1 -- 172.21.15.122:0/3426479831 learned_addr learned my addr 172.21.15.122:0/3426479831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f5126ffd700 1 -- 172.21.15.122:0/3426479831 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512807d200 con 0x7f512807e580 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.527+0000 7f5126ffd700 1 --2- 172.21.15.122:0/3426479831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51180098a0 tx=0x7f5118009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdb4b654efc9158 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f5125ffb700 1 -- 172.21.15.122:0/3426479831 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5118004160 con 0x7f512807e580 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f5125ffb700 1 -- 172.21.15.122:0/3426479831 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5118004300 con 0x7f512807e580 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f5125ffb700 1 -- 172.21.15.122:0/3426479831 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5118011b20 con 0x7f512807e580 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 -- 172.21.15.122:0/3426479831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 msgr2=0x7f512807cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 --2- 172.21.15.122:0/3426479831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51180098a0 tx=0x7f5118009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 -- 172.21.15.122:0/3426479831 shutdown_connections 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 --2- 172.21.15.122:0/3426479831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512807cb40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 -- 172.21.15.122:0/3426479831 >> 172.21.15.122:0/3426479831 conn(0x7f5128103690 msgr2=0x7f5128105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 -- 172.21.15.122:0/3426479831 shutdown_connections 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.528+0000 7f512d506700 1 -- 172.21.15.122:0/3426479831 wait complete. 2024-04-02T13:25:49.410 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f512d506700 1 Processor -- start 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f512d506700 1 -- start start 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f512d506700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f512d506700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f512810e4f0 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f5126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f5126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35708/0 (socket says 172.21.15.122:35708) 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.529+0000 7f5126ffd700 1 -- 172.21.15.122:0/2976153485 learned_addr learned my addr 172.21.15.122:0/2976153485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f5126ffd700 1 -- 172.21.15.122:0/2976153485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512819e640 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f5126ffd700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51180117e0 tx=0x7f5118011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5118003bf0 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f511801a910 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512819e8d0 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5118024970 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.530+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512819ed30 con 0x7f512807e580 2024-04-02T13:25:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5118022020 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5128063c20 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f510ffff700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 0x7f511003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f511804da60 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f51267fc700 1 -- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 msgr2=0x7f511003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.531+0000 7f51267fc700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 0x7f511003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.536+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5118011e20 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.732+0000 7f51267fc700 1 -- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 msgr2=0x7f511003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.732+0000 7f51267fc700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 0x7f511003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.797+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f51281a16e0 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.797+0000 7f510ffff700 1 -- 172.21.15.122:0/2976153485 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f511800ba30 con 0x7f512807e580 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 msgr2=0x7f511003bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 0x7f511003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 msgr2=0x7f512819e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:49.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f51180117e0 tx=0x7f5118011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 shutdown_connections 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7f5110039610 0x7f511003bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 --2- 172.21.15.122:0/2976153485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f512807e580 0x7f512819e100 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 >> 172.21.15.122:0/2976153485 conn(0x7f5128103690 msgr2=0x7f51281041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 shutdown_connections 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:48.799+0000 7f512d506700 1 -- 172.21.15.122:0/2976153485 wait complete. 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for the mgr to restart... 2024-04-02T13:25:49.413 INFO:teuthology.orchestra.run.smithi122.stdout:Waiting for mgr epoch 9... 2024-04-02T13:25:49.716 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2976153485' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-02T13:25:49.717 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.513+0000 7fc3e41f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:25:49.717 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.626+0000 7fc3e41f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:49.717 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.714+0000 7fc3e41f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:25:49.994 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.813+0000 7fc3e41f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:25:50.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:49.992+0000 7fc3e41f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:25:51.252 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:50 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:50.950+0000 7fc3e41f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:25:51.252 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:51.080+0000 7fc3e41f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:25:51.504 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:51.413+0000 7fc3e41f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:25:51.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:51.502+0000 7fc3e41f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:25:51.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:51.700+0000 7fc3e41f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:52.178 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:51.885+0000 7fc3e41f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:25:52.436 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:52 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:52.175+0000 7fc3e41f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:25:52.436 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:52 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:52.433+0000 7fc3e41f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:25:52.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:52 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:52.674+0000 7fc3e41f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:25:53.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:53 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:53.485+0000 7fc3e41f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:25:54.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:54.051+0000 7fc3e41f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:25:54.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:54.150+0000 7fc3e41f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:25:54.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:25:54.370+0000 7fc3e41f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: Active manager daemon x restarted 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: Activating manager daemon x 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: osdmap e3: 0 total, 0 up, 0 in 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: mgrmap e10: x(active, starting, since 0.00632911s) 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: Manager daemon x is now available 2024-04-02T13:25:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:25:54.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:25:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:25:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:55 smithi122 ceph-mon[32114]: mgrmap e11: x(active, since 1.00612s) 2024-04-02T13:25:56.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout { 2024-04-02T13:25:56.226 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout } 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.841+0000 7fe5def6c700 1 Processor -- start 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.841+0000 7fe5def6c700 1 -- start start 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5def6c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5def6c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d807bda0 con 0x7fe5d8108ab0 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5dcd08700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5dcd08700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35712/0 (socket says 172.21.15.122:35712) 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5dcd08700 1 -- 172.21.15.122:0/3459378979 learned_addr learned my addr 172.21.15.122:0/3459378979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5dcd08700 1 -- 172.21.15.122:0/3459378979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d810b3e0 con 0x7fe5d8108ab0 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5dcd08700 1 --2- 172.21.15.122:0/3459378979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9feebe56666bb819 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.842+0000 7fe5d77fe700 1 -- 172.21.15.122:0/3459378979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc004160 con 0x7fe5d8108ab0 2024-04-02T13:25:56.227 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5d77fe700 1 -- 172.21.15.122:0/3459378979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc004300 con 0x7fe5d8108ab0 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5d77fe700 1 -- 172.21.15.122:0/3459378979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc011b20 con 0x7fe5d8108ab0 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5def6c700 1 -- 172.21.15.122:0/3459378979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 msgr2=0x7fe5d810aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5def6c700 1 --2- 172.21.15.122:0/3459378979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0098a0 tx=0x7fe5cc009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5def6c700 1 -- 172.21.15.122:0/3459378979 shutdown_connections 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5def6c700 1 --2- 172.21.15.122:0/3459378979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d810aea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.843+0000 7fe5def6c700 1 -- 172.21.15.122:0/3459378979 >> 172.21.15.122:0/3459378979 conn(0x7fe5d8102460 msgr2=0x7fe5d8104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 -- 172.21.15.122:0/3459378979 shutdown_connections 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 -- 172.21.15.122:0/3459378979 wait complete. 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 Processor -- start 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 -- start start 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5def6c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d810c3e0 con 0x7fe5d8108ab0 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5dcd08700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:56.228 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5dcd08700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35726/0 (socket says 172.21.15.122:35726) 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.844+0000 7fe5dcd08700 1 -- 172.21.15.122:0/2050323376 learned_addr learned my addr 172.21.15.122:0/2050323376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5dcd08700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d819c470 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5dcd08700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0117e0 tx=0x7fe5cc011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc003bf0 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5cc01a910 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d819c700 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5cc024970 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.845+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d819cb60 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.846+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe5cc022020 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.847+0000 7fe5d5ffb700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.847+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5c803c180 con 0x7fe5c80395b0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.847+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe5cc04da40 con 0x7fe5d8108ab0 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.847+0000 7fe5d7fff700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:56.229 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:49.847+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:50.048+0000 7fe5d7fff700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:50.048+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:50.449+0000 7fe5d7fff700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:50.449+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:51.250+0000 7fe5d7fff700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:51.250+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:52.851+0000 7fe5d7fff700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/1783168866 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:52.851+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:54.377+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7fe5cc01a430 con 0x7fe5d8108ab0 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:54.377+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 msgr2=0x7fe5c803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:54.377+0000 7fe5d5ffb700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.375+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7fe5cc04f920 con 0x7fe5d8108ab0 2024-04-02T13:25:56.230 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.376+0000 7fe5d5ffb700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 0x7fe5c803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.376+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe5c803c180 con 0x7fe5c803cfe0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.376+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 0x7fe5c803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.377+0000 7fe5d7fff700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 0x7fe5c803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4003a10 tx=0x7fe5c40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.377+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fe5c803c180 con 0x7fe5c803cfe0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.384+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fe5d8063c20 con 0x7fe5c803cfe0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.384+0000 7fe5d5ffb700 1 -- 172.21.15.122:0/2050323376 <== mgr.14152 v2:172.21.15.122:6800/3450525377 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fe5d8063c20 con 0x7fe5c803cfe0 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 msgr2=0x7fe5c803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 0x7fe5c803f3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4003a10 tx=0x7fe5c40092b0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 msgr2=0x7fe5d819bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0117e0 tx=0x7fe5cc011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 shutdown_connections 2024-04-02T13:25:56.231 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5c803cfe0 0x7fe5c803f3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d8108ab0 0x7fe5d819bf30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 --2- 172.21.15.122:0/2050323376 >> [v2:172.21.15.122:6800/1783168866,v1:172.21.15.122:6801/1783168866] conn(0x7fe5c80395b0 0x7fe5c803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 >> 172.21.15.122:0/2050323376 conn(0x7fe5d8102460 msgr2=0x7fe5d8102f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 shutdown_connections 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:55.385+0000 7fe5def6c700 1 -- 172.21.15.122:0/2050323376 wait complete. 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:mgr epoch 9 is available 2024-04-02T13:25:56.232 INFO:teuthology.orchestra.run.smithi122.stdout:Generating a dashboard self-signed certificate... 2024-04-02T13:25:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:56 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:55] ENGINE Bus STARTING 2024-04-02T13:25:56.620 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:25:57.464 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f812e81e700 1 Processor -- start 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f812e81e700 1 -- start start 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f812e81e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f812e81e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8128107880 con 0x7f8128106f00 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f8127fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f8127fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35796/0 (socket says 172.21.15.122:35796) 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.637+0000 7f8127fff700 1 -- 172.21.15.122:0/3098075744 learned_addr learned my addr 172.21.15.122:0/3098075744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.638+0000 7f8127fff700 1 -- 172.21.15.122:0/3098075744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81281080e0 con 0x7f8128106f00 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.638+0000 7f8127fff700 1 --2- 172.21.15.122:0/3098075744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f811c0098a0 tx=0x7f811c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0de28d28f1eeb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.638+0000 7f8126ffd700 1 -- 172.21.15.122:0/3098075744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f811c004160 con 0x7f8128106f00 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.638+0000 7f8126ffd700 1 -- 172.21.15.122:0/3098075744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811c004300 con 0x7f8128106f00 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.638+0000 7f8126ffd700 1 -- 172.21.15.122:0/3098075744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f811c011b20 con 0x7f8128106f00 2024-04-02T13:25:57.465 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- 172.21.15.122:0/3098075744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 msgr2=0x7f8128107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 --2- 172.21.15.122:0/3098075744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f811c0098a0 tx=0x7f811c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- 172.21.15.122:0/3098075744 shutdown_connections 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 --2- 172.21.15.122:0/3098075744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f8128107340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- 172.21.15.122:0/3098075744 >> 172.21.15.122:0/3098075744 conn(0x7f81281025b0 msgr2=0x7f81281049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- 172.21.15.122:0/3098075744 shutdown_connections 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- 172.21.15.122:0/3098075744 wait complete. 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 Processor -- start 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.639+0000 7f812e81e700 1 -- start start 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f812e81e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f812e81e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f812810b3a0 con 0x7f8128106f00 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f8127fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f8127fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35800/0 (socket says 172.21.15.122:35800) 2024-04-02T13:25:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f8127fff700 1 -- 172.21.15.122:0/1374501479 learned_addr learned my addr 172.21.15.122:0/1374501479 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f8127fff700 1 -- 172.21.15.122:0/1374501479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812819ec30 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f8127fff700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f811c0117e0 tx=0x7f811c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f811c003bf0 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f811c01a910 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812819ee00 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.640+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f811c024970 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.641+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812819f1a0 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.641+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f811c022020 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.641+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8128063c20 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.642+0000 7f81257fa700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 0x7f811003ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.642+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f811c04da20 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.642+0000 7f81277fe700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 0x7f811003ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.642+0000 7f81277fe700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 0x7f811003ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8118006fd0 tx=0x7f8118006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.647+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f811c02cdf0 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.784+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f811c01a430 con 0x7f8128106f00 2024-04-02T13:25:57.467 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.859+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f8128103780 con 0x7f8110039550 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.935+0000 7f81257fa700 1 -- 172.21.15.122:0/1374501479 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f8128103780 con 0x7f8110039550 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.937+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 msgr2=0x7f811003ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.937+0000 7f812e81e700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 0x7f811003ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8118006fd0 tx=0x7f8118006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.937+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 msgr2=0x7f812807c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.937+0000 7f812e81e700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f811c0117e0 tx=0x7f811c011ba0 comp rx=0 tx=0).stop 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 shutdown_connections 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8110039550 0x7f811003ba10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 --2- 172.21.15.122:0/1374501479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8128106f00 0x7f812807c270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 >> 172.21.15.122:0/1374501479 conn(0x7f81281025b0 msgr2=0x7f8128103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 shutdown_connections 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:56.938+0000 7f812e81e700 1 -- 172.21.15.122:0/1374501479 wait complete. 2024-04-02T13:25:57.468 INFO:teuthology.orchestra.run.smithi122.stdout:Creating initial admin user... 2024-04-02T13:25:57.557 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:55] ENGINE Serving on https://172.21.15.122:7150 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:55] ENGINE Serving on http://172.21.15.122:8765 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: [02/Apr/2024:13:25:55] ENGINE Bus STARTED 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: mgrmap e12: x(active, since 2s) 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:25:57.558 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:25:58.784 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:58 smithi122 ceph-mon[32114]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:25:58.784 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:25:59.411 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4eYVPdBjjgf0wjC5weCJnuxsXLD/2cOTn2VxinuofhY3y2fICJxly", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1712064358, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.909+0000 7ff14bd07700 1 Processor -- start 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff14bd07700 1 -- start start 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff14bd07700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff14bd07700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff144108a20 con 0x7ff144108030 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff149aa3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff149aa3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35806/0 (socket says 172.21.15.122:35806) 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff149aa3700 1 -- 172.21.15.122:0/983953536 learned_addr learned my addr 172.21.15.122:0/983953536 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.910+0000 7ff149aa3700 1 -- 172.21.15.122:0/983953536 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1441091e0 con 0x7ff144108030 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff149aa3700 1 --2- 172.21.15.122:0/983953536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5547e8d1f2ee20c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff148aa1700 1 -- 172.21.15.122:0/983953536 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140004160 con 0x7ff144108030 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff148aa1700 1 -- 172.21.15.122:0/983953536 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140004300 con 0x7ff144108030 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff148aa1700 1 -- 172.21.15.122:0/983953536 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140011b20 con 0x7ff144108030 2024-04-02T13:25:59.412 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 -- 172.21.15.122:0/983953536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 msgr2=0x7ff144108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 --2- 172.21.15.122:0/983953536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff1400098a0 tx=0x7ff140009bb0 comp rx=0 tx=0).stop 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 -- 172.21.15.122:0/983953536 shutdown_connections 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 --2- 172.21.15.122:0/983953536 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff144108450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 -- 172.21.15.122:0/983953536 >> 172.21.15.122:0/983953536 conn(0x7ff1441035f0 msgr2=0x7ff144105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 -- 172.21.15.122:0/983953536 shutdown_connections 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.911+0000 7ff14bd07700 1 -- 172.21.15.122:0/983953536 wait complete. 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff14bd07700 1 Processor -- start 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff14bd07700 1 -- start start 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff14bd07700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff14bd07700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff14410c590 con 0x7ff144108030 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff149aa3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff149aa3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35818/0 (socket says 172.21.15.122:35818) 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff149aa3700 1 -- 172.21.15.122:0/1704617390 learned_addr learned my addr 172.21.15.122:0/1704617390 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:25:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.912+0000 7ff149aa3700 1 -- 172.21.15.122:0/1704617390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14407e8d0 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff149aa3700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1400117e0 tx=0x7ff140011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140003bf0 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff14001a910 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14407cac0 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1400249a0 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.913+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14407cf80 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.914+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff140022020 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.914+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff144063c20 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.914+0000 7ff13affd700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 0x7ff13003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.914+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff14004dc30 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.914+0000 7ff1492a2700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 0x7ff13003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.915+0000 7ff1492a2700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 0x7ff13003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:57.919+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff140011e20 con 0x7ff144108030 2024-04-02T13:25:59.414 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.146+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff14407d200 con 0x7ff130039610 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.371+0000 7ff13affd700 1 -- 172.21.15.122:0/1704617390 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff14407d200 con 0x7ff130039610 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 msgr2=0x7ff13003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 0x7ff13003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff134006fd0 tx=0x7ff134006e40 comp rx=0 tx=0).stop 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 msgr2=0x7ff14407e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff1400117e0 tx=0x7ff140011c10 comp rx=0 tx=0).stop 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 shutdown_connections 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff130039610 0x7ff13003bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 --2- 172.21.15.122:0/1704617390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff144108030 0x7ff14407e390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 >> 172.21.15.122:0/1704617390 conn(0x7ff1441035f0 msgr2=0x7ff1441040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 shutdown_connections 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:58.373+0000 7ff14bd07700 1 -- 172.21.15.122:0/1704617390 wait complete. 2024-04-02T13:25:59.415 INFO:teuthology.orchestra.run.smithi122.stdout:Fetching dashboard port number... 2024-04-02T13:25:59.705 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:25:59 smithi122 ceph-mon[32114]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stdout 8443 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.878+0000 7fd6f72f0700 1 Processor -- start 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.878+0000 7fd6f72f0700 1 -- start start 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.878+0000 7fd6f72f0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.878+0000 7fd6f72f0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f007c6d0 con 0x7fd6f0106840 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6f508c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:00.643 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6f508c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52658/0 (socket says 172.21.15.122:52658) 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6f508c700 1 -- 172.21.15.122:0/2978544433 learned_addr learned my addr 172.21.15.122:0/2978544433 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6f508c700 1 -- 172.21.15.122:0/2978544433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f01071a0 con 0x7fd6f0106840 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6f508c700 1 --2- 172.21.15.122:0/2978544433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009e80 tx=0x7fd6ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a832e4fba9e59ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6e7fff700 1 -- 172.21.15.122:0/2978544433 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ec017070 con 0x7fd6f0106840 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6e7fff700 1 -- 172.21.15.122:0/2978544433 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec00b8c0 con 0x7fd6f0106840 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.879+0000 7fd6e7fff700 1 -- 172.21.15.122:0/2978544433 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ec0045c0 con 0x7fd6f0106840 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 -- 172.21.15.122:0/2978544433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 msgr2=0x7fd6f0106c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/2978544433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009e80 tx=0x7fd6ec00b330 comp rx=0 tx=0).stop 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 -- 172.21.15.122:0/2978544433 shutdown_connections 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/2978544433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f0106c60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 -- 172.21.15.122:0/2978544433 >> 172.21.15.122:0/2978544433 conn(0x7fd6f0102500 msgr2=0x7fd6f0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 -- 172.21.15.122:0/2978544433 shutdown_connections 2024-04-02T13:26:00.644 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 -- 172.21.15.122:0/2978544433 wait complete. 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.880+0000 7fd6f72f0700 1 Processor -- start 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f72f0700 1 -- start start 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f72f0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f72f0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0108270 con 0x7fd6f0106840 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f508c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f508c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52664/0 (socket says 172.21.15.122:52664) 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f508c700 1 -- 172.21.15.122:0/3034682891 learned_addr learned my addr 172.21.15.122:0/3034682891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f508c700 1 -- 172.21.15.122:0/3034682891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f019e710 con 0x7fd6f0106840 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f508c700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec004350 tx=0x7fd6ec00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ec020020 con 0x7fd6f0106840 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec003d90 con 0x7fd6f0106840 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f019e9a0 con 0x7fd6f0106840 2024-04-02T13:26:00.645 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6ec017040 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.881+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01a1690 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.882+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd6ec023020 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.882+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0063c20 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.882+0000 7fd6e67fc700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 0x7fd6dc03bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.882+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6ec00f050 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.883+0000 7fd6f488b700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 0x7fd6dc03bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.883+0000 7fd6f488b700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 0x7fd6dc03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:25:59.887+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd6ec02a020 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.087+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd6f002e070 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.087+0000 7fd6e67fc700 1 -- 172.21.15.122:0/3034682891 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd6ec01b730 con 0x7fd6f0106840 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 msgr2=0x7fd6dc03bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:00.646 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 0x7fd6dc03bad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 msgr2=0x7fd6f019e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec004350 tx=0x7fd6ec00bd50 comp rx=0 tx=0).stop 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 shutdown_connections 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd6dc039610 0x7fd6dc03bad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 --2- 172.21.15.122:0/3034682891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6f0106840 0x7fd6f019e1d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.089+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 >> 172.21.15.122:0/3034682891 conn(0x7fd6f0102500 msgr2=0x7fd6f0194e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.090+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 shutdown_connections 2024-04-02T13:26:00.647 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:00.090+0000 7fd6f72f0700 1 -- 172.21.15.122:0/3034682891 wait complete. 2024-04-02T13:26:00.648 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-02T13:26:00.648 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout disabled 2024-04-02T13:26:00.655 INFO:teuthology.orchestra.run.smithi122.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-02T13:26:00.655 INFO:teuthology.orchestra.run.smithi122.stdout:systemctl: stdout inactive 2024-04-02T13:26:00.655 INFO:teuthology.orchestra.run.smithi122.stdout:firewalld.service is not enabled 2024-04-02T13:26:00.655 INFO:teuthology.orchestra.run.smithi122.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout:Ceph Dashboard is now available at: 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout: URL: https://smithi122.front.sepia.ceph.com:8443/ 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout: User: admin 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout: Password: iu65tniait 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout:Saving cluster configuration to /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config directory 2024-04-02T13:26:00.657 INFO:teuthology.orchestra.run.smithi122.stdout:Enabling autotune for osd_memory_target 2024-04-02T13:26:00.777 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3034682891' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-02T13:26:01.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:01 smithi122 ceph-mon[32114]: mgrmap e13: x(active, since 6s) 2024-04-02T13:26:01.806 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.085+0000 7f4d47b46700 1 Processor -- start 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.085+0000 7f4d47b46700 1 -- start start 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d47b46700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d47b46700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d401089d0 con 0x7f4d40108070 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d458e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d458e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52672/0 (socket says 172.21.15.122:52672) 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d458e2700 1 -- 172.21.15.122:0/4059903774 learned_addr learned my addr 172.21.15.122:0/4059903774 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d458e2700 1 -- 172.21.15.122:0/4059903774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d40108b50 con 0x7f4d40108070 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d458e2700 1 --2- 172.21.15.122:0/4059903774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9b8696b0d7aed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d448e0700 1 -- 172.21.15.122:0/4059903774 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c004160 con 0x7f4d40108070 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d448e0700 1 -- 172.21.15.122:0/4059903774 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c004300 con 0x7f4d40108070 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.086+0000 7f4d448e0700 1 -- 172.21.15.122:0/4059903774 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c011b20 con 0x7f4d40108070 2024-04-02T13:26:01.807 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 -- 172.21.15.122:0/4059903774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 msgr2=0x7f4d40108490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 --2- 172.21.15.122:0/4059903774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0098a0 tx=0x7f4d3c009bb0 comp rx=0 tx=0).stop 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 -- 172.21.15.122:0/4059903774 shutdown_connections 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 --2- 172.21.15.122:0/4059903774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d40108490 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 -- 172.21.15.122:0/4059903774 >> 172.21.15.122:0/4059903774 conn(0x7f4d401036d0 msgr2=0x7f4d40105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 -- 172.21.15.122:0/4059903774 shutdown_connections 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.087+0000 7f4d47b46700 1 -- 172.21.15.122:0/4059903774 wait complete. 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 Processor -- start 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 -- start start 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4010c490 con 0x7f4d40108070 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d458e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d458e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52674/0 (socket says 172.21.15.122:52674) 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d458e2700 1 -- 172.21.15.122:0/2457615103 learned_addr learned my addr 172.21.15.122:0/2457615103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:01.808 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d458e2700 1 -- 172.21.15.122:0/2457615103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4019c450 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d458e2700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0117e0 tx=0x7f4d3c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c003bf0 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4019c6e0 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c01a910 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3c024970 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.088+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4019cb40 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.089+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f4d3c022020 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.089+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d40063c20 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.089+0000 7f4d36ffd700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 0x7f4d2c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.090+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d3c04dd10 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.090+0000 7f4d450e1700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 0x7f4d2c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.090+0000 7f4d450e1700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 0x7f4d2c03bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.095+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4d3c053020 con 0x7f4d40108070 2024-04-02T13:26:01.809 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.293+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f4d4019f3d0 con 0x7f4d40108070 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.293+0000 7f4d36ffd700 1 -- 172.21.15.122:0/2457615103 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f4d3c011e20 con 0x7f4d40108070 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 msgr2=0x7f4d2c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 0x7f4d2c03bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006fd0 tx=0x7f4d30006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 msgr2=0x7f4d4019bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0117e0 tx=0x7f4d3c011c10 comp rx=0 tx=0).stop 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 shutdown_connections 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d2c039650 0x7f4d2c03bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 --2- 172.21.15.122:0/2457615103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d40108070 0x7f4d4019bf10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.295+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 >> 172.21.15.122:0/2457615103 conn(0x7f4d401036d0 msgr2=0x7f4d40104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.296+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 shutdown_connections 2024-04-02T13:26:01.810 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:01.296+0000 7f4d47b46700 1 -- 172.21.15.122:0/2457615103 wait complete. 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f277ae47700 1 Processor -- start 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f277ae47700 1 -- start start 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f277ae47700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f277ae47700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2774108a20 con 0x7f2774108030 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f2778be3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:03.091 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f2778be3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52684/0 (socket says 172.21.15.122:52684) 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f2778be3700 1 -- 172.21.15.122:0/4156963959 learned_addr learned my addr 172.21.15.122:0/4156963959 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f2778be3700 1 -- 172.21.15.122:0/4156963959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27741091e0 con 0x7f2774108030 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f2778be3700 1 --2- 172.21.15.122:0/4156963959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f27680098a0 tx=0x7f2768009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6a849782a74de6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.202+0000 7f27737fe700 1 -- 172.21.15.122:0/4156963959 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2768004160 con 0x7f2774108030 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f27737fe700 1 -- 172.21.15.122:0/4156963959 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2768004300 con 0x7f2774108030 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f27737fe700 1 -- 172.21.15.122:0/4156963959 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2768011b20 con 0x7f2774108030 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 -- 172.21.15.122:0/4156963959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 msgr2=0x7f2774108450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 --2- 172.21.15.122:0/4156963959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f27680098a0 tx=0x7f2768009bb0 comp rx=0 tx=0).stop 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 -- 172.21.15.122:0/4156963959 shutdown_connections 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 --2- 172.21.15.122:0/4156963959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f2774108450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 -- 172.21.15.122:0/4156963959 >> 172.21.15.122:0/4156963959 conn(0x7f27741035f0 msgr2=0x7f2774105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 -- 172.21.15.122:0/4156963959 shutdown_connections 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.203+0000 7f277ae47700 1 -- 172.21.15.122:0/4156963959 wait complete. 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f277ae47700 1 Processor -- start 2024-04-02T13:26:03.092 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f277ae47700 1 -- start start 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f277ae47700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f277ae47700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f277407c900 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f2778be3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f2778be3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52698/0 (socket says 172.21.15.122:52698) 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f2778be3700 1 -- 172.21.15.122:0/2752381160 learned_addr learned my addr 172.21.15.122:0/2752381160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f2778be3700 1 -- 172.21.15.122:0/2752381160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277419e7a0 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.204+0000 7f2778be3700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27680116a0 tx=0x7f2768011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.205+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2768003a40 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.205+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277419ea30 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.205+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2768003be0 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.205+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276801a720 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.205+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277419ee90 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f276801aa20 con 0x7f2774108030 2024-04-02T13:26:03.093 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2774063c20 con 0x7f2774108030 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f2771ffb700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 0x7f276403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f276804dfd0 con 0x7f2774108030 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f2773fff700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 0x7f276403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.206+0000 7f2773fff700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 0x7f276403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2760006fd0 tx=0x7f2760006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.211+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2768011e20 con 0x7f2774108030 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.516+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2774063200 con 0x7f2774108030 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.518+0000 7f2771ffb700 1 -- 172.21.15.122:0/2752381160 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f276800bc40 con 0x7f2774108030 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.520+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 msgr2=0x7f276403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.520+0000 7f277ae47700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 0x7f276403bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2760006fd0 tx=0x7f2760006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.520+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 msgr2=0x7f277419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.520+0000 7f277ae47700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f27680116a0 tx=0x7f2768011ba0 comp rx=0 tx=0).stop 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 shutdown_connections 2024-04-02T13:26:03.094 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2764039650 0x7f276403bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 --2- 172.21.15.122:0/2752381160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2774108030 0x7f277419e260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 >> 172.21.15.122:0/2752381160 conn(0x7f27741035f0 msgr2=0x7f2774195530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 shutdown_connections 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr 2024-04-02T13:26:02.521+0000 7f277ae47700 1 -- 172.21.15.122:0/2752381160 wait complete. 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:Or, if you are only running a single cluster on this host: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: ceph telemetry on 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout:For more information see: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:03.096 INFO:teuthology.orchestra.run.smithi122.stdout:Bootstrap complete. 2024-04-02T13:26:03.129 INFO:tasks.cephadm:Fetching config... 2024-04-02T13:26:03.129 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:03.130 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-02T13:26:03.145 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-02T13:26:03.145 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:03.145 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-02T13:26:03.200 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-02T13:26:03.200 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:03.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/keyring of=/dev/stdout 2024-04-02T13:26:03.267 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-02T13:26:03.267 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:03.267 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-02T13:26:03.321 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-02T13:26:03.325 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:26:03.402 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:03.416 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:26:03.461 INFO:teuthology.orchestra.run.smithi187.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:03.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-02T13:26:03.522 INFO:teuthology.orchestra.run.smithi188.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ3bkuapj2BzTUU2+MjyYkxwkYyGxH3H2lC83DIlB/HWTkGIvuj/ujBoDhgkwMj5XZqXrkWVRI6jZEyirGiWYhTXg835XCU7A6Q3b8X6c8AbKcayUx+VvgRPANToDAsbTAun7MJkscHEHpVdLKQ3nUMxN/S7it3N1noGCnOSHvqaYNy6ylWm7L67VmUGomAG9enjFXLhmiuV1MFGUxANJePFgwRKBGFa1WXe+L1rOtSPE8AVFsR0QfQ5cjky1wOe+5hxEe4jpVG7mRKMIrwKg1ZDCShT4zWtTJJQzK7Ff5KpBoLNtwqP/m0lSY8NP+bM2FciM6qCvrtVRXs6v5alm1C3DBF+jE0xx3sqQx/ILjB9985cUuVebwo+VJQHM+6FsWLa9sv3IqUv8VTIbE5bp/1tNltJT3olLKOolEq0LJuEoWOf4Wrzp8J+ttyuNuCBWB0X57s5eZ4GiUSS3F1OyHSh4jcPMiDt+J7rQyNZJPz7cD6xtBw0koT9ZptB6F1lM= ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:03.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-02T13:26:03.774 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2752381160' entity='client.admin' 2024-04-02T13:26:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.943+0000 7ff9eca51700 1 -- 172.21.15.122:0/2249976724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 msgr2=0x7ff9e8104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:05.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.943+0000 7ff9eca51700 1 --2- 172.21.15.122:0/2249976724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8104f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc009910 tx=0x7ff9dc009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- 172.21.15.122:0/2249976724 shutdown_connections 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 --2- 172.21.15.122:0/2249976724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8104f10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- 172.21.15.122:0/2249976724 >> 172.21.15.122:0/2249976724 conn(0x7ff9e80fc4b0 msgr2=0x7ff9e80fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- 172.21.15.122:0/2249976724 shutdown_connections 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- 172.21.15.122:0/2249976724 wait complete. 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 Processor -- start 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- start start 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.944+0000 7ff9eca51700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81080d0 con 0x7ff9e8102b20 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9e659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9e659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52720/0 (socket says 172.21.15.122:52720) 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9e659c700 1 -- 172.21.15.122:0/1913130560 learned_addr learned my addr 172.21.15.122:0/1913130560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:05.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9e659c700 1 -- 172.21.15.122:0/1913130560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e8197cb0 con 0x7ff9e8102b20 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9e659c700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0052c0 tx=0x7ff9dc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc026070 con 0x7ff9e8102b20 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9dc00bc30 con 0x7ff9e8102b20 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e8197f40 con 0x7ff9e8102b20 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9dc003b10 con 0x7ff9e8102b20 2024-04-02T13:26:05.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.945+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e819ac30 con 0x7ff9e8102b20 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.946+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7ff9dc029070 con 0x7ff9e8102b20 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.946+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e8108850 con 0x7ff9e8102b20 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.946+0000 7ff9d77fe700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 0x7ff9d003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.946+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff9dc01e070 con 0x7ff9e8102b20 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.946+0000 7ff9e5d9b700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 0x7ff9d003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:05.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.947+0000 7ff9e5d9b700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 0x7ff9d003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8006fd0 tx=0x7ff9d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:05.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:05.951+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff9dc013070 con 0x7ff9e8102b20 2024-04-02T13:26:06.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.148+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7ff9e802de40 con 0x7ff9e8102b20 2024-04-02T13:26:06.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.150+0000 7ff9d77fe700 1 -- 172.21.15.122:0/1913130560 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7ff9dc030020 con 0x7ff9e8102b20 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 msgr2=0x7ff9d003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 0x7ff9d003bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8006fd0 tx=0x7ff9d8006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 msgr2=0x7ff9e8197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0052c0 tx=0x7ff9dc005030 comp rx=0 tx=0).stop 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 shutdown_connections 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff9d0039650 0x7ff9d003bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:06.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 --2- 172.21.15.122:0/1913130560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9e8102b20 0x7ff9e8197770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:06.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 >> 172.21.15.122:0/1913130560 conn(0x7ff9e80fc4b0 msgr2=0x7ff9e80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:06.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 shutdown_connections 2024-04-02T13:26:06.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:06.153+0000 7ff9eca51700 1 -- 172.21.15.122:0/1913130560 wait complete. 2024-04-02T13:26:06.750 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-02T13:26:06.750 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-02T13:26:06.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:06.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1913130560' entity='client.admin' 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:06.776 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:06.983 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- 172.21.15.122:0/1038357772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 msgr2=0x7f2058100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 --2- 172.21.15.122:0/1038357772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f2058100730 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2054009910 tx=0x7f2054009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- 172.21.15.122:0/1038357772 shutdown_connections 2024-04-02T13:26:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 --2- 172.21.15.122:0/1038357772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f2058100730 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:08.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- 172.21.15.122:0/1038357772 >> 172.21.15.122:0/1038357772 conn(0x7f20580fbc50 msgr2=0x7f20580fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- 172.21.15.122:0/1038357772 shutdown_connections 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- 172.21.15.122:0/1038357772 wait complete. 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 Processor -- start 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.269+0000 7f205fa63700 1 -- start start 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205fa63700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205fa63700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2058108060 con 0x7f2058100310 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205d7ff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205d7ff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52736/0 (socket says 172.21.15.122:52736) 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205d7ff700 1 -- 172.21.15.122:0/188442211 learned_addr learned my addr 172.21.15.122:0/188442211 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205d7ff700 1 -- 172.21.15.122:0/188442211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2058197c20 con 0x7f2058100310 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205d7ff700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2054006010 tx=0x7f2054005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2054027070 con 0x7f2058100310 2024-04-02T13:26:08.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f205400be40 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2058197f10 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f205400f460 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.270+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205819aba0 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.271+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f205402a070 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.271+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20581087e0 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.271+0000 7f204a7fc700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 0x7f204403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.271+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f205401e030 con 0x7f2058100310 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.271+0000 7f205cffe700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 0x7f204403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:08.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.272+0000 7f205cffe700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 0x7f204403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f204c006fd0 tx=0x7f204c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:08.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.276+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2054013070 con 0x7f2058100310 2024-04-02T13:26:08.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.480+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f205802df80 con 0x7f2044039650 2024-04-02T13:26:08.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.484+0000 7f204a7fc700 1 -- 172.21.15.122:0/188442211 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f205802df80 con 0x7f2044039650 2024-04-02T13:26:08.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 msgr2=0x7f204403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 0x7f204403bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f204c006fd0 tx=0x7f204c006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 msgr2=0x7f20581976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2054006010 tx=0x7f2054005050 comp rx=0 tx=0).stop 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 shutdown_connections 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2044039650 0x7f204403bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:08.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 --2- 172.21.15.122:0/188442211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2058100310 0x7f20581976e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 >> 172.21.15.122:0/188442211 conn(0x7f20580fbc50 msgr2=0x7f20580fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 shutdown_connections 2024-04-02T13:26:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:08.486+0000 7f205fa63700 1 -- 172.21.15.122:0/188442211 wait complete. 2024-04-02T13:26:09.228 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi187 2024-04-02T13:26:09.229 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:26:09.229 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:26:09.247 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:26:09.247 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:09.304 INFO:tasks.cephadm:Adding host smithi187 to orchestrator... 2024-04-02T13:26:09.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi187 2024-04-02T13:26:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:09.741 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:26:09.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:09.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:09.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:10.666 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:10 smithi122 ceph-mon[32114]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:10.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:10 smithi122 ceph-mon[32114]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:10.667 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:10 smithi122 ceph-mon[32114]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:26:10.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 -- 172.21.15.122:0/2339448553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 msgr2=0x7fb0801062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 --2- 172.21.15.122:0/2339448553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb0801062d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb070009910 tx=0x7fb070009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 -- 172.21.15.122:0/2339448553 shutdown_connections 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 --2- 172.21.15.122:0/2339448553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb0801062d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 -- 172.21.15.122:0/2339448553 >> 172.21.15.122:0/2339448553 conn(0x7fb0800fc110 msgr2=0x7fb0800fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 -- 172.21.15.122:0/2339448553 shutdown_connections 2024-04-02T13:26:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.890+0000 7fb084c4e700 1 -- 172.21.15.122:0/2339448553 wait complete. 2024-04-02T13:26:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb084c4e700 1 Processor -- start 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb084c4e700 1 -- start start 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb084c4e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb084c4e700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb08010b540 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb07e59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb07e59c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36884/0 (socket says 172.21.15.122:36884) 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.891+0000 7fb07e59c700 1 -- 172.21.15.122:0/819164728 learned_addr learned my addr 172.21.15.122:0/819164728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb07e59c700 1 -- 172.21.15.122:0/819164728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08019dc40 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb07e59c700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb070005850 tx=0x7fb070005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070026070 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb07000bc30 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08019ded0 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb070003a20 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.892+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0801978c0 con 0x7fb080105ef0 2024-04-02T13:26:10.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fb070029070 con 0x7fb080105ef0 2024-04-02T13:26:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb06f7fe700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 0x7fb06803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb07001e070 con 0x7fb080105ef0 2024-04-02T13:26:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb07dd9b700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 0x7fb06803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb080102630 con 0x7fb080105ef0 2024-04-02T13:26:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.893+0000 7fb07dd9b700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 0x7fb06803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb074006fd0 tx=0x7fb074006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:10.898+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb070030070 con 0x7fb080105ef0 2024-04-02T13:26:11.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:11.102+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}) -- 0x7fb080198df0 con 0x7fb0680396b0 2024-04-02T13:26:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:11 smithi122 ceph-mon[32114]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi187", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:13.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:12 smithi122 ceph-mon[32114]: Deploying cephadm binary to smithi187 2024-04-02T13:26:15.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.216+0000 7fb06f7fe700 1 -- 172.21.15.122:0/819164728 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fb080198df0 con 0x7fb0680396b0 2024-04-02T13:26:15.220 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi187' with addr '172.21.15.187' 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 msgr2=0x7fb06803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 0x7fb06803bb70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb074006fd0 tx=0x7fb074006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 msgr2=0x7fb08019d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb070005850 tx=0x7fb070005030 comp rx=0 tx=0).stop 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 shutdown_connections 2024-04-02T13:26:15.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0680396b0 0x7fb06803bb70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:15.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 --2- 172.21.15.122:0/819164728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb080105ef0 0x7fb08019d700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:15.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.218+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 >> 172.21.15.122:0/819164728 conn(0x7fb0800fc110 msgr2=0x7fb080104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:15.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.219+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 shutdown_connections 2024-04-02T13:26:15.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:15.219+0000 7fb084c4e700 1 -- 172.21.15.122:0/819164728 wait complete. 2024-04-02T13:26:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:15 smithi122 ceph-mon[32114]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:15.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:26:16.092 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:16.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:16 smithi122 ceph-mon[32114]: Added host smithi187 2024-04-02T13:26:16.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:16.541 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.515+0000 7f96f6ca2700 1 -- 172.21.15.122:0/3948478944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 msgr2=0x7f96f0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.515+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/3948478944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f0106a80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009910 tx=0x7f96e4009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.515+0000 7f96f6ca2700 1 -- 172.21.15.122:0/3948478944 shutdown_connections 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.515+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/3948478944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f0106a80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.515+0000 7f96f6ca2700 1 -- 172.21.15.122:0/3948478944 >> 172.21.15.122:0/3948478944 conn(0x7f96f00fc160 msgr2=0x7f96f00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 -- 172.21.15.122:0/3948478944 shutdown_connections 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 -- 172.21.15.122:0/3948478944 wait complete. 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 Processor -- start 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 -- start start 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.516+0000 7f96f6ca2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96f010ac50 con 0x7f96f01066a0 2024-04-02T13:26:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f4a3e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f4a3e700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36890/0 (socket says 172.21.15.122:36890) 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f4a3e700 1 -- 172.21.15.122:0/2393498532 learned_addr learned my addr 172.21.15.122:0/2393498532 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f4a3e700 1 -- 172.21.15.122:0/2393498532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f0198ef0 con 0x7f96f01066a0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f4a3e700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004f80 tx=0x7f96e4005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e4026070 con 0x7f96f01066a0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e400b6f0 con 0x7f96f01066a0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f01991e0 con 0x7f96f01066a0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96e4003910 con 0x7f96f01066a0 2024-04-02T13:26:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.517+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f01996a0 con 0x7f96f01066a0 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.518+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f96e4029070 con 0x7f96f01066a0 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.518+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f0101d50 con 0x7f96f01066a0 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.518+0000 7f96edffb700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 0x7f96e003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.518+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96e401e070 con 0x7f96f01066a0 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.519+0000 7f96effff700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 0x7f96e003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.519+0000 7f96effff700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 0x7f96e003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.523+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f96e4013070 con 0x7f96f01066a0 2024-04-02T13:26:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:17 smithi122 ceph-mon[32114]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:17 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.733+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96f0063c20 con 0x7f96e00396b0 2024-04-02T13:26:17.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.734+0000 7f96edffb700 1 -- 172.21.15.122:0/2393498532 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f96f0063c20 con 0x7f96e00396b0 2024-04-02T13:26:17.738 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:17.738 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}] 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 msgr2=0x7f96e003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 0x7f96e003bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 msgr2=0x7f96f01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004f80 tx=0x7f96e4005060 comp rx=0 tx=0).stop 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 shutdown_connections 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96e00396b0 0x7f96e003bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:17.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 --2- 172.21.15.122:0/2393498532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96f01066a0 0x7f96f01989b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:17.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 >> 172.21.15.122:0/2393498532 conn(0x7f96f00fc160 msgr2=0x7f96f00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:17.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 shutdown_connections 2024-04-02T13:26:17.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:17.737+0000 7f96f6ca2700 1 -- 172.21.15.122:0/2393498532 wait complete. 2024-04-02T13:26:18.288 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi188 2024-04-02T13:26:18.288 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:26:18.288 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.conf 2024-04-02T13:26:18.307 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:26:18.307 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:18.364 INFO:tasks.cephadm:Adding host smithi188 to orchestrator... 2024-04-02T13:26:18.364 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host add smithi188 2024-04-02T13:26:18.602 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:19.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:19 smithi122 ceph-mon[32114]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:26:19.680 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:19 smithi122 ceph-mon[32114]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:20.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- 172.21.15.122:0/925728499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 msgr2=0x7f9644100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 --2- 172.21.15.122:0/925728499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f9644100ae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9638009910 tx=0x7f9638009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- 172.21.15.122:0/925728499 shutdown_connections 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 --2- 172.21.15.122:0/925728499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f9644100ae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- 172.21.15.122:0/925728499 >> 172.21.15.122:0/925728499 conn(0x7f96440fc120 msgr2=0x7f96440fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- 172.21.15.122:0/925728499 shutdown_connections 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- 172.21.15.122:0/925728499 wait complete. 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 Processor -- start 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.003+0000 7f964ab51700 1 -- start start 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f964ab51700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f964ab51700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f964410c5f0 con 0x7f9644100700 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f96488ed700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:20.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f96488ed700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52418/0 (socket says 172.21.15.122:52418) 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f96488ed700 1 -- 172.21.15.122:0/3322592583 learned_addr learned my addr 172.21.15.122:0/3322592583 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f96488ed700 1 -- 172.21.15.122:0/3322592583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964419dc60 con 0x7f9644100700 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f96488ed700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9638006010 tx=0x7f9638005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9638026070 con 0x7f9644100700 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f963800bbd0 con 0x7f9644100700 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964419df50 con 0x7f9644100700 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9638003910 con 0x7f9644100700 2024-04-02T13:26:20.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.004+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441978e0 con 0x7f9644100700 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.005+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f9638029070 con 0x7f9644100700 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.005+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96441037c0 con 0x7f9644100700 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.005+0000 7f9641ffb700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 0x7f963403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.006+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f963801e070 con 0x7f9644100700 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.006+0000 7f9643fff700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 0x7f963403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:20.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.006+0000 7f9643fff700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 0x7f963403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9630006fd0 tx=0x7f9630006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:20.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.010+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9638013070 con 0x7f9644100700 2024-04-02T13:26:20.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:20.210+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}) -- 0x7f9644198e10 con 0x7f9634039650 2024-04-02T13:26:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:22 smithi122 ceph-mon[32114]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:22 smithi122 ceph-mon[32114]: Deploying cephadm binary to smithi188 2024-04-02T13:26:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:22 smithi122 ceph-mon[32114]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:26:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:23 smithi122 ceph-mon[32114]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:24.387 INFO:teuthology.orchestra.run.smithi122.stdout:Added host 'smithi188' with addr '172.21.15.188' 2024-04-02T13:26:24.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.383+0000 7f9641ffb700 1 -- 172.21.15.122:0/3322592583 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9644198e10 con 0x7f9634039650 2024-04-02T13:26:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 msgr2=0x7f963403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:24.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 0x7f963403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9630006fd0 tx=0x7f9630006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 msgr2=0x7f964419d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9638006010 tx=0x7f9638005030 comp rx=0 tx=0).stop 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 shutdown_connections 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9634039650 0x7f963403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 --2- 172.21.15.122:0/3322592583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9644100700 0x7f964419d720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 >> 172.21.15.122:0/3322592583 conn(0x7f96440fc120 msgr2=0x7f9644109ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 shutdown_connections 2024-04-02T13:26:24.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:24.386+0000 7f964ab51700 1 -- 172.21.15.122:0/3322592583 wait complete. 2024-04-02T13:26:24.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch host ls --format=json 2024-04-02T13:26:25.146 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:25.561 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:25 smithi122 ceph-mon[32114]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:25.561 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:25 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:25.562 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:25 smithi122 ceph-mon[32114]: Added host smithi188 2024-04-02T13:26:25.562 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:25 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:25.562 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:25 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:26.626 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 -- 172.21.15.122:0/2899487548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 msgr2=0x7f3abc103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/2899487548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc103590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009910 tx=0x7f3aac009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 -- 172.21.15.122:0/2899487548 shutdown_connections 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/2899487548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc103590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 -- 172.21.15.122:0/2899487548 >> 172.21.15.122:0/2899487548 conn(0x7f3abc077c90 msgr2=0x7f3abc0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 -- 172.21.15.122:0/2899487548 shutdown_connections 2024-04-02T13:26:26.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.624+0000 7f3ac1a80700 1 -- 172.21.15.122:0/2899487548 wait complete. 2024-04-02T13:26:26.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.625+0000 7f3ac1a80700 1 Processor -- start 2024-04-02T13:26:26.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.625+0000 7f3ac1a80700 1 -- start start 2024-04-02T13:26:26.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.625+0000 7f3ac1a80700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.625+0000 7f3ac1a80700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc10c650 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.625+0000 7f3abb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3abb7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52430/0 (socket says 172.21.15.122:52430) 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3abb7fe700 1 -- 172.21.15.122:0/3474176744 learned_addr learned my addr 172.21.15.122:0/3474176744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3abb7fe700 1 -- 172.21.15.122:0/3474176744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc19dd00 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3abb7fe700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac026070 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac00bc30 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc19df90 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aac0039a0 con 0x7f3abc1031b0 2024-04-02T13:26:26.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.626+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc197980 con 0x7f3abc1031b0 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.627+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f3aac029070 con 0x7f3abc1031b0 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.627+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc06a280 con 0x7f3abc1031b0 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.627+0000 7f3ab97fa700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 0x7f3aa803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.627+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3aac01e070 con 0x7f3abc1031b0 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.627+0000 7f3ab2ffd700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 0x7f3aa803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:26.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.628+0000 7f3ab2ffd700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 0x7f3aa803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:26.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.632+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3aac013070 con 0x7f3abc1031b0 2024-04-02T13:26:26.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.835+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3abc198eb0 con 0x7f3aa8039650 2024-04-02T13:26:26.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.837+0000 7f3ab97fa700 1 -- 172.21.15.122:0/3474176744 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7f3abc198eb0 con 0x7f3aa8039650 2024-04-02T13:26:26.840 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:26:26.840 INFO:teuthology.orchestra.run.smithi122.stdout:[{"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}, {"addr": "172.21.15.187", "hostname": "smithi187", "labels": [], "status": ""}, {"addr": "172.21.15.188", "hostname": "smithi188", "labels": [], "status": ""}] 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 msgr2=0x7f3aa803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 0x7f3aa803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 msgr2=0x7f3abc19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac005030 comp rx=0 tx=0).stop 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 shutdown_connections 2024-04-02T13:26:26.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa8039650 0x7f3aa803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:26.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 --2- 172.21.15.122:0/3474176744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3abc1031b0 0x7f3abc19d7c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:26.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 >> 172.21.15.122:0/3474176744 conn(0x7f3abc077c90 msgr2=0x7f3abc103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:26.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 shutdown_connections 2024-04-02T13:26:26.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:26.839+0000 7f3ac1a80700 1 -- 172.21.15.122:0/3474176744 wait complete. 2024-04-02T13:26:27.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:27 smithi122 ceph-mon[32114]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:27.490 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:27.492 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-02T13:26:27.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd crush tunables default 2024-04-02T13:26:27.727 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:28.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:28 smithi122 ceph-mon[32114]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:26:29.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 -- 172.21.15.122:0/2918368725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 msgr2=0x7f14ec106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 --2- 172.21.15.122:0/2918368725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec106a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00aa20 tx=0x7f14dc00ad30 comp rx=0 tx=0).stop 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 -- 172.21.15.122:0/2918368725 shutdown_connections 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 --2- 172.21.15.122:0/2918368725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec106a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 -- 172.21.15.122:0/2918368725 >> 172.21.15.122:0/2918368725 conn(0x7f14ec0775e0 msgr2=0x7f14ec0779f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 -- 172.21.15.122:0/2918368725 shutdown_connections 2024-04-02T13:26:29.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.231+0000 7f14f29a3700 1 -- 172.21.15.122:0/2918368725 wait complete. 2024-04-02T13:26:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.232+0000 7f14f29a3700 1 Processor -- start 2024-04-02T13:26:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.232+0000 7f14f29a3700 1 -- start start 2024-04-02T13:26:29.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.232+0000 7f14f29a3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.232+0000 7f14f29a3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14ec10a410 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.232+0000 7f14ebfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14ebfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43000/0 (socket says 172.21.15.122:43000) 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14ebfff700 1 -- 172.21.15.122:0/341576202 learned_addr learned my addr 172.21.15.122:0/341576202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14ebfff700 1 -- 172.21.15.122:0/341576202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ec198110 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14ebfff700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14dc004c90 tx=0x7f14dc004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14dc005310 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14dc0054b0 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ec1983a0 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14dc00e790 con 0x7f14ec106650 2024-04-02T13:26:29.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.233+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ec19c4d0 con 0x7f14ec106650 2024-04-02T13:26:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.234+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f14dc00ea90 con 0x7f14ec106650 2024-04-02T13:26:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.234+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ec101d90 con 0x7f14ec106650 2024-04-02T13:26:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.234+0000 7f14e97fa700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 0x7f14d403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.234+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14dc01e070 con 0x7f14ec106650 2024-04-02T13:26:29.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.234+0000 7f14eb7fe700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 0x7f14d403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:29.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.235+0000 7f14eb7fe700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 0x7f14d403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14e0006fd0 tx=0x7f14e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:29.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.239+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f14dc020370 con 0x7f14ec106650 2024-04-02T13:26:29.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.444+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f14ec063c20 con 0x7f14ec106650 2024-04-02T13:26:29.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.464+0000 7f14e97fa700 1 -- 172.21.15.122:0/341576202 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f14dc00d3e0 con 0x7f14ec106650 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 msgr2=0x7f14d403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 0x7f14d403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14e0006fd0 tx=0x7f14e0006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 msgr2=0x7f14ec197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14dc004c90 tx=0x7f14dc004d70 comp rx=0 tx=0).stop 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 shutdown_connections 2024-04-02T13:26:29.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14d4039650 0x7f14d403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:29.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 --2- 172.21.15.122:0/341576202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ec106650 0x7f14ec197bd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:29.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 >> 172.21.15.122:0/341576202 conn(0x7f14ec0775e0 msgr2=0x7f14ec104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:29.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 shutdown_connections 2024-04-02T13:26:29.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:29.467+0000 7f14f29a3700 1 -- 172.21.15.122:0/341576202 wait complete. 2024-04-02T13:26:29.471 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-04-02T13:26:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:29 smithi122 ceph-mon[32114]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:29 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/341576202' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:26:29.989 INFO:tasks.cephadm:Adding mon.a on smithi122 2024-04-02T13:26:29.989 INFO:tasks.cephadm:Adding mon.b on smithi187 2024-04-02T13:26:29.989 INFO:tasks.cephadm:Adding mon.c on smithi188 2024-04-02T13:26:29.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mon '3;smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c' 2024-04-02T13:26:30.235 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T13:26:30.282 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-02T13:26:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/341576202' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:26:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:26:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:31.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 -- 172.21.15.188:0/2224989070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 msgr2=0x7f7ec4101510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:31.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/2224989070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec4101510 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009910 tx=0x7f7eb8009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:31.816 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 -- 172.21.15.188:0/2224989070 shutdown_connections 2024-04-02T13:26:31.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/2224989070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec4101510 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:31.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 -- 172.21.15.188:0/2224989070 >> 172.21.15.188:0/2224989070 conn(0x7f7ec40fc9c0 msgr2=0x7f7ec40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:31.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 -- 172.21.15.188:0/2224989070 shutdown_connections 2024-04-02T13:26:31.817 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.814+0000 7f7ecaeff700 1 -- 172.21.15.188:0/2224989070 wait complete. 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ecaeff700 1 Processor -- start 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ecaeff700 1 -- start start 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ecaeff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ecaeff700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec410b500 con 0x7f7ec4101130 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ec8c9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ec8c9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.188:49516/0 (socket says 172.21.15.188:49516) 2024-04-02T13:26:31.818 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.815+0000 7f7ec8c9b700 1 -- 172.21.15.188:0/3353777008 learned_addr learned my addr 172.21.15.188:0/3353777008 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ec8c9b700 1 -- 172.21.15.188:0/3353777008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec419cbb0 con 0x7f7ec4101130 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ec8c9b700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005850 tx=0x7f7eb8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb8026070 con 0x7f7ec4101130 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec419cea0 con 0x7f7ec4101130 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb800bbd0 con 0x7f7ec4101130 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7eb8003970 con 0x7f7ec4101130 2024-04-02T13:26:31.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.816+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec41967f0 con 0x7f7ec4101130 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.817+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f7eb8029070 con 0x7f7ec4101130 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.817+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec4102610 con 0x7f7ec4101130 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.817+0000 7f7ec1ffb700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 0x7f7eb403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.817+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7eb801e070 con 0x7f7ec4101130 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.818+0000 7f7ec3fff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 0x7f7eb403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:31.820 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.818+0000 7f7ec3fff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 0x7f7eb403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:31.825 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:31.823+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7eb8013070 con 0x7f7ec4101130 2024-04-02T13:26:32.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.031+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}) -- 0x7f7ec4197de0 con 0x7f7eb4039650 2024-04-02T13:26:32.039 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.038+0000 7f7ec1ffb700 1 -- 172.21.15.188:0/3353777008 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7ec4197de0 con 0x7f7eb4039650 2024-04-02T13:26:32.040 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mon update... 2024-04-02T13:26:32.042 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 msgr2=0x7f7eb403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 0x7f7eb403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006fd0 tx=0x7f7eb0006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 msgr2=0x7f7ec419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8005850 tx=0x7f7eb8005030 comp rx=0 tx=0).stop 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 shutdown_connections 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7eb4039650 0x7f7eb403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 --2- 172.21.15.188:0/3353777008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec4101130 0x7f7ec419c670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 >> 172.21.15.188:0/3353777008 conn(0x7f7ec40fc9c0 msgr2=0x7f7ec4102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 shutdown_connections 2024-04-02T13:26:32.047 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:32.040+0000 7f7ecaeff700 1 -- 172.21.15.188:0/3353777008 wait complete. 2024-04-02T13:26:32.605 DEBUG:teuthology.orchestra.run.smithi187:mon.b> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.b.service 2024-04-02T13:26:32.608 DEBUG:teuthology.orchestra.run.smithi188:mon.c> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.c.service 2024-04-02T13:26:32.610 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:26:32.610 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:26:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:26:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:33.282 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:33 smithi122 ceph-mon[32114]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:33 smithi122 ceph-mon[32114]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c;count:3 2024-04-02T13:26:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:33 smithi122 ceph-mon[32114]: Deploying daemon mon.c on smithi188 2024-04-02T13:26:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:33 smithi122 ceph-mon[32114]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:35 smithi122 ceph-mon[32114]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:36.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 -- 172.21.15.188:0/3145636731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 msgr2=0x7fb058101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:36.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/3145636731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb058101530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb048009910 tx=0x7fb048009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:36.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 -- 172.21.15.188:0/3145636731 shutdown_connections 2024-04-02T13:26:36.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/3145636731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb058101530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:36.402 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 -- 172.21.15.188:0/3145636731 >> 172.21.15.188:0/3145636731 conn(0x7fb058077c90 msgr2=0x7fb0580ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 -- 172.21.15.188:0/3145636731 shutdown_connections 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.397+0000 7fb05ca5d700 1 -- 172.21.15.188:0/3145636731 wait complete. 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.398+0000 7fb05ca5d700 1 Processor -- start 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.398+0000 7fb05ca5d700 1 -- start start 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05ca5d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05ca5d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb05810b4d0 con 0x7fb058101150 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.188:49546/0 (socket says 172.21.15.188:49546) 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05659c700 1 -- 172.21.15.188:0/2366857977 learned_addr learned my addr 172.21.15.188:0/2366857977 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05659c700 1 -- 172.21.15.188:0/2366857977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05819dc70 con 0x7fb058101150 2024-04-02T13:26:36.403 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.399+0000 7fb05659c700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb048005850 tx=0x7fb048005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:36.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.400+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb048026070 con 0x7fb058101150 2024-04-02T13:26:36.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.400+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04800bbd0 con 0x7fb058101150 2024-04-02T13:26:36.404 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.400+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05819df60 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.400+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb048003970 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.400+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb058197940 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.401+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7fb048029070 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.401+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0581025e0 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.401+0000 7fb0477fe700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 0x7fb04003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.401+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb04801e070 con 0x7fb058101150 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.401+0000 7fb055d9b700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 0x7fb04003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:36.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.402+0000 7fb055d9b700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 0x7fb04003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006fd0 tx=0x7fb04c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:36.409 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.407+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb048013070 con 0x7fb058101150 2024-04-02T13:26:36.705 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.701+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb058198e20 con 0x7fb058101150 2024-04-02T13:26:36.705 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.702+0000 7fb0477fe700 1 -- 172.21.15.188:0/2366857977 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7fb048030090 con 0x7fb058101150 2024-04-02T13:26:36.705 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:26:36.705 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":1,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:24:53.926501Z","created":"2024-04-02T13:24:53.926501Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-02T13:26:36.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 msgr2=0x7fb04003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:36.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 0x7fb04003bb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb04c006fd0 tx=0x7fb04c006e40 comp rx=0 tx=0).stop 2024-04-02T13:26:36.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 msgr2=0x7fb05819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:36.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb048005850 tx=0x7fb048005030 comp rx=0 tx=0).stop 2024-04-02T13:26:36.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 shutdown_connections 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0400396b0 0x7fb04003bb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 --2- 172.21.15.188:0/2366857977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb058101150 0x7fb05819d730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 >> 172.21.15.188:0/2366857977 conn(0x7fb058077c90 msgr2=0x7fb0580699f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 shutdown_connections 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:36.704+0000 7fb05ca5d700 1 -- 172.21.15.188:0/2366857977 wait complete. 2024-04-02T13:26:36.708 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 1 2024-04-02T13:26:36.993 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:36 smithi188 ceph-mon[36117]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T13:26:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:36 smithi188 bash[36063]: 0f49511e3d89640a5aa4f519e93ccdac509c933021a0d85e7ffe8ac365ea6f8f 2024-04-02T13:26:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:36 smithi188 systemd[1]: Started Ceph mon.c for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:26:38.431 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:26:38.431 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:26:38.678 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: mon.a calling monitor election 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: mon.c calling monitor election 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: monmap epoch 2 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: last_changed 2024-04-02T13:26:36.783501+0000 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: min_mon_release 19 (squid) 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: election_strategy: 1 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: fsmap 2024-04-02T13:26:42.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:42.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: mgrmap e13: x(active, since 47s) 2024-04-02T13:26:42.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: overall HEALTH_OK 2024-04-02T13:26:42.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:42.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: mon.a calling monitor election 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: mon.c calling monitor election 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:26:42.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: monmap epoch 2 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: last_changed 2024-04-02T13:26:36.783501+0000 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: min_mon_release 19 (squid) 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: election_strategy: 1 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: fsmap 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: mgrmap e13: x(active, since 47s) 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: overall HEALTH_OK 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:42.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:43.144 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:43.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:43.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:43.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: Deploying daemon mon.b on smithi187 2024-04-02T13:26:43.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:43.145 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:42 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:43.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 -- 172.21.15.188:0/2021404001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf40035c0 msgr2=0x7f4bf4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:43.146 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2021404001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf40035c0 0x7f4bf4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c100f30 tx=0x7f4c04009e80 comp rx=0 tx=0).stop 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 -- 172.21.15.188:0/2021404001 shutdown_connections 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2021404001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4bf40035c0 0x7f4bf4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 -- 172.21.15.188:0/2021404001 >> 172.21.15.188:0/2021404001 conn(0x7f4c0c0fcb10 msgr2=0x7f4c0c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 -- 172.21.15.188:0/2021404001 shutdown_connections 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.141+0000 7f4c113b3700 1 -- 172.21.15.188:0/2021404001 wait complete. 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.142+0000 7f4c113b3700 1 Processor -- start 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.142+0000 7f4c113b3700 1 -- start start 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c113b3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c113b3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 0x7f4c0c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c113b3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c109090 con 0x7f4c0c19dde0 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c113b3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0c108f10 con 0x7f4bf40035c0 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53816/0 (socket says 172.21.15.188:53816) 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 -- 172.21.15.188:0/2041663436 learned_addr learned my addr 172.21.15.188:0/2041663436 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:26:43.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0a7fc700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 0x7f4c0c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 -- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 msgr2=0x7f4c0c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 -- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 msgr2=0x7f4c0c19d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0affd700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0a7fc700 1 -- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 msgr2=0x7f4c0c19d8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0a7fc700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.143+0000 7f4c0a7fc700 1 -- 172.21.15.188:0/2041663436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0c19e420 con 0x7f4c0c19dde0 2024-04-02T13:26:43.148 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4c0a7fc700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 0x7f4c0c197b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b4d0 tx=0x7f4c0000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4c00011630 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c00011cb0 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0c198040 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4c00010e60 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.144+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0c1985d0 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.145+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f4c00018020 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.145+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0c102740 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.145+0000 7f4bf3fff700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 0x7f4bf803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.145+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4c0004d290 con 0x7f4c0c19dde0 2024-04-02T13:26:43.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.146+0000 7f4c0affd700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 0x7f4bf803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:43.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.146+0000 7f4c0affd700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 0x7f4bf803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c0401c040 tx=0x7f4c04005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:43.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.151+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4c0001f020 con 0x7f4c0c19dde0 2024-04-02T13:26:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: Deploying daemon mon.b on smithi187 2024-04-02T13:26:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:42 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:43.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.434+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4c0c198f90 con 0x7f4c0c19dde0 2024-04-02T13:26:43.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.434+0000 7f4bf3fff700 1 -- 172.21.15.188:0/2041663436 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7f4c00018570 con 0x7f4c0c19dde0 2024-04-02T13:26:43.437 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:26:43.438 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":2,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:26:36.783501Z","created":"2024-04-02T13:24:53.926501Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-02T13:26:43.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 msgr2=0x7f4bf803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:43.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 0x7f4bf803b8b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c0401c040 tx=0x7f4c04005040 comp rx=0 tx=0).stop 2024-04-02T13:26:43.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 msgr2=0x7f4c0c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:43.439 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 0x7f4c0c197b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4c0000b4d0 tx=0x7f4c0000b890 comp rx=0 tx=0).stop 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 shutdown_connections 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bf40035c0 0x7f4c0c19d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4bf80393f0 0x7f4bf803b8b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 --2- 172.21.15.188:0/2041663436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c0c19dde0 0x7f4c0c197b00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.436+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 >> 172.21.15.188:0/2041663436 conn(0x7f4c0c0fcb10 msgr2=0x7f4c0c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.437+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 shutdown_connections 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:43.437+0000 7f4c113b3700 1 -- 172.21.15.188:0/2041663436 wait complete. 2024-04-02T13:26:43.440 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 2 2024-04-02T13:26:43.997 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:43 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/2041663436' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:44.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:43 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/2041663436' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:45.025 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-02T13:26:45.026 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph mon dump -f json 2024-04-02T13:26:45.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:44 smithi122 ceph-mon[32114]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:44 smithi188 ceph-mon[36117]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.270 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:26:45.449 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: RocksDB version: 7.9.2 2024-04-02T13:26:45.449 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Git sha 0 2024-04-02T13:26:45.449 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Compile date 2024-04-02 01:02:34 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: DB SUMMARY 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: DB Session ID: VF24N26W655UH8XRL81S 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: CURRENT file: CURRENT 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: IDENTITY file: IDENTITY 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.error_if_exists: 0 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.create_if_missing: 0 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.paranoid_checks: 1 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-02T13:26:45.450 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.env: 0x55b2be7d8c60 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.fs: PosixFileSystem 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.info_log: 0x55b2c0c9dae0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_file_opening_threads: 16 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.statistics: (nil) 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.use_fsync: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_log_file_size: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.keep_log_file_num: 1000 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.recycle_log_file_num: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_fallocate: 1 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_mmap_reads: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_mmap_writes: 0 2024-04-02T13:26:45.451 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.use_direct_reads: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.create_missing_column_families: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.db_log_dir: 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.wal_dir: 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.advise_random_on_open: 1 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.db_write_buffer_size: 0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.write_buffer_manager: 0x55b2c0c6e1e0 2024-04-02T13:26:45.452 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-02T13:26:45.453 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-02T13:26:45.453 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-02T13:26:45.453 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.rate_limiter: (nil) 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.wal_recovery_mode: 2 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enable_thread_tracking: 0 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enable_pipelined_write: 0 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.unordered_write: 0 2024-04-02T13:26:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.row_cache: None 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.wal_filter: None 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_ingest_behind: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.two_write_queues: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.manual_wal_flush: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.wal_compression: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.atomic_flush: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-02T13:26:45.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.log_readahead_size: 0 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.best_efforts_recovery: 0 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.allow_data_in_errors: 0 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.db_host_id: __hostname__ 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_background_jobs: 2 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_background_compactions: -1 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_subcompactions: 1 2024-04-02T13:26:45.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_total_wal_size: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_open_files: -1 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bytes_per_sync: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_readahead_size: 0 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_background_flushes: -1 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Compression algorithms supported: 2024-04-02T13:26:45.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kZSTD supported: 0 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kXpressCompression supported: 0 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kLZ4HCCompression supported: 1 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kZlibCompression supported: 1 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kSnappyCompression supported: 1 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kLZ4Compression supported: 1 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: kBZip2Compression supported: 0 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.merge_operator: 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_filter: None 2024-04-02T13:26:45.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_filter_factory: None 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.sst_partitioner_factory: None 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b2c0c9dcc0) 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: cache_index_and_filter_blocks: 1 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: pin_top_level_index_and_filter: 1 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_type: 0 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: data_block_index_type: 0 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_shortening: 1 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: checksum: 4 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: no_block_cache: 0 2024-04-02T13:26:45.741 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache: 0x55b2c0ca1090 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_name: BinnedLRUCache 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_options: 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: capacity : 536870912 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: num_shard_bits : 4 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: strict_capacity_limit : 0 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: high_pri_pool_ratio: 0.000 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_cache_compressed: (nil) 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: persistent_cache: (nil) 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_size: 4096 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_size_deviation: 10 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_restart_interval: 16 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: index_block_restart_interval: 1 2024-04-02T13:26:45.742 INFO:journalctl@ceph.mon.b.smithi187.stdout: metadata_block_size: 4096 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: partition_filters: 0 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: use_delta_encoding: 1 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: filter_policy: bloomfilter 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: whole_key_filtering: 1 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: verify_compression: 0 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: read_amp_bytes_per_bit: 0 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: format_version: 5 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: enable_index_compression: 1 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: block_align: 0 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: max_auto_readahead_size: 262144 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: prepopulate_block_cache: 0 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: initial_auto_readahead_size: 8192 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout: num_file_reads_for_auto_readahead: 2 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.write_buffer_size: 33554432 2024-04-02T13:26:45.743 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_write_buffer_number: 2 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression: NoCompression 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression: Disabled 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.prefix_extractor: nullptr 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.num_levels: 7 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-02T13:26:45.744 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.level: 32767 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.strategy: 0 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-02T13:26:45.745 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.enabled: false 2024-04-02T13:26:45.746 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.target_file_size_base: 67108864 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-02T13:26:45.747 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.arena_block_size: 1048576 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.disable_auto_compactions: 0 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-02T13:26:45.748 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.inplace_update_support: 0 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-02T13:26:45.749 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.bloom_locality: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.max_successive_merges: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.paranoid_file_checks: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.force_consistency_checks: 1 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.report_bg_io_stats: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.ttl: 2592000 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enable_blob_files: false 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.min_blob_size: 0 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_file_size: 268435456 2024-04-02T13:26:45.750 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.blob_file_starting_level: 0 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 14c03a4c-944b-4c07-94b5-ced3ef5cb99c 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064405447028, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064405447487, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1712064405, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "14c03a4c-944b-4c07-94b5-ced3ef5cb99c", "db_session_id": "VF24N26W655UH8XRL81S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-02T13:26:45.751 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: EVENT_LOG_v1 {"time_micros": 1712064405447568, "job": 1, "event": "recovery_finished"} 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b2c0d90000 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: DB pointer 0x55b2c0d7a000 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** DB Stats ** 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-02T13:26:45.752 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** Compaction Stats [default] ** 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** Compaction Stats [default] ** 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.753 INFO:journalctl@ceph.mon.b.smithi187.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Block cache BinnedLRUCache@0x55b2c0ca1090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-02T13:26:45.754 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: using public_addr v2:172.21.15.187:0/0 -> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: starting mon.b rank -1 at public addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] at bind addrs [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(???) e0 preinit fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).mds e1 new map 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: e1 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: legacy client fscid: -1 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout: No filesystems configured 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-02T13:26:45.755 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi188", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi187", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Deploying cephadm binary to smithi188 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.757 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Added host smithi188 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/341576202' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/341576202' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.758 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi188", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:45.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Saving service mon spec with placement smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c;count:3 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Deploying daemon mon.c on smithi188 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.a calling monitor election 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.760 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.c calling monitor election 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: monmap epoch 2 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: last_changed 2024-04-02T13:26:36.783501+0000 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: min_mon_release 19 (squid) 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: election_strategy: 1 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: fsmap 2024-04-02T13:26:45.761 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mgrmap e13: x(active, since 47s) 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: overall HEALTH_OK 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: Deploying daemon mon.b on smithi187 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/2041663436' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 ceph-mon[36064]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-02T13:26:45.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 bash[36045]: 2d596404099a55f2602781b2030082928ee96b6dbd2eb02739ece9b9a90e77ff 2024-04-02T13:26:45.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:45 smithi187 systemd[1]: Started Ceph mon.b for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: mon.a calling monitor election 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: mon.c calling monitor election 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: mon.b calling monitor election 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: monmap epoch 3 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: last_changed 2024-04-02T13:26:45.469382+0000 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: min_mon_release 19 (squid) 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: election_strategy: 1 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:50.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: fsmap 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: mgrmap e13: x(active, since 56s) 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: overall HEALTH_OK 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: mon.a calling monitor election 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: mon.c calling monitor election 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: mon.b calling monitor election 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: monmap epoch 3 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: last_changed 2024-04-02T13:26:45.469382+0000 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: min_mon_release 19 (squid) 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: election_strategy: 1 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: fsmap 2024-04-02T13:26:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: mgrmap e13: x(active, since 56s) 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: overall HEALTH_OK 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:50.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:26:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: mon.a calling monitor election 2024-04-02T13:26:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: mon.c calling monitor election 2024-04-02T13:26:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: mon.b calling monitor election 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: monmap epoch 3 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: last_changed 2024-04-02T13:26:45.469382+0000 2024-04-02T13:26:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: created 2024-04-02T13:24:53.926501+0000 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: min_mon_release 19 (squid) 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: election_strategy: 1 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: 0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: 1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.c 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: 2: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: fsmap 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: osdmap e4: 0 total, 0 up, 0 in 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: mgrmap e13: x(active, since 56s) 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: overall HEALTH_OK 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:51.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 -- 172.21.15.188:0/978508263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 msgr2=0x7f4594005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/978508263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f4594005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45a8100f00 tx=0x7f459800fe60 comp rx=0 tx=0).stop 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 -- 172.21.15.188:0/978508263 shutdown_connections 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/978508263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f4594005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 -- 172.21.15.188:0/978508263 >> 172.21.15.188:0/978508263 conn(0x7f45a80fcaa0 msgr2=0x7f45a80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:51.158 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 -- 172.21.15.188:0/978508263 shutdown_connections 2024-04-02T13:26:51.159 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.155+0000 7f45ae0b7700 1 -- 172.21.15.188:0/978508263 wait complete. 2024-04-02T13:26:51.161 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 Processor -- start 2024-04-02T13:26:51.161 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 -- start start 2024-04-02T13:26:51.161 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f45a8197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8109010 con 0x7f45940035c0 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8108e90 con 0x7f45a819cb70 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45ae0b7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8109190 con 0x7f45a81982a0 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:51870/0 (socket says 172.21.15.188:51870) 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 -- 172.21.15.188:0/49303589 learned_addr learned my addr 172.21.15.188:0/49303589 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f45a8197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 msgr2=0x7f45a819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 msgr2=0x7f45a819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:26:51.162 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f459ffff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a7fff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a7fff700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 msgr2=0x7f45a819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a7fff700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 msgr2=0x7f45a819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a7fff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 msgr2=0x7f45a819c630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a7fff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 msgr2=0x7f45a819d020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.156+0000 7f45a77fe700 1 -- 172.21.15.188:0/49303589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81a48e0 con 0x7f45940035c0 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45a77fe700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f45a8197d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45a8100f00 tx=0x7f4598005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459802b020 con 0x7f45940035c0 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a81a4ab0 con 0x7f45940035c0 2024-04-02T13:26:51.163 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4598004180 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598022040 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.157+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a5040 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.158+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f459802e020 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.158+0000 7f45a57fa700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 0x7f458803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.158+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f459804f420 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.158+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a8063c20 con 0x7f45940035c0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.159+0000 7f459ffff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 0x7f458803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:51.164 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.159+0000 7f459ffff700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 0x7f458803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4590009180 tx=0x7f4590008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:51.167 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.164+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4598035020 con 0x7f45940035c0 2024-04-02T13:26:51.464 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.461+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f45a802dd30 con 0x7f45940035c0 2024-04-02T13:26:51.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.462+0000 7f45a57fa700 1 -- 172.21.15.188:0/49303589 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1289 (secure 0 0 0) 0x7f4598010c20 con 0x7f45940035c0 2024-04-02T13:26:51.465 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-04-02T13:26:51.465 INFO:teuthology.orchestra.run.smithi188.stdout:{"epoch":3,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","modified":"2024-04-02T13:26:45.469382Z","created":"2024-04-02T13:24:53.926501Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:3300","nonce":0},{"type":"v1","addr":"172.21.15.188:6789","nonce":0}]},"addr":"172.21.15.188:6789/0","public_addr":"172.21.15.188:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:3300","nonce":0},{"type":"v1","addr":"172.21.15.187:6789","nonce":0}]},"addr":"172.21.15.187:6789/0","public_addr":"172.21.15.187:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 msgr2=0x7f458803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 0x7f458803bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4590009180 tx=0x7f4590008040 comp rx=0 tx=0).stop 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 msgr2=0x7f45a8197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f45a8197d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45a8100f00 tx=0x7f4598005790 comp rx=0 tx=0).stop 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 shutdown_connections 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f45a819cb70 0x7f45a819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.467 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4588039ae0 0x7f458803bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45940035c0 0x7f45a8197d60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 --2- 172.21.15.188:0/49303589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45a81982a0 0x7f45a819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 >> 172.21.15.188:0/49303589 conn(0x7f45a80fcaa0 msgr2=0x7f45a806b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 shutdown_connections 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:51.464+0000 7f45ae0b7700 1 -- 172.21.15.188:0/49303589 wait complete. 2024-04-02T13:26:51.468 INFO:teuthology.orchestra.run.smithi188.stderr:dumped monmap epoch 3 2024-04-02T13:26:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:51 smithi187 ceph-mon[36064]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:51 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/49303589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:51 smithi188 ceph-mon[36117]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:51 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/49303589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:51 smithi122 ceph-mon[32114]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:51 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/49303589' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-02T13:26:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:26:52.161 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-02T13:26:52.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph config generate-minimal-conf 2024-04-02T13:26:52.417 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:26:53.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:53.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:26:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:52 smithi122 ceph-mon[32114]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:26:53.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:26:53.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:52 smithi187 ceph-mon[36064]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:26:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:26:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:52 smithi188 ceph-mon[36117]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 -- 172.21.15.122:0/3543296453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8102210 msgr2=0x7f69e81025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3543296453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8102210 0x7f69e81025f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009910 tx=0x7f69e4009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 -- 172.21.15.122:0/3543296453 shutdown_connections 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3543296453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8102210 0x7f69e81025f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 -- 172.21.15.122:0/3543296453 >> 172.21.15.122:0/3543296453 conn(0x7f69e80fdac0 msgr2=0x7f69e80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 -- 172.21.15.122:0/3543296453 shutdown_connections 2024-04-02T13:26:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.928+0000 7f69ef68b700 1 -- 172.21.15.122:0/3543296453 wait complete. 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 Processor -- start 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 -- start start 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e8102210 0x7f69e8197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 0x7f69e819cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810c5e0 con 0x7f69e8198270 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810c460 con 0x7f69e8102210 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ef68b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69e810c760 con 0x7f69e819cb40 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ecc26700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ecc26700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51308/0 (socket says 172.21.15.122:51308) 2024-04-02T13:26:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ecc26700 1 -- 172.21.15.122:0/3725384727 learned_addr learned my addr 172.21.15.122:0/3725384727 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.929+0000 7f69ed427700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e8102210 0x7f69e8197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69edc28700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 0x7f69e819cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 -- 172.21.15.122:0/3725384727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 msgr2=0x7f69e819cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 0x7f69e819cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 -- 172.21.15.122:0/3725384727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e8102210 msgr2=0x7f69e8197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e8102210 0x7f69e8197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 -- 172.21.15.122:0/3725384727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81a48b0 con 0x7f69e8198270 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69edc28700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 0x7f69e819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:26:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ecc26700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69d800b780 tx=0x7f69d800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8004230 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69d80043d0 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e81a4ae0 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8011c90 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.930+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e81a5020 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.931+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f69d801a400 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.931+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8063c20 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.931+0000 7f69de7fc700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 0x7f69d403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.931+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f69d804c4b0 con 0x7f69e8198270 2024-04-02T13:26:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.931+0000 7f69ed427700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 0x7f69d403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:53.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.932+0000 7f69ed427700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 0x7f69d403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69e4006010 tx=0x7f69e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:53.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:53.936+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f69d8019d60 con 0x7f69e8198270 2024-04-02T13:26:54.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.141+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f69e81a5340 con 0x7f69e8198270 2024-04-02T13:26:54.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.141+0000 7f69de7fc700 1 -- 172.21.15.122:0/3725384727 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f69d801a840 con 0x7f69e8198270 2024-04-02T13:26:54.145 INFO:teuthology.orchestra.run.smithi122.stdout:# minimal ceph.conf for 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:54.145 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-04-02T13:26:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:26:54.145 INFO:teuthology.orchestra.run.smithi122.stdout: mon_host = [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 msgr2=0x7f69d403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 0x7f69d403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f69e4006010 tx=0x7f69e40058e0 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 msgr2=0x7f69e819c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f69d800b780 tx=0x7f69d800ba90 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 shutdown_connections 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e819cb40 0x7f69e819cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69e8102210 0x7f69e8197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69d4039ae0 0x7f69d403bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 --2- 172.21.15.122:0/3725384727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69e8198270 0x7f69e819c600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 >> 172.21.15.122:0/3725384727 conn(0x7f69e80fdac0 msgr2=0x7f69e8103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:54.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 shutdown_connections 2024-04-02T13:26:54.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:26:54.143+0000 7f69ef68b700 1 -- 172.21.15.122:0/3725384727 wait complete. 2024-04-02T13:26:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:53 smithi187 ceph-mon[36064]: Reconfiguring daemon mon.a on smithi122 2024-04-02T13:26:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:53 smithi188 ceph-mon[36117]: Reconfiguring daemon mon.a on smithi122 2024-04-02T13:26:54.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: Reconfiguring mon.a (unknown last config time)... 2024-04-02T13:26:54.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:54.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:54.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:54.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:53 smithi122 ceph-mon[32114]: Reconfiguring daemon mon.a on smithi122 2024-04-02T13:26:54.790 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-02T13:26:54.790 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:54.790 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:26:54.820 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:54.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:54.888 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:26:54.888 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:26:54.919 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:26:54.919 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:54.987 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:26:54.988 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.conf 2024-04-02T13:26:55.022 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:26:55.022 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:26:55.093 INFO:tasks.cephadm:Adding mgr.x on smithi122 2024-04-02T13:26:55.093 INFO:tasks.cephadm:Adding mgr.y on smithi187 2024-04-02T13:26:55.094 INFO:tasks.cephadm:Adding mgr.z on smithi188 2024-04-02T13:26:55.094 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mgr '3;smithi122=x;smithi187=y;smithi188=z' 2024-04-02T13:26:55.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:54 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3725384727' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:55.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:54 smithi187 ceph-mon[36064]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:55.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:55.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:54 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3725384727' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:55.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:54 smithi188 ceph-mon[36117]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:55.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:55.285 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:54 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3725384727' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:55.285 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:54 smithi122 ceph-mon[32114]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:55.285 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:55.372 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:26:56.736 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:56.736 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: Reconfiguring daemon mon.b on smithi187 2024-04-02T13:26:56.737 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:56 smithi188 ceph-mon[36117]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 -- 172.21.15.188:0/4090947744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 msgr2=0x7f7ec0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 --2- 172.21.15.188:0/4090947744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec0101470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009910 tx=0x7f7eb0009c20 comp rx=0 tx=0).stop 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 -- 172.21.15.188:0/4090947744 shutdown_connections 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 --2- 172.21.15.188:0/4090947744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec0101470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 -- 172.21.15.188:0/4090947744 >> 172.21.15.188:0/4090947744 conn(0x7f7ec00fc5d0 msgr2=0x7f7ec00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 -- 172.21.15.188:0/4090947744 shutdown_connections 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.835+0000 7f7ec550c700 1 -- 172.21.15.188:0/4090947744 wait complete. 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 Processor -- start 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 -- start start 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:56.845 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 0x7f7ec019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 0x7f7ec019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec01070e0 con 0x7f7ec0101050 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0106f60 con 0x7f7ec019cff0 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ec550c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0107260 con 0x7f7ec019a670 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 0x7f7ec019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebeffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.188:44102/0 (socket says 172.21.15.188:44102) 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebeffd700 1 -- 172.21.15.188:0/2407115385 learned_addr learned my addr 172.21.15.188:0/2407115385 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebf7fe700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 0x7f7ec019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 -- 172.21.15.188:0/2407115385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 msgr2=0x7f7ec019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 0x7f7ec019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:56.846 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 -- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 msgr2=0x7f7ec006b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebe7fc700 1 -- 172.21.15.188:0/2407115385 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec019fd60 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebf7fe700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 0x7f7ec019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.836+0000 7f7ebeffd700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ebe7fc700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 0x7f7ec019ca60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b550 tx=0x7f7eb400b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4004230 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec019fff0 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb40043d0 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4011c20 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.837+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec01a05d0 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f7eb40036a0 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec0063c20 con 0x7f7ec019a670 2024-04-02T13:26:56.847 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ea7fff700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 0x7f7ea803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:26:56.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7eb404c300 con 0x7f7ec019a670 2024-04-02T13:26:56.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ebeffd700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 0x7f7ea803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:26:56.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.838+0000 7f7ebeffd700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 0x7f7ea803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006010 tx=0x7f7eb001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:26:56.848 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:56.843+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7eb4021e00 con 0x7f7ec019a670 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: Reconfiguring daemon mon.b on smithi187 2024-04-02T13:26:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:56 smithi187 ceph-mon[36064]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:57.054 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.052+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi187=y;smithi188=z", "target": ["mon-mgr", ""]}) -- 0x7f7ec0062ac0 con 0x7f7ea8039ae0 2024-04-02T13:26:57.062 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.060+0000 7f7ea7fff700 1 -- 172.21.15.188:0/2407115385 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7ec0062ac0 con 0x7f7ea8039ae0 2024-04-02T13:26:57.063 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mgr update... 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 msgr2=0x7f7ea803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 0x7f7ea803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0006010 tx=0x7f7eb001a040 comp rx=0 tx=0).stop 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 msgr2=0x7f7ec019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 0x7f7ec019ca60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400b550 tx=0x7f7eb400b860 comp rx=0 tx=0).stop 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 shutdown_connections 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ec019cff0 0x7f7ec019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7ea8039ae0 0x7f7ea803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:57.065 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ec0101050 0x7f7ec006b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:57.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 --2- 172.21.15.188:0/2407115385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec019a670 0x7f7ec019ca60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:26:57.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 >> 172.21.15.188:0/2407115385 conn(0x7f7ec00fc5d0 msgr2=0x7f7ec00fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:26:57.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 shutdown_connections 2024-04-02T13:26:57.066 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:26:57.062+0000 7f7ec550c700 1 -- 172.21.15.188:0/2407115385 wait complete. 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: Reconfiguring mon.b (monmap changed)... 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: Reconfiguring daemon mon.b on smithi187 2024-04-02T13:26:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:56 smithi122 ceph-mon[32114]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:57.625 DEBUG:teuthology.orchestra.run.smithi187:mgr.y> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.y.service 2024-04-02T13:26:57.628 DEBUG:teuthology.orchestra.run.smithi188:mgr.z> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service 2024-04-02T13:26:57.630 INFO:tasks.cephadm:Deploying OSDs... 2024-04-02T13:26:57.630 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:26:57.630 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:26:57.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:26:57.649 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-02 13:26:04.088007477 +0000 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-02 13:21:14.323303564 +0000 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-02 13:21:14.323303564 +0000 2024-04-02T13:26:57.707 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-02 13:21:14.323303564 +0000 2024-04-02T13:26:57.708 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:26:57.778 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-02T13:26:57.778 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-02T13:26:57.778 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00021428 s, 2.4 MB/s 2024-04-02T13:26:57.780 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:26:57.839 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-02 13:26:04.116006819 +0000 2024-04-02T13:26:57.897 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-02 13:21:14.670295359 +0000 2024-04-02T13:26:57.898 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-02 13:21:14.670295359 +0000 2024-04-02T13:26:57.898 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-02 13:21:14.670295359 +0000 2024-04-02T13:26:57.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:26:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-02T13:26:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-02T13:26:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000228159 s, 2.2 MB/s 2024-04-02T13:26:57.969 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:26:58.032 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-04-02T13:26:58.090 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:26:58.090 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-02 13:26:04.143006184 +0000 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-02 13:21:15.024286989 +0000 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-02 13:21:15.024286989 +0000 2024-04-02T13:26:58.091 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-02 13:21:15.024286989 +0000 2024-04-02T13:26:58.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:26:58.161 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-02T13:26:58.162 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-02T13:26:58.162 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000204386 s, 2.5 MB/s 2024-04-02T13:26:58.163 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:26:58.222 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi187=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: Saving service mgr spec with placement smithi122=x;smithi187=y;smithi188=z;count:3 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:58 smithi188 ceph-mon[36117]: Reconfiguring daemon mon.c on smithi188 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-02 13:26:04.170005550 +0000 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-02 13:21:15.354279187 +0000 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-02 13:21:15.354279187 +0000 2024-04-02T13:26:58.280 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-02 13:21:15.354279187 +0000 2024-04-02T13:26:58.281 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:26:58.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi187=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: Saving service mgr spec with placement smithi122=x;smithi187=y;smithi188=z;count:3 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:58 smithi122 ceph-mon[32114]: Reconfiguring daemon mon.c on smithi188 2024-04-02T13:26:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-04-02T13:26:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-04-02T13:26:58.351 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000218233 s, 2.3 MB/s 2024-04-02T13:26:58.352 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:26:58.413 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-02T13:26:58.414 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:26:58.432 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:26:58.432 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-02T13:26:58.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi122=x;smithi187=y;smithi188=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:26:58.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: Saving service mgr spec with placement smithi122=x;smithi187=y;smithi188=z;count:3 2024-04-02T13:26:58.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:58.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: Reconfiguring mon.c (monmap changed)... 2024-04-02T13:26:58.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-02T13:26:58.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-02T13:26:58.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:26:58.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:58 smithi187 ceph-mon[36064]: Reconfiguring daemon mon.c on smithi188 2024-04-02T13:26:58.490 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:26:58.490 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 658 Links: 1 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-02 13:26:44.923252790 +0000 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-02 13:21:34.428346294 +0000 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-02 13:21:34.428346294 +0000 2024-04-02T13:26:58.491 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-02 13:21:34.428346294 +0000 2024-04-02T13:26:58.491 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:26:58.561 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-02T13:26:58.561 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-02T13:26:58.561 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000239273 s, 2.1 MB/s 2024-04-02T13:26:58.562 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:26:58.621 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 675 Links: 1 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-02 13:26:44.924252774 +0000 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-02 13:21:34.783340196 +0000 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-02 13:21:34.783340196 +0000 2024-04-02T13:26:58.677 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-02 13:21:34.783340196 +0000 2024-04-02T13:26:58.678 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:26:58.745 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-02T13:26:58.745 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-02T13:26:58.746 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000212334 s, 2.4 MB/s 2024-04-02T13:26:58.747 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:26:58.803 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-02T13:26:58.859 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 694 Links: 1 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-02 13:26:44.924252774 +0000 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-02 13:21:35.132334199 +0000 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-02 13:21:35.132334199 +0000 2024-04-02T13:26:58.860 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-02 13:21:35.132334199 +0000 2024-04-02T13:26:58.860 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:26:58.927 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-02T13:26:58.927 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-02T13:26:58.927 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000285349 s, 1.8 MB/s 2024-04-02T13:26:58.928 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:26:58.985 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 714 Links: 1 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-02 13:26:44.924252774 +0000 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-02 13:21:35.456328633 +0000 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-02 13:21:35.456328633 +0000 2024-04-02T13:26:59.040 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-04-02 13:21:35.456328633 +0000 2024-04-02T13:26:59.041 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:26:59.106 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-02T13:26:59.106 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-02T13:26:59.106 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000225076 s, 2.3 MB/s 2024-04-02T13:26:59.107 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:26:59.163 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-04-02T13:26:59.164 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-04-02T13:26:59.180 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-02T13:26:59.180 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-04-02T13:26:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:59 smithi187 ceph-mon[36064]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:26:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:59.236 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-02T13:26:59.236 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:59.236 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-04-02T13:26:59.236 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:59.237 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:59.237 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 13:26:35.662941745 +0000 2024-04-02T13:26:59.237 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 13:21:34.491646313 +0000 2024-04-02T13:26:59.237 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 13:21:34.491646313 +0000 2024-04-02T13:26:59.237 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 13:21:34.491646313 +0000 2024-04-02T13:26:59.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-02T13:26:59.304 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T13:26:59.304 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T13:26:59.305 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000207366 s, 2.5 MB/s 2024-04-02T13:26:59.306 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-02T13:26:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:59 smithi122 ceph-mon[32114]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:26:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:59.362 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-04-02T13:26:59.416 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 13:26:35.662941745 +0000 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 13:21:34.842639393 +0000 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 13:21:34.842639393 +0000 2024-04-02T13:26:59.417 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 13:21:34.842639393 +0000 2024-04-02T13:26:59.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-02T13:26:59.480 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:59 smithi188 ceph-mon[36117]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:26:59.480 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.480 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:26:59.480 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:26:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:26:59.482 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T13:26:59.482 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T13:26:59.482 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000200231 s, 2.6 MB/s 2024-04-02T13:26:59.484 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-02T13:26:59.540 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-04-02T13:26:59.595 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-02T13:26:59.595 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 13:26:35.663941726 +0000 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 13:21:35.185632630 +0000 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 13:21:35.185632630 +0000 2024-04-02T13:26:59.596 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 13:21:35.185632630 +0000 2024-04-02T13:26:59.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-02T13:26:59.661 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T13:26:59.661 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T13:26:59.661 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00017657 s, 2.9 MB/s 2024-04-02T13:26:59.663 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-02T13:26:59.718 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-04-02T13:26:59.773 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-02T13:26:59.773 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-02T13:26:59.773 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-04-02T13:26:59.773 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:26:59.773 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-04-02T13:26:59.774 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-04-02 13:26:35.663941726 +0000 2024-04-02T13:26:59.774 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-04-02 13:21:35.548625470 +0000 2024-04-02T13:26:59.774 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-04-02 13:21:35.548625470 +0000 2024-04-02T13:26:59.774 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-04-02 13:21:35.548625470 +0000 2024-04-02T13:26:59.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-02T13:26:59.840 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-04-02T13:26:59.841 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-04-02T13:26:59.841 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000222752 s, 2.3 MB/s 2024-04-02T13:26:59.842 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-02T13:26:59.897 INFO:tasks.cephadm:Deploying osd.0 on smithi122 with /dev/vg_nvme/lv_4... 2024-04-02T13:26:59.897 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:27:00.129 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.149 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:00 smithi188 ceph-mon[36117]: Deploying daemon mgr.z on smithi188 2024-04-02T13:27:00.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:00.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:00.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:00.228 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:00.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:00.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.229 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:00 smithi122 ceph-mon[32114]: Deploying daemon mgr.z on smithi188 2024-04-02T13:27:00.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:00 smithi187 ceph-mon[36064]: Deploying daemon mgr.z on smithi188 2024-04-02T13:27:01.271 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:01 smithi122 ceph-mon[32114]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:01.479 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:01 smithi188 ceph-mon[36117]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:01.479 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 systemd[1]: Starting Ceph mgr.z for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:27:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:01 smithi187 ceph-mon[36064]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:01.764 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 podman[37964]: 2024-04-02 13:27:01.476531792 +0000 UTC m=+0.106510796 container create 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:27:01.764 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 podman[37964]: 2024-04-02 13:27:01.387473622 +0000 UTC m=+0.017452646 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:27:01.764 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 podman[37964]: 2024-04-02 13:27:01.662037903 +0000 UTC m=+0.292016930 container init 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:27:01.765 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 podman[37964]: 2024-04-02 13:27:01.664682415 +0000 UTC m=+0.294661439 container start 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-02T13:27:01.765 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 ceph-mgr[37982]: -- 172.21.15.188:0/1758276917 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644f93445a0 con 0x5644f9318c00 2024-04-02T13:27:01.765 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 bash[37964]: 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 2024-04-02T13:27:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:27:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:27:01.910 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-02T13:27:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-02T13:27:01.910 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011027 s, 951 MB/s 2024-04-02T13:27:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-02T13:27:02.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:01 smithi188 systemd[1]: Started Ceph mgr.z for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:27:02.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-04-02T13:27:02.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:02.205+0000 7fc7afce9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:27:02.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:02.297+0000 7fc7afce9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:27:02.734 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:27:03.153 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: Deploying daemon mgr.y on smithi187 2024-04-02T13:27:03.154 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:02 smithi122 ceph-mon[32114]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:03.165 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:02.932+0000 7fc7afce9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:27:03.165 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.022+0000 7fc7afce9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:27:03.165 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.161+0000 7fc7afce9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: Deploying daemon mgr.y on smithi187 2024-04-02T13:27:03.166 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:02 smithi188 ceph-mon[36117]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-02T13:27:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: Deploying daemon mgr.y on smithi187 2024-04-02T13:27:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:02 smithi187 ceph-mon[36064]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:03.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.256+0000 7fc7afce9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:27:03.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.337+0000 7fc7afce9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:27:03.887 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.598+0000 7fc7afce9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:27:03.887 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.707+0000 7fc7afce9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:27:03.887 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.792+0000 7fc7afce9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:27:03.888 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:03.884+0000 7fc7afce9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:27:03.984 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:03 smithi187 systemd[1]: Starting Ceph mgr.y for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:27:04.180 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:04.057+0000 7fc7afce9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.214+0000 7f0c16784700 1 -- 172.21.15.122:0/3984874894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101030a0 msgr2=0x7f0c10105490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.214+0000 7f0c16784700 1 --2- 172.21.15.122:0/3984874894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101030a0 0x7f0c10105490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009990 tx=0x7f0c04009ca0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 -- 172.21.15.122:0/3984874894 shutdown_connections 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 --2- 172.21.15.122:0/3984874894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c10108300 0x7f0c1010a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 --2- 172.21.15.122:0/3984874894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101030a0 0x7f0c10105490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 --2- 172.21.15.122:0/3984874894 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c101059d0 0x7f0c10107dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 -- 172.21.15.122:0/3984874894 >> 172.21.15.122:0/3984874894 conn(0x7f0c100fcab0 msgr2=0x7f0c100feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:27:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 -- 172.21.15.122:0/3984874894 shutdown_connections 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 -- 172.21.15.122:0/3984874894 wait complete. 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 Processor -- start 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.215+0000 7f0c16784700 1 -- start start 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c101030a0 0x7f0c10102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 0x7f0c10101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 0x7f0c10101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e220 con 0x7f0c101059d0 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e0a0 con 0x7f0c10108300 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1010e3a0 con 0x7f0c101030a0 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 0x7f0c10101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 0x7f0c10101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 0x7f0c10101050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40812/0 (socket says 172.21.15.122:40812) 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0f7fe700 1 -- 172.21.15.122:0/306931896 learned_addr learned my addr 172.21.15.122:0/306931896 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:27:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0ffff700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c101030a0 0x7f0c10102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 -- 172.21.15.122:0/306931896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c101030a0 msgr2=0x7f0c10102a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c101030a0 0x7f0c10102a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 -- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 msgr2=0x7f0c10101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 0x7f0c10101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 -- 172.21.15.122:0/306931896 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c10101e40 con 0x7f0c10108300 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c14d21700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 0x7f0c10101590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800ef90 tx=0x7f0c0800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080092e0 con 0x7f0c10108300 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c08004bb0 con 0x7f0c10108300 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c101a2930 con 0x7f0c10108300 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.216+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08004280 con 0x7f0c10108300 2024-04-02T13:27:04.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.217+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c101a2e40 con 0x7f0c10108300 2024-04-02T13:27:04.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.217+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 13) ==== 46615+0+0 (secure 0 0 0) 0x7f0c0801b030 con 0x7f0c10108300 2024-04-02T13:27:04.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.218+0000 7f0c0d7fa700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 0x7f0bf803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:04.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.218+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0c0804c6f0 con 0x7f0c10108300 2024-04-02T13:27:04.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.218+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c100692b0 con 0x7f0c10108300 2024-04-02T13:27:04.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.218+0000 7f0c0ffff700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 0x7f0bf803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:04.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.218+0000 7f0c0ffff700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 0x7f0bf803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b2e0 tx=0x7f0c04005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:04.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.224+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0c08022020 con 0x7f0c10108300 2024-04-02T13:27:04.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:03 smithi187 podman[37431]: 2024-04-02 13:27:03.981363853 +0000 UTC m=+0.100162554 container create 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T13:27:04.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:03 smithi187 podman[37431]: 2024-04-02 13:27:03.898871517 +0000 UTC m=+0.017670224 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:27:04.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 podman[37431]: 2024-04-02 13:27:04.192321 +0000 UTC m=+0.311119709 container init 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T13:27:04.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 podman[37431]: 2024-04-02 13:27:04.194957535 +0000 UTC m=+0.313756234 container start 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True) 2024-04-02T13:27:04.236 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 ceph-mgr[37451]: -- 172.21.15.187:0/282184886 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577eeac05a0 con 0x5577eea95000 2024-04-02T13:27:04.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:04.440+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0c10062a30 con 0x7f0bf8039890 2024-04-02T13:27:04.735 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 bash[37431]: 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf 2024-04-02T13:27:04.735 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 systemd[1]: Started Ceph mgr.y for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:27:05.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:04.755+0000 7f19ea093200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:27:05.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:04.851+0000 7f19ea093200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:27:05.354 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:05.075+0000 7fc7afce9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:27:05.354 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:05.255+0000 7fc7afce9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:27:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:05.609 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:05.539+0000 7f19ea093200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:05.632+0000 7f19ea093200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='client.24110 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:05.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:05.906 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:05.605+0000 7fc7afce9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:27:05.906 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:05.695+0000 7fc7afce9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:27:05.968 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:05.779+0000 7f19ea093200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:27:05.968 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:05.879+0000 7f19ea093200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:27:05.968 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:05.965+0000 7f19ea093200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:27:06.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:05.902+0000 7fc7afce9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:27:06.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:06.089+0000 7fc7afce9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:27:06.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:06.230+0000 7f19ea093200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:27:06.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:06.338+0000 7f19ea093200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:27:06.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:06.422+0000 7f19ea093200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:27:06.634 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:06.373+0000 7fc7afce9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:27:06.635 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:06.631+0000 7fc7afce9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:27:06.836 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:06.515+0000 7f19ea093200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:27:06.836 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:06.689+0000 7f19ea093200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:27:07.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:06.871+0000 7fc7afce9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:27:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: Reconfiguring daemon mgr.x on smithi122 2024-04-02T13:27:07.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:06 smithi188 ceph-mon[36117]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:07.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: Reconfiguring daemon mgr.x on smithi122 2024-04-02T13:27:07.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:06 smithi187 ceph-mon[36064]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:07.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: Reconfiguring mgr.x (unknown last config time)... 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: Reconfiguring daemon mgr.x on smithi122 2024-04-02T13:27:07.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:06 smithi122 ceph-mon[32114]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]: dispatch 2024-04-02T13:27:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]': finished 2024-04-02T13:27:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-mon[36064]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:27:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2126873615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:07.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:07.655+0000 7f19ea093200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:27:07.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:07.784+0000 7f19ea093200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:07.665+0000 7fc7afce9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]: dispatch 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]': finished 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-mon[36117]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2126873615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:08.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]: dispatch 2024-04-02T13:27:08.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1701223259' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "94d678b8-fbb0-485b-b98a-133e7a6de516"}]': finished 2024-04-02T13:27:08.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:07 smithi122 ceph-mon[32114]: osdmap e5: 1 total, 0 up, 1 in 2024-04-02T13:27:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2126873615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:08.404 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:08.119+0000 7f19ea093200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:27:08.404 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:08.207+0000 7f19ea093200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:27:08.404 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:08.401+0000 7f19ea093200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:27:08.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:08.233+0000 7fc7afce9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:27:08.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:08.329+0000 7fc7afce9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:27:08.735 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:08.584+0000 7f19ea093200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:27:08.902 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:27:08.545+0000 7fc7afce9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:27:08.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:08.905+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mgrmap(e 14) ==== 92603+0+0 (secure 0 0 0) 0x7f0c08012050 con 0x7f0c10108300 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: Standby manager daemon z started 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:08.866+0000 7f19ea093200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:27:09.136 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:09.132+0000 7f19ea093200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: Standby manager daemon z started 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:27:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:08 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:09.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:09.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: Standby manager daemon z started 2024-04-02T13:27:09.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:27:09.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:09.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:27:09.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:08 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/1716698837' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:09.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:09.365+0000 7f19ea093200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:27:10.159 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:09 smithi187 ceph-mon[36064]: mgrmap e14: x(active, since 74s), standbys: z 2024-04-02T13:27:10.159 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:09 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:27:10.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:09 smithi188 ceph-mon[36117]: mgrmap e14: x(active, since 74s), standbys: z 2024-04-02T13:27:10.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:09 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:27:10.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:09 smithi122 ceph-mon[32114]: mgrmap e14: x(active, since 74s), standbys: z 2024-04-02T13:27:10.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:09 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:27:10.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:10.156+0000 7f19ea093200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:27:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:10 smithi187 ceph-mon[36064]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:10.985 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:10.723+0000 7f19ea093200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:27:10.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:10.820+0000 7f19ea093200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:27:11.002 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:10 smithi122 ceph-mon[32114]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:11.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:10 smithi188 ceph-mon[36117]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:11.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:27:11.034+0000 7f19ea093200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:27:11.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:11.919+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mon.2 v2:172.21.15.187:3300/0 8 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f0c08007500 con 0x7f0c10108300 2024-04-02T13:27:12.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:11 smithi122 ceph-mon[32114]: Standby manager daemon y started 2024-04-02T13:27:12.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:11 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:27:12.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:11 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:12.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:11 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:27:12.193 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:11 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-mon[36064]: Standby manager daemon y started 2024-04-02T13:27:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:27:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:27:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:11 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:11 smithi188 ceph-mon[36117]: Standby manager daemon y started 2024-04-02T13:27:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:11 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:27:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:11 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:27:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:11 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:27:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:11 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2466482688' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:12.956 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.030 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-04-02T13:27:13.030 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:27:13.030 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:13.030 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:13.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:13.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:13.031 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: mgrmap e15: x(active, since 77s), standbys: z, y 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:13.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:15 smithi187 ceph-mon[36064]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:27:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:15.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:15 smithi122 ceph-mon[32114]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:15.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:27:15.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:15 smithi188 ceph-mon[36117]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-02T13:27:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:16.718 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:16 smithi122 ceph-mon[32114]: Deploying daemon osd.0 on smithi122 2024-04-02T13:27:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:16 smithi187 ceph-mon[36064]: Deploying daemon osd.0 on smithi122 2024-04-02T13:27:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:16 smithi188 ceph-mon[36117]: Deploying daemon osd.0 on smithi122 2024-04-02T13:27:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:17 smithi187 ceph-mon[36064]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:17 smithi188 ceph-mon[36117]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:17.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:17 smithi122 ceph-mon[32114]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:19.540 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:19 smithi122 ceph-mon[32114]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:19 smithi187 ceph-mon[36064]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:19 smithi188 ceph-mon[36117]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:21.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:21.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:21 smithi122 ceph-mon[32114]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:21 smithi187 ceph-mon[36064]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:21 smithi188 ceph-mon[36117]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:23.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.653+0000 7f0c0d7fa700 1 -- 172.21.15.122:0/306931896 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0c10062a30 con 0x7f0bf8039890 2024-04-02T13:27:23.656 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 0 on host 'smithi122' 2024-04-02T13:27:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 msgr2=0x7f0bf803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 0x7f0bf803bd50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400b2e0 tx=0x7f0c04005fd0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 msgr2=0x7f0c10101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 0x7f0c10101590 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800ef90 tx=0x7f0c0800c3a0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 shutdown_connections 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c10108300 0x7f0c10101590 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c101030a0 0x7f0c10102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0bf8039890 0x7f0bf803bd50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 --2- 172.21.15.122:0/306931896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c101059d0 0x7f0c10101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.655+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 >> 172.21.15.122:0/306931896 conn(0x7f0c100fcab0 msgr2=0x7f0c10103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.656+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 shutdown_connections 2024-04-02T13:27:23.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:27:23.656+0000 7f0c16784700 1 -- 172.21.15.122:0/306931896 wait complete. 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:24.244 DEBUG:teuthology.orchestra.run.smithi122:osd.0> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.0.service 2024-04-02T13:27:24.246 INFO:tasks.cephadm:Deploying osd.1 on smithi187 with /dev/vg_nvme/lv_4... 2024-04-02T13:27:24.247 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:27:24.487 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:27:24.771 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:24.771 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:24.771 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:24.771 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:24 smithi122 ceph-mon[32114]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:27:24.772 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:24 smithi122 ceph-mon[32114]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:24.855 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:24.855 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:24.855 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:24.855 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:24 smithi187 ceph-mon[36064]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:27:24.855 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:24 smithi187 ceph-mon[36064]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:24 smithi188 ceph-mon[36117]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-02T13:27:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:24 smithi188 ceph-mon[36117]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:25.868 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:25 smithi187 ceph-mon[36064]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:27:25.868 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:25 smithi187 ceph-mon[36064]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:27:25.868 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:25 smithi187 ceph-mon[36064]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:27:25.868 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:25 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:25.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:25 smithi122 ceph-mon[32114]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:27:25.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:25 smithi122 ceph-mon[32114]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:27:25.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:25 smithi122 ceph-mon[32114]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:27:25.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:25 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:25 smithi188 ceph-mon[36117]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-02T13:27:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:25 smithi188 ceph-mon[36117]: osdmap e6: 1 total, 0 up, 1 in 2024-04-02T13:27:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:25 smithi188 ceph-mon[36117]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:27:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:25 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0193954 s, 541 MB/s 2024-04-02T13:27:26.446 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-02T13:27:26.623 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0[44413]: 2024-04-02T13:27:26.344+0000 7fd475ca3700 -1 osd.0 0 waiting for initial osdmap 2024-04-02T13:27:26.624 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0[44413]: 2024-04-02T13:27:26.348+0000 7fd46da8e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:27:26.767 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:27:26.767 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:27:26.767 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:26.768 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:26.768 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' 2024-04-02T13:27:26.768 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:26 smithi187 ceph-mon[36064]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:27:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:27:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:27.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:27.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' 2024-04-02T13:27:27.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:26 smithi188 ceph-mon[36117]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:27.053 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi187:vg_nvme/lv_4 2024-04-02T13:27:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:27:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: osdmap e7: 1 total, 0 up, 1 in 2024-04-02T13:27:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: from='osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597]' entity='osd.0' 2024-04-02T13:27:27.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:26 smithi122 ceph-mon[32114]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-02T13:27:27.289 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:27:27.721 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:27:27.721 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:27:27.721 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:27.721 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi122 2024-04-02T13:27:27.721 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:27.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:27.723 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] boot 2024-04-02T13:27:27.723 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:27:27.723 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:27:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:27:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi122 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] boot 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:27:28.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi122 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 17088M 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: osd.0 [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] boot 2024-04-02T13:27:28.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: osdmap e8: 1 total, 1 up, 1 in 2024-04-02T13:27:28.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:27:28.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 -- 172.21.15.187:0/1638278549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 msgr2=0x7febdc103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 --2- 172.21.15.187:0/1638278549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc103ca0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7febcc0099c0 tx=0x7febcc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 -- 172.21.15.187:0/1638278549 shutdown_connections 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 --2- 172.21.15.187:0/1638278549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc1041e0 0x7febdc10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 --2- 172.21.15.187:0/1638278549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc102640 0x7febdc102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 --2- 172.21.15.187:0/1638278549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc103ca0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 -- 172.21.15.187:0/1638278549 >> 172.21.15.187:0/1638278549 conn(0x7febdc0fdc00 msgr2=0x7febdc100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 -- 172.21.15.187:0/1638278549 shutdown_connections 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 -- 172.21.15.187:0/1638278549 wait complete. 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.835+0000 7febe2ed6700 1 Processor -- start 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 -- start start 2024-04-02T13:27:28.840 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc102640 0x7febdc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc1041e0 0x7febdc19fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10e210 con 0x7febdc1041e0 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10e090 con 0x7febdc103840 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febdc10e390 con 0x7febdc102640 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46256/0 (socket says 172.21.15.187:46256) 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 -- 172.21.15.187:0/2584723749 learned_addr learned my addr 172.21.15.187:0/2584723749 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe0c72700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc102640 0x7febdc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe1473700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc1041e0 0x7febdc19fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 -- 172.21.15.187:0/2584723749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc102640 msgr2=0x7febdc198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc102640 0x7febdc198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 -- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc1041e0 msgr2=0x7febdc19fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:28.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc1041e0 0x7febdc19fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 -- 172.21.15.187:0/2584723749 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc1a0470 con 0x7febdc103840 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd3fff700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7febcc005910 tx=0x7febcc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc027070 con 0x7febdc103840 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febdc1a0700 con 0x7febdc103840 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febcc004150 con 0x7febdc103840 2024-04-02T13:27:28.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.836+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febcc022da0 con 0x7febdc103840 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.837+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febdc1a0bb0 con 0x7febdc103840 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.838+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febdc0692b0 con 0x7febdc103840 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.839+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7febcc02a070 con 0x7febdc103840 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.839+0000 7febd1ffb700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.840+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7febcc0cf6a0 con 0x7febdc103840 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.840+0000 7febe0c72700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:28.843 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.840+0000 7febe0c72700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 0x7febc40a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febd8009550 tx=0x7febd80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:28.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:28.844+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7febcc05ef20 con 0x7febdc103840 2024-04-02T13:27:28.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:28 smithi187 ceph-mon[36064]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:29.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:28 smithi188 ceph-mon[36117]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:29.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:29.051+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7febdc062af0 con 0x7febc40a3bb0 2024-04-02T13:27:29.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:28 smithi122 ceph-mon[32114]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:29.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:29 smithi187 ceph-mon[36064]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:27:29.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:29 smithi187 ceph-mon[36064]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:29.938 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:29.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:29.939 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:29 smithi188 ceph-mon[36117]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:27:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:29 smithi188 ceph-mon[36117]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:29 smithi122 ceph-mon[32114]: osdmap e9: 1 total, 1 up, 1 in 2024-04-02T13:27:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:29 smithi122 ceph-mon[32114]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:30.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:30 smithi187 ceph-mon[36064]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:30 smithi188 ceph-mon[36117]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:30 smithi122 ceph-mon[32114]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:31.948 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:31 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/2250545964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:31.948 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:31 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:31.948 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:31 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]': finished 2024-04-02T13:27:31.948 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:31 smithi187 ceph-mon[36064]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:27:31.948 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:31 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/2250545964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:31 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:31 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]': finished 2024-04-02T13:27:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:31 smithi188 ceph-mon[36117]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:27:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:31 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/2250545964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:31 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]: dispatch 2024-04-02T13:27:32.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:31 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "349c8fa1-e818-4c5e-862d-e30ff9fa14bc"}]': finished 2024-04-02T13:27:32.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:31 smithi122 ceph-mon[32114]: osdmap e10: 2 total, 1 up, 2 in 2024-04-02T13:27:32.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/3928080729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:32 smithi187 ceph-mon[36064]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/3928080729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:32 smithi188 ceph-mon[36117]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/3928080729' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:32 smithi122 ceph-mon[32114]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:35 smithi187 ceph-mon[36064]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:35.720 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:35 smithi188 ceph-mon[36117]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:35 smithi122 ceph-mon[32114]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:37 smithi187 ceph-mon[36064]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:37 smithi188 ceph-mon[36117]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:37 smithi122 ceph-mon[32114]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:39.666 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:39 smithi187 ceph-mon[36064]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:39.666 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:27:39.666 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:39 smithi188 ceph-mon[36117]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:27:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:39 smithi122 ceph-mon[32114]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-02T13:27:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:40 smithi187 ceph-mon[36064]: Deploying daemon osd.1 on smithi187 2024-04-02T13:27:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:40 smithi188 ceph-mon[36117]: Deploying daemon osd.1 on smithi187 2024-04-02T13:27:40.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:40 smithi122 ceph-mon[32114]: Deploying daemon osd.1 on smithi187 2024-04-02T13:27:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:41 smithi187 ceph-mon[36064]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:41 smithi188 ceph-mon[36117]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:41 smithi122 ceph-mon[32114]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:43.718 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:43 smithi187 ceph-mon[36064]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:43 smithi188 ceph-mon[36117]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:43 smithi122 ceph-mon[32114]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:45.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:45.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:44 smithi187 ceph-mon[36064]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:45.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:45.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:44 smithi188 ceph-mon[36117]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:45.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:45.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:45.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:44 smithi122 ceph-mon[32114]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:47.436 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.436 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.436 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:47.436 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:47.437 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.437 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:47 smithi187 ceph-mon[36064]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:47.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.515+0000 7febd1ffb700 1 -- 172.21.15.187:0/2584723749 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7febdc062af0 con 0x7febc40a3bb0 2024-04-02T13:27:47.518 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 1 on host 'smithi187' 2024-04-02T13:27:47.520 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 msgr2=0x7febc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:47.520 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 0x7febc40a6070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febd8009550 tx=0x7febd80091b0 comp rx=0 tx=0).stop 2024-04-02T13:27:47.520 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 msgr2=0x7febdc198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:47.520 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7febcc005910 tx=0x7febcc009580 comp rx=0 tx=0).stop 2024-04-02T13:27:47.520 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 shutdown_connections 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febc40a3bb0 0x7febc40a6070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7febdc1041e0 0x7febdc19fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc102640 0x7febdc198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 --2- 172.21.15.187:0/2584723749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7febdc103840 0x7febdc198d50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 >> 172.21.15.187:0/2584723749 conn(0x7febdc0fdc00 msgr2=0x7febdc106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 shutdown_connections 2024-04-02T13:27:47.521 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:27:47.517+0000 7febe2ed6700 1 -- 172.21.15.187:0/2584723749 wait complete. 2024-04-02T13:27:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:47 smithi188 ceph-mon[36117]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:47 smithi122 ceph-mon[32114]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:48.107 DEBUG:teuthology.orchestra.run.smithi187:osd.1> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.1.service 2024-04-02T13:27:48.109 INFO:tasks.cephadm:Deploying osd.2 on smithi188 with /dev/vg_nvme/lv_4... 2024-04-02T13:27:48.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-02T13:27:48.347 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:27:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:48 smithi187 ceph-mon[36064]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:48 smithi187 ceph-mon[36064]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:48.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:48.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:48 smithi188 ceph-mon[36117]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:48.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:48 smithi188 ceph-mon[36117]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:27:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:48 smithi122 ceph-mon[32114]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:48 smithi122 ceph-mon[32114]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:49.553 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-mon[36064]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:49.553 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1[40671]: 2024-04-02T13:27:49.491+0000 7f68b934b700 -1 osd.1 0 waiting for initial osdmap 2024-04-02T13:27:49.554 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:27:49 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1[40671]: 2024-04-02T13:27:49.496+0000 7f68b513e700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:49.682 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:49 smithi188 ceph-mon[36117]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-02T13:27:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: osdmap e11: 2 total, 1 up, 2 in 2024-04-02T13:27:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: from='osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:49 smithi122 ceph-mon[32114]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:27:50.245 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-02T13:27:50.246 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-02T13:27:50.246 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-02T13:27:50.246 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-02T13:27:50.246 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112698 s, 930 MB/s 2024-04-02T13:27:50.246 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-02T13:27:50.581 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:50 smithi188 ceph-mon[36117]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:27:50.581 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:50 smithi188 ceph-mon[36117]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:27:50.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:50 smithi187 ceph-mon[36064]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:27:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:50 smithi187 ceph-mon[36064]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:27:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:50 smithi122 ceph-mon[32114]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:27:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:50 smithi122 ceph-mon[32114]: osdmap e12: 2 total, 1 up, 2 in 2024-04-02T13:27:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:50.843 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi188:vg_nvme/lv_4 2024-04-02T13:27:51.083 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:27:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:27:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:27:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] boot 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] boot 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: osd.1 [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] boot 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: osdmap e13: 2 total, 2 up, 2 in 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:27:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 -- 172.21.15.188:0/2312805183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e301030c0 msgr2=0x7f0e30109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 --2- 172.21.15.188:0/2312805183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e301030c0 0x7f0e30109970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800b0a0 tx=0x7f0e2800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 -- 172.21.15.188:0/2312805183 shutdown_connections 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 --2- 172.21.15.188:0/2312805183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e301030c0 0x7f0e30109970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 --2- 172.21.15.188:0/2312805183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e30101520 0x7f0e30101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 --2- 172.21.15.188:0/2312805183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 0x7f0e30102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 -- 172.21.15.188:0/2312805183 >> 172.21.15.188:0/2312805183 conn(0x7f0e300fcac0 msgr2=0x7f0e300fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 -- 172.21.15.188:0/2312805183 shutdown_connections 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.498+0000 7f0e3538b700 1 -- 172.21.15.188:0/2312805183 wait complete. 2024-04-02T13:27:52.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 Processor -- start 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 -- start start 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 0x7f0e30198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 0x7f0e3019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010d0f0 con 0x7f0e301030c0 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010cf70 con 0x7f0e30102720 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e3538b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e3010d270 con 0x7f0e30101520 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e2effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e2effd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:57792/0 (socket says 172.21.15.188:57792) 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.499+0000 7f0e2effd700 1 -- 172.21.15.188:0/1973547276 learned_addr learned my addr 172.21.15.188:0/1973547276 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2e7fc700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 0x7f0e30198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2f7fe700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 0x7f0e3019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 -- 172.21.15.188:0/1973547276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 msgr2=0x7f0e30198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:52.505 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 0x7f0e30198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 -- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 msgr2=0x7f0e3019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 0x7f0e3019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 -- 172.21.15.188:0/1973547276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e301a03d0 con 0x7f0e30101520 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2f7fe700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 0x7f0e3019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e2effd700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c9c0 tx=0x7f0e2400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007ac0 con 0x7f0e30101520 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e301a06c0 con 0x7f0e30101520 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e24007c60 con 0x7f0e30101520 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e24018660 con 0x7f0e30101520 2024-04-02T13:27:52.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.500+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e301a0c00 con 0x7f0e30101520 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.502+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e300692b0 con 0x7f0e30101520 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.502+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7f0e24018800 con 0x7f0e30101520 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.503+0000 7f0e17fff700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.503+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f0e240ceb60 con 0x7f0e30101520 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.503+0000 7f0e2e7fc700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:27:52.507 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.503+0000 7f0e2e7fc700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 0x7f0e180a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007be0 tx=0x7f0e200073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:27:52.512 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.510+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0e24014030 con 0x7f0e30101520 2024-04-02T13:27:52.718 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:27:52.715+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f0e301a0ec0 con 0x7f0e180a3bb0 2024-04-02T13:27:52.719 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:52 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi187 2024-04-02T13:27:52.719 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:52 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 17080M 2024-04-02T13:27:52.719 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:52 smithi188 ceph-mon[36117]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:27:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:52 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi187 2024-04-02T13:27:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:52 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 17080M 2024-04-02T13:27:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:52 smithi122 ceph-mon[32114]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:27:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:52 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi187 2024-04-02T13:27:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:52 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 17080M 2024-04-02T13:27:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:52 smithi187 ceph-mon[36064]: osdmap e14: 2 total, 2 up, 2 in 2024-04-02T13:27:53.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:53 smithi188 ceph-mon[36117]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:53.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:53.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:53.576 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:53 smithi122 ceph-mon[32114]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:53 smithi187 ceph-mon[36064]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:27:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:27:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:27:54.658 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:54 smithi188 ceph-mon[36117]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:54.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:54 smithi187 ceph-mon[36064]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:54.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:54 smithi122 ceph-mon[32114]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/3380797208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]': finished 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:27:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:27:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/3380797208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]': finished 2024-04-02T13:27:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:27:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:27:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/3380797208' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]: dispatch 2024-04-02T13:27:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e9bc5736-0e61-468b-b9a8-f988e2d36bf9"}]': finished 2024-04-02T13:27:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: osdmap e15: 3 total, 2 up, 3 in 2024-04-02T13:27:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:27:56.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/549389367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/549389367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/549389367' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:27:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:57 smithi122 ceph-mon[32114]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:57 smithi187 ceph-mon[36064]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:57 smithi188 ceph-mon[36117]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:27:59 smithi122 ceph-mon[32114]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:27:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:27:59 smithi187 ceph-mon[36064]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:27:59 smithi188 ceph-mon[36117]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:01.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:01 smithi188 ceph-mon[36117]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:01 smithi122 ceph-mon[32114]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:01 smithi187 ceph-mon[36064]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:28:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:02.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:28:02.842 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-02T13:28:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:03.729 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:03 smithi188 ceph-mon[36117]: Deploying daemon osd.2 on smithi188 2024-04-02T13:28:03.729 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:03 smithi188 ceph-mon[36117]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:03 smithi122 ceph-mon[32114]: Deploying daemon osd.2 on smithi188 2024-04-02T13:28:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:03 smithi122 ceph-mon[32114]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:03 smithi187 ceph-mon[36064]: Deploying daemon osd.2 on smithi188 2024-04-02T13:28:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:03 smithi187 ceph-mon[36064]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:05 smithi188 ceph-mon[36117]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:05 smithi122 ceph-mon[32114]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:05 smithi187 ceph-mon[36064]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:07 smithi122 ceph-mon[32114]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:07.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:07 smithi188 ceph-mon[36117]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:07.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:07 smithi187 ceph-mon[36064]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:09.702 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:09 smithi188 ceph-mon[36117]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:09 smithi122 ceph-mon[32114]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:09 smithi187 ceph-mon[36064]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:10.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.030+0000 7f0e17fff700 1 -- 172.21.15.188:0/1973547276 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0e301a0ec0 con 0x7f0e180a3bb0 2024-04-02T13:28:10.033 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 2 on host 'smithi188' 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 msgr2=0x7f0e180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 0x7f0e180a6070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e20007be0 tx=0x7f0e200073d0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 msgr2=0x7f0e30198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400c9c0 tx=0x7f0e2400ccd0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 shutdown_connections 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0e180a3bb0 0x7f0e180a6070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e301030c0 0x7f0e3019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e30101520 0x7f0e30198770 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 --2- 172.21.15.188:0/1973547276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e30102720 0x7f0e30198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.032+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 >> 172.21.15.188:0/1973547276 conn(0x7f0e300fcac0 msgr2=0x7f0e300fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:10.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.033+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 shutdown_connections 2024-04-02T13:28:10.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:28:10.033+0000 7f0e3538b700 1 -- 172.21.15.188:0/1973547276 wait complete. 2024-04-02T13:28:10.568 DEBUG:teuthology.orchestra.run.smithi188:osd.2> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.2.service 2024-04-02T13:28:10.570 INFO:tasks.cephadm:Deploying osd.3 on smithi122 with /dev/vg_nvme/lv_3... 2024-04-02T13:28:10.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:28:10.819 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.195 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:10 smithi122 ceph-mon[32114]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:10 smithi187 ceph-mon[36064]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:11.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-mon[36117]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-02T13:28:11.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:28:10.808+0000 7f6580ba47c0 -1 osd.2 0 log_to_monitors true 2024-04-02T13:28:12.770 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:28:12.683+0000 7f6576209700 -1 osd.2 0 waiting for initial osdmap 2024-04-02T13:28:12.770 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:28:12.689+0000 7f6571ffc700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:12 smithi188 ceph-mon[36117]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:28:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:28:12.855 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-02T13:28:12.855 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-02T13:28:12.855 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110545 s, 949 MB/s 2024-04-02T13:28:12.856 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.856 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:12 smithi122 ceph-mon[32114]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-02T13:28:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: osdmap e16: 3 total, 2 up, 3 in 2024-04-02T13:28:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: from='osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:28:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:12 smithi187 ceph-mon[36064]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-02T13:28:13.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-04-02T13:28:13.780 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:28:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:13 smithi122 ceph-mon[32114]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:28:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:13 smithi122 ceph-mon[32114]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:28:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:13 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:13 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:13.939 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:13 smithi188 ceph-mon[36117]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:28:13.939 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:13 smithi188 ceph-mon[36117]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:28:13.939 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:13 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:13.939 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:13 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:13 smithi187 ceph-mon[36064]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:28:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:13 smithi187 ceph-mon[36064]: osdmap e17: 3 total, 2 up, 3 in 2024-04-02T13:28:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:13 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:13 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] boot 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi188 2024-04-02T13:28:14.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 17080M 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.963 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:14.964 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] boot 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi188 2024-04-02T13:28:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 17080M 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:14.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:14.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:14.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:14.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: osd.2 [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] boot 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: osdmap e18: 3 total, 3 up, 3 in 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi188 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 17080M 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:15.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1722077131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f4103840 msgr2=0x7fc6f4103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1722077131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f4103840 0x7fc6f4103ca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80099c0 tx=0x7fc6e8009cd0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1722077131 shutdown_connections 2024-04-02T13:28:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1722077131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f41041e0 0x7fc6f410aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1722077131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 0x7fc6f4102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1722077131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f4103840 0x7fc6f4103ca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1722077131 >> 172.21.15.122:0/1722077131 conn(0x7fc6f40fdc00 msgr2=0x7fc6f4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1722077131 shutdown_connections 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.355+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1722077131 wait complete. 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 Processor -- start 2024-04-02T13:28:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 -- start start 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 0x7fc6f4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f4103840 0x7fc6f4198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f410e260 con 0x7fc6f41041e0 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f410e0e0 con 0x7fc6f4103840 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f9e11700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f410e3e0 con 0x7fc6f4102640 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47518/0 (socket says 172.21.15.122:47518) 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f3fff700 1 -- 172.21.15.122:0/1890476056 learned_addr learned my addr 172.21.15.122:0/1890476056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f37fe700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 0x7fc6f4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.356+0000 7fc6f2ffd700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f4103840 0x7fc6f4198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 -- 172.21.15.122:0/1890476056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 msgr2=0x7fc6f4198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 0x7fc6f4198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 -- 172.21.15.122:0/1890476056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f4103840 msgr2=0x7fc6f4198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f4103840 0x7fc6f4198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 -- 172.21.15.122:0/1890476056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f41a04d0 con 0x7fc6f41041e0 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f3fff700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007f10 tx=0x7fc6ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:15.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec004bb0 con 0x7fc6f41041e0 2024-04-02T13:28:15.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41a07c0 con 0x7fc6f41041e0 2024-04-02T13:28:15.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec004d50 con 0x7fc6f41041e0 2024-04-02T13:28:15.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec0059f0 con 0x7fc6f41041e0 2024-04-02T13:28:15.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.357+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41a0d00 con 0x7fc6f41041e0 2024-04-02T13:28:15.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.358+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f40692b0 con 0x7fc6f41041e0 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.359+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 138591+0+0 (secure 0 0 0) 0x7fc6ec01a020 con 0x7fc6f41041e0 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.360+0000 7fc6f0ff9700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 0x7fc6dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.360+0000 7fc6f37fe700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 0x7fc6dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.360+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7fc6ec0cecf0 con 0x7fc6f41041e0 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.360+0000 7fc6f37fe700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 0x7fc6dc0a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6f41036a0 tx=0x7fc6e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:15.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.364+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc6ec010070 con 0x7fc6f41041e0 2024-04-02T13:28:15.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:15.577+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc6f4062a30 con 0x7fc6dc0a3bb0 2024-04-02T13:28:15.834 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:15.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:28:15.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:15.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:15.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:15.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:15.835 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:15.836 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:15.836 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49011]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:15.836 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49011]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:15.947 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:15.947 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:15.947 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45216]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:15.947 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45216]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:15.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:15.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:28:15.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:15.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:15.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:15.948 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:15.985 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44840]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:15.985 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44840]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44840]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44840]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44843]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:15.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:28:15 smithi187 sudo[44843]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: osdmap e19: 3 total, 3 up, 3 in 2024-04-02T13:28:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:28:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:15.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:16.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49014]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:16.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49014]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49014]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.220 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49014]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.220 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49017]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49017]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:15 smithi122 sudo[49017]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 sudo[49017]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 sudo[49020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 sudo[49020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 sudo[49020]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.221 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 sudo[49020]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.267 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:16.267 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.267 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45219]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.267 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:28:15 smithi188 sudo[45219]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45222]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45222]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45222]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45222]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45225]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 sudo[45225]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-02T13:28:16.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44846]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44846]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44849]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-02T13:28:16.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 sudo[44849]: pam_unix(sudo:session): session closed for user root 2024-04-02T13:28:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.317 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.318 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.319 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:16 smithi122 ceph-mon[32114]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.320 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:16 smithi188 ceph-mon[36117]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:17.321 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: osdmap e20: 3 total, 3 up, 3 in 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.322 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:28:17.323 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-02T13:28:17.324 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:16 smithi187 ceph-mon[36064]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:16.987+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc6ec098b00 con 0x7fc6f41041e0 2024-04-02T13:28:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:17 smithi187 ceph-mon[36064]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:28:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:17 smithi187 ceph-mon[36064]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:28:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:17 smithi188 ceph-mon[36117]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:28:18.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:17 smithi188 ceph-mon[36117]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:28:18.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:17 smithi122 ceph-mon[32114]: osdmap e21: 3 total, 3 up, 3 in 2024-04-02T13:28:18.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:17 smithi122 ceph-mon[32114]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-02T13:28:18.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:18 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]: dispatch 2024-04-02T13:28:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:18 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]': finished 2024-04-02T13:28:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:18 smithi187 ceph-mon[36064]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:28:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:18 smithi187 ceph-mon[36064]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:18 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]: dispatch 2024-04-02T13:28:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:18 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]': finished 2024-04-02T13:28:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:18 smithi188 ceph-mon[36117]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:28:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:18 smithi188 ceph-mon[36117]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:19.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:18 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]: dispatch 2024-04-02T13:28:19.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:18 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1217181518' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7608c20-4f88-434c-99ae-225f08577c91"}]': finished 2024-04-02T13:28:19.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:18 smithi122 ceph-mon[32114]: osdmap e22: 4 total, 3 up, 4 in 2024-04-02T13:28:19.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:19.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:18 smithi122 ceph-mon[32114]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2033083254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2033083254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:20.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2033083254' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:20.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:20 smithi187 ceph-mon[36064]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:21.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:20 smithi188 ceph-mon[36117]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:21.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:20 smithi122 ceph-mon[32114]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:23.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:23 smithi122 ceph-mon[32114]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:23.722 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:23 smithi187 ceph-mon[36064]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:23 smithi188 ceph-mon[36117]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:25.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:25 smithi122 ceph-mon[32114]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:25 smithi187 ceph-mon[36064]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:25 smithi188 ceph-mon[36117]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:26.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:28:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:28:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-02T13:28:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:27 smithi187 ceph-mon[36064]: Deploying daemon osd.3 on smithi122 2024-04-02T13:28:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:27 smithi187 ceph-mon[36064]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:27.743 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:27 smithi122 ceph-mon[32114]: Deploying daemon osd.3 on smithi122 2024-04-02T13:28:27.743 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:27 smithi122 ceph-mon[32114]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:27 smithi188 ceph-mon[36117]: Deploying daemon osd.3 on smithi122 2024-04-02T13:28:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:27 smithi188 ceph-mon[36117]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:29 smithi187 ceph-mon[36064]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:29.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:29 smithi188 ceph-mon[36117]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:29 smithi122 ceph-mon[32114]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:31 smithi187 ceph-mon[36064]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:31 smithi188 ceph-mon[36117]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:31.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:31 smithi122 ceph-mon[32114]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:31.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:31.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:33 smithi187 ceph-mon[36064]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:33.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:33 smithi122 ceph-mon[32114]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:33 smithi188 ceph-mon[36117]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:34.808 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 3 on host 'smithi122' 2024-04-02T13:28:34.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.804+0000 7fc6f0ff9700 1 -- 172.21.15.122:0/1890476056 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc6f4062a30 con 0x7fc6dc0a3bb0 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 msgr2=0x7fc6dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 0x7fc6dc0a6070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6f41036a0 tx=0x7fc6e400b410 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 msgr2=0x7fc6f419fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec007f10 tx=0x7fc6ec00d3b0 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 shutdown_connections 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6dc0a3bb0 0x7fc6dc0a6070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6f41041e0 0x7fc6f419fe30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6f4102640 0x7fc6f4198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 --2- 172.21.15.122:0/1890476056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6f4103840 0x7fc6f4198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:34.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 >> 172.21.15.122:0/1890476056 conn(0x7fc6f40fdc00 msgr2=0x7fc6f4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:34.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 shutdown_connections 2024-04-02T13:28:34.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:28:34.806+0000 7fc6f9e11700 1 -- 172.21.15.122:0/1890476056 wait complete. 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.053 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:35.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:35.451 DEBUG:teuthology.orchestra.run.smithi122:osd.3> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.3.service 2024-04-02T13:28:35.454 INFO:tasks.cephadm:Deploying osd.4 on smithi187 with /dev/vg_nvme/lv_3... 2024-04-02T13:28:35.454 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:28:35.700 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:28:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:35 smithi122 ceph-mon[32114]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:28:36.119 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:35 smithi187 ceph-mon[36064]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:28:36.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:35 smithi188 ceph-mon[36117]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-02T13:28:37.089 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:36 smithi187 ceph-mon[36064]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:28:37.089 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:36 smithi187 ceph-mon[36064]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:28:37.089 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:36 smithi187 ceph-mon[36064]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:28:37.089 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:37.089 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:36 smithi187 ceph-mon[36064]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:36 smithi188 ceph-mon[36117]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:28:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:36 smithi188 ceph-mon[36117]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:28:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:36 smithi188 ceph-mon[36117]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:28:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:36 smithi188 ceph-mon[36117]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:36 smithi122 ceph-mon[32114]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-02T13:28:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:36 smithi122 ceph-mon[32114]: osdmap e23: 4 total, 3 up, 4 in 2024-04-02T13:28:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:36 smithi122 ceph-mon[32114]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:28:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:36 smithi122 ceph-mon[32114]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-02T13:28:37.641 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:28:37.641 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:28:37.641 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-02T13:28:37.641 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-02T13:28:37.641 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128358 s, 817 MB/s 2024-04-02T13:28:37.642 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-02T13:28:37.942 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:28:37.684+0000 7f597e1bd700 -1 osd.3 0 waiting for initial osdmap 2024-04-02T13:28:37.942 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:28:37.689+0000 7f5975fa8700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:28:38.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] boot 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:28:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:28:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:28:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' 2024-04-02T13:28:38.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] boot 2024-04-02T13:28:38.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:28:38.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.297 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi187:vg_nvme/lv_3 2024-04-02T13:28:38.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: osdmap e24: 4 total, 3 up, 4 in 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: from='osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026]' entity='osd.3' 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: osd.3 [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] boot 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: osdmap e25: 4 total, 4 up, 4 in 2024-04-02T13:28:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:28:38.545 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:28:39.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:28:39.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi122 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:39.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:28:39.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:28:39.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi122 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:39.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi122 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 8544M 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:39.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:28:39.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.016+0000 7fe618347700 1 -- 172.21.15.187:0/4123573934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 msgr2=0x7fe61010a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.016+0000 7fe618347700 1 --2- 172.21.15.187:0/4123573934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61010a770 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe60800b0a0 tx=0x7fe60800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- 172.21.15.187:0/4123573934 shutdown_connections 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- 172.21.15.187:0/4123573934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61010a770 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- 172.21.15.187:0/4123573934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 0x7fe6101054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- 172.21.15.187:0/4123573934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 0x7fe610107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- 172.21.15.187:0/4123573934 >> 172.21.15.187:0/4123573934 conn(0x7fe6100fcab0 msgr2=0x7fe6100feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:40.022 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- 172.21.15.187:0/4123573934 shutdown_connections 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- 172.21.15.187:0/4123573934 wait complete. 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 Processor -- start 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- start start 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 0x7fe610198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 0x7fe610198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe61010e2c0 con 0x7fe610103100 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe61010e140 con 0x7fe610108360 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe618347700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe61010e440 con 0x7fe610105a30 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe6168e4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.017+0000 7fe6168e4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:56950/0 (socket says 172.21.15.187:56950) 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 -- 172.21.15.187:0/3615185940 learned_addr learned my addr 172.21.15.187:0/3615185940 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-02T13:28:40.023 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6160e3700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 0x7fe610198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6158e2700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 0x7fe610198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 -- 172.21.15.187:0/3615185940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 msgr2=0x7fe610198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 0x7fe610198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 -- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 msgr2=0x7fe610198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 0x7fe610198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 -- 172.21.15.187:0/3615185940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6101a04f0 con 0x7fe610108360 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6168e4700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe608007f00 tx=0x7fe608004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608021070 con 0x7fe610108360 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60801d930 con 0x7fe610108360 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6101a0780 con 0x7fe610108360 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608005640 con 0x7fe610108360 2024-04-02T13:28:40.024 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.018+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6101a0d00 con 0x7fe610108360 2024-04-02T13:28:40.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.019+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6100692b0 con 0x7fe610108360 2024-04-02T13:28:40.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.020+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe608024070 con 0x7fe610108360 2024-04-02T13:28:40.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.021+0000 7fe6077fe700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 0x7fe5fc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:28:40.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.021+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fe60800f020 con 0x7fe610108360 2024-04-02T13:28:40.025 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.021+0000 7fe6160e3700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 0x7fe5fc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:28:40.026 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.022+0000 7fe6160e3700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 0x7fe5fc0a6180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006fd0 tx=0x7fe60c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:28:40.028 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.025+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe60802b020 con 0x7fe610108360 2024-04-02T13:28:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:39 smithi187 ceph-mon[36064]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:28:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:39 smithi187 ceph-mon[36064]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:28:40.236 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:40.232+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fe610062a30 con 0x7fe5fc0a3cc0 2024-04-02T13:28:40.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:39 smithi188 ceph-mon[36117]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:28:40.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:39 smithi188 ceph-mon[36117]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:28:40.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:39 smithi122 ceph-mon[32114]: osdmap e26: 4 total, 4 up, 4 in 2024-04-02T13:28:40.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:39 smithi122 ceph-mon[32114]: osdmap e27: 4 total, 4 up, 4 in 2024-04-02T13:28:41.143 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:40 smithi187 ceph-mon[36064]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:41.143 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:41.143 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:41.143 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:41.143 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:40 smithi187 ceph-mon[36064]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:40 smithi188 ceph-mon[36117]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:40 smithi188 ceph-mon[36117]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:41.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:40 smithi122 ceph-mon[32114]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:28:41.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:28:41.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:28:41.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:41.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:40 smithi122 ceph-mon[32114]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:42.233 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:41 smithi187 ceph-mon[36064]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:28:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:41 smithi188 ceph-mon[36117]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:28:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:41 smithi122 ceph-mon[32114]: osdmap e28: 4 total, 4 up, 4 in 2024-04-02T13:28:43.239 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:28:43.239 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:28:43.239 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/857294031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.239 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.240 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]': finished 2024-04-02T13:28:43.240 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:28:43.240 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:42 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/857294031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]': finished 2024-04-02T13:28:43.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:28:43.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:42 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:28:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: osdmap e29: 4 total, 4 up, 4 in 2024-04-02T13:28:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 288 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:28:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/857294031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]: dispatch 2024-04-02T13:28:43.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "038779c8-ba3f-4165-a72d-61c31b3c3cbd"}]': finished 2024-04-02T13:28:43.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: osdmap e30: 5 total, 4 up, 5 in 2024-04-02T13:28:43.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:42 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:28:44.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:43 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/1853781940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:44.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:43 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/1853781940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:44.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:43 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/1853781940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:28:45.222 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:44 smithi122 ceph-mon[32114]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-02T13:28:45.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:44 smithi187 ceph-mon[36064]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-02T13:28:45.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:44 smithi188 ceph-mon[36117]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-04-02T13:28:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:47 smithi187 ceph-mon[36064]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:28:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:47 smithi188 ceph-mon[36117]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:28:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:47 smithi122 ceph-mon[32114]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-02T13:28:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:49 smithi187 ceph-mon[36064]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-02T13:28:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:49 smithi188 ceph-mon[36117]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-02T13:28:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:49 smithi122 ceph-mon[32114]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-02T13:28:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:28:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:28:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-02T13:28:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:28:51.592 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:51 smithi187 ceph-mon[36064]: Deploying daemon osd.4 on smithi187 2024-04-02T13:28:51.592 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:51 smithi187 ceph-mon[36064]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:51 smithi188 ceph-mon[36117]: Deploying daemon osd.4 on smithi187 2024-04-02T13:28:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:51 smithi188 ceph-mon[36117]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:51 smithi122 ceph-mon[32114]: Deploying daemon osd.4 on smithi187 2024-04-02T13:28:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:51 smithi122 ceph-mon[32114]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:53.687 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:53 smithi187 ceph-mon[36064]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:53 smithi188 ceph-mon[36117]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:53 smithi122 ceph-mon[32114]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:55 smithi187 ceph-mon[36064]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:55 smithi188 ceph-mon[36117]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:55 smithi122 ceph-mon[32114]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:56.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:56.930 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:56.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:56 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:56.931 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:56 smithi122 ceph-mon[32114]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:56 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:56 smithi187 ceph-mon[36064]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:56 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:28:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:56 smithi188 ceph-mon[36117]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:59.300 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.296+0000 7fe6077fe700 1 -- 172.21.15.187:0/3615185940 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe610062a30 con 0x7fe5fc0a3cc0 2024-04-02T13:28:59.300 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 4 on host 'smithi187' 2024-04-02T13:28:59.302 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 msgr2=0x7fe5fc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:59.302 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 0x7fe5fc0a6180 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe60c006fd0 tx=0x7fe60c008040 comp rx=0 tx=0).stop 2024-04-02T13:28:59.302 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 msgr2=0x7fe61019fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:28:59.302 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe608007f00 tx=0x7fe608004ef0 comp rx=0 tx=0).stop 2024-04-02T13:28:59.302 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 shutdown_connections 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe610108360 0x7fe61019fe50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe5fc0a3cc0 0x7fe5fc0a6180 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe610103100 0x7fe610198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 --2- 172.21.15.187:0/3615185940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe610105a30 0x7fe610198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 >> 172.21.15.187:0/3615185940 conn(0x7fe6100fcab0 msgr2=0x7fe610103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 shutdown_connections 2024-04-02T13:28:59.303 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:28:59.299+0000 7fe618347700 1 -- 172.21.15.187:0/3615185940 wait complete. 2024-04-02T13:28:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:59 smithi187 ceph-mon[36064]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:28:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:59 smithi188 ceph-mon[36117]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:28:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:59 smithi122 ceph-mon[32114]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:28:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:28:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:28:59.952 DEBUG:teuthology.orchestra.run.smithi187:osd.4> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.4.service 2024-04-02T13:28:59.955 INFO:tasks.cephadm:Deploying osd.5 on smithi188 with /dev/vg_nvme/lv_3... 2024-04-02T13:28:59.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-02T13:29:00.200 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:29:00.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:00 smithi188 ceph-mon[36117]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:00.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:00 smithi188 ceph-mon[36117]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:00 smithi187 ceph-mon[36064]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:00 smithi187 ceph-mon[36064]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:00 smithi122 ceph-mon[32114]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:00 smithi122 ceph-mon[32114]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-02T13:29:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:29:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:29:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-mon[36064]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:01.736 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:29:01.477+0000 7fa549976700 -1 osd.4 0 waiting for initial osdmap 2024-04-02T13:29:01.736 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:29:01 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:29:01.484+0000 7fa545769700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:29:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:29:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:29:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:01 smithi188 ceph-mon[36117]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-02T13:29:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: osdmap e31: 5 total, 4 up, 5 in 2024-04-02T13:29:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: from='osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:01 smithi122 ceph-mon[32114]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112333 s, 933 MB/s 2024-04-02T13:29:02.021 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-02T13:29:02.623 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:02 smithi187 ceph-mon[36064]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:29:02.623 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:02 smithi187 ceph-mon[36064]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:29:02.623 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.623 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:02 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:02 smithi188 ceph-mon[36117]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:29:02.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:02 smithi188 ceph-mon[36117]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:29:02.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.639 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:02 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.666 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi188:vg_nvme/lv_3 2024-04-02T13:29:02.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:02 smithi122 ceph-mon[32114]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:29:02.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:02 smithi122 ceph-mon[32114]: osdmap e32: 5 total, 4 up, 5 in 2024-04-02T13:29:02.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:02 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:02.905 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:29:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] boot 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:03.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:03 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:29:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:29:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] boot 2024-04-02T13:29:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:03 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: osd.4 [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] boot 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: osdmap e33: 5 total, 5 up, 5 in 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:03.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:03 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 -- 172.21.15.188:0/1832614050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c103840 msgr2=0x7f553c103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 --2- 172.21.15.188:0/1832614050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c103840 0x7f553c103ca0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5530009990 tx=0x7f5530009ca0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 -- 172.21.15.188:0/1832614050 shutdown_connections 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 --2- 172.21.15.188:0/1832614050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c1041e0 0x7f553c10aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 --2- 172.21.15.188:0/1832614050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c102640 0x7f553c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 --2- 172.21.15.188:0/1832614050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c103840 0x7f553c103ca0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.500 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 -- 172.21.15.188:0/1832614050 >> 172.21.15.188:0/1832614050 conn(0x7f553c0fdba0 msgr2=0x7f553c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 -- 172.21.15.188:0/1832614050 shutdown_connections 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.494+0000 7f55423b9700 1 -- 172.21.15.188:0/1832614050 wait complete. 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 Processor -- start 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 -- start start 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c102640 0x7f553c076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c103840 0x7f553c0749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10e230 con 0x7f553c102640 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10e0b0 con 0x7f553c103840 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f55423b9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f553c10e3b0 con 0x7f553c1041e0 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f5540956700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f5540956700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:45982/0 (socket says 172.21.15.188:45982) 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f5540956700 1 -- 172.21.15.188:0/3001615622 learned_addr learned my addr 172.21.15.188:0/3001615622 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f553bfff700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c102640 0x7f553c076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.495+0000 7f553b7fe700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c103840 0x7f553c0749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 -- 172.21.15.188:0/3001615622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c103840 msgr2=0x7f553c0749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c103840 0x7f553c0749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 -- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c102640 msgr2=0x7f553c076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c102640 0x7f553c076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 -- 172.21.15.188:0/3001615622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0757d0 con 0x7f553c1041e0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f5540956700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f553400ef30 tx=0x7f553400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340092e0 con 0x7f553c1041e0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5534004bb0 con 0x7f553c1041e0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c1a2970 con 0x7f553c1041e0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534004280 con 0x7f553c1041e0 2024-04-02T13:29:04.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.496+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a2e80 con 0x7f553c1041e0 2024-04-02T13:29:04.503 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.497+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f553c02dd30 con 0x7f553c1041e0 2024-04-02T13:29:04.503 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.498+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f553401b030 con 0x7f553c1041e0 2024-04-02T13:29:04.503 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.499+0000 7f55397fa700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 0x7f55240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:04.503 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.499+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f55340cedb0 con 0x7f553c1041e0 2024-04-02T13:29:04.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.499+0000 7f553bfff700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 0x7f55240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:04.504 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.499+0000 7f553bfff700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 0x7f55240a6180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f553c1036a0 tx=0x7f552c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:04.506 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.503+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5534022020 con 0x7f553c1041e0 2024-04-02T13:29:04.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:04 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi187 2024-04-02T13:29:04.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:04 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 8540M 2024-04-02T13:29:04.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:04 smithi188 ceph-mon[36117]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:29:04.714 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:04.711+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f553c062a30 con 0x7f55240a3cc0 2024-04-02T13:29:04.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:04 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi187 2024-04-02T13:29:04.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:04 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 8540M 2024-04-02T13:29:04.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:04 smithi122 ceph-mon[32114]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:29:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:04 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi187 2024-04-02T13:29:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:04 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 8540M 2024-04-02T13:29:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:04 smithi187 ceph-mon[36064]: osdmap e34: 5 total, 5 up, 5 in 2024-04-02T13:29:05.654 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:05 smithi188 ceph-mon[36117]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:05.654 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:05.654 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:05.654 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:05 smithi122 ceph-mon[32114]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:05 smithi187 ceph-mon[36064]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:06.619 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:06 smithi188 ceph-mon[36117]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:06.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:06 smithi122 ceph-mon[32114]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:06.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:06 smithi187 ceph-mon[36064]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/639627586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]': finished 2024-04-02T13:29:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:29:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/639627586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]': finished 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:29:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/639627586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]: dispatch 2024-04-02T13:29:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "266e5146-9b00-4cbe-8ced-85833fa9e959"}]': finished 2024-04-02T13:29:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: osdmap e35: 6 total, 5 up, 6 in 2024-04-02T13:29:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/778053913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:08.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/778053913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:08.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/778053913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:09 smithi188 ceph-mon[36117]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:09 smithi122 ceph-mon[32114]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:09 smithi187 ceph-mon[36064]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:11.706 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:11 smithi188 ceph-mon[36117]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:11 smithi122 ceph-mon[32114]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:11 smithi187 ceph-mon[36064]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:13.571 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:13 smithi188 ceph-mon[36117]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:13.788 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:13 smithi187 ceph-mon[36064]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:13 smithi122 ceph-mon[32114]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:29:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:14.800 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:29:14.801 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:14.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-02T13:29:14.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:15.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:15 smithi188 ceph-mon[36117]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:15.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:15 smithi188 ceph-mon[36117]: Deploying daemon osd.5 on smithi188 2024-04-02T13:29:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:15 smithi122 ceph-mon[32114]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:15 smithi122 ceph-mon[32114]: Deploying daemon osd.5 on smithi188 2024-04-02T13:29:15.969 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:15 smithi187 ceph-mon[36064]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:15.969 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:15 smithi187 ceph-mon[36064]: Deploying daemon osd.5 on smithi188 2024-04-02T13:29:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:17 smithi122 ceph-mon[32114]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:17.937 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:17 smithi188 ceph-mon[36117]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:17 smithi187 ceph-mon[36064]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:19 smithi122 ceph-mon[32114]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:19 smithi187 ceph-mon[36064]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:20.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:19 smithi188 ceph-mon[36117]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:20 smithi122 ceph-mon[32114]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:21.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:20 smithi187 ceph-mon[36064]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:21.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:21.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:20 smithi188 ceph-mon[36117]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:23.439 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:23.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:23.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.440 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:23 smithi188 ceph-mon[36117]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:23.529 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.524+0000 7f55397fa700 1 -- 172.21.15.188:0/3001615622 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f553c062a30 con 0x7f55240a3cc0 2024-04-02T13:29:23.529 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 5 on host 'smithi188' 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 msgr2=0x7f55240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 0x7f55240a6180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f553c1036a0 tx=0x7f552c009260 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 msgr2=0x7f553c074f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f553400ef30 tx=0x7f553400c3a0 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 shutdown_connections 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f553c1041e0 0x7f553c074f20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f55240a3cc0 0x7f55240a6180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f553c102640 0x7f553c076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 --2- 172.21.15.188:0/3001615622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f553c103840 0x7f553c0749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 >> 172.21.15.188:0/3001615622 conn(0x7f553c0fdba0 msgr2=0x7f553c106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 shutdown_connections 2024-04-02T13:29:23.531 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:29:23.527+0000 7f55423b9700 1 -- 172.21.15.188:0/3001615622 wait complete. 2024-04-02T13:29:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:23 smithi187 ceph-mon[36064]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:23 smithi122 ceph-mon[32114]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:24.111 DEBUG:teuthology.orchestra.run.smithi188:osd.5> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.5.service 2024-04-02T13:29:24.114 INFO:tasks.cephadm:Deploying osd.6 on smithi122 with /dev/vg_nvme/lv_2... 2024-04-02T13:29:24.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:29:24.363 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:24.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:24.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:24 smithi187 ceph-mon[36064]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:24.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:24 smithi187 ceph-mon[36064]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:24.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:24.759 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:24 smithi122 ceph-mon[32114]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:24.759 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:24 smithi122 ceph-mon[32114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:24.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:24.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:24.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:24 smithi188 ceph-mon[36117]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:24.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:24 smithi188 ceph-mon[36117]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-02T13:29:25.594 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:25.595 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:29:25.595 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:25.595 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:29:25.595 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:25.595 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:25 smithi188 ceph-mon[36117]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:29:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:29:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:25 smithi187 ceph-mon[36064]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: from='osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: osdmap e36: 6 total, 5 up, 6 in 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:25.785 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:25 smithi122 ceph-mon[32114]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110863 s, 946 MB/s 2024-04-02T13:29:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-02T13:29:26.488 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:29:26.200+0000 7f5e62d0c700 -1 osd.5 0 waiting for initial osdmap 2024-04-02T13:29:26.489 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:29:26.208+0000 7f5e5a2f6700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:29:26.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:29:26.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:29:26.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:29:26.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:26 smithi122 ceph-mon[32114]: from='osd.5 ' entity='osd.5' 2024-04-02T13:29:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:26 smithi188 ceph-mon[36117]: from='osd.5 ' entity='osd.5' 2024-04-02T13:29:26.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: osdmap e37: 6 total, 5 up, 6 in 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: osdmap e38: 6 total, 5 up, 6 in 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:26.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:26 smithi187 ceph-mon[36064]: from='osd.5 ' entity='osd.5' 2024-04-02T13:29:27.107 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-04-02T13:29:27.359 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] boot 2024-04-02T13:29:27.607 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:27.608 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] boot 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:27.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: osd.5 [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] boot 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: osdmap e39: 6 total, 6 up, 6 in 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:27.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:27.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:27.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:27.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:28.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:28 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi188 2024-04-02T13:29:28.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:28 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 8540M 2024-04-02T13:29:28.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:28 smithi122 ceph-mon[32114]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:29:28.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:28 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi188 2024-04-02T13:29:28.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:28 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 8540M 2024-04-02T13:29:28.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:28 smithi188 ceph-mon[36117]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:29:28.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:28 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi188 2024-04-02T13:29:28.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:28 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 8540M 2024-04-02T13:29:28.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:28 smithi187 ceph-mon[36064]: osdmap e40: 6 total, 6 up, 6 in 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 -- 172.21.15.122:0/1056385525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c1010b0 msgr2=0x7f6b1c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 --2- 172.21.15.122:0/1056385525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c10ac60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400b0d0 tx=0x7f6b1400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 -- 172.21.15.122:0/1056385525 shutdown_connections 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 --2- 172.21.15.122:0/1056385525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c10ac60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 --2- 172.21.15.122:0/1056385525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c103c70 0x7f6b1c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 --2- 172.21.15.122:0/1056385525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 -- 172.21.15.122:0/1056385525 >> 172.21.15.122:0/1056385525 conn(0x7f6b1c0fc290 msgr2=0x7f6b1c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:29.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 -- 172.21.15.122:0/1056385525 shutdown_connections 2024-04-02T13:29:29.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.062+0000 7f6b23c46700 1 -- 172.21.15.122:0/1056385525 wait complete. 2024-04-02T13:29:29.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 Processor -- start 2024-04-02T13:29:29.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 -- start start 2024-04-02T13:29:29.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c19af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c102cb0 con 0x7f6b1c103c70 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c102b30 con 0x7f6b1c1010b0 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b23c46700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c102e30 con 0x7f6b1c1006f0 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b221e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:29.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b221e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52230/0 (socket says 172.21.15.122:52230) 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b221e3700 1 -- 172.21.15.122:0/3009311406 learned_addr learned my addr 172.21.15.122:0/3009311406 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.063+0000 7f6b211e1700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c19af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b219e2700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 -- 172.21.15.122:0/3009311406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 msgr2=0x7f6b1c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 -- 172.21.15.122:0/3009311406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 msgr2=0x7f6b1c19af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c19af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 -- 172.21.15.122:0/3009311406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1a2670 con 0x7f6b1c103c70 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b211e1700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:29:29.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b219e2700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b221e3700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006070 tx=0x7f6b14007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14003c00 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14003da0 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c1a2900 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14005570 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.064+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a2e60 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.065+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c0692b0 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.066+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f6b14005710 con 0x7f6b1c103c70 2024-04-02T13:29:29.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.067+0000 7f6b12ffd700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 0x7f6b080a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:29.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.067+0000 7f6b219e2700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 0x7f6b080a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:29.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.067+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f6b1400f020 con 0x7f6b1c103c70 2024-04-02T13:29:29.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.067+0000 7f6b219e2700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 0x7f6b080a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06a9d0 tx=0x7f6b0c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:29.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.070+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6b1402b020 con 0x7f6b1c103c70 2024-04-02T13:29:29.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:29.280+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6b1c062a30 con 0x7f6b080a3c40 2024-04-02T13:29:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:29 smithi122 ceph-mon[32114]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:29 smithi122 ceph-mon[32114]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:29:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:29 smithi188 ceph-mon[36117]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:29 smithi188 ceph-mon[36117]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:29:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:29 smithi187 ceph-mon[36064]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:29 smithi187 ceph-mon[36064]: osdmap e41: 6 total, 6 up, 6 in 2024-04-02T13:29:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:30 smithi188 ceph-mon[36117]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:30 smithi188 ceph-mon[36117]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:29:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:30 smithi122 ceph-mon[32114]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:30 smithi122 ceph-mon[32114]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:29:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:30 smithi187 ceph-mon[36064]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:30.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:30 smithi187 ceph-mon[36064]: osdmap e42: 6 total, 6 up, 6 in 2024-04-02T13:29:31.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:31 smithi122 ceph-mon[32114]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:31.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:31 smithi122 ceph-mon[32114]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:29:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:31 smithi188 ceph-mon[36117]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:31 smithi188 ceph-mon[36117]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:29:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:31 smithi187 ceph-mon[36064]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:31.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:31 smithi187 ceph-mon[36064]: osdmap e43: 6 total, 6 up, 6 in 2024-04-02T13:29:32.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]: dispatch 2024-04-02T13:29:32.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]': finished 2024-04-02T13:29:32.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:32 smithi188 ceph-mon[36117]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:29:32.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]: dispatch 2024-04-02T13:29:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]': finished 2024-04-02T13:29:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:32 smithi122 ceph-mon[32114]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:29:32.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]: dispatch 2024-04-02T13:29:32.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/238777059' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2961ced1-b269-44f8-ad13-dc4435028869"}]': finished 2024-04-02T13:29:32.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:32 smithi187 ceph-mon[36064]: osdmap e44: 7 total, 6 up, 7 in 2024-04-02T13:29:32.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:33 smithi188 ceph-mon[36117]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:29:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2097736122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:33 smithi122 ceph-mon[32114]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:29:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2097736122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:33 smithi187 ceph-mon[36064]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:29:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2097736122' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:35 smithi188 ceph-mon[36117]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-02T13:29:35.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:35 smithi122 ceph-mon[32114]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-02T13:29:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:35 smithi187 ceph-mon[36064]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-02T13:29:37.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:37 smithi188 ceph-mon[36117]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:29:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:37 smithi122 ceph-mon[32114]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:29:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:37 smithi187 ceph-mon[36064]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:29:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:39 smithi188 ceph-mon[36117]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:29:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:29:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:39 smithi122 ceph-mon[32114]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:29:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:29:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:39.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:39 smithi187 ceph-mon[36064]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:29:39.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-02T13:29:39.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:40.763 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:40 smithi122 ceph-mon[32114]: Deploying daemon osd.6 on smithi122 2024-04-02T13:29:40.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:40 smithi187 ceph-mon[36064]: Deploying daemon osd.6 on smithi122 2024-04-02T13:29:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:40 smithi188 ceph-mon[36117]: Deploying daemon osd.6 on smithi122 2024-04-02T13:29:41.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:41 smithi122 ceph-mon[32114]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:29:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:41 smithi187 ceph-mon[36064]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:29:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:41 smithi188 ceph-mon[36117]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:29:43.788 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:43 smithi187 ceph-mon[36064]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:43 smithi122 ceph-mon[32114]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:43 smithi188 ceph-mon[36117]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:45.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:45 smithi122 ceph-mon[32114]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:45.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:45.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:45.709 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:45 smithi187 ceph-mon[36064]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:45 smithi188 ceph-mon[36117]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:47.755 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:47 smithi122 ceph-mon[32114]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:47 smithi187 ceph-mon[36064]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:47 smithi188 ceph-mon[36117]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:48.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.817+0000 7f6b12ffd700 1 -- 172.21.15.122:0/3009311406 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b1c062a30 con 0x7f6b080a3c40 2024-04-02T13:29:48.821 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 6 on host 'smithi122' 2024-04-02T13:29:48.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.819+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 msgr2=0x7f6b080a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.819+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 0x7f6b080a6100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06a9d0 tx=0x7f6b0c008040 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.819+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 msgr2=0x7f6b1c1a1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.819+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6b14006070 tx=0x7f6b14007e40 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 shutdown_connections 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f6b080a3c40 0x7f6b080a6100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b1c103c70 0x7f6b1c1a1fd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c1006f0 0x7f6b1c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 --2- 172.21.15.122:0/3009311406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6b1c1010b0 0x7f6b1c19af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 >> 172.21.15.122:0/3009311406 conn(0x7f6b1c0fc290 msgr2=0x7f6b1c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 shutdown_connections 2024-04-02T13:29:48.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:29:48.820+0000 7f6b23c46700 1 -- 172.21.15.122:0/3009311406 wait complete. 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:49.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:49.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:48 smithi122 ceph-mon[32114]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:29:49.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:49.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:48 smithi187 ceph-mon[36064]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:29:49.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:48 smithi188 ceph-mon[36117]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-02T13:29:49.543 DEBUG:teuthology.orchestra.run.smithi122:osd.6> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.6.service 2024-04-02T13:29:49.545 INFO:tasks.cephadm:Deploying osd.7 on smithi187 with /dev/vg_nvme/lv_2... 2024-04-02T13:29:49.546 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:29:49.793 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:29:51.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-mon[32114]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:29:51.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-mon[32114]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:29:51.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-mon[32114]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:29:51.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:51.040 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-mon[32114]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:51.041 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:29:50.809+0000 7f71104d6700 -1 osd.6 0 waiting for initial osdmap 2024-04-02T13:29:51.041 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:29:50 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:29:50.818+0000 7f710c2c9700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:29:51.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:50 smithi187 ceph-mon[36064]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:29:51.181 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:50 smithi187 ceph-mon[36064]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:29:51.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:50 smithi187 ceph-mon[36064]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:29:51.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:50 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:51.182 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:50 smithi187 ceph-mon[36064]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:51.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:50 smithi188 ceph-mon[36117]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-02T13:29:51.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:50 smithi188 ceph-mon[36117]: osdmap e45: 7 total, 6 up, 7 in 2024-04-02T13:29:51.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:50 smithi188 ceph-mon[36117]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:29:51.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:50 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:51.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:50 smithi188 ceph-mon[36117]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-02T13:29:51.857 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:29:51.857 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:29:51.858 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-02T13:29:51.858 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-02T13:29:51.858 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128652 s, 815 MB/s 2024-04-02T13:29:51.858 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-02T13:29:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:51 smithi122 ceph-mon[32114]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:29:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:51 smithi122 ceph-mon[32114]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:29:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.167 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:51 smithi187 ceph-mon[36064]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:29:52.167 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:51 smithi187 ceph-mon[36064]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:29:52.167 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.167 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:51 smithi188 ceph-mon[36117]: from='osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:29:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:51 smithi188 ceph-mon[36117]: osdmap e46: 7 total, 6 up, 7 in 2024-04-02T13:29:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:52.479 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi187:vg_nvme/lv_2 2024-04-02T13:29:52.715 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] boot 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:53.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:53.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:52 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.079 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:29:53.079 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:29:53.079 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] boot 2024-04-02T13:29:53.079 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:29:53.079 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:53.080 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:52 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: osd.6 [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] boot 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: osdmap e47: 7 total, 7 up, 7 in 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:29:53.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:52 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:29:54.134 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:53 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi122 2024-04-02T13:29:54.134 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:53 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-02T13:29:54.134 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:53 smithi187 ceph-mon[36064]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:29:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:53 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi122 2024-04-02T13:29:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:53 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-02T13:29:54.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:53 smithi188 ceph-mon[36117]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:29:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:53 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi122 2024-04-02T13:29:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:53 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 5696M 2024-04-02T13:29:54.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:53 smithi122 ceph-mon[32114]: osdmap e48: 7 total, 7 up, 7 in 2024-04-02T13:29:54.459 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 -- 172.21.15.187:0/279977724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364102720 msgr2=0x7f2364102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 --2- 172.21.15.187:0/279977724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364102720 0x7f2364102b80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f23540099c0 tx=0x7f2354009cd0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 -- 172.21.15.187:0/279977724 shutdown_connections 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 --2- 172.21.15.187:0/279977724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23641030c0 0x7f2364109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 --2- 172.21.15.187:0/279977724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2364101520 0x7f2364101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 --2- 172.21.15.187:0/279977724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364102720 0x7f2364102b80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 -- 172.21.15.187:0/279977724 >> 172.21.15.187:0/279977724 conn(0x7f23640fcac0 msgr2=0x7f23640fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 -- 172.21.15.187:0/279977724 shutdown_connections 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.454+0000 7f236a9b8700 1 -- 172.21.15.187:0/279977724 wait complete. 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 Processor -- start 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 -- start start 2024-04-02T13:29:54.460 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2364102720 0x7f2364197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23641030c0 0x7f236419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f236410d0f0 con 0x7f2364102720 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f236410cf70 con 0x7f2364101520 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f236a9b8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f236410d270 con 0x7f23641030c0 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f2363fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f2363fff700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:54344/0 (socket says 172.21.15.187:54344) 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f2363fff700 1 -- 172.21.15.187:0/3949740932 learned_addr learned my addr 172.21.15.187:0/3949740932 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.455+0000 7f23637fe700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2364102720 0x7f2364197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2368f55700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23641030c0 0x7f236419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 -- 172.21.15.187:0/3949740932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23641030c0 msgr2=0x7f236419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23641030c0 0x7f236419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 -- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2364102720 msgr2=0x7f2364197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:29:54.461 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2364102720 0x7f2364197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 -- 172.21.15.187:0/3949740932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236419f3c0 con 0x7f2364101520 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f2363fff700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f235800c970 tx=0x7f235800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358004220 con 0x7f2364101520 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236419f6b0 con 0x7f2364101520 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2358007d80 con 0x7f2364101520 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2358003df0 con 0x7f2364101520 2024-04-02T13:29:54.462 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.456+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236419fbf0 con 0x7f2364101520 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.457+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23640692b0 con 0x7f2364101520 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.459+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f23580078a0 con 0x7f2364101520 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.459+0000 7f23617fa700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.459+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f23580cf610 con 0x7f2364101520 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.460+0000 7f23637fe700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:29:54.463 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.460+0000 7f23637fe700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 0x7f234c0a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:29:54.466 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.463+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2358014030 con 0x7f2364101520 2024-04-02T13:29:54.672 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:29:54.669+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2364062a30 con 0x7f234c0a3c40 2024-04-02T13:29:55.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:54 smithi187 ceph-mon[36064]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:55.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:55.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:55.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:55.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:54 smithi188 ceph-mon[36117]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:55.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:55.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:55.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:54 smithi122 ceph-mon[32114]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:55.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:29:55.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:29:55.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:29:56.201 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:55 smithi187 ceph-mon[36064]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:56.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:55 smithi188 ceph-mon[36117]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:56.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:55 smithi122 ceph-mon[32114]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:29:57.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:56 smithi187 ceph-mon[36064]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:57.317 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:56 smithi188 ceph-mon[36117]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:57.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:56 smithi122 ceph-mon[32114]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:58.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:57 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/463704959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:57 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:57 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]': finished 2024-04-02T13:29:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:57 smithi187 ceph-mon[36064]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:29:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:57 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:29:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:57 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/463704959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:57 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:57 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]': finished 2024-04-02T13:29:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:57 smithi188 ceph-mon[36117]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:29:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:57 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:29:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:57 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/463704959' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:57 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]: dispatch 2024-04-02T13:29:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:57 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fd83ac7-db42-40ec-aee5-c69f77239ce9"}]': finished 2024-04-02T13:29:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:57 smithi122 ceph-mon[32114]: osdmap e49: 8 total, 7 up, 8 in 2024-04-02T13:29:58.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:29:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:58 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/1599016903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:59.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:29:58 smithi187 ceph-mon[36064]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:58 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/1599016903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:29:58 smithi188 ceph-mon[36117]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:29:59.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:58 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/1599016903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:29:59.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:29:58 smithi122 ceph-mon[32114]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:00.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:00 smithi122 ceph-mon[32114]: overall HEALTH_OK 2024-04-02T13:30:00.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:00 smithi187 ceph-mon[36064]: overall HEALTH_OK 2024-04-02T13:30:00.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:00 smithi188 ceph-mon[36117]: overall HEALTH_OK 2024-04-02T13:30:01.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:01 smithi122 ceph-mon[32114]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:01.376 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:01 smithi187 ceph-mon[36064]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:01.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:01 smithi188 ceph-mon[36117]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:03.610 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:03 smithi187 ceph-mon[36064]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:03.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:03 smithi188 ceph-mon[36117]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:03 smithi122 ceph-mon[32114]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:05.594 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:05 smithi187 ceph-mon[36064]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:05.594 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:30:05.594 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:05 smithi188 ceph-mon[36117]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:30:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:05 smithi122 ceph-mon[32114]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-02T13:30:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:06.650 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:06 smithi187 ceph-mon[36064]: Deploying daemon osd.7 on smithi187 2024-04-02T13:30:06.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:06 smithi188 ceph-mon[36117]: Deploying daemon osd.7 on smithi187 2024-04-02T13:30:06.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:06 smithi122 ceph-mon[32114]: Deploying daemon osd.7 on smithi187 2024-04-02T13:30:07.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:07 smithi188 ceph-mon[36117]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:07 smithi122 ceph-mon[32114]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:07.836 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:07 smithi187 ceph-mon[36064]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:09 smithi187 ceph-mon[36064]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:09 smithi188 ceph-mon[36117]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:09 smithi122 ceph-mon[32114]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:11.639 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:11.640 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:11 smithi187 ceph-mon[36064]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:11 smithi188 ceph-mon[36117]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:11 smithi122 ceph-mon[32114]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:13.387 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 7 on host 'smithi187' 2024-04-02T13:30:13.388 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.384+0000 7f23617fa700 1 -- 172.21.15.187:0/3949740932 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2364062a30 con 0x7f234c0a3c40 2024-04-02T13:30:13.389 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 msgr2=0x7f234c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 0x7f234c0a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2354000c00 tx=0x7f2354005fb0 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 msgr2=0x7f2364197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f235800c970 tx=0x7f235800cc80 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 shutdown_connections 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23641030c0 0x7f236419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2364101520 0x7f2364197760 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f234c0a3c40 0x7f234c0a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 --2- 172.21.15.187:0/3949740932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2364102720 0x7f2364197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 >> 172.21.15.187:0/3949740932 conn(0x7f23640fcac0 msgr2=0x7f2364105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 shutdown_connections 2024-04-02T13:30:13.390 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:30:13.386+0000 7f236a9b8700 1 -- 172.21.15.187:0/3949740932 wait complete. 2024-04-02T13:30:13.730 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:13 smithi187 ceph-mon[36064]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:13.730 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:13 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:13.731 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:13 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:13 smithi188 ceph-mon[36117]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:13 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:13 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:13 smithi122 ceph-mon[32114]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:13 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:13 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:14.494 DEBUG:teuthology.orchestra.run.smithi187:osd.7> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.7.service 2024-04-02T13:30:14.497 INFO:tasks.cephadm:Deploying osd.8 on smithi188 with /dev/vg_nvme/lv_2... 2024-04-02T13:30:14.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-02T13:30:14.708 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:14 smithi187 ceph-mon[36064]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:14.709 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:14 smithi187 ceph-mon[36064]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:14.741 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:30:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:14 smithi188 ceph-mon[36117]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:14 smithi188 ceph-mon[36117]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:14.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:14 smithi122 ceph-mon[32114]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:14.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:14 smithi122 ceph-mon[32114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:15.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:30:15.492+0000 7fc548cee700 -1 osd.7 0 waiting for initial osdmap 2024-04-02T13:30:15.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:30:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:30:15.504+0000 7fc544ae1700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:30:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: osdmap e50: 8 total, 7 up, 8 in 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01728 s, 607 MB/s 2024-04-02T13:30:16.739 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] boot 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:30:16.759 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:16 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:30:16.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:30:16.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] boot 2024-04-02T13:30:16.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:30:16.762 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:16 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: osdmap e51: 8 total, 7 up, 8 in 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: osd.7 [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] boot 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: osdmap e52: 8 total, 8 up, 8 in 2024-04-02T13:30:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:16 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:30:17.437 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi188:vg_nvme/lv_2 2024-04-02T13:30:17.684 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:30:17.720 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:17 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:30:17.720 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:17 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:30:17.720 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:17 smithi187 ceph-mon[36064]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:17.720 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:17 smithi187 ceph-mon[36064]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:30:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:17 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:30:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:17 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:30:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:17 smithi188 ceph-mon[36117]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:17 smithi188 ceph-mon[36117]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:30:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:17 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:30:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:17 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:30:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:17 smithi122 ceph-mon[32114]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-04-02T13:30:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:17 smithi122 ceph-mon[32114]: osdmap e53: 8 total, 8 up, 8 in 2024-04-02T13:30:19.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi187 2024-04-02T13:30:19.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.180 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 5693M 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.181 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:19 smithi188 ceph-mon[36117]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:19.283 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- 172.21.15.188:0/512106443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc640102730 msgr2=0x7fc640102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- 172.21.15.188:0/512106443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc640102730 0x7fc640102b90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009990 tx=0x7fc62c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- 172.21.15.188:0/512106443 shutdown_connections 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- 172.21.15.188:0/512106443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc6401030d0 0x7fc640109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- 172.21.15.188:0/512106443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640101530 0x7fc640101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- 172.21.15.188:0/512106443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc640102730 0x7fc640102b90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- 172.21.15.188:0/512106443 >> 172.21.15.188:0/512106443 conn(0x7fc6400fcaf0 msgr2=0x7fc6400fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- 172.21.15.188:0/512106443 shutdown_connections 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- 172.21.15.188:0/512106443 wait complete. 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 Processor -- start 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 -- start start 2024-04-02T13:30:19.284 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640101530 0x7fc640197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.278+0000 7fc646bcd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 0x7fc640197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc646bcd700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc646bcd700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010d120 con 0x7fc640102730 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc646bcd700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010cfa0 con 0x7fc640101530 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc646bcd700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc64010d2a0 con 0x7fc6401030d0 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:46598/0 (socket says 172.21.15.188:46598) 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 -- 172.21.15.188:0/130394183 learned_addr learned my addr 172.21.15.188:0/130394183 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc644969700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640101530 0x7fc640197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc63ffff700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 0x7fc640197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 -- 172.21.15.188:0/130394183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640101530 msgr2=0x7fc640197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640101530 0x7fc640197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.285 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 -- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 msgr2=0x7fc640197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 0x7fc640197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 -- 172.21.15.188:0/130394183 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64019f370 con 0x7fc6401030d0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc63ffff700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 0x7fc640197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc64516a700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc638007bd0 tx=0x7fc638007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638016030 con 0x7fc6401030d0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638012460 con 0x7fc6401030d0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64006adb0 con 0x7fc6401030d0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.279+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638005280 con 0x7fc6401030d0 2024-04-02T13:30:19.286 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.280+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64006b2f0 con 0x7fc6401030d0 2024-04-02T13:30:19.287 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.280+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6400692b0 con 0x7fc6401030d0 2024-04-02T13:30:19.287 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.281+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc638012610 con 0x7fc6401030d0 2024-04-02T13:30:19.287 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.282+0000 7fc63dffb700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 0x7fc6300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:19.288 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.282+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fc6380d1c80 con 0x7fc6401030d0 2024-04-02T13:30:19.288 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.282+0000 7fc644969700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 0x7fc6300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:19.288 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.283+0000 7fc644969700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 0x7fc6300a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc634005fd0 tx=0x7fc634005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:19.290 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.287+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc63809f1a0 con 0x7fc6401030d0 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi187 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 5693M 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:19 smithi187 ceph-mon[36064]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:19.498 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:19.494+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc64006b550 con 0x7fc6300a3c40 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi187 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: osdmap e54: 8 total, 8 up, 8 in 2024-04-02T13:30:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 5693M 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:19.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:19 smithi122 ceph-mon[32114]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:20.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:20 smithi188 ceph-mon[36117]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:30:20.432 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:20 smithi188 ceph-mon[36117]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:20.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:20.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:20.433 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:20 smithi122 ceph-mon[32114]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:30:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:20 smithi122 ceph-mon[32114]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:20.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:20 smithi187 ceph-mon[36064]: osdmap e55: 8 total, 8 up, 8 in 2024-04-02T13:30:20.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:20 smithi187 ceph-mon[36064]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:20.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:20.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:20.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:21 smithi122 ceph-mon[32114]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:30:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:21 smithi122 ceph-mon[32114]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:21 smithi187 ceph-mon[36064]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:30:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:21 smithi187 ceph-mon[36064]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:21 smithi188 ceph-mon[36117]: osdmap e56: 8 total, 8 up, 8 in 2024-04-02T13:30:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:21 smithi188 ceph-mon[36117]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:22.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:22 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/2668024598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:22 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:22 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]': finished 2024-04-02T13:30:22.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:22 smithi122 ceph-mon[32114]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:30:22.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:22 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:22 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/2668024598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:22 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:22 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]': finished 2024-04-02T13:30:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:22 smithi187 ceph-mon[36064]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:30:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:22 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:22 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/2668024598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:22 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]: dispatch 2024-04-02T13:30:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:22 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5093b075-74f5-494e-afef-c42bda5e04b6"}]': finished 2024-04-02T13:30:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:22 smithi188 ceph-mon[36117]: osdmap e57: 9 total, 8 up, 9 in 2024-04-02T13:30:22.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:22 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:23 smithi187 ceph-mon[36064]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:23 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/4089302077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:23 smithi188 ceph-mon[36117]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:23 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/4089302077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:23 smithi122 ceph-mon[32114]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:23 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/4089302077' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:25 smithi187 ceph-mon[36064]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:25 smithi188 ceph-mon[36117]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:25 smithi122 ceph-mon[32114]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:27 smithi187 ceph-mon[36064]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:27 smithi188 ceph-mon[36117]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:27 smithi122 ceph-mon[32114]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:29 smithi188 ceph-mon[36117]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:30:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:29 smithi122 ceph-mon[32114]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:30:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:29 smithi187 ceph-mon[36064]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-02T13:30:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:30 smithi188 ceph-mon[36117]: Deploying daemon osd.8 on smithi188 2024-04-02T13:30:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:30 smithi122 ceph-mon[32114]: Deploying daemon osd.8 on smithi188 2024-04-02T13:30:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:30 smithi187 ceph-mon[36064]: Deploying daemon osd.8 on smithi188 2024-04-02T13:30:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:31 smithi188 ceph-mon[36117]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:31 smithi122 ceph-mon[32114]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:31 smithi187 ceph-mon[36064]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:33 smithi188 ceph-mon[36117]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:33 smithi122 ceph-mon[32114]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:33 smithi187 ceph-mon[36064]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:35.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:35 smithi188 ceph-mon[36117]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:35.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.582 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:35 smithi122 ceph-mon[32114]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:35 smithi187 ceph-mon[36064]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:37.649 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:37 smithi188 ceph-mon[36117]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:37 smithi122 ceph-mon[32114]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:37 smithi187 ceph-mon[36064]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:38.862 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.859+0000 7fc63dffb700 1 -- 172.21.15.188:0/130394183 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc64006b550 con 0x7fc6300a3c40 2024-04-02T13:30:38.863 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 8 on host 'smithi188' 2024-04-02T13:30:38.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 msgr2=0x7fc6300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:38.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 0x7fc6300a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc634005fd0 tx=0x7fc634005e50 comp rx=0 tx=0).stop 2024-04-02T13:30:38.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 msgr2=0x7fc64019ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:38.865 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc638007bd0 tx=0x7fc638007f90 comp rx=0 tx=0).stop 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 shutdown_connections 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6401030d0 0x7fc64019ecd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640101530 0x7fc640197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc6300a3c40 0x7fc6300a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 --2- 172.21.15.188:0/130394183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc640102730 0x7fc640197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.861+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 >> 172.21.15.188:0/130394183 conn(0x7fc6400fcaf0 msgr2=0x7fc6401059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.862+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 shutdown_connections 2024-04-02T13:30:38.866 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:30:38.862+0000 7fc646bcd700 1 -- 172.21.15.188:0/130394183 wait complete. 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:39.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:39 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:39.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:39 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.554 DEBUG:teuthology.orchestra.run.smithi188:osd.8> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.8.service 2024-04-02T13:30:39.557 INFO:tasks.cephadm:Deploying osd.9 on smithi122 with /dev/vg_nvme/lv_1... 2024-04-02T13:30:39.557 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:30:39.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:39 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:39.812 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:30:40.248 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:40 smithi122 ceph-mon[32114]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:30:40.248 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:40 smithi122 ceph-mon[32114]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.248 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:40 smithi122 ceph-mon[32114]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:30:40.249 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:40.249 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:40 smithi122 ceph-mon[32114]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:40 smithi187 ceph-mon[36064]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:30:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:40 smithi187 ceph-mon[36064]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:40 smithi187 ceph-mon[36064]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:30:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:40 smithi187 ceph-mon[36064]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.512 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-mon[36117]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-02T13:30:40.512 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-mon[36117]: from='osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.512 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-mon[36117]: osdmap e58: 9 total, 8 up, 9 in 2024-04-02T13:30:40.512 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:40.512 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-mon[36117]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:30:40.512 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:30:40.136+0000 7fc6c57e8700 -1 osd.8 0 waiting for initial osdmap 2024-04-02T13:30:40.512 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:30:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:30:40.145+0000 7fc6c15db700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] boot 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:30:41.205 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] boot 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:30:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: osdmap e59: 9 total, 8 up, 9 in 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: osd.8 [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] boot 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: osdmap e60: 9 total, 9 up, 9 in 2024-04-02T13:30:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr: stderr: 10+0 records in 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:10+0 records out 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.015601 s, 672 MB/s 2024-04-02T13:30:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:--> Zapping successful for: 2024-04-02T13:30:42.293 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:42 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:30:42.293 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:42 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:30:42.294 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:42 smithi122 ceph-mon[32114]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:30:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:42 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:30:42.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:42 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:30:42.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:42 smithi187 ceph-mon[36064]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:30:42.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:42 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:30:42.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:42 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:30:42.505 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:42 smithi188 ceph-mon[36117]: osdmap e61: 9 total, 9 up, 9 in 2024-04-02T13:30:42.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-04-02T13:30:43.034 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi188 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 5693M 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:43 smithi122 ceph-mon[32114]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:30:43.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:43.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi188 2024-04-02T13:30:43.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 5693M 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:43.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:43 smithi187 ceph-mon[36064]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:30:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi188 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 5693M 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:30:44.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:30:44.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:43 smithi188 ceph-mon[36117]: osdmap e62: 9 total, 9 up, 9 in 2024-04-02T13:30:44.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.674+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/781457225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f4100920 msgr2=0x7ff0f4100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:44.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.674+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/781457225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f4100920 0x7ff0f4100d40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40099c0 tx=0x7ff0e4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/781457225 shutdown_connections 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/781457225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4104340 0x7ff0f410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/781457225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f4100920 0x7ff0f4100d40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/781457225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f41039a0 0x7ff0f4103e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/781457225 >> 172.21.15.122:0/781457225 conn(0x7ff0f40fc280 msgr2=0x7ff0f40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/781457225 shutdown_connections 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/781457225 wait complete. 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 Processor -- start 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- start start 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4100920 0x7ff0f4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f4104340 0x7ff0f419fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:44.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410e230 con 0x7ff0f41039a0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410e0b0 con 0x7ff0f4100920 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0fc0ce700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f410e3b0 con 0x7ff0f4104340 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0f9669700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.675+0000 7ff0f9669700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55524/0 (socket says 172.21.15.122:55524) 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 -- 172.21.15.122:0/1121322027 learned_addr learned my addr 172.21.15.122:0/1121322027 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9e6a700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4100920 0x7ff0f4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0fa66b700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f4104340 0x7ff0f419fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 -- 172.21.15.122:0/1121322027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f4104340 msgr2=0x7ff0f419fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f4104340 0x7ff0f419fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 -- 172.21.15.122:0/1121322027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4100920 msgr2=0x7ff0f4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4100920 0x7ff0f4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 -- 172.21.15.122:0/1121322027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f41a0480 con 0x7ff0f41039a0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0f9669700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d9b0 tx=0x7ff0f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:44.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0017070 con 0x7ff0f41039a0 2024-04-02T13:30:44.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f41a0770 con 0x7ff0f41039a0 2024-04-02T13:30:44.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f0005020 con 0x7ff0f41039a0 2024-04-02T13:30:44.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0009a90 con 0x7ff0f41039a0 2024-04-02T13:30:44.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.676+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a0c60 con 0x7ff0f41039a0 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.677+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f40692b0 con 0x7ff0f41039a0 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.678+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff0f00051d0 con 0x7ff0f41039a0 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.679+0000 7ff0eaffd700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 0x7ff0e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.679+0000 7ff0f9e6a700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 0x7ff0e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.679+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7ff0f00cf8c0 con 0x7ff0f41039a0 2024-04-02T13:30:44.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.679+0000 7ff0f9e6a700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 0x7ff0e00a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400b580 tx=0x7ff0e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:30:44.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.683+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff0f0020020 con 0x7ff0f41039a0 2024-04-02T13:30:44.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:30:44.893+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff0f4062a30 con 0x7ff0e00a3c40 2024-04-02T13:30:45.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:45 smithi187 ceph-mon[36064]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:30:45.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:45 smithi187 ceph-mon[36064]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:45.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:45.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:45.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:45 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:45.506 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:45 smithi122 ceph-mon[32114]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:30:45.506 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:45 smithi122 ceph-mon[32114]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:45.506 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:45.506 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:45.507 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:45 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:45 smithi188 ceph-mon[36117]: osdmap e63: 9 total, 9 up, 9 in 2024-04-02T13:30:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:45 smithi188 ceph-mon[36117]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:30:45.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:30:45.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:45 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:46.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:46 smithi187 ceph-mon[36064]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:46.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:46 smithi187 ceph-mon[36064]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:30:46.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:46 smithi188 ceph-mon[36117]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:46.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:46 smithi188 ceph-mon[36117]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:30:46.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:46 smithi122 ceph-mon[32114]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:30:46.536 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:46 smithi122 ceph-mon[32114]: osdmap e64: 9 total, 9 up, 9 in 2024-04-02T13:30:47.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:47 smithi187 ceph-mon[36064]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:47 smithi188 ceph-mon[36117]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:47 smithi122 ceph-mon[32114]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:48.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]: dispatch 2024-04-02T13:30:48.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]': finished 2024-04-02T13:30:48.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:48 smithi187 ceph-mon[36064]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:30:48.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:30:48.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]: dispatch 2024-04-02T13:30:48.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]': finished 2024-04-02T13:30:48.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:48 smithi188 ceph-mon[36117]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:30:48.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:30:48.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]: dispatch 2024-04-02T13:30:48.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1402199995' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a26c2ba8-71db-4d24-8add-1e8ec85b3b70"}]': finished 2024-04-02T13:30:48.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:48 smithi122 ceph-mon[32114]: osdmap e65: 10 total, 9 up, 10 in 2024-04-02T13:30:48.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:30:49.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:49 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1002421098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:49.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:49 smithi187 ceph-mon[36064]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T13:30:49.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:49 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1002421098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:49.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:49 smithi188 ceph-mon[36117]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T13:30:49.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:49 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1002421098' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:30:49.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:49 smithi122 ceph-mon[32114]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-02T13:30:51.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:51 smithi188 ceph-mon[36117]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-02T13:30:51.786 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:51 smithi122 ceph-mon[32114]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-02T13:30:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:51 smithi187 ceph-mon[36064]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-02T13:30:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:53 smithi188 ceph-mon[36117]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:30:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:53 smithi122 ceph-mon[32114]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:30:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:53 smithi187 ceph-mon[36064]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:30:55.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:55 smithi122 ceph-mon[32114]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:30:55.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:30:55.707 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:55 smithi187 ceph-mon[36064]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:30:55.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:30:55.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:55 smithi188 ceph-mon[36117]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-04-02T13:30:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-02T13:30:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:30:56.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:56 smithi188 ceph-mon[36117]: Deploying daemon osd.9 on smithi122 2024-04-02T13:30:56.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:56 smithi122 ceph-mon[32114]: Deploying daemon osd.9 on smithi122 2024-04-02T13:30:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:56 smithi187 ceph-mon[36064]: Deploying daemon osd.9 on smithi122 2024-04-02T13:30:57.755 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:57 smithi122 ceph-mon[32114]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:30:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:57 smithi188 ceph-mon[36117]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:30:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:57 smithi187 ceph-mon[36064]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-02T13:30:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:30:59 smithi188 ceph-mon[36117]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:30:59 smithi122 ceph-mon[32114]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:30:59.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:30:59 smithi187 ceph-mon[36064]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:01 smithi188 ceph-mon[36117]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:01.796 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:01 smithi122 ceph-mon[32114]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:01.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:01.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:01.797 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:01 smithi187 ceph-mon[36064]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:03 smithi188 ceph-mon[36117]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:03.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:03 smithi122 ceph-mon[32114]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:03 smithi187 ceph-mon[36064]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:05.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.488+0000 7ff0eaffd700 1 -- 172.21.15.122:0/1121322027 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff0f4062a30 con 0x7ff0e00a3c40 2024-04-02T13:31:05.493 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 9 on host 'smithi122' 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 msgr2=0x7ff0e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 0x7ff0e00a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400b580 tx=0x7ff0e4005fb0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 msgr2=0x7ff0f4198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000d9b0 tx=0x7ff0f000dcc0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 shutdown_connections 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0f4104340 0x7ff0f419fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff0f4100920 0x7ff0f4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff0e00a3c40 0x7ff0e00a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 --2- 172.21.15.122:0/1121322027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0f41039a0 0x7ff0f4198d60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 >> 172.21.15.122:0/1121322027 conn(0x7ff0f40fc280 msgr2=0x7ff0f4106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 shutdown_connections 2024-04-02T13:31:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:05.491+0000 7ff0fc0ce700 1 -- 172.21.15.122:0/1121322027 wait complete. 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:05.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:05 smithi187 ceph-mon[36064]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:31:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:05.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:05.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:05.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:05 smithi188 ceph-mon[36117]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:05.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:05 smithi122 ceph-mon[32114]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-02T13:31:06.201 DEBUG:teuthology.orchestra.run.smithi122:osd.9> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.9.service 2024-04-02T13:31:06.204 INFO:tasks.cephadm:Deploying osd.10 on smithi187 with /dev/vg_nvme/lv_1... 2024-04-02T13:31:06.204 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:31:06.451 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:06 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:06.756 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:31:06.756 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:31:06.756 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:31:06.756 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:06.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:06.757 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:31:06.432+0000 7f240ad72700 -1 osd.9 0 waiting for initial osdmap 2024-04-02T13:31:06.757 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:31:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:31:06.445+0000 7f2406b65700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:31:06.758 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: osdmap e66: 10 total, 9 up, 10 in 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:06.759 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:06 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:07.690 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:07 smithi187 ceph-mon[36064]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:31:07.690 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:07 smithi187 ceph-mon[36064]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:31:07.690 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:07.690 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:07 smithi187 ceph-mon[36064]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:07.690 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:07 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:07 smithi188 ceph-mon[36117]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:31:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:07 smithi188 ceph-mon[36117]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:31:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:07 smithi188 ceph-mon[36117]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:07 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:07.800 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:07 smithi122 ceph-mon[32114]: from='osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-04-02T13:31:07.800 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:07 smithi122 ceph-mon[32114]: osdmap e67: 10 total, 9 up, 10 in 2024-04-02T13:31:07.800 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:07.800 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:07 smithi122 ceph-mon[32114]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-02T13:31:07.800 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:07 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.367 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:31:08.367 INFO:teuthology.orchestra.run.smithi187.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:31:08.367 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10+0 records in 2024-04-02T13:31:08.367 INFO:teuthology.orchestra.run.smithi187.stderr:10+0 records out 2024-04-02T13:31:08.368 INFO:teuthology.orchestra.run.smithi187.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114974 s, 912 MB/s 2024-04-02T13:31:08.368 INFO:teuthology.orchestra.run.smithi187.stderr:--> Zapping successful for: 2024-04-02T13:31:08.667 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:31:08.668 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:31:08.668 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.668 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] boot 2024-04-02T13:31:08.668 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:31:08.668 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:08 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:31:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:31:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] boot 2024-04-02T13:31:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:31:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:08 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: osd.9 [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] boot 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: osdmap e68: 10 total, 10 up, 10 in 2024-04-02T13:31:08.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:08 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:31:09.043 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi187:vg_nvme/lv_1 2024-04-02T13:31:09.288 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:31:09.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:09 smithi187 ceph-mon[36064]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:09.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:09 smithi187 ceph-mon[36064]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:31:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:09 smithi188 ceph-mon[36117]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:09 smithi188 ceph-mon[36117]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:31:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:09 smithi122 ceph-mon[32114]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:09 smithi122 ceph-mon[32114]: osdmap e69: 10 total, 10 up, 10 in 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi122 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.683 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-02T13:31:10.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:10.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:10.684 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:10 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi122 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-02T13:31:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:10 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:10.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 -- 172.21.15.187:0/1089992944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe058100930 msgr2=0x7fe058100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:10.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 --2- 172.21.15.187:0/1089992944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe058100930 0x7fe058100d50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe054009990 tx=0x7fe054009ca0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.925 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 -- 172.21.15.187:0/1089992944 shutdown_connections 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 --2- 172.21.15.187:0/1089992944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0581042b0 0x7fe05810ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 --2- 172.21.15.187:0/1089992944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe058100930 0x7fe058100d50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 --2- 172.21.15.187:0/1089992944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058103910 0x7fe058103d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 -- 172.21.15.187:0/1089992944 >> 172.21.15.187:0/1089992944 conn(0x7fe0580fc290 msgr2=0x7fe0580fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 -- 172.21.15.187:0/1089992944 shutdown_connections 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.920+0000 7fe0606d0700 1 -- 172.21.15.187:0/1089992944 wait complete. 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 Processor -- start 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 -- start start 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058100930 0x7fe058198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe058103910 0x7fe058198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810e240 con 0x7fe058100930 2024-04-02T13:31:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810e0c0 con 0x7fe0581042b0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe0606d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe05810e3c0 con 0x7fe058103910 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:46434/0 (socket says 172.21.15.187:46434) 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 -- 172.21.15.187:0/2282685574 learned_addr learned my addr 172.21.15.187:0/2282685574 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05dc6b700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe058103910 0x7fe058198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05e46c700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058100930 0x7fe058198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 -- 172.21.15.187:0/2282685574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe058103910 msgr2=0x7fe058198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe058103910 0x7fe058198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 -- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058100930 msgr2=0x7fe058198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058100930 0x7fe058198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 -- 172.21.15.187:0/2282685574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0581a03f0 con 0x7fe0581042b0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.921+0000 7fe05ec6d700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe05000ef30 tx=0x7fe05000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500092e0 con 0x7fe0581042b0 2024-04-02T13:31:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050004bb0 con 0x7fe0581042b0 2024-04-02T13:31:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0581a0680 con 0x7fe0581042b0 2024-04-02T13:31:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004280 con 0x7fe0581042b0 2024-04-02T13:31:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581a0b70 con 0x7fe0581042b0 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.922+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0580692b0 con 0x7fe0581042b0 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.924+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fe05001b030 con 0x7fe0581042b0 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.924+0000 7fe04f7fe700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 0x7fe0440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.924+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fe0500d0020 con 0x7fe0581042b0 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.925+0000 7fe05e46c700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 0x7fe0440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:10.929 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.925+0000 7fe05e46c700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 0x7fe0440a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b7b0 tx=0x7fe054005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:10.931 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:10.928+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe050022020 con 0x7fe0581042b0 2024-04-02T13:31:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi122 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 4272M 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:10 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:11.140 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:11.136+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe058062af0 con 0x7fe0440a3c40 2024-04-02T13:31:11.892 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:11 smithi187 ceph-mon[36064]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:11.892 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:11.892 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:11.892 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:11 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:11 smithi188 ceph-mon[36117]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:11 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:12.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:11 smithi122 ceph-mon[32114]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:12.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:12.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:12.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:11 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:12.950 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:12 smithi187 ceph-mon[36064]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:12 smithi188 ceph-mon[36117]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:13.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:12 smithi122 ceph-mon[32114]: from='client.24341 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi187:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:13 smithi187 ceph-mon[36064]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:13 smithi188 ceph-mon[36117]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:14.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:13 smithi122 ceph-mon[32114]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/1523460161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]': finished 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:14.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.187:0/2095037233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/1523460161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]': finished 2024-04-02T13:31:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:31:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:15.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.187:0/2095037233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/1523460161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]: dispatch 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ee33545-53b3-428d-9b14-69e4bba42144"}]': finished 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: osdmap e70: 11 total, 10 up, 11 in 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:15.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.187:0/2095037233' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:15.972 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:15 smithi122 ceph-mon[32114]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:15 smithi187 ceph-mon[36064]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:15 smithi188 ceph-mon[36117]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:17.901 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:17 smithi187 ceph-mon[36064]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:17 smithi188 ceph-mon[36117]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:18.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:17 smithi122 ceph-mon[32114]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:19.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:19 smithi187 ceph-mon[36064]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:19 smithi188 ceph-mon[36117]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:20.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:19 smithi122 ceph-mon[32114]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:21 smithi187 ceph-mon[36064]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:31:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:21 smithi188 ceph-mon[36117]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:22.078 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:31:22.079 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:22.079 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:21 smithi122 ceph-mon[32114]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:22.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-02T13:31:22.080 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:22 smithi187 ceph-mon[36064]: Deploying daemon osd.10 on smithi187 2024-04-02T13:31:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:22 smithi188 ceph-mon[36117]: Deploying daemon osd.10 on smithi187 2024-04-02T13:31:23.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:22 smithi122 ceph-mon[32114]: Deploying daemon osd.10 on smithi187 2024-04-02T13:31:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:23 smithi187 ceph-mon[36064]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:23 smithi188 ceph-mon[36117]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:23 smithi122 ceph-mon[32114]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:24.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:24 smithi187 ceph-mon[36064]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:24 smithi188 ceph-mon[36117]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:24 smithi122 ceph-mon[32114]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:27.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:27 smithi122 ceph-mon[32114]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:27.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:27 smithi188 ceph-mon[36117]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:27.808 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:27 smithi187 ceph-mon[36064]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:27.808 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.808 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:27.808 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:29 smithi188 ceph-mon[36117]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:29 smithi122 ceph-mon[32114]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:29 smithi187 ceph-mon[36064]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:30.406 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.402+0000 7fe04f7fe700 1 -- 172.21.15.187:0/2282685574 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe058062af0 con 0x7fe0440a3c40 2024-04-02T13:31:30.406 INFO:teuthology.orchestra.run.smithi187.stdout:Created osd(s) 10 on host 'smithi187' 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 msgr2=0x7fe0440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 0x7fe0440a6100 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe05400b7b0 tx=0x7fe054005fd0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 msgr2=0x7fe05819fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe05000ef30 tx=0x7fe05000c3a0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 shutdown_connections 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0581042b0 0x7fe05819fd50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0440a3c40 0x7fe0440a6100 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe058100930 0x7fe058198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 --2- 172.21.15.187:0/2282685574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe058103910 0x7fe058198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 >> 172.21.15.187:0/2282685574 conn(0x7fe0580fc290 msgr2=0x7fe058106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 shutdown_connections 2024-04-02T13:31:30.408 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-02T13:31:30.404+0000 7fe0606d0700 1 -- 172.21.15.187:0/2282685574 wait complete. 2024-04-02T13:31:31.286 DEBUG:teuthology.orchestra.run.smithi187:osd.10> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.10.service 2024-04-02T13:31:31.288 INFO:tasks.cephadm:Deploying osd.11 on smithi188 with /dev/vg_nvme/lv_1... 2024-04-02T13:31:31.288 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-02T13:31:31.539 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:31:31.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:31 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:31 smithi188 ceph-mon[36117]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:31.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:31 smithi188 ceph-mon[36117]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:31.660 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:31 smithi188 ceph-mon[36117]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:31 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:31 smithi187 ceph-mon[36064]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:31 smithi187 ceph-mon[36064]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:31 smithi187 ceph-mon[36064]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:31 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:31 smithi122 ceph-mon[32114]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:31 smithi122 ceph-mon[32114]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:31 smithi122 ceph-mon[32114]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:32.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.679 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:32.680 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:32.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-02T13:31:32.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: osdmap e71: 11 total, 10 up, 11 in 2024-04-02T13:31:32.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.829 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]: dispatch 2024-04-02T13:31:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:32.830 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:33.425 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:31:33.103+0000 7f6fb3a29700 -1 osd.10 0 waiting for initial osdmap 2024-04-02T13:31:33.425 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:31:33.115+0000 7f6fab814700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:31:33.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:31:33.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:31:33.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:33.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:33 smithi188 ceph-mon[36117]: from='osd.10 ' entity='osd.10' 2024-04-02T13:31:33.663 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-02T13:31:33.663 INFO:teuthology.orchestra.run.smithi188.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-02T13:31:33.663 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10+0 records in 2024-04-02T13:31:33.663 INFO:teuthology.orchestra.run.smithi188.stderr:10+0 records out 2024-04-02T13:31:33.664 INFO:teuthology.orchestra.run.smithi188.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113487 s, 924 MB/s 2024-04-02T13:31:33.664 INFO:teuthology.orchestra.run.smithi188.stderr:--> Zapping successful for: 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:33 smithi187 ceph-mon[36064]: from='osd.10 ' entity='osd.10' 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi187", "root=default"]}]': finished 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: osdmap e72: 11 total, 10 up, 11 in 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-02T13:31:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:33 smithi122 ceph-mon[32114]: from='osd.10 ' entity='osd.10' 2024-04-02T13:31:34.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch daemon add osd smithi188:vg_nvme/lv_1 2024-04-02T13:31:34.639 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:31:34.686 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:31:34.686 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:31:34.687 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:34.687 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] boot 2024-04-02T13:31:34.687 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:31:34.687 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:34 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] boot 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:31:34.761 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:34 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: osd.10 [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] boot 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: osdmap e73: 11 total, 11 up, 11 in 2024-04-02T13:31:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:34 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:35.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:35.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:31:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:35.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: osdmap e74: 11 total, 11 up, 11 in 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:31:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:35.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:35.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:35.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 -- 172.21.15.188:0/3690074817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44105f30 msgr2=0x7fae44106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 --2- 172.21.15.188:0/3690074817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44105f30 0x7fae44106350 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fae380099c0 tx=0x7fae38009cd0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 -- 172.21.15.188:0/3690074817 shutdown_connections 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 --2- 172.21.15.188:0/3690074817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104e10 0x7fae4410cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 --2- 172.21.15.188:0/3690074817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44105f30 0x7fae44106350 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 --2- 172.21.15.188:0/3690074817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44104470 0x7fae441048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 -- 172.21.15.188:0/3690074817 >> 172.21.15.188:0/3690074817 conn(0x7fae440ffe10 msgr2=0x7fae44102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 -- 172.21.15.188:0/3690074817 shutdown_connections 2024-04-02T13:31:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.420+0000 7fae4a69a700 1 -- 172.21.15.188:0/3690074817 wait complete. 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 Processor -- start 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 -- start start 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104470 0x7fae4419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44105f30 0x7fae441a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae44110530 con 0x7fae44104470 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae441103b0 con 0x7fae44105f30 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae4a69a700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae441106b0 con 0x7fae44104e10 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae437fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae437fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:51334/0 (socket says 172.21.15.188:51334) 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae437fe700 1 -- 172.21.15.188:0/1575879390 learned_addr learned my addr 172.21.15.188:0/1575879390 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae43fff700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104470 0x7fae4419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.421+0000 7fae48c37700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44105f30 0x7fae441a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 -- 172.21.15.188:0/1575879390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44105f30 msgr2=0x7fae441a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44105f30 0x7fae441a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 -- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104470 msgr2=0x7fae4419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104470 0x7fae4419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 -- 172.21.15.188:0/1575879390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae441a2710 con 0x7fae44104e10 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae437fe700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d9b0 tx=0x7fae3400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34017070 con 0x7fae44104e10 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae441a2a00 con 0x7fae44104e10 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae34005020 con 0x7fae44104e10 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34009a90 con 0x7fae44104e10 2024-04-02T13:31:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.422+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae441a2ef0 con 0x7fae44104e10 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.423+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae440692b0 con 0x7fae44104e10 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.424+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fae340051d0 con 0x7fae44104e10 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.425+0000 7fae417fa700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 0x7fae2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.425+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fae340d0030 con 0x7fae44104e10 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.425+0000 7fae43fff700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 0x7fae2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.426+0000 7fae43fff700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 0x7fae2c0a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b580 tx=0x7fae38005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:36.433 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.428+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae34020020 con 0x7fae44104e10 2024-04-02T13:31:36.642 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:36.638+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fae4402e190 con 0x7fae2c0a3c40 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi187 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 4270M 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:36 smithi187 ceph-mon[36064]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:31:36.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi187 2024-04-02T13:31:36.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 4270M 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:36 smithi188 ceph-mon[36117]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi187 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 4270M 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: osdmap e75: 11 total, 11 up, 11 in 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:31:36.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:36.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:36 smithi122 ceph-mon[32114]: osdmap e76: 11 total, 11 up, 11 in 2024-04-02T13:31:37.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:37 smithi188 ceph-mon[36117]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:37.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:37.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:37.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:37 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:37.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:37 smithi188 ceph-mon[36117]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:31:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:37 smithi187 ceph-mon[36064]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:37 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:37 smithi187 ceph-mon[36064]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:31:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:37 smithi122 ceph-mon[32114]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-02T13:31:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-02T13:31:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:37 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:37 smithi122 ceph-mon[32114]: osdmap e77: 11 total, 11 up, 11 in 2024-04-02T13:31:38.547 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:38 smithi188 ceph-mon[36117]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:38.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:38 smithi187 ceph-mon[36064]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:38 smithi122 ceph-mon[32114]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi188:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:31:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:39 smithi187 ceph-mon[36064]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:31:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:39 smithi188 ceph-mon[36117]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:31:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:39 smithi122 ceph-mon[32114]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/3458391846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]': finished 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:40 smithi187 ceph-mon[36064]: from='client.? 172.21.15.188:0/309219787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/3458391846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]': finished 2024-04-02T13:31:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:31:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:40.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:40 smithi188 ceph-mon[36117]: from='client.? 172.21.15.188:0/309219787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/3458391846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]: dispatch 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14fa986c-a900-464f-a1d3-4bec844c1256"}]': finished 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: osdmap e78: 12 total, 11 up, 12 in 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:40 smithi122 ceph-mon[32114]: from='client.? 172.21.15.188:0/309219787' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-02T13:31:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:41 smithi188 ceph-mon[36117]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-02T13:31:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:41 smithi122 ceph-mon[32114]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-02T13:31:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:41 smithi187 ceph-mon[36064]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-02T13:31:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:43 smithi188 ceph-mon[36117]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:31:43.788 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:43 smithi187 ceph-mon[36064]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:31:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:43 smithi122 ceph-mon[32114]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-04-02T13:31:45.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:45 smithi188 ceph-mon[36117]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:31:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:45 smithi122 ceph-mon[32114]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:31:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:45 smithi187 ceph-mon[36064]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-02T13:31:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:47 smithi122 ceph-mon[32114]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:31:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:47.825 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:47 smithi188 ceph-mon[36117]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:47.826 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:31:47.826 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:47 smithi187 ceph-mon[36064]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:47.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-02T13:31:47.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:31:48.789 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:48 smithi188 ceph-mon[36117]: Deploying daemon osd.11 on smithi188 2024-04-02T13:31:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:48 smithi122 ceph-mon[32114]: Deploying daemon osd.11 on smithi188 2024-04-02T13:31:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:48 smithi187 ceph-mon[36064]: Deploying daemon osd.11 on smithi188 2024-04-02T13:31:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:49 smithi122 ceph-mon[32114]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:49.947 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:49 smithi188 ceph-mon[36117]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:49 smithi187 ceph-mon[36064]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:51.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:51 smithi188 ceph-mon[36117]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:51 smithi122 ceph-mon[32114]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:51 smithi187 ceph-mon[36064]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:53 smithi188 ceph-mon[36117]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:53 smithi122 ceph-mon[32114]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:53 smithi187 ceph-mon[36064]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:31:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:55 smithi188 ceph-mon[36117]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:55 smithi122 ceph-mon[32114]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:55 smithi187 ceph-mon[36064]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:56.462 INFO:teuthology.orchestra.run.smithi188.stdout:Created osd(s) 11 on host 'smithi188' 2024-04-02T13:31:56.463 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.458+0000 7fae417fa700 1 -- 172.21.15.188:0/1575879390 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fae3800b810 con 0x7fae2c0a3c40 2024-04-02T13:31:56.464 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 msgr2=0x7fae2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:56.464 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 0x7fae2c0a6100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b580 tx=0x7fae38005fb0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 msgr2=0x7fae4419aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fae3400d9b0 tx=0x7fae3400dcc0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 shutdown_connections 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae44105f30 0x7fae441a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae2c0a3c40 0x7fae2c0a6100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae44104470 0x7fae4419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 --2- 172.21.15.188:0/1575879390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae44104e10 0x7fae4419aff0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 >> 172.21.15.188:0/1575879390 conn(0x7fae440ffe10 msgr2=0x7fae44108c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 shutdown_connections 2024-04-02T13:31:56.465 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:31:56.461+0000 7fae4a69a700 1 -- 172.21.15.188:0/1575879390 wait complete. 2024-04-02T13:31:57.132 DEBUG:teuthology.orchestra.run.smithi188:osd.11> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.11.service 2024-04-02T13:31:57.135 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-02T13:31:57.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T13:31:57.396 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:31:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:57 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:57 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-02T13:31:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:57 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:31:58.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:58 smithi122 ceph-mon[32114]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:31:58.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:58 smithi122 ceph-mon[32114]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:31:58.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:58 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:58.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:58 smithi122 ceph-mon[32114]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:58.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:58 smithi122 ceph-mon[32114]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:58.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:58 smithi187 ceph-mon[36064]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:31:58.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:58 smithi187 ceph-mon[36064]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:31:58.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:58 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:58.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:58 smithi187 ceph-mon[36064]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:58.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:58 smithi187 ceph-mon[36064]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:58 smithi188 ceph-mon[36117]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-02T13:31:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:58 smithi188 ceph-mon[36117]: osdmap e79: 12 total, 11 up, 12 in 2024-04-02T13:31:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:58 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:58 smithi188 ceph-mon[36117]: from='osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:58.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:58 smithi188 ceph-mon[36117]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]: dispatch 2024-04-02T13:31:59.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.126+0000 7f71e2e53700 1 -- 172.21.15.122:0/1795510424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 msgr2=0x7f71dc103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:59.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.126+0000 7f71e2e53700 1 --2- 172.21.15.122:0/1795510424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc103c70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71c80099c0 tx=0x7f71c8009cd0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- 172.21.15.122:0/1795510424 shutdown_connections 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- 172.21.15.122:0/1795510424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc1041b0 0x7f71dc10aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- 172.21.15.122:0/1795510424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc102610 0x7f71dc102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- 172.21.15.122:0/1795510424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc103c70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- 172.21.15.122:0/1795510424 >> 172.21.15.122:0/1795510424 conn(0x7f71dc0fdbd0 msgr2=0x7f71dc0ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- 172.21.15.122:0/1795510424 shutdown_connections 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- 172.21.15.122:0/1795510424 wait complete. 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 Processor -- start 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- start start 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc102610 0x7f71dc076390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1041b0 0x7f71dc074f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10e1e0 con 0x7f71dc103810 2024-04-02T13:31:59.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10e060 con 0x7f71dc1041b0 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e2e53700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc10e360 con 0x7f71dc102610 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71dbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47760/0 (socket says 172.21.15.122:47760) 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71e0bef700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc102610 0x7f71dc076390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.127+0000 7f71dbfff700 1 -- 172.21.15.122:0/2500196585 learned_addr learned my addr 172.21.15.122:0/2500196585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71e13f0700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1041b0 0x7f71dc074f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 -- 172.21.15.122:0/2500196585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc102610 msgr2=0x7f71dc076390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc102610 0x7f71dc076390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 -- 172.21.15.122:0/2500196585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1041b0 msgr2=0x7f71dc074f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1041b0 0x7f71dc074f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 -- 172.21.15.122:0/2500196585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0757d0 con 0x7f71dc103810 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71dbfff700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005910 tx=0x7f71c8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8027070 con 0x7f71dc103810 2024-04-02T13:31:59.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc1a28a0 con 0x7f71dc103810 2024-04-02T13:31:59.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71c8004150 con 0x7f71dc103810 2024-04-02T13:31:59.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8022da0 con 0x7f71dc103810 2024-04-02T13:31:59.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.128+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc1a2cf0 con 0x7f71dc103810 2024-04-02T13:31:59.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.129+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc0692b0 con 0x7f71dc103810 2024-04-02T13:31:59.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.130+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f71c802a070 con 0x7f71dc103810 2024-04-02T13:31:59.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.131+0000 7f71d9ffb700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 0x7f71cc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:31:59.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.131+0000 7f71e0bef700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 0x7f71cc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:31:59.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.131+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f71c80d0fb0 con 0x7f71dc103810 2024-04-02T13:31:59.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.131+0000 7f71e0bef700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 0x7f71cc0a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71dc103670 tx=0x7f71d000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:31:59.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.135+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f71c8031020 con 0x7f71dc103810 2024-04-02T13:31:59.224 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:31:59.160+0000 7f7350e0f700 -1 osd.11 0 waiting for initial osdmap 2024-04-02T13:31:59.224 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:31:59.172+0000 7f7348bfa700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-02T13:31:59.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.337+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f71dc063200 con 0x7f71dc103810 2024-04-02T13:31:59.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.337+0000 7f71d9ffb700 1 -- 172.21.15.122:0/2500196585 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f71c80a2e20 con 0x7f71dc103810 2024-04-02T13:31:59.341 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:31:59.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 msgr2=0x7f71cc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:59.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 0x7f71cc0a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71dc103670 tx=0x7f71d000a3b0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 msgr2=0x7f71dc0749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:31:59.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f71c8005910 tx=0x7f71c8009580 comp rx=0 tx=0).stop 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 shutdown_connections 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71dc1041b0 0x7f71dc074f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc102610 0x7f71dc076390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f71cc0a3cc0 0x7f71cc0a6180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 --2- 172.21.15.122:0/2500196585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71dc103810 0x7f71dc0749e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 >> 172.21.15.122:0/2500196585 conn(0x7f71dc0fdbd0 msgr2=0x7f71dc106ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 shutdown_connections 2024-04-02T13:31:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:31:59.339+0000 7f71e2e53700 1 -- 172.21.15.122:0/2500196585 wait complete. 2024-04-02T13:31:59.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: from='osd.11 ' entity='osd.11' 2024-04-02T13:31:59.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:31:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2500196585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: from='osd.11 ' entity='osd.11' 2024-04-02T13:31:59.639 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:31:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2500196585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:31:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi188", "root=default"]}]': finished 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: osdmap e80: 12 total, 11 up, 12 in 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: from='osd.11 ' entity='osd.11' 2024-04-02T13:31:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:31:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2500196585' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:32:00.012 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1712064693,"num_in_osds":12,"osd_in_since":1712064699,"num_remapped_pgs":0} 2024-04-02T13:32:00.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: purged_snaps scrub starts 2024-04-02T13:32:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: purged_snaps scrub ok 2024-04-02T13:32:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] boot 2024-04-02T13:32:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:32:00.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:00 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:00.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: purged_snaps scrub starts 2024-04-02T13:32:00.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: purged_snaps scrub ok 2024-04-02T13:32:00.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:00.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] boot 2024-04-02T13:32:00.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:32:00.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:00 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: purged_snaps scrub starts 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: purged_snaps scrub ok 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: osd.11 [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] boot 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: osdmap e81: 12 total, 12 up, 12 in 2024-04-02T13:32:00.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:00 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:32:01.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd stat -f json 2024-04-02T13:32:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:32:01.688 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:01.689 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.689 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.689 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:01 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:01 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-04-02T13:32:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: osdmap e82: 12 total, 12 up, 12 in 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:01.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:01 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:02.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:02 smithi187 ceph-mon[36064]: Detected new or changed devices on smithi188 2024-04-02T13:32:02.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:02 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 4270M 2024-04-02T13:32:02.766 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:02 smithi122 ceph-mon[32114]: Detected new or changed devices on smithi188 2024-04-02T13:32:02.766 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:02 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 4270M 2024-04-02T13:32:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:02 smithi188 ceph-mon[36117]: Detected new or changed devices on smithi188 2024-04-02T13:32:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:02 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 4270M 2024-04-02T13:32:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 -- 172.21.15.122:0/165446335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 msgr2=0x7f5194101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 --2- 172.21.15.122:0/165446335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f5194101110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51900099c0 tx=0x7f5190009cd0 comp rx=0 tx=0).stop 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 -- 172.21.15.122:0/165446335 shutdown_connections 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 --2- 172.21.15.122:0/165446335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5194102890 0x7f5194109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 --2- 172.21.15.122:0/165446335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f5194101110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 --2- 172.21.15.122:0/165446335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5194101ef0 0x7f5194102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 -- 172.21.15.122:0/165446335 >> 172.21.15.122:0/165446335 conn(0x7f51940fc290 msgr2=0x7f51940fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 -- 172.21.15.122:0/165446335 shutdown_connections 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 -- 172.21.15.122:0/165446335 wait complete. 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.993+0000 7f519bc9b700 1 Processor -- start 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- start start 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5194101ef0 0x7f519419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5194102890 0x7f519419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:02.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f519410c8d0 con 0x7f5194100cf0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f519410c750 con 0x7f5194102890 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f519410ca50 con 0x7f5194101ef0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47772/0 (socket says 172.21.15.122:47772) 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 -- 172.21.15.122:0/1474142037 learned_addr learned my addr 172.21.15.122:0/1474142037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 -- 172.21.15.122:0/1474142037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5194101ef0 msgr2=0x7f519419d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5194101ef0 0x7f519419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 -- 172.21.15.122:0/1474142037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5194102890 msgr2=0x7f519419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5194102890 0x7f519419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 -- 172.21.15.122:0/1474142037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51941a0500 con 0x7f5194100cf0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f5199a37700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f519000c010 tx=0x7f5190009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190027070 con 0x7f5194100cf0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5190004150 con 0x7f5194100cf0 2024-04-02T13:32:03.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51941a0790 con 0x7f5194100cf0 2024-04-02T13:32:03.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519000f460 con 0x7f5194100cf0 2024-04-02T13:32:03.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.994+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51941a0c70 con 0x7f5194100cf0 2024-04-02T13:32:03.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.995+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51941084a0 con 0x7f5194100cf0 2024-04-02T13:32:03.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.996+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f519002a070 con 0x7f5194100cf0 2024-04-02T13:32:03.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.997+0000 7f518affd700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 0x7f51800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:03.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.997+0000 7f5199236700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 0x7f51800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:03.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.997+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f51900d1290 con 0x7f5194100cf0 2024-04-02T13:32:03.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:02.997+0000 7f5199236700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 0x7f51800a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5184005950 tx=0x7f51840058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:03.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.001+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5190031070 con 0x7f5194100cf0 2024-04-02T13:32:03.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.199+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f519402ddb0 con 0x7f5194100cf0 2024-04-02T13:32:03.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.200+0000 7f518affd700 1 -- 172.21.15.122:0/1474142037 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f5190013070 con 0x7f5194100cf0 2024-04-02T13:32:03.204 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 msgr2=0x7f51800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 0x7f51800a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5184005950 tx=0x7f51840058e0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 msgr2=0x7f51940fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f519000c010 tx=0x7f5190009f10 comp rx=0 tx=0).stop 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 shutdown_connections 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5194102890 0x7f519419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51800a3c40 0x7f51800a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5194100cf0 0x7f51940fe450 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 --2- 172.21.15.122:0/1474142037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5194101ef0 0x7f519419d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:03.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 >> 172.21.15.122:0/1474142037 conn(0x7f51940fc290 msgr2=0x7f51941055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:03.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 shutdown_connections 2024-04-02T13:32:03.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:03.202+0000 7f519bc9b700 1 -- 172.21.15.122:0/1474142037 wait complete. 2024-04-02T13:32:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:03 smithi187 ceph-mon[36064]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:03.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1474142037' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:32:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:03 smithi188 ceph-mon[36117]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1474142037' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:32:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:03 smithi122 ceph-mon[32114]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1474142037' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-02T13:32:03.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1712064720,"num_in_osds":12,"osd_in_since":1712064699,"num_remapped_pgs":0} 2024-04-02T13:32:03.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:32:04.178 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:05.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:05 smithi122 ceph-mon[32114]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:05 smithi187 ceph-mon[36064]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:05 smithi188 ceph-mon[36117]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 -- 172.21.15.122:0/599993513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc101eb0 msgr2=0x7f38fc10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 --2- 172.21.15.122:0/599993513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc101eb0 0x7f38fc10e340 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b0a0 tx=0x7f38f400b3b0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 -- 172.21.15.122:0/599993513 shutdown_connections 2024-04-02T13:32:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 --2- 172.21.15.122:0/599993513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc101eb0 0x7f38fc10e340 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 --2- 172.21.15.122:0/599993513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc107500 0x7f38fc1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 --2- 172.21.15.122:0/599993513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc1014f0 0x7f38fc101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 -- 172.21.15.122:0/599993513 >> 172.21.15.122:0/599993513 conn(0x7f38fc078420 msgr2=0x7f38fc078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 -- 172.21.15.122:0/599993513 shutdown_connections 2024-04-02T13:32:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.802+0000 7f3901d44700 1 -- 172.21.15.122:0/599993513 wait complete. 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 Processor -- start 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 -- start start 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc1014f0 0x7f38fc199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 0x7f38fc19a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111790 con 0x7f38fc107500 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111610 con 0x7f38fc101eb0 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.803+0000 7f3901d44700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111910 con 0x7f38fc1014f0 2024-04-02T13:32:05.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fb7fe700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc1014f0 0x7f38fc199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47798/0 (socket says 172.21.15.122:47798) 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 -- 172.21.15.122:0/272344621 learned_addr learned my addr 172.21.15.122:0/272344621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38faffd700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 0x7f38fc19a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 -- 172.21.15.122:0/272344621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc1014f0 msgr2=0x7f38fc199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc1014f0 0x7f38fc199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 -- 172.21.15.122:0/272344621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 msgr2=0x7f38fc19a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 0x7f38fc19a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 -- 172.21.15.122:0/272344621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc1a5130 con 0x7f38fc107500 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38faffd700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 0x7f38fc19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38fbfff700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b780 tx=0x7f38f4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4021070 con 0x7f38fc107500 2024-04-02T13:32:05.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc1a5300 con 0x7f38fc107500 2024-04-02T13:32:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f4007b80 con 0x7f38fc107500 2024-04-02T13:32:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4005370 con 0x7f38fc107500 2024-04-02T13:32:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.804+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a57b0 con 0x7f38fc107500 2024-04-02T13:32:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.805+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc0692b0 con 0x7f38fc107500 2024-04-02T13:32:05.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.806+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f38f4024070 con 0x7f38fc107500 2024-04-02T13:32:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.807+0000 7f38f8ff9700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 0x7f38e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.807+0000 7f38fb7fe700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 0x7f38e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.807+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f38f400f020 con 0x7f38fc107500 2024-04-02T13:32:05.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.808+0000 7f38fb7fe700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 0x7f38e80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0045b0 tx=0x7f38ec00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:05.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:05.811+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f38f402b020 con 0x7f38fc107500 2024-04-02T13:32:06.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.007+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38fc02dd30 con 0x7f38fc107500 2024-04-02T13:32:06.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.008+0000 7f38f8ff9700 1 -- 172.21.15.122:0/272344621 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18502 (secure 0 0 0) 0x7f38f409e7d0 con 0x7f38fc107500 2024-04-02T13:32:06.012 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:06.012 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":82,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:01.165058+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:06.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 msgr2=0x7f38e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:06.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 0x7f38e80a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f38ec0045b0 tx=0x7f38ec00b3c0 comp rx=0 tx=0).stop 2024-04-02T13:32:06.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 msgr2=0x7f38fc1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b780 tx=0x7f38f4004d00 comp rx=0 tx=0).stop 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 shutdown_connections 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f38e80a3c40 0x7f38e80a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38fc107500 0x7f38fc1a4bf0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38fc1014f0 0x7f38fc199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 --2- 172.21.15.122:0/272344621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38fc101eb0 0x7f38fc19a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 >> 172.21.15.122:0/272344621 conn(0x7f38fc078420 msgr2=0x7f38fc100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 shutdown_connections 2024-04-02T13:32:06.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:06.010+0000 7f3901d44700 1 -- 172.21.15.122:0/272344621 wait complete. 2024-04-02T13:32:06.632 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:32:06.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd pool get .mgr pg_num 2024-04-02T13:32:06.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:06 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/272344621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:06.764 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:06 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/272344621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:06.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/272344621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:07 smithi122 ceph-mon[32114]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:07 smithi187 ceph-mon[36064]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:07 smithi188 ceph-mon[36117]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- 172.21.15.122:0/3458009116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4101780 msgr2=0x7f9bd4101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/3458009116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4101780 0x7f9bd4101b60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40099c0 tx=0x7f9bc4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- 172.21.15.122:0/3458009116 shutdown_connections 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/3458009116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 0x7f9bd410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/3458009116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4101780 0x7f9bd4101b60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/3458009116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4102130 0x7f9bd4106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- 172.21.15.122:0/3458009116 >> 172.21.15.122:0/3458009116 conn(0x7f9bd4077510 msgr2=0x7f9bd4077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- 172.21.15.122:0/3458009116 shutdown_connections 2024-04-02T13:32:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- 172.21.15.122:0/3458009116 wait complete. 2024-04-02T13:32:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 Processor -- start 2024-04-02T13:32:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.551+0000 7f9bda6e3700 1 -- start start 2024-04-02T13:32:08.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 0x7f9bd4197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:08.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:08.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 0x7f9bd4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:08.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4111820 con 0x7f9bd4102130 2024-04-02T13:32:08.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd41116a0 con 0x7f9bd4101780 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bda6e3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd41119a0 con 0x7f9bd4106880 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49572/0 (socket says 172.21.15.122:49572) 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 -- 172.21.15.122:0/1963792626 learned_addr learned my addr 172.21.15.122:0/1963792626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd3fff700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 0x7f9bd4197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 -- 172.21.15.122:0/1963792626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 msgr2=0x7f9bd4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd8c80700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 0x7f9bd4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 0x7f9bd4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 -- 172.21.15.122:0/1963792626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 msgr2=0x7f9bd4197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 0x7f9bd4197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 -- 172.21.15.122:0/1963792626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd41a7200 con 0x7f9bd4102130 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd3fff700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 0x7f9bd4197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:08.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.552+0000 7f9bd37fe700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d950 tx=0x7f9bc800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8016070 con 0x7f9bd4102130 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd41a74f0 con 0x7f9bd4102130 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc8009b40 con 0x7f9bd4102130 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800f620 con 0x7f9bd4102130 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41a7b00 con 0x7f9bd4102130 2024-04-02T13:32:08.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.553+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bd4109ac0 con 0x7f9bd4102130 2024-04-02T13:32:08.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.555+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9bc8019070 con 0x7f9bd4102130 2024-04-02T13:32:08.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.556+0000 7f9bd17fa700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 0x7f9bbc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:08.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.556+0000 7f9bd3fff700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 0x7f9bbc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:08.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.556+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9bc80cff40 con 0x7f9bd4102130 2024-04-02T13:32:08.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.556+0000 7f9bd3fff700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 0x7f9bbc0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009750 tx=0x7f9bc4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:08.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.559+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9bc809cdc0 con 0x7f9bd4102130 2024-04-02T13:32:08.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.756+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9bd406b2e0 con 0x7f9bd4102130 2024-04-02T13:32:08.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.756+0000 7f9bd17fa700 1 -- 172.21.15.122:0/1963792626 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f9bc809c6c0 con 0x7f9bd4102130 2024-04-02T13:32:08.761 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 msgr2=0x7f9bbc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 0x7f9bbc0a6100 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009750 tx=0x7f9bc4009f10 comp rx=0 tx=0).stop 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 msgr2=0x7f9bd4197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d950 tx=0x7f9bc800dd10 comp rx=0 tx=0).stop 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 shutdown_connections 2024-04-02T13:32:08.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bd4106880 0x7f9bd4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bd4101780 0x7f9bd4197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9bbc0a3c40 0x7f9bbc0a6100 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 --2- 172.21.15.122:0/1963792626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bd4102130 0x7f9bd4197ed0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 >> 172.21.15.122:0/1963792626 conn(0x7f9bd4077510 msgr2=0x7f9bd41079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 shutdown_connections 2024-04-02T13:32:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:08.759+0000 7f9bda6e3700 1 -- 172.21.15.122:0/1963792626 wait complete. 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.a on smithi122 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.d on smithi122 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.g on smithi122 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.j on smithi122 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.b on smithi187 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.e on smithi187 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.h on smithi187 2024-04-02T13:32:09.371 INFO:tasks.cephadm:Adding mds.k on smithi187 2024-04-02T13:32:09.372 INFO:tasks.cephadm:Adding mds.c on smithi188 2024-04-02T13:32:09.372 INFO:tasks.cephadm:Adding mds.f on smithi188 2024-04-02T13:32:09.372 INFO:tasks.cephadm:Adding mds.i on smithi188 2024-04-02T13:32:09.372 INFO:tasks.cephadm:Adding mds.l on smithi188 2024-04-02T13:32:09.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph orch apply mds all '12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l' 2024-04-02T13:32:09.626 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:32:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:09 smithi187 ceph-mon[36064]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:09.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1963792626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:09.747 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:09 smithi188 ceph-mon[36117]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:09.747 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1963792626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:09 smithi122 ceph-mon[32114]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1963792626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 -- 172.21.15.188:0/1275645124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90100f10 msgr2=0x7f7f90103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/1275645124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90100f10 0x7f7f90103340 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009990 tx=0x7f7f80009ca0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 -- 172.21.15.188:0/1275645124 shutdown_connections 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/1275645124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90106250 0x7f7f901066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/1275645124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90100f10 0x7f7f90103340 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.269 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/1275645124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90103880 0x7f7f90105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.265+0000 7f7f97fb3700 1 -- 172.21.15.188:0/1275645124 >> 172.21.15.188:0/1275645124 conn(0x7f7f900fcb50 msgr2=0x7f7f900fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:11.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- 172.21.15.188:0/1275645124 shutdown_connections 2024-04-02T13:32:11.270 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- 172.21.15.188:0/1275645124 wait complete. 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 Processor -- start 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- start start 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 0x7f7f9019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90103880 0x7f7f9019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f90114860 con 0x7f7f90100f10 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901146e0 con 0x7f7f90103880 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.266+0000 7f7f97fb3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f901149e0 con 0x7f7f90106250 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:11.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:45246/0 (socket says 172.21.15.188:45246) 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 -- 172.21.15.188:0/2943104644 learned_addr learned my addr 172.21.15.188:0/2943104644 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f95d4f700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 0x7f7f9019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f9554e700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90103880 0x7f7f9019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 -- 172.21.15.188:0/2943104644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90103880 msgr2=0x7f7f9019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90103880 0x7f7f9019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 -- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 msgr2=0x7f7f9019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 0x7f7f9019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 -- 172.21.15.188:0/2943104644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f901a6b30 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f95d4f700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 0x7f7f9019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f96550700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800b9a0 tx=0x7f7f8800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88019070 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f88014440 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f901a6e20 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880075d0 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.267+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f901a73a0 con 0x7f7f90106250 2024-04-02T13:32:11.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.268+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f900692b0 con 0x7f7f90106250 2024-04-02T13:32:11.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.269+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7f8801c070 con 0x7f7f90106250 2024-04-02T13:32:11.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.270+0000 7f7f86ffd700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 0x7f7f7c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:11.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.270+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f7f880cf770 con 0x7f7f90106250 2024-04-02T13:32:11.274 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.270+0000 7f7f95d4f700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 0x7f7f7c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:11.275 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.271+0000 7f7f95d4f700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 0x7f7f7c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b5c0 tx=0x7f7f80005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:11.279 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.276+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7f8809c680 con 0x7f7f90106250 2024-04-02T13:32:11.484 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.481+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}) -- 0x7f7f90062b30 con 0x7f7f7c0a3cc0 2024-04-02T13:32:11.492 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.489+0000 7f7f86ffd700 1 -- 172.21.15.188:0/2943104644 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7f90062b30 con 0x7f7f7c0a3cc0 2024-04-02T13:32:11.493 INFO:teuthology.orchestra.run.smithi188.stdout:Scheduled mds.all update... 2024-04-02T13:32:11.495 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 msgr2=0x7f7f7c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:11.495 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 0x7f7f7c0a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000b5c0 tx=0x7f7f80005fb0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.495 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 msgr2=0x7f7f901a6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:11.495 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f8800b9a0 tx=0x7f7f8800bd60 comp rx=0 tx=0).stop 2024-04-02T13:32:11.495 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 shutdown_connections 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f90106250 0x7f7f901a6490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f7c0a3cc0 0x7f7f7c0a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f90100f10 0x7f7f9019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 --2- 172.21.15.188:0/2943104644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f90103880 0x7f7f9019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 >> 172.21.15.188:0/2943104644 conn(0x7f7f900fcb50 msgr2=0x7f7f90104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 shutdown_connections 2024-04-02T13:32:11.496 INFO:teuthology.orchestra.run.smithi188.stderr:2024-04-02T13:32:11.491+0000 7f7f97fb3700 1 -- 172.21.15.188:0/2943104644 wait complete. 2024-04-02T13:32:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:11 smithi187 ceph-mon[36064]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:11 smithi188 ceph-mon[36117]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:11 smithi122 ceph-mon[32114]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:12.180 DEBUG:teuthology.orchestra.run.smithi122:mds.a> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.a.service 2024-04-02T13:32:12.182 DEBUG:teuthology.orchestra.run.smithi122:mds.d> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.d.service 2024-04-02T13:32:12.225 DEBUG:teuthology.orchestra.run.smithi122:mds.g> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.g.service 2024-04-02T13:32:12.268 DEBUG:teuthology.orchestra.run.smithi122:mds.j> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.j.service 2024-04-02T13:32:12.311 DEBUG:teuthology.orchestra.run.smithi187:mds.b> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.b.service 2024-04-02T13:32:12.314 DEBUG:teuthology.orchestra.run.smithi187:mds.e> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.e.service 2024-04-02T13:32:12.357 DEBUG:teuthology.orchestra.run.smithi187:mds.h> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.h.service 2024-04-02T13:32:12.400 DEBUG:teuthology.orchestra.run.smithi187:mds.k> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.k.service 2024-04-02T13:32:12.443 DEBUG:teuthology.orchestra.run.smithi188:mds.c> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.c.service 2024-04-02T13:32:12.446 DEBUG:teuthology.orchestra.run.smithi188:mds.f> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.f.service 2024-04-02T13:32:12.489 DEBUG:teuthology.orchestra.run.smithi188:mds.i> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.i.service 2024-04-02T13:32:12.532 DEBUG:teuthology.orchestra.run.smithi188:mds.l> sudo journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.l.service 2024-04-02T13:32:12.575 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-02T13:32:12.575 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 -- 172.21.15.122:0/2226487940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6468101c10 msgr2=0x7f6468102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 --2- 172.21.15.122:0/2226487940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6468101c10 0x7f6468102090 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6450009900 tx=0x7f645002bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 -- 172.21.15.122:0/2226487940 shutdown_connections 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 --2- 172.21.15.122:0/2226487940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64681025d0 0x7f6468108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 --2- 172.21.15.122:0/2226487940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6468101c10 0x7f6468102090 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 --2- 172.21.15.122:0/2226487940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 0x7f6468100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 -- 172.21.15.122:0/2226487940 >> 172.21.15.122:0/2226487940 conn(0x7f64680fbfb0 msgr2=0x7f64680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 -- 172.21.15.122:0/2226487940 shutdown_connections 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.680+0000 7f646d522640 1 -- 172.21.15.122:0/2226487940 wait complete. 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 Processor -- start 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 -- start start 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 0x7f6468197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6468101c10 0x7f6468198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f646810d7c0 con 0x7f64681025d0 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f646810d640 con 0x7f6468100a20 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f646d522640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f646810d940 con 0x7f6468101c10 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49594/0 (socket says 172.21.15.122:49594) 2024-04-02T13:32:12.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 -- 172.21.15.122:0/4199183656 learned_addr learned my addr 172.21.15.122:0/4199183656 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64667fc640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6468101c10 0x7f6468198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f6466ffd640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 0x7f6468197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 -- 172.21.15.122:0/4199183656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6468101c10 msgr2=0x7f6468198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6468101c10 0x7f6468198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 -- 172.21.15.122:0/4199183656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 msgr2=0x7f6468197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 0x7f6468197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 -- 172.21.15.122:0/4199183656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646819fa40 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.681+0000 7f64677fe640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f645800d950 tx=0x7f645800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.682+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458017030 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.682+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6458004020 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.682+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646819fd30 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.682+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6458003c00 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.682+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64681a01f0 con 0x7f64681025d0 2024-04-02T13:32:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.683+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6468107d40 con 0x7f64681025d0 2024-04-02T13:32:12.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.684+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f645801a070 con 0x7f64681025d0 2024-04-02T13:32:12.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.684+0000 7f645ffff640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 0x7f64540a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.684+0000 7f6466ffd640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 0x7f64540a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.684+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f64580cf6d0 con 0x7f64681025d0 2024-04-02T13:32:12.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.685+0000 7f6466ffd640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 0x7f64540a60f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6468101a70 tx=0x7f6460009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:12.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.690+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6458021070 con 0x7f64681025d0 2024-04-02T13:32:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:12 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:12 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: Saving service mds.all spec with placement smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:12 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:12.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.869+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64680627a0 con 0x7f64681025d0 2024-04-02T13:32:12.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.870+0000 7f645ffff640 1 -- 172.21.15.122:0/4199183656 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18502 (secure 0 0 0) 0x7f645809c550 con 0x7f64681025d0 2024-04-02T13:32:12.874 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:12.874 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":82,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:01.165058+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:12.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 msgr2=0x7f64540a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 0x7f64540a60f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6468101a70 tx=0x7f6460009500 comp rx=0 tx=0).stop 2024-04-02T13:32:12.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 msgr2=0x7f646819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f645800d950 tx=0x7f645800de10 comp rx=0 tx=0).stop 2024-04-02T13:32:12.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 shutdown_connections 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64681025d0 0x7f646819f3d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6468101c10 0x7f6468198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64540a3c30 0x7f64540a60f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 --2- 172.21.15.122:0/4199183656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6468100a20 0x7f6468197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 >> 172.21.15.122:0/4199183656 conn(0x7f64680fbfb0 msgr2=0x7f6468104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 shutdown_connections 2024-04-02T13:32:12.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.873+0000 7f646d522640 1 -- 172.21.15.122:0/4199183656 wait complete. 2024-04-02T13:32:12.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:32:12.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- 172.21.15.122:0/2029317289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 msgr2=0x7fd130100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 --2- 172.21.15.122:0/2029317289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130100e30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009900 tx=0x7fd12c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- 172.21.15.122:0/2029317289 shutdown_connections 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 --2- 172.21.15.122:0/2029317289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1301025c0 0x7fd130108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 --2- 172.21.15.122:0/2029317289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd130101c00 0x7fd130102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 --2- 172.21.15.122:0/2029317289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130100e30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- 172.21.15.122:0/2029317289 >> 172.21.15.122:0/2029317289 conn(0x7fd1300fbfa0 msgr2=0x7fd1300fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- 172.21.15.122:0/2029317289 shutdown_connections 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- 172.21.15.122:0/2029317289 wait complete. 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 Processor -- start 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.989+0000 7fd1373eb640 1 -- start start 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd130101c00 0x7fd130198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 0x7fd13019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd13010d7b0 con 0x7fd130100a10 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd13010d630 con 0x7fd1301025c0 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd13010d930 con 0x7fd130101c00 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49606/0 (socket says 172.21.15.122:49606) 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 -- 172.21.15.122:0/1554736061 learned_addr learned my addr 172.21.15.122:0/1554736061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd134986640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd130101c00 0x7fd130198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135988640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 0x7fd13019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 -- 172.21.15.122:0/1554736061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd130101c00 msgr2=0x7fd130198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd130101c00 0x7fd130198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 -- 172.21.15.122:0/1554736061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 msgr2=0x7fd13019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 0x7fd13019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 -- 172.21.15.122:0/1554736061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13019f9f0 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135988640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 0x7fd13019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd135187640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005ab0 tx=0x7fd12c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c041070 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd12c0054e0 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13019fc80 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c034440 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.990+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1301a0170 con 0x7fd130100a10 2024-04-02T13:32:12.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.991+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd130107d80 con 0x7fd130100a10 2024-04-02T13:32:12.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.992+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd12c044070 con 0x7fd130100a10 2024-04-02T13:32:12.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.993+0000 7fd1267fc640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 0x7fd11c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:12.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.993+0000 7fd134986640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 0x7fd11c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:12.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.993+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd12c0f1da0 con 0x7fd130100a10 2024-04-02T13:32:12.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.993+0000 7fd134986640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 0x7fd11c0a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1301992e0 tx=0x7fd12000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:13.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:12.997+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd12c0bec20 con 0x7fd130100a10 2024-04-02T13:32:13.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.175+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd13002d500 con 0x7fd130100a10 2024-04-02T13:32:13.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.176+0000 7fd1267fc640 1 -- 172.21.15.122:0/1554736061 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fd12c0be520 con 0x7fd130100a10 2024-04-02T13:32:13.179 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 msgr2=0x7fd11c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 0x7fd11c0a6070 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd1301992e0 tx=0x7fd12000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 msgr2=0x7fd130197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005ab0 tx=0x7fd12c005dc0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 shutdown_connections 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd11c0a3bb0 0x7fd11c0a6070 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1301025c0 0x7fd13019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd130101c00 0x7fd130198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 --2- 172.21.15.122:0/1554736061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd130100a10 0x7fd130197dd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 >> 172.21.15.122:0/1554736061 conn(0x7fd1300fbfa0 msgr2=0x7fd130104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:13.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 shutdown_connections 2024-04-02T13:32:13.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.178+0000 7fd1373eb640 1 -- 172.21.15.122:0/1554736061 wait complete. 2024-04-02T13:32:13.192 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:13.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 -- 172.21.15.122:0/2607027430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 msgr2=0x7f39e0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 --2- 172.21.15.122:0/2607027430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e0100e40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f39dc009900 tx=0x7f39dc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 -- 172.21.15.122:0/2607027430 shutdown_connections 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 --2- 172.21.15.122:0/2607027430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e01025d0 0x7f39e0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 --2- 172.21.15.122:0/2607027430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e0101c10 0x7f39e0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 --2- 172.21.15.122:0/2607027430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e0100e40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 -- 172.21.15.122:0/2607027430 >> 172.21.15.122:0/2607027430 conn(0x7f39e00fbf90 msgr2=0x7f39e00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 -- 172.21.15.122:0/2607027430 shutdown_connections 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.289+0000 7f39e73f2640 1 -- 172.21.15.122:0/2607027430 wait complete. 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 Processor -- start 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 -- start start 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:13.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 0x7f39e019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e01025d0 0x7f39e019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010c690 con 0x7f39e0100a20 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010c510 con 0x7f39e0101c10 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e73f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39e010c810 con 0x7f39e01025d0 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e518e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e518e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49630/0 (socket says 172.21.15.122:49630) 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e518e640 1 -- 172.21.15.122:0/1370019240 learned_addr learned my addr 172.21.15.122:0/1370019240 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e598f640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e01025d0 0x7f39e019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.290+0000 7f39e498d640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 0x7f39e019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 -- 172.21.15.122:0/1370019240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e01025d0 msgr2=0x7f39e019f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e01025d0 0x7f39e019f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 -- 172.21.15.122:0/1370019240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 msgr2=0x7f39e019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 0x7f39e019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 -- 172.21.15.122:0/1370019240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e019faf0 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e498d640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 0x7f39e019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e518e640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f39dc000c00 tx=0x7f39dc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc041070 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39dc0044d0 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc034440 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e019fd80 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.291+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01a0210 con 0x7f39e0100a20 2024-04-02T13:32:13.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.292+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0107e30 con 0x7f39e0100a20 2024-04-02T13:32:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.293+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f39dc044070 con 0x7f39e0100a20 2024-04-02T13:32:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.293+0000 7f39d67fc640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 0x7f39cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.294+0000 7f39e498d640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 0x7f39cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.294+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f39dc0f1cc0 con 0x7f39e0100a20 2024-04-02T13:32:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.294+0000 7f39e498d640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 0x7f39cc0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39e0069560 tx=0x7f39d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.297+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f39dc0beb40 con 0x7f39e0100a20 2024-04-02T13:32:13.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.514+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e002d530 con 0x7f39e0100a20 2024-04-02T13:32:13.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.514+0000 7f39d67fc640 1 -- 172.21.15.122:0/1370019240 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f39dc0be440 con 0x7f39e0100a20 2024-04-02T13:32:13.517 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:13.517 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-02T13:32:13.517 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 1 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 msgr2=0x7f39cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 0x7f39cc0a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f39e0069560 tx=0x7f39d0005f90 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 msgr2=0x7f39e00697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f39dc000c00 tx=0x7f39dc005d50 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 shutdown_connections 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39e01025d0 0x7f39e019f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39cc0a3bb0 0x7f39cc0a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39e0101c10 0x7f39e019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 --2- 172.21.15.122:0/1370019240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e0100a20 0x7f39e00697f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:13.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 >> 172.21.15.122:0/1370019240 conn(0x7f39e00fbf90 msgr2=0x7f39e0105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:13.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 shutdown_connections 2024-04-02T13:32:13.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:13.516+0000 7f39e73f2640 1 -- 172.21.15.122:0/1370019240 wait complete. 2024-04-02T13:32:13.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:13 smithi188 ceph-mon[36117]: Deploying daemon mds.l on smithi188 2024-04-02T13:32:13.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:13 smithi188 ceph-mon[36117]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:13.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4199183656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:13.755 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1554736061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:13 smithi122 ceph-mon[32114]: Deploying daemon mds.l on smithi188 2024-04-02T13:32:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:13 smithi122 ceph-mon[32114]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4199183656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1554736061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:13 smithi187 ceph-mon[36064]: Deploying daemon mds.l on smithi188 2024-04-02T13:32:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:13 smithi187 ceph-mon[36064]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4199183656' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1554736061' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:14.433 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 systemd[1]: Starting Ceph mds.l for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:14.694 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1370019240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:14.695 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 podman[66540]: 2024-04-02 13:32:14.43032964 +0000 UTC m=+0.103112567 container create e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T13:32:14.695 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 podman[66540]: 2024-04-02 13:32:14.34495333 +0000 UTC m=+0.017736243 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:14.695 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 podman[66540]: 2024-04-02 13:32:14.66651301 +0000 UTC m=+0.339295917 container init e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:32:14.695 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 podman[66540]: 2024-04-02 13:32:14.669064585 +0000 UTC m=+0.341847489 container start e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-02T13:32:14.695 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 ceph-mds[66560]: -- 172.21.15.188:0/3249706829 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b10f843a40 con 0x55b10f816c00 2024-04-02T13:32:14.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1370019240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:14.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1370019240' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:15.017 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: starting mds.l at 2024-04-02T13:32:15.017 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 bash[66540]: e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 2024-04-02T13:32:15.017 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:32:14 smithi188 systemd[1]: Started Ceph mds.l for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:15.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 -- 172.21.15.122:0/3768685759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa604103840 msgr2=0x7fa60410a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 --2- 172.21.15.122:0/3768685759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa604103840 0x7fa60410a0d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009cd0 tx=0x7fa5fc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 -- 172.21.15.122:0/3768685759 shutdown_connections 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 --2- 172.21.15.122:0/3768685759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa604103840 0x7fa60410a0d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 --2- 172.21.15.122:0/3768685759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 0x7fa604103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 --2- 172.21.15.122:0/3768685759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6040ffe10 0x7fa604100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 -- 172.21.15.122:0/3768685759 >> 172.21.15.122:0/3768685759 conn(0x7fa6040fb720 msgr2=0x7fa6040fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 -- 172.21.15.122:0/3768685759 shutdown_connections 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.641+0000 7fa60a109640 1 -- 172.21.15.122:0/3768685759 wait complete. 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 Processor -- start 2024-04-02T13:32:15.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 -- start start 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 0x7fa604198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa604103840 0x7fa60419f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa60410d790 con 0x7fa6040ffe10 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa60410d610 con 0x7fa604102e80 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa60a109640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa60410d910 con 0x7fa604103840 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa6037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa6037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49660/0 (socket says 172.21.15.122:49660) 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa6037fe640 1 -- 172.21.15.122:0/1866957438 learned_addr learned my addr 172.21.15.122:0/1866957438 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa603fff640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa604103840 0x7fa60419f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.642+0000 7fa602ffd640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 0x7fa604198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 -- 172.21.15.122:0/1866957438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa604103840 msgr2=0x7fa60419f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa604103840 0x7fa60419f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 -- 172.21.15.122:0/1866957438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 msgr2=0x7fa604198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 0x7fa604198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 -- 172.21.15.122:0/1866957438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60419f9f0 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa6037fe640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d770 tx=0x7fa5f400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4017070 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60419fce0 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4009c30 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4003dd0 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.643+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041a0180 con 0x7fa6040ffe10 2024-04-02T13:32:15.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.644+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa604108fb0 con 0x7fa6040ffe10 2024-04-02T13:32:15.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.645+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa5f4004000 con 0x7fa6040ffe10 2024-04-02T13:32:15.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.646+0000 7fa600ff9640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 0x7fa5ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:15.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.646+0000 7fa602ffd640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 0x7fa5ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:15.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.646+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa5f40cf840 con 0x7fa6040ffe10 2024-04-02T13:32:15.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.647+0000 7fa602ffd640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 0x7fa5ec0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6041992e0 tx=0x7fa5f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:15.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.651+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa5f4009de0 con 0x7fa6040ffe10 2024-04-02T13:32:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6832/546075479,v1:172.21.15.188:6833/546075479] up:boot 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: fsmap 1 up:standby 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:15 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6832/546075479,v1:172.21.15.188:6833/546075479] up:boot 2024-04-02T13:32:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: fsmap 1 up:standby 2024-04-02T13:32:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:32:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:15.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:15 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6832/546075479,v1:172.21.15.188:6833/546075479] up:boot 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: fsmap 1 up:standby 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:15 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.875+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa60402d370 con 0x7fa6040ffe10 2024-04-02T13:32:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.876+0000 7fa600ff9640 1 -- 172.21.15.122:0/1866957438 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fa60402d370 con 0x7fa6040ffe10 2024-04-02T13:32:15.879 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:15.879 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:32:15.879 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 2 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 msgr2=0x7fa5ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 0x7fa5ec0a6070 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa6041992e0 tx=0x7fa5f8005f90 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 msgr2=0x7fa604197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d770 tx=0x7fa5f400dc30 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 shutdown_connections 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa604103840 0x7fa60419f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa5ec0a3bb0 0x7fa5ec0a6070 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa604102e80 0x7fa604198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 --2- 172.21.15.122:0/1866957438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6040ffe10 0x7fa604197dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 >> 172.21.15.122:0/1866957438 conn(0x7fa6040fb720 msgr2=0x7fa604106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 shutdown_connections 2024-04-02T13:32:15.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:15.878+0000 7fa60a109640 1 -- 172.21.15.122:0/1866957438 wait complete. 2024-04-02T13:32:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:16 smithi187 ceph-mon[36064]: Deploying daemon mds.k on smithi187 2024-04-02T13:32:16.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:16 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1866957438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:16 smithi188 ceph-mon[36117]: Deploying daemon mds.k on smithi187 2024-04-02T13:32:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:16 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1866957438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:16.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:16 smithi122 ceph-mon[32114]: Deploying daemon mds.k on smithi187 2024-04-02T13:32:16.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:16 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1866957438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:17.236 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 systemd[1]: Starting Ceph mds.k for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:17.555 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:17 smithi187 ceph-mon[36064]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:17.556 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 podman[66014]: 2024-04-02 13:32:17.165716195 +0000 UTC m=+0.017570876 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:17.556 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 podman[66014]: 2024-04-02 13:32:17.283824007 +0000 UTC m=+0.135678667 container create 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T13:32:17.556 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 podman[66014]: 2024-04-02 13:32:17.553486058 +0000 UTC m=+0.405340740 container init 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T13:32:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:17 smithi188 ceph-mon[36117]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:17 smithi122 ceph-mon[32114]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:17.892 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:17.985 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 podman[66014]: 2024-04-02 13:32:17.55628679 +0000 UTC m=+0.408141459 container start 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-02T13:32:17.986 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 ceph-mds[66033]: -- 172.21.15.187:0/4214094585 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fdb9a0ba40 con 0x55fdb99de800 2024-04-02T13:32:17.986 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: starting mds.k at 2024-04-02T13:32:17.986 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 bash[66014]: 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf 2024-04-02T13:32:17.986 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:32:17 smithi187 systemd[1]: Started Ceph mds.k for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 -- 172.21.15.122:0/2788823544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 msgr2=0x7f3a541072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 --2- 172.21.15.122:0/2788823544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541072c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009900 tx=0x7f3a3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:18.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 -- 172.21.15.122:0/2788823544 shutdown_connections 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 --2- 172.21.15.122:0/2788823544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a54109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 --2- 172.21.15.122:0/2788823544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541072c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 --2- 172.21.15.122:0/2788823544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 0x7f3a54104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 -- 172.21.15.122:0/2788823544 >> 172.21.15.122:0/2788823544 conn(0x7f3a540fbf90 msgr2=0x7f3a540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 -- 172.21.15.122:0/2788823544 shutdown_connections 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.995+0000 7f3a59041640 1 -- 172.21.15.122:0/2788823544 wait complete. 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 Processor -- start 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 -- start start 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 0x7f3a54197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:18.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a5419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410d790 con 0x7f3a54104ed0 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410d610 con 0x7f3a54107800 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a59041640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a5410d910 con 0x7f3a541025a0 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a5359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a5419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49684/0 (socket says 172.21.15.122:49684) 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.996+0000 7f3a4bfff640 1 -- 172.21.15.122:0/520546724 learned_addr learned my addr 172.21.15.122:0/520546724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a52d9d640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 0x7f3a54197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 -- 172.21.15.122:0/520546724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 msgr2=0x7f3a54197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 0x7f3a54197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 -- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 msgr2=0x7f3a541983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 -- 172.21.15.122:0/520546724 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5419fa80 con 0x7f3a54107800 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a5359e640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a5419f410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fd0 tx=0x7f3a4c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c00ce20 con 0x7f3a54107800 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a4c005020 con 0x7f3a54107800 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a5419fd70 con 0x7f3a54107800 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4c010720 con 0x7f3a54107800 2024-04-02T13:32:18.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.997+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a541a01e0 con 0x7f3a54107800 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.998+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a540fda80 con 0x7f3a54107800 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.999+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3a4c004b20 con 0x7f3a54107800 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.999+0000 7f3a50d99640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 0x7f3a400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:17.999+0000 7f3a52d9d640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 0x7f3a400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.000+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3a4c0d0710 con 0x7f3a54107800 2024-04-02T13:32:18.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.000+0000 7f3a52d9d640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 0x7f3a400a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005d00 tx=0x7f3a44005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:18.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.004+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a4c09d4f0 con 0x7f3a54107800 2024-04-02T13:32:18.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.226+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a540627a0 con 0x7f3a54107800 2024-04-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.227+0000 7f3a50d99640 1 -- 172.21.15.122:0/520546724 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7f3a4c09cdf0 con 0x7f3a54107800 2024-04-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:18.229 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-02T13:32:18.230 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 2 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 msgr2=0x7f3a400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 0x7f3a400a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a44005d00 tx=0x7f3a44005c50 comp rx=0 tx=0).stop 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 msgr2=0x7f3a5419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a5419f410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009fd0 tx=0x7f3a4c00ef10 comp rx=0 tx=0).stop 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 shutdown_connections 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a400a3bb0 0x7f3a400a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a54107800 0x7f3a5419f410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a54104ed0 0x7f3a541983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 --2- 172.21.15.122:0/520546724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a541025a0 0x7f3a54197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 >> 172.21.15.122:0/520546724 conn(0x7f3a540fbf90 msgr2=0x7f3a54103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 shutdown_connections 2024-04-02T13:32:18.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:18.229+0000 7f3a59041640 1 -- 172.21.15.122:0/520546724 wait complete. 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: Deploying daemon mds.j on smithi122 2024-04-02T13:32:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/520546724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:18.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:18.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6832/1223173792,v1:172.21.15.187:6833/1223173792] up:boot 2024-04-02T13:32:18.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: fsmap 2 up:standby 2024-04-02T13:32:18.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:18 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: Deploying daemon mds.j on smithi122 2024-04-02T13:32:18.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/520546724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:18.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:18.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6832/1223173792,v1:172.21.15.187:6833/1223173792] up:boot 2024-04-02T13:32:18.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: fsmap 2 up:standby 2024-04-02T13:32:18.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:18 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:32:19.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: Deploying daemon mds.j on smithi122 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/520546724' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6832/1223173792,v1:172.21.15.187:6833/1223173792] up:boot 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: fsmap 2 up:standby 2024-04-02T13:32:19.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:18 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:32:20.225 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 systemd[1]: Starting Ceph mds.j for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:20.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- 172.21.15.122:0/432591493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc103110 msgr2=0x7f60fc103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 --2- 172.21.15.122:0/432591493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc103110 0x7f60fc103530 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009900 tx=0x7f60f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- 172.21.15.122:0/432591493 shutdown_connections 2024-04-02T13:32:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 --2- 172.21.15.122:0/432591493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc100540 0x7f60fc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 --2- 172.21.15.122:0/432591493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc0ffb80 0x7f60fc100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 --2- 172.21.15.122:0/432591493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc103110 0x7f60fc103530 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- 172.21.15.122:0/432591493 >> 172.21.15.122:0/432591493 conn(0x7f60fc0fb700 msgr2=0x7f60fc0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- 172.21.15.122:0/432591493 shutdown_connections 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- 172.21.15.122:0/432591493 wait complete. 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 Processor -- start 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.348+0000 7f6103b15640 1 -- start start 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc100540 0x7f60fc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc103110 0x7f60fc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10d770 con 0x7f60fc0ffb80 2024-04-02T13:32:20.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10d5f0 con 0x7f60fc100540 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc10d8f0 con 0x7f60fc103110 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33686/0 (socket says 172.21.15.122:33686) 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 -- 172.21.15.122:0/2009797349 learned_addr learned my addr 172.21.15.122:0/2009797349 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61010b0640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc100540 0x7f60fc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61020b2640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc103110 0x7f60fc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 -- 172.21.15.122:0/2009797349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc103110 msgr2=0x7f60fc19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc103110 0x7f60fc19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 -- 172.21.15.122:0/2009797349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc100540 msgr2=0x7f60fc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc100540 0x7f60fc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 -- 172.21.15.122:0/2009797349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc19fa30 con 0x7f60fc0ffb80 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f61018b1640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005ab0 tx=0x7f60f8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8041070 con 0x7f60fc0ffb80 2024-04-02T13:32:20.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60f80054e0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc19fcc0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f8034440 con 0x7f60fc0ffb80 2024-04-02T13:32:20.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.349+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc1a01b0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.350+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc108fd0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.351+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f60f8044070 con 0x7f60fc0ffb80 2024-04-02T13:32:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.352+0000 7f60f2ffd640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 0x7f60e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.352+0000 7f61010b0640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 0x7f60e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.352+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f60f80f1c70 con 0x7f60fc0ffb80 2024-04-02T13:32:20.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.353+0000 7f61010b0640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 0x7f60e80a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60fc199320 tx=0x7f60ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:20.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.356+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f60f80be4a0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.502 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 podman[70094]: 2024-04-02 13:32:20.124728851 +0000 UTC m=+0.017569815 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:20.503 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 podman[70094]: 2024-04-02 13:32:20.230048409 +0000 UTC m=+0.122889359 container create 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T13:32:20.503 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 podman[70094]: 2024-04-02 13:32:20.500100343 +0000 UTC m=+0.392941305 container init 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T13:32:20.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.582+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60fc02d370 con 0x7f60fc0ffb80 2024-04-02T13:32:20.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.582+0000 7f60f2ffd640 1 -- 172.21.15.122:0/2009797349 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f60f80c27a0 con 0x7f60fc0ffb80 2024-04-02T13:32:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:20.585 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:32:20.586 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 3 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 msgr2=0x7f60e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 0x7f60e80a60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60fc199320 tx=0x7f60ec00b390 comp rx=0 tx=0).stop 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 msgr2=0x7f60fc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60f8005ab0 tx=0x7f60f8005dc0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 shutdown_connections 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc103110 0x7f60fc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f60e80a3c30 0x7f60e80a60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f60fc100540 0x7f60fc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 --2- 172.21.15.122:0/2009797349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f60fc0ffb80 0x7f60fc197e10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 >> 172.21.15.122:0/2009797349 conn(0x7f60fc0fb700 msgr2=0x7f60fc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 shutdown_connections 2024-04-02T13:32:20.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:20.584+0000 7f6103b15640 1 -- 172.21.15.122:0/2009797349 wait complete. 2024-04-02T13:32:20.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 podman[70094]: 2024-04-02 13:32:20.503293284 +0000 UTC m=+0.396134241 container start 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:32:20.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 ceph-mds[70153]: -- 172.21.15.122:0/1021230070 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e76061ba40 con 0x55e7605ee800 2024-04-02T13:32:20.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j[70149]: starting mds.j at 2024-04-02T13:32:20.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 bash[70094]: 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 2024-04-02T13:32:20.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 13:32:20 smithi122 systemd[1]: Started Ceph mds.j for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2009797349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:21.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:21 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2009797349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:21 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2009797349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:21 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:22.598 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:22.729 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:22 smithi122 ceph-mon[32114]: Deploying daemon mds.d on smithi122 2024-04-02T13:32:22.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:22 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6834/2838975565,v1:172.21.15.122:6835/2838975565] up:boot 2024-04-02T13:32:22.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:22 smithi122 ceph-mon[32114]: fsmap 3 up:standby 2024-04-02T13:32:22.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:22 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:32:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:22 smithi188 ceph-mon[36117]: Deploying daemon mds.d on smithi122 2024-04-02T13:32:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:22 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6834/2838975565,v1:172.21.15.122:6835/2838975565] up:boot 2024-04-02T13:32:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:22 smithi188 ceph-mon[36117]: fsmap 3 up:standby 2024-04-02T13:32:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:22 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3522018810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 msgr2=0x7f4d68102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3522018810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d68102090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009900 tx=0x7f4d5802bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3522018810 shutdown_connections 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3522018810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d681025d0 0x7f4d68108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3522018810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d68102090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3522018810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d68100a20 0x7f4d68100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3522018810 >> 172.21.15.122:0/3522018810 conn(0x7f4d680fbf90 msgr2=0x7f4d680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3522018810 shutdown_connections 2024-04-02T13:32:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.813+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3522018810 wait complete. 2024-04-02T13:32:22.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 Processor -- start 2024-04-02T13:32:22.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 -- start start 2024-04-02T13:32:22.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 0x7f4d6810d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:22.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d681025d0 0x7f4d6810c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68069910 con 0x7f4d68101c10 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68069790 con 0x7f4d68100a20 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.814+0000 7f4d6dd74640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d68069a90 con 0x7f4d681025d0 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33706/0 (socket says 172.21.15.122:33706) 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 -- 172.21.15.122:0/3521344149 learned_addr learned my addr 172.21.15.122:0/3521344149 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d67fff640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d681025d0 0x7f4d6810c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d677fe640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 0x7f4d6810d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 -- 172.21.15.122:0/3521344149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d681025d0 msgr2=0x7f4d6810c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d681025d0 0x7f4d6810c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 -- 172.21.15.122:0/3521344149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 msgr2=0x7f4d6810d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 0x7f4d6810d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 -- 172.21.15.122:0/3521344149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6810c9d0 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d677fe640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 0x7f4d6810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d66ffd640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000c00 tx=0x7f4d58005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58043020 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d681a1ef0 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d580054e0 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58003e60 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.815+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d681a2360 con 0x7f4d68101c10 2024-04-02T13:32:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.816+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d68107d40 con 0x7f4d68101c10 2024-04-02T13:32:22.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.817+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4d58046020 con 0x7f4d68101c10 2024-04-02T13:32:22.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.818+0000 7f4d64ff9640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 0x7f4d500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:22.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.818+0000 7f4d677fe640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 0x7f4d500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:22.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.818+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4d580f25a0 con 0x7f4d68101c10 2024-04-02T13:32:22.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.818+0000 7f4d677fe640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 0x7f4d500a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d68101a70 tx=0x7f4d5c005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:22.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:22.822+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4d5804d020 con 0x7f4d68101c10 2024-04-02T13:32:22.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:22 smithi187 ceph-mon[36064]: Deploying daemon mds.d on smithi122 2024-04-02T13:32:22.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:22 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6834/2838975565,v1:172.21.15.122:6835/2838975565] up:boot 2024-04-02T13:32:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:22 smithi187 ceph-mon[36064]: fsmap 3 up:standby 2024-04-02T13:32:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:22 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:32:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.054+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d6802d410 con 0x7f4d68101c10 2024-04-02T13:32:23.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.055+0000 7f4d64ff9640 1 -- 172.21.15.122:0/3521344149 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7f4d580bf380 con 0x7f4d68101c10 2024-04-02T13:32:23.058 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:23.058 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:32:23.058 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 4 2024-04-02T13:32:23.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 msgr2=0x7f4d500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:23.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 0x7f4d500a6070 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4d68101a70 tx=0x7f4d5c005e70 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 msgr2=0x7f4d6810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000c00 tx=0x7f4d58005dd0 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 shutdown_connections 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d681025d0 0x7f4d6810c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d68101c10 0x7f4d6810bc00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d500a3bb0 0x7f4d500a6070 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 --2- 172.21.15.122:0/3521344149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d68100a20 0x7f4d6810d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 >> 172.21.15.122:0/3521344149 conn(0x7f4d680fbf90 msgr2=0x7f4d68104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 shutdown_connections 2024-04-02T13:32:23.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:23.057+0000 7f4d6dd74640 1 -- 172.21.15.122:0/3521344149 wait complete. 2024-04-02T13:32:23.070 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:22 smithi122 systemd[1]: Starting Ceph mds.d for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:23.377 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 podman[70484]: 2024-04-02 13:32:23.155478191 +0000 UTC m=+0.111774381 container create 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS) 2024-04-02T13:32:23.378 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 podman[70484]: 2024-04-02 13:32:23.064719164 +0000 UTC m=+0.021015320 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:23.378 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 podman[70484]: 2024-04-02 13:32:23.375177915 +0000 UTC m=+0.331474062 container init 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:23 smithi122 ceph-mon[32114]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:23 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3521344149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 podman[70484]: 2024-04-02 13:32:23.378020724 +0000 UTC m=+0.334316874 container start 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 ceph-mds[70505]: -- 172.21.15.122:0/2116589799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559b30a9da40 con 0x559b30a70800 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d[70501]: starting mds.d at 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 bash[70484]: 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d 2024-04-02T13:32:23.821 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 13:32:23 smithi122 systemd[1]: Started Ceph mds.d for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:23 smithi187 ceph-mon[36064]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:23 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3521344149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:23 smithi188 ceph-mon[36117]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:23 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3521344149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6836/1110044990,v1:172.21.15.122:6837/1110044990] up:boot 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: fsmap 4 up:standby 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:24.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:24.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:24 smithi122 ceph-mon[32114]: Deploying daemon mds.e on smithi187 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6836/1110044990,v1:172.21.15.122:6837/1110044990] up:boot 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: fsmap 4 up:standby 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:24 smithi187 ceph-mon[36064]: Deploying daemon mds.e on smithi187 2024-04-02T13:32:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6836/1110044990,v1:172.21.15.122:6837/1110044990] up:boot 2024-04-02T13:32:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: fsmap 4 up:standby 2024-04-02T13:32:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:25.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:24 smithi188 ceph-mon[36117]: Deploying daemon mds.e on smithi187 2024-04-02T13:32:25.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- 172.21.15.122:0/759884269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188100a20 msgr2=0x7f5188100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 --2- 172.21.15.122:0/759884269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188100a20 0x7f5188100e40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5184009ca0 tx=0x7f518402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- 172.21.15.122:0/759884269 shutdown_connections 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 --2- 172.21.15.122:0/759884269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51881025d0 0x7f5188108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 --2- 172.21.15.122:0/759884269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5188101c10 0x7f5188102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 --2- 172.21.15.122:0/759884269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188100a20 0x7f5188100e40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- 172.21.15.122:0/759884269 >> 172.21.15.122:0/759884269 conn(0x7f51880fbf90 msgr2=0x7f51880fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- 172.21.15.122:0/759884269 shutdown_connections 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- 172.21.15.122:0/759884269 wait complete. 2024-04-02T13:32:25.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 Processor -- start 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 -- start start 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.192+0000 7f518f80d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5188100a20 0x7f5188197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 0x7f518819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d7c0 con 0x7f5188101c10 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d640 con 0x7f5188100a20 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f518810d940 con 0x7f51881025d0 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33752/0 (socket says 172.21.15.122:33752) 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 -- 172.21.15.122:0/805414530 learned_addr learned my addr 172.21.15.122:0/805414530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518d5a9640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5188100a20 0x7f5188197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518ddaa640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 0x7f518819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:25.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 -- 172.21.15.122:0/805414530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 msgr2=0x7f518819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 0x7f518819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 -- 172.21.15.122:0/805414530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5188100a20 msgr2=0x7f5188197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5188100a20 0x7f5188197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 -- 172.21.15.122:0/805414530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518819fa90 con 0x7f5188101c10 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518ddaa640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 0x7f518819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518cda8640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f517800d770 tx=0x7f517800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178017070 con 0x7f5188101c10 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5178005500 con 0x7f5188101c10 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518819fd80 con 0x7f5188101c10 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5178004e00 con 0x7f5188101c10 2024-04-02T13:32:25.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.193+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51881a0220 con 0x7f5188101c10 2024-04-02T13:32:25.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.195+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5188107d90 con 0x7f5188101c10 2024-04-02T13:32:25.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.195+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f5178004fa0 con 0x7f5188101c10 2024-04-02T13:32:25.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.196+0000 7f517e7fc640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 0x7f51740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:25.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.196+0000 7f518d5a9640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 0x7f51740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:25.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.196+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f51780cf9a0 con 0x7f5188101c10 2024-04-02T13:32:25.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.197+0000 7f518d5a9640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 0x7f51740a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f518402c700 tx=0x7f5184032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:25.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.201+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f517809c820 con 0x7f5188101c10 2024-04-02T13:32:25.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.416+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f518802d370 con 0x7f5188101c10 2024-04-02T13:32:25.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.416+0000 7f517e7fc640 1 -- 172.21.15.122:0/805414530 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f517809c120 con 0x7f5188101c10 2024-04-02T13:32:25.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:25.419 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-02T13:32:25.420 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 5 2024-04-02T13:32:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 msgr2=0x7f51740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 0x7f51740a6070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f518402c700 tx=0x7f5184032590 comp rx=0 tx=0).stop 2024-04-02T13:32:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 msgr2=0x7f51881983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f517800d770 tx=0x7f517800dc30 comp rx=0 tx=0).stop 2024-04-02T13:32:25.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 shutdown_connections 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51881025d0 0x7f518819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5188101c10 0x7f51881983b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51740a3bb0 0x7f51740a6070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 --2- 172.21.15.122:0/805414530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5188100a20 0x7f5188197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 >> 172.21.15.122:0/805414530 conn(0x7f51880fbf90 msgr2=0x7f5188104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 shutdown_connections 2024-04-02T13:32:25.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:25.418+0000 7f518f80d640 1 -- 172.21.15.122:0/805414530 wait complete. 2024-04-02T13:32:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:25 smithi122 ceph-mon[32114]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:25 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/805414530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:25.943 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:25 smithi187 ceph-mon[36064]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:25.944 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:25 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/805414530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:25.944 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:25 smithi187 systemd[1]: Starting Ceph mds.e for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:25.944 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:25 smithi187 podman[66330]: 2024-04-02 13:32:25.843132318 +0000 UTC m=+0.017804807 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:25 smithi188 ceph-mon[36117]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:25 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/805414530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:26.219 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:25 smithi187 podman[66330]: 2024-04-02 13:32:25.956559489 +0000 UTC m=+0.131232024 container create 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 podman[66330]: 2024-04-02 13:32:26.216939561 +0000 UTC m=+0.391612056 container init 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 podman[66330]: 2024-04-02 13:32:26.219737311 +0000 UTC m=+0.394409791 container start 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 ceph-mds[66349]: -- 172.21.15.187:0/732906720 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565cc323a40 con 0x5565cc2f6c00 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: starting mds.e at 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 bash[66330]: 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 2024-04-02T13:32:26.486 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:32:26 smithi187 systemd[1]: Started Ceph mds.e for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:27.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:27.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 -- 172.21.15.122:0/445067495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 msgr2=0x7fdefc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:27.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/445067495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc1031c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009900 tx=0x7fdee402bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 -- 172.21.15.122:0/445067495 shutdown_connections 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/445067495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 0x7fdefc109f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/445067495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc1031c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/445067495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 -- 172.21.15.122:0/445067495 >> 172.21.15.122:0/445067495 conn(0x7fdefc0fd0a0 msgr2=0x7fdefc0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 -- 172.21.15.122:0/445067495 shutdown_connections 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 -- 172.21.15.122:0/445067495 wait complete. 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.549+0000 7fdf01f1b640 1 Processor -- start 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 -- start start 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:27.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 0x7fdefc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc10d7d0 con 0x7fdefc102d40 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc10d650 con 0x7fdefc101b50 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdf01f1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc10d950 con 0x7fdefc103700 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33780/0 (socket says 172.21.15.122:33780) 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 -- 172.21.15.122:0/434663619 learned_addr learned my addr 172.21.15.122:0/434663619 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefb7fe640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefbfff640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 0x7fdefc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 -- 172.21.15.122:0/434663619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 msgr2=0x7fdefc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:27.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 0x7fdefc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 -- 172.21.15.122:0/434663619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 msgr2=0x7fdefc197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefaffd640 1 -- 172.21.15.122:0/434663619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc19fa40 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.550+0000 7fdefb7fe640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdefaffd640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009230 tx=0x7fdee4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4041070 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdee40044d0 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4034440 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc19fcd0 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.551+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc1a01c0 con 0x7fdefc102d40 2024-04-02T13:32:27.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.552+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc108e70 con 0x7fdefc102d40 2024-04-02T13:32:27.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.553+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdee4044070 con 0x7fdefc102d40 2024-04-02T13:32:27.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.554+0000 7fdef8ff9640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 0x7fdee80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:27.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.554+0000 7fdefb7fe640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 0x7fdee80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.554+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdee40f1f60 con 0x7fdefc102d40 2024-04-02T13:32:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.554+0000 7fdefb7fe640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 0x7fdee80a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdefc102ba0 tx=0x7fdeec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.557+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdee40bede0 con 0x7fdefc102d40 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: Deploying daemon mds.f on smithi188 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6834/4231455845,v1:172.21.15.187:6835/4231455845] up:boot 2024-04-02T13:32:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: fsmap 5 up:standby 2024-04-02T13:32:27.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:27 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:32:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: Deploying daemon mds.f on smithi188 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6834/4231455845,v1:172.21.15.187:6835/4231455845] up:boot 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: fsmap 5 up:standby 2024-04-02T13:32:27.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:27 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:32:27.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.772+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdefc02d370 con 0x7fdefc102d40 2024-04-02T13:32:27.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.773+0000 7fdef8ff9640 1 -- 172.21.15.122:0/434663619 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7fdee40be6e0 con 0x7fdefc102d40 2024-04-02T13:32:27.776 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:27.776 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:32:27.776 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 msgr2=0x7fdee80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 0x7fdee80a60f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdefc102ba0 tx=0x7fdeec009500 comp rx=0 tx=0).stop 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 msgr2=0x7fdefc198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009230 tx=0x7fdee4005d50 comp rx=0 tx=0).stop 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 shutdown_connections 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdefc103700 0x7fdefc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdefc102d40 0x7fdefc198360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdee80a3c30 0x7fdee80a60f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 --2- 172.21.15.122:0/434663619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdefc101b50 0x7fdefc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 >> 172.21.15.122:0/434663619 conn(0x7fdefc0fd0a0 msgr2=0x7fdefc106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 shutdown_connections 2024-04-02T13:32:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:27.775+0000 7fdf01f1b640 1 -- 172.21.15.122:0/434663619 wait complete. 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: Deploying daemon mds.f on smithi188 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6834/4231455845,v1:172.21.15.187:6835/4231455845] up:boot 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: fsmap 5 up:standby 2024-04-02T13:32:27.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:27 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:32:28.720 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:28 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/434663619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:28.721 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 systemd[1]: Starting Ceph mds.f for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:28.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:28 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/434663619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:28.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:28 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/434663619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 podman[66854]: 2024-04-02 13:32:28.717660934 +0000 UTC m=+0.112757774 container create 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 podman[66854]: 2024-04-02 13:32:28.622077359 +0000 UTC m=+0.017174208 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 podman[66854]: 2024-04-02 13:32:28.936959105 +0000 UTC m=+0.332055945 container init 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 podman[66854]: 2024-04-02 13:32:28.939907185 +0000 UTC m=+0.335004026 container start 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 ceph-mds[66873]: -- 172.21.15.188:0/3029870287 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55718d219a40 con 0x55718d1ec800 2024-04-02T13:32:29.018 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: starting mds.f at 2024-04-02T13:32:29.421 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:29 smithi188 bash[66854]: 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e 2024-04-02T13:32:29.421 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:32:29 smithi188 systemd[1]: Started Ceph mds.f for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:29.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:29.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.707 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.708 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.708 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:29.708 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:29.708 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:29 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:29 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:29.791 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:29.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:29 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.899+0000 7f3ef763c640 1 -- 172.21.15.122:0/2421884995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0101b20 msgr2=0x7f3ef0101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.899+0000 7f3ef763c640 1 --2- 172.21.15.122:0/2421884995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0101b20 0x7f3ef0101f40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009900 tx=0x7f3eec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- 172.21.15.122:0/2421884995 shutdown_connections 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- 172.21.15.122:0/2421884995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 0x7f3ef0109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- 172.21.15.122:0/2421884995 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0102d10 0x7f3ef0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- 172.21.15.122:0/2421884995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0101b20 0x7f3ef0101f40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- 172.21.15.122:0/2421884995 >> 172.21.15.122:0/2421884995 conn(0x7f3ef00fd0b0 msgr2=0x7f3ef00ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- 172.21.15.122:0/2421884995 shutdown_connections 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- 172.21.15.122:0/2421884995 wait complete. 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 Processor -- start 2024-04-02T13:32:29.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- start start 2024-04-02T13:32:29.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 0x7f3ef0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 0x7f3ef019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010d7a0 con 0x7f3ef0102d10 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010d620 con 0x7f3ef0101b20 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.900+0000 7f3ef763c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef010d920 con 0x7f3ef01036d0 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40390/0 (socket says 172.21.15.122:40390) 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 -- 172.21.15.122:0/977278561 learned_addr learned my addr 172.21.15.122:0/977278561 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef5bd9640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 0x7f3ef019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef53d8640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 0x7f3ef0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 -- 172.21.15.122:0/977278561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 msgr2=0x7f3ef019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:29.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 0x7f3ef019f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 -- 172.21.15.122:0/977278561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 msgr2=0x7f3ef0197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 0x7f3ef0197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 -- 172.21.15.122:0/977278561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef019fa30 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef53d8640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 0x7f3ef0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef4bd7640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d950 tx=0x7f3ee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0017070 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ee0009c30 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee000f6b0 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef019fd20 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.901+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef01a0260 con 0x7f3ef0102d10 2024-04-02T13:32:29.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.902+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef0108e40 con 0x7f3ef0102d10 2024-04-02T13:32:29.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.903+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3ee00109d0 con 0x7f3ef0102d10 2024-04-02T13:32:29.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.904+0000 7f3ee67fc640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 0x7f3edc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:29.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.904+0000 7f3ef53d8640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 0x7f3edc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:29.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.904+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3ee00cfaf0 con 0x7f3ef0102d10 2024-04-02T13:32:29.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.905+0000 7f3ef53d8640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 0x7f3edc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3eec036040 tx=0x7f3eec000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:29.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:29.908+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ee0009de0 con 0x7f3ef0102d10 2024-04-02T13:32:30.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.122+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ef002d5c0 con 0x7f3ef0102d10 2024-04-02T13:32:30.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.122+0000 7f3ee67fc640 1 -- 172.21.15.122:0/977278561 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5530 (secure 0 0 0) 0x7f3ee009c9f0 con 0x7f3ef0102d10 2024-04-02T13:32:30.127 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:30.127 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 7 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 msgr2=0x7f3edc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 0x7f3edc0a60f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3eec036040 tx=0x7f3eec000bc0 comp rx=0 tx=0).stop 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 msgr2=0x7f3ef0198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000d950 tx=0x7f3ee000de10 comp rx=0 tx=0).stop 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 shutdown_connections 2024-04-02T13:32:30.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ef01036d0 0x7f3ef019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ef0102d10 0x7f3ef0198350 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3edc0a3c30 0x7f3edc0a60f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 --2- 172.21.15.122:0/977278561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ef0101b20 0x7f3ef0197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 >> 172.21.15.122:0/977278561 conn(0x7f3ef00fd0b0 msgr2=0x7f3ef0106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 shutdown_connections 2024-04-02T13:32:30.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:30.124+0000 7f3ef763c640 1 -- 172.21.15.122:0/977278561 wait complete. 2024-04-02T13:32:30.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:30 smithi188 ceph-mon[36117]: Deploying daemon mds.c on smithi188 2024-04-02T13:32:30.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:30 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6834/179772221,v1:172.21.15.188:6835/179772221] up:boot 2024-04-02T13:32:30.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:30 smithi188 ceph-mon[36117]: fsmap 6 up:standby 2024-04-02T13:32:30.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:30 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:32:30.699 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:30 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/977278561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:30 smithi187 ceph-mon[36064]: Deploying daemon mds.c on smithi188 2024-04-02T13:32:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:30 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6834/179772221,v1:172.21.15.188:6835/179772221] up:boot 2024-04-02T13:32:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:30 smithi187 ceph-mon[36064]: fsmap 6 up:standby 2024-04-02T13:32:30.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:30 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:32:30.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:30 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/977278561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:30 smithi122 ceph-mon[32114]: Deploying daemon mds.c on smithi188 2024-04-02T13:32:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:30 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6834/179772221,v1:172.21.15.188:6835/179772221] up:boot 2024-04-02T13:32:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:30 smithi122 ceph-mon[32114]: fsmap 6 up:standby 2024-04-02T13:32:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:30 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:32:30.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:30 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/977278561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:31.267 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 systemd[1]: Starting Ceph mds.c for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:31.662 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:31 smithi188 ceph-mon[36117]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:31.666 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 podman[67162]: 2024-04-02 13:32:31.301827411 +0000 UTC m=+0.017550447 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:31.666 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 podman[67162]: 2024-04-02 13:32:31.406664728 +0000 UTC m=+0.122387790 container create 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:32:31.666 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 podman[67162]: 2024-04-02 13:32:31.659853638 +0000 UTC m=+0.375576664 container init 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T13:32:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:31 smithi187 ceph-mon[36064]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:31 smithi122 ceph-mon[32114]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:32.017 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 podman[67162]: 2024-04-02 13:32:31.662548245 +0000 UTC m=+0.378271267 container start 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:32:32.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 ceph-mds[67181]: -- 172.21.15.188:0/1371794998 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5626c0e71a40 con 0x5626c0e44800 2024-04-02T13:32:32.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: starting mds.c at 2024-04-02T13:32:32.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 bash[67162]: 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 2024-04-02T13:32:32.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:32:31 smithi188 systemd[1]: Started Ceph mds.c for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:32.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:32.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.249+0000 7f2b941d4640 1 -- 172.21.15.122:0/2807932931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 msgr2=0x7f2b8c102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:32.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.249+0000 7f2b941d4640 1 --2- 172.21.15.122:0/2807932931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c102090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009900 tx=0x7f2b7c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:32.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- 172.21.15.122:0/2807932931 shutdown_connections 2024-04-02T13:32:32.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- 172.21.15.122:0/2807932931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c1025d0 0x7f2b8c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- 172.21.15.122:0/2807932931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c102090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- 172.21.15.122:0/2807932931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c100a20 0x7f2b8c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- 172.21.15.122:0/2807932931 >> 172.21.15.122:0/2807932931 conn(0x7f2b8c0fbf90 msgr2=0x7f2b8c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- 172.21.15.122:0/2807932931 shutdown_connections 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- 172.21.15.122:0/2807932931 wait complete. 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 Processor -- start 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- start start 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c100a20 0x7f2b8c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:32.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c1025d0 0x7f2b8c19e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c6a0 con 0x7f2b8c101c10 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c520 con 0x7f2b8c1025d0 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b941d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c10c820 con 0x7f2b8c100a20 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.250+0000 7f2b9176f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40398/0 (socket says 172.21.15.122:40398) 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 -- 172.21.15.122:0/3787050990 learned_addr learned my addr 172.21.15.122:0/3787050990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b92771640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c1025d0 0x7f2b8c19e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b91f70640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c100a20 0x7f2b8c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 -- 172.21.15.122:0/3787050990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c100a20 msgr2=0x7f2b8c196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c100a20 0x7f2b8c196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 -- 172.21.15.122:0/3787050990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c1025d0 msgr2=0x7f2b8c19e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:32.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c1025d0 0x7f2b8c19e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 -- 172.21.15.122:0/3787050990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c19e960 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b9176f640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009230 tx=0x7f2b7c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c041070 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b7c0044d0 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c19ebf0 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c034440 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.251+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c19f0e0 con 0x7f2b8c101c10 2024-04-02T13:32:32.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.252+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c107d40 con 0x7f2b8c101c10 2024-04-02T13:32:32.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.253+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2b7c044070 con 0x7f2b8c101c10 2024-04-02T13:32:32.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.254+0000 7f2b82ffd640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 0x7f2b780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:32.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.254+0000 7f2b91f70640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 0x7f2b780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:32.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.254+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2b7c0f1fb0 con 0x7f2b8c101c10 2024-04-02T13:32:32.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.254+0000 7f2b91f70640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 0x7f2b780a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c101a70 tx=0x7f2b84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:32.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.257+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2b7c0bed90 con 0x7f2b8c101c10 2024-04-02T13:32:32.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.475+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8c02d370 con 0x7f2b8c101c10 2024-04-02T13:32:32.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.476+0000 7f2b82ffd640 1 -- 172.21.15.122:0/3787050990 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7f2b7c0be690 con 0x7f2b8c101c10 2024-04-02T13:32:32.480 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:32.480 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T13:32:32.480 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 msgr2=0x7f2b780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 0x7f2b780a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c101a70 tx=0x7f2b84005f90 comp rx=0 tx=0).stop 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 msgr2=0x7f2b8c197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c009230 tx=0x7f2b7c005d50 comp rx=0 tx=0).stop 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 shutdown_connections 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2b780a3c30 0x7f2b780a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b8c1025d0 0x7f2b8c19e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b8c101c10 0x7f2b8c197280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 --2- 172.21.15.122:0/3787050990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b8c100a20 0x7f2b8c196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 >> 172.21.15.122:0/3787050990 conn(0x7f2b8c0fbf90 msgr2=0x7f2b8c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 shutdown_connections 2024-04-02T13:32:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:32.478+0000 7f2b941d4640 1 -- 172.21.15.122:0/3787050990 wait complete. 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: Deploying daemon mds.h on smithi187 2024-04-02T13:32:33.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:boot 2024-04-02T13:32:33.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: fsmap 7 up:standby 2024-04-02T13:32:33.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:32:33.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:33.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3787050990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: Deploying daemon mds.h on smithi187 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:boot 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: fsmap 7 up:standby 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:33.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3787050990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:33.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: Deploying daemon mds.h on smithi187 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:boot 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: fsmap 7 up:standby 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:33.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3787050990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:34.236 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 systemd[1]: Starting Ceph mds.h for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:34.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:34.524 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 podman[66639]: 2024-04-02 13:32:34.168059815 +0000 UTC m=+0.017762261 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:34.525 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 podman[66639]: 2024-04-02 13:32:34.278109068 +0000 UTC m=+0.127811506 container create 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-02T13:32:34.525 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 podman[66639]: 2024-04-02 13:32:34.522414491 +0000 UTC m=+0.372116912 container init 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 -- 172.21.15.122:0/1442739388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194101c10 msgr2=0x7fa194102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 --2- 172.21.15.122:0/1442739388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194101c10 0x7fa194102090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa184009900 tx=0x7fa18402bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 -- 172.21.15.122:0/1442739388 shutdown_connections 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 --2- 172.21.15.122:0/1442739388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1941025d0 0x7fa194108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 --2- 172.21.15.122:0/1442739388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194101c10 0x7fa194102090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.601+0000 7fa19c175640 1 --2- 172.21.15.122:0/1442739388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa194100a20 0x7fa194100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- 172.21.15.122:0/1442739388 >> 172.21.15.122:0/1442739388 conn(0x7fa1940fbf90 msgr2=0x7fa1940fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- 172.21.15.122:0/1442739388 shutdown_connections 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- 172.21.15.122:0/1442739388 wait complete. 2024-04-02T13:32:34.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 Processor -- start 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- start start 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa194101c10 0x7fa1941972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 0x7fa19419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410c6a0 con 0x7fa194100a20 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410c520 con 0x7fa194101c10 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19c175640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410c820 con 0x7fa1941025d0 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa199f11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa199f11640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40430/0 (socket says 172.21.15.122:40430) 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa199f11640 1 -- 172.21.15.122:0/2414866071 learned_addr learned my addr 172.21.15.122:0/2414866071 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa199710640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa194101c10 0x7fa1941972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.602+0000 7fa19a712640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 0x7fa19419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 -- 172.21.15.122:0/2414866071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 msgr2=0x7fa19419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:34.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 0x7fa19419e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 -- 172.21.15.122:0/2414866071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa194101c10 msgr2=0x7fa1941972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa194101c10 0x7fa1941972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 -- 172.21.15.122:0/2414866071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19419e990 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa19a712640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 0x7fa19419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa199f11640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e9c0 tx=0x7fa19000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19000cd80 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1900035f0 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa194068cc0 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa190010690 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.603+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa194069190 con 0x7fa194100a20 2024-04-02T13:32:34.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.604+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1940688f0 con 0x7fa194100a20 2024-04-02T13:32:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.605+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa1900037a0 con 0x7fa194100a20 2024-04-02T13:32:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.605+0000 7fa18affd640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 0x7fa1800a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.606+0000 7fa199710640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 0x7fa1800a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.606+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa1900d0370 con 0x7fa194100a20 2024-04-02T13:32:34.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.606+0000 7fa199710640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 0x7fa1800a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa18402c7b0 tx=0x7fa18403d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:34.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.610+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa19009d1f0 con 0x7fa194100a20 2024-04-02T13:32:34.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.832+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa194107830 con 0x7fa194100a20 2024-04-02T13:32:34.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.832+0000 7fa18affd640 1 -- 172.21.15.122:0/2414866071 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7fa19009de20 con 0x7fa194100a20 2024-04-02T13:32:34.836 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:34.836 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-02T13:32:34.836 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 8 2024-04-02T13:32:34.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 msgr2=0x7fa1800a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:34.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 0x7fa1800a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa18402c7b0 tx=0x7fa18403d040 comp rx=0 tx=0).stop 2024-04-02T13:32:34.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 msgr2=0x7fa194196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:34.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa19000e9c0 tx=0x7fa19000ee80 comp rx=0 tx=0).stop 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 shutdown_connections 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1941025d0 0x7fa19419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1800a3bb0 0x7fa1800a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa194101c10 0x7fa1941972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 --2- 172.21.15.122:0/2414866071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa194100a20 0x7fa194196d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 >> 172.21.15.122:0/2414866071 conn(0x7fa1940fbf90 msgr2=0x7fa194104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 shutdown_connections 2024-04-02T13:32:34.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:34.834+0000 7fa19c175640 1 -- 172.21.15.122:0/2414866071 wait complete. 2024-04-02T13:32:34.985 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 podman[66639]: 2024-04-02 13:32:34.525127484 +0000 UTC m=+0.374829903 container start 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:32:34.985 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 ceph-mds[66658]: -- 172.21.15.187:0/1971210955 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c87195a40 con 0x563c87169000 2024-04-02T13:32:34.986 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: starting mds.h at 2024-04-02T13:32:34.986 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 bash[66639]: 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 2024-04-02T13:32:34.986 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:32:34 smithi187 systemd[1]: Started Ceph mds.h for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:35.605 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:35 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2414866071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:35 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2414866071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:35 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2414866071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:36 smithi122 ceph-mon[32114]: Deploying daemon mds.i on smithi188 2024-04-02T13:32:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:36 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:boot 2024-04-02T13:32:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:36 smithi122 ceph-mon[32114]: fsmap 8 up:standby 2024-04-02T13:32:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:36 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:32:36.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:36 smithi188 ceph-mon[36117]: Deploying daemon mds.i on smithi188 2024-04-02T13:32:36.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:36 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:boot 2024-04-02T13:32:36.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:36 smithi188 ceph-mon[36117]: fsmap 8 up:standby 2024-04-02T13:32:36.823 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:36 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:32:36.850 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.968+0000 7f9797a63640 1 -- 172.21.15.122:0/1442279885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97901025d0 msgr2=0x7f9790108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.968+0000 7f9797a63640 1 --2- 172.21.15.122:0/1442279885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97901025d0 0x7f9790108e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9788009be0 tx=0x7f978802bef0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 -- 172.21.15.122:0/1442279885 shutdown_connections 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 --2- 172.21.15.122:0/1442279885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97901025d0 0x7f9790108e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 --2- 172.21.15.122:0/1442279885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790101c10 0x7f9790102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 --2- 172.21.15.122:0/1442279885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9790100a20 0x7f9790100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 -- 172.21.15.122:0/1442279885 >> 172.21.15.122:0/1442279885 conn(0x7f97900fbf90 msgr2=0x7f97900fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 -- 172.21.15.122:0/1442279885 shutdown_connections 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 -- 172.21.15.122:0/1442279885 wait complete. 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 Processor -- start 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.969+0000 7f9797a63640 1 -- start start 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 0x7f97900697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 0x7f97901a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f979010c6a0 con 0x7f9790101c10 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f979010c520 con 0x7f97901025d0 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9797a63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f979010c820 con 0x7f9790100a20 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40452/0 (socket says 172.21.15.122:40452) 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 -- 172.21.15.122:0/1432819432 learned_addr learned my addr 172.21.15.122:0/1432819432 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f97957ff640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 0x7f97900697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9796000640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 0x7f97901a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 -- 172.21.15.122:0/1432819432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 msgr2=0x7f97900697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 0x7f97900697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 -- 172.21.15.122:0/1432819432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 msgr2=0x7f97901a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 0x7f97901a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 -- 172.21.15.122:0/1432819432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901a1c70 con 0x7f9790101c10 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f97957ff640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 0x7f97900697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9796000640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 0x7f97901a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.970+0000 7f9794ffe640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f978000e9f0 tx=0x7f978000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.971+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978000ccc0 con 0x7f9790101c10 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.971+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97901a1f00 con 0x7f9790101c10 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.971+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9780005500 con 0x7f9790101c10 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.971+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9780010690 con 0x7f9790101c10 2024-04-02T13:32:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.971+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901a2370 con 0x7f9790101c10 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.972+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9790107d40 con 0x7f9790101c10 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.973+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f97800046c0 con 0x7f9790101c10 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.974+0000 7f97867fc640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 0x7f977c0a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.974+0000 7f97957ff640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 0x7f977c0a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.974+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97800d0390 con 0x7f9790101c10 2024-04-02T13:32:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.974+0000 7f97957ff640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 0x7f977c0a6120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f978c0058c0 tx=0x7f978c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:36.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:36.979+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f978009d170 con 0x7f9790101c10 2024-04-02T13:32:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:36 smithi187 ceph-mon[36064]: Deploying daemon mds.i on smithi188 2024-04-02T13:32:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:36 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:boot 2024-04-02T13:32:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:36 smithi187 ceph-mon[36064]: fsmap 8 up:standby 2024-04-02T13:32:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:36 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:32:37.116 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 systemd[1]: Starting Ceph mds.i for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.195+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f979002d370 con 0x7f9790101c10 2024-04-02T13:32:37.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.196+0000 7f97867fc640 1 -- 172.21.15.122:0/1432819432 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7180 (secure 0 0 0) 0x7f978009ca70 con 0x7f9790101c10 2024-04-02T13:32:37.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:37.200 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-02T13:32:37.200 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 9 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 msgr2=0x7f977c0a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 0x7f977c0a6120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f978c0058c0 tx=0x7f978c005850 comp rx=0 tx=0).stop 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 msgr2=0x7f979019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f978000e9f0 tx=0x7f978000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 shutdown_connections 2024-04-02T13:32:37.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f977c0a3c60 0x7f977c0a6120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f97901025d0 0x7f97901a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9790101c10 0x7f979019ef30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 --2- 172.21.15.122:0/1432819432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9790100a20 0x7f97900697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 >> 172.21.15.122:0/1432819432 conn(0x7f97900fbf90 msgr2=0x7f9790104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 shutdown_connections 2024-04-02T13:32:37.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:37.198+0000 7f9797a63640 1 -- 172.21.15.122:0/1432819432 wait complete. 2024-04-02T13:32:37.469 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 podman[67484]: 2024-04-02 13:32:37.232906993 +0000 UTC m=+0.017509161 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:37.470 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 podman[67484]: 2024-04-02 13:32:37.338129383 +0000 UTC m=+0.122731543 container create c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T13:32:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:37 smithi188 ceph-mon[36117]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:37 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1432819432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 podman[67484]: 2024-04-02 13:32:37.590948261 +0000 UTC m=+0.375550429 container init c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 podman[67484]: 2024-04-02 13:32:37.594060846 +0000 UTC m=+0.378663005 container start c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 ceph-mds[67504]: -- 172.21.15.188:0/1479963205 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b14631a40 con 0x562b14604800 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: starting mds.i at 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 bash[67484]: c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 2024-04-02T13:32:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:32:37 smithi188 systemd[1]: Started Ceph mds.i for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:37 smithi122 ceph-mon[32114]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:37 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1432819432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:37 smithi187 ceph-mon[36064]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:37 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1432819432' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:38.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: Deploying daemon mds.g on smithi122 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:boot 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: fsmap 9 up:standby 2024-04-02T13:32:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:38 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:32:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: Deploying daemon mds.g on smithi122 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:boot 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: fsmap 9 up:standby 2024-04-02T13:32:39.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:38 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:39.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: Deploying daemon mds.g on smithi122 2024-04-02T13:32:39.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:39.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:boot 2024-04-02T13:32:39.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: fsmap 9 up:standby 2024-04-02T13:32:39.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:38 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:32:39.214 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1840610884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 msgr2=0x7fd144108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1840610884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd144108f50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd13c009ca0 tx=0x7fd13c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1840610884 shutdown_connections 2024-04-02T13:32:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1840610884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd144108f50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1840610884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1840610884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd1441023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1840610884 >> 172.21.15.122:0/1840610884 conn(0x7fd1440fb720 msgr2=0x7fd1440fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1840610884 shutdown_connections 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.322+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1840610884 wait complete. 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 Processor -- start 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 -- start start 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd14419f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410d7e0 con 0x7fd1441016b0 2024-04-02T13:32:39.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410d660 con 0x7fd144101fc0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd14a5e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd14410d960 con 0x7fd144100cf0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58424/0 (socket says 172.21.15.122:58424) 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 -- 172.21.15.122:0/1040352128 learned_addr learned my addr 172.21.15.122:0/1040352128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd148b7f640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd14419f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd143fff640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 -- 172.21.15.122:0/1040352128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 msgr2=0x7fd144197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 -- 172.21.15.122:0/1040352128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 msgr2=0x7fd14419f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd14419f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.323+0000 7fd1437fe640 1 -- 172.21.15.122:0/1040352128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14419fab0 con 0x7fd1441016b0 2024-04-02T13:32:39.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd148b7f640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd14419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd143fff640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd1437fe640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d770 tx=0x7fd13800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138017070 con 0x7fd1441016b0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd138005500 con 0x7fd1441016b0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138004e00 con 0x7fd1441016b0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd14419fda0 con 0x7fd1441016b0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.324+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1441a0210 con 0x7fd1441016b0 2024-04-02T13:32:39.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.325+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd144107dd0 con 0x7fd1441016b0 2024-04-02T13:32:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.326+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fd138004fa0 con 0x7fd1441016b0 2024-04-02T13:32:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.327+0000 7fd1417fa640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 0x7fd12c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.327+0000 7fd143fff640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 0x7fd12c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.327+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd1380cfe90 con 0x7fd1441016b0 2024-04-02T13:32:39.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.327+0000 7fd143fff640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 0x7fd12c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd144069f30 tx=0x7fd134005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:39.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.331+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd13809cc70 con 0x7fd1441016b0 2024-04-02T13:32:39.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.552+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd14402d5c0 con 0x7fd1441016b0 2024-04-02T13:32:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.553+0000 7fd1417fa640 1 -- 172.21.15.122:0/1040352128 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fd1380cf070 con 0x7fd1441016b0 2024-04-02T13:32:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:39.557 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:32:39.557 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 10 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 msgr2=0x7fd12c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 0x7fd12c0a6070 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd144069f30 tx=0x7fd134005f90 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 msgr2=0x7fd1441983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd13800d770 tx=0x7fd13800dc30 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 shutdown_connections 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd12c0a3bb0 0x7fd12c0a6070 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd144101fc0 0x7fd14419f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1441016b0 0x7fd1441983d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 --2- 172.21.15.122:0/1040352128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd144100cf0 0x7fd144197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 >> 172.21.15.122:0/1040352128 conn(0x7fd1440fb720 msgr2=0x7fd144104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 shutdown_connections 2024-04-02T13:32:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:39.555+0000 7fd14a5e2640 1 -- 172.21.15.122:0/1040352128 wait complete. 2024-04-02T13:32:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:39 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1040352128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:39 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1040352128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:40.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:39 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1040352128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:40.292 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 systemd[1]: Starting Ceph mds.g for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:32:40.571 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 podman[71079]: 2024-04-02 13:32:40.19262423 +0000 UTC m=+0.017772029 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:32:40.571 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 podman[71079]: 2024-04-02 13:32:40.305214211 +0000 UTC m=+0.130361982 container create 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:40 smithi122 ceph-mon[32114]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:40 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 podman[71079]: 2024-04-02 13:32:40.600147074 +0000 UTC m=+0.425294848 container init 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 podman[71079]: 2024-04-02 13:32:40.602847146 +0000 UTC m=+0.427994922 container start 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 ceph-mds[71098]: -- 172.21.15.122:0/1522246896 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b60cbcba40 con 0x55b60cb9ec00 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: starting mds.g at 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 bash[71079]: 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 2024-04-02T13:32:41.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:32:40 smithi122 systemd[1]: Started Ceph mds.g for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:41.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:40 smithi187 ceph-mon[36064]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:41.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:40 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:40 smithi188 ceph-mon[36117]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:40 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:41.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 -- 172.21.15.122:0/121357859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de81025d0 msgr2=0x7f0de8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 --2- 172.21.15.122:0/121357859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de81025d0 0x7f0de8108e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009ca0 tx=0x7f0de002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 -- 172.21.15.122:0/121357859 shutdown_connections 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 --2- 172.21.15.122:0/121357859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de81025d0 0x7f0de8108e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 --2- 172.21.15.122:0/121357859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de8101c10 0x7f0de8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 --2- 172.21.15.122:0/121357859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 0x7f0de8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 -- 172.21.15.122:0/121357859 >> 172.21.15.122:0/121357859 conn(0x7f0de80fbf90 msgr2=0x7f0de80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 -- 172.21.15.122:0/121357859 shutdown_connections 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.674+0000 7f0defa93640 1 -- 172.21.15.122:0/121357859 wait complete. 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 Processor -- start 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 -- start start 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 0x7f0de80697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de81025d0 0x7f0de819f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0de810c700 con 0x7f0de8101c10 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0de810c580 con 0x7f0de81025d0 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0defa93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0de810c880 con 0x7f0de8100a20 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58456/0 (socket says 172.21.15.122:58456) 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 -- 172.21.15.122:0/2947566911 learned_addr learned my addr 172.21.15.122:0/2947566911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:41.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0dee030640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de81025d0 0x7f0de819f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded82f640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 0x7f0de80697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 -- 172.21.15.122:0/2947566911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 msgr2=0x7f0de80697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 0x7f0de80697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 -- 172.21.15.122:0/2947566911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de81025d0 msgr2=0x7f0de819f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de81025d0 0x7f0de819f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.675+0000 7f0ded02e640 1 -- 172.21.15.122:0/2947566911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de819fa70 con 0x7f0de8101c10 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0ded02e640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b4a0 tx=0x7f0dd800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8014070 con 0x7f0de8101c10 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de819fd60 con 0x7f0de8101c10 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dd8010af0 con 0x7f0de8101c10 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8017d10 con 0x7f0de8101c10 2024-04-02T13:32:41.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.676+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de81a0250 con 0x7f0de8101c10 2024-04-02T13:32:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.677+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de8107d40 con 0x7f0de8101c10 2024-04-02T13:32:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.678+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0dd8017070 con 0x7f0de8101c10 2024-04-02T13:32:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.678+0000 7f0ddeffd640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 0x7f0dd40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.679+0000 7f0ded82f640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 0x7f0dd40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.679+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f0dd80cf2d0 con 0x7f0de8101c10 2024-04-02T13:32:41.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.679+0000 7f0ded82f640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 0x7f0dd40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005fd0 tx=0x7f0de4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.683+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0dd809c150 con 0x7f0de8101c10 2024-04-02T13:32:41.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.907+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0de802d5b0 con 0x7f0de8101c10 2024-04-02T13:32:41.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.907+0000 7f0ddeffd640 1 -- 172.21.15.122:0/2947566911 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8836 (secure 0 0 0) 0x7f0dd809ba50 con 0x7f0de8101c10 2024-04-02T13:32:41.911 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:41.911 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:32:41.911 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 11 2024-04-02T13:32:41.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 msgr2=0x7f0dd40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:41.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 0x7f0dd40a6070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0de4005fd0 tx=0x7f0de4005850 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 msgr2=0x7f0de819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0dd800b4a0 tx=0x7f0dd800b960 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 shutdown_connections 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0dd40a3bb0 0x7f0dd40a6070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0de81025d0 0x7f0de819f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0de8101c10 0x7f0de819cb70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 --2- 172.21.15.122:0/2947566911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0de8100a20 0x7f0de80697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.909+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 >> 172.21.15.122:0/2947566911 conn(0x7f0de80fbf90 msgr2=0x7f0de8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.910+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 shutdown_connections 2024-04-02T13:32:41.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:41.910+0000 7f0defa93640 1 -- 172.21.15.122:0/2947566911 wait complete. 2024-04-02T13:32:42.043 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6838/1672540384,v1:172.21.15.122:6839/1672540384] up:boot 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: fsmap 10 up:standby 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:42.044 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:41 smithi122 ceph-mon[32114]: Deploying daemon mds.a on smithi122 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6838/1672540384,v1:172.21.15.122:6839/1672540384] up:boot 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: fsmap 10 up:standby 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:41 smithi187 ceph-mon[36064]: Deploying daemon mds.a on smithi122 2024-04-02T13:32:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6838/1672540384,v1:172.21.15.122:6839/1672540384] up:boot 2024-04-02T13:32:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: fsmap 10 up:standby 2024-04-02T13:32:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:32:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:42.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:41 smithi188 ceph-mon[36117]: Deploying daemon mds.a on smithi122 2024-04-02T13:32:42.896 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:42 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2947566911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:42.896 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:42 smithi122 ceph-mon[32114]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:43.137 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:42 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2947566911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:43.137 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:42 smithi187 ceph-mon[36064]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:43.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:42 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2947566911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:43.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:42 smithi188 ceph-mon[36117]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:43.820 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:32:43 smithi122 systemd[1]: Started Ceph mds.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:43.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 -- 172.21.15.122:0/2295381112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 msgr2=0x7fc748108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/2295381112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc748108e60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc740009ca0 tx=0x7fc74002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 -- 172.21.15.122:0/2295381112 shutdown_connections 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/2295381112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc748108e60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/2295381112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748101c10 0x7fc748102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/2295381112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748100a20 0x7fc748100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 -- 172.21.15.122:0/2295381112 >> 172.21.15.122:0/2295381112 conn(0x7fc7480fbf90 msgr2=0x7fc7480fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 -- 172.21.15.122:0/2295381112 shutdown_connections 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.026+0000 7fc74cbe9640 1 -- 172.21.15.122:0/2295381112 wait complete. 2024-04-02T13:32:44.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 Processor -- start 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 -- start start 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748100a20 0x7fc748197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748101c10 0x7fc748198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc74810c790 con 0x7fc7481025d0 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc74810c610 con 0x7fc748101c10 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74cbe9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc74810c910 con 0x7fc748100a20 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58494/0 (socket says 172.21.15.122:58494) 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 -- 172.21.15.122:0/342570291 learned_addr learned my addr 172.21.15.122:0/342570291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc74659c640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748100a20 0x7fc748197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc73ffff640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748101c10 0x7fc748198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 -- 172.21.15.122:0/342570291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748100a20 msgr2=0x7fc748197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:44.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748100a20 0x7fc748197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 -- 172.21.15.122:0/342570291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748101c10 msgr2=0x7fc748198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748101c10 0x7fc748198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.027+0000 7fc746d9d640 1 -- 172.21.15.122:0/342570291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74819faf0 con 0x7fc7481025d0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc746d9d640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc740037040 tx=0x7fc740032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740039040 con 0x7fc7481025d0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74819fd80 con 0x7fc7481025d0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc740044070 con 0x7fc7481025d0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc740004260 con 0x7fc7481025d0 2024-04-02T13:32:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.028+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7481a0290 con 0x7fc7481025d0 2024-04-02T13:32:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.029+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7481a04c0 con 0x7fc7481025d0 2024-04-02T13:32:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.030+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc740003970 con 0x7fc7481025d0 2024-04-02T13:32:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.031+0000 7fc73f7fe640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 0x7fc7340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.031+0000 7fc74659c640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 0x7fc7340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.031+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7400f2c70 con 0x7fc7481025d0 2024-04-02T13:32:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.031+0000 7fc74659c640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 0x7fc7340a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc748101a70 tx=0x7fc738009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.034+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc7400bfaf0 con 0x7fc7481025d0 2024-04-02T13:32:44.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.249+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7481a0850 con 0x7fc7481025d0 2024-04-02T13:32:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.249+0000 7fc73f7fe640 1 -- 172.21.15.122:0/342570291 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9662 (secure 0 0 0) 0x7fc7400bf3f0 con 0x7fc7481025d0 2024-04-02T13:32:44.253 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:44.253 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-02T13:32:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 12 2024-04-02T13:32:44.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 msgr2=0x7fc7340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:44.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 0x7fc7340a6070 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc748101a70 tx=0x7fc738009110 comp rx=0 tx=0).stop 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 msgr2=0x7fc74819f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc740037040 tx=0x7fc740032f90 comp rx=0 tx=0).stop 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 shutdown_connections 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7481025d0 0x7fc74819f480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7340a3bb0 0x7fc7340a6070 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc748101c10 0x7fc748198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 --2- 172.21.15.122:0/342570291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc748100a20 0x7fc748197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 >> 172.21.15.122:0/342570291 conn(0x7fc7480fbf90 msgr2=0x7fc7480fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 shutdown_connections 2024-04-02T13:32:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:44.252+0000 7fc74cbe9640 1 -- 172.21.15.122:0/342570291 wait complete. 2024-04-02T13:32:44.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: Deploying daemon mds.b on smithi187 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:boot 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: fsmap 11 up:standby 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:32:44.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:44 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/342570291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: Deploying daemon mds.b on smithi187 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:boot 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: fsmap 11 up:standby 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:32:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:44 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/342570291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: Deploying daemon mds.b on smithi187 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:boot 2024-04-02T13:32:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: fsmap 11 up:standby 2024-04-02T13:32:45.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:32:45.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:44 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/342570291' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:45 smithi187 ceph-mon[36064]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:45 smithi122 ceph-mon[32114]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:45 smithi188 ceph-mon[36117]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:46.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:46.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 -- 172.21.15.122:0/3733277027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0100a40 msgr2=0x7f72b0100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/3733277027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0100a40 0x7f72b0100e60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009900 tx=0x7f72a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 -- 172.21.15.122:0/3733277027 shutdown_connections 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/3733277027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b01025f0 0x7f72b0108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/3733277027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b0101c30 0x7f72b01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/3733277027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0100a40 0x7f72b0100e60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 -- 172.21.15.122:0/3733277027 >> 172.21.15.122:0/3733277027 conn(0x7f72b00fbfb0 msgr2=0x7f72b00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.383+0000 7f72b4dfa640 1 -- 172.21.15.122:0/3733277027 shutdown_connections 2024-04-02T13:32:46.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 -- 172.21.15.122:0/3733277027 wait complete. 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 Processor -- start 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 -- start start 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0101c30 0x7f72b01983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b01025f0 0x7f72b019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d7e0 con 0x7f72b0101c30 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d660 con 0x7f72b0100a40 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.384+0000 7f72b4dfa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72b010d960 con 0x7f72b01025f0 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:32936/0 (socket says 172.21.15.122:32936) 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 -- 172.21.15.122:0/1112379828 learned_addr learned my addr 172.21.15.122:0/1112379828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:46.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72a7fff640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0101c30 0x7f72b01983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72aed9d640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b01025f0 0x7f72b019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 -- 172.21.15.122:0/1112379828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b01025f0 msgr2=0x7f72b019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b01025f0 0x7f72b019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 -- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0101c30 msgr2=0x7f72b01983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0101c30 0x7f72b01983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 -- 172.21.15.122:0/1112379828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b019fac0 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72ae59c640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005ab0 tx=0x7f72a0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0041070 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72a00054e0 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b019fd50 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0034440 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.385+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b01a01d0 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.386+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b0107d80 con 0x7f72b0100a40 2024-04-02T13:32:46.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.387+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f72a0044070 con 0x7f72b0100a40 2024-04-02T13:32:46.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.388+0000 7f72a77fe640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.388+0000 7f72a7fff640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.388+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f72a00f1d10 con 0x7f72b0100a40 2024-04-02T13:32:46.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.388+0000 7f72a7fff640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 0x7f729c0a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72b01993b0 tx=0x7f729800a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:46.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.392+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f72a00beb90 con 0x7f72b0100a40 2024-04-02T13:32:46.485 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:32:46 smithi187 systemd[1]: Started Ceph mds.b for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:32:46.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.616+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b002d370 con 0x7f72b0100a40 2024-04-02T13:32:46.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.617+0000 7f72a77fe640 1 -- 172.21.15.122:0/1112379828 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10491 (secure 0 0 0) 0x7f72a00be490 con 0x7f72b0100a40 2024-04-02T13:32:46.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:46.620 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-02T13:32:46.621 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 msgr2=0x7f729c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 0x7f729c0a60f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f72b01993b0 tx=0x7f729800a250 comp rx=0 tx=0).stop 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 msgr2=0x7f72b0197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f72a0005ab0 tx=0x7f72a0005dc0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 shutdown_connections 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b01025f0 0x7f72b019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72b0101c30 0x7f72b01983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f729c0a3c30 0x7f729c0a60f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 --2- 172.21.15.122:0/1112379828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f72b0100a40 0x7f72b0197ea0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 >> 172.21.15.122:0/1112379828 conn(0x7f72b00fbfb0 msgr2=0x7f72b0104f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:46.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 shutdown_connections 2024-04-02T13:32:46.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.619+0000 7f72b4dfa640 1 -- 172.21.15.122:0/1112379828 wait complete. 2024-04-02T13:32:46.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:32:46.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/434558118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 msgr2=0x7f2ccc1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/434558118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc1072c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8009900 tx=0x7f2cc802bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:46.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/434558118 shutdown_connections 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/434558118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc107800 0x7f2ccc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/434558118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc1072c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/434558118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/434558118 >> 172.21.15.122:0/434558118 conn(0x7f2ccc0fbf90 msgr2=0x7f2ccc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/434558118 shutdown_connections 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.736+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/434558118 wait complete. 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 Processor -- start 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 -- start start 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc107800 0x7f2ccc19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10d7a0 con 0x7f2ccc104ed0 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10d620 con 0x7f2ccc107800 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd3f8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc10d920 con 0x7f2ccc1025a0 2024-04-02T13:32:46.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58518/0 (socket says 172.21.15.122:58518) 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 -- 172.21.15.122:0/1067388069 learned_addr learned my addr 172.21.15.122:0/1067388069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd1d2b640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd252c640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc107800 0x7f2ccc19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 -- 172.21.15.122:0/1067388069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 msgr2=0x7f2ccc197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 -- 172.21.15.122:0/1067388069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc107800 msgr2=0x7f2ccc19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc107800 0x7f2ccc19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 -- 172.21.15.122:0/1067388069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19fa60 con 0x7f2ccc104ed0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd1d2b640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.737+0000 7f2cd152a640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.738+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8041070 con 0x7f2ccc104ed0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.738+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19fcf0 con 0x7f2ccc104ed0 2024-04-02T13:32:46.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.738+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8004450 con 0x7f2ccc104ed0 2024-04-02T13:32:46.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.738+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8034440 con 0x7f2ccc104ed0 2024-04-02T13:32:46.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.738+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a01d0 con 0x7f2ccc104ed0 2024-04-02T13:32:46.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.739+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ccc0fda80 con 0x7f2ccc104ed0 2024-04-02T13:32:46.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.741+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2cc8044070 con 0x7f2ccc104ed0 2024-04-02T13:32:46.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.741+0000 7f2cc2ffd640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 0x7f2cb80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:46.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.741+0000 7f2cd1d2b640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 0x7f2cb80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:46.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.741+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f2cc80f1bd0 con 0x7f2ccc104ed0 2024-04-02T13:32:46.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.742+0000 7f2cd1d2b640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 0x7f2cb80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009fc0 tx=0x7f2cbc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:46.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.744+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2cc80be490 con 0x7f2ccc104ed0 2024-04-02T13:32:46.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.927+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ccc02d4f0 con 0x7f2ccc104ed0 2024-04-02T13:32:46.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.928+0000 7f2cc2ffd640 1 -- 172.21.15.122:0/1067388069 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18502 (secure 0 0 0) 0x7f2cc80c2790 con 0x7f2ccc104ed0 2024-04-02T13:32:46.932 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:46.932 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":82,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:01.165058+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:46.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.930+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 msgr2=0x7f2cb80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.930+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 0x7f2cb80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009fc0 tx=0x7f2cbc005f90 comp rx=0 tx=0).stop 2024-04-02T13:32:46.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.930+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 msgr2=0x7f2ccc198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:46.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8000c00 tx=0x7f2cc802c640 comp rx=0 tx=0).stop 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 shutdown_connections 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f2cb80a3bb0 0x7f2cb80a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ccc107800 0x7f2ccc19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2ccc104ed0 0x7f2ccc198380 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 --2- 172.21.15.122:0/1067388069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ccc1025a0 0x7f2ccc197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 >> 172.21.15.122:0/1067388069 conn(0x7f2ccc0fbf90 msgr2=0x7f2ccc102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 shutdown_connections 2024-04-02T13:32:46.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:46.931+0000 7f2cd3f8f640 1 -- 172.21.15.122:0/1067388069 wait complete. 2024-04-02T13:32:46.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-02T13:32:46.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 -- 172.21.15.122:0/4100064610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 msgr2=0x7fb0b0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 --2- 172.21.15.122:0/4100064610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0108e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009ca0 tx=0x7fb0a802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 -- 172.21.15.122:0/4100064610 shutdown_connections 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 --2- 172.21.15.122:0/4100064610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0108e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 --2- 172.21.15.122:0/4100064610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0101bf0 0x7fb0b0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 --2- 172.21.15.122:0/4100064610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0100a00 0x7fb0b0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 -- 172.21.15.122:0/4100064610 >> 172.21.15.122:0/4100064610 conn(0x7fb0b00fbf90 msgr2=0x7fb0b00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:47.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 -- 172.21.15.122:0/4100064610 shutdown_connections 2024-04-02T13:32:47.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.048+0000 7fb0b49db640 1 -- 172.21.15.122:0/4100064610 wait complete. 2024-04-02T13:32:47.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 Processor -- start 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 -- start start 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 0x7fb0b010b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0101bf0 0x7fb0b01099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010d760 con 0x7fb0b01025b0 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010d5e0 con 0x7fb0b0100a00 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0b49db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b010d8e0 con 0x7fb0b0101bf0 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58536/0 (socket says 172.21.15.122:58536) 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0ae59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 0x7fb0b010b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 -- 172.21.15.122:0/2990773186 learned_addr learned my addr 172.21.15.122:0/2990773186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0add9b640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0101bf0 0x7fb0b01099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 -- 172.21.15.122:0/2990773186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0101bf0 msgr2=0x7fb0b01099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0101bf0 0x7fb0b01099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 -- 172.21.15.122:0/2990773186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 msgr2=0x7fb0b010b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.049+0000 7fb0aed9d640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 0x7fb0b010b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb0aed9d640 1 -- 172.21.15.122:0/2990773186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b010a790 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb0ae59c640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 0x7fb0b010b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb0aed9d640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8037040 tx=0x7fb0a8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8041020 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01a1ec0 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0a8034d90 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01a2380 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.050+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8039040 con 0x7fb0b01025b0 2024-04-02T13:32:47.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.051+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b0107d20 con 0x7fb0b01025b0 2024-04-02T13:32:47.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.052+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb0a8044020 con 0x7fb0b01025b0 2024-04-02T13:32:47.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.053+0000 7fb09f7fe640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 0x7fb0980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.053+0000 7fb0ae59c640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 0x7fb0980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.053+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb0a80f1dd0 con 0x7fb0b01025b0 2024-04-02T13:32:47.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.053+0000 7fb0ae59c640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 0x7fb0980a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005fd0 tx=0x7fb0a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.062+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb0a804b020 con 0x7fb0b01025b0 2024-04-02T13:32:47.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.263+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0b002d370 con 0x7fb0b01025b0 2024-04-02T13:32:47.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.263+0000 7fb09f7fe640 1 -- 172.21.15.122:0/2990773186 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb0a80bec50 con 0x7fb0b01025b0 2024-04-02T13:32:47.267 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:32:47.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.265+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 msgr2=0x7fb0980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.265+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 0x7fb0980a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0005fd0 tx=0x7fb0a000b410 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.265+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 msgr2=0x7fb0b0109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.265+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8037040 tx=0x7fb0a8032f20 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 shutdown_connections 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b01025b0 0x7fb0b0109f00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0b0101bf0 0x7fb0b01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0980a3bb0 0x7fb0980a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 --2- 172.21.15.122:0/2990773186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0b0100a00 0x7fb0b010b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 >> 172.21.15.122:0/2990773186 conn(0x7fb0b00fbf90 msgr2=0x7fb0b0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 shutdown_connections 2024-04-02T13:32:47.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.266+0000 7fb0b49db640 1 -- 172.21.15.122:0/2990773186 wait complete. 2024-04-02T13:32:47.281 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-02T13:32:47.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.380+0000 7ffacbd8f640 1 -- 172.21.15.122:0/2510735775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 msgr2=0x7ffac4109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.380+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/2510735775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac4109cd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009ca0 tx=0x7ffabc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- 172.21.15.122:0/2510735775 shutdown_connections 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/2510735775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac4109cd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/2510735775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 0x7ffac4107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/2510735775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 0x7ffac4104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- 172.21.15.122:0/2510735775 >> 172.21.15.122:0/2510735775 conn(0x7ffac40fc030 msgr2=0x7ffac40fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- 172.21.15.122:0/2510735775 shutdown_connections 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- 172.21.15.122:0/2510735775 wait complete. 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 Processor -- start 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- start start 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 0x7ffac4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 0x7ffac4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410d820 con 0x7ffac4107880 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410d6a0 con 0x7ffac4102620 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffacbd8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffac410d9a0 con 0x7ffac4104f50 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffaca32c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffaca32c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58554/0 (socket says 172.21.15.122:58554) 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.381+0000 7ffaca32c640 1 -- 172.21.15.122:0/1561527965 learned_addr learned my addr 172.21.15.122:0/1561527965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffac932a640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 0x7ffac4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffac9b2b640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 0x7ffac4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 -- 172.21.15.122:0/1561527965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 msgr2=0x7ffac4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 0x7ffac4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 -- 172.21.15.122:0/1561527965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 msgr2=0x7ffac4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 0x7ffac4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 -- 172.21.15.122:0/1561527965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac419fa70 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffaca32c640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffabc02c700 tx=0x7ffabc032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc039040 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffabc044070 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac419fd00 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffabc0042a0 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.382+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac41a01a0 con 0x7ffac4107880 2024-04-02T13:32:47.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.383+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac40fdb20 con 0x7ffac4107880 2024-04-02T13:32:47.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.384+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ffabc032690 con 0x7ffac4107880 2024-04-02T13:32:47.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.384+0000 7ffabaffd640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 0x7ffab00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.385+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffabc0f2420 con 0x7ffac4107880 2024-04-02T13:32:47.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.385+0000 7ffac9b2b640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 0x7ffab00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.385+0000 7ffac9b2b640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 0x7ffab00a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffac000bbe0 tx=0x7ffac000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.389+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffabc0bf330 con 0x7ffac4107880 2024-04-02T13:32:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:boot 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: fsmap 12 up:standby 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1112379828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1067388069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:47 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2990773186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.579+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ffac402d5c0 con 0x7ffac4107880 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.579+0000 7ffabaffd640 1 -- 172.21.15.122:0/1561527965 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7ffabc0bec30 con 0x7ffac4107880 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-04-02T13:32:47.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:32:47.584 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-04-02T13:32:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.581+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 msgr2=0x7ffab00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.581+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 0x7ffab00a6070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffac000bbe0 tx=0x7ffac000b390 comp rx=0 tx=0).stop 2024-04-02T13:32:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.581+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 msgr2=0x7ffac419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.581+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffabc02c700 tx=0x7ffabc032f20 comp rx=0 tx=0).stop 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 shutdown_connections 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffac4107880 0x7ffac419f400 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffac4104f50 0x7ffac4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffab00a3bb0 0x7ffab00a6070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 --2- 172.21.15.122:0/1561527965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffac4102620 0x7ffac4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 >> 172.21.15.122:0/1561527965 conn(0x7ffac40fc030 msgr2=0x7ffac4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 shutdown_connections 2024-04-02T13:32:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.582+0000 7ffacbd8f640 1 -- 172.21.15.122:0/1561527965 wait complete. 2024-04-02T13:32:47.596 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-02T13:32:47.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-02T13:32:47.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 -- 172.21.15.122:0/340913304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c103840 msgr2=0x7f286c10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 --2- 172.21.15.122:0/340913304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c103840 0x7f286c10a0d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2864009cd0 tx=0x7f286402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 -- 172.21.15.122:0/340913304 shutdown_connections 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 --2- 172.21.15.122:0/340913304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c103840 0x7f286c10a0d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 --2- 172.21.15.122:0/340913304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286c102e80 0x7f286c103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 --2- 172.21.15.122:0/340913304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f286c0ffe10 0x7f286c100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 -- 172.21.15.122:0/340913304 >> 172.21.15.122:0/340913304 conn(0x7f286c0fb700 msgr2=0x7f286c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 -- 172.21.15.122:0/340913304 shutdown_connections 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 -- 172.21.15.122:0/340913304 wait complete. 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.700+0000 7f2871c0d640 1 Processor -- start 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 -- start start 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f286c102e80 0x7f286c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286c103840 0x7f286c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10d790 con 0x7f286c0ffe10 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10d610 con 0x7f286c103840 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f286c10d910 con 0x7f286c102e80 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58564/0 (socket says 172.21.15.122:58564) 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 -- 172.21.15.122:0/2381656888 learned_addr learned my addr 172.21.15.122:0/2381656888 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 -- 172.21.15.122:0/2381656888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f286c102e80 msgr2=0x7f286c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f286c102e80 0x7f286c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 -- 172.21.15.122:0/2381656888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286c103840 msgr2=0x7f286c19f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286c103840 0x7f286c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 -- 172.21.15.122:0/2381656888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286c19fa20 con 0x7f286c0ffe10 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f286b7fe640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d950 tx=0x7f285c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c017020 con 0x7f286c0ffe10 2024-04-02T13:32:47.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f286c19fd10 con 0x7f286c0ffe10 2024-04-02T13:32:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f285c009c30 con 0x7f286c0ffe10 2024-04-02T13:32:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.701+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c00f740 con 0x7f286c0ffe10 2024-04-02T13:32:47.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.702+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286c1a0200 con 0x7f286c0ffe10 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.702+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f286c109000 con 0x7f286c0ffe10 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.703+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f285c01a020 con 0x7f286c0ffe10 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.704+0000 7f2868ff9640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 0x7f28540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.704+0000 7f286affd640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 0x7f28540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.704+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f285c0cfa30 con 0x7f286c0ffe10 2024-04-02T13:32:47.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.705+0000 7f286affd640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 0x7f28540a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f286c199310 tx=0x7f2860005b60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:47.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.709+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f285c021070 con 0x7f286c0ffe10 2024-04-02T13:32:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:boot 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: fsmap 12 up:standby 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1112379828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1067388069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:47 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2990773186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:boot 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: fsmap 12 up:standby 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1112379828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1067388069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:47 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2990773186' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:32:47.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:47.890+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f286c02d370 con 0x7f286c0ffe10 2024-04-02T13:32:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.498+0000 7f2868ff9640 1 -- 172.21.15.122:0/2381656888 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f285c009de0 con 0x7f286c0ffe10 2024-04-02T13:32:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-04-02T13:32:48.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 msgr2=0x7f28540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 0x7f28540a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f286c199310 tx=0x7f2860005b60 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 msgr2=0x7f286c197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f285c00d950 tx=0x7f285c00de10 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 shutdown_connections 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f28540a3bb0 0x7f28540a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f286c103840 0x7f286c19f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f286c102e80 0x7f286c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 --2- 172.21.15.122:0/2381656888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f286c0ffe10 0x7f286c197e00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.500+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 >> 172.21.15.122:0/2381656888 conn(0x7f286c0fb700 msgr2=0x7f286c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.501+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 shutdown_connections 2024-04-02T13:32:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.501+0000 7f2871c0d640 1 -- 172.21.15.122:0/2381656888 wait complete. 2024-04-02T13:32:48.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-02T13:32:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 -- 172.21.15.122:0/874874187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24104f50 msgr2=0x7f8b24107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 --2- 172.21.15.122:0/874874187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24104f50 0x7f8b24107340 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009900 tx=0x7f8b0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 -- 172.21.15.122:0/874874187 shutdown_connections 2024-04-02T13:32:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 --2- 172.21.15.122:0/874874187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24107880 0x7f8b24109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 --2- 172.21.15.122:0/874874187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24104f50 0x7f8b24107340 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 --2- 172.21.15.122:0/874874187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24102620 0x7f8b24104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 -- 172.21.15.122:0/874874187 >> 172.21.15.122:0/874874187 conn(0x7f8b240fbfb0 msgr2=0x7f8b240fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 -- 172.21.15.122:0/874874187 shutdown_connections 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.623+0000 7f8b299a8640 1 -- 172.21.15.122:0/874874187 wait complete. 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 Processor -- start 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 -- start start 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24102620 0x7f8b24197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24104f50 0x7f8b241983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2410d770 con 0x7f8b24107880 2024-04-02T13:32:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2410d5f0 con 0x7f8b24104f50 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b299a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2410d8f0 con 0x7f8b24102620 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46470/0 (socket says 172.21.15.122:46470) 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 -- 172.21.15.122:0/2416742801 learned_addr learned my addr 172.21.15.122:0/2416742801 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b22ffd640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24102620 0x7f8b24197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b1bfff640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24104f50 0x7f8b241983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 -- 172.21.15.122:0/2416742801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24102620 msgr2=0x7f8b24197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24102620 0x7f8b24197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 -- 172.21.15.122:0/2416742801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24104f50 msgr2=0x7f8b241983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24104f50 0x7f8b241983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 -- 172.21.15.122:0/2416742801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b2419fa90 con 0x7f8b24107880 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b237fe640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009fd0 tx=0x7f8b1c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c017070 con 0x7f8b24107880 2024-04-02T13:32:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.624+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b1c003e50 con 0x7f8b24107880 2024-04-02T13:32:48.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.625+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b2419fd80 con 0x7f8b24107880 2024-04-02T13:32:48.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.625+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1c0106c0 con 0x7f8b24107880 2024-04-02T13:32:48.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.625+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b241a0240 con 0x7f8b24107880 2024-04-02T13:32:48.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.625+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b240fdaa0 con 0x7f8b24107880 2024-04-02T13:32:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.627+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8b1c003970 con 0x7f8b24107880 2024-04-02T13:32:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.627+0000 7f8b289a6640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 0x7f8b100a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.627+0000 7f8b22ffd640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 0x7f8b100a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.627+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f8b1c0cfd10 con 0x7f8b24107880 2024-04-02T13:32:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.628+0000 7f8b22ffd640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 0x7f8b100a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400bbe0 tx=0x7f8b1400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:48.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.631+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8b1c09c450 con 0x7f8b24107880 2024-04-02T13:32:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1561527965' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:32:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:32:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:48 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:48.809+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8b2402d370 con 0x7f8b24107880 2024-04-02T13:32:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1561527965' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:32:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:48 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1561527965' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-02T13:32:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:48.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:48 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.505+0000 7f8b289a6640 1 -- 172.21.15.122:0/2416742801 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7f8b1c0a0750 con 0x7f8b24107880 2024-04-02T13:32:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-04-02T13:32:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 msgr2=0x7f8b100a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 0x7f8b100a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400bbe0 tx=0x7f8b1400b390 comp rx=0 tx=0).stop 2024-04-02T13:32:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 msgr2=0x7f8b2419f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c009fd0 tx=0x7f8b1c00b260 comp rx=0 tx=0).stop 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 shutdown_connections 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b24107880 0x7f8b2419f420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b100a3c30 0x7f8b100a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b24104f50 0x7f8b241983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 --2- 172.21.15.122:0/2416742801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b24102620 0x7f8b24197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.508+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 >> 172.21.15.122:0/2416742801 conn(0x7f8b240fbfb0 msgr2=0x7f8b24103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:49.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.512+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 shutdown_connections 2024-04-02T13:32:49.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.512+0000 7f8b299a8640 1 -- 172.21.15.122:0/2416742801 wait complete. 2024-04-02T13:32:49.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-02T13:32:49.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 -- 172.21.15.122:0/4131904034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56781077f0 msgr2=0x7f5678109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:49.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 --2- 172.21.15.122:0/4131904034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56781077f0 0x7f5678109c40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5670009ca0 tx=0x7f567002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 -- 172.21.15.122:0/4131904034 shutdown_connections 2024-04-02T13:32:49.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 --2- 172.21.15.122:0/4131904034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56781077f0 0x7f5678109c40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 --2- 172.21.15.122:0/4131904034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678104ec0 0x7f56781072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 --2- 172.21.15.122:0/4131904034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5678102590 0x7f5678104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 -- 172.21.15.122:0/4131904034 >> 172.21.15.122:0/4131904034 conn(0x7f56780fbfa0 msgr2=0x7f56780fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 -- 172.21.15.122:0/4131904034 shutdown_connections 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.651+0000 7f567e9cd640 1 -- 172.21.15.122:0/4131904034 wait complete. 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 Processor -- start 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 -- start start 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678102590 0x7f5678197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 0x7f567819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:49.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f567810d790 con 0x7f5678104ec0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f567810d610 con 0x7f56781077f0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f567e9cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f567810d910 con 0x7f5678102590 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f56777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f56777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46484/0 (socket says 172.21.15.122:46484) 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.652+0000 7f56777fe640 1 -- 172.21.15.122:0/3340298496 learned_addr learned my addr 172.21.15.122:0/3340298496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f567cf6a640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 0x7f567819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f5677fff640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678102590 0x7f5678197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 -- 172.21.15.122:0/3340298496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678102590 msgr2=0x7f5678197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678102590 0x7f5678197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 -- 172.21.15.122:0/3340298496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 msgr2=0x7f567819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 0x7f567819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 -- 172.21.15.122:0/3340298496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567819fa20 con 0x7f5678104ec0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f567cf6a640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 0x7f567819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56777fe640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d770 tx=0x7f566c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:49.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c017070 con 0x7f5678104ec0 2024-04-02T13:32:49.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f566c009c30 con 0x7f5678104ec0 2024-04-02T13:32:49.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567819fd10 con 0x7f5678104ec0 2024-04-02T13:32:49.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c00f620 con 0x7f5678104ec0 2024-04-02T13:32:49.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.653+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a0200 con 0x7f5678104ec0 2024-04-02T13:32:49.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.654+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56780fda90 con 0x7f5678104ec0 2024-04-02T13:32:49.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.655+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f566c01a070 con 0x7f5678104ec0 2024-04-02T13:32:49.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.656+0000 7f56757fa640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 0x7f56600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:49.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.656+0000 7f5677fff640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 0x7f56600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:49.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.656+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7f566c0cfdc0 con 0x7f5678104ec0 2024-04-02T13:32:49.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.656+0000 7f5677fff640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 0x7f56600a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f5668005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:49.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.660+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f566c009de0 con 0x7f5678104ec0 2024-04-02T13:32:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:32:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:32:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:32:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:49 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:49.817 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:49 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2381656888' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: osdmap e83: 12 total, 12 up, 12 in 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:49.818 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:32:49.819 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:49 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:49.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:49.882+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f567802d370 con 0x7f5678104ec0 2024-04-02T13:32:50.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.521+0000 7f56757fa640 1 -- 172.21.15.122:0/3340298496 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f566c09c950 con 0x7f5678104ec0 2024-04-02T13:32:50.526 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 2 and data pool 3 2024-04-02T13:32:50.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 msgr2=0x7f56600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:50.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 0x7f56600a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f5668005850 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 msgr2=0x7f5678198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f566c00d770 tx=0x7f566c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 shutdown_connections 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56600a3bb0 0x7f56600a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56781077f0 0x7f567819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5678104ec0 0x7f5678198340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 --2- 172.21.15.122:0/3340298496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5678102590 0x7f5678197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.530+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 >> 172.21.15.122:0/3340298496 conn(0x7f56780fbfa0 msgr2=0x7f5678102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:50.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.531+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 shutdown_connections 2024-04-02T13:32:50.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.531+0000 7f567e9cd640 1 -- 172.21.15.122:0/3340298496 wait complete. 2024-04-02T13:32:50.553 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-02T13:32:50.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-02T13:32:50.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 -- 172.21.15.122:0/1656322423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 msgr2=0x7fdc5c1073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:50.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 --2- 172.21.15.122:0/1656322423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c1073c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 -- 172.21.15.122:0/1656322423 shutdown_connections 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 --2- 172.21.15.122:0/1656322423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 0x7fdc5c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 --2- 172.21.15.122:0/1656322423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c1073c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 --2- 172.21.15.122:0/1656322423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 0x7fdc5c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 -- 172.21.15.122:0/1656322423 >> 172.21.15.122:0/1656322423 conn(0x7fdc5c0fc010 msgr2=0x7fdc5c0fe450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 -- 172.21.15.122:0/1656322423 shutdown_connections 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.677+0000 7fdc61567640 1 -- 172.21.15.122:0/1656322423 wait complete. 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 Processor -- start 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 -- start start 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 0x7fdc5c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 0x7fdc5c19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d800 con 0x7fdc5c104fd0 2024-04-02T13:32:50.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d680 con 0x7fdc5c1026a0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc61567640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d980 con 0x7fdc5c107900 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc5a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc5a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46490/0 (socket says 172.21.15.122:46490) 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc5a7fc640 1 -- 172.21.15.122:0/1073754504 learned_addr learned my addr 172.21.15.122:0/1073754504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc5affd640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 0x7fdc5c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.678+0000 7fdc5b7fe640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 0x7fdc5c19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 -- 172.21.15.122:0/1073754504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 msgr2=0x7fdc5c19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 0x7fdc5c19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 -- 172.21.15.122:0/1073754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 msgr2=0x7fdc5c197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 0x7fdc5c197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 -- 172.21.15.122:0/1073754504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c19faf0 con 0x7fdc5c104fd0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc5a7fc640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005ab0 tx=0x7fdc4c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:50.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c041070 con 0x7fdc5c104fd0 2024-04-02T13:32:50.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c19fd80 con 0x7fdc5c104fd0 2024-04-02T13:32:50.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc4c0054e0 con 0x7fdc5c104fd0 2024-04-02T13:32:50.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c034440 con 0x7fdc5c104fd0 2024-04-02T13:32:50.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.679+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c1a0260 con 0x7fdc5c104fd0 2024-04-02T13:32:50.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.680+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c0fdb00 con 0x7fdc5c104fd0 2024-04-02T13:32:50.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.681+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fdc4c044070 con 0x7fdc5c104fd0 2024-04-02T13:32:50.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.682+0000 7fdc43fff640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 0x7fdc440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:50.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.682+0000 7fdc5affd640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 0x7fdc440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:50.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.682+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fdc4c0f1ea0 con 0x7fdc5c104fd0 2024-04-02T13:32:50.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.682+0000 7fdc5affd640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 0x7fdc440a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000bbe0 tx=0x7fdc5000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:50.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.686+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdc4c0be380 con 0x7fdc5c104fd0 2024-04-02T13:32:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:50 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:32:50.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:50 smithi122 ceph-mon[32114]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:32:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:50 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:32:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:50 smithi122 ceph-mon[32114]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:32:50.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:50.884+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdc5c02d540 con 0x7fdc5c104fd0 2024-04-02T13:32:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:50 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:32:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:50 smithi187 ceph-mon[36064]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:32:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:50 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:32:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:50 smithi187 ceph-mon[36064]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:32:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:50 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2416742801' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-02T13:32:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:50 smithi188 ceph-mon[36117]: osdmap e84: 12 total, 12 up, 12 in 2024-04-02T13:32:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:50 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-02T13:32:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:50 smithi188 ceph-mon[36117]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-02T13:32:51.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.526+0000 7fdc43fff640 1 -- 172.21.15.122:0/1073754504 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fdc4c0c2680 con 0x7fdc5c104fd0 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 msgr2=0x7fdc440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 0x7fdc440a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000bbe0 tx=0x7fdc5000b390 comp rx=0 tx=0).stop 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 msgr2=0x7fdc5c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c005ab0 tx=0x7fdc4c005dc0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 shutdown_connections 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc5c107900 0x7fdc5c19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc5c104fd0 0x7fdc5c198410 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdc440a3bb0 0x7fdc440a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 --2- 172.21.15.122:0/1073754504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc5c1026a0 0x7fdc5c197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 >> 172.21.15.122:0/1073754504 conn(0x7fdc5c0fc010 msgr2=0x7fdc5c103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:51.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 shutdown_connections 2024-04-02T13:32:51.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.529+0000 7fdc61567640 1 -- 172.21.15.122:0/1073754504 wait complete. 2024-04-02T13:32:51.545 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- 172.21.15.122:0/2916715908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 msgr2=0x7fae8c1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 --2- 172.21.15.122:0/2916715908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c1031a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009900 tx=0x7fae7c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- 172.21.15.122:0/2916715908 shutdown_connections 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 --2- 172.21.15.122:0/2916715908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 0x7fae8c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 --2- 172.21.15.122:0/2916715908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c1031a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 --2- 172.21.15.122:0/2916715908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 0x7fae8c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- 172.21.15.122:0/2916715908 >> 172.21.15.122:0/2916715908 conn(0x7fae8c0fd0e0 msgr2=0x7fae8c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- 172.21.15.122:0/2916715908 shutdown_connections 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- 172.21.15.122:0/2916715908 wait complete. 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 Processor -- start 2024-04-02T13:32:51.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.656+0000 7fae93d60640 1 -- start start 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 0x7fae8c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 0x7fae8c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10d7b0 con 0x7fae8c102d20 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10d630 con 0x7fae8c1036e0 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10d930 con 0x7fae8c101b30 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46508/0 (socket says 172.21.15.122:46508) 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 -- 172.21.15.122:0/49359927 learned_addr learned my addr 172.21.15.122:0/49359927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae922fd640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 0x7fae8c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae91afc640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 0x7fae8c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 -- 172.21.15.122:0/49359927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 msgr2=0x7fae8c197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 0x7fae8c197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 -- 172.21.15.122:0/49359927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 msgr2=0x7fae8c19f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 0x7fae8c19f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 -- 172.21.15.122:0/49359927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c19fa70 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae912fb640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009230 tx=0x7fae7c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c041070 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c19fd00 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7c0054e0 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c034440 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.657+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a01e0 con 0x7fae8c102d20 2024-04-02T13:32:51.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.658+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c108e50 con 0x7fae8c102d20 2024-04-02T13:32:51.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.659+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fae7c044070 con 0x7fae8c102d20 2024-04-02T13:32:51.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.660+0000 7fae82ffd640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 0x7fae780a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:51.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.660+0000 7fae91afc640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 0x7fae780a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:51.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.660+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7fae7c0f2010 con 0x7fae8c102d20 2024-04-02T13:32:51.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.660+0000 7fae91afc640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 0x7fae780a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae880058c0 tx=0x7fae8800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:51.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.664+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae7c0be370 con 0x7fae8c102d20 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: pgmap v282: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:32:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:active 2024-04-02T13:32:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:51 smithi122 ceph-mon[32114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:51.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:51.845+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fae8c02d370 con 0x7fae8c102d20 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: 2024-04-02T13:32:51.525+0000 7fb1c4b90700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: pgmap v282: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:active 2024-04-02T13:32:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:51 smithi187 ceph-mon[36064]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: pgmap v282: 97 pgs: 6 creating+peering, 1 active+clean, 90 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: osdmap e85: 12 total, 12 up, 12 in 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3340298496' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] up:active 2024-04-02T13:32:52.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:51 smithi188 ceph-mon[36117]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:52 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:32:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:52 smithi122 ceph-mon[32114]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:32:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:52 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:32:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:52 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:32:52.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:52 smithi187 ceph-mon[36064]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:32:52.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:52 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:32:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:52 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1073754504' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-02T13:32:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:52 smithi188 ceph-mon[36117]: osdmap e86: 12 total, 12 up, 12 in 2024-04-02T13:32:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:52 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-02T13:32:53.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.538+0000 7fae82ffd640 1 -- 172.21.15.122:0/49359927 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7fae7c0c2670 con 0x7fae8c102d20 2024-04-02T13:32:53.542 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data_ec' created 2024-04-02T13:32:53.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 msgr2=0x7fae780a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 0x7fae780a60f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae880058c0 tx=0x7fae8800a2d0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 msgr2=0x7fae8c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009230 tx=0x7fae7c005dd0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 shutdown_connections 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fae780a3c30 0x7fae780a60f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8c1036e0 0x7fae8c19f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae8c102d20 0x7fae8c198390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 --2- 172.21.15.122:0/49359927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae8c101b30 0x7fae8c197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.541+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 >> 172.21.15.122:0/49359927 conn(0x7fae8c0fd0e0 msgr2=0x7fae8c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:53.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.542+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 shutdown_connections 2024-04-02T13:32:53.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.542+0000 7fae93d60640 1 -- 172.21.15.122:0/49359927 wait complete. 2024-04-02T13:32:53.563 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 -- 172.21.15.122:0/443091268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62941025b0 msgr2=0x7f6294108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 --2- 172.21.15.122:0/443091268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62941025b0 0x7f6294108e40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f628c009ca0 tx=0x7f628c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 -- 172.21.15.122:0/443091268 shutdown_connections 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 --2- 172.21.15.122:0/443091268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62941025b0 0x7f6294108e40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 --2- 172.21.15.122:0/443091268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 0x7f6294102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 --2- 172.21.15.122:0/443091268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6294100a00 0x7f6294100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 -- 172.21.15.122:0/443091268 >> 172.21.15.122:0/443091268 conn(0x7f62940fbf70 msgr2=0x7f62940fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 -- 172.21.15.122:0/443091268 shutdown_connections 2024-04-02T13:32:53.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.670+0000 7f629ada8640 1 -- 172.21.15.122:0/443091268 wait complete. 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 Processor -- start 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 -- start start 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 0x7f6294198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 0x7f629419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c680 con 0x7f6294100a00 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c500 con 0x7f62941025b0 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f629ada8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f629410c800 con 0x7f6294101bf0 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f6298b44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f6298b44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46520/0 (socket says 172.21.15.122:46520) 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f6298b44640 1 -- 172.21.15.122:0/1521021067 learned_addr learned my addr 172.21.15.122:0/1521021067 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.671+0000 7f6299345640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 0x7f629419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 -- 172.21.15.122:0/1521021067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 msgr2=0x7f6294198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6293fff640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 0x7f6294198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 0x7f6294198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 -- 172.21.15.122:0/1521021067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 msgr2=0x7f629419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 0x7f629419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 -- 172.21.15.122:0/1521021067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629419fa00 con 0x7f6294100a00 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6299345640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 0x7f629419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6298b44640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f628400d950 tx=0x7f628400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284017020 con 0x7f6294100a00 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629419fcf0 con 0x7f6294100a00 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6284009c30 con 0x7f6294100a00 2024-04-02T13:32:53.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628400f720 con 0x7f6294100a00 2024-04-02T13:32:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.672+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62941a0190 con 0x7f6294100a00 2024-04-02T13:32:53.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.673+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62941a0480 con 0x7f6294100a00 2024-04-02T13:32:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.674+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f628401a020 con 0x7f6294100a00 2024-04-02T13:32:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.674+0000 7f6291ffb640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 0x7f627c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.675+0000 7f6293fff640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 0x7f627c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.675+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f62840d0660 con 0x7f6294100a00 2024-04-02T13:32:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.675+0000 7f6293fff640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 0x7f627c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6288005d80 tx=0x7f6288005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:53.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.679+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6284021020 con 0x7f6294100a00 2024-04-02T13:32:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:53 smithi122 ceph-mon[32114]: pgmap v285: 97 pgs: 38 active+clean, 13 creating+peering, 46 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-02T13:32:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:53 smithi122 ceph-mon[32114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:32:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:53 smithi122 ceph-mon[32114]: Cluster is now healthy 2024-04-02T13:32:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:53 smithi122 ceph-mon[32114]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:32:53.851 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:53 smithi187 ceph-mon[36064]: pgmap v285: 97 pgs: 38 active+clean, 13 creating+peering, 46 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-02T13:32:53.851 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:53 smithi187 ceph-mon[36064]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:32:53.851 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:53 smithi187 ceph-mon[36064]: Cluster is now healthy 2024-04-02T13:32:53.851 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:53 smithi187 ceph-mon[36064]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:32:53.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:53.857+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f62941a0650 con 0x7f6294100a00 2024-04-02T13:32:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:53 smithi188 ceph-mon[36117]: pgmap v285: 97 pgs: 38 active+clean, 13 creating+peering, 46 unknown; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-04-02T13:32:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:53 smithi188 ceph-mon[36117]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-02T13:32:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:53 smithi188 ceph-mon[36117]: Cluster is now healthy 2024-04-02T13:32:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:53 smithi188 ceph-mon[36117]: osdmap e87: 12 total, 12 up, 12 in 2024-04-02T13:32:54.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.543+0000 7f6291ffb640 1 -- 172.21.15.122:0/1521021067 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7f6284009de0 con 0x7f6294100a00 2024-04-02T13:32:54.547 INFO:teuthology.orchestra.run.smithi122.stderr:set pool 4 allow_ec_overwrites to true 2024-04-02T13:32:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 msgr2=0x7f627c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 0x7f627c0a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6288005d80 tx=0x7f6288005c90 comp rx=0 tx=0).stop 2024-04-02T13:32:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 msgr2=0x7f6294197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f628400d950 tx=0x7f628400de10 comp rx=0 tx=0).stop 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 shutdown_connections 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f627c0a3bb0 0x7f627c0a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62941025b0 0x7f629419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294101bf0 0x7f6294198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 --2- 172.21.15.122:0/1521021067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6294100a00 0x7f6294197de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 >> 172.21.15.122:0/1521021067 conn(0x7f62940fbf70 msgr2=0x7f62940fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 shutdown_connections 2024-04-02T13:32:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.547+0000 7f629ada8640 1 -- 172.21.15.122:0/1521021067 wait complete. 2024-04-02T13:32:54.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-02T13:32:54.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 -- 172.21.15.122:0/162352951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 msgr2=0x7f3a98108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:54.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 --2- 172.21.15.122:0/162352951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a98108e40 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009900 tx=0x7f3a9002bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 -- 172.21.15.122:0/162352951 shutdown_connections 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 --2- 172.21.15.122:0/162352951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a98108e40 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 --2- 172.21.15.122:0/162352951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 --2- 172.21.15.122:0/162352951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 0x7f3a98100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 -- 172.21.15.122:0/162352951 >> 172.21.15.122:0/162352951 conn(0x7f3a980fbf90 msgr2=0x7f3a980fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 -- 172.21.15.122:0/162352951 shutdown_connections 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.673+0000 7f3a9f332640 1 -- 172.21.15.122:0/162352951 wait complete. 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 Processor -- start 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 -- start start 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 0x7f3a98197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:54.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9810d7a0 con 0x7f3a981025b0 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9810d620 con 0x7f3a98100a00 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9f332640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a9810d920 con 0x7f3a98101bf0 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9d8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9d8cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46538/0 (socket says 172.21.15.122:46538) 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9d8cf640 1 -- 172.21.15.122:0/1570330138 learned_addr learned my addr 172.21.15.122:0/1570330138 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9d0ce640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 0x7f3a98197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.674+0000 7f3a9c8cd640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 -- 172.21.15.122:0/1570330138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 msgr2=0x7f3a98198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 -- 172.21.15.122:0/1570330138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 msgr2=0x7f3a98197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 0x7f3a98197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:54.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 -- 172.21.15.122:0/1570330138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9819fa40 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9c8cd640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9d8cf640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a9002c7b0 tx=0x7f3a90005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90041070 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a900054e0 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a9819fcd0 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90034440 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.675+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a981a01c0 con 0x7f3a981025b0 2024-04-02T13:32:54.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.676+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a98107d20 con 0x7f3a981025b0 2024-04-02T13:32:54.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.677+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3a90044070 con 0x7f3a981025b0 2024-04-02T13:32:54.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.677+0000 7f3a8e7fc640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 0x7f3a840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:54.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.678+0000 7f3a9d0ce640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 0x7f3a840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:54.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.678+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f3a900f2640 con 0x7f3a981025b0 2024-04-02T13:32:54.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.678+0000 7f3a9d0ce640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 0x7f3a840a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a98101a50 tx=0x7f3a94009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:54.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.681+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3a900bee00 con 0x7f3a981025b0 2024-04-02T13:32:54.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:54 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:32:54.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:54 smithi187 ceph-mon[36064]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:32:54.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:54 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:32:54.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:54 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:32:54.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:54 smithi122 ceph-mon[32114]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:32:54.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:54 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:32:54.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:54.912+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f3a9802d410 con 0x7f3a981025b0 2024-04-02T13:32:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:54 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/49359927' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-02T13:32:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:54 smithi188 ceph-mon[36117]: osdmap e88: 12 total, 12 up, 12 in 2024-04-02T13:32:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:54 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-02T13:32:55.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.545+0000 7f3a8e7fc640 1 -- 172.21.15.122:0/1570330138 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f3a900be700 con 0x7f3a981025b0 2024-04-02T13:32:55.549 INFO:teuthology.orchestra.run.smithi122.stderr:added data pool 4 to fsmap 2024-04-02T13:32:55.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.547+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 msgr2=0x7f3a840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.547+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 0x7f3a840a6070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3a98101a50 tx=0x7f3a94009500 comp rx=0 tx=0).stop 2024-04-02T13:32:55.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.547+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 msgr2=0x7f3a9819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.547+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3a9002c7b0 tx=0x7f3a90005dd0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 shutdown_connections 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a981025b0 0x7f3a9819f3d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a98101bf0 0x7f3a98198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3a840a3bb0 0x7f3a840a6070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 --2- 172.21.15.122:0/1570330138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3a98100a00 0x7f3a98197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 >> 172.21.15.122:0/1570330138 conn(0x7f3a980fbf90 msgr2=0x7f3a98104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 shutdown_connections 2024-04-02T13:32:55.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.548+0000 7f3a9f332640 1 -- 172.21.15.122:0/1570330138 wait complete. 2024-04-02T13:32:55.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:55.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 -- 172.21.15.122:0/2383637408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c81025d0 msgr2=0x7ff2c8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/2383637408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c81025d0 0x7ff2c8108e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009ca0 tx=0x7ff2c002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 -- 172.21.15.122:0/2383637408 shutdown_connections 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/2383637408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c81025d0 0x7ff2c8108e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/2383637408 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 0x7ff2c8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/2383637408 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c8100a20 0x7ff2c8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.664+0000 7ff2cf86a640 1 -- 172.21.15.122:0/2383637408 >> 172.21.15.122:0/2383637408 conn(0x7ff2c80fbf90 msgr2=0x7ff2c80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- 172.21.15.122:0/2383637408 shutdown_connections 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- 172.21.15.122:0/2383637408 wait complete. 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 Processor -- start 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- start start 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:55.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 0x7ff2c8197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c81025d0 0x7ff2c819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810c790 con 0x7ff2c8100a20 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810c610 con 0x7ff2c8101c10 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cf86a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c810c910 con 0x7ff2c81025d0 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cd606640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cd606640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46556/0 (socket says 172.21.15.122:46556) 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.665+0000 7ff2cd606640 1 -- 172.21.15.122:0/192198424 learned_addr learned my addr 172.21.15.122:0/192198424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cce05640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 0x7ff2c8197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cde07640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c81025d0 0x7ff2c819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 -- 172.21.15.122:0/192198424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c81025d0 msgr2=0x7ff2c819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c81025d0 0x7ff2c819e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 -- 172.21.15.122:0/192198424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 msgr2=0x7ff2c8197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 0x7ff2c8197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 -- 172.21.15.122:0/192198424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c819e9f0 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cd606640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b450 tx=0x7ff2b800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b8015020 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b8011d70 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c819ece0 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b800f340 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.666+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c819f180 con 0x7ff2c8100a20 2024-04-02T13:32:55.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.667+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c8107ce0 con 0x7ff2c8100a20 2024-04-02T13:32:55.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.668+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff2b8018020 con 0x7ff2c8100a20 2024-04-02T13:32:55.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.668+0000 7ff2be7fc640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 0x7ff2b40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:55.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.669+0000 7ff2cce05640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 0x7ff2b40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:55.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.669+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff2b80d01e0 con 0x7ff2c8100a20 2024-04-02T13:32:55.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.669+0000 7ff2cce05640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 0x7ff2b40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2c81982e0 tx=0x7ff2c4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:55.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.674+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff2b801f020 con 0x7ff2c8100a20 2024-04-02T13:32:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:55 smithi122 ceph-mon[32114]: pgmap v288: 129 pgs: 90 active+clean, 11 creating+peering, 28 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-02T13:32:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:55 smithi122 ceph-mon[32114]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:55 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:32:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:55 smithi122 ceph-mon[32114]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:32:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:55 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:32:55.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.897+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2c8062700 con 0x7ff2c8100a20 2024-04-02T13:32:55.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.898+0000 7ff2be7fc640 1 -- 172.21.15.122:0/192198424 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7ff2b800d040 con 0x7ff2c8100a20 2024-04-02T13:32:55.900 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:55.901 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:32:54.914221+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24512,"qdb_cluster":[24512]},"id":1}]} 2024-04-02T13:32:55.901 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 16 2024-04-02T13:32:55.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 msgr2=0x7ff2b40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 0x7ff2b40a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2c81982e0 tx=0x7ff2c4008040 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 msgr2=0x7ff2c8196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800b450 tx=0x7ff2b800b910 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 shutdown_connections 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2c81025d0 0x7ff2c819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff2b40a3bb0 0x7ff2b40a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2c8101c10 0x7ff2c8197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 --2- 172.21.15.122:0/192198424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2c8100a20 0x7ff2c8196dd0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 >> 172.21.15.122:0/192198424 conn(0x7ff2c80fbf90 msgr2=0x7ff2c8104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 shutdown_connections 2024-04-02T13:32:55.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:55.900+0000 7ff2cf86a640 1 -- 172.21.15.122:0/192198424 wait complete. 2024-04-02T13:32:55.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:32:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:55 smithi187 ceph-mon[36064]: pgmap v288: 129 pgs: 90 active+clean, 11 creating+peering, 28 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-02T13:32:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:55 smithi187 ceph-mon[36064]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:55 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:32:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:55 smithi187 ceph-mon[36064]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:32:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:55 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:32:56.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 -- 172.21.15.122:0/1807764648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f4104ed0 msgr2=0x7fa0f41072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1807764648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f4104ed0 0x7fa0f41072c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009900 tx=0x7fa0e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 -- 172.21.15.122:0/1807764648 shutdown_connections 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1807764648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 0x7fa0f4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1807764648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f4104ed0 0x7fa0f41072c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1807764648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f41025a0 0x7fa0f4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 -- 172.21.15.122:0/1807764648 >> 172.21.15.122:0/1807764648 conn(0x7fa0f40fbf90 msgr2=0x7fa0f40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 -- 172.21.15.122:0/1807764648 shutdown_connections 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.011+0000 7fa0f9932640 1 -- 172.21.15.122:0/1807764648 wait complete. 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 Processor -- start 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 -- start start 2024-04-02T13:32:56.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f4104ed0 0x7fa0f419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 0x7fa0f41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410d7e0 con 0x7fa0f41025a0 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410d660 con 0x7fa0f4104ed0 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f9932640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f410d960 con 0x7fa0f4107800 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46584/0 (socket says 172.21.15.122:46584) 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.012+0000 7fa0f2ffd640 1 -- 172.21.15.122:0/1131303447 learned_addr learned my addr 172.21.15.122:0/1131303447 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f27fc640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f4104ed0 0x7fa0f419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f37fe640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 0x7fa0f41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 -- 172.21.15.122:0/1131303447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 msgr2=0x7fa0f41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 0x7fa0f41a1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 -- 172.21.15.122:0/1131303447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f4104ed0 msgr2=0x7fa0f419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f4104ed0 0x7fa0f419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 -- 172.21.15.122:0/1131303447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f41a1cb0 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f2ffd640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400cce0 tx=0x7fa0e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4004b20 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4004cc0 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f41a1fa0 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400f7d0 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.013+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41a2400 con 0x7fa0f41025a0 2024-04-02T13:32:56.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.014+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f4109350 con 0x7fa0f41025a0 2024-04-02T13:32:56.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:55 smithi188 ceph-mon[36117]: pgmap v288: 129 pgs: 90 active+clean, 11 creating+peering, 28 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-02T13:32:56.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:55 smithi188 ceph-mon[36117]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-02T13:32:56.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:55 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1521021067' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-02T13:32:56.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:55 smithi188 ceph-mon[36117]: osdmap e89: 12 total, 12 up, 12 in 2024-04-02T13:32:56.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:55 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-02T13:32:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.016+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa0e400f970 con 0x7fa0f41025a0 2024-04-02T13:32:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.016+0000 7fa0f8930640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 0x7fa0dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.016+0000 7fa0f27fc640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 0x7fa0dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.016+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa0e40d0b10 con 0x7fa0f41025a0 2024-04-02T13:32:56.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.017+0000 7fa0f27fc640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 0x7fa0dc0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0e802c170 tx=0x7fa0e802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.020+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa0e4014030 con 0x7fa0f41025a0 2024-04-02T13:32:56.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.199+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0f402d600 con 0x7fa0f41025a0 2024-04-02T13:32:56.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.200+0000 7fa0f8930640 1 -- 172.21.15.122:0/1131303447 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7fa0e409d2b0 con 0x7fa0f41025a0 2024-04-02T13:32:56.203 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:56.203 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:56.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.202+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 msgr2=0x7fa0dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.202+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 0x7fa0dc0a6070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa0e802c170 tx=0x7fa0e802df90 comp rx=0 tx=0).stop 2024-04-02T13:32:56.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.202+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 msgr2=0x7fa0f419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.202+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400cce0 tx=0x7fa0e400a620 comp rx=0 tx=0).stop 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 shutdown_connections 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f4107800 0x7fa0f41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa0dc0a3bb0 0x7fa0dc0a6070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f4104ed0 0x7fa0f419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 --2- 172.21.15.122:0/1131303447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa0f41025a0 0x7fa0f419a090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 >> 172.21.15.122:0/1131303447 conn(0x7fa0f40fbf90 msgr2=0x7fa0f4101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 shutdown_connections 2024-04-02T13:32:56.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.203+0000 7fa0f9932640 1 -- 172.21.15.122:0/1131303447 wait complete. 2024-04-02T13:32:56.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:32:56.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/3866034869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa0101d80 msgr2=0x7f9fa0108610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/3866034869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa0101d80 0x7f9fa0108610 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009900 tx=0x7f9f9002bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/3866034869 shutdown_connections 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/3866034869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa0101d80 0x7f9fa0108610 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/3866034869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01013c0 0x7f9fa0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/3866034869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa01001d0 0x7f9fa01005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/3866034869 >> 172.21.15.122:0/3866034869 conn(0x7f9fa00fb740 msgr2=0x7f9fa00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/3866034869 shutdown_connections 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/3866034869 wait complete. 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 Processor -- start 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.316+0000 7f9fa6f7b640 1 -- start start 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01001d0 0x7f9fa0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 0x7f9fa019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa010be30 con 0x7f9fa01013c0 2024-04-02T13:32:56.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa010bcb0 con 0x7f9fa01001d0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa010bfb0 con 0x7f9fa0101d80 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46598/0 (socket says 172.21.15.122:46598) 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 -- 172.21.15.122:0/1542522304 learned_addr learned my addr 172.21.15.122:0/1542522304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa4d17640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01001d0 0x7f9fa0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa5518640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 0x7f9fa019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 -- 172.21.15.122:0/1542522304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 msgr2=0x7f9fa019f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 0x7f9fa019f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 -- 172.21.15.122:0/1542522304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01001d0 msgr2=0x7f9fa0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01001d0 0x7f9fa0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 -- 172.21.15.122:0/1542522304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa019f9b0 con 0x7f9fa01013c0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa5518640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 0x7f9fa019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9ffff640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800d950 tx=0x7f9f8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88017070 con 0x7f9fa01013c0 2024-04-02T13:32:56.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f88009c30 con 0x7f9fa01013c0 2024-04-02T13:32:56.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa019fca0 con 0x7f9fa01013c0 2024-04-02T13:32:56.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8800f6b0 con 0x7f9fa01013c0 2024-04-02T13:32:56.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.317+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01a0190 con 0x7f9fa01013c0 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.318+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa01a0420 con 0x7f9fa01013c0 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.319+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9f880109d0 con 0x7f9fa01013c0 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.320+0000 7f9f9dffb640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 0x7f9f8c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.320+0000 7f9fa4d17640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 0x7f9f8c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.320+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9f880d0270 con 0x7f9fa01013c0 2024-04-02T13:32:56.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.320+0000 7f9fa4d17640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 0x7f9f8c0a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0199270 tx=0x7f9f98007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.324+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f88009de0 con 0x7f9fa01013c0 2024-04-02T13:32:56.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.505+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9fa01a07b0 con 0x7f9fa01013c0 2024-04-02T13:32:56.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.506+0000 7f9f9dffb640 1 -- 172.21.15.122:0/1542522304 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f9f8809ca10 con 0x7f9fa01013c0 2024-04-02T13:32:56.509 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:56.509 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:56.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 msgr2=0x7f9f8c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 0x7f9f8c0a60f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0199270 tx=0x7f9f98007600 comp rx=0 tx=0).stop 2024-04-02T13:32:56.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 msgr2=0x7f9fa0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800d950 tx=0x7f9f8800de10 comp rx=0 tx=0).stop 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 shutdown_connections 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0101d80 0x7f9fa019f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9fa01013c0 0x7f9fa0198330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9f8c0a3c30 0x7f9f8c0a60f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 --2- 172.21.15.122:0/1542522304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fa01001d0 0x7f9fa0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 >> 172.21.15.122:0/1542522304 conn(0x7f9fa00fb740 msgr2=0x7f9fa00fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 shutdown_connections 2024-04-02T13:32:56.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.509+0000 7f9fa6f7b640 1 -- 172.21.15.122:0/1542522304 wait complete. 2024-04-02T13:32:56.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:32:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- 172.21.15.122:0/2856019343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 msgr2=0x7f1ae010a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- 172.21.15.122:0/2856019343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae010a010 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009cd0 tx=0x7f1ad802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- 172.21.15.122:0/2856019343 shutdown_connections 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- 172.21.15.122:0/2856019343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae010a010 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- 172.21.15.122:0/2856019343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae0102dc0 0x7f1ae0103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- 172.21.15.122:0/2856019343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- 172.21.15.122:0/2856019343 >> 172.21.15.122:0/2856019343 conn(0x7f1ae00fb6e0 msgr2=0x7f1ae00fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- 172.21.15.122:0/2856019343 shutdown_connections 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- 172.21.15.122:0/2856019343 wait complete. 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 Processor -- start 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- start start 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae0102dc0 0x7f1ae01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d840 con 0x7f1ae0103780 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d6c0 con 0x7f1ae0102dc0 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.637+0000 7f1ae553d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae010d9c0 con 0x7f1ae00ffdf0 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46628/0 (socket says 172.21.15.122:46628) 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 -- 172.21.15.122:0/1279784620 learned_addr learned my addr 172.21.15.122:0/1279784620 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adeffd640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 -- 172.21.15.122:0/1279784620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 msgr2=0x7f1ae0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 -- 172.21.15.122:0/1279784620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae0102dc0 msgr2=0x7f1ae01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae0102dc0 0x7f1ae01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 -- 172.21.15.122:0/1279784620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae019faa0 con 0x7f1ae0103780 2024-04-02T13:32:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adeffd640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adf7fe640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8005e30 tx=0x7f1ad8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8039040 con 0x7f1ae0103780 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae019fd30 con 0x7f1ae0103780 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ad8044070 con 0x7f1ae0103780 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8004280 con 0x7f1ae0103780 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.638+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a0220 con 0x7f1ae0103780 2024-04-02T13:32:56.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.639+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0108ef0 con 0x7f1ae0103780 2024-04-02T13:32:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.640+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1ad8003970 con 0x7f1ae0103780 2024-04-02T13:32:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.641+0000 7f1adcff9640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 0x7f1acc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.641+0000 7f1adeffd640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 0x7f1acc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.641+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1ad80f3130 con 0x7f1ae0103780 2024-04-02T13:32:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.641+0000 7f1adeffd640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 0x7f1acc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0069f10 tx=0x7f1ad0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.645+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1ad80bf8d0 con 0x7f1ae0103780 2024-04-02T13:32:56.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:32:56.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:32:56.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:56.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/192198424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1131303447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1542522304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.862+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae002d370 con 0x7f1ae0103780 2024-04-02T13:32:56.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.863+0000 7f1adcff9640 1 -- 172.21.15.122:0/1279784620 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11809 (secure 0 0 0) 0x7f1ad8035d60 con 0x7f1ae0103780 2024-04-02T13:32:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:56.866 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24448,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24455,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:32:54.914221+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24512},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24512,"qdb_cluster":[24512]},"id":1}]} 2024-04-02T13:32:56.866 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 16 2024-04-02T13:32:56.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 msgr2=0x7f1acc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 0x7f1acc0a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0069f10 tx=0x7f1ad0008040 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 msgr2=0x7f1ae019f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8005e30 tx=0x7f1ad8034760 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 shutdown_connections 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ae0103780 0x7f1ae019f430 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1acc0a3bb0 0x7f1acc0a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ae0102dc0 0x7f1ae01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 --2- 172.21.15.122:0/1279784620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae00ffdf0 0x7f1ae0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 >> 172.21.15.122:0/1279784620 conn(0x7f1ae00fb6e0 msgr2=0x7f1ae01060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 shutdown_connections 2024-04-02T13:32:56.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.865+0000 7f1ae553d640 1 -- 172.21.15.122:0/1279784620 wait complete. 2024-04-02T13:32:56.879 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3146391739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 msgr2=0x7f9ad0107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3146391739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0107380 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009900 tx=0x7f9abc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3146391739 shutdown_connections 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3146391739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad01078c0 0x7f9ad0109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3146391739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0107380 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3146391739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad0102660 0x7f9ad0104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3146391739 >> 172.21.15.122:0/3146391739 conn(0x7f9ad00fbfb0 msgr2=0x7f9ad00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3146391739 shutdown_connections 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.977+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3146391739 wait complete. 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 Processor -- start 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 -- start start 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad0102660 0x7f9ad0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 0x7f9ad019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010d890 con 0x7f9ad0104f90 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010d710 con 0x7f9ad0102660 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010da10 con 0x7f9ad01078c0 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46640/0 (socket says 172.21.15.122:46640) 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 -- 172.21.15.122:0/3743195781 learned_addr learned my addr 172.21.15.122:0/3743195781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad556a640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 0x7f9ad019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 -- 172.21.15.122:0/3743195781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 msgr2=0x7f9ad019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 0x7f9ad019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 -- 172.21.15.122:0/3743195781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad0102660 msgr2=0x7f9ad0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad0102660 0x7f9ad0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 -- 172.21.15.122:0/3743195781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad019fb60 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad556a640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 0x7f9ad019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acffff640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc041070 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad019fdf0 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9abc0054e0 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.978+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0346a0 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.979+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01a0280 con 0x7f9ad0104f90 2024-04-02T13:32:56.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.980+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad00fdaa0 con 0x7f9ad0104f90 2024-04-02T13:32:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.981+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f9abc044070 con 0x7f9ad0104f90 2024-04-02T13:32:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.981+0000 7f9acdffb640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 0x7f9ac00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.981+0000 7f9ad4d69640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 0x7f9ac00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.981+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f9abc0f2500 con 0x7f9ad0104f90 2024-04-02T13:32:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.982+0000 7f9ad4d69640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 0x7f9ac00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009710 tx=0x7f9ac4006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/192198424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1131303447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1542522304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:56.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:56.986+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9abc0beca0 con 0x7f9ad0104f90 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1570330138' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: osdmap e90: 12 total, 12 up, 12 in 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/192198424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1131303447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1542522304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.167+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad00627a0 con 0x7f9ad0104f90 2024-04-02T13:32:57.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.168+0000 7f9acdffb640 1 -- 172.21.15.122:0/3743195781 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f9abc0be5a0 con 0x7f9ad0104f90 2024-04-02T13:32:57.171 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:32:57.171 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:32:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 msgr2=0x7f9ac00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 0x7f9ac00a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4009710 tx=0x7f9ac4006c90 comp rx=0 tx=0).stop 2024-04-02T13:32:57.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 msgr2=0x7f9ad0198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000c00 tx=0x7f9abc004040 comp rx=0 tx=0).stop 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 shutdown_connections 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ad01078c0 0x7f9ad019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad0104f90 0x7f9ad0198480 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f9ac00a3bb0 0x7f9ac00a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 --2- 172.21.15.122:0/3743195781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad0102660 0x7f9ad0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.170+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 >> 172.21.15.122:0/3743195781 conn(0x7f9ad00fbfb0 msgr2=0x7f9ad0102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.171+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 shutdown_connections 2024-04-02T13:32:57.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.171+0000 7f9ad6fcd640 1 -- 172.21.15.122:0/3743195781 wait complete. 2024-04-02T13:32:57.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-02T13:32:49.884932+0000', 'modified': '2024-04-02T13:32:54.914221+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24512}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6839/2503909279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 2503909279}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 2503909279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24512, 'qdb_cluster': [24512]} 2024-04-02T13:32:57.186 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-02T13:32:57.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:32:57.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1954588986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec1025c0 msgr2=0x7f1fec108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1954588986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec1025c0 0x7f1fec108e50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009900 tx=0x7f1fe402bc30 comp rx=0 tx=0).stop 2024-04-02T13:32:57.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1954588986 shutdown_connections 2024-04-02T13:32:57.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1954588986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec1025c0 0x7f1fec108e50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1954588986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec101c00 0x7f1fec102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1954588986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec100a10 0x7f1fec100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.286+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1954588986 >> 172.21.15.122:0/1954588986 conn(0x7f1fec0fbfa0 msgr2=0x7f1fec0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1954588986 shutdown_connections 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1954588986 wait complete. 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 Processor -- start 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- start start 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec100a10 0x7f1fec06ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec1025c0 0x7f1fec1a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10c690 con 0x7f1fec101c00 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10c510 con 0x7f1fec1025c0 2024-04-02T13:32:57.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff3e55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec10c810 con 0x7f1fec100a10 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46654/0 (socket says 172.21.15.122:46654) 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1207525223 learned_addr learned my addr 172.21.15.122:0/1207525223 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec100a10 0x7f1fec06ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.287+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec1025c0 0x7f1fec1a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1207525223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec100a10 msgr2=0x7f1fec06ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec100a10 0x7f1fec06ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1207525223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec1025c0 msgr2=0x7f1fec1a1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec1025c0 0x7f1fec1a1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1207525223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec1a1c70 con 0x7f1fec101c00 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c970 tx=0x7f1fe000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1feaffd640 1 -- 172.21.15.122:0/1207525223 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0007bb0 con 0x7f1fec101c00 2024-04-02T13:32:57.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1feaffd640 1 -- 172.21.15.122:0/1207525223 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe0007d50 con 0x7f1fec101c00 2024-04-02T13:32:57.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1feaffd640 1 -- 172.21.15.122:0/1207525223 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000f450 con 0x7f1fec101c00 2024-04-02T13:32:57.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec1a1f00 con 0x7f1fec101c00 2024-04-02T13:32:57.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.288+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a23c0 con 0x7f1fec101c00 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1feaffd640 1 -- 172.21.15.122:0/1207525223 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1fe0004fe0 con 0x7f1fec101c00 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 Processor -- start 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 -- start start 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ad9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603d6a39060 con 0x5603d6ada640 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603d69e1650 con 0x5603d6ad4ae0 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.291+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603d6a35010 con 0x5603d6ad54d0 2024-04-02T13:32:57.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46656/0 (socket says 172.21.15.122:46656) 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 -- 172.21.15.122:0/2270689129 learned_addr learned my addr 172.21.15.122:0/2270689129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ad9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1feaffd640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 0x7f1fd40a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 -- 172.21.15.122:0/2270689129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 msgr2=0x5603d6ad9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ad9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 -- 172.21.15.122:0/2270689129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 msgr2=0x5603d6ad4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 -- 172.21.15.122:0/2270689129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d674d2d0 con 0x5603d6ada640 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:57.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 0x7f1fd40a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1feaffd640 1 -- 172.21.15.122:0/1207525223 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1fe00d0200 con 0x7f1fec101c00 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0094c20 tx=0x7f1fe00918d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6331dbd731528c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1fd3fff640 1 -- 172.21.15.122:0/2270689129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe009bf30 con 0x5603d6ada640 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 0x7f1fd40a6120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005ab0 tx=0x7f1fe402e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1fd3fff640 1 -- 172.21.15.122:0/2270689129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe00951f0 con 0x5603d6ada640 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.292+0000 7f1fd3fff640 1 -- 172.21.15.122:0/2270689129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00d3e60 con 0x5603d6ada640 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/2270689129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 msgr2=0x5603d6adcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0094c20 tx=0x7f1fe00918d0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/2270689129 shutdown_connections 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ada640 0x5603d6adcad0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ad9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/2270689129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/2270689129 >> 172.21.15.122:0/2270689129 conn(0x5603d6ad0050 msgr2=0x5603d6ad0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/2270689129 shutdown_connections 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.293+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/2270689129 wait complete. 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 -- start start 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 Processor -- start 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6adf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 0x5603d6ae6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603d6add590 con 0x5603d6ad54d0 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603d69f4a10 con 0x5603d6ad4ae0 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603d6add710 con 0x5603d6ada640 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff6a3b740 10 client.0 _ll_register_callbacks cb 0x5603d6a310b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46660/0 (socket says 172.21.15.122:46660) 2024-04-02T13:32:57.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 -- 172.21.15.122:0/3508058662 learned_addr learned my addr 172.21.15.122:0/3508058662 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 0x5603d6ae6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 -- 172.21.15.122:0/3508058662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 msgr2=0x5603d6ae6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6adf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 0x5603d6ae6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 -- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 msgr2=0x5603d6adf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6adf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff13f0640 1 -- 172.21.15.122:0/3508058662 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d68176b0 con 0x5603d6ad54d0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 0x5603d6ae6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.294+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6adf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c940 tx=0x7f1fe00076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9d9d2bf900b583 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe001d070 con 0x5603d6ad54d0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1fe0098280 con 0x5603d6ad54d0 2024-04-02T13:32:57.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603d69e0cf0 con 0x5603d6ad54d0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1fe00d3e60 con 0x5603d6ad54d0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 10 client.14748 fetch_fsmap learned FSMap version 16 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5603d6ae98b0 con 0x5603d6ad54d0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5603d69e0cf0 con 0x5603d6ad54d0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f1fe00052d0 con 0x5603d6ad54d0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 10 client.14748 fetch_fsmap finished waiting for FSMap version 16 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 20 client.14748 populate_metadata read hostname 'smithi122' 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6aeba40 0x5603d6aeceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 4 client.14748 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d6a465a0 con 0x5603d6aeba40 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.295+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6aeba40 0x5603d6aeceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.296+0000 7f1fd17fa640 10 client.14748 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:32:57.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.296+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6aeba40 0x5603d6aeceb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb2f9658f5b3cdec server_cookie=86fc030ffa6c68a1 in_seq=0 out_seq=0 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1fd17fa640 1 -- 172.21.15.122:0/3508058662 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603d6a465a0 con 0x5603d6aeba40 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1fd17fa640 10 client.14748 handle_command_reply: tid=0 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 client.14748 shutdown 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 msgr2=0x5603d6adf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c940 tx=0x7f1fe00076c0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 shutdown_connections 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6aeba40 0x5603d6aeceb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ada640 0x5603d6ae6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6adf570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/3508058662 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ad4ae0 0x5603d6adf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1fd17fa640 0 client.14748 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 >> 172.21.15.122:0/3508058662 conn(0x5603d6ad0050 msgr2=0x5603d6acf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 shutdown_connections 2024-04-02T13:32:57.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/3508058662 wait complete. 2024-04-02T13:32:57.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.297+0000 7f1ff6a3b740 20 client.14748 trim_cache size 0 max 16384 2024-04-02T13:32:57.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 Processor -- start 2024-04-02T13:32:57.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 -- start start 2024-04-02T13:32:57.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ae6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ace730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aeb1f0 con 0x5603d6ad4ae0 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aeb8c0 con 0x5603d6ada640 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aebf90 con 0x5603d6ad54d0 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46662/0 (socket says 172.21.15.122:46662) 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 -- 172.21.15.122:0/1196504612 learned_addr learned my addr 172.21.15.122:0/1196504612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ae6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 -- 172.21.15.122:0/1196504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 msgr2=0x5603d6ae6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ae6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 -- 172.21.15.122:0/1196504612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 msgr2=0x5603d6ace730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ace730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff1bf1640 1 -- 172.21.15.122:0/1196504612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d6aed1e0 con 0x5603d6ad4ae0 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.315+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ae6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5603d6ad5f40 tx=0x7f1fdc013c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6319b27ecf39b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1fd37fe640 1 -- 172.21.15.122:0/1196504612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc01f640 con 0x5603d6ad4ae0 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1fd37fe640 1 -- 172.21.15.122:0/1196504612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc023030 con 0x5603d6ad4ae0 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1fd37fe640 1 -- 172.21.15.122:0/1196504612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc027aa0 con 0x5603d6ad4ae0 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1196504612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 msgr2=0x5603d6ad0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5603d6ad5f40 tx=0x7f1fdc013c10 comp rx=0 tx=0).stop 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1196504612 shutdown_connections 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ace730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad54d0 0x5603d6ae6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1196504612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad4ae0 0x5603d6ad0430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1196504612 >> 172.21.15.122:0/1196504612 conn(0x5603d6ad0050 msgr2=0x5603d6ad5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1196504612 shutdown_connections 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.316+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1196504612 wait complete. 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 -- start start 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 Processor -- start 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad4ae0 0x5603d6ae2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ae5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aecd30 con 0x5603d6ad54d0 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aec660 con 0x5603d6ada640 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603d6aed400 con 0x5603d6ad4ae0 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff6a3b740 10 client.0 _ll_register_callbacks cb 0x7f1fdc009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46674/0 (socket says 172.21.15.122:46674) 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1332781929 learned_addr learned my addr 172.21.15.122:0/1332781929 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ae5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1332781929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad4ae0 msgr2=0x5603d6ae2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad4ae0 0x5603d6ae2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 msgr2=0x5603d6ae5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ae5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 -- 172.21.15.122:0/1332781929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d6ae5c20 con 0x5603d6ad54d0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff23f2640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ae5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1ff13f0640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0098d70 tx=0x7f1fe000a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87007512425266 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.317+0000 7f1fd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe001d070 con 0x5603d6ad54d0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1fe00953b0 con 0x5603d6ad54d0 2024-04-02T13:32:57.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fdc023030 con 0x5603d6ad54d0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f1fe009bd50 con 0x5603d6ad54d0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 10 client.14754 fetch_fsmap learned FSMap version 16 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5603d6b51e40 con 0x5603d6ad54d0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc023030 con 0x5603d6ad54d0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 16) ==== 8342+0+0 (secure 0 0 0) 0x7f1fe00cbbd0 con 0x5603d6ad54d0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 10 client.14754 fetch_fsmap finished waiting for FSMap version 16 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 10 client.14754 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 20 client.14754 populate_metadata read hostname 'smithi122' 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6b568b0 0x5603d6b555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 4 client.14754 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "status"}) -- 0x5603d6a408f0 con 0x5603d6b568b0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.318+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6b568b0 0x5603d6b555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff1bf1640 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6b568b0 0x5603d6b555c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17e07cdcc6f6ecd server_cookie=269eea697be469e9 in_seq=0 out_seq=0 2024-04-02T13:32:57.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1fd17fa640 10 client.14754 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1fd17fa640 1 -- 172.21.15.122:0/1332781929 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5603d6a408f0 con 0x5603d6b568b0 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1fd17fa640 10 client.14754 handle_command_reply: tid=0 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 client.14754 shutdown 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 msgr2=0x5603d6ae2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0098d70 tx=0x7f1fe000a4b0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 shutdown_connections 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603d6b568b0 0x5603d6b555c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603d6ada640 0x5603d6ae5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603d6ad54d0 0x5603d6ae2c70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 --2- 172.21.15.122:0/1332781929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603d6ad4ae0 0x5603d6ae2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1fd17fa640 0 client.14754 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 >> 172.21.15.122:0/1332781929 conn(0x5603d6ad0050 msgr2=0x5603d6ad0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 shutdown_connections 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 1 -- 172.21.15.122:0/1332781929 wait complete. 2024-04-02T13:32:57.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.319+0000 7f1ff6a3b740 20 client.14754 trim_cache size 0 max 16384 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24512, 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 6.7957883160000003, 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 16, 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 90, 2024-04-02T13:32:57.330 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 89, 2024-04-02T13:32:57.331 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 10.946363387 2024-04-02T13:32:57.331 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:32:57.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 msgr2=0x7f1fd40a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 0x7f1fd40a6120 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4005ab0 tx=0x7f1fe402e040 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 msgr2=0x7f1fec19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000c970 tx=0x7f1fe000ce30 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 shutdown_connections 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1fd40a3c60 0x7f1fd40a6120 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fec1025c0 0x7f1fec1a1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fec101c00 0x7f1fec19ef30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 --2- 172.21.15.122:0/1207525223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fec100a10 0x7f1fec06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 >> 172.21.15.122:0/1207525223 conn(0x7f1fec0fbfa0 msgr2=0x7f1fec104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 shutdown_connections 2024-04-02T13:32:57.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:57.320+0000 7f1ff3e55640 1 -- 172.21.15.122:0/1207525223 wait complete. 2024-04-02T13:32:57.333 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:32:57.333 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:32:57.334 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:32:57.570 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:32:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:57 smithi122 ceph-mon[32114]: pgmap v291: 129 pgs: 101 active+clean, 4 creating+peering, 24 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-02T13:32:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:57 smithi122 ceph-mon[32114]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:32:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:57 smithi122 ceph-mon[32114]: Cluster is now healthy 2024-04-02T13:32:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:57 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1279784620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:57 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3743195781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:57 smithi187 ceph-mon[36064]: pgmap v291: 129 pgs: 101 active+clean, 4 creating+peering, 24 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-02T13:32:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:57 smithi187 ceph-mon[36064]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:32:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:57 smithi187 ceph-mon[36064]: Cluster is now healthy 2024-04-02T13:32:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:57 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1279784620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:57 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3743195781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:57 smithi188 ceph-mon[36117]: pgmap v291: 129 pgs: 101 active+clean, 4 creating+peering, 24 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2024-04-02T13:32:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:57 smithi188 ceph-mon[36117]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-02T13:32:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:57 smithi188 ceph-mon[36117]: Cluster is now healthy 2024-04-02T13:32:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:57 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1279784620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:32:58.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:57 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3743195781' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 -- 172.21.15.122:0/1457761435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0107500 msgr2=0x7f8cd01078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 --2- 172.21.15.122:0/1457761435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0107500 0x7f8cd01078e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009990 tx=0x7f8cc4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 -- 172.21.15.122:0/1457761435 shutdown_connections 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 --2- 172.21.15.122:0/1457761435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0101eb0 0x7f8cd010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 --2- 172.21.15.122:0/1457761435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0107500 0x7f8cd01078e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 --2- 172.21.15.122:0/1457761435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 -- 172.21.15.122:0/1457761435 >> 172.21.15.122:0/1457761435 conn(0x7f8cd0078420 msgr2=0x7f8cd0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 -- 172.21.15.122:0/1457761435 shutdown_connections 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.137+0000 7f8cd568c700 1 -- 172.21.15.122:0/1457761435 wait complete. 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 Processor -- start 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 -- start start 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0107500 0x7f8cd019d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01117f0 con 0x7f8cd0101eb0 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0111670 con 0x7f8cd01014f0 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cd568c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0111970 con 0x7f8cd0107500 2024-04-02T13:32:59.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.138+0000 7f8cce7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43040/0 (socket says 172.21.15.122:43040) 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 -- 172.21.15.122:0/998602329 learned_addr learned my addr 172.21.15.122:0/998602329 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cceffd700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8ccf7fe700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0107500 0x7f8cd019d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 -- 172.21.15.122:0/998602329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0107500 msgr2=0x7f8cd019d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0107500 0x7f8cd019d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 -- 172.21.15.122:0/998602329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 msgr2=0x7f8cd0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 -- 172.21.15.122:0/998602329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd019dd60 con 0x7f8cd0101eb0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cceffd700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cce7fc700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000d950 tx=0x7f8cc000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0016070 con 0x7f8cd0101eb0 2024-04-02T13:32:59.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd019e050 con 0x7f8cd0101eb0 2024-04-02T13:32:59.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc0009b40 con 0x7f8cd0101eb0 2024-04-02T13:32:59.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000f620 con 0x7f8cd0101eb0 2024-04-02T13:32:59.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.139+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd01a58d0 con 0x7f8cd0101eb0 2024-04-02T13:32:59.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.140+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd00692b0 con 0x7f8cd0101eb0 2024-04-02T13:32:59.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.141+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8cc0019070 con 0x7f8cd0101eb0 2024-04-02T13:32:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.142+0000 7f8cb7fff700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 0x7f8cb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:32:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.142+0000 7f8cceffd700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 0x7f8cb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:32:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.142+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8cc00d0640 con 0x7f8cd0101eb0 2024-04-02T13:32:59.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.142+0000 7f8cceffd700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 0x7f8cb80a6100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006010 tx=0x7f8cc400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:32:59.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.146+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8cc009cde0 con 0x7f8cd0101eb0 2024-04-02T13:32:59.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.393+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8cd01a5b00 con 0x7f8cd0101eb0 2024-04-02T13:32:59.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.393+0000 7f8cb7fff700 1 -- 172.21.15.122:0/998602329 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f8cc009c6e0 con 0x7f8cd0101eb0 2024-04-02T13:32:59.396 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 msgr2=0x7f8cb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 0x7f8cb80a6100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006010 tx=0x7f8cc400b540 comp rx=0 tx=0).stop 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 msgr2=0x7f8cd0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000d950 tx=0x7f8cc000dd10 comp rx=0 tx=0).stop 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 shutdown_connections 2024-04-02T13:32:59.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd0107500 0x7f8cd019d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd01014f0 0x7f8cd0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cb80a3c40 0x7f8cb80a6100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 --2- 172.21.15.122:0/998602329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0101eb0 0x7f8cd0199290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.395+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 >> 172.21.15.122:0/998602329 conn(0x7f8cd0078420 msgr2=0x7f8cd0100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.396+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 shutdown_connections 2024-04-02T13:32:59.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:32:59.396+0000 7f8cd568c700 1 -- 172.21.15.122:0/998602329 wait complete. 2024-04-02T13:32:59.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:59 smithi122 ceph-mon[32114]: pgmap v292: 129 pgs: 118 active+clean, 4 creating+peering, 7 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-04-02T13:32:59.791 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:32:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/998602329' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:32:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:59 smithi187 ceph-mon[36064]: pgmap v292: 129 pgs: 118 active+clean, 4 creating+peering, 7 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-04-02T13:32:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:32:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/998602329' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:33:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:59 smithi188 ceph-mon[36117]: pgmap v292: 129 pgs: 118 active+clean, 4 creating+peering, 7 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 2 op/s 2024-04-02T13:33:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:32:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/998602329' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-02T13:33:00.057 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:33:00.058 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:33:00.058 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:33:00.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:00.058 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-04-02T13:33:00.074 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:33:00.074 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:33:00.074 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:33:00.074 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:33:00.074 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42255sec preferred_lft 42255sec 2024-04-02T13:33:00.075 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-04-02T13:33:00.076 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:33:00.076 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:33:00.076 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:33:00.076 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:33:00.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:33:00.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-04-02T13:33:00.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-04-02T13:33:00.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:33:00.077 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:33:00.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:33:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:00.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:00.268 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:33:00.294 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-04-02T13:33:00.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:00.296 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-04-02T13:33:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-04-02T13:33:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:33:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T13:33:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T13:33:00.367 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T13:33:00.368 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:33:00.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:33:00.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:00.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:00.669 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-02T13:33:00.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:00.684 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-04-02T13:33:00.739 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:33:00.739 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:33:00.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:33:00.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-02T13:33:00.739 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:33:00.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:33:00.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:00.862 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-02T13:33:00.862 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-02T13:33:00.863 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:33:00.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:33:00.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:00.999 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:33:01.000 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:33:01.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-04-02T13:33:01.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:33:01.000 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:33:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:33:01.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:01.081 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-02T13:33:01.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:33:01.082 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:01.096 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-02T13:33:01.097 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:33:01.097 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:01.151 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-04-02T13:33:01.219 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-04-02T13:33:01.275 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/selinux 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:33:01.276 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/c64155b78fb13b609d9c4f294b1fdb10983436719569a0bb06950e090939d78e/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/2fbe0fca3dec6b76d42faa79ed116252b1b2c5f4ed08eee0bd31ac3e07be3f54/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/0 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/7dbc86dbf097ece5a1b05987aea0abccddbd5f78ace454acd44130f7e2382830/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/25f08bcd060207c9c353f0c4d4079dc6ca7269484e5b8c8fb2a1cbfeefdbd5e3/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/fff47eb5376a3596c07dc1350efc42eb52969d8e958faf4370de9c569f66cf64/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/8c677ea45e6fbd7b98f4ece9c9571774e07778ade5d5442099777dfc369af454/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/daec06a726f98e9e18fe8c0ba40ddbc31f579bb97680c55b89690bc01e045786/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/260c8f53fc8e18fc897ebc3994086e8bd85b93d0cb421b067f42293e989edf4d/merged 2024-04-02T13:33:01.277 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/2ed3e4dd02e827002ab4bd1c40a80184de1d2a27e3c847e2ba0dd712848b9981/merged 2024-04-02T13:33:01.278 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/020ae7ee976dc72a393301fde814cdae59a8b8c46406f86a5b494709add047de/merged 2024-04-02T13:33:01.278 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-04-02T13:33:01.278 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:33:01.278 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:33:01.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:01.278 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-04-02T13:33:01.330 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-02T13:33:01.330 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-02T13:33:01.373 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-04-02T13:33:01.399 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/selinux 2024-04-02T13:33:01.455 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysctl.service 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/nfs/rpc_pipefs 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay 2024-04-02T13:33:01.456 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/c64155b78fb13b609d9c4f294b1fdb10983436719569a0bb06950e090939d78e/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/2fbe0fca3dec6b76d42faa79ed116252b1b2c5f4ed08eee0bd31ac3e07be3f54/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/0 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/7dbc86dbf097ece5a1b05987aea0abccddbd5f78ace454acd44130f7e2382830/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/25f08bcd060207c9c353f0c4d4079dc6ca7269484e5b8c8fb2a1cbfeefdbd5e3/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/fff47eb5376a3596c07dc1350efc42eb52969d8e958faf4370de9c569f66cf64/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/8c677ea45e6fbd7b98f4ece9c9571774e07778ade5d5442099777dfc369af454/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/daec06a726f98e9e18fe8c0ba40ddbc31f579bb97680c55b89690bc01e045786/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/260c8f53fc8e18fc897ebc3994086e8bd85b93d0cb421b067f42293e989edf4d/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/2ed3e4dd02e827002ab4bd1c40a80184de1d2a27e3c847e2ba0dd712848b9981/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/containers/storage/overlay/020ae7ee976dc72a393301fde814cdae59a8b8c46406f86a5b494709add047de/merged 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:33:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:33:01.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:01.458 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-04-02T13:33:01.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi122.stderr:2024-04-02T13:33:01.467+0000 7fda4623a480 -1 init, newargv = 0x55a5dacb9bf0 newargc=15 2024-04-02T13:33:01.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi122.stderr:ceph-fuse[73569]: starting ceph client 2024-04-02T13:33:01.475 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi122.stderr:ceph-fuse[73569]: starting fuse 2024-04-02T13:33:01.484 INFO:teuthology.orchestra.run.smithi122.stdout:119 2024-04-02T13:33:01.484 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo stdin-killer -- python3 -c ' 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> import glob 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> import re 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> import os 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> import subprocess 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> def _find_admin_socket(client_name): 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> files = glob.glob(asok_path) 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> # Given a non-glob path, it better be there 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> if "*" not in asok_path: 2024-04-02T13:33:01.485 DEBUG:teuthology.orchestra.run.smithi122:> assert(len(files) == 1) 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> return files[0] 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> for f in files: 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> contents = proc_f.read() 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> if mountpoint in contents: 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> return f 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> print(_find_admin_socket("client.admin")) 2024-04-02T13:33:01.486 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-04-02T13:33:01.606 INFO:teuthology.orchestra.run.smithi122.stdout:/var/run/ceph/ceph-client.admin.73569.asok 2024-04-02T13:33:01.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:01.616 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73569.asok 2024-04-02T13:33:01.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:01.616 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73569.asok status 2024-04-02T13:33:01.752 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": { 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_sha1": "191d963ac9de7322da8aff20b567b3fa458e2ef6", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-2609-g191d963a (191d963ac9de7322da8aff20b567b3fa458e2ef6) squid (dev)", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "entity_id": "admin", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": "smithi122", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "pid": "73569", 2024-04-02T13:33:01.753 INFO:teuthology.orchestra.run.smithi122.stdout: "root": "/" 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_count": 0, 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_pinned_count": 0, 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 14766, 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "inst": { 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "name": { 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "client", 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "num": 14766 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-04-02T13:33:01.754 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 4225539960 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 4225539960 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "inst_str": "client.14766 192.168.0.1:0/4225539960", 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "addr_str": "192.168.0.1:0/4225539960", 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "inode_count": 1, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_epoch": 16, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch": 90, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch_barrier": 0, 2024-04-02T13:33:01.755 INFO:teuthology.orchestra.run.smithi122.stdout: "blocklisted": false, 2024-04-02T13:33:01.756 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs" 2024-04-02T13:33:01.756 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:33:01.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:01.761 DEBUG:teuthology.orchestra.run.smithi122:> stat --file-system '--printf=%T 2024-04-02T13:33:01.761 DEBUG:teuthology.orchestra.run.smithi122:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:01.777 INFO:teuthology.orchestra.run.smithi122.stdout:fuseblk 2024-04-02T13:33:01.777 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:01.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:01.777 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:01 smithi122 ceph-mon[32114]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-04-02T13:33:01.845 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-02T13:33:01.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:33:01.955 INFO:teuthology.orchestra.run.smithi122.stdout:# file: . 2024-04-02T13:33:01.955 INFO:teuthology.orchestra.run.smithi122.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-02T13:33:01.955 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:01.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:33:01.961 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:33:01.961 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:33:01.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:33:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:01 smithi187 ceph-mon[36064]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-04-02T13:33:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:01 smithi188 ceph-mon[36117]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 1 op/s 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 -- 172.21.15.122:0/1223084472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4100a30 msgr2=0x7f36e4100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/1223084472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4100a30 0x7f36e4100e50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009900 tx=0x7f36d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 -- 172.21.15.122:0/1223084472 shutdown_connections 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/1223084472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e41025e0 0x7f36e4108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/1223084472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e4101c20 0x7f36e41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/1223084472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4100a30 0x7f36e4100e50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 -- 172.21.15.122:0/1223084472 >> 172.21.15.122:0/1223084472 conn(0x7f36e40fbfe0 msgr2=0x7f36e40fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 -- 172.21.15.122:0/1223084472 shutdown_connections 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.116+0000 7f36ea0e4640 1 -- 172.21.15.122:0/1223084472 wait complete. 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 Processor -- start 2024-04-02T13:33:02.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 -- start start 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 0x7f36e4197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e41025e0 0x7f36e419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c750 con 0x7f36e4101c20 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c5d0 con 0x7f36e4100a30 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36ea0e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36e410c8d0 con 0x7f36e41025e0 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43054/0 (socket says 172.21.15.122:43054) 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 -- 172.21.15.122:0/4254699560 learned_addr learned my addr 172.21.15.122:0/4254699560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e37fe640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 0x7f36e4197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 -- 172.21.15.122:0/4254699560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e41025e0 msgr2=0x7f36e419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e41025e0 0x7f36e419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 -- 172.21.15.122:0/4254699560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 msgr2=0x7f36e4197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 0x7f36e4197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 -- 172.21.15.122:0/4254699560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e419fae0 con 0x7f36e4101c20 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e37fe640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 0x7f36e4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.117+0000 7f36e2ffd640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b4a0 tx=0x7f36d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.118+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8014070 con 0x7f36e4101c20 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.118+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d8010af0 con 0x7f36e4101c20 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.118+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e419fdd0 con 0x7f36e4101c20 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.118+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8017d10 con 0x7f36e4101c20 2024-04-02T13:33:02.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.118+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41a0290 con 0x7f36e4101c20 2024-04-02T13:33:02.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.119+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e41a0520 con 0x7f36e4101c20 2024-04-02T13:33:02.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.120+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f36d8017070 con 0x7f36e4101c20 2024-04-02T13:33:02.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.120+0000 7f36e0ff9640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 0x7f36cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.120+0000 7f36e37fe640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 0x7f36cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.120+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f36d80cfa60 con 0x7f36e4101c20 2024-04-02T13:33:02.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.121+0000 7f36e37fe640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 0x7f36cc0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36d40096c0 tx=0x7f36d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.125+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f36d809c200 con 0x7f36e4101c20 2024-04-02T13:33:02.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.306+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36e41a06f0 con 0x7f36e4101c20 2024-04-02T13:33:02.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.307+0000 7f36e0ff9640 1 -- 172.21.15.122:0/4254699560 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f36d809bb00 con 0x7f36e4101c20 2024-04-02T13:33:02.310 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:02.310 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.309+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 msgr2=0x7f36cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.309+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 0x7f36cc0a6070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f36d40096c0 tx=0x7f36d4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.309+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 msgr2=0x7f36e4198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.309+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f36d800b4a0 tx=0x7f36d800b960 comp rx=0 tx=0).stop 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 shutdown_connections 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36e41025e0 0x7f36e419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36e4101c20 0x7f36e4198400 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36cc0a3bb0 0x7f36cc0a6070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 --2- 172.21.15.122:0/4254699560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36e4100a30 0x7f36e4197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 >> 172.21.15.122:0/4254699560 conn(0x7f36e40fbfe0 msgr2=0x7f36e406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 shutdown_connections 2024-04-02T13:33:02.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.310+0000 7f36ea0e4640 1 -- 172.21.15.122:0/4254699560 wait complete. 2024-04-02T13:33:02.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:33:02.324 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 -- 172.21.15.122:0/1847737616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc104eb0 msgr2=0x7feccc1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 --2- 172.21.15.122:0/1847737616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc104eb0 0x7feccc1072a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fecc0009900 tx=0x7fecc002bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 -- 172.21.15.122:0/1847737616 shutdown_connections 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 --2- 172.21.15.122:0/1847737616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 --2- 172.21.15.122:0/1847737616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc104eb0 0x7feccc1072a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 --2- 172.21.15.122:0/1847737616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc102580 0x7feccc104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 -- 172.21.15.122:0/1847737616 >> 172.21.15.122:0/1847737616 conn(0x7feccc0fbf90 msgr2=0x7feccc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 -- 172.21.15.122:0/1847737616 shutdown_connections 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.426+0000 7fecd2396640 1 -- 172.21.15.122:0/1847737616 wait complete. 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 Processor -- start 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 -- start start 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc104eb0 0x7feccc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feccc10d780 con 0x7feccc102580 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feccc10d600 con 0x7feccc104eb0 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7fecd2396640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feccc10d900 con 0x7feccc1077e0 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7feccbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7feccbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43078/0 (socket says 172.21.15.122:43078) 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7feccbfff640 1 -- 172.21.15.122:0/3167605770 learned_addr learned my addr 172.21.15.122:0/3167605770 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:02.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.427+0000 7feccb7fe640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc104eb0 0x7feccc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecd0933640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 -- 172.21.15.122:0/3167605770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 msgr2=0x7feccc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc19f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 -- 172.21.15.122:0/3167605770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc104eb0 msgr2=0x7feccc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc104eb0 0x7feccc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 -- 172.21.15.122:0/3167605770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc19faf0 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecd0933640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7feccbfff640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00d950 tx=0x7fecbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc017030 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecbc004020 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc0047a0 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc19fde0 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.428+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc1a01f0 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.429+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc0fda80 con 0x7feccc102580 2024-04-02T13:33:02.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.430+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fecbc0041d0 con 0x7feccc102580 2024-04-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.431+0000 7fecc97fa640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 0x7fecb40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.431+0000 7feccb7fe640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 0x7fecb40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.431+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fecbc0d00e0 con 0x7feccc102580 2024-04-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.431+0000 7feccb7fe640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 0x7fecb40a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecc002c7b0 tx=0x7fecc0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.435+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fecbc09c880 con 0x7feccc102580 2024-04-02T13:33:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.619+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feccc061f40 con 0x7feccc102580 2024-04-02T13:33:02.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.620+0000 7fecc97fa640 1 -- 172.21.15.122:0/3167605770 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fecbc09c180 con 0x7feccc102580 2024-04-02T13:33:02.623 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:33:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 msgr2=0x7fecb40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 0x7fecb40a60f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fecc002c7b0 tx=0x7fecc0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 msgr2=0x7feccc197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00d950 tx=0x7fecbc00de10 comp rx=0 tx=0).stop 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 shutdown_connections 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc1077e0 0x7feccc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fecb40a3c30 0x7fecb40a60f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feccc104eb0 0x7feccc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 --2- 172.21.15.122:0/3167605770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feccc102580 0x7feccc197f20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 >> 172.21.15.122:0/3167605770 conn(0x7feccc0fbf90 msgr2=0x7feccc1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 shutdown_connections 2024-04-02T13:33:02.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.622+0000 7fecd2396640 1 -- 172.21.15.122:0/3167605770 wait complete. 2024-04-02T13:33:02.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:02 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4254699560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:02.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.738+0000 7f3abf263640 1 -- 172.21.15.122:0/301858837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8101b10 msgr2=0x7f3ab8101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.738+0000 7f3abf263640 1 --2- 172.21.15.122:0/301858837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8101b10 0x7f3ab8101f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009900 tx=0x7f3aac02bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- 172.21.15.122:0/301858837 shutdown_connections 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- 172.21.15.122:0/301858837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab81036c0 0x7f3ab8109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- 172.21.15.122:0/301858837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab8102d00 0x7f3ab8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- 172.21.15.122:0/301858837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8101b10 0x7f3ab8101f30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- 172.21.15.122:0/301858837 >> 172.21.15.122:0/301858837 conn(0x7f3ab80fd080 msgr2=0x7f3ab80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- 172.21.15.122:0/301858837 shutdown_connections 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- 172.21.15.122:0/301858837 wait complete. 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 Processor -- start 2024-04-02T13:33:02.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- start start 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 0x7f3ab8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab81036c0 0x7f3ab819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d790 con 0x7f3ab8102d00 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d610 con 0x7f3ab81036c0 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.739+0000 7f3abf263640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab810d910 con 0x7f3ab8101b10 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43104/0 (socket says 172.21.15.122:43104) 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 -- 172.21.15.122:0/2122476467 learned_addr learned my addr 172.21.15.122:0/2122476467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3abcfff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 0x7f3ab8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3abd800640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab81036c0 0x7f3ab819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 -- 172.21.15.122:0/2122476467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 msgr2=0x7f3ab8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 0x7f3ab8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 -- 172.21.15.122:0/2122476467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab81036c0 msgr2=0x7f3ab819f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab81036c0 0x7f3ab819f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 -- 172.21.15.122:0/2122476467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab819fa80 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3abcfff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 0x7f3ab8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab7fff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400d950 tx=0x7f3aa400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4017030 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4004020 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40047a0 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab819fd70 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.740+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab81a0260 con 0x7f3ab8102d00 2024-04-02T13:33:02.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.741+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab8108e80 con 0x7f3ab8102d00 2024-04-02T13:33:02.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.742+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f3aa40041d0 con 0x7f3ab8102d00 2024-04-02T13:33:02.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.743+0000 7f3ab5ffb640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 0x7f3aa80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:02.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.743+0000 7f3abcfff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 0x7f3aa80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:02.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.743+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3aa40d0160 con 0x7f3ab8102d00 2024-04-02T13:33:02.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.743+0000 7f3abcfff640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 0x7f3aa80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02c7b0 tx=0x7f3aac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:02.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.747+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3aa409c900 con 0x7f3ab8102d00 2024-04-02T13:33:02.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.925+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3ab8062700 con 0x7f3ab8102d00 2024-04-02T13:33:02.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.926+0000 7f3ab5ffb640 1 -- 172.21.15.122:0/2122476467 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f3aa409c200 con 0x7f3ab8102d00 2024-04-02T13:33:02.929 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 msgr2=0x7f3aa80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 0x7f3aa80a6070 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3aac02c7b0 tx=0x7f3aac005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 msgr2=0x7f3ab81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400d950 tx=0x7f3aa400de10 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 shutdown_connections 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3aa80a3bb0 0x7f3aa80a6070 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab81036c0 0x7f3ab819f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab8102d00 0x7f3ab81983a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 --2- 172.21.15.122:0/2122476467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab8101b10 0x7f3ab8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 >> 172.21.15.122:0/2122476467 conn(0x7f3ab80fd080 msgr2=0x7f3ab8106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 shutdown_connections 2024-04-02T13:33:02.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:02.928+0000 7f3abf263640 1 -- 172.21.15.122:0/2122476467 wait complete. 2024-04-02T13:33:02.942 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:33:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:02 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4254699560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:02 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4254699560' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- 172.21.15.122:0/1113747445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4320101b10 msgr2=0x7f4320101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- 172.21.15.122:0/1113747445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4320101b10 0x7f4320101f30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4318009900 tx=0x7f431802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- 172.21.15.122:0/1113747445 shutdown_connections 2024-04-02T13:33:03.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- 172.21.15.122:0/1113747445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43201036c0 0x7f4320109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- 172.21.15.122:0/1113747445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320102d00 0x7f4320103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- 172.21.15.122:0/1113747445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4320101b10 0x7f4320101f30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- 172.21.15.122:0/1113747445 >> 172.21.15.122:0/1113747445 conn(0x7f43200fd0a0 msgr2=0x7f43200ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- 172.21.15.122:0/1113747445 shutdown_connections 2024-04-02T13:33:03.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- 172.21.15.122:0/1113747445 wait complete. 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 Processor -- start 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- start start 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 0x7f4320197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4320102d00 0x7f43201983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d780 con 0x7f43201036c0 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d600 con 0x7f4320102d00 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.043+0000 7f43279e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f432010d900 con 0x7f4320101b10 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43116/0 (socket says 172.21.15.122:43116) 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 -- 172.21.15.122:0/3627186258 learned_addr learned my addr 172.21.15.122:0/3627186258 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4324f84640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4320102d00 0x7f43201983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325785640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 0x7f4320197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 -- 172.21.15.122:0/3627186258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 msgr2=0x7f4320197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 0x7f4320197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 -- 172.21.15.122:0/3627186258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4320102d00 msgr2=0x7f43201983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4320102d00 0x7f43201983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 -- 172.21.15.122:0/3627186258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432019fac0 con 0x7f43201036c0 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325785640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 0x7f4320197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:03.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f4325f86640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f431c009fd0 tx=0x7f431c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f43201036c0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f431c003e50 con 0x7f43201036c0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c0106c0 con 0x7f43201036c0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432019fdb0 con 0x7f43201036c0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.044+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43201a0270 con 0x7f43201036c0 2024-04-02T13:33:03.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.045+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4320108f50 con 0x7f43201036c0 2024-04-02T13:33:03.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.046+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f431c003970 con 0x7f43201036c0 2024-04-02T13:33:03.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.047+0000 7f43167fc640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 0x7f430c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.047+0000 7f4325785640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 0x7f430c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.047+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f431c0d0360 con 0x7f43201036c0 2024-04-02T13:33:03.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.047+0000 7f4325785640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 0x7f430c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f431802c7b0 tx=0x7f4318005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.051+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f431c09cb00 con 0x7f43201036c0 2024-04-02T13:33:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.228+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43200627a0 con 0x7f43201036c0 2024-04-02T13:33:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.229+0000 7f43167fc640 1 -- 172.21.15.122:0/3627186258 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f431c09c400 con 0x7f43201036c0 2024-04-02T13:33:03.232 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 msgr2=0x7f430c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 0x7f430c0a60f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f431802c7b0 tx=0x7f4318005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 msgr2=0x7f432019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f431c009fd0 tx=0x7f431c00b260 comp rx=0 tx=0).stop 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 shutdown_connections 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43201036c0 0x7f432019f450 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f430c0a3c30 0x7f430c0a60f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4320102d00 0x7f43201983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 --2- 172.21.15.122:0/3627186258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4320101b10 0x7f4320197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 >> 172.21.15.122:0/3627186258 conn(0x7f43200fd0a0 msgr2=0x7f4320106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 shutdown_connections 2024-04-02T13:33:03.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.231+0000 7f43279e9640 1 -- 172.21.15.122:0/3627186258 wait complete. 2024-04-02T13:33:03.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 -- 172.21.15.122:0/792933406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 msgr2=0x7fc9e0109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 --2- 172.21.15.122:0/792933406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e0109f70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009900 tx=0x7fc9d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 -- 172.21.15.122:0/792933406 shutdown_connections 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 --2- 172.21.15.122:0/792933406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e0109f70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 --2- 172.21.15.122:0/792933406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 0x7fc9e01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 --2- 172.21.15.122:0/792933406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 0x7fc9e0101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 -- 172.21.15.122:0/792933406 >> 172.21.15.122:0/792933406 conn(0x7fc9e00fd0a0 msgr2=0x7fc9e00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 -- 172.21.15.122:0/792933406 shutdown_connections 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.346+0000 7fc9e5055640 1 -- 172.21.15.122:0/792933406 wait complete. 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 Processor -- start 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 -- start start 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 0x7fc9e0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 0x7fc9e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010d7b0 con 0x7fc9e01036e0 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010d630 con 0x7fc9e0102d20 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9e5055640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e010d930 con 0x7fc9e0101b30 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9df59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9df59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43138/0 (socket says 172.21.15.122:43138) 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.347+0000 7fc9df59e640 1 -- 172.21.15.122:0/3948727633 learned_addr learned my addr 172.21.15.122:0/3948727633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9ded9d640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 0x7fc9e0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9de59c640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 0x7fc9e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 -- 172.21.15.122:0/3948727633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 msgr2=0x7fc9e0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 0x7fc9e0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 -- 172.21.15.122:0/3948727633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 msgr2=0x7fc9e0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 0x7fc9e0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 -- 172.21.15.122:0/3948727633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e019fa10 con 0x7fc9e01036e0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9df59e640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802ce70 tx=0x7fc9d8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8041070 con 0x7fc9e01036e0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e019fca0 con 0x7fc9e01036e0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d80054e0 con 0x7fc9e01036e0 2024-04-02T13:33:03.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d80344a0 con 0x7fc9e01036e0 2024-04-02T13:33:03.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.348+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a0190 con 0x7fc9e01036e0 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.349+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0108df0 con 0x7fc9e01036e0 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.350+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fc9d8044070 con 0x7fc9e01036e0 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.351+0000 7fc9d7fff640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 0x7fc9cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.351+0000 7fc9ded9d640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 0x7fc9cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.351+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc9d80f2650 con 0x7fc9e01036e0 2024-04-02T13:33:03.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.351+0000 7fc9ded9d640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 0x7fc9cc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0102b80 tx=0x7fc9d0009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.354+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc9d80be7a0 con 0x7fc9e01036e0 2024-04-02T13:33:03.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.534+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc9e002d370 con 0x7fc9e01036e0 2024-04-02T13:33:03.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.534+0000 7fc9d7fff640 1 -- 172.21.15.122:0/3948727633 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7fc9d80c2aa0 con 0x7fc9e01036e0 2024-04-02T13:33:03.537 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 msgr2=0x7fc9cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 0x7fc9cc0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0102b80 tx=0x7fc9d0009680 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 msgr2=0x7fc9e019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9d802ce70 tx=0x7fc9d8005e40 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 shutdown_connections 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9e01036e0 0x7fc9e019f3a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9cc0a3bb0 0x7fc9cc0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc9e0102d20 0x7fc9e0198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 --2- 172.21.15.122:0/3948727633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e0101b30 0x7fc9e0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 >> 172.21.15.122:0/3948727633 conn(0x7fc9e00fd0a0 msgr2=0x7fc9e0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 shutdown_connections 2024-04-02T13:33:03.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.536+0000 7fc9e5055640 1 -- 172.21.15.122:0/3948727633 wait complete. 2024-04-02T13:33:03.550 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:33:03.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- 172.21.15.122:0/2974109963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 msgr2=0x7f7484109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 --2- 172.21.15.122:0/2974109963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f7484109f50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f747c009ca0 tx=0x7f747c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- 172.21.15.122:0/2974109963 shutdown_connections 2024-04-02T13:33:03.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 --2- 172.21.15.122:0/2974109963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f7484109f50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 --2- 172.21.15.122:0/2974109963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484102d00 0x7f7484103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 --2- 172.21.15.122:0/2974109963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484101b10 0x7f7484101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- 172.21.15.122:0/2974109963 >> 172.21.15.122:0/2974109963 conn(0x7f74840fd0a0 msgr2=0x7f74840ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- 172.21.15.122:0/2974109963 shutdown_connections 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- 172.21.15.122:0/2974109963 wait complete. 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 Processor -- start 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.650+0000 7f748af4a640 1 -- start start 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484101b10 0x7f7484197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 0x7f7484198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f748410d790 con 0x7f74841036c0 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f748410d610 con 0x7f7484101b10 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f748410d910 con 0x7f7484102d00 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43152/0 (socket says 172.21.15.122:43152) 2024-04-02T13:33:03.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 -- 172.21.15.122:0/2748822308 learned_addr learned my addr 172.21.15.122:0/2748822308 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7483fff640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 0x7f7484198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 -- 172.21.15.122:0/2748822308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 msgr2=0x7f7484198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 0x7f7484198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7488ce6640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484101b10 0x7f7484197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 -- 172.21.15.122:0/2748822308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484101b10 msgr2=0x7f7484197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484101b10 0x7f7484197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 -- 172.21.15.122:0/2748822308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748419f970 con 0x7f74841036c0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7483fff640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 0x7f7484198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f74894e7640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c700 tx=0x7f747c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c039040 con 0x7f74841036c0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f747c044070 con 0x7f74841036c0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748419fc00 con 0x7f74841036c0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.651+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c0042a0 con 0x7f74841036c0 2024-04-02T13:33:03.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.652+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74841a00f0 con 0x7f74841036c0 2024-04-02T13:33:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.653+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7484108dd0 con 0x7f74841036c0 2024-04-02T13:33:03.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.654+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f747c003970 con 0x7f74841036c0 2024-04-02T13:33:03.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.654+0000 7f7481ffb640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 0x7f746c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:03.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.654+0000 7f7488ce6640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 0x7f746c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:03.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.654+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f747c0f2bb0 con 0x7f74841036c0 2024-04-02T13:33:03.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.655+0000 7f7488ce6640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 0x7f746c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7474005fd0 tx=0x7f7474005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:03.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.658+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f747c0333a0 con 0x7f74841036c0 2024-04-02T13:33:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:03 smithi122 ceph-mon[32114]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:33:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3167605770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2122476467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3627186258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3948727633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.849+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f748402d770 con 0x7f74841036c0 2024-04-02T13:33:03.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.850+0000 7f7481ffb640 1 -- 172.21.15.122:0/2748822308 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+600 (secure 0 0 0) 0x7f747c0bf3e0 con 0x7f74841036c0 2024-04-02T13:33:03.852 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3421882822 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:03.852 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/684561021 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/776403907 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/4063470519 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1474593903 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1325797276 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:03.853 INFO:teuthology.orchestra.run.smithi122.stderr:listed 10 entries 2024-04-02T13:33:03.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 msgr2=0x7f746c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 0x7f746c0a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7474005fd0 tx=0x7f7474005850 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 msgr2=0x7f748419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f747c02c700 tx=0x7f747c032f20 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 shutdown_connections 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74841036c0 0x7f748419f300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7484102d00 0x7f7484198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f746c0a3bb0 0x7f746c0a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 --2- 172.21.15.122:0/2748822308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7484101b10 0x7f7484197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 >> 172.21.15.122:0/2748822308 conn(0x7f74840fd0a0 msgr2=0x7f7484105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 shutdown_connections 2024-04-02T13:33:03.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:03.852+0000 7f748af4a640 1 -- 172.21.15.122:0/2748822308 wait complete. 2024-04-02T13:33:03.865 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:33:03.865 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:33:03.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:33:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:03 smithi187 ceph-mon[36064]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:33:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3167605770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2122476467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3627186258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3948727633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:03 smithi188 ceph-mon[36117]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:33:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3167605770' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2122476467' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3627186258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3948727633' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/790368646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e58100a20 msgr2=0x7f8e58100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/790368646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e58100a20 0x7f8e58100e40 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009900 tx=0x7f8e4802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:04.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/790368646 shutdown_connections 2024-04-02T13:33:04.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/790368646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e581025d0 0x7f8e58108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/790368646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58101c10 0x7f8e58102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/790368646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e58100a20 0x7f8e58100e40 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/790368646 >> 172.21.15.122:0/790368646 conn(0x7f8e580fbf90 msgr2=0x7f8e580fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/790368646 shutdown_connections 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.020+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/790368646 wait complete. 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 Processor -- start 2024-04-02T13:33:04.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 -- start start 2024-04-02T13:33:04.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58100a20 0x7f8e58197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e58101c10 0x7f8e58198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810c6a0 con 0x7f8e581025d0 2024-04-02T13:33:04.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810c520 con 0x7f8e58100a20 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5e9ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5810c820 con 0x7f8e58101c10 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5cf57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5cf57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43168/0 (socket says 172.21.15.122:43168) 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e5cf57640 1 -- 172.21.15.122:0/1305402506 learned_addr learned my addr 172.21.15.122:0/1305402506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.021+0000 7f8e577fe640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e58101c10 0x7f8e58198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e57fff640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58100a20 0x7f8e58197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 -- 172.21.15.122:0/1305402506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e58101c10 msgr2=0x7f8e58198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e58101c10 0x7f8e58198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 -- 172.21.15.122:0/1305402506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58100a20 msgr2=0x7f8e58197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58100a20 0x7f8e58197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 -- 172.21.15.122:0/1305402506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5819fa60 con 0x7f8e581025d0 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5cf57640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eb10 tx=0x7f8e5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50004b20 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e50004cc0 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e50010760 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5819fd50 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.022+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a0240 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.023+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e581a04d0 con 0x7f8e581025d0 2024-04-02T13:33:04.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.024+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f8e50005020 con 0x7f8e581025d0 2024-04-02T13:33:04.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.024+0000 7f8e557fa640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.025+0000 7f8e57fff640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.025+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8e500d0d90 con 0x7f8e581025d0 2024-04-02T13:33:04.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.025+0000 7f8e57fff640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802c7b0 tx=0x7f8e48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.029+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e5009d530 con 0x7f8e581025d0 2024-04-02T13:33:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.207+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8e581a0830 con 0x7f8e581025d0 2024-04-02T13:33:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.208+0000 7f8e557fa640 1 -- 172.21.15.122:0/1305402506 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f8e5009ce30 con 0x7f8e581025d0 2024-04-02T13:33:04.211 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:04.211 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 msgr2=0x7f8e400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 0x7f8e400a6070 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8e4802c7b0 tx=0x7f8e48005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 msgr2=0x7f8e5819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000eb10 tx=0x7f8e5000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 shutdown_connections 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e581025d0 0x7f8e5819f3f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e58101c10 0x7f8e58198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8e400a3bb0 0x7f8e400a6070 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 --2- 172.21.15.122:0/1305402506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e58100a20 0x7f8e58197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 >> 172.21.15.122:0/1305402506 conn(0x7f8e580fbf90 msgr2=0x7f8e58068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 shutdown_connections 2024-04-02T13:33:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.210+0000 7f8e5e9ba640 1 -- 172.21.15.122:0/1305402506 wait complete. 2024-04-02T13:33:04.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:33:04.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:33:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.326+0000 7ffa6b98d640 1 -- 172.21.15.122:0/2265167591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 msgr2=0x7ffa64103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.326+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/2265167591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64103190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009900 tx=0x7ffa5402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- 172.21.15.122:0/2265167591 shutdown_connections 2024-04-02T13:33:04.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/2265167591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa641036d0 0x7ffa64109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/2265167591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64103190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/2265167591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa64101b20 0x7ffa64101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- 172.21.15.122:0/2265167591 >> 172.21.15.122:0/2265167591 conn(0x7ffa640fd090 msgr2=0x7ffa640ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- 172.21.15.122:0/2265167591 shutdown_connections 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- 172.21.15.122:0/2265167591 wait complete. 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 Processor -- start 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- start start 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa64101b20 0x7ffa64197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa641036d0 0x7ffa6419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410d780 con 0x7ffa64102d10 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410d600 con 0x7ffa64101b20 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa6b98d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6410d900 con 0x7ffa641036d0 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa68f28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa68f28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43186/0 (socket says 172.21.15.122:43186) 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.327+0000 7ffa68f28640 1 -- 172.21.15.122:0/3298631537 learned_addr learned my addr 172.21.15.122:0/3298631537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa69f2a640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa641036d0 0x7ffa6419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa69729640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa64101b20 0x7ffa64197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 -- 172.21.15.122:0/3298631537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa641036d0 msgr2=0x7ffa6419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa641036d0 0x7ffa6419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 -- 172.21.15.122:0/3298631537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa64101b20 msgr2=0x7ffa64197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa64101b20 0x7ffa64197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 -- 172.21.15.122:0/3298631537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6419fa40 con 0x7ffa64102d10 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa68f28640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009230 tx=0x7ffa54005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54041070 con 0x7ffa64102d10 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa540054e0 con 0x7ffa64102d10 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6419fcd0 con 0x7ffa64102d10 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54034440 con 0x7ffa64102d10 2024-04-02T13:33:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.328+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa641a01c0 con 0x7ffa64102d10 2024-04-02T13:33:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.329+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa64108f10 con 0x7ffa64102d10 2024-04-02T13:33:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.330+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ffa54044070 con 0x7ffa64102d10 2024-04-02T13:33:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.330+0000 7ffa5a7fc640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 0x7ffa500a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.331+0000 7ffa69729640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 0x7ffa500a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.331+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ffa540f2220 con 0x7ffa64102d10 2024-04-02T13:33:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.331+0000 7ffa69729640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 0x7ffa500a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa64102b70 tx=0x7ffa5c00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.335+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ffa540be370 con 0x7ffa64102d10 2024-04-02T13:33:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.511+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa64062700 con 0x7ffa64102d10 2024-04-02T13:33:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.512+0000 7ffa5a7fc640 1 -- 172.21.15.122:0/3298631537 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7ffa540c2670 con 0x7ffa64102d10 2024-04-02T13:33:04.515 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:33:04.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 msgr2=0x7ffa500a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 0x7ffa500a60f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ffa64102b70 tx=0x7ffa5c00a2d0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 msgr2=0x7ffa64198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009230 tx=0x7ffa54005dd0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 shutdown_connections 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa641036d0 0x7ffa6419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa64102d10 0x7ffa64198360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ffa500a3c30 0x7ffa500a60f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 --2- 172.21.15.122:0/3298631537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffa64101b20 0x7ffa64197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 >> 172.21.15.122:0/3298631537 conn(0x7ffa640fd090 msgr2=0x7ffa641060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 shutdown_connections 2024-04-02T13:33:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.514+0000 7ffa6b98d640 1 -- 172.21.15.122:0/3298631537 wait complete. 2024-04-02T13:33:04.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 -- 172.21.15.122:0/1626496035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18101c10 msgr2=0x7f0a18102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/1626496035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18101c10 0x7f0a18102090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009900 tx=0x7f0a0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 -- 172.21.15.122:0/1626496035 shutdown_connections 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/1626496035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a18108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/1626496035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18101c10 0x7f0a18102090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/1626496035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18100a20 0x7f0a18100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 -- 172.21.15.122:0/1626496035 >> 172.21.15.122:0/1626496035 conn(0x7f0a180fbf90 msgr2=0x7f0a180fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 -- 172.21.15.122:0/1626496035 shutdown_connections 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.630+0000 7f0a1dd47640 1 -- 172.21.15.122:0/1626496035 wait complete. 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 Processor -- start 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 -- start start 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18101c10 0x7f0a18197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a1819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810c6a0 con 0x7f0a18100a20 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810c520 con 0x7f0a18101c10 2024-04-02T13:33:04.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a1dd47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810c820 con 0x7f0a181025d0 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43200/0 (socket says 172.21.15.122:43200) 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 -- 172.21.15.122:0/3859226613 learned_addr learned my addr 172.21.15.122:0/3859226613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a16ffd640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18101c10 0x7f0a18197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a17fff640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a1819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 -- 172.21.15.122:0/3859226613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 msgr2=0x7f0a1819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a1819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 -- 172.21.15.122:0/3859226613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18101c10 msgr2=0x7f0a18197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18101c10 0x7f0a18197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a177fe640 1 -- 172.21.15.122:0/3859226613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1819e9f0 con 0x7f0a18100a20 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.631+0000 7f0a17fff640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a1819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:04.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a177fe640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d770 tx=0x7f0a0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08017070 con 0x7f0a18100a20 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a08009c30 con 0x7f0a18100a20 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1819ece0 con 0x7f0a18100a20 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08003dd0 con 0x7f0a18100a20 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.632+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1819f1b0 con 0x7f0a18100a20 2024-04-02T13:33:04.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.633+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a18107e60 con 0x7f0a18100a20 2024-04-02T13:33:04.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.634+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f0a08003f70 con 0x7f0a18100a20 2024-04-02T13:33:04.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.634+0000 7f0a14ff9640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 0x7f0a000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.635+0000 7f0a16ffd640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 0x7f0a000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.635+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0a080cffb0 con 0x7f0a18100a20 2024-04-02T13:33:04.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.635+0000 7f0a16ffd640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 0x7f0a000a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009230 tx=0x7f0a0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.639+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0a0809c750 con 0x7f0a18100a20 2024-04-02T13:33:04.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.815+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a180627a0 con 0x7f0a18100a20 2024-04-02T13:33:04.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.816+0000 7f0a14ff9640 1 -- 172.21.15.122:0/3859226613 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f0a08009de0 con 0x7f0a18100a20 2024-04-02T13:33:04.819 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:33:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 msgr2=0x7f0a000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 0x7f0a000a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009230 tx=0x7f0a0c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 msgr2=0x7f0a18196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d770 tx=0x7f0a0800dc30 comp rx=0 tx=0).stop 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 shutdown_connections 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a181025d0 0x7f0a1819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0a000a3bb0 0x7f0a000a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0a18101c10 0x7f0a18197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 --2- 172.21.15.122:0/3859226613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0a18100a20 0x7f0a18196dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2748822308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:04.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1305402506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:04.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3298631537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 >> 172.21.15.122:0/3859226613 conn(0x7f0a180fbf90 msgr2=0x7f0a18105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 shutdown_connections 2024-04-02T13:33:04.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.818+0000 7f0a1dd47640 1 -- 172.21.15.122:0/3859226613 wait complete. 2024-04-02T13:33:04.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- 172.21.15.122:0/1799898158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03041019e0 msgr2=0x7f0304101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 --2- 172.21.15.122:0/1799898158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03041019e0 0x7f0304101e00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02f4009900 tx=0x7f02f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- 172.21.15.122:0/1799898158 shutdown_connections 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 --2- 172.21.15.122:0/1799898158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304103590 0x7f0304109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 --2- 172.21.15.122:0/1799898158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0304102bd0 0x7f0304103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 --2- 172.21.15.122:0/1799898158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03041019e0 0x7f0304101e00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- 172.21.15.122:0/1799898158 >> 172.21.15.122:0/1799898158 conn(0x7f03040fcf70 msgr2=0x7f03040ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- 172.21.15.122:0/1799898158 shutdown_connections 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- 172.21.15.122:0/1799898158 wait complete. 2024-04-02T13:33:04.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 Processor -- start 2024-04-02T13:33:04.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.933+0000 7f030939d640 1 -- start start 2024-04-02T13:33:04.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03041019e0 0x7f0304197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 0x7f03041981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d630 con 0x7f0304103590 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d4b0 con 0x7f03041019e0 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f030410d7b0 con 0x7f0304102bd0 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43204/0 (socket says 172.21.15.122:43204) 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 -- 172.21.15.122:0/605973244 learned_addr learned my addr 172.21.15.122:0/605973244 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 -- 172.21.15.122:0/605973244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 msgr2=0x7f03041981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f03037fe640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 0x7f03041981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 0x7f03041981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 -- 172.21.15.122:0/605973244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03041019e0 msgr2=0x7f0304197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0303fff640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03041019e0 0x7f0304197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03041019e0 0x7f0304197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 -- 172.21.15.122:0/605973244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030419f8c0 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f03037fe640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 0x7f03041981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f0308b9c640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00cce0 tx=0x7f02fc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc00f450 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.934+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030419fbb0 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.935+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc00fad0 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.935+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc018c10 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.935+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041a0100 con 0x7f0304103590 2024-04-02T13:33:04.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.935+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0304108d00 con 0x7f0304103590 2024-04-02T13:33:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.937+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f02fc00f5f0 con 0x7f0304103590 2024-04-02T13:33:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.937+0000 7f03017fa640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 0x7f02ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.937+0000 7f0303fff640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 0x7f02ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.937+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f02fc0d0dc0 con 0x7f0304103590 2024-04-02T13:33:04.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.937+0000 7f0303fff640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 0x7f02ec0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02f402c8e0 tx=0x7f02f4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:04.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:04.941+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f02fc014030 con 0x7f0304103590 2024-04-02T13:33:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2748822308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1305402506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3298631537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2748822308' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1305402506' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3298631537' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.119+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0304062700 con 0x7f0304103590 2024-04-02T13:33:05.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.119+0000 7f03017fa640 1 -- 172.21.15.122:0/605973244 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f02fc09d560 con 0x7f0304103590 2024-04-02T13:33:05.122 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:33:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 msgr2=0x7f02ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 0x7f02ec0a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f02f402c8e0 tx=0x7f02f4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 msgr2=0x7f030419f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00cce0 tx=0x7f02fc00a620 comp rx=0 tx=0).stop 2024-04-02T13:33:05.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 shutdown_connections 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0304103590 0x7f030419f220 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0304102bd0 0x7f03041981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f02ec0a3bb0 0x7f02ec0a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 --2- 172.21.15.122:0/605973244 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03041019e0 0x7f0304197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.121+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 >> 172.21.15.122:0/605973244 conn(0x7f03040fcf70 msgr2=0x7f0304105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.122+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 shutdown_connections 2024-04-02T13:33:05.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.122+0000 7f030939d640 1 -- 172.21.15.122:0/605973244 wait complete. 2024-04-02T13:33:05.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 -- 172.21.15.122:0/1279045350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47680ffde0 msgr2=0x7f4768100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 --2- 172.21.15.122:0/1279045350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47680ffde0 0x7f4768100200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4764009930 tx=0x7f476402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 -- 172.21.15.122:0/1279045350 shutdown_connections 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 --2- 172.21.15.122:0/1279045350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4768103770 0x7f476810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 --2- 172.21.15.122:0/1279045350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4768102db0 0x7f4768103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 --2- 172.21.15.122:0/1279045350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47680ffde0 0x7f4768100200 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 -- 172.21.15.122:0/1279045350 >> 172.21.15.122:0/1279045350 conn(0x7f47680fb710 msgr2=0x7f47680fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 -- 172.21.15.122:0/1279045350 shutdown_connections 2024-04-02T13:33:05.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.236+0000 7f476fd82640 1 -- 172.21.15.122:0/1279045350 wait complete. 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 Processor -- start 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 -- start start 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47680ffde0 0x7f4768197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4768102db0 0x7f4768198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f476810d890 con 0x7f4768103770 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f476810d710 con 0x7f47680ffde0 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f476810da10 con 0x7f4768102db0 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43214/0 (socket says 172.21.15.122:43214) 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 -- 172.21.15.122:0/2546141989 learned_addr learned my addr 172.21.15.122:0/2546141989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476db1e640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47680ffde0 0x7f4768197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 -- 172.21.15.122:0/2546141989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4768102db0 msgr2=0x7f4768198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4768102db0 0x7f4768198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 -- 172.21.15.122:0/2546141989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47680ffde0 msgr2=0x7f4768197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47680ffde0 0x7f4768197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 -- 172.21.15.122:0/2546141989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476819fb50 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476e31f640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f476000b970 tx=0x7f476000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760019070 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476000cb40 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476819fe40 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.237+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760007650 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.238+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681a0310 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.239+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4768108ee0 con 0x7f4768103770 2024-04-02T13:33:05.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.240+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f476001c070 con 0x7f4768103770 2024-04-02T13:33:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.240+0000 7f475effd640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.240+0000 7f476db1e640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.240+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f47600cfc90 con 0x7f4768103770 2024-04-02T13:33:05.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.241+0000 7f476db1e640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 0x7f47540a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f476402c7b0 tx=0x7f476402cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:05.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.245+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f476009c430 con 0x7f4768103770 2024-04-02T13:33:05.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.420+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f47680627a0 con 0x7f4768103770 2024-04-02T13:33:05.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.421+0000 7f475effd640 1 -- 172.21.15.122:0/2546141989 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v90) ==== 103+0+11 (secure 0 0 0) 0x7f476009bd30 con 0x7f4768103770 2024-04-02T13:33:05.424 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 msgr2=0x7f47540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 0x7f47540a6070 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f476402c7b0 tx=0x7f476402cf90 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 msgr2=0x7f476819f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f476000b970 tx=0x7f476000be30 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 shutdown_connections 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4768103770 0x7f476819f4e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4768102db0 0x7f4768198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f47540a3bb0 0x7f47540a6070 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 --2- 172.21.15.122:0/2546141989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47680ffde0 0x7f4768197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 >> 172.21.15.122:0/2546141989 conn(0x7f47680fb710 msgr2=0x7f47681060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:05.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 shutdown_connections 2024-04-02T13:33:05.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.423+0000 7f476fd82640 1 -- 172.21.15.122:0/2546141989 wait complete. 2024-04-02T13:33:05.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:33:05.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- 172.21.15.122:0/2982728818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 msgr2=0x7f1d78102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/2982728818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d78102070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009900 tx=0x7f1d6c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- 172.21.15.122:0/2982728818 shutdown_connections 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/2982728818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d781025b0 0x7f1d78108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/2982728818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d78102070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/2982728818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d78100a00 0x7f1d78100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- 172.21.15.122:0/2982728818 >> 172.21.15.122:0/2982728818 conn(0x7f1d780fbf90 msgr2=0x7f1d780fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- 172.21.15.122:0/2982728818 shutdown_connections 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- 172.21.15.122:0/2982728818 wait complete. 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 Processor -- start 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.538+0000 7f1d7f20d640 1 -- start start 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d78100a00 0x7f1d780697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d781025b0 0x7f1d7819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7810c660 con 0x7f1d78101bf0 2024-04-02T13:33:05.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7810c4e0 con 0x7f1d781025b0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7810c7e0 con 0x7f1d78100a00 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43236/0 (socket says 172.21.15.122:43236) 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 -- 172.21.15.122:0/1122874821 learned_addr learned my addr 172.21.15.122:0/1122874821 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7cfa9640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d78100a00 0x7f1d780697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7d7aa640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d781025b0 0x7f1d7819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 -- 172.21.15.122:0/1122874821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d78100a00 msgr2=0x7f1d780697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d78100a00 0x7f1d780697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 -- 172.21.15.122:0/1122874821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d781025b0 msgr2=0x7f1d7819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d781025b0 0x7f1d7819f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 -- 172.21.15.122:0/1122874821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7819fab0 con 0x7f1d78101bf0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d77fff640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009230 tx=0x7f1d6c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c041070 con 0x7f1d78101bf0 2024-04-02T13:33:05.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7819fce0 con 0x7f1d78101bf0 2024-04-02T13:33:05.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d6c0054e0 con 0x7f1d78101bf0 2024-04-02T13:33:05.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.539+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c034440 con 0x7f1d78101bf0 2024-04-02T13:33:05.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.540+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d781a01d0 con 0x7f1d78101bf0 2024-04-02T13:33:05.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.540+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d78107d70 con 0x7f1d78101bf0 2024-04-02T13:33:05.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.542+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f1d6c044070 con 0x7f1d78101bf0 2024-04-02T13:33:05.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.542+0000 7f1d75ffb640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 0x7f1d600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:05.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.543+0000 7f1d7cfa9640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 0x7f1d600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:05.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.543+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f1d6c0f27b0 con 0x7f1d78101bf0 2024-04-02T13:33:05.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.543+0000 7f1d7cfa9640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 0x7f1d600a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d78101a50 tx=0x7f1d68009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:05.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.546+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1d6c0bef50 con 0x7f1d78101bf0 2024-04-02T13:33:05.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.738+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d7802d370 con 0x7f1d78101bf0 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.738+0000 7f1d75ffb640 1 -- 172.21.15.122:0/1122874821 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+600 (secure 0 0 0) 0x7f1d6c0be850 con 0x7f1d78101bf0 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3421882822 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/684561021 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:05.741 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/776403907 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/4063470519 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1474593903 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1325797276 2024-04-03T13:25:27.690608+0000 2024-04-02T13:33:05.742 INFO:teuthology.orchestra.run.smithi122.stderr:listed 10 entries 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 msgr2=0x7f1d600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 0x7f1d600a6070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1d78101a50 tx=0x7f1d68009870 comp rx=0 tx=0).stop 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 msgr2=0x7f1d7819cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009230 tx=0x7f1d6c005dd0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 shutdown_connections 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1d600a3bb0 0x7f1d600a6070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d781025b0 0x7f1d7819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d78101bf0 0x7f1d7819cb80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 --2- 172.21.15.122:0/1122874821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d78100a00 0x7f1d780697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 >> 172.21.15.122:0/1122874821 conn(0x7f1d780fbf90 msgr2=0x7f1d78104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 shutdown_connections 2024-04-02T13:33:05.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:05.740+0000 7f1d7f20d640 1 -- 172.21.15.122:0/1122874821 wait complete. 2024-04-02T13:33:05.754 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:05 smithi122 ceph-mon[32114]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s 2024-04-02T13:33:05.754 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3859226613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.754 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/605973244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.754 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2546141989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.755 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi122.front.sepia.ceph.com... 2024-04-02T13:33:05.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:05.755 DEBUG:teuthology.orchestra.run.smithi122:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-02T13:33:05.784 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi122.stderr:ceph-fuse[73569]: fuse finished with error 0 and tester_r 0 2024-04-02T13:33:05.786 INFO:teuthology.orchestra.run:waiting for 300 2024-04-02T13:33:05.852 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:33:05.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:33:05.853 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-02T13:33:05.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:33:05.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:05 smithi188 ceph-mon[36117]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s 2024-04-02T13:33:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3859226613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/605973244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2546141989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:05 smithi187 ceph-mon[36064]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 309 B/s wr, 0 op/s 2024-04-02T13:33:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3859226613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/605973244' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:33:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2546141989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 -- 172.21.15.122:0/1555046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c103700 msgr2=0x7fa96c109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 --2- 172.21.15.122:0/1555046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c103700 0x7fa96c109f90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa964009900 tx=0x7fa96402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 -- 172.21.15.122:0/1555046797 shutdown_connections 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 --2- 172.21.15.122:0/1555046797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c103700 0x7fa96c109f90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 --2- 172.21.15.122:0/1555046797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 0x7fa96c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 --2- 172.21.15.122:0/1555046797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c101b50 0x7fa96c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 -- 172.21.15.122:0/1555046797 >> 172.21.15.122:0/1555046797 conn(0x7fa96c0fd0a0 msgr2=0x7fa96c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 -- 172.21.15.122:0/1555046797 shutdown_connections 2024-04-02T13:33:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.033+0000 7fa971111640 1 -- 172.21.15.122:0/1555046797 wait complete. 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 Processor -- start 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 -- start start 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 0x7fa96c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 0x7fa96c19f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c10d7b0 con 0x7fa96c101b50 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c10d630 con 0x7fa96c103700 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa971111640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c10d930 con 0x7fa96c102d40 2024-04-02T13:33:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa96ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa96ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43254/0 (socket says 172.21.15.122:43254) 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa96ad9d640 1 -- 172.21.15.122:0/1531985421 learned_addr learned my addr 172.21.15.122:0/1531985421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.034+0000 7fa96a59c640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 0x7fa96c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96b59e640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 0x7fa96c19f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 -- 172.21.15.122:0/1531985421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 msgr2=0x7fa96c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 0x7fa96c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 -- 172.21.15.122:0/1531985421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 msgr2=0x7fa96c19f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 0x7fa96c19f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 -- 172.21.15.122:0/1531985421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c19f9f0 con 0x7fa96c101b50 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96b59e640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 0x7fa96c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa96ad9d640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00ea80 tx=0x7fa95c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00ce40 con 0x7fa96c101b50 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c19fce0 con 0x7fa96c101b50 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c011950 con 0x7fa96c101b50 2024-04-02T13:33:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c01ab80 con 0x7fa96c101b50 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.035+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c1a0180 con 0x7fa96c101b50 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.036+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c108e70 con 0x7fa96c101b50 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.037+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fa95c011b00 con 0x7fa96c101b50 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.038+0000 7fa953fff640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 0x7fa9540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.038+0000 7fa96a59c640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 0x7fa9540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.038+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa95c0d0cf0 con 0x7fa96c101b50 2024-04-02T13:33:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.038+0000 7fa96a59c640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 0x7fa9540a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa96c1992e0 tx=0x7fa96000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.043+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa95c0d5070 con 0x7fa96c101b50 2024-04-02T13:33:06.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.223+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96c062800 con 0x7fa96c101b50 2024-04-02T13:33:06.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.225+0000 7fa953fff640 1 -- 172.21.15.122:0/1531985421 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7fa95c016030 con 0x7fa96c101b50 2024-04-02T13:33:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:06.228 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:06.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 msgr2=0x7fa9540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 0x7fa9540a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa96c1992e0 tx=0x7fa96000b410 comp rx=0 tx=0).stop 2024-04-02T13:33:06.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 msgr2=0x7fa96c197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00ea80 tx=0x7fa95c00ef40 comp rx=0 tx=0).stop 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 shutdown_connections 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa9540a3bb0 0x7fa9540a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96c103700 0x7fa96c19f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa96c102d40 0x7fa96c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 --2- 172.21.15.122:0/1531985421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa96c101b50 0x7fa96c197dd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 >> 172.21.15.122:0/1531985421 conn(0x7fa96c0fd0a0 msgr2=0x7fa96c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 shutdown_connections 2024-04-02T13:33:06.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.227+0000 7fa971111640 1 -- 172.21.15.122:0/1531985421 wait complete. 2024-04-02T13:33:06.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:33:06.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 -- 172.21.15.122:0/1991515749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77981025d0 msgr2=0x7f7798108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 --2- 172.21.15.122:0/1991515749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77981025d0 0x7f7798108e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7790009ca0 tx=0x7f779002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 -- 172.21.15.122:0/1991515749 shutdown_connections 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 --2- 172.21.15.122:0/1991515749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77981025d0 0x7f7798108e60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 --2- 172.21.15.122:0/1991515749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7798101c10 0x7f7798102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 --2- 172.21.15.122:0/1991515749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 0x7f7798100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 -- 172.21.15.122:0/1991515749 >> 172.21.15.122:0/1991515749 conn(0x7f77980fbf90 msgr2=0x7f77980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 -- 172.21.15.122:0/1991515749 shutdown_connections 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.347+0000 7f779df5f640 1 -- 172.21.15.122:0/1991515749 wait complete. 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 Processor -- start 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 -- start start 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 0x7f7798197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77981025d0 0x7f779819f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d780 con 0x7f7798101c10 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d600 con 0x7f7798100a20 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f779df5f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f779810d900 con 0x7f77981025d0 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f778effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f778effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43266/0 (socket says 172.21.15.122:43266) 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.348+0000 7f778effd640 1 -- 172.21.15.122:0/1815800704 learned_addr learned my addr 172.21.15.122:0/1815800704 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f7797fff640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77981025d0 0x7f779819f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f77977fe640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 0x7f7798197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 -- 172.21.15.122:0/1815800704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77981025d0 msgr2=0x7f779819f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77981025d0 0x7f779819f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 -- 172.21.15.122:0/1815800704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 msgr2=0x7f7798197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 0x7f7798197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 -- 172.21.15.122:0/1815800704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779819f960 con 0x7f7798101c10 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f778effd640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f778000c920 tx=0x7f778000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780007c70 con 0x7f7798101c10 2024-04-02T13:33:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7780007e10 con 0x7f7798101c10 2024-04-02T13:33:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780018890 con 0x7f7798101c10 2024-04-02T13:33:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779819fc50 con 0x7f7798101c10 2024-04-02T13:33:06.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.349+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77981a01a0 con 0x7f7798101c10 2024-04-02T13:33:06.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.350+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7798107ce0 con 0x7f7798101c10 2024-04-02T13:33:06.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.351+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7780018a50 con 0x7f7798101c10 2024-04-02T13:33:06.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.352+0000 7f77957fa640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 0x7f77840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.352+0000 7f77977fe640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 0x7f77840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.352+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f77800d09b0 con 0x7f7798101c10 2024-04-02T13:33:06.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.352+0000 7f77977fe640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 0x7f77840a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7798199220 tx=0x7f778800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.355+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7780014030 con 0x7f7798101c10 2024-04-02T13:33:06.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.539+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f779802d540 con 0x7f7798101c10 2024-04-02T13:33:06.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.540+0000 7f77957fa640 1 -- 172.21.15.122:0/1815800704 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f778009d150 con 0x7f7798101c10 2024-04-02T13:33:06.544 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:06.544 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 msgr2=0x7f77840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 0x7f77840a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7798199220 tx=0x7f778800b410 comp rx=0 tx=0).stop 2024-04-02T13:33:06.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 msgr2=0x7f7798198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f778000c920 tx=0x7f778000cde0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 shutdown_connections 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77981025d0 0x7f779819f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7798101c10 0x7f7798198290 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f77840a3bb0 0x7f77840a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 --2- 172.21.15.122:0/1815800704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7798100a20 0x7f7798197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.542+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 >> 172.21.15.122:0/1815800704 conn(0x7f77980fbf90 msgr2=0x7f7798104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.543+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 shutdown_connections 2024-04-02T13:33:06.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.543+0000 7f779df5f640 1 -- 172.21.15.122:0/1815800704 wait complete. 2024-04-02T13:33:06.557 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 -- 172.21.15.122:0/1180507561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 msgr2=0x7f4a70101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/1180507561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70101f40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009900 tx=0x7f4a6c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 -- 172.21.15.122:0/1180507561 shutdown_connections 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/1180507561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a701036d0 0x7f4a70109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/1180507561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a70102d10 0x7f4a70103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/1180507561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70101f40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 -- 172.21.15.122:0/1180507561 >> 172.21.15.122:0/1180507561 conn(0x7f4a700fd0b0 msgr2=0x7f4a700ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 -- 172.21.15.122:0/1180507561 shutdown_connections 2024-04-02T13:33:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.660+0000 7f4a77ceb640 1 -- 172.21.15.122:0/1180507561 wait complete. 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 Processor -- start 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 -- start start 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a70102d10 0x7f4a70198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a701036d0 0x7f4a7019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010d790 con 0x7f4a70101b20 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010d610 con 0x7f4a70102d10 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a77ceb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7010d910 con 0x7f4a701036d0 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43290/0 (socket says 172.21.15.122:43290) 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 -- 172.21.15.122:0/3911736849 learned_addr learned my addr 172.21.15.122:0/3911736849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75286640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a70102d10 0x7f4a70198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a76288640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a701036d0 0x7f4a7019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 -- 172.21.15.122:0/3911736849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a701036d0 msgr2=0x7f4a7019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a701036d0 0x7f4a7019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 -- 172.21.15.122:0/3911736849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a70102d10 msgr2=0x7f4a70198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a70102d10 0x7f4a70198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.661+0000 7f4a75a87640 1 -- 172.21.15.122:0/3911736849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7019f9f0 con 0x7f4a70101b20 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a75a87640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c000c00 tx=0x7f4a6c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c041070 con 0x7f4a70101b20 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c0044d0 con 0x7f4a70101b20 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7019fc80 con 0x7f4a70101b20 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c034440 con 0x7f4a70101b20 2024-04-02T13:33:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.662+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a701a0170 con 0x7f4a70101b20 2024-04-02T13:33:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.663+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a70108e40 con 0x7f4a70101b20 2024-04-02T13:33:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.664+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4a6c044070 con 0x7f4a70101b20 2024-04-02T13:33:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.664+0000 7f4a66ffd640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 0x7f4a5c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.664+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4a6c0f2120 con 0x7f4a70101b20 2024-04-02T13:33:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.664+0000 7f4a75286640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 0x7f4a5c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.665+0000 7f4a75286640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 0x7f4a5c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a701992e0 tx=0x7f4a60005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.669+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4a6c0be270 con 0x7f4a70101b20 2024-04-02T13:33:06.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1122874821' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:06.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1531985421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:06.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1815800704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:06.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:06.885+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4a7002d370 con 0x7f4a70101b20 2024-04-02T13:33:06.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:06 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1122874821' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:06.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:06 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1531985421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:06.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:06 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1815800704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:06 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1122874821' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:33:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:06 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1531985421' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:07.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:06 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1815800704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.592+0000 7f4a66ffd640 1 -- 172.21.15.122:0/3911736849 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4a6c0c2570 con 0x7f4a70101b20 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 msgr2=0x7f4a5c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 0x7f4a5c0a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4a701992e0 tx=0x7f4a60005f90 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 msgr2=0x7f4a70197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c000c00 tx=0x7f4a6c005d50 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 shutdown_connections 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a701036d0 0x7f4a7019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a5c0a3bb0 0x7f4a5c0a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a70102d10 0x7f4a70198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 --2- 172.21.15.122:0/3911736849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a70101b20 0x7f4a70197dd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 >> 172.21.15.122:0/3911736849 conn(0x7f4a700fd0b0 msgr2=0x7f4a70106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 shutdown_connections 2024-04-02T13:33:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.595+0000 7f4a77ceb640 1 -- 172.21.15.122:0/3911736849 wait complete. 2024-04-02T13:33:07.611 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-02T13:33:07.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-02T13:33:07.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 -- 172.21.15.122:0/1131859905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634100a00 msgr2=0x7f2634100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:07.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 --2- 172.21.15.122:0/1131859905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634100a00 0x7f2634100e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2624009900 tx=0x7f262402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 -- 172.21.15.122:0/1131859905 shutdown_connections 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 --2- 172.21.15.122:0/1131859905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26341025b0 0x7f2634108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 --2- 172.21.15.122:0/1131859905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634101bf0 0x7f2634102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 --2- 172.21.15.122:0/1131859905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2634100a00 0x7f2634100e20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 -- 172.21.15.122:0/1131859905 >> 172.21.15.122:0/1131859905 conn(0x7f26340fbf90 msgr2=0x7f26340fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 -- 172.21.15.122:0/1131859905 shutdown_connections 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.718+0000 7f263a359640 1 -- 172.21.15.122:0/1131859905 wait complete. 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 Processor -- start 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 -- start start 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 0x7f263410b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:07.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 0x7f26341099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f263410d7a0 con 0x7f26341025b0 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f263410d620 con 0x7f2634100a00 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f263a359640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f263410d920 con 0x7f2634101bf0 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f26388f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f26388f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43316/0 (socket says 172.21.15.122:43316) 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.719+0000 7f26388f6640 1 -- 172.21.15.122:0/3476110723 learned_addr learned my addr 172.21.15.122:0/3476110723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26337fe640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 0x7f26341099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f2633fff640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 0x7f263410b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:07.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 -- 172.21.15.122:0/3476110723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 msgr2=0x7f26341099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 0x7f26341099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 -- 172.21.15.122:0/3476110723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 msgr2=0x7f263410b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 0x7f263410b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 -- 172.21.15.122:0/3476110723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263410a790 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26337fe640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 0x7f26341099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f2633fff640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 0x7f263410b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26388f6640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f262c00eb10 tx=0x7f262c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c004b20 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26341a1ee0 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262c004cc0 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c010760 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.720+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26341a2340 con 0x7f26341025b0 2024-04-02T13:33:07.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.721+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634107cc0 con 0x7f26341025b0 2024-04-02T13:33:07.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.722+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f262c005000 con 0x7f26341025b0 2024-04-02T13:33:07.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.723+0000 7f26317fa640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 0x7f261c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:07.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.723+0000 7f2633fff640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 0x7f261c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:07.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.723+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f262c0d11a0 con 0x7f26341025b0 2024-04-02T13:33:07.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.723+0000 7f2633fff640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 0x7f261c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2624036040 tx=0x7f2624000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:07.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.726+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f262c09d940 con 0x7f26341025b0 2024-04-02T13:33:07.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:07.944+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f263402d370 con 0x7f26341025b0 2024-04-02T13:33:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:07 smithi187 ceph-mon[36064]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s 2024-04-02T13:33:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:33:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:07 smithi188 ceph-mon[36117]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s 2024-04-02T13:33:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:33:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:07 smithi122 ceph-mon[32114]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 281 B/s wr, 0 op/s 2024-04-02T13:33:08.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-02T13:33:08.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.606+0000 7f26317fa640 1 -- 172.21.15.122:0/3476110723 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f262c09d240 con 0x7f26341025b0 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 msgr2=0x7f261c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 0x7f261c0a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2624036040 tx=0x7f2624000bc0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 msgr2=0x7f2634109f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f262c00eb10 tx=0x7f262c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 shutdown_connections 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26341025b0 0x7f2634109f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2634101bf0 0x7f26341099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f261c0a3bb0 0x7f261c0a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 --2- 172.21.15.122:0/3476110723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2634100a00 0x7f263410b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 >> 172.21.15.122:0/3476110723 conn(0x7f26340fbf90 msgr2=0x7f2634104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:08.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 shutdown_connections 2024-04-02T13:33:08.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.609+0000 7f263a359640 1 -- 172.21.15.122:0/3476110723 wait complete. 2024-04-02T13:33:08.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 -- 172.21.15.122:0/3650369871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 msgr2=0x7f31e4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 --2- 172.21.15.122:0/3650369871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4102070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009900 tx=0x7f31d402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 -- 172.21.15.122:0/3650369871 shutdown_connections 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 --2- 172.21.15.122:0/3650369871 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e41025b0 0x7f31e4108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 --2- 172.21.15.122:0/3650369871 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4102070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 --2- 172.21.15.122:0/3650369871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e4100a00 0x7f31e4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 -- 172.21.15.122:0/3650369871 >> 172.21.15.122:0/3650369871 conn(0x7f31e40fbf70 msgr2=0x7f31e40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 -- 172.21.15.122:0/3650369871 shutdown_connections 2024-04-02T13:33:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 -- 172.21.15.122:0/3650369871 wait complete. 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.734+0000 7f31e9e82640 1 Processor -- start 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 -- start start 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 0x7f31e4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 0x7f31e419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410c6f0 con 0x7f31e4101bf0 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410c570 con 0x7f31e4100a00 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e9e82640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31e410c870 con 0x7f31e41025b0 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33578/0 (socket says 172.21.15.122:33578) 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 -- 172.21.15.122:0/2520967672 learned_addr learned my addr 172.21.15.122:0/2520967672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e3fff640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 0x7f31e419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e37fe640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 0x7f31e4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 -- 172.21.15.122:0/2520967672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 msgr2=0x7f31e419e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 0x7f31e419e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 -- 172.21.15.122:0/2520967672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 msgr2=0x7f31e4196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 0x7f31e4196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 -- 172.21.15.122:0/2520967672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e419e960 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e3fff640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 0x7f31e419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e37fe640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 0x7f31e4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.735+0000 7f31e2ffd640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009230 tx=0x7f31d4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.736+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d4041070 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.736+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31e419ebf0 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.736+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31d40054e0 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.736+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40346a0 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.736+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31e419f130 con 0x7f31e4101bf0 2024-04-02T13:33:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.737+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31e4107cc0 con 0x7f31e4101bf0 2024-04-02T13:33:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.738+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f31d4044070 con 0x7f31e4101bf0 2024-04-02T13:33:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.738+0000 7f31e0ff9640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.738+0000 7f31e37fe640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.739+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f31d40f27f0 con 0x7f31e4101bf0 2024-04-02T13:33:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.739+0000 7f31e37fe640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006fd0 tx=0x7f31d8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:08.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.742+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f31d40bef90 con 0x7f31e4101bf0 2024-04-02T13:33:08.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:08.972+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f31e402d590 con 0x7f31e4101bf0 2024-04-02T13:33:08.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:33:08.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:08 smithi187 ceph-mon[36064]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:08.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:33:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:33:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:08 smithi188 ceph-mon[36117]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:33:09.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3911736849' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-02T13:33:09.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:08 smithi122 ceph-mon[32114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:09.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-02T13:33:09.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.616+0000 7f31e0ff9640 1 -- 172.21.15.122:0/2520967672 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f31d40a7e20 con 0x7f31e4101bf0 2024-04-02T13:33:09.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 msgr2=0x7f31cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f31d8006fd0 tx=0x7f31d8008040 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 msgr2=0x7f31e4197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31d4009230 tx=0x7f31d4004040 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 shutdown_connections 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e41025b0 0x7f31e419e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31e4101bf0 0x7f31e4197280 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31cc0a3bb0 0x7f31cc0a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 --2- 172.21.15.122:0/2520967672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31e4100a00 0x7f31e4196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 >> 172.21.15.122:0/2520967672 conn(0x7f31e40fbf70 msgr2=0x7f31e4104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 shutdown_connections 2024-04-02T13:33:09.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.619+0000 7f31e9e82640 1 -- 172.21.15.122:0/2520967672 wait complete. 2024-04-02T13:33:09.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-02T13:33:09.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 -- 172.21.15.122:0/1396729400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4101b10 msgr2=0x7ff8b4101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:09.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 --2- 172.21.15.122:0/1396729400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4101b10 0x7ff8b4101f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4009900 tx=0x7ff8a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 -- 172.21.15.122:0/1396729400 shutdown_connections 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 --2- 172.21.15.122:0/1396729400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 0x7ff8b4109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 --2- 172.21.15.122:0/1396729400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4102d00 0x7ff8b4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 --2- 172.21.15.122:0/1396729400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4101b10 0x7ff8b4101f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 -- 172.21.15.122:0/1396729400 >> 172.21.15.122:0/1396729400 conn(0x7ff8b40fd0a0 msgr2=0x7ff8b40ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.744+0000 7ff8ba260640 1 -- 172.21.15.122:0/1396729400 shutdown_connections 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 -- 172.21.15.122:0/1396729400 wait complete. 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 Processor -- start 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 -- start start 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4101b10 0x7ff8b4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 0x7ff8b419f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:09.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410d780 con 0x7ff8b4102d00 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410d600 con 0x7ff8b41036c0 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8ba260640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b410d900 con 0x7ff8b4101b10 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.745+0000 7ff8aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33602/0 (socket says 172.21.15.122:33602) 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 -- 172.21.15.122:0/541283902 learned_addr learned my addr 172.21.15.122:0/541283902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8b37fe640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4101b10 0x7ff8b4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8b3fff640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 0x7ff8b419f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 -- 172.21.15.122:0/541283902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4101b10 msgr2=0x7ff8b4197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4101b10 0x7ff8b4197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 -- 172.21.15.122:0/541283902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 msgr2=0x7ff8b419f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 0x7ff8b419f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 -- 172.21.15.122:0/541283902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b419faa0 con 0x7ff8b4102d00 2024-04-02T13:33:09.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8aaffd640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00d770 tx=0x7ff89c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:09.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c017070 con 0x7ff8b4102d00 2024-04-02T13:33:09.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89c005500 con 0x7ff8b4102d00 2024-04-02T13:33:09.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b419fd90 con 0x7ff8b4102d00 2024-04-02T13:33:09.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004e00 con 0x7ff8b4102d00 2024-04-02T13:33:09.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.746+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b41a0230 con 0x7ff8b4102d00 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.747+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8b4108ea0 con 0x7ff8b4102d00 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.748+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7ff89c004fa0 con 0x7ff8b4102d00 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.749+0000 7ff8b17fa640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 0x7ff8a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.749+0000 7ff8b37fe640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 0x7ff8a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.749+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff89c0cffe0 con 0x7ff8b4102d00 2024-04-02T13:33:09.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.749+0000 7ff8b37fe640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 0x7ff8a00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:09.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.754+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff89c09c130 con 0x7ff8b4102d00 2024-04-02T13:33:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:09.968+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff8b4062700 con 0x7ff8b4102d00 2024-04-02T13:33:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:33:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:33:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:33:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:33:09.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:09.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:33:09.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:33:09.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:33:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:33:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:33:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:33:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:33:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:33:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:33:10.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:33:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3476110723' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-02T13:33:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-02T13:33:10.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.626+0000 7ff8b17fa640 1 -- 172.21.15.122:0/541283902 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7ff89c0a0430 con 0x7ff8b4102d00 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 msgr2=0x7ff8a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 0x7ff8a00a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4000c00 tx=0x7ff8a4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 msgr2=0x7ff8b41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00d770 tx=0x7ff89c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 shutdown_connections 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7ff8a00a3bb0 0x7ff8a00a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b41036c0 0x7ff8b419f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4102d00 0x7ff8b41983c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 --2- 172.21.15.122:0/541283902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b4101b10 0x7ff8b4197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 >> 172.21.15.122:0/541283902 conn(0x7ff8b40fd0a0 msgr2=0x7ff8b4106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 shutdown_connections 2024-04-02T13:33:10.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.629+0000 7ff8ba260640 1 -- 172.21.15.122:0/541283902 wait complete. 2024-04-02T13:33:10.647 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-02T13:33:10.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 -- 172.21.15.122:0/3320328701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b4100900 msgr2=0x7f99b4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 --2- 172.21.15.122:0/3320328701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b4100900 0x7f99b4100d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009900 tx=0x7f99a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 -- 172.21.15.122:0/3320328701 shutdown_connections 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 --2- 172.21.15.122:0/3320328701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b41024b0 0x7f99b4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 --2- 172.21.15.122:0/3320328701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4101af0 0x7f99b4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 --2- 172.21.15.122:0/3320328701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b4100900 0x7f99b4100d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 -- 172.21.15.122:0/3320328701 >> 172.21.15.122:0/3320328701 conn(0x7f99b40fbe70 msgr2=0x7f99b40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 -- 172.21.15.122:0/3320328701 shutdown_connections 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 -- 172.21.15.122:0/3320328701 wait complete. 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.750+0000 7f99b359e640 1 Processor -- start 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 -- start start 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 0x7f99b406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b4101af0 0x7f99b419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99b410c570 con 0x7f99b41024b0 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99b410c3f0 con 0x7f99b4101af0 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99b410c6f0 con 0x7f99b4100900 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33622/0 (socket says 172.21.15.122:33622) 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 -- 172.21.15.122:0/1290342617 learned_addr learned my addr 172.21.15.122:0/1290342617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99a9d9b640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b4101af0 0x7f99b419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b259c640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 0x7f99b406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 -- 172.21.15.122:0/1290342617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 msgr2=0x7f99b406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 0x7f99b406abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 -- 172.21.15.122:0/1290342617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b4101af0 msgr2=0x7f99b419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b4101af0 0x7f99b419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 -- 172.21.15.122:0/1290342617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b419f940 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b259c640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 0x7f99b406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.751+0000 7f99b2d9d640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00eb10 tx=0x7f99ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.752+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac004b20 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.752+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ac004cc0 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.752+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b419fc30 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.752+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac010760 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.752+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41a0130 con 0x7f99b41024b0 2024-04-02T13:33:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.753+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b4107ca0 con 0x7f99b41024b0 2024-04-02T13:33:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.754+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f99ac005020 con 0x7f99b41024b0 2024-04-02T13:33:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.754+0000 7f99abfff640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 0x7f99a00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.754+0000 7f99b259c640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 0x7f99a00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.754+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f99ac0d0c60 con 0x7f99b41024b0 2024-04-02T13:33:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.755+0000 7f99b259c640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 0x7f99a00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99a4036040 tx=0x7f99a4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.759+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f99ac09d400 con 0x7f99b41024b0 2024-04-02T13:33:10.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.964+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f99b4061f40 con 0x7f99a00a3bb0 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:active 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-02T13:33:10.974 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:33:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:33:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:active 2024-04-02T13:33:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:33:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:33:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-02T13:33:10.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-02T13:33:10.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:33:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.984+0000 7f99abfff640 1 -- 172.21.15.122:0/1290342617 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f99b4061f40 con 0x7f99a00a3bb0 2024-04-02T13:33:10.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 msgr2=0x7f99a00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 0x7f99a00a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99a4036040 tx=0x7f99a4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 msgr2=0x7f99b419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00eb10 tx=0x7f99ac00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 shutdown_connections 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99b41024b0 0x7f99b419f080 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f99a00a3bb0 0x7f99a00a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99b4101af0 0x7f99b419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 --2- 172.21.15.122:0/1290342617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b4100900 0x7f99b406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 >> 172.21.15.122:0/1290342617 conn(0x7f99b40fbe70 msgr2=0x7f99b4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 shutdown_connections 2024-04-02T13:33:10.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:10.987+0000 7f99b359e640 1 -- 172.21.15.122:0/1290342617 wait complete. 2024-04-02T13:33:11.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2520967672' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] up:active 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-02T13:33:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-02T13:33:11.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-02T13:33:11.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.103+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2683470958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d44104eb0 msgr2=0x7f4d441072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.103+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2683470958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d44104eb0 0x7f4d441072a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009900 tx=0x7f4d3402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:11.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2683470958 shutdown_connections 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2683470958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d441077e0 0x7f4d44109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2683470958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d44104eb0 0x7f4d441072a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2683470958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44102580 0x7f4d44104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2683470958 >> 172.21.15.122:0/2683470958 conn(0x7f4d440fbf90 msgr2=0x7f4d440fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2683470958 shutdown_connections 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2683470958 wait complete. 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 Processor -- start 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- start start 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d44102580 0x7f4d44197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 0x7f4d44198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4410d780 con 0x7f4d441077e0 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4410d600 con 0x7f4d44104eb0 2024-04-02T13:33:11.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d4b3c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4410d900 con 0x7f4d44102580 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d49965640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d49965640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33670/0 (socket says 172.21.15.122:33670) 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d48963640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 0x7f4d44198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.104+0000 7f4d49965640 1 -- 172.21.15.122:0/2811901845 learned_addr learned my addr 172.21.15.122:0/2811901845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49164640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d44102580 0x7f4d44197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 -- 172.21.15.122:0/2811901845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d44102580 msgr2=0x7f4d44197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d44102580 0x7f4d44197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 -- 172.21.15.122:0/2811901845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 msgr2=0x7f4d44198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 0x7f4d44198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 -- 172.21.15.122:0/2811901845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4419fae0 con 0x7f4d441077e0 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d48963640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 0x7f4d44198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d49965640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000cad0 tx=0x7f4d4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000ae20 con 0x7f4d441077e0 2024-04-02T13:33:11.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40005020 con 0x7f4d441077e0 2024-04-02T13:33:11.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4419fdd0 con 0x7f4d441077e0 2024-04-02T13:33:11.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000f6f0 con 0x7f4d441077e0 2024-04-02T13:33:11.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.105+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d441a02e0 con 0x7f4d441077e0 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.106+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d440fda80 con 0x7f4d441077e0 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.107+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f4d400051d0 con 0x7f4d441077e0 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.108+0000 7f4d3a7fc640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 0x7f4d300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.108+0000 7f4d49164640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 0x7f4d300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.108+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4d400d0860 con 0x7f4d441077e0 2024-04-02T13:33:11.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.108+0000 7f4d49164640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 0x7f4d300a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009bd0 tx=0x7f4d3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.112+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4d40014030 con 0x7f4d441077e0 2024-04-02T13:33:11.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:33:10.969+0000 7fc346684700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:33:11.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:33:10.969+0000 7fc346684700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:33:11.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:33:10.969+0000 7fc346684700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:33:11.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:33:10.969+0000 7fc346684700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:33:11.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:33:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:33:10.969+0000 7fc346684700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:33:11.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.327+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4d44061f40 con 0x7f4d300a3c30 2024-04-02T13:33:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.353+0000 7f4d3a7fc640 1 -- 172.21.15.122:0/2811901845 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d44061f40 con 0x7f4d300a3c30 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 msgr2=0x7f4d300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 0x7f4d300a60f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009bd0 tx=0x7f4d3c005f90 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 msgr2=0x7f4d4419f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000cad0 tx=0x7f4d4000cf90 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 shutdown_connections 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d441077e0 0x7f4d4419f470 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4d300a3c30 0x7f4d300a60f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d44104eb0 0x7f4d44198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 --2- 172.21.15.122:0/2811901845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d44102580 0x7f4d44197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 >> 172.21.15.122:0/2811901845 conn(0x7f4d440fbf90 msgr2=0x7f4d44102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 shutdown_connections 2024-04-02T13:33:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.355+0000 7f4d4b3c8640 1 -- 172.21.15.122:0/2811901845 wait complete. 2024-04-02T13:33:11.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 -- 172.21.15.122:0/3828929927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 msgr2=0x7f7a9c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/3828929927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c100e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009900 tx=0x7f7a9002bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 -- 172.21.15.122:0/3828929927 shutdown_connections 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/3828929927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c1025b0 0x7f7a9c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/3828929927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c101bf0 0x7f7a9c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/3828929927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c100e20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 -- 172.21.15.122:0/3828929927 >> 172.21.15.122:0/3828929927 conn(0x7f7a9c0fbf90 msgr2=0x7f7a9c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 -- 172.21.15.122:0/3828929927 shutdown_connections 2024-04-02T13:33:11.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.473+0000 7f7aa3e69640 1 -- 172.21.15.122:0/3828929927 wait complete. 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 Processor -- start 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 -- start start 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c101bf0 0x7f7a9c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c1025b0 0x7f7a9c19f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c10c720 con 0x7f7a9c100a00 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c10c5a0 con 0x7f7a9c1025b0 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa3e69640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9c10c8a0 con 0x7f7a9c101bf0 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33688/0 (socket says 172.21.15.122:33688) 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 -- 172.21.15.122:0/280501669 learned_addr learned my addr 172.21.15.122:0/280501669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa2406640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c1025b0 0x7f7a9c19f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1404640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c101bf0 0x7f7a9c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 -- 172.21.15.122:0/280501669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c101bf0 msgr2=0x7f7a9c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c101bf0 0x7f7a9c19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 -- 172.21.15.122:0/280501669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c1025b0 msgr2=0x7f7a9c19f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c1025b0 0x7f7a9c19f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 -- 172.21.15.122:0/280501669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a9c19fb60 con 0x7f7a9c100a00 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7aa1c05640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a90000c00 tx=0x7f7a90005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.474+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90043020 con 0x7f7a9c100a00 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.475+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a900054e0 con 0x7f7a9c100a00 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.475+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a9c19fd90 con 0x7f7a9c100a00 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.475+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a90003e60 con 0x7f7a9c100a00 2024-04-02T13:33:11.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.475+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a9c1a0220 con 0x7f7a9c100a00 2024-04-02T13:33:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.476+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a9c107d20 con 0x7f7a9c100a00 2024-04-02T13:33:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.477+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7a90046020 con 0x7f7a9c100a00 2024-04-02T13:33:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.477+0000 7f7a8effd640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 0x7f7a880a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.477+0000 7f7aa1404640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 0x7f7a880a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.477+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7a900f28d0 con 0x7f7a9c100a00 2024-04-02T13:33:11.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.478+0000 7f7aa1404640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 0x7f7a880a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005fd0 tx=0x7f7a98005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.482+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7a9004d020 con 0x7f7a9c100a00 2024-04-02T13:33:11.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.692+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f7a9c061f40 con 0x7f7a880a3bb0 2024-04-02T13:33:11.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.697+0000 7f7a8effd640 1 -- 172.21.15.122:0/280501669 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7a9c061f40 con 0x7f7a880a3bb0 2024-04-02T13:33:11.701 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_0/c7374354-edc6-4a5e-8a29-4c187d3cab66 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 msgr2=0x7f7a880a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 0x7f7a880a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7a98005fd0 tx=0x7f7a98005850 comp rx=0 tx=0).stop 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 msgr2=0x7f7a9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7a90000c00 tx=0x7f7a90005dd0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 shutdown_connections 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7a880a3bb0 0x7f7a880a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7a9c1025b0 0x7f7a9c19f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a9c101bf0 0x7f7a9c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 --2- 172.21.15.122:0/280501669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7a9c100a00 0x7f7a9c0697d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 >> 172.21.15.122:0/280501669 conn(0x7f7a9c0fbf90 msgr2=0x7f7a9c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:11.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 shutdown_connections 2024-04-02T13:33:11.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.699+0000 7f7aa3e69640 1 -- 172.21.15.122:0/280501669 wait complete. 2024-04-02T13:33:11.714 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 -- 172.21.15.122:0/2704179068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac100a00 msgr2=0x7f14ac100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2704179068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac100a00 0x7f14ac100e20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009900 tx=0x7f14a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 -- 172.21.15.122:0/2704179068 shutdown_connections 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2704179068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac1025b0 0x7f14ac108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2704179068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14ac101bf0 0x7f14ac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2704179068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac100a00 0x7f14ac100e20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 -- 172.21.15.122:0/2704179068 >> 172.21.15.122:0/2704179068 conn(0x7f14ac0fbf90 msgr2=0x7f14ac0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 -- 172.21.15.122:0/2704179068 shutdown_connections 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.816+0000 7f14b3f96640 1 -- 172.21.15.122:0/2704179068 wait complete. 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 Processor -- start 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 -- start start 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac100a00 0x7f14ac197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14ac1025b0 0x7f14ac19f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10d7a0 con 0x7f14ac101bf0 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10d620 con 0x7f14ac100a00 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b3f96640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14ac10d920 con 0x7f14ac1025b0 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33714/0 (socket says 172.21.15.122:33714) 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 -- 172.21.15.122:0/2855232134 learned_addr learned my addr 172.21.15.122:0/2855232134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1d32640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac100a00 0x7f14ac197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 -- 172.21.15.122:0/2855232134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14ac1025b0 msgr2=0x7f14ac19f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14ac1025b0 0x7f14ac19f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 -- 172.21.15.122:0/2855232134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac100a00 msgr2=0x7f14ac197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac100a00 0x7f14ac197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.817+0000 7f14b1531640 1 -- 172.21.15.122:0/2855232134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14ac19fa80 con 0x7f14ac101bf0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14b1531640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f149c00d950 tx=0x7f149c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c017030 con 0x7f14ac101bf0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14ac19fd70 con 0x7f14ac101bf0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c004020 con 0x7f14ac101bf0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c0047a0 con 0x7f14ac101bf0 2024-04-02T13:33:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.818+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14ac1a0230 con 0x7f14ac101bf0 2024-04-02T13:33:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.819+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14ac107dc0 con 0x7f14ac101bf0 2024-04-02T13:33:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.820+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f149c0041d0 con 0x7f14ac101bf0 2024-04-02T13:33:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.820+0000 7f14a2ffd640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 0x7f14980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.820+0000 7f14b1d32640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 0x7f14980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.820+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f149c0d00b0 con 0x7f14ac101bf0 2024-04-02T13:33:11.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.821+0000 7f14b1d32640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 0x7f14980a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000c00 tx=0x7f14a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:11.825+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f149c09c850 con 0x7f14ac101bf0 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: 2024-04-02T13:33:11.527+0000 7fb1c4b90700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:active 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:33:11.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:active 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: Cluster is now healthy 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-02T13:33:11.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:11 smithi187 ceph-mon[36064]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-02T13:33:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-02T13:33:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:33:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:active 2024-04-02T13:33:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:33:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:active 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: Cluster is now healthy 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-02T13:33:12.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:11 smithi188 ceph-mon[36117]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-02T13:33:12.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.030+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f14ac061f40 con 0x7f14980a3bb0 2024-04-02T13:33:12.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.054+0000 7f14a2ffd640 1 -- 172.21.15.122:0/2855232134 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14ac061f40 con 0x7f14980a3bb0 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 msgr2=0x7f14980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 0x7f14980a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f14a8000c00 tx=0x7f14a8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 msgr2=0x7f14ac1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f149c00d950 tx=0x7f149c00de10 comp rx=0 tx=0).stop 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 shutdown_connections 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14ac1025b0 0x7f14ac19f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14ac101bf0 0x7f14ac1983a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f14980a3bb0 0x7f14980a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 --2- 172.21.15.122:0/2855232134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14ac100a00 0x7f14ac197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 >> 172.21.15.122:0/2855232134 conn(0x7f14ac0fbf90 msgr2=0x7f14ac104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 shutdown_connections 2024-04-02T13:33:12.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.056+0000 7f14b3f96640 1 -- 172.21.15.122:0/2855232134 wait complete. 2024-04-02T13:33:12.069 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/541283902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] up:active 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] up:active 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: daemon mds.a assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: Cluster is now healthy 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 4 up:standby-replay 4 up:standby 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-04-02T13:33:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:11 smithi122 ceph-mon[32114]: daemon mds.a is now active in filesystem cephfs as rank 4 2024-04-02T13:33:12.071 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- 172.21.15.122:0/3291535139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2688104f10 msgr2=0x7f2688107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 --2- 172.21.15.122:0/3291535139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2688104f10 0x7f2688107300 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2678009900 tx=0x7f267802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- 172.21.15.122:0/3291535139 shutdown_connections 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 --2- 172.21.15.122:0/3291535139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688107840 0x7f2688109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 --2- 172.21.15.122:0/3291535139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2688104f10 0x7f2688107300 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 --2- 172.21.15.122:0/3291535139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26881025e0 0x7f26881049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- 172.21.15.122:0/3291535139 >> 172.21.15.122:0/3291535139 conn(0x7f26880fbf90 msgr2=0x7f26880fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- 172.21.15.122:0/3291535139 shutdown_connections 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- 172.21.15.122:0/3291535139 wait complete. 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 Processor -- start 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.179+0000 7f268f9e4640 1 -- start start 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 0x7f26881983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 0x7f268819f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f268810d780 con 0x7f26881025e0 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f268810d600 con 0x7f2688107840 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f268810d900 con 0x7f2688104f10 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33734/0 (socket says 172.21.15.122:33734) 2024-04-02T13:33:12.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 -- 172.21.15.122:0/1951439729 learned_addr learned my addr 172.21.15.122:0/1951439729 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268df81640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 0x7f268819f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268cf7f640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 0x7f26881983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 -- 172.21.15.122:0/1951439729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 msgr2=0x7f26881983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 0x7f26881983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 -- 172.21.15.122:0/1951439729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 msgr2=0x7f268819f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 0x7f268819f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 -- 172.21.15.122:0/1951439729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268819fad0 con 0x7f26881025e0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268cf7f640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 0x7f26881983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268df81640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 0x7f268819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268d780640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f268400e9f0 tx=0x7f268400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268400cd80 con 0x7f26881025e0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2684010910 con 0x7f26881025e0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268819fdc0 con 0x7f26881025e0 2024-04-02T13:33:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2684018990 con 0x7f26881025e0 2024-04-02T13:33:12.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.180+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881a0230 con 0x7f26881025e0 2024-04-02T13:33:12.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.181+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26880fda80 con 0x7f26881025e0 2024-04-02T13:33:12.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.182+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f2684010ac0 con 0x7f26881025e0 2024-04-02T13:33:12.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.183+0000 7f267e7fc640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 0x7f26740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.183+0000 7f268cf7f640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 0x7f26740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.183+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f26840d0e80 con 0x7f26881025e0 2024-04-02T13:33:12.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.184+0000 7f268cf7f640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 0x7f26740a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2678006010 tx=0x7f2678000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.187+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f268409d620 con 0x7f26881025e0 2024-04-02T13:33:12.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.393+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f2688061f40 con 0x7f26740a3bb0 2024-04-02T13:33:12.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.398+0000 7f267e7fc640 1 -- 172.21.15.122:0/1951439729 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2688061f40 con 0x7f26740a3bb0 2024-04-02T13:33:12.401 INFO:teuthology.orchestra.run.smithi122.stdout:/volumes/qa/sv_1/7cc47f64-78ec-40c5-b8db-79497ae99943 2024-04-02T13:33:12.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 msgr2=0x7f26740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 0x7f26740a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2678006010 tx=0x7f2678000bc0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 msgr2=0x7f2688197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f268400e9f0 tx=0x7f268400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 shutdown_connections 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26740a3bb0 0x7f26740a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2688107840 0x7f268819f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2688104f10 0x7f26881983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 --2- 172.21.15.122:0/1951439729 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26881025e0 0x7f2688197eb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 >> 172.21.15.122:0/1951439729 conn(0x7f26880fbf90 msgr2=0x7f2688102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 shutdown_connections 2024-04-02T13:33:12.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.400+0000 7f268f9e4640 1 -- 172.21.15.122:0/1951439729 wait complete. 2024-04-02T13:33:12.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:33:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 -- 172.21.15.122:0/3479796956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 msgr2=0x7f7e9c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/3479796956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c100e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e94009900 tx=0x7f7e9402bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 -- 172.21.15.122:0/3479796956 shutdown_connections 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/3479796956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/3479796956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 0x7f7e9c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/3479796956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c100e40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 -- 172.21.15.122:0/3479796956 >> 172.21.15.122:0/3479796956 conn(0x7f7e9c0fbf90 msgr2=0x7f7e9c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 -- 172.21.15.122:0/3479796956 shutdown_connections 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.512+0000 7f7ea47eb640 1 -- 172.21.15.122:0/3479796956 wait complete. 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 Processor -- start 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 -- start start 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 0x7f7e9c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10c6f0 con 0x7f7e9c100a20 2024-04-02T13:33:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10c570 con 0x7f7e9c101c10 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea47eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9c10c870 con 0x7f7e9c1025d0 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33756/0 (socket says 172.21.15.122:33756) 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 -- 172.21.15.122:0/143523396 learned_addr learned my addr 172.21.15.122:0/143523396 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea1d86640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 0x7f7e9c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2d88640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 -- 172.21.15.122:0/143523396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 msgr2=0x7f7e9c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 -- 172.21.15.122:0/143523396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 msgr2=0x7f7e9c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 0x7f7e9c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2587640 1 -- 172.21.15.122:0/143523396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e9c19fb10 con 0x7f7e9c100a20 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.513+0000 7f7ea2d88640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7ea2587640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94041070 con 0x7f7e9c100a20 2024-04-02T13:33:12.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e940044d0 con 0x7f7e9c100a20 2024-04-02T13:33:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e9c19fd40 con 0x7f7e9c100a20 2024-04-02T13:33:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94034440 con 0x7f7e9c100a20 2024-04-02T13:33:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.514+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e9c1a01d0 con 0x7f7e9c100a20 2024-04-02T13:33:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.515+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9c107dc0 con 0x7f7e9c100a20 2024-04-02T13:33:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.516+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7f7e94044070 con 0x7f7e9c100a20 2024-04-02T13:33:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.517+0000 7f7e9b7fe640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 0x7f7e840a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.517+0000 7f7ea1d86640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 0x7f7e840a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.517+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7e940f21c0 con 0x7f7e9c100a20 2024-04-02T13:33:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.517+0000 7f7ea1d86640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 0x7f7e840a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c069540 tx=0x7f7e90005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.521+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7e940be960 con 0x7f7e9c100a20 2024-04-02T13:33:12.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.742+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e9c02d370 con 0x7f7e9c100a20 2024-04-02T13:33:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.742+0000 7f7e9b7fe640 1 -- 172.21.15.122:0/143523396 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f7e940be260 con 0x7f7e9c100a20 2024-04-02T13:33:12.746 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:12.746 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:33:12.746 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:33:12.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 msgr2=0x7f7e840a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 0x7f7e840a60f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c069540 tx=0x7f7e90005f90 comp rx=0 tx=0).stop 2024-04-02T13:33:12.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 msgr2=0x7f7e9c0697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7e94000c00 tx=0x7f7e94005d50 comp rx=0 tx=0).stop 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 shutdown_connections 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e9c1025d0 0x7f7e9c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7e840a3c30 0x7f7e840a60f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e9c101c10 0x7f7e9c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.744+0000 7f7ea47eb640 1 --2- 172.21.15.122:0/143523396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e9c100a20 0x7f7e9c0697d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.745+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 >> 172.21.15.122:0/143523396 conn(0x7f7e9c0fbf90 msgr2=0x7f7e9c104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.745+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 shutdown_connections 2024-04-02T13:33:12.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.745+0000 7f7ea47eb640 1 -- 172.21.15.122:0/143523396 wait complete. 2024-04-02T13:33:12.757 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:12 smithi122 ceph-mon[32114]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:12.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:12 smithi122 ceph-mon[32114]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:12.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:12 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:active 2024-04-02T13:33:12.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:12 smithi122 ceph-mon[32114]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T13:33:12.758 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:12 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:33:12.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- 172.21.15.122:0/1361789800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238101c30 msgr2=0x7fb2381020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/1361789800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238101c30 0x7fb2381020b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb228009900 tx=0x7fb22802bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- 172.21.15.122:0/1361789800 shutdown_connections 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/1361789800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb238108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/1361789800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238101c30 0x7fb2381020b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/1361789800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238100a40 0x7fb238100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- 172.21.15.122:0/1361789800 >> 172.21.15.122:0/1361789800 conn(0x7fb2380fbfb0 msgr2=0x7fb2380fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- 172.21.15.122:0/1361789800 shutdown_connections 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- 172.21.15.122:0/1361789800 wait complete. 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 Processor -- start 2024-04-02T13:33:12.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.857+0000 7fb23e9e2640 1 -- start start 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238101c30 0x7fb238198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb23819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810d840 con 0x7fb238100a40 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810d6c0 con 0x7fb238101c30 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23e9e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb23810d9c0 con 0x7fb2381025f0 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33788/0 (socket says 172.21.15.122:33788) 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 -- 172.21.15.122:0/171152774 learned_addr learned my addr 172.21.15.122:0/171152774 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb2377fe640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238101c30 0x7fb238198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23cf7f640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb23819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 -- 172.21.15.122:0/171152774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 msgr2=0x7fb23819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb23819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 -- 172.21.15.122:0/171152774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238101c30 msgr2=0x7fb238198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238101c30 0x7fb238198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 -- 172.21.15.122:0/171152774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23819fa00 con 0x7fb238100a40 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb23cf7f640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb23819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb237fff640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9c0 tx=0x7fb22c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.858+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00cd80 con 0x7fb238100a40 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.859+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb22c0035f0 con 0x7fb238100a40 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.859+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c010690 con 0x7fb238100a40 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.859+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23819fcf0 con 0x7fb238100a40 2024-04-02T13:33:12.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.859+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2381a01b0 con 0x7fb238100a40 2024-04-02T13:33:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.860+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb238107d00 con 0x7fb238100a40 2024-04-02T13:33:12.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.861+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138647+0+0 (secure 0 0 0) 0x7fb22c0037a0 con 0x7fb238100a40 2024-04-02T13:33:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.861+0000 7fb2357fa640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 0x7fb2200a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.861+0000 7fb2377fe640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 0x7fb2200a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.861+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb22c0d0a30 con 0x7fb238100a40 2024-04-02T13:33:12.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.862+0000 7fb2377fe640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 0x7fb2200a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb22802c240 tx=0x7fb22802df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:12.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:12.866+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb22c09d1d0 con 0x7fb238100a40 2024-04-02T13:33:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:12 smithi187 ceph-mon[36064]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:12 smithi187 ceph-mon[36064]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:12 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:active 2024-04-02T13:33:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:12 smithi187 ceph-mon[36064]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T13:33:12.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:12 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:33:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:12 smithi188 ceph-mon[36117]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:12 smithi188 ceph-mon[36117]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:12 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] up:active 2024-04-02T13:33:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:12 smithi188 ceph-mon[36117]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-04-02T13:33:13.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:12 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:33:13.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.045+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb238062800 con 0x7fb238100a40 2024-04-02T13:33:13.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.046+0000 7fb2357fa640 1 -- 172.21.15.122:0/171152774 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7fb22c09cad0 con 0x7fb238100a40 2024-04-02T13:33:13.050 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:13.050 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:13.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.048+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 msgr2=0x7fb2200a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 0x7fb2200a60f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb22802c240 tx=0x7fb22802df90 comp rx=0 tx=0).stop 2024-04-02T13:33:13.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 msgr2=0x7fb238197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00e9c0 tx=0x7fb22c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 shutdown_connections 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2381025f0 0x7fb23819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb2200a3c30 0x7fb2200a60f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb238101c30 0x7fb238198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 --2- 172.21.15.122:0/171152774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb238100a40 0x7fb238197de0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 >> 172.21.15.122:0/171152774 conn(0x7fb2380fbfb0 msgr2=0x7fb238104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 shutdown_connections 2024-04-02T13:33:13.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.049+0000 7fb23e9e2640 1 -- 172.21.15.122:0/171152774 wait complete. 2024-04-02T13:33:13.064 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-02T13:33:13.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 -- 172.21.15.122:0/2150385800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 msgr2=0x7fec0c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 --2- 172.21.15.122:0/2150385800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1031b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7febfc009900 tx=0x7febfc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 -- 172.21.15.122:0/2150385800 shutdown_connections 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 --2- 172.21.15.122:0/2150385800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c1036f0 0x7fec0c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 --2- 172.21.15.122:0/2150385800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1031b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 --2- 172.21.15.122:0/2150385800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c101b40 0x7fec0c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.162+0000 7fec139a8640 1 -- 172.21.15.122:0/2150385800 >> 172.21.15.122:0/2150385800 conn(0x7fec0c0fd0b0 msgr2=0x7fec0c0ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- 172.21.15.122:0/2150385800 shutdown_connections 2024-04-02T13:33:13.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- 172.21.15.122:0/2150385800 wait complete. 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 Processor -- start 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- start start 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c101b40 0x7fec0c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 0x7fec0c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10d7c0 con 0x7fec0c102d30 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10d640 con 0x7fec0c101b40 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.163+0000 7fec139a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10d940 con 0x7fec0c1036f0 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33806/0 (socket says 172.21.15.122:33806) 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 -- 172.21.15.122:0/4264490891 learned_addr learned my addr 172.21.15.122:0/4264490891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec11744640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c101b40 0x7fec0c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec11f45640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 0x7fec0c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 -- 172.21.15.122:0/4264490891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 msgr2=0x7fec0c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 0x7fec0c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 -- 172.21.15.122:0/4264490891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c101b40 msgr2=0x7fec0c197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c101b40 0x7fec0c197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 -- 172.21.15.122:0/4264490891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c19fab0 con 0x7fec0c102d30 2024-04-02T13:33:13.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec11f45640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 0x7fec0c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:13.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.164+0000 7fec10f43640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febfc009230 tx=0x7febfc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:13.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.173+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc041070 con 0x7fec0c102d30 2024-04-02T13:33:13.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.173+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febfc004450 con 0x7fec0c102d30 2024-04-02T13:33:13.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.173+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc003e20 con 0x7fec0c102d30 2024-04-02T13:33:13.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.173+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c19fd40 con 0x7fec0c102d30 2024-04-02T13:33:13.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.173+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c1a0220 con 0x7fec0c102d30 2024-04-02T13:33:13.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.174+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c108f00 con 0x7fec0c102d30 2024-04-02T13:33:13.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.175+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7febfc044070 con 0x7fec0c102d30 2024-04-02T13:33:13.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.175+0000 7fec027fc640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 0x7febf80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.175+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7febfc0f2210 con 0x7fec0c102d30 2024-04-02T13:33:13.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.176+0000 7fec11744640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 0x7febf80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.176+0000 7fec11744640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 0x7febf80a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec0c102b90 tx=0x7fec08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:13.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.180+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7febfc0be960 con 0x7fec0c102d30 2024-04-02T13:33:13.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.356+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec0c061f40 con 0x7febf80a3cc0 2024-04-02T13:33:13.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.366+0000 7fec027fc640 1 -- 172.21.15.122:0/4264490891 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340286 (secure 0 0 0) 0x7fec0c061f40 con 0x7febf80a3cc0 2024-04-02T13:33:13.374 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:13.376 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 msgr2=0x7febf80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 0x7febf80a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fec0c102b90 tx=0x7fec08005f90 comp rx=0 tx=0).stop 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 msgr2=0x7fec0c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7febfc009230 tx=0x7febfc02c650 comp rx=0 tx=0).stop 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 shutdown_connections 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec0c1036f0 0x7fec0c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec0c102d30 0x7fec0c1983d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7febf80a3cc0 0x7febf80a6180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 --2- 172.21.15.122:0/4264490891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec0c101b40 0x7fec0c197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.369+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 >> 172.21.15.122:0/4264490891 conn(0x7fec0c0fd0b0 msgr2=0x7fec0c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:13.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.370+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 shutdown_connections 2024-04-02T13:33:13.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.370+0000 7fec139a8640 1 -- 172.21.15.122:0/4264490891 wait complete. 2024-04-02T13:33:13.384 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-02T13:33:12.460240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596049,"num_objects":52,"num_object_clones":0,"num_object_copies":157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":292,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371128,"kb_used_data":13600,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570696,"statfs":{"total":1151940427776,"available":1151560392704,"internally_reserved":0,"allocated":13926400,"data_stored":7590761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3271,"num_objects":28,"num_object_clones":0,"num_object_copies":85,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003201"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619941+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:32:55.619941+0000","last_peered":"2024-04-02T13:32:55.619941+0000","last_clean":"2024-04-02T13:32:55.619941+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:32:55.619941+0000","last_undegraded":"2024-04-02T13:32:55.619941+0000","last_fullsized":"2024-04-02T13:32:55.619941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646880+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:32:56.646880+0000","last_peered":"2024-04-02T13:32:56.646880+0000","last_clean":"2024-04-02T13:32:56.646880+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:32:56.646880+0000","last_undegraded":"2024-04-02T13:32:56.646880+0000","last_fullsized":"2024-04-02T13:32:56.646880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646719+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:32:56.646719+0000","last_peered":"2024-04-02T13:32:56.646719+0000","last_clean":"2024-04-02T13:32:56.646719+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:32:56.646719+0000","last_undegraded":"2024-04-02T13:32:56.646719+0000","last_fullsized":"2024-04-02T13:32:56.646719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651875+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:32:56.651875+0000","last_peered":"2024-04-02T13:32:56.651875+0000","last_clean":"2024-04-02T13:32:56.651875+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:32:56.651875+0000","last_undegraded":"2024-04-02T13:32:56.651875+0000","last_fullsized":"2024-04-02T13:32:56.651875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649801+0000","last_change":"2024-04-02T13:32:50.536139+0000","last_active":"2024-04-02T13:32:56.649801+0000","last_peered":"2024-04-02T13:32:56.649801+0000","last_clean":"2024-04-02T13:32:56.649801+0000","last_became_active":"2024-04-02T13:32:50.535691+0000","last_became_peered":"2024-04-02T13:32:50.535691+0000","last_unstale":"2024-04-02T13:32:56.649801+0000","last_undegraded":"2024-04-02T13:32:56.649801+0000","last_fullsized":"2024-04-02T13:32:56.649801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:54:18.031530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650412+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:32:56.650412+0000","last_peered":"2024-04-02T13:32:56.650412+0000","last_clean":"2024-04-02T13:32:56.650412+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:32:56.650412+0000","last_undegraded":"2024-04-02T13:32:56.650412+0000","last_fullsized":"2024-04-02T13:32:56.650412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550060+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:32:55.550060+0000","last_peered":"2024-04-02T13:32:55.550060+0000","last_clean":"2024-04-02T13:32:55.550060+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:32:55.550060+0000","last_undegraded":"2024-04-02T13:32:55.550060+0000","last_fullsized":"2024-04-02T13:32:55.550060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649687+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:32:56.649687+0000","last_peered":"2024-04-02T13:32:56.649687+0000","last_clean":"2024-04-02T13:32:56.649687+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:32:56.649687+0000","last_undegraded":"2024-04-02T13:32:56.649687+0000","last_fullsized":"2024-04-02T13:32:56.649687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651830+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:32:56.651830+0000","last_peered":"2024-04-02T13:32:56.651830+0000","last_clean":"2024-04-02T13:32:56.651830+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:32:56.651830+0000","last_undegraded":"2024-04-02T13:32:56.651830+0000","last_fullsized":"2024-04-02T13:32:56.651830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651257+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:32:56.651257+0000","last_peered":"2024-04-02T13:32:56.651257+0000","last_clean":"2024-04-02T13:32:56.651257+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:32:56.651257+0000","last_undegraded":"2024-04-02T13:32:56.651257+0000","last_fullsized":"2024-04-02T13:32:56.651257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550604+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:32:55.550604+0000","last_peered":"2024-04-02T13:32:55.550604+0000","last_clean":"2024-04-02T13:32:55.550604+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:32:55.550604+0000","last_undegraded":"2024-04-02T13:32:55.550604+0000","last_fullsized":"2024-04-02T13:32:55.550604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651801+0000","last_change":"2024-04-02T13:32:50.538066+0000","last_active":"2024-04-02T13:32:56.651801+0000","last_peered":"2024-04-02T13:32:56.651801+0000","last_clean":"2024-04-02T13:32:56.651801+0000","last_became_active":"2024-04-02T13:32:50.537761+0000","last_became_peered":"2024-04-02T13:32:50.537761+0000","last_unstale":"2024-04-02T13:32:56.651801+0000","last_undegraded":"2024-04-02T13:32:56.651801+0000","last_fullsized":"2024-04-02T13:32:56.651801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:43.535184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551249+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:32:55.551249+0000","last_peered":"2024-04-02T13:32:55.551249+0000","last_clean":"2024-04-02T13:32:55.551249+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:32:55.551249+0000","last_undegraded":"2024-04-02T13:32:55.551249+0000","last_fullsized":"2024-04-02T13:32:55.551249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646482+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:32:56.646482+0000","last_peered":"2024-04-02T13:32:56.646482+0000","last_clean":"2024-04-02T13:32:56.646482+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:32:56.646482+0000","last_undegraded":"2024-04-02T13:32:56.646482+0000","last_fullsized":"2024-04-02T13:32:56.646482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646574+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:32:56.646574+0000","last_peered":"2024-04-02T13:32:56.646574+0000","last_clean":"2024-04-02T13:32:56.646574+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:32:56.646574+0000","last_undegraded":"2024-04-02T13:32:56.646574+0000","last_fullsized":"2024-04-02T13:32:56.646574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650205+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:32:56.650205+0000","last_peered":"2024-04-02T13:32:56.650205+0000","last_clean":"2024-04-02T13:32:56.650205+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:32:56.650205+0000","last_undegraded":"2024-04-02T13:32:56.650205+0000","last_fullsized":"2024-04-02T13:32:56.650205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619937+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:32:55.619937+0000","last_peered":"2024-04-02T13:32:55.619937+0000","last_clean":"2024-04-02T13:32:55.619937+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:32:55.619937+0000","last_undegraded":"2024-04-02T13:32:55.619937+0000","last_fullsized":"2024-04-02T13:32:55.619937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687978+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:32:55.687978+0000","last_peered":"2024-04-02T13:32:55.687978+0000","last_clean":"2024-04-02T13:32:55.687978+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:32:55.687978+0000","last_undegraded":"2024-04-02T13:32:55.687978+0000","last_fullsized":"2024-04-02T13:32:55.687978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149762+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:32:56.149762+0000","last_peered":"2024-04-02T13:32:56.149762+0000","last_clean":"2024-04-02T13:32:56.149762+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:32:56.149762+0000","last_undegraded":"2024-04-02T13:32:56.149762+0000","last_fullsized":"2024-04-02T13:32:56.149762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149728+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:32:56.149728+0000","last_peered":"2024-04-02T13:32:56.149728+0000","last_clean":"2024-04-02T13:32:56.149728+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:32:56.149728+0000","last_undegraded":"2024-04-02T13:32:56.149728+0000","last_fullsized":"2024-04-02T13:32:56.149728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688011+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:32:55.688011+0000","last_peered":"2024-04-02T13:32:55.688011+0000","last_clean":"2024-04-02T13:32:55.688011+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:32:55.688011+0000","last_undegraded":"2024-04-02T13:32:55.688011+0000","last_fullsized":"2024-04-02T13:32:55.688011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650194+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:32:56.650194+0000","last_peered":"2024-04-02T13:32:56.650194+0000","last_clean":"2024-04-02T13:32:56.650194+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:32:56.650194+0000","last_undegraded":"2024-04-02T13:32:56.650194+0000","last_fullsized":"2024-04-02T13:32:56.650194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650754+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:32:56.650754+0000","last_peered":"2024-04-02T13:32:56.650754+0000","last_clean":"2024-04-02T13:32:56.650754+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:32:56.650754+0000","last_undegraded":"2024-04-02T13:32:56.650754+0000","last_fullsized":"2024-04-02T13:32:56.650754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650380+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:32:56.650380+0000","last_peered":"2024-04-02T13:32:56.650380+0000","last_clean":"2024-04-02T13:32:56.650380+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:32:56.650380+0000","last_undegraded":"2024-04-02T13:32:56.650380+0000","last_fullsized":"2024-04-02T13:32:56.650380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646813+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:32:56.646813+0000","last_peered":"2024-04-02T13:32:56.646813+0000","last_clean":"2024-04-02T13:32:56.646813+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:32:56.646813+0000","last_undegraded":"2024-04-02T13:32:56.646813+0000","last_fullsized":"2024-04-02T13:32:56.646813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651025+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:32:56.651025+0000","last_peered":"2024-04-02T13:32:56.651025+0000","last_clean":"2024-04-02T13:32:56.651025+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:32:56.651025+0000","last_undegraded":"2024-04-02T13:32:56.651025+0000","last_fullsized":"2024-04-02T13:32:56.651025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.643007+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:33:10.643007+0000","last_peered":"2024-04-02T13:33:10.643007+0000","last_clean":"2024-04-02T13:33:10.643007+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:33:10.643007+0000","last_undegraded":"2024-04-02T13:33:10.643007+0000","last_fullsized":"2024-04-02T13:33:10.643007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651074+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:32:56.651074+0000","last_peered":"2024-04-02T13:32:56.651074+0000","last_clean":"2024-04-02T13:32:56.651074+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:32:56.651074+0000","last_undegraded":"2024-04-02T13:32:56.651074+0000","last_fullsized":"2024-04-02T13:32:56.651074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650834+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:32:56.650834+0000","last_peered":"2024-04-02T13:32:56.650834+0000","last_clean":"2024-04-02T13:32:56.650834+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:32:56.650834+0000","last_undegraded":"2024-04-02T13:32:56.650834+0000","last_fullsized":"2024-04-02T13:32:56.650834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649937+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:32:56.649937+0000","last_peered":"2024-04-02T13:32:56.649937+0000","last_clean":"2024-04-02T13:32:56.649937+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:32:56.649937+0000","last_undegraded":"2024-04-02T13:32:56.649937+0000","last_fullsized":"2024-04-02T13:32:56.649937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550382+0000","last_change":"2024-04-02T13:32:50.536179+0000","last_active":"2024-04-02T13:32:55.550382+0000","last_peered":"2024-04-02T13:32:55.550382+0000","last_clean":"2024-04-02T13:32:55.550382+0000","last_became_active":"2024-04-02T13:32:50.535848+0000","last_became_peered":"2024-04-02T13:32:50.535848+0000","last_unstale":"2024-04-02T13:32:55.550382+0000","last_undegraded":"2024-04-02T13:32:55.550382+0000","last_fullsized":"2024-04-02T13:32:55.550382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:08.589683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149699+0000","last_change":"2024-04-02T13:32:54.560990+0000","last_active":"2024-04-02T13:32:56.149699+0000","last_peered":"2024-04-02T13:32:56.149699+0000","last_clean":"2024-04-02T13:32:56.149699+0000","last_became_active":"2024-04-02T13:32:54.560528+0000","last_became_peered":"2024-04-02T13:32:54.560528+0000","last_unstale":"2024-04-02T13:32:56.149699+0000","last_undegraded":"2024-04-02T13:32:56.149699+0000","last_fullsized":"2024-04-02T13:32:56.149699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:41:41.675349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550098+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:32:55.550098+0000","last_peered":"2024-04-02T13:32:55.550098+0000","last_clean":"2024-04-02T13:32:55.550098+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:32:55.550098+0000","last_undegraded":"2024-04-02T13:32:55.550098+0000","last_fullsized":"2024-04-02T13:32:55.550098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649929+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:32:56.649929+0000","last_peered":"2024-04-02T13:32:56.649929+0000","last_clean":"2024-04-02T13:32:56.649929+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:32:56.649929+0000","last_undegraded":"2024-04-02T13:32:56.649929+0000","last_fullsized":"2024-04-02T13:32:56.649929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149448+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:32:56.149448+0000","last_peered":"2024-04-02T13:32:56.149448+0000","last_clean":"2024-04-02T13:32:56.149448+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:32:56.149448+0000","last_undegraded":"2024-04-02T13:32:56.149448+0000","last_fullsized":"2024-04-02T13:32:56.149448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650298+0000","last_change":"2024-04-02T13:32:49.523267+0000","last_active":"2024-04-02T13:32:56.650298+0000","last_peered":"2024-04-02T13:32:56.650298+0000","last_clean":"2024-04-02T13:32:56.650298+0000","last_became_active":"2024-04-02T13:32:49.522953+0000","last_became_peered":"2024-04-02T13:32:49.522953+0000","last_unstale":"2024-04-02T13:32:56.650298+0000","last_undegraded":"2024-04-02T13:32:56.650298+0000","last_fullsized":"2024-04-02T13:32:56.650298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:03:10.578642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550271+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:32:55.550271+0000","last_peered":"2024-04-02T13:32:55.550271+0000","last_clean":"2024-04-02T13:32:55.550271+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:32:55.550271+0000","last_undegraded":"2024-04-02T13:32:55.550271+0000","last_fullsized":"2024-04-02T13:32:55.550271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619847+0000","last_change":"2024-04-02T13:32:54.568499+0000","last_active":"2024-04-02T13:32:55.619847+0000","last_peered":"2024-04-02T13:32:55.619847+0000","last_clean":"2024-04-02T13:32:55.619847+0000","last_became_active":"2024-04-02T13:32:54.568140+0000","last_became_peered":"2024-04-02T13:32:54.568140+0000","last_unstale":"2024-04-02T13:32:55.619847+0000","last_undegraded":"2024-04-02T13:32:55.619847+0000","last_fullsized":"2024-04-02T13:32:55.619847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:03.120174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:09.635952+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:33:09.635952+0000","last_peered":"2024-04-02T13:33:09.635952+0000","last_clean":"2024-04-02T13:33:09.635952+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:33:09.635952+0000","last_undegraded":"2024-04-02T13:33:09.635952+0000","last_fullsized":"2024-04-02T13:33:09.635952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650058+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:32:56.650058+0000","last_peered":"2024-04-02T13:32:56.650058+0000","last_clean":"2024-04-02T13:32:56.650058+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:32:56.650058+0000","last_undegraded":"2024-04-02T13:32:56.650058+0000","last_fullsized":"2024-04-02T13:32:56.650058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646925+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:32:56.646925+0000","last_peered":"2024-04-02T13:32:56.646925+0000","last_clean":"2024-04-02T13:32:56.646925+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:32:56.646925+0000","last_undegraded":"2024-04-02T13:32:56.646925+0000","last_fullsized":"2024-04-02T13:32:56.646925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549717+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:32:55.549717+0000","last_peered":"2024-04-02T13:32:55.549717+0000","last_clean":"2024-04-02T13:32:55.549717+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:32:55.549717+0000","last_undegraded":"2024-04-02T13:32:55.549717+0000","last_fullsized":"2024-04-02T13:32:55.549717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687520+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:32:55.687520+0000","last_peered":"2024-04-02T13:32:55.687520+0000","last_clean":"2024-04-02T13:32:55.687520+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:32:55.687520+0000","last_undegraded":"2024-04-02T13:32:55.687520+0000","last_fullsized":"2024-04-02T13:32:55.687520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649884+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:32:56.649884+0000","last_peered":"2024-04-02T13:32:56.649884+0000","last_clean":"2024-04-02T13:32:56.649884+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:32:56.649884+0000","last_undegraded":"2024-04-02T13:32:56.649884+0000","last_fullsized":"2024-04-02T13:32:56.649884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625861+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:33:08.625861+0000","last_peered":"2024-04-02T13:33:08.625861+0000","last_clean":"2024-04-02T13:33:08.625861+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:33:08.625861+0000","last_undegraded":"2024-04-02T13:33:08.625861+0000","last_fullsized":"2024-04-02T13:33:08.625861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651763+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:32:56.651763+0000","last_peered":"2024-04-02T13:32:56.651763+0000","last_clean":"2024-04-02T13:32:56.651763+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:32:56.651763+0000","last_undegraded":"2024-04-02T13:32:56.651763+0000","last_fullsized":"2024-04-02T13:32:56.651763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687671+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:32:55.687671+0000","last_peered":"2024-04-02T13:32:55.687671+0000","last_clean":"2024-04-02T13:32:55.687671+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:32:55.687671+0000","last_undegraded":"2024-04-02T13:32:55.687671+0000","last_fullsized":"2024-04-02T13:32:55.687671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.157890+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:33:11.157890+0000","last_peered":"2024-04-02T13:33:11.157890+0000","last_clean":"2024-04-02T13:33:11.157890+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:33:11.157890+0000","last_undegraded":"2024-04-02T13:33:11.157890+0000","last_fullsized":"2024-04-02T13:33:11.157890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646955+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:32:56.646955+0000","last_peered":"2024-04-02T13:32:56.646955+0000","last_clean":"2024-04-02T13:32:56.646955+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:32:56.646955+0000","last_undegraded":"2024-04-02T13:32:56.646955+0000","last_fullsized":"2024-04-02T13:32:56.646955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651536+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:32:56.651536+0000","last_peered":"2024-04-02T13:32:56.651536+0000","last_clean":"2024-04-02T13:32:56.651536+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:32:56.651536+0000","last_undegraded":"2024-04-02T13:32:56.651536+0000","last_fullsized":"2024-04-02T13:32:56.651536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550320+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:32:55.550320+0000","last_peered":"2024-04-02T13:32:55.550320+0000","last_clean":"2024-04-02T13:32:55.550320+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:32:55.550320+0000","last_undegraded":"2024-04-02T13:32:55.550320+0000","last_fullsized":"2024-04-02T13:32:55.550320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651257+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:32:56.651257+0000","last_peered":"2024-04-02T13:32:56.651257+0000","last_clean":"2024-04-02T13:32:56.651257+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:32:56.651257+0000","last_undegraded":"2024-04-02T13:32:56.651257+0000","last_fullsized":"2024-04-02T13:32:56.651257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687694+0000","last_change":"2024-04-02T13:32:54.562030+0000","last_active":"2024-04-02T13:32:55.687694+0000","last_peered":"2024-04-02T13:32:55.687694+0000","last_clean":"2024-04-02T13:32:55.687694+0000","last_became_active":"2024-04-02T13:32:54.560872+0000","last_became_peered":"2024-04-02T13:32:54.560872+0000","last_unstale":"2024-04-02T13:32:55.687694+0000","last_undegraded":"2024-04-02T13:32:55.687694+0000","last_fullsized":"2024-04-02T13:32:55.687694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:25:08.994729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625616+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:33:08.625616+0000","last_peered":"2024-04-02T13:33:08.625616+0000","last_clean":"2024-04-02T13:33:08.625616+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:33:08.625616+0000","last_undegraded":"2024-04-02T13:33:08.625616+0000","last_fullsized":"2024-04-02T13:33:08.625616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650098+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:32:56.650098+0000","last_peered":"2024-04-02T13:32:56.650098+0000","last_clean":"2024-04-02T13:32:56.650098+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:32:56.650098+0000","last_undegraded":"2024-04-02T13:32:56.650098+0000","last_fullsized":"2024-04-02T13:32:56.650098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550797+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:32:55.550797+0000","last_peered":"2024-04-02T13:32:55.550797+0000","last_clean":"2024-04-02T13:32:55.550797+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:32:55.550797+0000","last_undegraded":"2024-04-02T13:32:55.550797+0000","last_fullsized":"2024-04-02T13:32:55.550797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625213+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:33:08.625213+0000","last_peered":"2024-04-02T13:33:08.625213+0000","last_clean":"2024-04-02T13:33:08.625213+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:33:08.625213+0000","last_undegraded":"2024-04-02T13:33:08.625213+0000","last_fullsized":"2024-04-02T13:33:08.625213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551137+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:32:55.551137+0000","last_peered":"2024-04-02T13:32:55.551137+0000","last_clean":"2024-04-02T13:32:55.551137+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:32:55.551137+0000","last_undegraded":"2024-04-02T13:32:55.551137+0000","last_fullsized":"2024-04-02T13:32:55.551137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619613+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:32:55.619613+0000","last_peered":"2024-04-02T13:32:55.619613+0000","last_clean":"2024-04-02T13:32:55.619613+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:32:55.619613+0000","last_undegraded":"2024-04-02T13:32:55.619613+0000","last_fullsized":"2024-04-02T13:32:55.619613+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550167+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:32:55.550167+0000","last_peered":"2024-04-02T13:32:55.550167+0000","last_clean":"2024-04-02T13:32:55.550167+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:32:55.550167+0000","last_undegraded":"2024-04-02T13:32:55.550167+0000","last_fullsized":"2024-04-02T13:32:55.550167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550299+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:32:55.550299+0000","last_peered":"2024-04-02T13:32:55.550299+0000","last_clean":"2024-04-02T13:32:55.550299+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:32:55.550299+0000","last_undegraded":"2024-04-02T13:32:55.550299+0000","last_fullsized":"2024-04-02T13:32:55.550299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687956+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:32:55.687956+0000","last_peered":"2024-04-02T13:32:55.687956+0000","last_clean":"2024-04-02T13:32:55.687956+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:32:55.687956+0000","last_undegraded":"2024-04-02T13:32:55.687956+0000","last_fullsized":"2024-04-02T13:32:55.687956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650800+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:32:56.650800+0000","last_peered":"2024-04-02T13:32:56.650800+0000","last_clean":"2024-04-02T13:32:56.650800+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:32:56.650800+0000","last_undegraded":"2024-04-02T13:32:56.650800+0000","last_fullsized":"2024-04-02T13:32:56.650800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550126+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:32:55.550126+0000","last_peered":"2024-04-02T13:32:55.550126+0000","last_clean":"2024-04-02T13:32:55.550126+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:32:55.550126+0000","last_undegraded":"2024-04-02T13:32:55.550126+0000","last_fullsized":"2024-04-02T13:32:55.550126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551712+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:32:55.551712+0000","last_peered":"2024-04-02T13:32:55.551712+0000","last_clean":"2024-04-02T13:32:55.551712+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:32:55.551712+0000","last_undegraded":"2024-04-02T13:32:55.551712+0000","last_fullsized":"2024-04-02T13:32:55.551712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'16","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.641914+0000","last_change":"2024-04-02T13:32:49.525732+0000","last_active":"2024-04-02T13:33:10.641914+0000","last_peered":"2024-04-02T13:33:10.641914+0000","last_clean":"2024-04-02T13:33:10.641914+0000","last_became_active":"2024-04-02T13:32:49.525404+0000","last_became_peered":"2024-04-02T13:32:49.525404+0000","last_unstale":"2024-04-02T13:33:10.641914+0000","last_undegraded":"2024-04-02T13:33:10.641914+0000","last_fullsized":"2024-04-02T13:33:10.641914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:21.317513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550163+0000","last_change":"2024-04-02T13:32:50.533696+0000","last_active":"2024-04-02T13:32:55.550163+0000","last_peered":"2024-04-02T13:32:55.550163+0000","last_clean":"2024-04-02T13:32:55.550163+0000","last_became_active":"2024-04-02T13:32:50.533369+0000","last_became_peered":"2024-04-02T13:32:50.533369+0000","last_unstale":"2024-04-02T13:32:55.550163+0000","last_undegraded":"2024-04-02T13:32:55.550163+0000","last_fullsized":"2024-04-02T13:32:55.550163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:29:09.847846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550010+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:32:55.550010+0000","last_peered":"2024-04-02T13:32:55.550010+0000","last_clean":"2024-04-02T13:32:55.550010+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:32:55.550010+0000","last_undegraded":"2024-04-02T13:32:55.550010+0000","last_fullsized":"2024-04-02T13:32:55.550010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.630205+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:33:10.630205+0000","last_peered":"2024-04-02T13:33:10.630205+0000","last_clean":"2024-04-02T13:33:10.630205+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:33:10.630205+0000","last_undegraded":"2024-04-02T13:33:10.630205+0000","last_fullsized":"2024-04-02T13:33:10.630205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619722+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:32:55.619722+0000","last_peered":"2024-04-02T13:32:55.619722+0000","last_clean":"2024-04-02T13:32:55.619722+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:32:55.619722+0000","last_undegraded":"2024-04-02T13:32:55.619722+0000","last_fullsized":"2024-04-02T13:32:55.619722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650001+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:32:56.650001+0000","last_peered":"2024-04-02T13:32:56.650001+0000","last_clean":"2024-04-02T13:32:56.650001+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:32:56.650001+0000","last_undegraded":"2024-04-02T13:32:56.650001+0000","last_fullsized":"2024-04-02T13:32:56.650001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687798+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:32:55.687798+0000","last_peered":"2024-04-02T13:32:55.687798+0000","last_clean":"2024-04-02T13:32:55.687798+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:32:55.687798+0000","last_undegraded":"2024-04-02T13:32:55.687798+0000","last_fullsized":"2024-04-02T13:32:55.687798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650888+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:32:56.650888+0000","last_peered":"2024-04-02T13:32:56.650888+0000","last_clean":"2024-04-02T13:32:56.650888+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:32:56.650888+0000","last_undegraded":"2024-04-02T13:32:56.650888+0000","last_fullsized":"2024-04-02T13:32:56.650888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551589+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:32:55.551589+0000","last_peered":"2024-04-02T13:32:55.551589+0000","last_clean":"2024-04-02T13:32:55.551589+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:32:55.551589+0000","last_undegraded":"2024-04-02T13:32:55.551589+0000","last_fullsized":"2024-04-02T13:32:55.551589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.158741+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:33:11.158741+0000","last_peered":"2024-04-02T13:33:11.158741+0000","last_clean":"2024-04-02T13:33:11.158741+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:33:11.158741+0000","last_undegraded":"2024-04-02T13:33:11.158741+0000","last_fullsized":"2024-04-02T13:33:11.158741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650443+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:32:56.650443+0000","last_peered":"2024-04-02T13:32:56.650443+0000","last_clean":"2024-04-02T13:32:56.650443+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:32:56.650443+0000","last_undegraded":"2024-04-02T13:32:56.650443+0000","last_fullsized":"2024-04-02T13:32:56.650443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551638+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:32:55.551638+0000","last_peered":"2024-04-02T13:32:55.551638+0000","last_clean":"2024-04-02T13:32:55.551638+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:32:55.551638+0000","last_undegraded":"2024-04-02T13:32:55.551638+0000","last_fullsized":"2024-04-02T13:32:55.551638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549973+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:32:55.549973+0000","last_peered":"2024-04-02T13:32:55.549973+0000","last_clean":"2024-04-02T13:32:55.549973+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:32:55.549973+0000","last_undegraded":"2024-04-02T13:32:55.549973+0000","last_fullsized":"2024-04-02T13:32:55.549973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.150626+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:33:11.150626+0000","last_peered":"2024-04-02T13:33:11.150626+0000","last_clean":"2024-04-02T13:33:11.150626+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:33:11.150626+0000","last_undegraded":"2024-04-02T13:33:11.150626+0000","last_fullsized":"2024-04-02T13:33:11.150626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.508447+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:32:56.508447+0000","last_peered":"2024-04-02T13:32:56.508447+0000","last_clean":"2024-04-02T13:32:56.508447+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:32:56.508447+0000","last_undegraded":"2024-04-02T13:32:56.508447+0000","last_fullsized":"2024-04-02T13:32:56.508447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649785+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:32:56.649785+0000","last_peered":"2024-04-02T13:32:56.649785+0000","last_clean":"2024-04-02T13:32:56.649785+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:32:56.649785+0000","last_undegraded":"2024-04-02T13:32:56.649785+0000","last_fullsized":"2024-04-02T13:32:56.649785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651245+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:32:56.651245+0000","last_peered":"2024-04-02T13:32:56.651245+0000","last_clean":"2024-04-02T13:32:56.651245+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:32:56.651245+0000","last_undegraded":"2024-04-02T13:32:56.651245+0000","last_fullsized":"2024-04-02T13:32:56.651245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549912+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:32:55.549912+0000","last_peered":"2024-04-02T13:32:55.549912+0000","last_clean":"2024-04-02T13:32:55.549912+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:32:55.549912+0000","last_undegraded":"2024-04-02T13:32:55.549912+0000","last_fullsized":"2024-04-02T13:32:55.549912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550583+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:32:55.550583+0000","last_peered":"2024-04-02T13:32:55.550583+0000","last_clean":"2024-04-02T13:32:55.550583+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:32:55.550583+0000","last_undegraded":"2024-04-02T13:32:55.550583+0000","last_fullsized":"2024-04-02T13:32:55.550583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550789+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:32:55.550789+0000","last_peered":"2024-04-02T13:32:55.550789+0000","last_clean":"2024-04-02T13:32:55.550789+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:32:55.550789+0000","last_undegraded":"2024-04-02T13:32:55.550789+0000","last_fullsized":"2024-04-02T13:32:55.550789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:09.634870+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:33:09.634870+0000","last_peered":"2024-04-02T13:33:09.634870+0000","last_clean":"2024-04-02T13:33:09.634870+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:33:09.634870+0000","last_undegraded":"2024-04-02T13:33:09.634870+0000","last_fullsized":"2024-04-02T13:33:09.634870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619566+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:32:55.619566+0000","last_peered":"2024-04-02T13:32:55.619566+0000","last_clean":"2024-04-02T13:32:55.619566+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:32:55.619566+0000","last_undegraded":"2024-04-02T13:32:55.619566+0000","last_fullsized":"2024-04-02T13:32:55.619566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.642732+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:33:10.642732+0000","last_peered":"2024-04-02T13:33:10.642732+0000","last_clean":"2024-04-02T13:33:10.642732+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:33:10.642732+0000","last_undegraded":"2024-04-02T13:33:10.642732+0000","last_fullsized":"2024-04-02T13:33:10.642732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551277+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:32:55.551277+0000","last_peered":"2024-04-02T13:32:55.551277+0000","last_clean":"2024-04-02T13:32:55.551277+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:32:55.551277+0000","last_undegraded":"2024-04-02T13:32:55.551277+0000","last_fullsized":"2024-04-02T13:32:55.551277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649850+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:32:56.649850+0000","last_peered":"2024-04-02T13:32:56.649850+0000","last_clean":"2024-04-02T13:32:56.649850+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:32:56.649850+0000","last_undegraded":"2024-04-02T13:32:56.649850+0000","last_fullsized":"2024-04-02T13:32:56.649850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688010+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:32:55.688010+0000","last_peered":"2024-04-02T13:32:55.688010+0000","last_clean":"2024-04-02T13:32:55.688010+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:32:55.688010+0000","last_undegraded":"2024-04-02T13:32:55.688010+0000","last_fullsized":"2024-04-02T13:32:55.688010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650315+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:32:56.650315+0000","last_peered":"2024-04-02T13:32:56.650315+0000","last_clean":"2024-04-02T13:32:56.650315+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:32:56.650315+0000","last_undegraded":"2024-04-02T13:32:56.650315+0000","last_fullsized":"2024-04-02T13:32:56.650315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149111+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:32:56.149111+0000","last_peered":"2024-04-02T13:32:56.149111+0000","last_clean":"2024-04-02T13:32:56.149111+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:32:56.149111+0000","last_undegraded":"2024-04-02T13:32:56.149111+0000","last_fullsized":"2024-04-02T13:32:56.149111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651954+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:32:56.651954+0000","last_peered":"2024-04-02T13:32:56.651954+0000","last_clean":"2024-04-02T13:32:56.651954+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:32:56.651954+0000","last_undegraded":"2024-04-02T13:32:56.651954+0000","last_fullsized":"2024-04-02T13:32:56.651954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646937+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:32:56.646937+0000","last_peered":"2024-04-02T13:32:56.646937+0000","last_clean":"2024-04-02T13:32:56.646937+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:32:56.646937+0000","last_undegraded":"2024-04-02T13:32:56.646937+0000","last_fullsized":"2024-04-02T13:32:56.646937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651822+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:32:56.651822+0000","last_peered":"2024-04-02T13:32:56.651822+0000","last_clean":"2024-04-02T13:32:56.651822+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:32:56.651822+0000","last_undegraded":"2024-04-02T13:32:56.651822+0000","last_fullsized":"2024-04-02T13:32:56.651822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.626322+0000","last_change":"2024-04-02T13:32:49.524826+0000","last_active":"2024-04-02T13:33:08.626322+0000","last_peered":"2024-04-02T13:33:08.626322+0000","last_clean":"2024-04-02T13:33:08.626322+0000","last_became_active":"2024-04-02T13:32:49.524516+0000","last_became_peered":"2024-04-02T13:32:49.524516+0000","last_unstale":"2024-04-02T13:33:08.626322+0000","last_undegraded":"2024-04-02T13:33:08.626322+0000","last_fullsized":"2024-04-02T13:33:08.626322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:07.176783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649676+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:32:56.649676+0000","last_peered":"2024-04-02T13:32:56.649676+0000","last_clean":"2024-04-02T13:32:56.649676+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:32:56.649676+0000","last_undegraded":"2024-04-02T13:32:56.649676+0000","last_fullsized":"2024-04-02T13:32:56.649676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688046+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:32:55.688046+0000","last_peered":"2024-04-02T13:32:55.688046+0000","last_clean":"2024-04-02T13:32:55.688046+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:32:55.688046+0000","last_undegraded":"2024-04-02T13:32:55.688046+0000","last_fullsized":"2024-04-02T13:32:55.688046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649773+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:32:56.649773+0000","last_peered":"2024-04-02T13:32:56.649773+0000","last_clean":"2024-04-02T13:32:56.649773+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:32:56.649773+0000","last_undegraded":"2024-04-02T13:32:56.649773+0000","last_fullsized":"2024-04-02T13:32:56.649773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619365+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:32:55.619365+0000","last_peered":"2024-04-02T13:32:55.619365+0000","last_clean":"2024-04-02T13:32:55.619365+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:32:55.619365+0000","last_undegraded":"2024-04-02T13:32:55.619365+0000","last_fullsized":"2024-04-02T13:32:55.619365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651802+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:32:56.651802+0000","last_peered":"2024-04-02T13:32:56.651802+0000","last_clean":"2024-04-02T13:32:56.651802+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:32:56.651802+0000","last_undegraded":"2024-04-02T13:32:56.651802+0000","last_fullsized":"2024-04-02T13:32:56.651802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149231+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:32:56.149231+0000","last_peered":"2024-04-02T13:32:56.149231+0000","last_clean":"2024-04-02T13:32:56.149231+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:32:56.149231+0000","last_undegraded":"2024-04-02T13:32:56.149231+0000","last_fullsized":"2024-04-02T13:32:56.149231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650027+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:32:56.650027+0000","last_peered":"2024-04-02T13:32:56.650027+0000","last_clean":"2024-04-02T13:32:56.650027+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:32:56.650027+0000","last_undegraded":"2024-04-02T13:32:56.650027+0000","last_fullsized":"2024-04-02T13:32:56.650027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651565+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:32:56.651565+0000","last_peered":"2024-04-02T13:32:56.651565+0000","last_clean":"2024-04-02T13:32:56.651565+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:32:56.651565+0000","last_undegraded":"2024-04-02T13:32:56.651565+0000","last_fullsized":"2024-04-02T13:32:56.651565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.626398+0000","last_change":"2024-04-02T13:32:49.528077+0000","last_active":"2024-04-02T13:33:08.626398+0000","last_peered":"2024-04-02T13:33:08.626398+0000","last_clean":"2024-04-02T13:33:08.626398+0000","last_became_active":"2024-04-02T13:32:49.527754+0000","last_became_peered":"2024-04-02T13:32:49.527754+0000","last_unstale":"2024-04-02T13:33:08.626398+0000","last_undegraded":"2024-04-02T13:33:08.626398+0000","last_fullsized":"2024-04-02T13:33:08.626398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:01.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550267+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:32:55.550267+0000","last_peered":"2024-04-02T13:32:55.550267+0000","last_clean":"2024-04-02T13:32:55.550267+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:32:55.550267+0000","last_undegraded":"2024-04-02T13:32:55.550267+0000","last_fullsized":"2024-04-02T13:32:55.550267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650437+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:32:56.650437+0000","last_peered":"2024-04-02T13:32:56.650437+0000","last_clean":"2024-04-02T13:32:56.650437+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:32:56.650437+0000","last_undegraded":"2024-04-02T13:32:56.650437+0000","last_fullsized":"2024-04-02T13:32:56.650437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149562+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:32:56.149562+0000","last_peered":"2024-04-02T13:32:56.149562+0000","last_clean":"2024-04-02T13:32:56.149562+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:32:56.149562+0000","last_undegraded":"2024-04-02T13:32:56.149562+0000","last_fullsized":"2024-04-02T13:32:56.149562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550610+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:32:55.550610+0000","last_peered":"2024-04-02T13:32:55.550610+0000","last_clean":"2024-04-02T13:32:55.550610+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:32:55.550610+0000","last_undegraded":"2024-04-02T13:32:55.550610+0000","last_fullsized":"2024-04-02T13:32:55.550610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646631+0000","last_change":"2024-04-02T13:32:54.566345+0000","last_active":"2024-04-02T13:32:56.646631+0000","last_peered":"2024-04-02T13:32:56.646631+0000","last_clean":"2024-04-02T13:32:56.646631+0000","last_became_active":"2024-04-02T13:32:54.564927+0000","last_became_peered":"2024-04-02T13:32:54.564927+0000","last_unstale":"2024-04-02T13:32:56.646631+0000","last_undegraded":"2024-04-02T13:32:56.646631+0000","last_fullsized":"2024-04-02T13:32:56.646631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:34.439915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550540+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:32:55.550540+0000","last_peered":"2024-04-02T13:32:55.550540+0000","last_clean":"2024-04-02T13:32:55.550540+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:32:55.550540+0000","last_undegraded":"2024-04-02T13:32:55.550540+0000","last_fullsized":"2024-04-02T13:32:55.550540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550127+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:32:55.550127+0000","last_peered":"2024-04-02T13:32:55.550127+0000","last_clean":"2024-04-02T13:32:55.550127+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:32:55.550127+0000","last_undegraded":"2024-04-02T13:32:55.550127+0000","last_fullsized":"2024-04-02T13:32:55.550127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549758+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:32:55.549758+0000","last_peered":"2024-04-02T13:32:55.549758+0000","last_clean":"2024-04-02T13:32:55.549758+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:32:55.549758+0000","last_undegraded":"2024-04-02T13:32:55.549758+0000","last_fullsized":"2024-04-02T13:32:55.549758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.159142+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:33:11.159142+0000","last_peered":"2024-04-02T13:33:11.159142+0000","last_clean":"2024-04-02T13:33:11.159142+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:33:11.159142+0000","last_undegraded":"2024-04-02T13:33:11.159142+0000","last_fullsized":"2024-04-02T13:33:11.159142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550471+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:32:55.550471+0000","last_peered":"2024-04-02T13:32:55.550471+0000","last_clean":"2024-04-02T13:32:55.550471+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:32:55.550471+0000","last_undegraded":"2024-04-02T13:32:55.550471+0000","last_fullsized":"2024-04-02T13:32:55.550471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619439+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:32:55.619439+0000","last_peered":"2024-04-02T13:32:55.619439+0000","last_clean":"2024-04-02T13:32:55.619439+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:32:55.619439+0000","last_undegraded":"2024-04-02T13:32:55.619439+0000","last_fullsized":"2024-04-02T13:32:55.619439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149733+0000","last_change":"2024-04-02T13:32:50.530731+0000","last_active":"2024-04-02T13:32:56.149733+0000","last_peered":"2024-04-02T13:32:56.149733+0000","last_clean":"2024-04-02T13:32:56.149733+0000","last_became_active":"2024-04-02T13:32:50.530286+0000","last_became_peered":"2024-04-02T13:32:50.530286+0000","last_unstale":"2024-04-02T13:32:56.149733+0000","last_undegraded":"2024-04-02T13:32:56.149733+0000","last_fullsized":"2024-04-02T13:32:56.149733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.111854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149143+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:32:56.149143+0000","last_peered":"2024-04-02T13:32:56.149143+0000","last_clean":"2024-04-02T13:32:56.149143+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:32:56.149143+0000","last_undegraded":"2024-04-02T13:32:56.149143+0000","last_fullsized":"2024-04-02T13:32:56.149143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646674+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:32:56.646674+0000","last_peered":"2024-04-02T13:32:56.646674+0000","last_clean":"2024-04-02T13:32:56.646674+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:32:56.646674+0000","last_undegraded":"2024-04-02T13:32:56.646674+0000","last_fullsized":"2024-04-02T13:32:56.646674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551187+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:32:55.551187+0000","last_peered":"2024-04-02T13:32:55.551187+0000","last_clean":"2024-04-02T13:32:55.551187+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:32:55.551187+0000","last_undegraded":"2024-04-02T13:32:55.551187+0000","last_fullsized":"2024-04-02T13:32:55.551187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619848+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:32:55.619848+0000","last_peered":"2024-04-02T13:32:55.619848+0000","last_clean":"2024-04-02T13:32:55.619848+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:32:55.619848+0000","last_undegraded":"2024-04-02T13:32:55.619848+0000","last_fullsized":"2024-04-02T13:32:55.619848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149582+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:32:56.149582+0000","last_peered":"2024-04-02T13:32:56.149582+0000","last_clean":"2024-04-02T13:32:56.149582+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:32:56.149582+0000","last_undegraded":"2024-04-02T13:32:56.149582+0000","last_fullsized":"2024-04-02T13:32:56.149582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650396+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:32:56.650396+0000","last_peered":"2024-04-02T13:32:56.650396+0000","last_clean":"2024-04-02T13:32:56.650396+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:32:56.650396+0000","last_undegraded":"2024-04-02T13:32:56.650396+0000","last_fullsized":"2024-04-02T13:32:56.650396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549836+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:32:55.549836+0000","last_peered":"2024-04-02T13:32:55.549836+0000","last_clean":"2024-04-02T13:32:55.549836+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:32:55.549836+0000","last_undegraded":"2024-04-02T13:32:55.549836+0000","last_fullsized":"2024-04-02T13:32:55.549836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649872+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:32:56.649872+0000","last_peered":"2024-04-02T13:32:56.649872+0000","last_clean":"2024-04-02T13:32:56.649872+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:32:56.649872+0000","last_undegraded":"2024-04-02T13:32:56.649872+0000","last_fullsized":"2024-04-02T13:32:56.649872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650800+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:32:56.650800+0000","last_peered":"2024-04-02T13:32:56.650800+0000","last_clean":"2024-04-02T13:32:56.650800+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:32:56.650800+0000","last_undegraded":"2024-04-02T13:32:56.650800+0000","last_fullsized":"2024-04-02T13:32:56.650800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551071+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:32:55.551071+0000","last_peered":"2024-04-02T13:32:55.551071+0000","last_clean":"2024-04-02T13:32:55.551071+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:32:55.551071+0000","last_undegraded":"2024-04-02T13:32:55.551071+0000","last_fullsized":"2024-04-02T13:32:55.551071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619506+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:32:55.619506+0000","last_peered":"2024-04-02T13:32:55.619506+0000","last_clean":"2024-04-02T13:32:55.619506+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:32:55.619506+0000","last_undegraded":"2024-04-02T13:32:55.619506+0000","last_fullsized":"2024-04-02T13:32:55.619506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5564,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":44984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1626112,"data_stored":1076875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":482291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037792,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":482447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1075219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":482431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":481799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":481813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":481731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1601536,"data_stored":1072259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":29524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:33:13.389 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:33:13.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- 172.21.15.122:0/3372457693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c1025d0 msgr2=0x7f8f2c108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 --2- 172.21.15.122:0/3372457693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c108e60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ca0 tx=0x7f8f2402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- 172.21.15.122:0/3372457693 shutdown_connections 2024-04-02T13:33:13.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 --2- 172.21.15.122:0/3372457693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c108e60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 --2- 172.21.15.122:0/3372457693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c101c10 0x7f8f2c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 --2- 172.21.15.122:0/3372457693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c100a20 0x7f8f2c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- 172.21.15.122:0/3372457693 >> 172.21.15.122:0/3372457693 conn(0x7f8f2c0fbf90 msgr2=0x7f8f2c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- 172.21.15.122:0/3372457693 shutdown_connections 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- 172.21.15.122:0/3372457693 wait complete. 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 Processor -- start 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.523+0000 7f8f31c93640 1 -- start start 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c101c10 0x7f8f2c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c19f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c6a0 con 0x7f8f2c100a20 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c520 con 0x7f8f2c1025d0 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f31c93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c10c820 con 0x7f8f2c101c10 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33830/0 (socket says 172.21.15.122:33830) 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2b7fe640 1 -- 172.21.15.122:0/948258971 learned_addr learned my addr 172.21.15.122:0/948258971 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c19f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2affd640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c101c10 0x7f8f2c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 -- 172.21.15.122:0/948258971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c101c10 msgr2=0x7f8f2c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c101c10 0x7f8f2c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 -- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 msgr2=0x7f8f2c197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 -- 172.21.15.122:0/948258971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c19fa20 con 0x7f8f2c1025d0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2b7fe640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f2bfff640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c19f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402c630 tx=0x7f8f24032f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.524+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24039040 con 0x7f8f2c1025d0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.525+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c19fcb0 con 0x7f8f2c1025d0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.525+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24044070 con 0x7f8f2c1025d0 2024-04-02T13:33:13.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.525+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240042a0 con 0x7f8f2c1025d0 2024-04-02T13:33:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.525+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a01c0 con 0x7f8f2c1025d0 2024-04-02T13:33:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.526+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f2c1a0390 con 0x7f8f2c1025d0 2024-04-02T13:33:13.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.527+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8f24032690 con 0x7f8f2c1025d0 2024-04-02T13:33:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.527+0000 7f8f28ff9640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 0x7f8f140a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.527+0000 7f8f2b7fe640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 0x7f8f140a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.527+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8f240f29a0 con 0x7f8f2c1025d0 2024-04-02T13:33:13.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.528+0000 7f8f2b7fe640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 0x7f8f140a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005fd0 tx=0x7f8f1c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:13.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.532+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8f240bf180 con 0x7f8f2c1025d0 2024-04-02T13:33:13.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.708+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8f2c0fe100 con 0x7f8f140a3c40 2024-04-02T13:33:13.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.717+0000 7f8f28ff9640 1 -- 172.21.15.122:0/948258971 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340286 (secure 0 0 0) 0x7f8f2c0fe100 con 0x7f8f140a3c40 2024-04-02T13:33:13.724 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:13.725 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:33:13.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 msgr2=0x7f8f140a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 0x7f8f140a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c005fd0 tx=0x7f8f1c00b410 comp rx=0 tx=0).stop 2024-04-02T13:33:13.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 msgr2=0x7f8f2c19f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:13.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c19f3b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402c630 tx=0x7f8f24032f20 comp rx=0 tx=0).stop 2024-04-02T13:33:13.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 shutdown_connections 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8f140a3c40 0x7f8f140a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f2c1025d0 0x7f8f2c19f3b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f2c101c10 0x7f8f2c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 --2- 172.21.15.122:0/948258971 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2c100a20 0x7f8f2c197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 >> 172.21.15.122:0/948258971 conn(0x7f8f2c0fbf90 msgr2=0x7f8f2c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 shutdown_connections 2024-04-02T13:33:13.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:13.720+0000 7f8f31c93640 1 -- 172.21.15.122:0/948258971 wait complete. 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: pgmap v299: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/143523396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/171152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:13.734 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:13 smithi122 ceph-mon[32114]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:33:13.735 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-04-02T13:33:12.460240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596049,"num_objects":52,"num_object_clones":0,"num_object_copies":157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":144,"num_read_kb":127,"num_write":292,"num_write_kb":2977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371128,"kb_used_data":13600,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570696,"statfs":{"total":1151940427776,"available":1151560392704,"internally_reserved":0,"allocated":13926400,"data_stored":7590761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3271,"num_objects":28,"num_object_clones":0,"num_object_copies":85,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":66,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":20,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003201"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619941+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:32:55.619941+0000","last_peered":"2024-04-02T13:32:55.619941+0000","last_clean":"2024-04-02T13:32:55.619941+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:32:55.619941+0000","last_undegraded":"2024-04-02T13:32:55.619941+0000","last_fullsized":"2024-04-02T13:32:55.619941+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646880+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:32:56.646880+0000","last_peered":"2024-04-02T13:32:56.646880+0000","last_clean":"2024-04-02T13:32:56.646880+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:32:56.646880+0000","last_undegraded":"2024-04-02T13:32:56.646880+0000","last_fullsized":"2024-04-02T13:32:56.646880+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646719+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:32:56.646719+0000","last_peered":"2024-04-02T13:32:56.646719+0000","last_clean":"2024-04-02T13:32:56.646719+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:32:56.646719+0000","last_undegraded":"2024-04-02T13:32:56.646719+0000","last_fullsized":"2024-04-02T13:32:56.646719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651875+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:32:56.651875+0000","last_peered":"2024-04-02T13:32:56.651875+0000","last_clean":"2024-04-02T13:32:56.651875+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:32:56.651875+0000","last_undegraded":"2024-04-02T13:32:56.651875+0000","last_fullsized":"2024-04-02T13:32:56.651875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649801+0000","last_change":"2024-04-02T13:32:50.536139+0000","last_active":"2024-04-02T13:32:56.649801+0000","last_peered":"2024-04-02T13:32:56.649801+0000","last_clean":"2024-04-02T13:32:56.649801+0000","last_became_active":"2024-04-02T13:32:50.535691+0000","last_became_peered":"2024-04-02T13:32:50.535691+0000","last_unstale":"2024-04-02T13:32:56.649801+0000","last_undegraded":"2024-04-02T13:32:56.649801+0000","last_fullsized":"2024-04-02T13:32:56.649801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:54:18.031530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650412+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:32:56.650412+0000","last_peered":"2024-04-02T13:32:56.650412+0000","last_clean":"2024-04-02T13:32:56.650412+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:32:56.650412+0000","last_undegraded":"2024-04-02T13:32:56.650412+0000","last_fullsized":"2024-04-02T13:32:56.650412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550060+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:32:55.550060+0000","last_peered":"2024-04-02T13:32:55.550060+0000","last_clean":"2024-04-02T13:32:55.550060+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:32:55.550060+0000","last_undegraded":"2024-04-02T13:32:55.550060+0000","last_fullsized":"2024-04-02T13:32:55.550060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649687+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:32:56.649687+0000","last_peered":"2024-04-02T13:32:56.649687+0000","last_clean":"2024-04-02T13:32:56.649687+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:32:56.649687+0000","last_undegraded":"2024-04-02T13:32:56.649687+0000","last_fullsized":"2024-04-02T13:32:56.649687+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651830+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:32:56.651830+0000","last_peered":"2024-04-02T13:32:56.651830+0000","last_clean":"2024-04-02T13:32:56.651830+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:32:56.651830+0000","last_undegraded":"2024-04-02T13:32:56.651830+0000","last_fullsized":"2024-04-02T13:32:56.651830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651257+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:32:56.651257+0000","last_peered":"2024-04-02T13:32:56.651257+0000","last_clean":"2024-04-02T13:32:56.651257+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:32:56.651257+0000","last_undegraded":"2024-04-02T13:32:56.651257+0000","last_fullsized":"2024-04-02T13:32:56.651257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550604+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:32:55.550604+0000","last_peered":"2024-04-02T13:32:55.550604+0000","last_clean":"2024-04-02T13:32:55.550604+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:32:55.550604+0000","last_undegraded":"2024-04-02T13:32:55.550604+0000","last_fullsized":"2024-04-02T13:32:55.550604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651801+0000","last_change":"2024-04-02T13:32:50.538066+0000","last_active":"2024-04-02T13:32:56.651801+0000","last_peered":"2024-04-02T13:32:56.651801+0000","last_clean":"2024-04-02T13:32:56.651801+0000","last_became_active":"2024-04-02T13:32:50.537761+0000","last_became_peered":"2024-04-02T13:32:50.537761+0000","last_unstale":"2024-04-02T13:32:56.651801+0000","last_undegraded":"2024-04-02T13:32:56.651801+0000","last_fullsized":"2024-04-02T13:32:56.651801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:43.535184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551249+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:32:55.551249+0000","last_peered":"2024-04-02T13:32:55.551249+0000","last_clean":"2024-04-02T13:32:55.551249+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:32:55.551249+0000","last_undegraded":"2024-04-02T13:32:55.551249+0000","last_fullsized":"2024-04-02T13:32:55.551249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646482+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:32:56.646482+0000","last_peered":"2024-04-02T13:32:56.646482+0000","last_clean":"2024-04-02T13:32:56.646482+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:32:56.646482+0000","last_undegraded":"2024-04-02T13:32:56.646482+0000","last_fullsized":"2024-04-02T13:32:56.646482+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646574+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:32:56.646574+0000","last_peered":"2024-04-02T13:32:56.646574+0000","last_clean":"2024-04-02T13:32:56.646574+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:32:56.646574+0000","last_undegraded":"2024-04-02T13:32:56.646574+0000","last_fullsized":"2024-04-02T13:32:56.646574+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650205+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:32:56.650205+0000","last_peered":"2024-04-02T13:32:56.650205+0000","last_clean":"2024-04-02T13:32:56.650205+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:32:56.650205+0000","last_undegraded":"2024-04-02T13:32:56.650205+0000","last_fullsized":"2024-04-02T13:32:56.650205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619937+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:32:55.619937+0000","last_peered":"2024-04-02T13:32:55.619937+0000","last_clean":"2024-04-02T13:32:55.619937+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:32:55.619937+0000","last_undegraded":"2024-04-02T13:32:55.619937+0000","last_fullsized":"2024-04-02T13:32:55.619937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687978+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:32:55.687978+0000","last_peered":"2024-04-02T13:32:55.687978+0000","last_clean":"2024-04-02T13:32:55.687978+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:32:55.687978+0000","last_undegraded":"2024-04-02T13:32:55.687978+0000","last_fullsized":"2024-04-02T13:32:55.687978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149762+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:32:56.149762+0000","last_peered":"2024-04-02T13:32:56.149762+0000","last_clean":"2024-04-02T13:32:56.149762+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:32:56.149762+0000","last_undegraded":"2024-04-02T13:32:56.149762+0000","last_fullsized":"2024-04-02T13:32:56.149762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149728+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:32:56.149728+0000","last_peered":"2024-04-02T13:32:56.149728+0000","last_clean":"2024-04-02T13:32:56.149728+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:32:56.149728+0000","last_undegraded":"2024-04-02T13:32:56.149728+0000","last_fullsized":"2024-04-02T13:32:56.149728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688011+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:32:55.688011+0000","last_peered":"2024-04-02T13:32:55.688011+0000","last_clean":"2024-04-02T13:32:55.688011+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:32:55.688011+0000","last_undegraded":"2024-04-02T13:32:55.688011+0000","last_fullsized":"2024-04-02T13:32:55.688011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650194+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:32:56.650194+0000","last_peered":"2024-04-02T13:32:56.650194+0000","last_clean":"2024-04-02T13:32:56.650194+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:32:56.650194+0000","last_undegraded":"2024-04-02T13:32:56.650194+0000","last_fullsized":"2024-04-02T13:32:56.650194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650754+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:32:56.650754+0000","last_peered":"2024-04-02T13:32:56.650754+0000","last_clean":"2024-04-02T13:32:56.650754+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:32:56.650754+0000","last_undegraded":"2024-04-02T13:32:56.650754+0000","last_fullsized":"2024-04-02T13:32:56.650754+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650380+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:32:56.650380+0000","last_peered":"2024-04-02T13:32:56.650380+0000","last_clean":"2024-04-02T13:32:56.650380+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:32:56.650380+0000","last_undegraded":"2024-04-02T13:32:56.650380+0000","last_fullsized":"2024-04-02T13:32:56.650380+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646813+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:32:56.646813+0000","last_peered":"2024-04-02T13:32:56.646813+0000","last_clean":"2024-04-02T13:32:56.646813+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:32:56.646813+0000","last_undegraded":"2024-04-02T13:32:56.646813+0000","last_fullsized":"2024-04-02T13:32:56.646813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651025+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:32:56.651025+0000","last_peered":"2024-04-02T13:32:56.651025+0000","last_clean":"2024-04-02T13:32:56.651025+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:32:56.651025+0000","last_undegraded":"2024-04-02T13:32:56.651025+0000","last_fullsized":"2024-04-02T13:32:56.651025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'7","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.643007+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:33:10.643007+0000","last_peered":"2024-04-02T13:33:10.643007+0000","last_clean":"2024-04-02T13:33:10.643007+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:33:10.643007+0000","last_undegraded":"2024-04-02T13:33:10.643007+0000","last_fullsized":"2024-04-02T13:33:10.643007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651074+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:32:56.651074+0000","last_peered":"2024-04-02T13:32:56.651074+0000","last_clean":"2024-04-02T13:32:56.651074+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:32:56.651074+0000","last_undegraded":"2024-04-02T13:32:56.651074+0000","last_fullsized":"2024-04-02T13:32:56.651074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650834+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:32:56.650834+0000","last_peered":"2024-04-02T13:32:56.650834+0000","last_clean":"2024-04-02T13:32:56.650834+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:32:56.650834+0000","last_undegraded":"2024-04-02T13:32:56.650834+0000","last_fullsized":"2024-04-02T13:32:56.650834+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649937+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:32:56.649937+0000","last_peered":"2024-04-02T13:32:56.649937+0000","last_clean":"2024-04-02T13:32:56.649937+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:32:56.649937+0000","last_undegraded":"2024-04-02T13:32:56.649937+0000","last_fullsized":"2024-04-02T13:32:56.649937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550382+0000","last_change":"2024-04-02T13:32:50.536179+0000","last_active":"2024-04-02T13:32:55.550382+0000","last_peered":"2024-04-02T13:32:55.550382+0000","last_clean":"2024-04-02T13:32:55.550382+0000","last_became_active":"2024-04-02T13:32:50.535848+0000","last_became_peered":"2024-04-02T13:32:50.535848+0000","last_unstale":"2024-04-02T13:32:55.550382+0000","last_undegraded":"2024-04-02T13:32:55.550382+0000","last_fullsized":"2024-04-02T13:32:55.550382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:47:08.589683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149699+0000","last_change":"2024-04-02T13:32:54.560990+0000","last_active":"2024-04-02T13:32:56.149699+0000","last_peered":"2024-04-02T13:32:56.149699+0000","last_clean":"2024-04-02T13:32:56.149699+0000","last_became_active":"2024-04-02T13:32:54.560528+0000","last_became_peered":"2024-04-02T13:32:54.560528+0000","last_unstale":"2024-04-02T13:32:56.149699+0000","last_undegraded":"2024-04-02T13:32:56.149699+0000","last_fullsized":"2024-04-02T13:32:56.149699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:41:41.675349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550098+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:32:55.550098+0000","last_peered":"2024-04-02T13:32:55.550098+0000","last_clean":"2024-04-02T13:32:55.550098+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:32:55.550098+0000","last_undegraded":"2024-04-02T13:32:55.550098+0000","last_fullsized":"2024-04-02T13:32:55.550098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649929+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:32:56.649929+0000","last_peered":"2024-04-02T13:32:56.649929+0000","last_clean":"2024-04-02T13:32:56.649929+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:32:56.649929+0000","last_undegraded":"2024-04-02T13:32:56.649929+0000","last_fullsized":"2024-04-02T13:32:56.649929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149448+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:32:56.149448+0000","last_peered":"2024-04-02T13:32:56.149448+0000","last_clean":"2024-04-02T13:32:56.149448+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:32:56.149448+0000","last_undegraded":"2024-04-02T13:32:56.149448+0000","last_fullsized":"2024-04-02T13:32:56.149448+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650298+0000","last_change":"2024-04-02T13:32:49.523267+0000","last_active":"2024-04-02T13:32:56.650298+0000","last_peered":"2024-04-02T13:32:56.650298+0000","last_clean":"2024-04-02T13:32:56.650298+0000","last_became_active":"2024-04-02T13:32:49.522953+0000","last_became_peered":"2024-04-02T13:32:49.522953+0000","last_unstale":"2024-04-02T13:32:56.650298+0000","last_undegraded":"2024-04-02T13:32:56.650298+0000","last_fullsized":"2024-04-02T13:32:56.650298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:03:10.578642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550271+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:32:55.550271+0000","last_peered":"2024-04-02T13:32:55.550271+0000","last_clean":"2024-04-02T13:32:55.550271+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:32:55.550271+0000","last_undegraded":"2024-04-02T13:32:55.550271+0000","last_fullsized":"2024-04-02T13:32:55.550271+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619847+0000","last_change":"2024-04-02T13:32:54.568499+0000","last_active":"2024-04-02T13:32:55.619847+0000","last_peered":"2024-04-02T13:32:55.619847+0000","last_clean":"2024-04-02T13:32:55.619847+0000","last_became_active":"2024-04-02T13:32:54.568140+0000","last_became_peered":"2024-04-02T13:32:54.568140+0000","last_unstale":"2024-04-02T13:32:55.619847+0000","last_undegraded":"2024-04-02T13:32:55.619847+0000","last_fullsized":"2024-04-02T13:32:55.619847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:03.120174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:09.635952+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:33:09.635952+0000","last_peered":"2024-04-02T13:33:09.635952+0000","last_clean":"2024-04-02T13:33:09.635952+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:33:09.635952+0000","last_undegraded":"2024-04-02T13:33:09.635952+0000","last_fullsized":"2024-04-02T13:33:09.635952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650058+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:32:56.650058+0000","last_peered":"2024-04-02T13:32:56.650058+0000","last_clean":"2024-04-02T13:32:56.650058+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:32:56.650058+0000","last_undegraded":"2024-04-02T13:32:56.650058+0000","last_fullsized":"2024-04-02T13:32:56.650058+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646925+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:32:56.646925+0000","last_peered":"2024-04-02T13:32:56.646925+0000","last_clean":"2024-04-02T13:32:56.646925+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:32:56.646925+0000","last_undegraded":"2024-04-02T13:32:56.646925+0000","last_fullsized":"2024-04-02T13:32:56.646925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549717+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:32:55.549717+0000","last_peered":"2024-04-02T13:32:55.549717+0000","last_clean":"2024-04-02T13:32:55.549717+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:32:55.549717+0000","last_undegraded":"2024-04-02T13:32:55.549717+0000","last_fullsized":"2024-04-02T13:32:55.549717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687520+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:32:55.687520+0000","last_peered":"2024-04-02T13:32:55.687520+0000","last_clean":"2024-04-02T13:32:55.687520+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:32:55.687520+0000","last_undegraded":"2024-04-02T13:32:55.687520+0000","last_fullsized":"2024-04-02T13:32:55.687520+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649884+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:32:56.649884+0000","last_peered":"2024-04-02T13:32:56.649884+0000","last_clean":"2024-04-02T13:32:56.649884+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:32:56.649884+0000","last_undegraded":"2024-04-02T13:32:56.649884+0000","last_fullsized":"2024-04-02T13:32:56.649884+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625861+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:33:08.625861+0000","last_peered":"2024-04-02T13:33:08.625861+0000","last_clean":"2024-04-02T13:33:08.625861+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:33:08.625861+0000","last_undegraded":"2024-04-02T13:33:08.625861+0000","last_fullsized":"2024-04-02T13:33:08.625861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651763+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:32:56.651763+0000","last_peered":"2024-04-02T13:32:56.651763+0000","last_clean":"2024-04-02T13:32:56.651763+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:32:56.651763+0000","last_undegraded":"2024-04-02T13:32:56.651763+0000","last_fullsized":"2024-04-02T13:32:56.651763+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687671+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:32:55.687671+0000","last_peered":"2024-04-02T13:32:55.687671+0000","last_clean":"2024-04-02T13:32:55.687671+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:32:55.687671+0000","last_undegraded":"2024-04-02T13:32:55.687671+0000","last_fullsized":"2024-04-02T13:32:55.687671+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.157890+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:33:11.157890+0000","last_peered":"2024-04-02T13:33:11.157890+0000","last_clean":"2024-04-02T13:33:11.157890+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:33:11.157890+0000","last_undegraded":"2024-04-02T13:33:11.157890+0000","last_fullsized":"2024-04-02T13:33:11.157890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646955+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:32:56.646955+0000","last_peered":"2024-04-02T13:32:56.646955+0000","last_clean":"2024-04-02T13:32:56.646955+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:32:56.646955+0000","last_undegraded":"2024-04-02T13:32:56.646955+0000","last_fullsized":"2024-04-02T13:32:56.646955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651536+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:32:56.651536+0000","last_peered":"2024-04-02T13:32:56.651536+0000","last_clean":"2024-04-02T13:32:56.651536+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:32:56.651536+0000","last_undegraded":"2024-04-02T13:32:56.651536+0000","last_fullsized":"2024-04-02T13:32:56.651536+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"85'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550320+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:32:55.550320+0000","last_peered":"2024-04-02T13:32:55.550320+0000","last_clean":"2024-04-02T13:32:55.550320+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:32:55.550320+0000","last_undegraded":"2024-04-02T13:32:55.550320+0000","last_fullsized":"2024-04-02T13:32:55.550320+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651257+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:32:56.651257+0000","last_peered":"2024-04-02T13:32:56.651257+0000","last_clean":"2024-04-02T13:32:56.651257+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:32:56.651257+0000","last_undegraded":"2024-04-02T13:32:56.651257+0000","last_fullsized":"2024-04-02T13:32:56.651257+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687694+0000","last_change":"2024-04-02T13:32:54.562030+0000","last_active":"2024-04-02T13:32:55.687694+0000","last_peered":"2024-04-02T13:32:55.687694+0000","last_clean":"2024-04-02T13:32:55.687694+0000","last_became_active":"2024-04-02T13:32:54.560872+0000","last_became_peered":"2024-04-02T13:32:54.560872+0000","last_unstale":"2024-04-02T13:32:55.687694+0000","last_undegraded":"2024-04-02T13:32:55.687694+0000","last_fullsized":"2024-04-02T13:32:55.687694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:25:08.994729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625616+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:33:08.625616+0000","last_peered":"2024-04-02T13:33:08.625616+0000","last_clean":"2024-04-02T13:33:08.625616+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:33:08.625616+0000","last_undegraded":"2024-04-02T13:33:08.625616+0000","last_fullsized":"2024-04-02T13:33:08.625616+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650098+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:32:56.650098+0000","last_peered":"2024-04-02T13:32:56.650098+0000","last_clean":"2024-04-02T13:32:56.650098+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:32:56.650098+0000","last_undegraded":"2024-04-02T13:32:56.650098+0000","last_fullsized":"2024-04-02T13:32:56.650098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550797+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:32:55.550797+0000","last_peered":"2024-04-02T13:32:55.550797+0000","last_clean":"2024-04-02T13:32:55.550797+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:32:55.550797+0000","last_undegraded":"2024-04-02T13:32:55.550797+0000","last_fullsized":"2024-04-02T13:32:55.550797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.625213+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:33:08.625213+0000","last_peered":"2024-04-02T13:33:08.625213+0000","last_clean":"2024-04-02T13:33:08.625213+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:33:08.625213+0000","last_undegraded":"2024-04-02T13:33:08.625213+0000","last_fullsized":"2024-04-02T13:33:08.625213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551137+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:32:55.551137+0000","last_peered":"2024-04-02T13:32:55.551137+0000","last_clean":"2024-04-02T13:32:55.551137+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:32:55.551137+0000","last_undegraded":"2024-04-02T13:32:55.551137+0000","last_fullsized":"2024-04-02T13:32:55.551137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619613+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:32:55.619613+0000","last_peered":"2024-04-02T13:32:55.619613+0000","last_clean":"2024-04-02T13:32:55.619613+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:32:55.619613+0000","last_undegraded":"2024-04-02T13:32:55.619613+0000","last_fullsized":"2024-04-02T13:32:55.619613+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550167+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:32:55.550167+0000","last_peered":"2024-04-02T13:32:55.550167+0000","last_clean":"2024-04-02T13:32:55.550167+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:32:55.550167+0000","last_undegraded":"2024-04-02T13:32:55.550167+0000","last_fullsized":"2024-04-02T13:32:55.550167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550299+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:32:55.550299+0000","last_peered":"2024-04-02T13:32:55.550299+0000","last_clean":"2024-04-02T13:32:55.550299+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:32:55.550299+0000","last_undegraded":"2024-04-02T13:32:55.550299+0000","last_fullsized":"2024-04-02T13:32:55.550299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687956+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:32:55.687956+0000","last_peered":"2024-04-02T13:32:55.687956+0000","last_clean":"2024-04-02T13:32:55.687956+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:32:55.687956+0000","last_undegraded":"2024-04-02T13:32:55.687956+0000","last_fullsized":"2024-04-02T13:32:55.687956+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650800+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:32:56.650800+0000","last_peered":"2024-04-02T13:32:56.650800+0000","last_clean":"2024-04-02T13:32:56.650800+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:32:56.650800+0000","last_undegraded":"2024-04-02T13:32:56.650800+0000","last_fullsized":"2024-04-02T13:32:56.650800+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550126+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:32:55.550126+0000","last_peered":"2024-04-02T13:32:55.550126+0000","last_clean":"2024-04-02T13:32:55.550126+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:32:55.550126+0000","last_undegraded":"2024-04-02T13:32:55.550126+0000","last_fullsized":"2024-04-02T13:32:55.550126+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551712+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:32:55.551712+0000","last_peered":"2024-04-02T13:32:55.551712+0000","last_clean":"2024-04-02T13:32:55.551712+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:32:55.551712+0000","last_undegraded":"2024-04-02T13:32:55.551712+0000","last_fullsized":"2024-04-02T13:32:55.551712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'16","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.641914+0000","last_change":"2024-04-02T13:32:49.525732+0000","last_active":"2024-04-02T13:33:10.641914+0000","last_peered":"2024-04-02T13:33:10.641914+0000","last_clean":"2024-04-02T13:33:10.641914+0000","last_became_active":"2024-04-02T13:32:49.525404+0000","last_became_peered":"2024-04-02T13:32:49.525404+0000","last_unstale":"2024-04-02T13:33:10.641914+0000","last_undegraded":"2024-04-02T13:33:10.641914+0000","last_fullsized":"2024-04-02T13:33:10.641914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:21.317513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550163+0000","last_change":"2024-04-02T13:32:50.533696+0000","last_active":"2024-04-02T13:32:55.550163+0000","last_peered":"2024-04-02T13:32:55.550163+0000","last_clean":"2024-04-02T13:32:55.550163+0000","last_became_active":"2024-04-02T13:32:50.533369+0000","last_became_peered":"2024-04-02T13:32:50.533369+0000","last_unstale":"2024-04-02T13:32:55.550163+0000","last_undegraded":"2024-04-02T13:32:55.550163+0000","last_fullsized":"2024-04-02T13:32:55.550163+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:29:09.847846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550010+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:32:55.550010+0000","last_peered":"2024-04-02T13:32:55.550010+0000","last_clean":"2024-04-02T13:32:55.550010+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:32:55.550010+0000","last_undegraded":"2024-04-02T13:32:55.550010+0000","last_fullsized":"2024-04-02T13:32:55.550010+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.630205+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:33:10.630205+0000","last_peered":"2024-04-02T13:33:10.630205+0000","last_clean":"2024-04-02T13:33:10.630205+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:33:10.630205+0000","last_undegraded":"2024-04-02T13:33:10.630205+0000","last_fullsized":"2024-04-02T13:33:10.630205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619722+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:32:55.619722+0000","last_peered":"2024-04-02T13:32:55.619722+0000","last_clean":"2024-04-02T13:32:55.619722+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:32:55.619722+0000","last_undegraded":"2024-04-02T13:32:55.619722+0000","last_fullsized":"2024-04-02T13:32:55.619722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650001+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:32:56.650001+0000","last_peered":"2024-04-02T13:32:56.650001+0000","last_clean":"2024-04-02T13:32:56.650001+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:32:56.650001+0000","last_undegraded":"2024-04-02T13:32:56.650001+0000","last_fullsized":"2024-04-02T13:32:56.650001+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.687798+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:32:55.687798+0000","last_peered":"2024-04-02T13:32:55.687798+0000","last_clean":"2024-04-02T13:32:55.687798+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:32:55.687798+0000","last_undegraded":"2024-04-02T13:32:55.687798+0000","last_fullsized":"2024-04-02T13:32:55.687798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650888+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:32:56.650888+0000","last_peered":"2024-04-02T13:32:56.650888+0000","last_clean":"2024-04-02T13:32:56.650888+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:32:56.650888+0000","last_undegraded":"2024-04-02T13:32:56.650888+0000","last_fullsized":"2024-04-02T13:32:56.650888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551589+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:32:55.551589+0000","last_peered":"2024-04-02T13:32:55.551589+0000","last_clean":"2024-04-02T13:32:55.551589+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:32:55.551589+0000","last_undegraded":"2024-04-02T13:32:55.551589+0000","last_fullsized":"2024-04-02T13:32:55.551589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.158741+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:33:11.158741+0000","last_peered":"2024-04-02T13:33:11.158741+0000","last_clean":"2024-04-02T13:33:11.158741+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:33:11.158741+0000","last_undegraded":"2024-04-02T13:33:11.158741+0000","last_fullsized":"2024-04-02T13:33:11.158741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650443+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:32:56.650443+0000","last_peered":"2024-04-02T13:32:56.650443+0000","last_clean":"2024-04-02T13:32:56.650443+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:32:56.650443+0000","last_undegraded":"2024-04-02T13:32:56.650443+0000","last_fullsized":"2024-04-02T13:32:56.650443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551638+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:32:55.551638+0000","last_peered":"2024-04-02T13:32:55.551638+0000","last_clean":"2024-04-02T13:32:55.551638+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:32:55.551638+0000","last_undegraded":"2024-04-02T13:32:55.551638+0000","last_fullsized":"2024-04-02T13:32:55.551638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549973+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:32:55.549973+0000","last_peered":"2024-04-02T13:32:55.549973+0000","last_clean":"2024-04-02T13:32:55.549973+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:32:55.549973+0000","last_undegraded":"2024-04-02T13:32:55.549973+0000","last_fullsized":"2024-04-02T13:32:55.549973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.150626+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:33:11.150626+0000","last_peered":"2024-04-02T13:33:11.150626+0000","last_clean":"2024-04-02T13:33:11.150626+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:33:11.150626+0000","last_undegraded":"2024-04-02T13:33:11.150626+0000","last_fullsized":"2024-04-02T13:33:11.150626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":15,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.508447+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:32:56.508447+0000","last_peered":"2024-04-02T13:32:56.508447+0000","last_clean":"2024-04-02T13:32:56.508447+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:32:56.508447+0000","last_undegraded":"2024-04-02T13:32:56.508447+0000","last_fullsized":"2024-04-02T13:32:56.508447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649785+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:32:56.649785+0000","last_peered":"2024-04-02T13:32:56.649785+0000","last_clean":"2024-04-02T13:32:56.649785+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:32:56.649785+0000","last_undegraded":"2024-04-02T13:32:56.649785+0000","last_fullsized":"2024-04-02T13:32:56.649785+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651245+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:32:56.651245+0000","last_peered":"2024-04-02T13:32:56.651245+0000","last_clean":"2024-04-02T13:32:56.651245+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:32:56.651245+0000","last_undegraded":"2024-04-02T13:32:56.651245+0000","last_fullsized":"2024-04-02T13:32:56.651245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549912+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:32:55.549912+0000","last_peered":"2024-04-02T13:32:55.549912+0000","last_clean":"2024-04-02T13:32:55.549912+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:32:55.549912+0000","last_undegraded":"2024-04-02T13:32:55.549912+0000","last_fullsized":"2024-04-02T13:32:55.549912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550583+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:32:55.550583+0000","last_peered":"2024-04-02T13:32:55.550583+0000","last_clean":"2024-04-02T13:32:55.550583+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:32:55.550583+0000","last_undegraded":"2024-04-02T13:32:55.550583+0000","last_fullsized":"2024-04-02T13:32:55.550583+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550789+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:32:55.550789+0000","last_peered":"2024-04-02T13:32:55.550789+0000","last_clean":"2024-04-02T13:32:55.550789+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:32:55.550789+0000","last_undegraded":"2024-04-02T13:32:55.550789+0000","last_fullsized":"2024-04-02T13:32:55.550789+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:09.634870+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:33:09.634870+0000","last_peered":"2024-04-02T13:33:09.634870+0000","last_clean":"2024-04-02T13:33:09.634870+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:33:09.634870+0000","last_undegraded":"2024-04-02T13:33:09.634870+0000","last_fullsized":"2024-04-02T13:33:09.634870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619566+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:32:55.619566+0000","last_peered":"2024-04-02T13:32:55.619566+0000","last_clean":"2024-04-02T13:32:55.619566+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:32:55.619566+0000","last_undegraded":"2024-04-02T13:32:55.619566+0000","last_fullsized":"2024-04-02T13:32:55.619566+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:10.642732+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:33:10.642732+0000","last_peered":"2024-04-02T13:33:10.642732+0000","last_clean":"2024-04-02T13:33:10.642732+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:33:10.642732+0000","last_undegraded":"2024-04-02T13:33:10.642732+0000","last_fullsized":"2024-04-02T13:33:10.642732+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551277+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:32:55.551277+0000","last_peered":"2024-04-02T13:32:55.551277+0000","last_clean":"2024-04-02T13:32:55.551277+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:32:55.551277+0000","last_undegraded":"2024-04-02T13:32:55.551277+0000","last_fullsized":"2024-04-02T13:32:55.551277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649850+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:32:56.649850+0000","last_peered":"2024-04-02T13:32:56.649850+0000","last_clean":"2024-04-02T13:32:56.649850+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:32:56.649850+0000","last_undegraded":"2024-04-02T13:32:56.649850+0000","last_fullsized":"2024-04-02T13:32:56.649850+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688010+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:32:55.688010+0000","last_peered":"2024-04-02T13:32:55.688010+0000","last_clean":"2024-04-02T13:32:55.688010+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:32:55.688010+0000","last_undegraded":"2024-04-02T13:32:55.688010+0000","last_fullsized":"2024-04-02T13:32:55.688010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650315+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:32:56.650315+0000","last_peered":"2024-04-02T13:32:56.650315+0000","last_clean":"2024-04-02T13:32:56.650315+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:32:56.650315+0000","last_undegraded":"2024-04-02T13:32:56.650315+0000","last_fullsized":"2024-04-02T13:32:56.650315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149111+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:32:56.149111+0000","last_peered":"2024-04-02T13:32:56.149111+0000","last_clean":"2024-04-02T13:32:56.149111+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:32:56.149111+0000","last_undegraded":"2024-04-02T13:32:56.149111+0000","last_fullsized":"2024-04-02T13:32:56.149111+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"85'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651954+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:32:56.651954+0000","last_peered":"2024-04-02T13:32:56.651954+0000","last_clean":"2024-04-02T13:32:56.651954+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:32:56.651954+0000","last_undegraded":"2024-04-02T13:32:56.651954+0000","last_fullsized":"2024-04-02T13:32:56.651954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646937+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:32:56.646937+0000","last_peered":"2024-04-02T13:32:56.646937+0000","last_clean":"2024-04-02T13:32:56.646937+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:32:56.646937+0000","last_undegraded":"2024-04-02T13:32:56.646937+0000","last_fullsized":"2024-04-02T13:32:56.646937+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651822+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:32:56.651822+0000","last_peered":"2024-04-02T13:32:56.651822+0000","last_clean":"2024-04-02T13:32:56.651822+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:32:56.651822+0000","last_undegraded":"2024-04-02T13:32:56.651822+0000","last_fullsized":"2024-04-02T13:32:56.651822+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.626322+0000","last_change":"2024-04-02T13:32:49.524826+0000","last_active":"2024-04-02T13:33:08.626322+0000","last_peered":"2024-04-02T13:33:08.626322+0000","last_clean":"2024-04-02T13:33:08.626322+0000","last_became_active":"2024-04-02T13:32:49.524516+0000","last_became_peered":"2024-04-02T13:32:49.524516+0000","last_unstale":"2024-04-02T13:33:08.626322+0000","last_undegraded":"2024-04-02T13:33:08.626322+0000","last_fullsized":"2024-04-02T13:33:08.626322+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:20:07.176783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649676+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:32:56.649676+0000","last_peered":"2024-04-02T13:32:56.649676+0000","last_clean":"2024-04-02T13:32:56.649676+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:32:56.649676+0000","last_undegraded":"2024-04-02T13:32:56.649676+0000","last_fullsized":"2024-04-02T13:32:56.649676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.688046+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:32:55.688046+0000","last_peered":"2024-04-02T13:32:55.688046+0000","last_clean":"2024-04-02T13:32:55.688046+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:32:55.688046+0000","last_undegraded":"2024-04-02T13:32:55.688046+0000","last_fullsized":"2024-04-02T13:32:55.688046+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649773+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:32:56.649773+0000","last_peered":"2024-04-02T13:32:56.649773+0000","last_clean":"2024-04-02T13:32:56.649773+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:32:56.649773+0000","last_undegraded":"2024-04-02T13:32:56.649773+0000","last_fullsized":"2024-04-02T13:32:56.649773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619365+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:32:55.619365+0000","last_peered":"2024-04-02T13:32:55.619365+0000","last_clean":"2024-04-02T13:32:55.619365+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:32:55.619365+0000","last_undegraded":"2024-04-02T13:32:55.619365+0000","last_fullsized":"2024-04-02T13:32:55.619365+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651802+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:32:56.651802+0000","last_peered":"2024-04-02T13:32:56.651802+0000","last_clean":"2024-04-02T13:32:56.651802+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:32:56.651802+0000","last_undegraded":"2024-04-02T13:32:56.651802+0000","last_fullsized":"2024-04-02T13:32:56.651802+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149231+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:32:56.149231+0000","last_peered":"2024-04-02T13:32:56.149231+0000","last_clean":"2024-04-02T13:32:56.149231+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:32:56.149231+0000","last_undegraded":"2024-04-02T13:32:56.149231+0000","last_fullsized":"2024-04-02T13:32:56.149231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650027+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:32:56.650027+0000","last_peered":"2024-04-02T13:32:56.650027+0000","last_clean":"2024-04-02T13:32:56.650027+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:32:56.650027+0000","last_undegraded":"2024-04-02T13:32:56.650027+0000","last_fullsized":"2024-04-02T13:32:56.650027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.651565+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:32:56.651565+0000","last_peered":"2024-04-02T13:32:56.651565+0000","last_clean":"2024-04-02T13:32:56.651565+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:32:56.651565+0000","last_undegraded":"2024-04-02T13:32:56.651565+0000","last_fullsized":"2024-04-02T13:32:56.651565+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:08.626398+0000","last_change":"2024-04-02T13:32:49.528077+0000","last_active":"2024-04-02T13:33:08.626398+0000","last_peered":"2024-04-02T13:33:08.626398+0000","last_clean":"2024-04-02T13:33:08.626398+0000","last_became_active":"2024-04-02T13:32:49.527754+0000","last_became_peered":"2024-04-02T13:32:49.527754+0000","last_unstale":"2024-04-02T13:33:08.626398+0000","last_undegraded":"2024-04-02T13:33:08.626398+0000","last_fullsized":"2024-04-02T13:33:08.626398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:26:01.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550267+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:32:55.550267+0000","last_peered":"2024-04-02T13:32:55.550267+0000","last_clean":"2024-04-02T13:32:55.550267+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:32:55.550267+0000","last_undegraded":"2024-04-02T13:32:55.550267+0000","last_fullsized":"2024-04-02T13:32:55.550267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650437+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:32:56.650437+0000","last_peered":"2024-04-02T13:32:56.650437+0000","last_clean":"2024-04-02T13:32:56.650437+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:32:56.650437+0000","last_undegraded":"2024-04-02T13:32:56.650437+0000","last_fullsized":"2024-04-02T13:32:56.650437+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149562+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:32:56.149562+0000","last_peered":"2024-04-02T13:32:56.149562+0000","last_clean":"2024-04-02T13:32:56.149562+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:32:56.149562+0000","last_undegraded":"2024-04-02T13:32:56.149562+0000","last_fullsized":"2024-04-02T13:32:56.149562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550610+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:32:55.550610+0000","last_peered":"2024-04-02T13:32:55.550610+0000","last_clean":"2024-04-02T13:32:55.550610+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:32:55.550610+0000","last_undegraded":"2024-04-02T13:32:55.550610+0000","last_fullsized":"2024-04-02T13:32:55.550610+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646631+0000","last_change":"2024-04-02T13:32:54.566345+0000","last_active":"2024-04-02T13:32:56.646631+0000","last_peered":"2024-04-02T13:32:56.646631+0000","last_clean":"2024-04-02T13:32:56.646631+0000","last_became_active":"2024-04-02T13:32:54.564927+0000","last_became_peered":"2024-04-02T13:32:54.564927+0000","last_unstale":"2024-04-02T13:32:56.646631+0000","last_undegraded":"2024-04-02T13:32:56.646631+0000","last_fullsized":"2024-04-02T13:32:56.646631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:34.439915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550540+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:32:55.550540+0000","last_peered":"2024-04-02T13:32:55.550540+0000","last_clean":"2024-04-02T13:32:55.550540+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:32:55.550540+0000","last_undegraded":"2024-04-02T13:32:55.550540+0000","last_fullsized":"2024-04-02T13:32:55.550540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550127+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:32:55.550127+0000","last_peered":"2024-04-02T13:32:55.550127+0000","last_clean":"2024-04-02T13:32:55.550127+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:32:55.550127+0000","last_undegraded":"2024-04-02T13:32:55.550127+0000","last_fullsized":"2024-04-02T13:32:55.550127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549758+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:32:55.549758+0000","last_peered":"2024-04-02T13:32:55.549758+0000","last_clean":"2024-04-02T13:32:55.549758+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:32:55.549758+0000","last_undegraded":"2024-04-02T13:32:55.549758+0000","last_fullsized":"2024-04-02T13:32:55.549758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:33:11.159142+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:33:11.159142+0000","last_peered":"2024-04-02T13:33:11.159142+0000","last_clean":"2024-04-02T13:33:11.159142+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:33:11.159142+0000","last_undegraded":"2024-04-02T13:33:11.159142+0000","last_fullsized":"2024-04-02T13:33:11.159142+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.550471+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:32:55.550471+0000","last_peered":"2024-04-02T13:32:55.550471+0000","last_clean":"2024-04-02T13:32:55.550471+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:32:55.550471+0000","last_undegraded":"2024-04-02T13:32:55.550471+0000","last_fullsized":"2024-04-02T13:32:55.550471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619439+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:32:55.619439+0000","last_peered":"2024-04-02T13:32:55.619439+0000","last_clean":"2024-04-02T13:32:55.619439+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:32:55.619439+0000","last_undegraded":"2024-04-02T13:32:55.619439+0000","last_fullsized":"2024-04-02T13:32:55.619439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149733+0000","last_change":"2024-04-02T13:32:50.530731+0000","last_active":"2024-04-02T13:32:56.149733+0000","last_peered":"2024-04-02T13:32:56.149733+0000","last_clean":"2024-04-02T13:32:56.149733+0000","last_became_active":"2024-04-02T13:32:50.530286+0000","last_became_peered":"2024-04-02T13:32:50.530286+0000","last_unstale":"2024-04-02T13:32:56.149733+0000","last_undegraded":"2024-04-02T13:32:56.149733+0000","last_fullsized":"2024-04-02T13:32:56.149733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:43:48.111854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149143+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:32:56.149143+0000","last_peered":"2024-04-02T13:32:56.149143+0000","last_clean":"2024-04-02T13:32:56.149143+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:32:56.149143+0000","last_undegraded":"2024-04-02T13:32:56.149143+0000","last_fullsized":"2024-04-02T13:32:56.149143+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.646674+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:32:56.646674+0000","last_peered":"2024-04-02T13:32:56.646674+0000","last_clean":"2024-04-02T13:32:56.646674+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:32:56.646674+0000","last_undegraded":"2024-04-02T13:32:56.646674+0000","last_fullsized":"2024-04-02T13:32:56.646674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551187+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:32:55.551187+0000","last_peered":"2024-04-02T13:32:55.551187+0000","last_clean":"2024-04-02T13:32:55.551187+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:32:55.551187+0000","last_undegraded":"2024-04-02T13:32:55.551187+0000","last_fullsized":"2024-04-02T13:32:55.551187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619848+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:32:55.619848+0000","last_peered":"2024-04-02T13:32:55.619848+0000","last_clean":"2024-04-02T13:32:55.619848+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:32:55.619848+0000","last_undegraded":"2024-04-02T13:32:55.619848+0000","last_fullsized":"2024-04-02T13:32:55.619848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.149582+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:32:56.149582+0000","last_peered":"2024-04-02T13:32:56.149582+0000","last_clean":"2024-04-02T13:32:56.149582+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:32:56.149582+0000","last_undegraded":"2024-04-02T13:32:56.149582+0000","last_fullsized":"2024-04-02T13:32:56.149582+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650396+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:32:56.650396+0000","last_peered":"2024-04-02T13:32:56.650396+0000","last_clean":"2024-04-02T13:32:56.650396+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:32:56.650396+0000","last_undegraded":"2024-04-02T13:32:56.650396+0000","last_fullsized":"2024-04-02T13:32:56.650396+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.549836+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:32:55.549836+0000","last_peered":"2024-04-02T13:32:55.549836+0000","last_clean":"2024-04-02T13:32:55.549836+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:32:55.549836+0000","last_undegraded":"2024-04-02T13:32:55.549836+0000","last_fullsized":"2024-04-02T13:32:55.549836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.649872+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:32:56.649872+0000","last_peered":"2024-04-02T13:32:56.649872+0000","last_clean":"2024-04-02T13:32:56.649872+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:32:56.649872+0000","last_undegraded":"2024-04-02T13:32:56.649872+0000","last_fullsized":"2024-04-02T13:32:56.649872+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:56.650800+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:32:56.650800+0000","last_peered":"2024-04-02T13:32:56.650800+0000","last_clean":"2024-04-02T13:32:56.650800+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:32:56.650800+0000","last_undegraded":"2024-04-02T13:32:56.650800+0000","last_fullsized":"2024-04-02T13:32:56.650800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.551071+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:32:55.551071+0000","last_peered":"2024-04-02T13:32:55.551071+0000","last_clean":"2024-04-02T13:32:55.551071+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:32:55.551071+0000","last_undegraded":"2024-04-02T13:32:55.551071+0000","last_fullsized":"2024-04-02T13:32:55.551071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-02T13:32:55.619506+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:32:55.619506+0000","last_peered":"2024-04-02T13:32:55.619506+0000","last_clean":"2024-04-02T13:32:55.619506+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:32:55.619506+0000","last_undegraded":"2024-04-02T13:32:55.619506+0000","last_fullsized":"2024-04-02T13:32:55.619506+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5564,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":110,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":44984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":995328,"data_stored":481709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612629,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1626112,"data_stored":1076875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776155,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":999424,"data_stored":482291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037792,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":482447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299429,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1075219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":482431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":506602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":481799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":485585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182456,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":481813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920820,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":481731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712400,"statfs":{"total":95995035648,"available":95961497600,"internally_reserved":0,"allocated":1601536,"data_stored":1072259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":29524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:33:13.740 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-02T13:33:13.740 INFO:tasks.cephadm:Setting up client nodes... 2024-04-02T13:33:13.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: pgmap v299: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/143523396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/171152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:13 smithi187 ceph-mon[36064]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:33:14.015 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: pgmap v299: 129 pgs: 129 active+clean; 582 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 2.1 KiB/s wr, 6 op/s 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/143523396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/171152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:13 smithi188 ceph-mon[36117]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-02T13:33:14.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:14 smithi122 ceph-mon[32114]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:14.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:14 smithi187 ceph-mon[36064]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:14 smithi188 ceph-mon[36117]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:15.737 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:15 smithi122 ceph-mon[32114]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:15.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:15 smithi122 ceph-mon[32114]: pgmap v300: 129 pgs: 129 active+clean; 622 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-02T13:33:15.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- 172.21.15.122:0/3958492157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50100cf0 msgr2=0x7f5b50101110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/3958492157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50100cf0 0x7f5b50101110 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5b400099c0 tx=0x7f5b40009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- 172.21.15.122:0/3958492157 shutdown_connections 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/3958492157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50102890 0x7f5b50109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/3958492157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50100cf0 0x7f5b50101110 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/3958492157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- 172.21.15.122:0/3958492157 >> 172.21.15.122:0/3958492157 conn(0x7f5b500fc290 msgr2=0x7f5b500fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- 172.21.15.122:0/3958492157 shutdown_connections 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- 172.21.15.122:0/3958492157 wait complete. 2024-04-02T13:33:15.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 Processor -- start 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.847+0000 7f5b55b4b700 1 -- start start 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50100cf0 0x7f5b50198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5010d160 con 0x7f5b50102890 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5010cfe0 con 0x7f5b50100cf0 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5010d2e0 con 0x7f5b50101ef0 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33848/0 (socket says 172.21.15.122:33848) 2024-04-02T13:33:15.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 -- 172.21.15.122:0/301600842 learned_addr learned my addr 172.21.15.122:0/301600842 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4f7fe700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50100cf0 0x7f5b50198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4effd700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 -- 172.21.15.122:0/301600842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 msgr2=0x7f5b50198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 -- 172.21.15.122:0/301600842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50100cf0 msgr2=0x7f5b50198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50100cf0 0x7f5b50198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 -- 172.21.15.122:0/301600842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b501a0360 con 0x7f5b50102890 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4effd700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4ffff700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b48007da0 tx=0x7f5b4800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48016020 con 0x7f5b50102890 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.848+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b501a0650 con 0x7f5b50102890 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.849+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b4800ad00 con 0x7f5b50102890 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.849+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48011620 con 0x7f5b50102890 2024-04-02T13:33:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.849+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b501a0b90 con 0x7f5b50102890 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.850+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b50050b00 con 0x7f5b50102890 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.851+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5b48019020 con 0x7f5b50102890 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.851+0000 7f5b4cff9700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 0x7f5b380a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.851+0000 7f5b4f7fe700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 0x7f5b380a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.851+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f5b480d0680 con 0x7f5b50102890 2024-04-02T13:33:15.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.852+0000 7f5b4f7fe700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 0x7f5b380a6240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b580 tx=0x7f5b40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:15.856+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5b48020020 con 0x7f5b50102890 2024-04-02T13:33:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:15 smithi187 ceph-mon[36064]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:15 smithi187 ceph-mon[36064]: pgmap v300: 129 pgs: 129 active+clean; 622 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-02T13:33:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:15 smithi188 ceph-mon[36117]: from='client.24725 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:33:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:15 smithi188 ceph-mon[36117]: pgmap v300: 129 pgs: 129 active+clean; 622 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 8.0 KiB/s wr, 18 op/s 2024-04-02T13:33:16.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.134+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f5b501a0fc0 con 0x7f5b50102890 2024-04-02T13:33:16.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.137+0000 7f5b4cff9700 1 -- 172.21.15.122:0/301600842 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f5b48019310 con 0x7f5b50102890 2024-04-02T13:33:16.140 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-04-02T13:33:16.140 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAcCQxmHmkhCBAATTBtVyrYQ4XZw8njGnfOhw== 2024-04-02T13:33:16.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 msgr2=0x7f5b380a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:16.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 0x7f5b380a6240 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000b580 tx=0x7f5b40005fb0 comp rx=0 tx=0).stop 2024-04-02T13:33:16.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 msgr2=0x7f5b5019fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b48007da0 tx=0x7f5b4800a340 comp rx=0 tx=0).stop 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 shutdown_connections 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5b380a3d80 0x7f5b380a6240 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b50102890 0x7f5b5019fcc0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b50100cf0 0x7f5b50198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 --2- 172.21.15.122:0/301600842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b50101ef0 0x7f5b50198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 >> 172.21.15.122:0/301600842 conn(0x7f5b500fc290 msgr2=0x7f5b50109b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 shutdown_connections 2024-04-02T13:33:16.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:16.139+0000 7f5b55b4b700 1 -- 172.21.15.122:0/301600842 wait complete. 2024-04-02T13:33:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:16 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:33:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:16 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:33:16.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:16 smithi122 ceph-mon[32114]: pgmap v301: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-02T13:33:16.947 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:33:16.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-02T13:33:16.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-02T13:33:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:16 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:33:16.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:16 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:33:16.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:16 smithi187 ceph-mon[36064]: pgmap v301: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-02T13:33:16.989 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-02T13:33:16.990 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-02T13:33:16.990 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph mgr dump --format=json 2024-04-02T13:33:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:16 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-02T13:33:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:16 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/301600842' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-02T13:33:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:16 smithi188 ceph-mon[36117]: pgmap v301: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-04-02T13:33:17.278 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 -- 172.21.15.122:0/670623471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0102fe0 msgr2=0x7fdad010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 --2- 172.21.15.122:0/670623471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0102fe0 0x7fdad010f490 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdac800b0a0 tx=0x7fdac800b3b0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 -- 172.21.15.122:0/670623471 shutdown_connections 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 --2- 172.21.15.122:0/670623471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0102fe0 0x7fdad010f490 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 --2- 172.21.15.122:0/670623471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0107520 0x7fdad0107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 --2- 172.21.15.122:0/670623471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102640 0x7fdad0102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 -- 172.21.15.122:0/670623471 >> 172.21.15.122:0/670623471 conn(0x7fdad0079720 msgr2=0x7fdad0079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 -- 172.21.15.122:0/670623471 shutdown_connections 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 -- 172.21.15.122:0/670623471 wait complete. 2024-04-02T13:33:19.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.014+0000 7fdad5815700 1 Processor -- start 2024-04-02T13:33:19.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 -- start start 2024-04-02T13:33:19.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 0x7fdad019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:19.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 0x7fdad019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdad01128f0 con 0x7fdad0107520 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0112770 con 0x7fdad0102fe0 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdad5815700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdad0112a70 con 0x7fdad0102640 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33088/0 (socket says 172.21.15.122:33088) 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 -- 172.21.15.122:0/2883801234 learned_addr learned my addr 172.21.15.122:0/2883801234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdaceffd700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 0x7fdad019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 -- 172.21.15.122:0/2883801234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 msgr2=0x7fdad019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 0x7fdad019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdace7fc700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 0x7fdad019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 -- 172.21.15.122:0/2883801234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 msgr2=0x7fdad019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:19.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 0x7fdad019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdacf7fe700 1 -- 172.21.15.122:0/2883801234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad01992b0 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdaceffd700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 0x7fdad019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.015+0000 7fdace7fc700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 0x7fdad019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdacf7fe700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac80062a0 tx=0x7fdac8004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8021070 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad0199540 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdac8007c80 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac801d450 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.016+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad01a5810 con 0x7fdad0107520 2024-04-02T13:33:19.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.017+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad010ab50 con 0x7fdad0107520 2024-04-02T13:33:19.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.018+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fdac8004000 con 0x7fdad0107520 2024-04-02T13:33:19.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.018+0000 7fdad4813700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 0x7fdab80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:19.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.019+0000 7fdaceffd700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 0x7fdab80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:19.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.019+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdac800f020 con 0x7fdad0107520 2024-04-02T13:33:19.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.019+0000 7fdaceffd700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 0x7fdab80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005e10 tx=0x7fdac0005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:19.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.022+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdac802a020 con 0x7fdad0107520 2024-04-02T13:33:19.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.276+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdad0199ff0 con 0x7fdad0107520 2024-04-02T13:33:19.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.280+0000 7fdad4813700 1 -- 172.21.15.122:0/2883801234 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+265006 (secure 0 0 0) 0x7fdac809e480 con 0x7fdad0107520 2024-04-02T13:33:19.288 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:19.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 msgr2=0x7fdab80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 0x7fdab80a6190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdac0005e10 tx=0x7fdac0005d20 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 msgr2=0x7fdad0198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdac80062a0 tx=0x7fdac8004e20 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 shutdown_connections 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdab80a3cd0 0x7fdab80a6190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdad0107520 0x7fdad0198b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad0102640 0x7fdad019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 --2- 172.21.15.122:0/2883801234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdad0102fe0 0x7fdad019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.283+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 >> 172.21.15.122:0/2883801234 conn(0x7fdad0079720 msgr2=0x7fdad0100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.284+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 shutdown_connections 2024-04-02T13:33:19.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:19.284+0000 7fdad5815700 1 -- 172.21.15.122:0/2883801234 wait complete. 2024-04-02T13:33:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:19 smithi122 ceph-mon[32114]: pgmap v302: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-02T13:33:19.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2883801234' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:33:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":3450525377},{"type":"v1","addr":"172.21.15.122:6801","nonce":3450525377}]},"active_addr":"172.21.15.122:6801/3450525377","active_change":"2024-04-02T13:25:54.372337+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.122:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1773721972}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2213768328}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3229862468}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":3032683818}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":168772214}]}]} 2024-04-02T13:33:19.912 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-02T13:33:19.912 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-02T13:33:19.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:33:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:19 smithi187 ceph-mon[36064]: pgmap v302: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-02T13:33:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2883801234' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:33:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:19 smithi188 ceph-mon[36117]: pgmap v302: 129 pgs: 129 active+clean; 623 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-02T13:33:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2883801234' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-02T13:33:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:21.548 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:21 smithi122 ceph-mon[32114]: pgmap v303: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-04-02T13:33:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 -- 172.21.15.122:0/820019251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 msgr2=0x7fc99c108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/820019251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c108a30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc99400b0d0 tx=0x7fc99400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 -- 172.21.15.122:0/820019251 shutdown_connections 2024-04-02T13:33:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/820019251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 0x7fc99c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/820019251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c108a30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/820019251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 0x7fc99c06b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 -- 172.21.15.122:0/820019251 >> 172.21.15.122:0/820019251 conn(0x7fc99c077510 msgr2=0x7fc99c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 -- 172.21.15.122:0/820019251 shutdown_connections 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.776+0000 7fc9a14d0700 1 -- 172.21.15.122:0/820019251 wait complete. 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 Processor -- start 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 -- start start 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 0x7fc99c1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 0x7fc99c101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112960 con 0x7fc99c108650 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1127e0 con 0x7fc99c06acf0 2024-04-02T13:33:21.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.777+0000 7fc9a14d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112ae0 con 0x7fc99c06b6b0 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33108/0 (socket says 172.21.15.122:33108) 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99affd700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 0x7fc99c1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 -- 172.21.15.122:0/3282179280 learned_addr learned my addr 172.21.15.122:0/3282179280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99a7fc700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 0x7fc99c101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 -- 172.21.15.122:0/3282179280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 msgr2=0x7fc99c101b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 0x7fc99c101b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 -- 172.21.15.122:0/3282179280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 msgr2=0x7fc99c1015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 0x7fc99c1015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 -- 172.21.15.122:0/3282179280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1a7330 con 0x7fc99c108650 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc99b7fe700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b550 tx=0x7fc98c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c015020 con 0x7fc99c108650 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c011d10 con 0x7fc99c108650 2024-04-02T13:33:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c010e00 con 0x7fc99c108650 2024-04-02T13:33:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1a7620 con 0x7fc99c108650 2024-04-02T13:33:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.778+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a7b10 con 0x7fc99c108650 2024-04-02T13:33:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.779+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c103d20 con 0x7fc99c108650 2024-04-02T13:33:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.781+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc98c018020 con 0x7fc99c108650 2024-04-02T13:33:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.781+0000 7fc983fff700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 0x7fc9840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.781+0000 7fc99affd700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 0x7fc9840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.781+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fc98c0d0220 con 0x7fc99c108650 2024-04-02T13:33:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.782+0000 7fc99affd700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 0x7fc9840a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc994009fd0 tx=0x7fc99400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.785+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc98c01f020 con 0x7fc99c108650 2024-04-02T13:33:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:21 smithi187 ceph-mon[36064]: pgmap v303: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-04-02T13:33:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.985+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc99c02dd30 con 0x7fc99c108650 2024-04-02T13:33:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.987+0000 7fc983fff700 1 -- 172.21.15.122:0/3282179280 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7fc98c09c970 con 0x7fc99c108650 2024-04-02T13:33:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:21.991 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 msgr2=0x7fc9840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 0x7fc9840a6190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc994009fd0 tx=0x7fc99400b040 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 msgr2=0x7fc99c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00b550 tx=0x7fc98c00b860 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 shutdown_connections 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc9840a3cd0 0x7fc9840a6190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc99c108650 0x7fc99c102070 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc99c06acf0 0x7fc99c1015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 --2- 172.21.15.122:0/3282179280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc99c06b6b0 0x7fc99c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 >> 172.21.15.122:0/3282179280 conn(0x7fc99c077510 msgr2=0x7fc99c100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 shutdown_connections 2024-04-02T13:33:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:21.989+0000 7fc9a14d0700 1 -- 172.21.15.122:0/3282179280 wait complete. 2024-04-02T13:33:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:21 smithi188 ceph-mon[36117]: pgmap v303: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.0 KiB/s wr, 24 op/s 2024-04-02T13:33:22.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:22 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3282179280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:22.787 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-02T13:33:22.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd dump --format=json 2024-04-02T13:33:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:22 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3282179280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:22 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3282179280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:23.026 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:23 smithi122 ceph-mon[32114]: pgmap v304: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-02T13:33:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:23 smithi187 ceph-mon[36064]: pgmap v304: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-02T13:33:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:23 smithi188 ceph-mon[36117]: pgmap v304: 129 pgs: 129 active+clean; 631 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.7 KiB/s wr, 24 op/s 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 -- 172.21.15.122:0/1389313232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c102670 msgr2=0x7f233c102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1389313232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c102670 0x7f233c102af0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f23300099c0 tx=0x7f2330009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 -- 172.21.15.122:0/1389313232 shutdown_connections 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1389313232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f233c103030 0x7f233c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1389313232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c108690 0x7f233c108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1389313232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c102670 0x7f233c102af0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 -- 172.21.15.122:0/1389313232 >> 172.21.15.122:0/1389313232 conn(0x7f233c0796b0 msgr2=0x7f233c079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.812+0000 7f2341ea6700 1 -- 172.21.15.122:0/1389313232 shutdown_connections 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 -- 172.21.15.122:0/1389313232 wait complete. 2024-04-02T13:33:24.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 Processor -- start 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 -- start start 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 0x7f233c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f233c108690 0x7f233c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f233c112930 con 0x7f233c103030 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f233c1127b0 con 0x7f233c108690 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.813+0000 7f2341ea6700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f233c112ab0 con 0x7f233c102670 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33130/0 (socket says 172.21.15.122:33130) 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 -- 172.21.15.122:0/1119490907 learned_addr learned my addr 172.21.15.122:0/1119490907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233b7fe700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 0x7f233c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 -- 172.21.15.122:0/1119490907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 msgr2=0x7f233c198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 0x7f233c198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 -- 172.21.15.122:0/1119490907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f233c108690 msgr2=0x7f233c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f233c108690 0x7f233c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 -- 172.21.15.122:0/1119490907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233c19ddb0 con 0x7f233c103030 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233b7fe700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 0x7f233c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f233affd700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2330009fd0 tx=0x7f2330003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330027070 con 0x7f233c103030 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233c19e040 con 0x7f233c103030 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2330004150 con 0x7f233c103030 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000f460 con 0x7f233c103030 2024-04-02T13:33:24.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.814+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f233c1a5920 con 0x7f233c103030 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.815+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233c0692b0 con 0x7f233c103030 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.816+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f233002a070 con 0x7f233c103030 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.817+0000 7f2338ff9700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 0x7f23240a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.817+0000 7f233b7fe700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 0x7f23240a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.817+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f23300d1ea0 con 0x7f233c103030 2024-04-02T13:33:24.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.817+0000 7f233b7fe700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 0x7f23240a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232c005950 tx=0x7f232c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:24.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:24.821+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2330031070 con 0x7f233c103030 2024-04-02T13:33:25.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.019+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f233c02dd30 con 0x7f233c103030 2024-04-02T13:33:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.021+0000 7f2338ff9700 1 -- 172.21.15.122:0/1119490907 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24174 (secure 0 0 0) 0x7f2330013070 con 0x7f233c103030 2024-04-02T13:33:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:33:25.025 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":90,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:32:54.914059+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 msgr2=0x7f23240a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 0x7f23240a6190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232c005950 tx=0x7f232c005f50 comp rx=0 tx=0).stop 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 msgr2=0x7f233c1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2330009fd0 tx=0x7f2330003730 comp rx=0 tx=0).stop 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 shutdown_connections 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f233c108690 0x7f233c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f233c102670 0x7f233c198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f23240a3cd0 0x7f23240a6190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 --2- 172.21.15.122:0/1119490907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f233c103030 0x7f233c1992e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 >> 172.21.15.122:0/1119490907 conn(0x7f233c0796b0 msgr2=0x7f233c101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 shutdown_connections 2024-04-02T13:33:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:25.023+0000 7f2341ea6700 1 -- 172.21.15.122:0/1119490907 wait complete. 2024-04-02T13:33:25.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:25 smithi122 ceph-mon[32114]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-02T13:33:25.788 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:25 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1119490907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:25.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.0 flush_pg_stats 2024-04-02T13:33:25.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.1 flush_pg_stats 2024-04-02T13:33:25.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.2 flush_pg_stats 2024-04-02T13:33:25.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.3 flush_pg_stats 2024-04-02T13:33:25.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.4 flush_pg_stats 2024-04-02T13:33:25.824 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.5 flush_pg_stats 2024-04-02T13:33:25.824 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.6 flush_pg_stats 2024-04-02T13:33:25.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.7 flush_pg_stats 2024-04-02T13:33:25.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.8 flush_pg_stats 2024-04-02T13:33:25.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.9 flush_pg_stats 2024-04-02T13:33:25.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.10 flush_pg_stats 2024-04-02T13:33:25.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph tell osd.11 flush_pg_stats 2024-04-02T13:33:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:25 smithi187 ceph-mon[36064]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-02T13:33:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:25 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1119490907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:25 smithi188 ceph-mon[36117]: pgmap v305: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 23 op/s 2024-04-02T13:33:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:25 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1119490907' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:33:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.647 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.647 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.648 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.648 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.648 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.649 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.649 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.868 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.888 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:26.888 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:27.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:27 smithi122 ceph-mon[32114]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-02T13:33:27.947 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:27 smithi187 ceph-mon[36064]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-02T13:33:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:27 smithi188 ceph-mon[36117]: pgmap v306: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.1 KiB/s wr, 10 op/s 2024-04-02T13:33:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:29 smithi122 ceph-mon[32114]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-02T13:33:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:29 smithi187 ceph-mon[36064]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-02T13:33:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:29 smithi188 ceph-mon[36117]: pgmap v307: 129 pgs: 129 active+clean; 640 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-04-02T13:33:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:31 smithi122 ceph-mon[32114]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-02T13:33:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:31 smithi187 ceph-mon[36064]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-02T13:33:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:31 smithi188 ceph-mon[36117]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 11 op/s 2024-04-02T13:33:33.652 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:33 smithi122 ceph-mon[32114]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-02T13:33:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:33 smithi187 ceph-mon[36064]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-02T13:33:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:33 smithi188 ceph-mon[36117]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-02T13:33:34.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 -- 172.21.15.122:0/2772875027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c1086f0 msgr2=0x7f4a5c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 --2- 172.21.15.122:0/2772875027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c108ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00b0d0 tx=0x7f4a4c00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 -- 172.21.15.122:0/2772875027 shutdown_connections 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 --2- 172.21.15.122:0/2772875027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c103060 0x7f4a5c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 --2- 172.21.15.122:0/2772875027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c108ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 --2- 172.21.15.122:0/2772875027 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1026a0 0x7f4a5c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 -- 172.21.15.122:0/2772875027 >> 172.21.15.122:0/2772875027 conn(0x7f4a5c0796c0 msgr2=0x7f4a5c079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 -- 172.21.15.122:0/2772875027 shutdown_connections 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.887+0000 7f4a61e70700 1 -- 172.21.15.122:0/2772875027 wait complete. 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 Processor -- start 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 -- start start 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c1026a0 0x7f4a5c0748f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 0x7f4a5c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:34.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c1129b0 con 0x7f4a5c103060 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c112830 con 0x7f4a5c1086f0 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a61e70700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5c112b30 con 0x7f4a5c1026a0 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 0x7f4a5c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 0x7f4a5c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57044/0 (socket says 172.21.15.122:57044) 2024-04-02T13:33:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 -- 172.21.15.122:0/296468463 learned_addr learned my addr 172.21.15.122:0/296468463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:34.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5bfff700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5b7fe700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c1026a0 0x7f4a5c0748f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 -- 172.21.15.122:0/296468463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c1026a0 msgr2=0x7f4a5c0748f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c1026a0 0x7f4a5c0748f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 -- 172.21.15.122:0/296468463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1086f0 msgr2=0x7f4a5c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5affd700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5c1a7390 con 0x7f4a5c103060 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.888+0000 7f4a5bfff700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a5c1086f0 0x7f4a5c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a5affd700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 0x7f4a5c074e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400b530 tx=0x7f4a4400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44015020 con 0x7f4a5c103060 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5c1a7680 con 0x7f4a5c103060 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a44011d10 con 0x7f4a5c103060 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44010e50 con 0x7f4a5c103060 2024-04-02T13:33:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.889+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a5c1a7c10 con 0x7f4a5c103060 2024-04-02T13:33:34.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.890+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a5c062d80 con 0x7f4a5c103060 2024-04-02T13:33:34.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.891+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4a44018020 con 0x7f4a5c103060 2024-04-02T13:33:34.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.891+0000 7f4a58ff9700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a480a3e70 0x7f4a480a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.891+0000 7f4a5b7fe700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a480a3e70 0x7f4a480a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.891+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4a440d0590 con 0x7f4a5c103060 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a58ff9700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] conn(0x7f4a480aeb30 0x7f4a480b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a44027ea0 con 0x7f4a480aeb30 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4a440d09e0 con 0x7f4a5c103060 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a5bfff700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] conn(0x7f4a480aeb30 0x7f4a480b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a5b7fe700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a480a3e70 0x7f4a480a6330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00b820 tx=0x7f4a4c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.892+0000 7f4a5bfff700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] conn(0x7f4a480aeb30 0x7f4a480b0f70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.893+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== osd.3 v2:172.21.15.122:6810/2685457026 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a44027ea0 con 0x7f4a480aeb30 2024-04-02T13:33:34.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.909+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 --> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a5c063180 con 0x7f4a480aeb30 2024-04-02T13:33:34.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.909+0000 7f4a58ff9700 1 -- 172.21.15.122:0/296468463 <== osd.3 v2:172.21.15.122:6810/2685457026 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a5c063180 con 0x7f4a480aeb30 2024-04-02T13:33:34.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.909+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] conn(0x7f4a480aeb30 msgr2=0x7f4a480b0f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.909+0000 7f4a61e70700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] conn(0x7f4a480aeb30 0x7f4a480b0f70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a480a3e70 msgr2=0x7f4a480a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4a480a3e70 0x7f4a480a6330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c00b820 tx=0x7f4a4c00b040 comp rx=0 tx=0).stop 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 msgr2=0x7f4a5c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 --2- 172.21.15.122:0/296468463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a5c103060 0x7f4a5c074e30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400b530 tx=0x7f4a4400b8f0 comp rx=0 tx=0).stop 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a5bfff700 1 -- 172.21.15.122:0/296468463 reap_dead start 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 shutdown_connections 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 >> 172.21.15.122:0/296468463 conn(0x7f4a5c0796c0 msgr2=0x7f4a5c1013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 shutdown_connections 2024-04-02T13:33:34.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:34.910+0000 7f4a61e70700 1 -- 172.21.15.122:0/296468463 wait complete. 2024-04-02T13:33:35.805 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:35 smithi122 ceph-mon[32114]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T13:33:35.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:35 smithi188 ceph-mon[36117]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T13:33:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:35 smithi187 ceph-mon[36064]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-02T13:33:37.193 INFO:teuthology.orchestra.run.smithi122.stdout:107374182462 2024-04-02T13:33:37.193 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.3 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 -- 172.21.15.122:0/306495324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 msgr2=0x7f0c88101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/306495324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c88101b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0c780099c0 tx=0x7f0c78009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 -- 172.21.15.122:0/306495324 shutdown_connections 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/306495324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 0x7f0c8810e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/306495324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c88101b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/306495324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c88102130 0x7f0c88106140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 -- 172.21.15.122:0/306495324 >> 172.21.15.122:0/306495324 conn(0x7f0c88077500 msgr2=0x7f0c88077910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 -- 172.21.15.122:0/306495324 shutdown_connections 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.466+0000 7f0c8cce9700 1 -- 172.21.15.122:0/306495324 wait complete. 2024-04-02T13:33:37.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.467+0000 7f0c8cce9700 1 Processor -- start 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.467+0000 7f0c8cce9700 1 -- start start 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c8819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c88102130 0x7f0c8819dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 0x7f0c881979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88111790 con 0x7f0c88101780 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c8819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88111610 con 0x7f0c88102130 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c88111910 con 0x7f0c88106870 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c8819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57070/0 (socket says 172.21.15.122:57070) 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 -- 172.21.15.122:0/1263764932 learned_addr learned my addr 172.21.15.122:0/1263764932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c86d9d700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 0x7f0c881979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 -- 172.21.15.122:0/1263764932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 msgr2=0x7f0c881979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:37.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 0x7f0c881979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 -- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c88102130 msgr2=0x7f0c8819dd00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c88102130 0x7f0c8819dd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c88198130 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c86d9d700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c88106870 0x7f0c881979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8659c700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c8819d7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fd0 tx=0x7f0c7801ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800bba0 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c881983c0 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c7800bd40 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7800f750 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.468+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8806ad50 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.469+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c8806b360 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.470+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f0c7802f060 con 0x7f0c88101780 2024-04-02T13:33:37.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c777fe700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0c700a3cd0 0x7f0c700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0c780d61d0 con 0x7f0c88101780 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c85d9b700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0c700a3cd0 0x7f0c700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c777fe700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] conn(0x7f0c700ae990 0x7f0c700b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c700b14a0 con 0x7f0c700ae990 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.471+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f0c780d6620 con 0x7f0c88101780 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.472+0000 7f0c86d9d700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] conn(0x7f0c700ae990 0x7f0c700b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.472+0000 7f0c85d9b700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0c700a3cd0 0x7f0c700a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0c88107d00 tx=0x7f0c7c005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:37.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.472+0000 7f0c86d9d700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] conn(0x7f0c700ae990 0x7f0c700b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.473+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== osd.7 v2:172.21.15.187:6816/3961503181 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c700b14a0 con 0x7f0c700ae990 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.490+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 --> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c8806b7e0 con 0x7f0c700ae990 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c777fe700 1 -- 172.21.15.122:0/1263764932 <== osd.7 v2:172.21.15.187:6816/3961503181 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c8806b7e0 con 0x7f0c700ae990 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] conn(0x7f0c700ae990 msgr2=0x7f0c700b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] conn(0x7f0c700ae990 0x7f0c700b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0c700a3cd0 msgr2=0x7f0c700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f0c700a3cd0 0x7f0c700a6190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0c88107d00 tx=0x7f0c7c005c10 comp rx=0 tx=0).stop 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 msgr2=0x7f0c8819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 --2- 172.21.15.122:0/1263764932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0c88101780 0x7f0c8819d7c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0c78009fd0 tx=0x7f0c7801ed80 comp rx=0 tx=0).stop 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c86d9d700 1 -- 172.21.15.122:0/1263764932 reap_dead start 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 shutdown_connections 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 >> 172.21.15.122:0/1263764932 conn(0x7f0c88077500 msgr2=0x7f0c880ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 shutdown_connections 2024-04-02T13:33:37.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:35.491+0000 7f0c8cce9700 1 -- 172.21.15.122:0/1263764932 wait complete. 2024-04-02T13:33:37.809 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:37 smithi122 ceph-mon[32114]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T13:33:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:37 smithi187 ceph-mon[36064]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T13:33:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:37 smithi188 ceph-mon[36117]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-02T13:33:38.413 INFO:teuthology.orchestra.run.smithi122.stdout:223338299434 2024-04-02T13:33:38.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.7 2024-04-02T13:33:39.773 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:39 smithi122 ceph-mon[32114]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T13:33:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:39 smithi187 ceph-mon[36064]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T13:33:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:39 smithi188 ceph-mon[36117]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-04-02T13:33:40.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 -- 172.21.15.122:0/840214903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208102fb0 msgr2=0x7fb20810f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/840214903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208102fb0 0x7fb20810f4c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb20000b0a0 tx=0x7fb20000b3b0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 -- 172.21.15.122:0/840214903 shutdown_connections 2024-04-02T13:33:40.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/840214903 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208102fb0 0x7fb20810f4c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/840214903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208108680 0x7fb208108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/840214903 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2081025f0 0x7fb208102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 -- 172.21.15.122:0/840214903 >> 172.21.15.122:0/840214903 conn(0x7fb2080fe0c0 msgr2=0x7fb2081004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:40.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 -- 172.21.15.122:0/840214903 shutdown_connections 2024-04-02T13:33:40.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.194+0000 7fb20f9a7700 1 -- 172.21.15.122:0/840214903 wait complete. 2024-04-02T13:33:40.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 Processor -- start 2024-04-02T13:33:40.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 -- start start 2024-04-02T13:33:40.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2081025f0 0x7fb20819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 0x7fb20819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208108680 0x7fb208198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112940 con 0x7fb208108680 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081127c0 con 0x7fb2081025f0 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20f9a7700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb208112ac0 con 0x7fb208102fb0 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20cf42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 0x7fb20819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20d743700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2081025f0 0x7fb20819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.195+0000 7fb20cf42700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 0x7fb20819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:45274/0 (socket says 172.21.15.122:45274) 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20df44700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208108680 0x7fb208198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 -- 172.21.15.122:0/3733997911 learned_addr learned my addr 172.21.15.122:0/3733997911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 -- 172.21.15.122:0/3733997911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2081025f0 msgr2=0x7fb20819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb2081025f0 0x7fb20819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 -- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208108680 msgr2=0x7fb208198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb208108680 0x7fb208198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb208199400 con 0x7fb208102fb0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20cf42700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 0x7fb20819ef90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c920 tx=0x7fb1f800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004a30 con 0x7fb208102fb0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1f8005500 con 0x7fb208102fb0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8007d50 con 0x7fb208102fb0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081996f0 con 0x7fb208102fb0 2024-04-02T13:33:40.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.196+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a59d0 con 0x7fb208102fb0 2024-04-02T13:33:40.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.197+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb208100010 con 0x7fb208102fb0 2024-04-02T13:33:40.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.199+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb1f8004d60 con 0x7fb208102fb0 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb1fe7fc700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1f40a3cd0 0x7fb1f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb20d743700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1f40a3cd0 0x7fb1f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb1f80d0720 con 0x7fb208102fb0 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb1fe7fc700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] conn(0x7fb1f40ae990 0x7fb1f40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1f40b14a0 con 0x7fb1f40ae990 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb1f80d0ad0 con 0x7fb208102fb0 2024-04-02T13:33:40.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb20d743700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1f40a3cd0 0x7fb1f40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb204005950 tx=0x7fb20400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.200+0000 7fb20df44700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] conn(0x7fb1f40ae990 0x7fb1f40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.201+0000 7fb20df44700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] conn(0x7fb1f40ae990 0x7fb1f40b0dd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.201+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== osd.0 v2:172.21.15.122:6802/933440597 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1f40b14a0 con 0x7fb1f40ae990 2024-04-02T13:33:40.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.218+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 --> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb208103d30 con 0x7fb1f40ae990 2024-04-02T13:33:40.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb1fe7fc700 1 -- 172.21.15.122:0/3733997911 <== osd.0 v2:172.21.15.122:6802/933440597 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb208103d30 con 0x7fb1f40ae990 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] conn(0x7fb1f40ae990 msgr2=0x7fb1f40b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] conn(0x7fb1f40ae990 0x7fb1f40b0dd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1f40a3cd0 msgr2=0x7fb1f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1f40a3cd0 0x7fb1f40a6190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb204005950 tx=0x7fb20400a300 comp rx=0 tx=0).stop 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 msgr2=0x7fb20819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 --2- 172.21.15.122:0/3733997911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb208102fb0 0x7fb20819ef90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c920 tx=0x7fb1f800cc30 comp rx=0 tx=0).stop 2024-04-02T13:33:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20df44700 1 -- 172.21.15.122:0/3733997911 reap_dead start 2024-04-02T13:33:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 shutdown_connections 2024-04-02T13:33:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.219+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 >> 172.21.15.122:0/3733997911 conn(0x7fb2080fe0c0 msgr2=0x7fb2080ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.220+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 shutdown_connections 2024-04-02T13:33:40.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.220+0000 7fb20f9a7700 1 -- 172.21.15.122:0/3733997911 wait complete. 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 -- 172.21.15.122:0/3205378365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 msgr2=0x7f8aa806b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 --2- 172.21.15.122:0/3205378365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa806b170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8a900099c0 tx=0x7f8a90009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 -- 172.21.15.122:0/3205378365 shutdown_connections 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 --2- 172.21.15.122:0/3205378365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa806b6b0 0x7f8aa810f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 --2- 172.21.15.122:0/3205378365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa8108660 0x7f8aa8108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 --2- 172.21.15.122:0/3205378365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa806b170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 -- 172.21.15.122:0/3205378365 >> 172.21.15.122:0/3205378365 conn(0x7f8aa8077540 msgr2=0x7f8aa8077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 -- 172.21.15.122:0/3205378365 shutdown_connections 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 -- 172.21.15.122:0/3205378365 wait complete. 2024-04-02T13:33:40.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.928+0000 7f8aad18d700 1 Processor -- start 2024-04-02T13:33:40.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 -- start start 2024-04-02T13:33:40.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa8101620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa806b6b0 0x7f8aa8101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa8108660 0x7f8aa81020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8112960 con 0x7f8aa806acf0 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa81127e0 con 0x7f8aa8108660 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aad18d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa8112ae0 con 0x7f8aa806b6b0 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa8101620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa8101620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45802/0 (socket says 172.21.15.122:45802) 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 -- 172.21.15.122:0/1347084504 learned_addr learned my addr 172.21.15.122:0/1347084504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8a9e59c700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa806b6b0 0x7f8aa8101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 -- 172.21.15.122:0/1347084504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa806b6b0 msgr2=0x7f8aa8101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa759e700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa8108660 0x7f8aa81020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa806b6b0 0x7f8aa8101b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 -- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa8108660 msgr2=0x7f8aa81020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa8108660 0x7f8aa81020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa81a72e0 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa759e700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa8108660 0x7f8aa81020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.929+0000 7f8aa6d9d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa8101620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b530 tx=0x7f8a9800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98015020 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa81a75d0 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a98011d10 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a98010e50 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa81a7ac0 con 0x7f8aa806acf0 2024-04-02T13:33:40.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.930+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8aa8062d00 con 0x7f8aa806acf0 2024-04-02T13:33:40.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.932+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8a98018020 con 0x7f8aa806acf0 2024-04-02T13:33:40.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.932+0000 7f8aa4d99700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8a940a3e70 0x7f8a940a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.932+0000 7f8a9e59c700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8a940a3e70 0x7f8a940a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.932+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f8a980d05f0 con 0x7f8aa806acf0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8aa4d99700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] conn(0x7f8a940aeb30 0x7f8a940b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a98010e50 con 0x7f8a940aeb30 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8a980d0a40 con 0x7f8aa806acf0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8a9e59c700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8a940a3e70 0x7f8a940a6330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006010 tx=0x7f8a90000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8aa759e700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] conn(0x7f8a940aeb30 0x7f8a940b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.933+0000 7f8aa759e700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] conn(0x7f8a940aeb30 0x7f8a940b0f70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:40.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.934+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== osd.10 v2:172.21.15.187:6824/442100564 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8a98010e50 con 0x7f8a940aeb30 2024-04-02T13:33:40.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.951+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 --> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8aa8063100 con 0x7f8a940aeb30 2024-04-02T13:33:40.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.951+0000 7f8aa4d99700 1 -- 172.21.15.122:0/1347084504 <== osd.10 v2:172.21.15.187:6824/442100564 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8aa8063100 con 0x7f8a940aeb30 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] conn(0x7f8a940aeb30 msgr2=0x7f8a940b0f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] conn(0x7f8a940aeb30 0x7f8a940b0f70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8a940a3e70 msgr2=0x7f8a940a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8a940a3e70 0x7f8a940a6330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8a90006010 tx=0x7f8a90000bc0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 msgr2=0x7f8aa8101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 --2- 172.21.15.122:0/1347084504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa806acf0 0x7f8aa8101620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b530 tx=0x7f8a9800b8f0 comp rx=0 tx=0).stop 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aa759e700 1 -- 172.21.15.122:0/1347084504 reap_dead start 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 shutdown_connections 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 >> 172.21.15.122:0/1347084504 conn(0x7f8aa8077540 msgr2=0x7f8aa81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 shutdown_connections 2024-04-02T13:33:40.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:40.952+0000 7f8aad18d700 1 -- 172.21.15.122:0/1347084504 wait complete. 2024-04-02T13:33:41.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:41 smithi122 ceph-mon[32114]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T13:33:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:41 smithi187 ceph-mon[36064]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T13:33:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:41 smithi188 ceph-mon[36117]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-02T13:33:43.718 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:43 smithi122 ceph-mon[32114]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:33:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:43 smithi187 ceph-mon[36064]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:33:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:43 smithi188 ceph-mon[36117]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:33:44.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 -- 172.21.15.122:0/342267762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341076e0 msgr2=0x7f2234107ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 --2- 172.21.15.122:0/342267762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341076e0 0x7f2234107ac0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f222400b0d0 tx=0x7f222400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 -- 172.21.15.122:0/342267762 shutdown_connections 2024-04-02T13:33:44.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 --2- 172.21.15.122:0/342267762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2234102190 0x7f223410e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 --2- 172.21.15.122:0/342267762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341076e0 0x7f2234107ac0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 --2- 172.21.15.122:0/342267762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22341017d0 0x7f2234101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 -- 172.21.15.122:0/342267762 >> 172.21.15.122:0/342267762 conn(0x7f2234078350 msgr2=0x7f2234078760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 -- 172.21.15.122:0/342267762 shutdown_connections 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.054+0000 7f2238e6d700 1 -- 172.21.15.122:0/342267762 wait complete. 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 Processor -- start 2024-04-02T13:33:44.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 -- start start 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 0x7f2234197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2234102190 0x7f2234198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22341076e0 0x7f223419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2234111860 con 0x7f22341017d0 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22341116e0 con 0x7f2234102190 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2238e6d700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22341119e0 con 0x7f22341076e0 2024-04-02T13:33:44.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 0x7f2234197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 0x7f2234197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45828/0 (socket says 172.21.15.122:45828) 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 -- 172.21.15.122:0/9681805 learned_addr learned my addr 172.21.15.122:0/9681805 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2231d9b700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2234102190 0x7f2234198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2232d9d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22341076e0 0x7f223419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 -- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22341076e0 msgr2=0x7f223419c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22341076e0 0x7f223419c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 -- 172.21.15.122:0/9681805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2234102190 msgr2=0x7f2234198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2234102190 0x7f2234198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f223259c700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223419cc50 con 0x7f22341017d0 2024-04-02T13:33:44.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2232d9d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22341076e0 0x7f223419c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.055+0000 7f2231d9b700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2234102190 0x7f2234198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f223259c700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 0x7f2234197c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f222400b820 tx=0x7f2224004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224021020 con 0x7f22341017d0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2224007d40 con 0x7f22341017d0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2224005370 con 0x7f22341017d0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223419cee0 con 0x7f22341017d0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.056+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a4800 con 0x7f22341017d0 2024-04-02T13:33:44.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.057+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2234062d00 con 0x7f22341017d0 2024-04-02T13:33:44.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.058+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2224024020 con 0x7f22341017d0 2024-04-02T13:33:44.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f222b7fe700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22200a3cd0 0x7f22200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f222400f020 con 0x7f22341017d0 2024-04-02T13:33:44.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f2231d9b700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22200a3cd0 0x7f22200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f222b7fe700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] conn(0x7f22200ae860 0x7f22200b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22200b1370 con 0x7f22200ae860 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f22240d21c0 con 0x7f22341017d0 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f2231d9b700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22200a3cd0 0x7f22200a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f221c005fd0 tx=0x7f221c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.059+0000 7f2232d9d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] conn(0x7f22200ae860 0x7f22200b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.060+0000 7f2232d9d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] conn(0x7f22200ae860 0x7f22200b0ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.061+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== osd.8 v2:172.21.15.188:6816/2378797977 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f22200b1370 con 0x7f22200ae860 2024-04-02T13:33:44.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.077+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 --> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2234063100 con 0x7f22200ae860 2024-04-02T13:33:44.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.077+0000 7f222b7fe700 1 -- 172.21.15.122:0/9681805 <== osd.8 v2:172.21.15.188:6816/2378797977 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2234063100 con 0x7f22200ae860 2024-04-02T13:33:44.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] conn(0x7f22200ae860 msgr2=0x7f22200b0ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] conn(0x7f22200ae860 0x7f22200b0ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22200a3cd0 msgr2=0x7f22200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22200a3cd0 0x7f22200a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f221c005fd0 tx=0x7f221c005e20 comp rx=0 tx=0).stop 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 msgr2=0x7f2234197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 --2- 172.21.15.122:0/9681805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341017d0 0x7f2234197c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f222400b820 tx=0x7f2224004e60 comp rx=0 tx=0).stop 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2232d9d700 1 -- 172.21.15.122:0/9681805 reap_dead start 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 shutdown_connections 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 >> 172.21.15.122:0/9681805 conn(0x7f2234078350 msgr2=0x7f2234100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 shutdown_connections 2024-04-02T13:33:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.078+0000 7f2238e6d700 1 -- 172.21.15.122:0/9681805 wait complete. 2024-04-02T13:33:44.399 INFO:teuthology.orchestra.run.smithi122.stdout:313532612636 2024-04-02T13:33:44.399 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.10 2024-04-02T13:33:44.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 -- 172.21.15.122:0/3842649236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 msgr2=0x7f4f101078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3842649236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101078e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009990 tx=0x7f4f00009ca0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 -- 172.21.15.122:0/3842649236 shutdown_connections 2024-04-02T13:33:44.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3842649236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f10101eb0 0x7f4f1010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3842649236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101078e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3842649236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f101014f0 0x7f4f10101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 -- 172.21.15.122:0/3842649236 >> 172.21.15.122:0/3842649236 conn(0x7f4f10078420 msgr2=0x7f4f10078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:44.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 -- 172.21.15.122:0/3842649236 shutdown_connections 2024-04-02T13:33:44.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.590+0000 7f4f168ea700 1 -- 172.21.15.122:0/3842649236 wait complete. 2024-04-02T13:33:44.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.591+0000 7f4f168ea700 1 Processor -- start 2024-04-02T13:33:44.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.591+0000 7f4f168ea700 1 -- start start 2024-04-02T13:33:44.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f101014f0 0x7f4f10199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f10101eb0 0x7f4f1019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10111770 con 0x7f4f10107500 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101115f0 con 0x7f4f101014f0 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f168ea700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101118f0 con 0x7f4f10101eb0 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45854/0 (socket says 172.21.15.122:45854) 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 -- 172.21.15.122:0/3733657451 learned_addr learned my addr 172.21.15.122:0/3733657451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f0f7fe700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f10101eb0 0x7f4f1019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f0ffff700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f101014f0 0x7f4f10199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 -- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f10101eb0 msgr2=0x7f4f1019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f10101eb0 0x7f4f1019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 -- 172.21.15.122:0/3733657451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f101014f0 msgr2=0x7f4f10199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f101014f0 0x7f4f10199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101a5090 con 0x7f4f10107500 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f14e87700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101a4b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f08007d70 tx=0x7f4f0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.592+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08016020 con 0x7f4f10107500 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.593+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f101a5260 con 0x7f4f10107500 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.593+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f0800ad40 con 0x7f4f10107500 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.593+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08011760 con 0x7f4f10107500 2024-04-02T13:33:44.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.593+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101a5720 con 0x7f4f10107500 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.593+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f10062d00 con 0x7f4f10107500 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.595+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f4f08019020 con 0x7f4f10107500 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.595+0000 7f4f0d7fa700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4ef80a3cd0 0x7f4ef80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.595+0000 7f4f0ffff700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4ef80a3cd0 0x7f4ef80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.595+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4f080d0c70 con 0x7f4f10107500 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0d7fa700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] conn(0x7f4ef80ae990 0x7f4ef80b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f08019ec0 con 0x7f4ef80ae990 2024-04-02T13:33:44.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4f080d10c0 con 0x7f4f10107500 2024-04-02T13:33:44.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0ffff700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4ef80a3cd0 0x7f4ef80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f0000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0f7fe700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] conn(0x7f4ef80ae990 0x7f4ef80b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:44.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.596+0000 7f4f0f7fe700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] conn(0x7f4ef80ae990 0x7f4ef80b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:44.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.597+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== osd.11 v2:172.21.15.188:6824/993879353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f08019ec0 con 0x7f4ef80ae990 2024-04-02T13:33:44.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.614+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 --> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f10063180 con 0x7f4ef80ae990 2024-04-02T13:33:44.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f0d7fa700 1 -- 172.21.15.122:0/3733657451 <== osd.11 v2:172.21.15.188:6824/993879353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f10063180 con 0x7f4ef80ae990 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] conn(0x7f4ef80ae990 msgr2=0x7f4ef80b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] conn(0x7f4ef80ae990 0x7f4ef80b0dd0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4ef80a3cd0 msgr2=0x7f4ef80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4ef80a3cd0 0x7f4ef80a6190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f0000b540 comp rx=0 tx=0).stop 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 msgr2=0x7f4f101a4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 --2- 172.21.15.122:0/3733657451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f10107500 0x7f4f101a4b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4f08007d70 tx=0x7f4f0800a340 comp rx=0 tx=0).stop 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f14e87700 1 -- 172.21.15.122:0/3733657451 reap_dead start 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 shutdown_connections 2024-04-02T13:33:44.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 >> 172.21.15.122:0/3733657451 conn(0x7f4f10078420 msgr2=0x7f4f100fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:44.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 shutdown_connections 2024-04-02T13:33:44.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.615+0000 7f4f168ea700 1 -- 172.21.15.122:0/3733657451 wait complete. 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- 172.21.15.122:0/4092575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 msgr2=0x7f228c10f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- 172.21.15.122:0/4092575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c10f450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f228400b0d0 tx=0x7f228400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- 172.21.15.122:0/4092575157 shutdown_connections 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- 172.21.15.122:0/4092575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c10f450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- 172.21.15.122:0/4092575157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228c108630 0x7f228c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- 172.21.15.122:0/4092575157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f228c06acf0 0x7f228c06b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- 172.21.15.122:0/4092575157 >> 172.21.15.122:0/4092575157 conn(0x7f228c077510 msgr2=0x7f228c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- 172.21.15.122:0/4092575157 shutdown_connections 2024-04-02T13:33:45.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- 172.21.15.122:0/4092575157 wait complete. 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 Processor -- start 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- start start 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228c06acf0 0x7f228c198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f228c108630 0x7f228c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f228c112950 con 0x7f228c06b6b0 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f228c1127d0 con 0x7f228c108630 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.818+0000 7f2291340700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f228c112ad0 con 0x7f228c06acf0 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45870/0 (socket says 172.21.15.122:45870) 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 -- 172.21.15.122:0/1671373342 learned_addr learned my addr 172.21.15.122:0/1671373342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f228affd700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228c06acf0 0x7f228c198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:45.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 -- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228c06acf0 msgr2=0x7f228c198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f228c06acf0 0x7f228c198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 -- 172.21.15.122:0/1671373342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f228c108630 msgr2=0x7f228c19d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f228c108630 0x7f228c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c19dd90 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2283fff700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c1992c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f227400b530 tx=0x7f227400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274015020 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c19e080 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2274011d10 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1a5900 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.819+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274010df0 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.820+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f228c062d80 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.821+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f2274018020 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f2288ff9700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22780a3cd0 0x7f22780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f228affd700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22780a3cd0 0x7f22780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f22740d05c0 con 0x7f228c06b6b0 2024-04-02T13:33:45.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f2288ff9700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] conn(0x7f22780ae990 0x7f22780b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2274021e50 con 0x7f22780ae990 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f22740d0a10 con 0x7f228c06b6b0 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f228affd700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22780a3cd0 0x7f22780a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f227c005950 tx=0x7f227c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.822+0000 7f228b7fe700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] conn(0x7f22780ae990 0x7f22780b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.823+0000 7f228b7fe700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] conn(0x7f22780ae990 0x7f22780b0dd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.824+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== osd.2 v2:172.21.15.188:6800/2840236580 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2274021e50 con 0x7f22780ae990 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.840+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 --> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f228c063180 con 0x7f22780ae990 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.840+0000 7f2288ff9700 1 -- 172.21.15.122:0/1671373342 <== osd.2 v2:172.21.15.188:6800/2840236580 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f228c063180 con 0x7f22780ae990 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.840+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] conn(0x7f22780ae990 msgr2=0x7f22780b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.840+0000 7f2291340700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] conn(0x7f22780ae990 0x7f22780b0dd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22780a3cd0 msgr2=0x7f22780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f22780a3cd0 0x7f22780a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f227c005950 tx=0x7f227c0058e0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 msgr2=0x7f228c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 --2- 172.21.15.122:0/1671373342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c06b6b0 0x7f228c1992c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f227400b530 tx=0x7f227400b8f0 comp rx=0 tx=0).stop 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f228b7fe700 1 -- 172.21.15.122:0/1671373342 reap_dead start 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 shutdown_connections 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 >> 172.21.15.122:0/1671373342 conn(0x7f228c077510 msgr2=0x7f228c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 shutdown_connections 2024-04-02T13:33:45.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:44.841+0000 7f2291340700 1 -- 172.21.15.122:0/1671373342 wait complete. 2024-04-02T13:33:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:45 smithi122 ceph-mon[32114]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:45.881 INFO:teuthology.orchestra.run.smithi122.stdout:77309411397 2024-04-02T13:33:45.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.2 2024-04-02T13:33:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:45 smithi187 ceph-mon[36064]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:45 smithi188 ceph-mon[36117]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:46.484 INFO:teuthology.orchestra.run.smithi122.stdout:347892350999 2024-04-02T13:33:46.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.11 2024-04-02T13:33:46.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 -- 172.21.15.122:0/2702099348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06ab30 msgr2=0x7f62ac06afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 --2- 172.21.15.122:0/2702099348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06ab30 0x7f62ac06afb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f62940099c0 tx=0x7f6294009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 -- 172.21.15.122:0/2702099348 shutdown_connections 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 --2- 172.21.15.122:0/2702099348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06b4f0 0x7f62ac10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 --2- 172.21.15.122:0/2702099348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62ac108630 0x7f62ac108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 --2- 172.21.15.122:0/2702099348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06ab30 0x7f62ac06afb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 -- 172.21.15.122:0/2702099348 >> 172.21.15.122:0/2702099348 conn(0x7f62ac077540 msgr2=0x7f62ac077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 -- 172.21.15.122:0/2702099348 shutdown_connections 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 -- 172.21.15.122:0/2702099348 wait complete. 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.934+0000 7f62b207c700 1 Processor -- start 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 -- start start 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06ab30 0x7f62ac1015f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 0x7f62ac101b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62ac108630 0x7f62ac102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac112930 con 0x7f62ac06b4f0 2024-04-02T13:33:46.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac1127b0 con 0x7f62ac06ab30 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62b207c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac112ab0 con 0x7f62ac108630 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 0x7f62ac101b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 0x7f62ac101b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45888/0 (socket says 172.21.15.122:45888) 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 -- 172.21.15.122:0/390226225 learned_addr learned my addr 172.21.15.122:0/390226225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62ab7fe700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06ab30 0x7f62ac1015f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62abfff700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62ac108630 0x7f62ac102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 -- 172.21.15.122:0/390226225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62ac108630 msgr2=0x7f62ac102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62ac108630 0x7f62ac102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 -- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06ab30 msgr2=0x7f62ac1015f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62ac06ab30 0x7f62ac1015f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.935+0000 7f62aaffd700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac1a7390 con 0x7f62ac06b4f0 2024-04-02T13:33:46.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62aaffd700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 0x7f62ac101b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f62940059d0 tx=0x7f62940038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:46.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294027070 con 0x7f62ac06b4f0 2024-04-02T13:33:46.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac1a7620 con 0x7f62ac06b4f0 2024-04-02T13:33:46.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6294003e40 con 0x7f62ac06b4f0 2024-04-02T13:33:46.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6294022d00 con 0x7f62ac06b4f0 2024-04-02T13:33:46.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1a7a70 con 0x7f62ac06b4f0 2024-04-02T13:33:46.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.936+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62ac062d00 con 0x7f62ac06b4f0 2024-04-02T13:33:46.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.938+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f629402a070 con 0x7f62ac06b4f0 2024-04-02T13:33:46.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.938+0000 7f62a8ff9700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f62980a3cd0 0x7f62980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:46.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62ab7fe700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f62980a3cd0 0x7f62980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:46.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f62940d2420 con 0x7f62ac06b4f0 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62a8ff9700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] conn(0x7f62980ae990 0x7f62980b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62940095d0 con 0x7f62980ae990 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f62940d2870 con 0x7f62ac06b4f0 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62ab7fe700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f62980a3cd0 0x7f62980a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f629c007be0 tx=0x7f629c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.939+0000 7f62abfff700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] conn(0x7f62980ae990 0x7f62980b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:46.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.940+0000 7f62abfff700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] conn(0x7f62980ae990 0x7f62980b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:46.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.942+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== osd.1 v2:172.21.15.187:6800/2690363076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62940095d0 con 0x7f62980ae990 2024-04-02T13:33:46.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.975+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 --> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62ac063100 con 0x7f62980ae990 2024-04-02T13:33:46.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.976+0000 7f62a8ff9700 1 -- 172.21.15.122:0/390226225 <== osd.1 v2:172.21.15.187:6800/2690363076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62ac063100 con 0x7f62980ae990 2024-04-02T13:33:46.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.976+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] conn(0x7f62980ae990 msgr2=0x7f62980b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.976+0000 7f62b207c700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] conn(0x7f62980ae990 0x7f62980b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f62980a3cd0 msgr2=0x7f62980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f62980a3cd0 0x7f62980a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f629c007be0 tx=0x7f629c0073d0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 msgr2=0x7f62ac101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 --2- 172.21.15.122:0/390226225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62ac06b4f0 0x7f62ac101b30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f62940059d0 tx=0x7f62940038b0 comp rx=0 tx=0).stop 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62abfff700 1 -- 172.21.15.122:0/390226225 reap_dead start 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 shutdown_connections 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 >> 172.21.15.122:0/390226225 conn(0x7f62ac077540 msgr2=0x7f62ac1002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 shutdown_connections 2024-04-02T13:33:46.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:46.977+0000 7f62b207c700 1 -- 172.21.15.122:0/390226225 wait complete. 2024-04-02T13:33:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- 172.21.15.122:0/3749094612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24108690 msgr2=0x7f3b24108a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/3749094612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24108690 0x7f3b24108a70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3b140099c0 tx=0x7f3b14009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- 172.21.15.122:0/3749094612 shutdown_connections 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/3749094612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24103030 0x7f3b2410f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/3749094612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24108690 0x7f3b24108a70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/3749094612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b24102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- 172.21.15.122:0/3749094612 >> 172.21.15.122:0/3749094612 conn(0x7f3b240796b0 msgr2=0x7f3b24079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- 172.21.15.122:0/3749094612 shutdown_connections 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- 172.21.15.122:0/3749094612 wait complete. 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 Processor -- start 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.543+0000 7f3b29cd5700 1 -- start start 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b2419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24103030 0x7f3b2419ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24108690 0x7f3b24198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b241128f0 con 0x7f3b24103030 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24112770 con 0x7f3b24102670 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b29cd5700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24112a70 con 0x7f3b24108690 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b2419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24103030 0x7f3b2419ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b237fe700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b2419e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49444/0 (socket says 172.21.15.122:49444) 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b237fe700 1 -- 172.21.15.122:0/2518266503 learned_addr learned my addr 172.21.15.122:0/2518266503 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b23fff700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24108690 0x7f3b24198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 -- 172.21.15.122:0/2518266503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24108690 msgr2=0x7f3b24198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b24108690 0x7f3b24198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 -- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 msgr2=0x7f3b2419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b2419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b24199230 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b237fe700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b24102670 0x7f3b2419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.544+0000 7f3b22ffd700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24103030 0x7f3b2419ee50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18016070 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b24199520 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b1800fd70 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1800e3b0 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b241a57e0 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.545+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b24062d80 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.547+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3b18019070 con 0x7f3b24103030 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b20ff9700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3b0c0a3cd0 0x7f3b0c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b237fe700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3b0c0a3cd0 0x7f3b0c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f3b180d06b0 con 0x7f3b24103030 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b20ff9700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] conn(0x7f3b0c0ae990 0x7f3b0c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b0c0b14a0 con 0x7f3b0c0ae990 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.548+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3b180d0b00 con 0x7f3b24103030 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.549+0000 7f3b23fff700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] conn(0x7f3b0c0ae990 0x7f3b0c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.549+0000 7f3b237fe700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3b0c0a3cd0 0x7f3b0c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009750 tx=0x7f3b14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.549+0000 7f3b23fff700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] conn(0x7f3b0c0ae990 0x7f3b0c0b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.550+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== osd.4 v2:172.21.15.187:6808/101695590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b0c0b14a0 con 0x7f3b0c0ae990 2024-04-02T13:33:47.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.566+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 --> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b24063200 con 0x7f3b0c0ae990 2024-04-02T13:33:47.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b20ff9700 1 -- 172.21.15.122:0/2518266503 <== osd.4 v2:172.21.15.187:6808/101695590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b24063200 con 0x7f3b0c0ae990 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] conn(0x7f3b0c0ae990 msgr2=0x7f3b0c0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] conn(0x7f3b0c0ae990 0x7f3b0c0b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3b0c0a3cd0 msgr2=0x7f3b0c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3b0c0a3cd0 0x7f3b0c0a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3b14009750 tx=0x7f3b14005960 comp rx=0 tx=0).stop 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24103030 msgr2=0x7f3b2419ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:47.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 --2- 172.21.15.122:0/2518266503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b24103030 0x7f3b2419ee50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b1800d950 tx=0x7f3b1800dc60 comp rx=0 tx=0).stop 2024-04-02T13:33:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b23fff700 1 -- 172.21.15.122:0/2518266503 reap_dead start 2024-04-02T13:33:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 shutdown_connections 2024-04-02T13:33:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 >> 172.21.15.122:0/2518266503 conn(0x7f3b240796b0 msgr2=0x7f3b24101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.567+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 shutdown_connections 2024-04-02T13:33:47.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:47.568+0000 7f3b29cd5700 1 -- 172.21.15.122:0/2518266503 wait complete. 2024-04-02T13:33:47.755 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:47 smithi122 ceph-mon[32114]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:33:47.942 INFO:teuthology.orchestra.run.smithi122.stdout:34359738446 2024-04-02T13:33:47.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.0 2024-04-02T13:33:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:47 smithi187 ceph-mon[36064]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:33:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:47 smithi188 ceph-mon[36117]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:33:49.159 INFO:teuthology.orchestra.run.smithi122.stdout:141733920828 2024-04-02T13:33:49.160 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.4 2024-04-02T13:33:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:49 smithi122 ceph-mon[32114]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:33:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:49 smithi187 ceph-mon[36064]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:33:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:49 smithi188 ceph-mon[36117]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:33:50.022 INFO:teuthology.orchestra.run.smithi122.stdout:55834574922 2024-04-02T13:33:50.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.1 2024-04-02T13:33:50.492 INFO:teuthology.orchestra.run.smithi122.stdout:257698037799 2024-04-02T13:33:50.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.8 2024-04-02T13:33:50.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 -- 172.21.15.122:0/23183592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 msgr2=0x7f89f4108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/23183592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4108a40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009990 tx=0x7f89e4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 -- 172.21.15.122:0/23183592 shutdown_connections 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/23183592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89f4102f90 0x7f89f410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/23183592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4108a40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/23183592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f41025d0 0x7f89f4102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 -- 172.21.15.122:0/23183592 >> 172.21.15.122:0/23183592 conn(0x7f89f40fe0a0 msgr2=0x7f89f41004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 -- 172.21.15.122:0/23183592 shutdown_connections 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.676+0000 7f89f9bd3700 1 -- 172.21.15.122:0/23183592 wait complete. 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 Processor -- start 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 -- start start 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89f41025d0 0x7f89f419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f4102f90 0x7f89f419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:50.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4112920 con 0x7f89f4108660 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89f41127a0 con 0x7f89f4102f90 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f9bd3700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89f4112aa0 con 0x7f89f41025d0 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.677+0000 7f89f3fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60066/0 (socket says 172.21.15.122:60066) 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 -- 172.21.15.122:0/1137068128 learned_addr learned my addr 172.21.15.122:0/1137068128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f2ffd700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f4102f90 0x7f89f419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f37fe700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89f41025d0 0x7f89f419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:50.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 -- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89f41025d0 msgr2=0x7f89f419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89f41025d0 0x7f89f419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 -- 172.21.15.122:0/1137068128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f4102f90 msgr2=0x7f89f419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f89f4102f90 0x7f89f419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f41991e0 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f3fff700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4198a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007ee0 tx=0x7f89ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00dbd0 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f41994d0 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89ec00dd70 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ec005a60 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.678+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f41a5740 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.679+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f89f40fff30 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.680+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f89ec01a020 con 0x7f89f4108660 2024-04-02T13:33:50.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f0ff9700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89dc0a3cd0 0x7f89dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f37fe700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89dc0a3cd0 0x7f89dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f89ec0d05e0 con 0x7f89f4108660 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f0ff9700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] conn(0x7f89dc0ae990 0x7f89dc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f89ec012ec0 con 0x7f89dc0ae990 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f89ec0d0a30 con 0x7f89f4108660 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f37fe700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89dc0a3cd0 0x7f89dc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.681+0000 7f89f2ffd700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] conn(0x7f89dc0ae990 0x7f89dc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.682+0000 7f89f2ffd700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] conn(0x7f89dc0ae990 0x7f89dc0b0dd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.683+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== osd.5 v2:172.21.15.188:6808/1528823096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f89ec012ec0 con 0x7f89dc0ae990 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.699+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 --> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f89f4103d10 con 0x7f89dc0ae990 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f0ff9700 1 -- 172.21.15.122:0/1137068128 <== osd.5 v2:172.21.15.188:6808/1528823096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f89f4103d10 con 0x7f89dc0ae990 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] conn(0x7f89dc0ae990 msgr2=0x7f89dc0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] conn(0x7f89dc0ae990 0x7f89dc0b0dd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89dc0a3cd0 msgr2=0x7f89dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89dc0a3cd0 0x7f89dc0a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e400b540 comp rx=0 tx=0).stop 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 msgr2=0x7f89f4198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 --2- 172.21.15.122:0/1137068128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89f4108660 0x7f89f4198a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89ec007ee0 tx=0x7f89ec00d3b0 comp rx=0 tx=0).stop 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f3fff700 1 -- 172.21.15.122:0/1137068128 reap_dead start 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 shutdown_connections 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 >> 172.21.15.122:0/1137068128 conn(0x7f89f40fe0a0 msgr2=0x7f89f40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 shutdown_connections 2024-04-02T13:33:50.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:50.700+0000 7f89f9bd3700 1 -- 172.21.15.122:0/1137068128 wait complete. 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 -- 172.21.15.122:0/2152925301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 msgr2=0x7f7b90105950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 --2- 172.21.15.122:0/2152925301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 0x7f7b90105950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b840099c0 tx=0x7f7b84009cd0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 -- 172.21.15.122:0/2152925301 shutdown_connections 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 --2- 172.21.15.122:0/2152925301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b90106030 0x7f7b9010dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 --2- 172.21.15.122:0/2152925301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 --2- 172.21.15.122:0/2152925301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 0x7f7b90105950 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 -- 172.21.15.122:0/2152925301 >> 172.21.15.122:0/2152925301 conn(0x7f7b90077af0 msgr2=0x7f7b900feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 -- 172.21.15.122:0/2152925301 shutdown_connections 2024-04-02T13:33:51.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.366+0000 7f7b964f1700 1 -- 172.21.15.122:0/2152925301 wait complete. 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 Processor -- start 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 -- start start 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 0x7f7b9019a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b90106030 0x7f7b901a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90111890 con 0x7f7b90101900 2024-04-02T13:33:51.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90111710 con 0x7f7b90106030 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b964f1700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b90111a10 con 0x7f7b90100f50 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b8f7fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 0x7f7b9019a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.367+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:56684/0 (socket says 172.21.15.122:56684) 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 -- 172.21.15.122:0/4245627403 learned_addr learned my addr 172.21.15.122:0/4245627403 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b94a8e700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b90106030 0x7f7b901a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 -- 172.21.15.122:0/4245627403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b90106030 msgr2=0x7f7b901a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b90106030 0x7f7b901a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 -- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 msgr2=0x7f7b9019a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b90101900 0x7f7b9019a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b901a5230 con 0x7f7b90100f50 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8ffff700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90199c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000d980 tx=0x7f7b8000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:51.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80016070 con 0x7f7b90100f50 2024-04-02T13:33:51.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b80004d10 con 0x7f7b90100f50 2024-04-02T13:33:51.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b90197ac0 con 0x7f7b90100f50 2024-04-02T13:33:51.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b80005670 con 0x7f7b90100f50 2024-04-02T13:33:51.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.368+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b90197fd0 con 0x7f7b90100f50 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.369+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7b90062d00 con 0x7f7b90100f50 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.370+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f7b80004750 con 0x7f7b90100f50 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8d7fa700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7b780a3cd0 0x7f7b780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8f7fe700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7b780a3cd0 0x7f7b780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7b800d0c40 con 0x7f7b90100f50 2024-04-02T13:33:51.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8d7fa700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] conn(0x7f7b780ae990 0x7f7b780b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b80015ec0 con 0x7f7b780ae990 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.371+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f7b800d0ff0 con 0x7f7b90100f50 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.372+0000 7f7b94a8e700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] conn(0x7f7b780ae990 0x7f7b780b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.372+0000 7f7b8f7fe700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7b780a3cd0 0x7f7b780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b84000c00 tx=0x7f7b84005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.372+0000 7f7b94a8e700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] conn(0x7f7b780ae990 0x7f7b780b0dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:51.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.373+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== osd.6 v2:172.21.15.122:6818/2482601616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7b80015ec0 con 0x7f7b780ae990 2024-04-02T13:33:51.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.392+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 --> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7b90063100 con 0x7f7b780ae990 2024-04-02T13:33:51.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.392+0000 7f7b8d7fa700 1 -- 172.21.15.122:0/4245627403 <== osd.6 v2:172.21.15.122:6818/2482601616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7b90063100 con 0x7f7b780ae990 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] conn(0x7f7b780ae990 msgr2=0x7f7b780b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] conn(0x7f7b780ae990 0x7f7b780b0dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7b780a3cd0 msgr2=0x7f7b780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7b780a3cd0 0x7f7b780a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b84000c00 tx=0x7f7b84005fb0 comp rx=0 tx=0).stop 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 msgr2=0x7f7b90199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b964f1700 1 --2- 172.21.15.122:0/4245627403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b90100f50 0x7f7b90199c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7b8000d980 tx=0x7f7b8000dc90 comp rx=0 tx=0).stop 2024-04-02T13:33:51.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.393+0000 7f7b94a8e700 1 -- 172.21.15.122:0/4245627403 reap_dead start 2024-04-02T13:33:51.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.394+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 shutdown_connections 2024-04-02T13:33:51.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.394+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 >> 172.21.15.122:0/4245627403 conn(0x7f7b90077af0 msgr2=0x7f7b90104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:51.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.395+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 shutdown_connections 2024-04-02T13:33:51.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:51.395+0000 7f7b964f1700 1 -- 172.21.15.122:0/4245627403 wait complete. 2024-04-02T13:33:51.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:51 smithi188 ceph-mon[36117]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:51.840 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:51 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:51.843 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:51 smithi122 ceph-mon[32114]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:51.843 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:51 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:51.852 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:51 smithi187 ceph-mon[36064]: pgmap v318: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:33:51.852 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:51 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:33:52.140 INFO:teuthology.orchestra.run.smithi122.stdout:167503724600 2024-04-02T13:33:52.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.5 2024-04-02T13:33:52.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 -- 172.21.15.122:0/3574276741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9806b4f0 msgr2=0x7eff9810f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 --2- 172.21.15.122:0/3574276741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9806b4f0 0x7eff9810f450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7eff90009f90 tx=0x7eff9000b330 comp rx=0 tx=0).stop 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 -- 172.21.15.122:0/3574276741 shutdown_connections 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 --2- 172.21.15.122:0/3574276741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff9806b4f0 0x7eff9810f450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 --2- 172.21.15.122:0/3574276741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff98108630 0x7eff98108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 --2- 172.21.15.122:0/3574276741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806ab30 0x7eff9806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 -- 172.21.15.122:0/3574276741 >> 172.21.15.122:0/3574276741 conn(0x7eff98077540 msgr2=0x7eff98077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 -- 172.21.15.122:0/3574276741 shutdown_connections 2024-04-02T13:33:52.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.336+0000 7eff9cf29700 1 -- 172.21.15.122:0/3574276741 wait complete. 2024-04-02T13:33:52.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.337+0000 7eff9cf29700 1 Processor -- start 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.337+0000 7eff9cf29700 1 -- start start 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 0x7eff9819f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806b4f0 0x7eff9819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff98108630 0x7eff9810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff981128d0 con 0x7eff9806ab30 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff98112750 con 0x7eff9806b4f0 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9cf29700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff98112a50 con 0x7eff98108630 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 0x7eff9819f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 0x7eff9819f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60104/0 (socket says 172.21.15.122:60104) 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 -- 172.21.15.122:0/2546614500 learned_addr learned my addr 172.21.15.122:0/2546614500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff95d9b700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806b4f0 0x7eff9819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 -- 172.21.15.122:0/2546614500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff98108630 msgr2=0x7eff9810d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff96d9d700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff98108630 0x7eff9810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff98108630 0x7eff9810d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 -- 172.21.15.122:0/2546614500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806b4f0 msgr2=0x7eff9819fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806b4f0 0x7eff9819fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff9659c700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9810db60 con 0x7eff9806ab30 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.338+0000 7eff95d9b700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff9806b4f0 0x7eff9819fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff9659c700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 0x7eff9819f8b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff8800ec10 tx=0x7eff8800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:52.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8800cd40 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff88004500 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88010690 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff9cf29700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9810de50 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.339+0000 7eff9cf29700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9819b540 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.340+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff7c001030 con 0x7eff9806ab30 2024-04-02T13:33:52.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.341+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7eff88003f10 con 0x7eff9806ab30 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.342+0000 7eff877fe700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7eff800a3d50 0x7eff800a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.342+0000 7eff95d9b700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7eff800a3d50 0x7eff800a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.342+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7eff880d0d30 con 0x7eff9806ab30 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.342+0000 7eff95d9b700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7eff800a3d50 0x7eff800a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff981034a0 tx=0x7eff8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.342+0000 7eff877fe700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] conn(0x7eff800aea10 0x7eff800b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.343+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff88011e50 con 0x7eff800aea10 2024-04-02T13:33:52.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.343+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7eff880d1180 con 0x7eff9806ab30 2024-04-02T13:33:52.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.343+0000 7eff96d9d700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] conn(0x7eff800aea10 0x7eff800b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:52.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.343+0000 7eff96d9d700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] conn(0x7eff800aea10 0x7eff800b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:52.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.344+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== osd.9 v2:172.21.15.122:6826/3872606263 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff88011e50 con 0x7eff800aea10 2024-04-02T13:33:52.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.361+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 --> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff7c002de0 con 0x7eff800aea10 2024-04-02T13:33:52.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.361+0000 7eff877fe700 1 -- 172.21.15.122:0/2546614500 <== osd.9 v2:172.21.15.122:6826/3872606263 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7eff7c002de0 con 0x7eff800aea10 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] conn(0x7eff800aea10 msgr2=0x7eff800b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] conn(0x7eff800aea10 0x7eff800b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7eff800a3d50 msgr2=0x7eff800a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7eff800a3d50 0x7eff800a6210 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff981034a0 tx=0x7eff8c009260 comp rx=0 tx=0).stop 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 msgr2=0x7eff9819f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 --2- 172.21.15.122:0/2546614500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff9806ab30 0x7eff9819f8b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7eff8800ec10 tx=0x7eff8800ef20 comp rx=0 tx=0).stop 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff96d9d700 1 -- 172.21.15.122:0/2546614500 reap_dead start 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 shutdown_connections 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 >> 172.21.15.122:0/2546614500 conn(0x7eff98077540 msgr2=0x7eff98101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 shutdown_connections 2024-04-02T13:33:52.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:52.362+0000 7eff857fa700 1 -- 172.21.15.122:0/2546614500 wait complete. 2024-04-02T13:33:53.146 INFO:teuthology.orchestra.run.smithi122.stdout:292057776164 2024-04-02T13:33:53.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.9 2024-04-02T13:33:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:53 smithi122 ceph-mon[32114]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-02T13:33:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:53 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.801 INFO:teuthology.orchestra.run.smithi122.stdout:201863462962 2024-04-02T13:33:53.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph osd last-stat-seq osd.6 2024-04-02T13:33:53.887 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.891 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.892 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.898 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.901 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:33:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:53 smithi187 ceph-mon[36064]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-02T13:33:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:53 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:53 smithi188 ceph-mon[36117]: pgmap v319: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s 2024-04-02T13:33:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:53 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:33:54.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-04-02T13:33:54.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:33:54.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-04-02T13:33:54.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-02T13:33:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-04-02T13:33:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-04-02T13:33:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-02T13:33:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]: dispatch 2024-04-02T13:33:55.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-02T13:33:55.798 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-04-02T13:33:55.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:33:55.799 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:55 smithi122 ceph-mon[32114]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:33:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:33:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:55 smithi187 ceph-mon[36064]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: pgmap v320: 129 pgs: 129 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-02T13:33:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-04-02T13:33:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-04-02T13:33:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-02T13:33:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3b", "id": [3, 9]}]': finished 2024-04-02T13:33:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-02T13:33:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:55 smithi188 ceph-mon[36117]: osdmap e91: 12 total, 12 up, 12 in 2024-04-02T13:33:56.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:56 smithi122 ceph-mon[32114]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:33:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:56 smithi187 ceph-mon[36064]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:33:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:56 smithi188 ceph-mon[36117]: osdmap e92: 12 total, 12 up, 12 in 2024-04-02T13:33:57.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:57 smithi122 ceph-mon[32114]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:33:57.816 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:57 smithi122 ceph-mon[32114]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T13:33:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:57 smithi187 ceph-mon[36064]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:33:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:57 smithi187 ceph-mon[36064]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T13:33:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:57 smithi188 ceph-mon[36117]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:33:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:57 smithi188 ceph-mon[36117]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-02T13:33:59.742 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:33:59 smithi122 ceph-mon[32114]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:33:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 -- 172.21.15.122:0/1829693892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128107520 msgr2=0x7f3128107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 --2- 172.21.15.122:0/1829693892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128107520 0x7f3128107900 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3118009990 tx=0x7f3118009ca0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 -- 172.21.15.122:0/1829693892 shutdown_connections 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 --2- 172.21.15.122:0/1829693892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f312810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 --2- 172.21.15.122:0/1829693892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128107520 0x7f3128107900 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 --2- 172.21.15.122:0/1829693892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128102640 0x7f3128102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 -- 172.21.15.122:0/1829693892 >> 172.21.15.122:0/1829693892 conn(0x7f3128079720 msgr2=0x7f3128079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.839+0000 7f312d7a0700 1 -- 172.21.15.122:0/1829693892 shutdown_connections 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.840+0000 7f312d7a0700 1 -- 172.21.15.122:0/1829693892 wait complete. 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.840+0000 7f312d7a0700 1 Processor -- start 2024-04-02T13:33:59.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.840+0000 7f312d7a0700 1 -- start start 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f3128199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 0x7f312819d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112990 con 0x7f3128102640 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112810 con 0x7f3128102fe0 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3128112b10 con 0x7f3128107520 2024-04-02T13:33:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50878/0 (socket says 172.21.15.122:50878) 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 -- 172.21.15.122:0/341989672 learned_addr learned my addr 172.21.15.122:0/341989672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f31277fe700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 0x7f312819d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f31267fc700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f3128199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 -- 172.21.15.122:0/341989672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 msgr2=0x7f312819d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 0x7f312819d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 -- 172.21.15.122:0/341989672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 msgr2=0x7f3128199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f3128199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 -- 172.21.15.122:0/341989672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312819def0 con 0x7f3128102640 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f31277fe700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 0x7f312819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f31267fc700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f3128199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f3126ffd700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118027070 con 0x7f3128102640 2024-04-02T13:33:59.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312819e180 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3118004110 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.841+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118022dd0 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.842+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31281a5a60 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.843+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f312810ab50 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.844+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f3118005030 con 0x7f3128102640 2024-04-02T13:33:59.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.844+0000 7f310ffff700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 0x7f31100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:33:59.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.844+0000 7f31267fc700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 0x7f31100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:33:59.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.844+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f31180d1e00 con 0x7f3128102640 2024-04-02T13:33:59.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.845+0000 7f31267fc700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 0x7f31100a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3128079e40 tx=0x7f311c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:33:59.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:33:59.848+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3118031020 con 0x7f3128102640 2024-04-02T13:33:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:33:59 smithi187 ceph-mon[36064]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:34:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:33:59 smithi188 ceph-mon[36117]: pgmap v324: 129 pgs: 2 peering, 127 active+clean; 644 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s; 0 B/s, 0 objects/s recovering 2024-04-02T13:34:00.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.055+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f312819a240 con 0x7f3128102640 2024-04-02T13:34:00.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.056+0000 7f310ffff700 1 -- 172.21.15.122:0/341989672 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3118013070 con 0x7f3128102640 2024-04-02T13:34:00.059 INFO:teuthology.orchestra.run.smithi122.stdout:347892351002 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 -- 172.21.15.122:0/1331694878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688103030 msgr2=0x7f068810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1331694878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688103030 0x7f068810f460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f068000b0d0 tx=0x7f068000b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 -- 172.21.15.122:0/1331694878 shutdown_connections 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1331694878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688103030 0x7f068810f460 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1331694878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 0x7f0688108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1331694878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688102670 0x7f0688102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 -- 172.21.15.122:0/1331694878 >> 172.21.15.122:0/1331694878 conn(0x7f06880796b0 msgr2=0x7f0688079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 -- 172.21.15.122:0/1331694878 shutdown_connections 2024-04-02T13:34:00.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.057+0000 7f068ea0c700 1 -- 172.21.15.122:0/1331694878 wait complete. 2024-04-02T13:34:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f068ea0c700 1 Processor -- start 2024-04-02T13:34:00.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f068ea0c700 1 -- start start 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 msgr2=0x7f31100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 0x7f31100a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3128079e40 tx=0x7f311c008040 comp rx=0 tx=0).stop 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 msgr2=0x7f3128198ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3118000c00 tx=0x7f3118005590 comp rx=0 tx=0).stop 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 shutdown_connections 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3128107520 0x7f312819d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f31100a3cd0 0x7f31100a6190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3128102640 0x7f3128198ee0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 --2- 172.21.15.122:0/341989672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3128102fe0 0x7f3128199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.058+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 >> 172.21.15.122:0/341989672 conn(0x7f3128079720 msgr2=0x7f31281007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 0x7f068819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 0x7f0688198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06881128e0 con 0x7f0688102670 2024-04-02T13:34:00.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0688112760 con 0x7f0688103030 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068ea0c700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0688112a60 con 0x7f0688108690 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50884/0 (socket says 172.21.15.122:50884) 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 -- 172.21.15.122:0/1933429686 learned_addr learned my addr 172.21.15.122:0/1933429686 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 shutdown_connections 2024-04-02T13:34:00.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f068cfa9700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 0x7f0688198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:00.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f312d7a0700 1 -- 172.21.15.122:0/341989672 wait complete. 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f06877fe700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 0x7f068819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 -- 172.21.15.122:0/1933429686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 msgr2=0x7f0688198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 0x7f0688198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 -- 172.21.15.122:0/1933429686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 msgr2=0x7f068819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 0x7f068819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 -- 172.21.15.122:0/1933429686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0688199160 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f06877fe700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 0x7f068819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.059+0000 7f0687fff700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f067800c9b0 tx=0x7f067800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.060+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678007b00 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.060+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678007ca0 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.060+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0688199450 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.060+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06881a5710 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.060+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678018730 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.061+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06880692b0 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.063+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f06780188d0 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.063+0000 7f06857fa700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 0x7f06700aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.064+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f06780d0a70 con 0x7f0688102670 2024-04-02T13:34:00.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.064+0000 7f06877fe700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 0x7f06700aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:00.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.064+0000 7f06877fe700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 0x7f06700aa620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f067c005fd0 tx=0x7f067c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:00.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.068+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0678014030 con 0x7f0688102670 2024-04-02T13:34:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.272+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f0688199f30 con 0x7f0688102670 2024-04-02T13:34:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.272+0000 7f06857fa700 1 -- 172.21.15.122:0/1933429686 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f067809d0a0 con 0x7f0688102670 2024-04-02T13:34:00.276 INFO:teuthology.orchestra.run.smithi122.stdout:313532612639 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 msgr2=0x7f06700aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 0x7f06700aa620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f067c005fd0 tx=0x7f067c005f60 comp rx=0 tx=0).stop 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 msgr2=0x7f068819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f067800c9b0 tx=0x7f067800ccc0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 shutdown_connections 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.274+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0688108690 0x7f0688198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f06700a8160 0x7f06700aa620 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0688102670 0x7f068819e840 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 --2- 172.21.15.122:0/1933429686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0688103030 0x7f068819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 >> 172.21.15.122:0/1933429686 conn(0x7f06880796b0 msgr2=0x7f0688101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 shutdown_connections 2024-04-02T13:34:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:00.275+0000 7f068ea0c700 1 -- 172.21.15.122:0/1933429686 wait complete. 2024-04-02T13:34:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:00.597 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:00.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/341989672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:34:00.683 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1933429686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:34:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/341989672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:34:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1933429686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:34:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/341989672' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-02T13:34:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1933429686' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-02T13:34:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:01 smithi122 ceph-mon[32114]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:01 smithi122 ceph-mon[32114]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-02T13:34:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:01 smithi122 ceph-mon[32114]: Cluster is now healthy 2024-04-02T13:34:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:01 smithi187 ceph-mon[36064]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:01 smithi187 ceph-mon[36064]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-02T13:34:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:01 smithi187 ceph-mon[36064]: Cluster is now healthy 2024-04-02T13:34:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:01 smithi188 ceph-mon[36117]: pgmap v325: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:01 smithi188 ceph-mon[36117]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-02T13:34:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:01 smithi188 ceph-mon[36117]: Cluster is now healthy 2024-04-02T13:34:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:03 smithi122 ceph-mon[32114]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/954133936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:34:03.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 -- 172.21.15.122:0/1221353651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 msgr2=0x7f107c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 --2- 172.21.15.122:0/1221353651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c102b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f10700099c0 tx=0x7f1070009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 -- 172.21.15.122:0/1221353651 shutdown_connections 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 --2- 172.21.15.122:0/1221353651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107c103060 0x7f107c10f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 --2- 172.21.15.122:0/1221353651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c1086f0 0x7f107c108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 --2- 172.21.15.122:0/1221353651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c102b20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 -- 172.21.15.122:0/1221353651 >> 172.21.15.122:0/1221353651 conn(0x7f107c0796c0 msgr2=0x7f107c079ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 -- 172.21.15.122:0/1221353651 shutdown_connections 2024-04-02T13:34:03.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.012+0000 7f1082580700 1 -- 172.21.15.122:0/1221353651 wait complete. 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 Processor -- start 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 -- start start 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c103060 0x7f107c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107c1086f0 0x7f107c19d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f107c112970 con 0x7f107c1026a0 2024-04-02T13:34:03.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f107c1127f0 con 0x7f107c103060 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f1082580700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f107c112af0 con 0x7f107c1086f0 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50896/0 (socket says 172.21.15.122:50896) 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 -- 172.21.15.122:0/954133936 learned_addr learned my addr 172.21.15.122:0/954133936 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107b7fe700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c103060 0x7f107c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 -- 172.21.15.122:0/954133936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107c1086f0 msgr2=0x7f107c19d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:03.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107c1086f0 0x7f107c19d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 -- 172.21.15.122:0/954133936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c103060 msgr2=0x7f107c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:03.930 INFO:teuthology.orchestra.run.smithi122.stdout:77309411400 2024-04-02T13:34:03.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c103060 0x7f107c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.013+0000 7f107bfff700 1 -- 172.21.15.122:0/954133936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f107c19ddb0 con 0x7f107c1026a0 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f107bfff700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c016070 con 0x7f107c1026a0 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f107c19e0a0 con 0x7f107c1026a0 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f106c009e20 con 0x7f107c1026a0 2024-04-02T13:34:03.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106c00f620 con 0x7f107c1026a0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.014+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f107c1a5920 con 0x7f107c1026a0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.015+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f107c0692b0 con 0x7f107c1026a0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.016+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f106c019070 con 0x7f107c1026a0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.017+0000 7f10797fa700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 0x7f10640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.017+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f106c0d04a0 con 0x7f107c1026a0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.017+0000 7f107b7fe700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 0x7f10640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:03.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.017+0000 7f107b7fe700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 0x7f10640a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f107c079cc0 tx=0x7f1070005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.021+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f106c09c9c0 con 0x7f107c1026a0 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.227+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f107c02dd30 con 0x7f107c1026a0 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.227+0000 7f10797fa700 1 -- 172.21.15.122:0/954133936 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f106c09c2c0 con 0x7f107c1026a0 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.229+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 msgr2=0x7f10640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.229+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 0x7f10640a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f107c079cc0 tx=0x7f1070005fb0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.229+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 msgr2=0x7f107c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.229+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d950 tx=0x7f106c00dc60 comp rx=0 tx=0).stop 2024-04-02T13:34:03.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 shutdown_connections 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107c1086f0 0x7f107c19d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10640a3cd0 0x7f10640a6190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f107c1026a0 0x7f107c198da0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 --2- 172.21.15.122:0/954133936 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f107c103060 0x7f107c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 >> 172.21.15.122:0/954133936 conn(0x7f107c0796c0 msgr2=0x7f107c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:03.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 shutdown_connections 2024-04-02T13:34:03.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:03.230+0000 7f1082580700 1 -- 172.21.15.122:0/954133936 wait complete. 2024-04-02T13:34:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:03 smithi187 ceph-mon[36064]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/954133936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:34:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:03 smithi188 ceph-mon[36117]: pgmap v326: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s; 13 B/s, 1 objects/s recovering 2024-04-02T13:34:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/954133936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-02T13:34:04.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411400 for osd.2 2024-04-02T13:34:04.566 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:05.753 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350999 got 347892351002 for osd.11 2024-04-02T13:34:05.753 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:05.834 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:05 smithi188 ceph-mon[36117]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-04-02T13:34:05.901 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:05 smithi122 ceph-mon[32114]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-04-02T13:34:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:05 smithi187 ceph-mon[36064]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 9 op/s; 11 B/s, 1 objects/s recovering 2024-04-02T13:34:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:07 smithi122 ceph-mon[32114]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-02T13:34:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:07 smithi187 ceph-mon[36064]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-02T13:34:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:07 smithi188 ceph-mon[36117]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 10 B/s, 1 objects/s recovering 2024-04-02T13:34:08.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612636 got 313532612639 for osd.10 2024-04-02T13:34:08.776 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:09.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 -- 172.21.15.122:0/1953754683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710102640 msgr2=0x7f9710102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 --2- 172.21.15.122:0/1953754683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710102640 0x7f9710102aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9704009990 tx=0x7f9704009ca0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 -- 172.21.15.122:0/1953754683 shutdown_connections 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 --2- 172.21.15.122:0/1953754683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 0x7f971010f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 --2- 172.21.15.122:0/1953754683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710107520 0x7f9710107900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 --2- 172.21.15.122:0/1953754683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710102640 0x7f9710102aa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 -- 172.21.15.122:0/1953754683 >> 172.21.15.122:0/1953754683 conn(0x7f9710079720 msgr2=0x7f9710079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 -- 172.21.15.122:0/1953754683 shutdown_connections 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.411+0000 7f97166d0700 1 -- 172.21.15.122:0/1953754683 wait complete. 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 Processor -- start 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 -- start start 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710102640 0x7f9710198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 0x7f9710199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:09.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97101128b0 con 0x7f9710107520 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9710112730 con 0x7f9710102640 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f97166d0700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9710112a30 con 0x7f9710102fe0 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f9714c6d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f9714c6d700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60890/0 (socket says 172.21.15.122:60890) 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f9714c6d700 1 -- 172.21.15.122:0/3965103054 learned_addr learned my addr 172.21.15.122:0/3965103054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f970f7fe700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 0x7f9710199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.412+0000 7f970ffff700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710102640 0x7f9710198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 -- 172.21.15.122:0/3965103054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 msgr2=0x7f9710199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 0x7f9710199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 -- 172.21.15.122:0/3965103054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710102640 msgr2=0x7f9710198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710102640 0x7f9710198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 -- 172.21.15.122:0/3965103054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971019ddf0 con 0x7f9710107520 2024-04-02T13:34:09.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f9714c6d700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f970800ada0 tx=0x7f970800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708017020 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f970800a0d0 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971019e0e0 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708005480 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.413+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101a5960 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.414+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97100692b0 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.415+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f9708005620 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.416+0000 7f970d7fa700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 0x7f96f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.416+0000 7f970ffff700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 0x7f96f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.416+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f97080cfff0 con 0x7f9710107520 2024-04-02T13:34:09.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.416+0000 7f970ffff700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 0x7f96f80a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9700005fd0 tx=0x7f970000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:09.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.421+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9708021020 con 0x7f9710107520 2024-04-02T13:34:09.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.629+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f971002dd30 con 0x7f9710107520 2024-04-02T13:34:09.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.630+0000 7f970d7fa700 1 -- 172.21.15.122:0/3965103054 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f970809c5a0 con 0x7f9710107520 2024-04-02T13:34:09.633 INFO:teuthology.orchestra.run.smithi122.stdout:292057776168 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 msgr2=0x7f96f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 0x7f96f80a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9700005fd0 tx=0x7f970000a4c0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 msgr2=0x7f971019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f970800ada0 tx=0x7f970800c330 comp rx=0 tx=0).stop 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 shutdown_connections 2024-04-02T13:34:09.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f96f80a3cd0 0x7f96f80a6190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9710107520 0x7f971019d6b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9710102640 0x7f9710198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 --2- 172.21.15.122:0/3965103054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9710102fe0 0x7f9710199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 >> 172.21.15.122:0/3965103054 conn(0x7f9710079720 msgr2=0x7f9710100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 shutdown_connections 2024-04-02T13:34:09.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:09.632+0000 7f97166d0700 1 -- 172.21.15.122:0/3965103054 wait complete. 2024-04-02T13:34:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:09 smithi187 ceph-mon[36064]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:09.990 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:09 smithi122 ceph-mon[32114]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:09 smithi188 ceph-mon[36117]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:10.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 -- 172.21.15.122:0/3251956895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 msgr2=0x7f8ba810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 --2- 172.21.15.122:0/3251956895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba810f460 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000b0d0 tx=0x7f8ba000b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 -- 172.21.15.122:0/3251956895 shutdown_connections 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 --2- 172.21.15.122:0/3251956895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba810f460 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 --2- 172.21.15.122:0/3251956895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 0x7f8ba8108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 --2- 172.21.15.122:0/3251956895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 0x7f8ba8102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 -- 172.21.15.122:0/3251956895 >> 172.21.15.122:0/3251956895 conn(0x7f8ba80796b0 msgr2=0x7f8ba8079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 -- 172.21.15.122:0/3251956895 shutdown_connections 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.089+0000 7f8baddc4700 1 -- 172.21.15.122:0/3251956895 wait complete. 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 Processor -- start 2024-04-02T13:34:10.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 -- start start 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 0x7f8ba819e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 0x7f8ba8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81128f0 con 0x7f8ba8103030 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112770 con 0x7f8ba8108690 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.090+0000 7f8baddc4700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8112a70 con 0x7f8ba8102670 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60900/0 (socket says 172.21.15.122:60900) 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 -- 172.21.15.122:0/1835629556 learned_addr learned my addr 172.21.15.122:0/1835629556 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba77fe700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 0x7f8ba819e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba7fff700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 0x7f8ba8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 -- 172.21.15.122:0/1835629556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 msgr2=0x7f8ba819e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 0x7f8ba819e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 -- 172.21.15.122:0/1835629556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 msgr2=0x7f8ba8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 0x7f8ba8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 -- 172.21.15.122:0/1835629556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8199230 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba6ffd700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00d980 tx=0x7f8b9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c017070 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c005020 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba8199520 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c009bc0 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.091+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81a57e0 con 0x7f8ba8103030 2024-04-02T13:34:10.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.092+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba80692b0 con 0x7f8ba8103030 2024-04-02T13:34:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.093+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8b9c0051d0 con 0x7f8ba8103030 2024-04-02T13:34:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.094+0000 7f8ba4ff9700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 0x7f8b900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.094+0000 7f8ba77fe700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 0x7f8b900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.094+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8b9c0d0c60 con 0x7f8ba8103030 2024-04-02T13:34:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.095+0000 7f8ba77fe700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 0x7f8b900a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005950 tx=0x7f8b9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:10.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.098+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8b9c020020 con 0x7f8ba8103030 2024-04-02T13:34:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.305+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8ba802dd30 con 0x7f8ba8103030 2024-04-02T13:34:10.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.306+0000 7f8ba4ff9700 1 -- 172.21.15.122:0/1835629556 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b9c09d120 con 0x7f8ba8103030 2024-04-02T13:34:10.309 INFO:teuthology.orchestra.run.smithi122.stdout:55834574926 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 msgr2=0x7f8b900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 0x7f8b900a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005950 tx=0x7f8b9800b410 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 msgr2=0x7f8ba819ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00d980 tx=0x7f8b9c00dc90 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 shutdown_connections 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ba8108690 0x7f8ba8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba8102670 0x7f8ba819e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8b900a3cd0 0x7f8b900a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 --2- 172.21.15.122:0/1835629556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ba8103030 0x7f8ba819ee50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 >> 172.21.15.122:0/1835629556 conn(0x7f8ba80796b0 msgr2=0x7f8ba8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 shutdown_connections 2024-04-02T13:34:10.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.308+0000 7f8baddc4700 1 -- 172.21.15.122:0/1835629556 wait complete. 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- 172.21.15.122:0/4266085661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249806a2a0 msgr2=0x7f249806a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 --2- 172.21.15.122:0/4266085661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249806a2a0 0x7f249806a720 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2480009990 tx=0x7f2480009ca0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- 172.21.15.122:0/4266085661 shutdown_connections 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 --2- 172.21.15.122:0/4266085661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f249810f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 --2- 172.21.15.122:0/4266085661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2498108650 0x7f2498108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 --2- 172.21.15.122:0/4266085661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f249806a2a0 0x7f249806a720 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- 172.21.15.122:0/4266085661 >> 172.21.15.122:0/4266085661 conn(0x7f2498078420 msgr2=0x7f2498078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- 172.21.15.122:0/4266085661 shutdown_connections 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- 172.21.15.122:0/4266085661 wait complete. 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 Processor -- start 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.475+0000 7f249df2b700 1 -- start start 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f249806a2a0 0x7f2498198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f24981992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2498112900 con 0x7f2498108650 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2498112780 con 0x7f249806a2a0 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f249df2b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2498112a80 con 0x7f249806ac60 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60914/0 (socket says 172.21.15.122:60914) 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 -- 172.21.15.122:0/2795661100 learned_addr learned my addr 172.21.15.122:0/2795661100 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f248effd700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f24981992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f24977fe700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f249806a2a0 0x7f2498198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 -- 172.21.15.122:0/2795661100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 msgr2=0x7f24981992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f24981992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 -- 172.21.15.122:0/2795661100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f249806a2a0 msgr2=0x7f2498198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f249806a2a0 0x7f2498198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 -- 172.21.15.122:0/2795661100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f249819dd80 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f248effd700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f24981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f2497fff700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2490007ee0 tx=0x7f249000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.476+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249000dbd0 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.477+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249000dd70 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.477+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f249819e070 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.477+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24900133f0 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.477+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24981a58f0 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.478+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24980692b0 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.479+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f249001a020 con 0x7f2498108650 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.479+0000 7f24957fa700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 0x7f24840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:10.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.480+0000 7f24977fe700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 0x7f24840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:10.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.480+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f24900cfab0 con 0x7f2498108650 2024-04-02T13:34:10.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.480+0000 7f24977fe700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 0x7f24840a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2488005950 tx=0x7f24880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:10.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.484+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2490010070 con 0x7f2498108650 2024-04-02T13:34:10.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.694+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f249802dd30 con 0x7f2498108650 2024-04-02T13:34:10.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.694+0000 7f24957fa700 1 -- 172.21.15.122:0/2795661100 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f249009bfd0 con 0x7f2498108650 2024-04-02T13:34:10.698 INFO:teuthology.orchestra.run.smithi122.stdout:107374182469 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 msgr2=0x7f24840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 0x7f24840a6190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2488005950 tx=0x7f24880058e0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 msgr2=0x7f249819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2490007ee0 tx=0x7f249000d3b0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 shutdown_connections 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24840a3cd0 0x7f24840a6190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2498108650 0x7f249819d640 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f249806a2a0 0x7f2498198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 --2- 172.21.15.122:0/2795661100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f249806ac60 0x7f24981992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 >> 172.21.15.122:0/2795661100 conn(0x7f2498078420 msgr2=0x7f2498100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 shutdown_connections 2024-04-02T13:34:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:10.696+0000 7f249df2b700 1 -- 172.21.15.122:0/2795661100 wait complete. 2024-04-02T13:34:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3965103054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:34:10.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1835629556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:34:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3965103054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:34:11.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1835629556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:34:11.038 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3965103054' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-02T13:34:11.038 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1835629556' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-02T13:34:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:11 smithi122 ceph-mon[32114]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2795661100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:34:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:11 smithi187 ceph-mon[36064]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2795661100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:34:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:11 smithi188 ceph-mon[36117]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s; 9 B/s, 0 objects/s recovering 2024-04-02T13:34:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2795661100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 -- 172.21.15.122:0/4271354323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f4107500 msgr2=0x7fb1f41078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 --2- 172.21.15.122:0/4271354323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f4107500 0x7fb1f41078e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4009990 tx=0x7fb1e4009ca0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 -- 172.21.15.122:0/4271354323 shutdown_connections 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 --2- 172.21.15.122:0/4271354323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 0x7fb1f410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 --2- 172.21.15.122:0/4271354323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f4107500 0x7fb1f41078e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 --2- 172.21.15.122:0/4271354323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f41014f0 0x7fb1f4101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 -- 172.21.15.122:0/4271354323 >> 172.21.15.122:0/4271354323 conn(0x7fb1f4078420 msgr2=0x7fb1f4078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 -- 172.21.15.122:0/4271354323 shutdown_connections 2024-04-02T13:34:12.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.068+0000 7fb1fa021700 1 -- 172.21.15.122:0/4271354323 wait complete. 2024-04-02T13:34:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.069+0000 7fb1fa021700 1 Processor -- start 2024-04-02T13:34:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.069+0000 7fb1fa021700 1 -- start start 2024-04-02T13:34:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 0x7fb1f419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f4107500 0x7fb1f41a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f41117a0 con 0x7fb1f41014f0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4111620 con 0x7fb1f4101eb0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f4111920 con 0x7fb1f4107500 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60934/0 (socket says 172.21.15.122:60934) 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 -- 172.21.15.122:0/2995139366 learned_addr learned my addr 172.21.15.122:0/2995139366 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f2ffd700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 0x7fb1f419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f3fff700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f4107500 0x7fb1f41a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 -- 172.21.15.122:0/2995139366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f4107500 msgr2=0x7fb1f41a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f4107500 0x7fb1f41a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 -- 172.21.15.122:0/2995139366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 msgr2=0x7fb1f419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 0x7fb1f419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 -- 172.21.15.122:0/2995139366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f41a5100 con 0x7fb1f41014f0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f37fe700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000c00 tx=0x7fb1e4003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4027070 con 0x7fb1f41014f0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1e4004190 con 0x7fb1f41014f0 2024-04-02T13:34:12.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f41a52d0 con 0x7fb1f41014f0 2024-04-02T13:34:12.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400f460 con 0x7fb1f41014f0 2024-04-02T13:34:12.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.070+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f41a5720 con 0x7fb1f41014f0 2024-04-02T13:34:12.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.071+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f40692b0 con 0x7fb1f41014f0 2024-04-02T13:34:12.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.073+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fb1e402a070 con 0x7fb1f41014f0 2024-04-02T13:34:12.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.073+0000 7fb1f0ff9700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:12.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.074+0000 7fb1f2ffd700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:12.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.074+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb1e40d1c50 con 0x7fb1f41014f0 2024-04-02T13:34:12.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.074+0000 7fb1f2ffd700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4078990 tx=0x7fb1e8005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:12.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.077+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb1e4013070 con 0x7fb1f41014f0 2024-04-02T13:34:12.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.281+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb1f402df90 con 0x7fb1f41014f0 2024-04-02T13:34:12.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.282+0000 7fb1f0ff9700 1 -- 172.21.15.122:0/2995139366 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb1e409e200 con 0x7fb1f41014f0 2024-04-02T13:34:12.285 INFO:teuthology.orchestra.run.smithi122.stdout:223338299441 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 msgr2=0x7fb1dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4078990 tx=0x7fb1e8005eb0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 msgr2=0x7fb1f4199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000c00 tx=0x7fb1e4003730 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 shutdown_connections 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1f4107500 0x7fb1f41a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb1dc0a3cd0 0x7fb1dc0a6190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1f41014f0 0x7fb1f4199b40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 --2- 172.21.15.122:0/2995139366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1f4101eb0 0x7fb1f419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:12.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 >> 172.21.15.122:0/2995139366 conn(0x7fb1f4078420 msgr2=0x7fb1f4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:12.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 shutdown_connections 2024-04-02T13:34:12.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.284+0000 7fb1fa021700 1 -- 172.21.15.122:0/2995139366 wait complete. 2024-04-02T13:34:12.945 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:12 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2995139366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:34:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:12 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2995139366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:34:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:12 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2995139366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-02T13:34:13.431 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776168 for osd.9 2024-04-02T13:34:13.432 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:13 smithi122 ceph-mon[32114]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/243471728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:34:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2617825102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:34:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:13 smithi187 ceph-mon[36064]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/243471728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:34:13.887 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2617825102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:34:14.017 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182469 for osd.3 2024-04-02T13:34:14.017 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:13 smithi188 ceph-mon[36117]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/243471728' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-02T13:34:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2617825102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-02T13:34:14.458 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:14.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 -- 172.21.15.122:0/2721584481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8107520 msgr2=0x7f5dd8107900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/2721584481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8107520 0x7f5dd8107900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000b0a0 tx=0x7f5dd000b3b0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 -- 172.21.15.122:0/2721584481 shutdown_connections 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/2721584481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8102fe0 0x7f5dd810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/2721584481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8107520 0x7f5dd8107900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/2721584481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102640 0x7f5dd8102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 -- 172.21.15.122:0/2721584481 >> 172.21.15.122:0/2721584481 conn(0x7f5dd8079720 msgr2=0x7f5dd8079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 -- 172.21.15.122:0/2721584481 shutdown_connections 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.722+0000 7f5dde1f8700 1 -- 172.21.15.122:0/2721584481 wait complete. 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 Processor -- start 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 -- start start 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102fe0 0x7f5dd8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 0x7f5dd819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd81128e0 con 0x7f5dd8102640 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8112760 con 0x7f5dd8107520 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dde1f8700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd8112a60 con 0x7f5dd8102fe0 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dd77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dd77fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60962/0 (socket says 172.21.15.122:60962) 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dd77fe700 1 -- 172.21.15.122:0/243471728 learned_addr learned my addr 172.21.15.122:0/243471728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.723+0000 7f5dd6ffd700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102fe0 0x7f5dd8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd7fff700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 0x7f5dd819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 -- 172.21.15.122:0/243471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102fe0 msgr2=0x7f5dd8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102fe0 0x7f5dd8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 -- 172.21.15.122:0/243471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 msgr2=0x7f5dd819d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 0x7f5dd819d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 -- 172.21.15.122:0/243471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd819ddd0 con 0x7f5dd8102640 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd7fff700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 0x7f5dd819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd77fe700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00062a0 tx=0x7f5dd0004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0021070 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5dd0007c00 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd819e060 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00048b0 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.724+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd81a5940 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.725+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd810ab50 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.727+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f5dd0004000 con 0x7f5dd8102640 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stdout:141733920833 2024-04-02T13:34:14.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.727+0000 7f5dd4ff9700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 0x7f5dc40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.727+0000 7f5dd6ffd700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 0x7f5dc40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.727+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5dd000f020 con 0x7f5dd8102640 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.728+0000 7f5dd6ffd700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 0x7f5dc40a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8079e00 tx=0x7f5dc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.731+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5dd002a030 con 0x7f5dd8102640 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.941+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5dd819a120 con 0x7f5dd8102640 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.942+0000 7f5dd4ff9700 1 -- 172.21.15.122:0/243471728 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5dd0007db0 con 0x7f5dd8102640 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 msgr2=0x7f5dc40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 0x7f5dc40a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8079e00 tx=0x7f5dc000b410 comp rx=0 tx=0).stop 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 msgr2=0x7f5dd8198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00062a0 tx=0x7f5dd0004d70 comp rx=0 tx=0).stop 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 shutdown_connections 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5dd8107520 0x7f5dd819d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f5dc40a3cd0 0x7f5dc40a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5dd8102640 0x7f5dd8198e10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 --2- 172.21.15.122:0/243471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dd8102fe0 0x7f5dd8199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 >> 172.21.15.122:0/243471728 conn(0x7f5dd8079720 msgr2=0x7f5dd8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 shutdown_connections 2024-04-02T13:34:14.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.945+0000 7f5dde1f8700 1 -- 172.21.15.122:0/243471728 wait complete. 2024-04-02T13:34:14.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2442921572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 msgr2=0x7fbd2410f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2442921572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd2410f580 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00b0d0 tx=0x7fbd1c00b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2442921572 shutdown_connections 2024-04-02T13:34:14.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2442921572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd2410f580 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2442921572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 0x7fbd24101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2442921572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 0x7fbd24107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2442921572 >> 172.21.15.122:0/2442921572 conn(0x7fbd24077af0 msgr2=0x7fbd240fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2442921572 shutdown_connections 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2442921572 wait complete. 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.825+0000 7fbd2ad79700 1 Processor -- start 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 -- start start 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 0x7fbd2419ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 0x7fbd2419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24112930 con 0x7fbd24107970 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd241127b0 con 0x7fbd241017f0 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd2ad79700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24112ab0 con 0x7fbd24100e40 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60978/0 (socket says 172.21.15.122:60978) 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 -- 172.21.15.122:0/2617825102 learned_addr learned my addr 172.21.15.122:0/2617825102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:14.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd23fff700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 0x7fbd2419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd28b15700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 0x7fbd2419ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stdout:167503724604 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 -- 172.21.15.122:0/2617825102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 msgr2=0x7fbd2419ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 0x7fbd2419ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 -- 172.21.15.122:0/2617825102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 msgr2=0x7fbd2419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 0x7fbd2419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 -- 172.21.15.122:0/2617825102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd241993f0 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.826+0000 7fbd29316700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c006070 tx=0x7fbd1c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.827+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c021070 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.827+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c007b80 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.827+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd24199680 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.827+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c005330 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.827+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241a59c0 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.828+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd241999f0 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.830+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fbd1c024070 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.830+0000 7fbd21ffb700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.830+0000 7fbd28b15700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.830+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbd1c00f020 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.831+0000 7fbd28b15700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 0x7fbd140a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800bbd0 tx=0x7fbd1800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:12.833+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbd241999f0 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.058+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fbd24199fd0 con 0x7fbd24107970 2024-04-02T13:34:14.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.058+0000 7fbd21ffb700 1 -- 172.21.15.122:0/2617825102 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd1c02b020 con 0x7fbd24107970 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 msgr2=0x7fbd140a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 0x7fbd140a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800bbd0 tx=0x7fbd1800b5b0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 msgr2=0x7fbd24198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c006070 tx=0x7fbd1c004d00 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 shutdown_connections 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fbd140a3cd0 0x7fbd140a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd24107970 0x7fbd24198b10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd24100e40 0x7fbd2419ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 --2- 172.21.15.122:0/2617825102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd241017f0 0x7fbd2419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.060+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 >> 172.21.15.122:0/2617825102 conn(0x7fbd24077af0 msgr2=0x7fbd24106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.061+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 shutdown_connections 2024-04-02T13:34:14.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:13.061+0000 7fbd2ad79700 1 -- 172.21.15.122:0/2617825102 wait complete. 2024-04-02T13:34:15.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724600 got 167503724604 for osd.5 2024-04-02T13:34:15.513 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:15.760 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:15 smithi122 ceph-mon[32114]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:15 smithi187 ceph-mon[36064]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:15 smithi188 ceph-mon[36117]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:16.439 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574926 for osd.1 2024-04-02T13:34:16.440 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 -- 172.21.15.122:0/2649392804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 msgr2=0x7f8ce8101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 --2- 172.21.15.122:0/2649392804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8101a00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8cd40099c0 tx=0x7f8cd4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 -- 172.21.15.122:0/2649392804 shutdown_connections 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 --2- 172.21.15.122:0/2649392804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 0x7f8ce810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 --2- 172.21.15.122:0/2649392804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 0x7f8ce8107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 --2- 172.21.15.122:0/2649392804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8101a00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 -- 172.21.15.122:0/2649392804 >> 172.21.15.122:0/2649392804 conn(0x7f8ce8078620 msgr2=0x7f8ce8078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 -- 172.21.15.122:0/2649392804 shutdown_connections 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.613+0000 7f8ceec25700 1 -- 172.21.15.122:0/2649392804 wait complete. 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 Processor -- start 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 -- start start 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 0x7f8ce81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 0x7f8ce819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce81118e0 con 0x7f8ce8101580 2024-04-02T13:34:16.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8111760 con 0x7f8ce8107630 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ceec25700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8111a60 con 0x7f8ce8101f40 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8cec9c1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8cec9c1700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32770/0 (socket says 172.21.15.122:32770) 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8cec9c1700 1 -- 172.21.15.122:0/595690613 learned_addr learned my addr 172.21.15.122:0/595690613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ced1c2700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 0x7f8ce819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.614+0000 7f8ce7fff700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 0x7f8ce81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 -- 172.21.15.122:0/595690613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 msgr2=0x7f8ce81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 0x7f8ce81992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 -- 172.21.15.122:0/595690613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 msgr2=0x7f8ce819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 0x7f8ce819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 -- 172.21.15.122:0/595690613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce819ddb0 con 0x7f8ce8101580 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8cec9c1700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00b530 tx=0x7f8cdc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc015020 con 0x7f8ce8101580 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cdc011d10 con 0x7f8ce8101580 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce819e0a0 con 0x7f8ce8101580 2024-04-02T13:34:16.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc00f2e0 con 0x7f8ce8101580 2024-04-02T13:34:16.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.615+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce81a5920 con 0x7f8ce8101580 2024-04-02T13:34:16.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.616+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce80692b0 con 0x7f8ce8101580 2024-04-02T13:34:16.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.617+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8cdc018020 con 0x7f8ce8101580 2024-04-02T13:34:16.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.618+0000 7f8ce5ffb700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 0x7f8cd80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:16.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.618+0000 7f8ce7fff700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 0x7f8cd80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:16.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.618+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8cdc0d0030 con 0x7f8ce8101580 2024-04-02T13:34:16.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.618+0000 7f8ce7fff700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 0x7f8cd80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8106380 tx=0x7f8cd4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:16.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.622+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8cdc01f020 con 0x7f8ce8101580 2024-04-02T13:34:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.827+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8ce819a100 con 0x7f8ce8101580 2024-04-02T13:34:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.828+0000 7f8ce5ffb700 1 -- 172.21.15.122:0/595690613 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ce819a100 con 0x7f8ce8101580 2024-04-02T13:34:16.832 INFO:teuthology.orchestra.run.smithi122.stdout:34359738453 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 msgr2=0x7f8cd80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 0x7f8cd80a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8106380 tx=0x7f8cd4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 msgr2=0x7f8ce8198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc00b530 tx=0x7f8cdc00b840 comp rx=0 tx=0).stop 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 shutdown_connections 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ce8107630 0x7f8ce819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f8cd80a3cd0 0x7f8cd80a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ce8101580 0x7f8ce8198da0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 --2- 172.21.15.122:0/595690613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ce8101f40 0x7f8ce81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:16.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 >> 172.21.15.122:0/595690613 conn(0x7f8ce8078620 msgr2=0x7f8ce8100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:16.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 shutdown_connections 2024-04-02T13:34:16.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:16.830+0000 7f8ceec25700 1 -- 172.21.15.122:0/595690613 wait complete. 2024-04-02T13:34:17.122 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920828 got 141733920833 for osd.4 2024-04-02T13:34:17.122 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:17.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:17 smithi122 ceph-mon[32114]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:17.891 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:17 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/595690613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:34:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:17 smithi187 ceph-mon[36064]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:17 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/595690613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:34:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:17 smithi188 ceph-mon[36117]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:17 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/595690613' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 -- 172.21.15.122:0/2619928032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 msgr2=0x7f8a1410f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 --2- 172.21.15.122:0/2619928032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a1410f540 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00b0a0 tx=0x7f8a0c00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 -- 172.21.15.122:0/2619928032 shutdown_connections 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 --2- 172.21.15.122:0/2619928032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a1410f540 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 --2- 172.21.15.122:0/2619928032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14108670 0x7f8a14108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 --2- 172.21.15.122:0/2619928032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14100ba0 0x7f8a14101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 -- 172.21.15.122:0/2619928032 >> 172.21.15.122:0/2619928032 conn(0x7f8a14077af0 msgr2=0x7f8a140feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:18.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 -- 172.21.15.122:0/2619928032 shutdown_connections 2024-04-02T13:34:18.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.613+0000 7f8a1a310700 1 -- 172.21.15.122:0/2619928032 wait complete. 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 Processor -- start 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 -- start start 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14100ba0 0x7f8a14101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14108670 0x7f8a14102800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14112a50 con 0x7f8a14101560 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a141128d0 con 0x7f8a14108670 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.614+0000 7f8a1a310700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a14112bd0 con 0x7f8a14100ba0 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:18.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39266/0 (socket says 172.21.15.122:39266) 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 -- 172.21.15.122:0/3655697389 learned_addr learned my addr 172.21.15.122:0/3655697389 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a13fff700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14100ba0 0x7f8a14101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a188ad700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14108670 0x7f8a14102800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 -- 172.21.15.122:0/3655697389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14100ba0 msgr2=0x7f8a14101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14100ba0 0x7f8a14101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 -- 172.21.15.122:0/3655697389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14108670 msgr2=0x7f8a14102800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14108670 0x7f8a14102800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 -- 172.21.15.122:0/3655697389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a141a7310 con 0x7f8a14101560 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a137fe700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d950 tx=0x7f8a0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08016070 con 0x7f8a14101560 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a141a7600 con 0x7f8a14101560 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a0800fd70 con 0x7f8a14101560 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0800e610 con 0x7f8a14101560 2024-04-02T13:34:18.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.615+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a141a7b40 con 0x7f8a14101560 2024-04-02T13:34:18.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.616+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a140692b0 con 0x7f8a14101560 2024-04-02T13:34:18.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.618+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7f8a08019070 con 0x7f8a14101560 2024-04-02T13:34:18.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.619+0000 7f8a117fa700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 0x7f89fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:18.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.619+0000 7f8a13fff700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 0x7f89fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:18.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.619+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f8a080d0330 con 0x7f8a14101560 2024-04-02T13:34:18.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.619+0000 7f8a13fff700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 0x7f89fc0a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8a040045b0 tx=0x7f8a0400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:18.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.622+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8a0809c850 con 0x7f8a14101560 2024-04-02T13:34:18.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.827+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8a141a7da0 con 0x7f8a14101560 2024-04-02T13:34:18.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.827+0000 7f8a117fa700 1 -- 172.21.15.122:0/3655697389 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8a0809c150 con 0x7f8a14101560 2024-04-02T13:34:18.831 INFO:teuthology.orchestra.run.smithi122.stdout:257698037806 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 msgr2=0x7f89fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 0x7f89fc0a6190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8a040045b0 tx=0x7f8a0400b3c0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 msgr2=0x7f8a141022c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8a0800d950 tx=0x7f8a0800dc60 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 shutdown_connections 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a14108670 0x7f8a14102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a14100ba0 0x7f8a14101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f89fc0a3cd0 0x7f89fc0a6190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 --2- 172.21.15.122:0/3655697389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a14101560 0x7f8a141022c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 >> 172.21.15.122:0/3655697389 conn(0x7f8a14077af0 msgr2=0x7f8a140ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 shutdown_connections 2024-04-02T13:34:18.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:18.829+0000 7f8a1a310700 1 -- 172.21.15.122:0/3655697389 wait complete. 2024-04-02T13:34:19.262 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738453 for osd.0 2024-04-02T13:34:19.262 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:19.388 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299434 got 223338299441 for osd.7 2024-04-02T13:34:19.388 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:19.779 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:19 smithi122 ceph-mon[32114]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:19.779 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3655697389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:34:19.807 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037799 got 257698037806 for osd.8 2024-04-02T13:34:19.808 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:19 smithi187 ceph-mon[36064]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3655697389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:34:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:19 smithi188 ceph-mon[36117]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3655697389' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-02T13:34:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 -- 172.21.15.122:0/3353862607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78107630 msgr2=0x7fca78107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 --2- 172.21.15.122:0/3353862607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78107630 0x7fca78107a10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fca740099c0 tx=0x7fca74009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 -- 172.21.15.122:0/3353862607 shutdown_connections 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 --2- 172.21.15.122:0/3353862607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78101f40 0x7fca7810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 --2- 172.21.15.122:0/3353862607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78107630 0x7fca78107a10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 --2- 172.21.15.122:0/3353862607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101580 0x7fca78101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 -- 172.21.15.122:0/3353862607 >> 172.21.15.122:0/3353862607 conn(0x7fca78078620 msgr2=0x7fca78078a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 -- 172.21.15.122:0/3353862607 shutdown_connections 2024-04-02T13:34:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 -- 172.21.15.122:0/3353862607 wait complete. 2024-04-02T13:34:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.813+0000 7fca7f909700 1 Processor -- start 2024-04-02T13:34:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 -- start start 2024-04-02T13:34:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 0x7fca7819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78107630 0x7fca781a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca781118e0 con 0x7fca78101580 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fca78111760 con 0x7fca78107630 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7f909700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca78111a60 con 0x7fca78101f40 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39284/0 (socket says 172.21.15.122:39284) 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 -- 172.21.15.122:0/133133542 learned_addr learned my addr 172.21.15.122:0/133133542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7dea6700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78107630 0x7fca781a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7cea4700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 0x7fca7819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 -- 172.21.15.122:0/133133542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 msgr2=0x7fca7819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 0x7fca7819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 -- 172.21.15.122:0/133133542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78107630 msgr2=0x7fca781a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78107630 0x7fca781a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7d6a5700 1 -- 172.21.15.122:0/133133542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca781a5220 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.814+0000 7fca7cea4700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 0x7fca7819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca7d6a5700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca740059d0 tx=0x7fca740038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74027070 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca74004050 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca781a53f0 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca74022db0 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.815+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca781a58a0 con 0x7fca78101580 2024-04-02T13:34:20.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.816+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca78109ac0 con 0x7fca78101580 2024-04-02T13:34:20.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.817+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fca7402a070 con 0x7fca78101580 2024-04-02T13:34:20.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.818+0000 7fca6e7fc700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 0x7fca640a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:20.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.818+0000 7fca7cea4700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 0x7fca640a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:20.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.818+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fca740d1ec0 con 0x7fca78101580 2024-04-02T13:34:20.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.818+0000 7fca7cea4700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 0x7fca640a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca78078c20 tx=0x7fca68008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:20.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:20.821+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fca74013070 con 0x7fca78101580 2024-04-02T13:34:21.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.020+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fca7806b320 con 0x7fca78101580 2024-04-02T13:34:21.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.020+0000 7fca6e7fc700 1 -- 172.21.15.122:0/133133542 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fca7409e470 con 0x7fca78101580 2024-04-02T13:34:21.024 INFO:teuthology.orchestra.run.smithi122.stdout:201863462968 2024-04-02T13:34:21.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.022+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 msgr2=0x7fca640a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:21.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.022+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 0x7fca640a6210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fca78078c20 tx=0x7fca68008040 comp rx=0 tx=0).stop 2024-04-02T13:34:21.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.022+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 msgr2=0x7fca78199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:21.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fca740059d0 tx=0x7fca740038b0 comp rx=0 tx=0).stop 2024-04-02T13:34:21.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 shutdown_connections 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fca78107630 0x7fca781a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fca640a3d50 0x7fca640a6210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca78101580 0x7fca78199c60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 --2- 172.21.15.122:0/133133542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca78101f40 0x7fca7819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 >> 172.21.15.122:0/133133542 conn(0x7fca78078620 msgr2=0x7fca78100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 shutdown_connections 2024-04-02T13:34:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:21.023+0000 7fca7f909700 1 -- 172.21.15.122:0/133133542 wait complete. 2024-04-02T13:34:21.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:21.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:21.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:20 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:20 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:34:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:20 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' 2024-04-02T13:34:21.722 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462968 for osd.6 2024-04-02T13:34:21.722 DEBUG:teuthology.parallel:result is None 2024-04-02T13:34:21.722 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-02T13:34:21.723 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:34:21.966 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:22.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:21 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/133133542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:34:22.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:21 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/133133542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:34:22.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:21 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/133133542' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-02T13:34:23.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:22 smithi122 ceph-mon[32114]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:23.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:22 smithi187 ceph-mon[36064]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:22 smithi188 ceph-mon[36117]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 -- 172.21.15.122:0/166062157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 msgr2=0x7fc2c410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 --2- 172.21.15.122:0/166062157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c410f4a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00b0a0 tx=0x7fc2bc00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 -- 172.21.15.122:0/166062157 shutdown_connections 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 --2- 172.21.15.122:0/166062157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c410f4a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 --2- 172.21.15.122:0/166062157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 0x7fc2c4108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 --2- 172.21.15.122:0/166062157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 0x7fc2c4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 -- 172.21.15.122:0/166062157 >> 172.21.15.122:0/166062157 conn(0x7fc2c4079720 msgr2=0x7fc2c4079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:23.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 -- 172.21.15.122:0/166062157 shutdown_connections 2024-04-02T13:34:23.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.695+0000 7fc2c900b700 1 -- 172.21.15.122:0/166062157 wait complete. 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 Processor -- start 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 -- start start 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 0x7fc2c419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 0x7fc2c4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4112920 con 0x7fc2c4103000 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41127a0 con 0x7fc2c4102640 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c900b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4112aa0 con 0x7fc2c41086d0 2024-04-02T13:34:23.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.696+0000 7fc2c259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c359e700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 0x7fc2c4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39304/0 (socket says 172.21.15.122:39304) 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 -- 172.21.15.122:0/4056128183 learned_addr learned my addr 172.21.15.122:0/4056128183 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c2d9d700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 0x7fc2c419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 -- 172.21.15.122:0/4056128183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 msgr2=0x7fc2c4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 0x7fc2c4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 -- 172.21.15.122:0/4056128183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 msgr2=0x7fc2c419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 0x7fc2c419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 -- 172.21.15.122:0/4056128183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c4199260 con 0x7fc2c4103000 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c259c700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400d980 tx=0x7fc2b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4016070 con 0x7fc2c4103000 2024-04-02T13:34:23.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c4199550 con 0x7fc2c4103000 2024-04-02T13:34:23.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b4004d10 con 0x7fc2c4103000 2024-04-02T13:34:23.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a5810 con 0x7fc2c4103000 2024-04-02T13:34:23.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.697+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4005670 con 0x7fc2c4103000 2024-04-02T13:34:23.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.698+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2c40692b0 con 0x7fc2c4103000 2024-04-02T13:34:23.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.699+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 17) ==== 138698+0+0 (secure 0 0 0) 0x7fc2b4004750 con 0x7fc2c4103000 2024-04-02T13:34:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.700+0000 7fc2abfff700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 0x7fc2ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.700+0000 7fc2c2d9d700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 0x7fc2ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.700+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc2b40d0670 con 0x7fc2c4103000 2024-04-02T13:34:23.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.700+0000 7fc2c2d9d700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 0x7fc2ac0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800abd0 tx=0x7fc2b800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:23.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.705+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc2b401f020 con 0x7fc2c4103000 2024-04-02T13:34:23.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.898+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2c41017a0 con 0x7fc2ac0a3cd0 2024-04-02T13:34:23.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.908+0000 7fc2abfff700 1 -- 172.21.15.122:0/4056128183 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340402 (secure 0 0 0) 0x7fc2c41017a0 con 0x7fc2ac0a3cd0 2024-04-02T13:34:23.916 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 msgr2=0x7fc2ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 0x7fc2ac0a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800abd0 tx=0x7fc2b800a380 comp rx=0 tx=0).stop 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 msgr2=0x7fc2c419ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400d980 tx=0x7fc2b400dc90 comp rx=0 tx=0).stop 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 shutdown_connections 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2c41086d0 0x7fc2c4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2c4102640 0x7fc2c419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc2ac0a3cd0 0x7fc2ac0a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 --2- 172.21.15.122:0/4056128183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc2c4103000 0x7fc2c419ee80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 >> 172.21.15.122:0/4056128183 conn(0x7fc2c4079720 msgr2=0x7fc2c4101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 shutdown_connections 2024-04-02T13:34:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:23.911+0000 7fc2c900b700 1 -- 172.21.15.122:0/4056128183 wait complete. 2024-04-02T13:34:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:34:24.720 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-02T13:34:22.480272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659357,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":859,"num_read_kb":557,"num_write":424,"num_write_kb":3080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380268,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561556,"statfs":{"total":1151940427776,"available":1151551033344,"internally_reserved":0,"allocated":14729216,"data_stored":8017805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003533"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554078+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:33:55.554078+0000","last_peered":"2024-04-02T13:33:55.554078+0000","last_clean":"2024-04-02T13:33:55.554078+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:33:55.554078+0000","last_undegraded":"2024-04-02T13:33:55.554078+0000","last_fullsized":"2024-04-02T13:33:55.554078+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555120+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:33:55.555120+0000","last_peered":"2024-04-02T13:33:55.555120+0000","last_clean":"2024-04-02T13:33:55.555120+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:33:55.555120+0000","last_undegraded":"2024-04-02T13:33:55.555120+0000","last_fullsized":"2024-04-02T13:33:55.555120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554964+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:33:55.554964+0000","last_peered":"2024-04-02T13:33:55.554964+0000","last_clean":"2024-04-02T13:33:55.554964+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:33:55.554964+0000","last_undegraded":"2024-04-02T13:33:55.554964+0000","last_fullsized":"2024-04-02T13:33:55.554964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551897+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:33:55.551897+0000","last_peered":"2024-04-02T13:33:55.551897+0000","last_clean":"2024-04-02T13:33:55.551897+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:33:55.551897+0000","last_undegraded":"2024-04-02T13:33:55.551897+0000","last_fullsized":"2024-04-02T13:33:55.551897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560120+0000","last_change":"2024-04-02T13:33:55.560120+0000","last_active":"2024-04-02T13:33:55.560120+0000","last_peered":"2024-04-02T13:33:55.560120+0000","last_clean":"2024-04-02T13:33:55.560120+0000","last_became_active":"2024-04-02T13:33:55.559117+0000","last_became_peered":"2024-04-02T13:33:55.559117+0000","last_unstale":"2024-04-02T13:33:55.560120+0000","last_undegraded":"2024-04-02T13:33:55.560120+0000","last_fullsized":"2024-04-02T13:33:55.560120+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:03:42.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554581+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:33:55.554581+0000","last_peered":"2024-04-02T13:33:55.554581+0000","last_clean":"2024-04-02T13:33:55.554581+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:33:55.554581+0000","last_undegraded":"2024-04-02T13:33:55.554581+0000","last_fullsized":"2024-04-02T13:33:55.554581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555398+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:33:55.555398+0000","last_peered":"2024-04-02T13:33:55.555398+0000","last_clean":"2024-04-02T13:33:55.555398+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:33:55.555398+0000","last_undegraded":"2024-04-02T13:33:55.555398+0000","last_fullsized":"2024-04-02T13:33:55.555398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657098+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:33:55.657098+0000","last_peered":"2024-04-02T13:33:55.657098+0000","last_clean":"2024-04-02T13:33:55.657098+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:33:55.657098+0000","last_undegraded":"2024-04-02T13:33:55.657098+0000","last_fullsized":"2024-04-02T13:33:55.657098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557525+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:33:55.557525+0000","last_peered":"2024-04-02T13:33:55.557525+0000","last_clean":"2024-04-02T13:33:55.557525+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:33:55.557525+0000","last_undegraded":"2024-04-02T13:33:55.557525+0000","last_fullsized":"2024-04-02T13:33:55.557525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554912+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:33:55.554912+0000","last_peered":"2024-04-02T13:33:55.554912+0000","last_clean":"2024-04-02T13:33:55.554912+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:33:55.554912+0000","last_undegraded":"2024-04-02T13:33:55.554912+0000","last_fullsized":"2024-04-02T13:33:55.554912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556315+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:33:55.556315+0000","last_peered":"2024-04-02T13:33:55.556315+0000","last_clean":"2024-04-02T13:33:55.556315+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:33:55.556315+0000","last_undegraded":"2024-04-02T13:33:55.556315+0000","last_fullsized":"2024-04-02T13:33:55.556315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556490+0000","last_change":"2024-04-02T13:33:55.555987+0000","last_active":"2024-04-02T13:33:55.556490+0000","last_peered":"2024-04-02T13:33:55.556490+0000","last_clean":"2024-04-02T13:33:55.556490+0000","last_became_active":"2024-04-02T13:33:55.555623+0000","last_became_peered":"2024-04-02T13:33:55.555623+0000","last_unstale":"2024-04-02T13:33:55.556490+0000","last_undegraded":"2024-04-02T13:33:55.556490+0000","last_fullsized":"2024-04-02T13:33:55.556490+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:32.669173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556489+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:33:55.556489+0000","last_peered":"2024-04-02T13:33:55.556489+0000","last_clean":"2024-04-02T13:33:55.556489+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:33:55.556489+0000","last_undegraded":"2024-04-02T13:33:55.556489+0000","last_fullsized":"2024-04-02T13:33:55.556489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554719+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:33:55.554719+0000","last_peered":"2024-04-02T13:33:55.554719+0000","last_clean":"2024-04-02T13:33:55.554719+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:33:55.554719+0000","last_undegraded":"2024-04-02T13:33:55.554719+0000","last_fullsized":"2024-04-02T13:33:55.554719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554871+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:33:55.554871+0000","last_peered":"2024-04-02T13:33:55.554871+0000","last_clean":"2024-04-02T13:33:55.554871+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:33:55.554871+0000","last_undegraded":"2024-04-02T13:33:55.554871+0000","last_fullsized":"2024-04-02T13:33:55.554871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557286+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:33:55.557286+0000","last_peered":"2024-04-02T13:33:55.557286+0000","last_clean":"2024-04-02T13:33:55.557286+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:33:55.557286+0000","last_undegraded":"2024-04-02T13:33:55.557286+0000","last_fullsized":"2024-04-02T13:33:55.557286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553938+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:33:55.553938+0000","last_peered":"2024-04-02T13:33:55.553938+0000","last_clean":"2024-04-02T13:33:55.553938+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:33:55.553938+0000","last_undegraded":"2024-04-02T13:33:55.553938+0000","last_fullsized":"2024-04-02T13:33:55.553938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551256+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:33:55.551256+0000","last_peered":"2024-04-02T13:33:55.551256+0000","last_clean":"2024-04-02T13:33:55.551256+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:33:55.551256+0000","last_undegraded":"2024-04-02T13:33:55.551256+0000","last_fullsized":"2024-04-02T13:33:55.551256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560137+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:33:55.560137+0000","last_peered":"2024-04-02T13:33:55.560137+0000","last_clean":"2024-04-02T13:33:55.560137+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:33:55.560137+0000","last_undegraded":"2024-04-02T13:33:55.560137+0000","last_fullsized":"2024-04-02T13:33:55.560137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558983+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:33:55.558983+0000","last_peered":"2024-04-02T13:33:55.558983+0000","last_clean":"2024-04-02T13:33:55.558983+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:33:55.558983+0000","last_undegraded":"2024-04-02T13:33:55.558983+0000","last_fullsized":"2024-04-02T13:33:55.558983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:17.246340+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:34:17.246340+0000","last_peered":"2024-04-02T13:34:17.246340+0000","last_clean":"2024-04-02T13:34:17.246340+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:34:17.246340+0000","last_undegraded":"2024-04-02T13:34:17.246340+0000","last_fullsized":"2024-04-02T13:34:17.246340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657619+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:33:55.657619+0000","last_peered":"2024-04-02T13:33:55.657619+0000","last_clean":"2024-04-02T13:33:55.657619+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:33:55.657619+0000","last_undegraded":"2024-04-02T13:33:55.657619+0000","last_fullsized":"2024-04-02T13:33:55.657619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551401+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:33:55.551401+0000","last_peered":"2024-04-02T13:33:55.551401+0000","last_clean":"2024-04-02T13:33:55.551401+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:33:55.551401+0000","last_undegraded":"2024-04-02T13:33:55.551401+0000","last_fullsized":"2024-04-02T13:33:55.551401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657763+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:33:55.657763+0000","last_peered":"2024-04-02T13:33:55.657763+0000","last_clean":"2024-04-02T13:33:55.657763+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:33:55.657763+0000","last_undegraded":"2024-04-02T13:33:55.657763+0000","last_fullsized":"2024-04-02T13:33:55.657763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555002+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:33:55.555002+0000","last_peered":"2024-04-02T13:33:55.555002+0000","last_clean":"2024-04-02T13:33:55.555002+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:33:55.555002+0000","last_undegraded":"2024-04-02T13:33:55.555002+0000","last_fullsized":"2024-04-02T13:33:55.555002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551528+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:33:55.551528+0000","last_peered":"2024-04-02T13:33:55.551528+0000","last_clean":"2024-04-02T13:33:55.551528+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:33:55.551528+0000","last_undegraded":"2024-04-02T13:33:55.551528+0000","last_fullsized":"2024-04-02T13:33:55.551528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.552031+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:33:55.552031+0000","last_peered":"2024-04-02T13:33:55.552031+0000","last_clean":"2024-04-02T13:33:55.552031+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:33:55.552031+0000","last_undegraded":"2024-04-02T13:33:55.552031+0000","last_fullsized":"2024-04-02T13:33:55.552031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557467+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:33:55.557467+0000","last_peered":"2024-04-02T13:33:55.557467+0000","last_clean":"2024-04-02T13:33:55.557467+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:33:55.557467+0000","last_undegraded":"2024-04-02T13:33:55.557467+0000","last_fullsized":"2024-04-02T13:33:55.557467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551483+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:33:55.551483+0000","last_peered":"2024-04-02T13:33:55.551483+0000","last_clean":"2024-04-02T13:33:55.551483+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:33:55.551483+0000","last_undegraded":"2024-04-02T13:33:55.551483+0000","last_fullsized":"2024-04-02T13:33:55.551483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551427+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:33:55.551427+0000","last_peered":"2024-04-02T13:33:55.551427+0000","last_clean":"2024-04-02T13:33:55.551427+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:33:55.551427+0000","last_undegraded":"2024-04-02T13:33:55.551427+0000","last_fullsized":"2024-04-02T13:33:55.551427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559746+0000","last_change":"2024-04-02T13:33:55.559231+0000","last_active":"2024-04-02T13:33:55.559746+0000","last_peered":"2024-04-02T13:33:55.559746+0000","last_clean":"2024-04-02T13:33:55.559746+0000","last_became_active":"2024-04-02T13:33:55.558856+0000","last_became_peered":"2024-04-02T13:33:55.558856+0000","last_unstale":"2024-04-02T13:33:55.559746+0000","last_undegraded":"2024-04-02T13:33:55.559746+0000","last_fullsized":"2024-04-02T13:33:55.559746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:12:47.406198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559868+0000","last_change":"2024-04-02T13:32:54.560990+0000","last_active":"2024-04-02T13:33:55.559868+0000","last_peered":"2024-04-02T13:33:55.559868+0000","last_clean":"2024-04-02T13:33:55.559868+0000","last_became_active":"2024-04-02T13:32:54.560528+0000","last_became_peered":"2024-04-02T13:32:54.560528+0000","last_unstale":"2024-04-02T13:33:55.559868+0000","last_undegraded":"2024-04-02T13:33:55.559868+0000","last_fullsized":"2024-04-02T13:33:55.559868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:41:41.675349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:16.246316+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:34:16.246316+0000","last_peered":"2024-04-02T13:34:16.246316+0000","last_clean":"2024-04-02T13:34:16.246316+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:34:16.246316+0000","last_undegraded":"2024-04-02T13:34:16.246316+0000","last_fullsized":"2024-04-02T13:34:16.246316+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551544+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:33:55.551544+0000","last_peered":"2024-04-02T13:33:55.551544+0000","last_clean":"2024-04-02T13:33:55.551544+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:33:55.551544+0000","last_undegraded":"2024-04-02T13:33:55.551544+0000","last_fullsized":"2024-04-02T13:33:55.551544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559110+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:33:55.559110+0000","last_peered":"2024-04-02T13:33:55.559110+0000","last_clean":"2024-04-02T13:33:55.559110+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:33:55.559110+0000","last_undegraded":"2024-04-02T13:33:55.559110+0000","last_fullsized":"2024-04-02T13:33:55.559110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.565630+0000","last_change":"2024-04-02T13:33:55.565630+0000","last_active":"2024-04-02T13:33:55.565630+0000","last_peered":"2024-04-02T13:33:55.565630+0000","last_clean":"2024-04-02T13:33:55.565630+0000","last_became_active":"2024-04-02T13:33:55.559239+0000","last_became_peered":"2024-04-02T13:33:55.559239+0000","last_unstale":"2024-04-02T13:33:55.565630+0000","last_undegraded":"2024-04-02T13:33:55.565630+0000","last_fullsized":"2024-04-02T13:33:55.565630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:46.161608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555606+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:33:55.555606+0000","last_peered":"2024-04-02T13:33:55.555606+0000","last_clean":"2024-04-02T13:33:55.555606+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:33:55.555606+0000","last_undegraded":"2024-04-02T13:33:55.555606+0000","last_fullsized":"2024-04-02T13:33:55.555606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558751+0000","last_change":"2024-04-02T13:33:55.558751+0000","last_active":"2024-04-02T13:33:55.558751+0000","last_peered":"2024-04-02T13:33:55.558751+0000","last_clean":"2024-04-02T13:33:55.558751+0000","last_became_active":"2024-04-02T13:33:55.558278+0000","last_became_peered":"2024-04-02T13:33:55.558278+0000","last_unstale":"2024-04-02T13:33:55.558751+0000","last_undegraded":"2024-04-02T13:33:55.558751+0000","last_fullsized":"2024-04-02T13:33:55.558751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:15.164871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556460+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:33:55.556460+0000","last_peered":"2024-04-02T13:33:55.556460+0000","last_clean":"2024-04-02T13:33:55.556460+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:33:55.556460+0000","last_undegraded":"2024-04-02T13:33:55.556460+0000","last_fullsized":"2024-04-02T13:33:55.556460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657234+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:33:55.657234+0000","last_peered":"2024-04-02T13:33:55.657234+0000","last_clean":"2024-04-02T13:33:55.657234+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:33:55.657234+0000","last_undegraded":"2024-04-02T13:33:55.657234+0000","last_fullsized":"2024-04-02T13:33:55.657234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555029+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:33:55.555029+0000","last_peered":"2024-04-02T13:33:55.555029+0000","last_clean":"2024-04-02T13:33:55.555029+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:33:55.555029+0000","last_undegraded":"2024-04-02T13:33:55.555029+0000","last_fullsized":"2024-04-02T13:33:55.555029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554795+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:33:55.554795+0000","last_peered":"2024-04-02T13:33:55.554795+0000","last_clean":"2024-04-02T13:33:55.554795+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:33:55.554795+0000","last_undegraded":"2024-04-02T13:33:55.554795+0000","last_fullsized":"2024-04-02T13:33:55.554795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550716+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:33:55.550716+0000","last_peered":"2024-04-02T13:33:55.550716+0000","last_clean":"2024-04-02T13:33:55.550716+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:33:55.550716+0000","last_undegraded":"2024-04-02T13:33:55.550716+0000","last_fullsized":"2024-04-02T13:33:55.550716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657222+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:33:55.657222+0000","last_peered":"2024-04-02T13:33:55.657222+0000","last_clean":"2024-04-02T13:33:55.657222+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:33:55.657222+0000","last_undegraded":"2024-04-02T13:33:55.657222+0000","last_fullsized":"2024-04-02T13:33:55.657222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":168,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:19.250037+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:34:19.250037+0000","last_peered":"2024-04-02T13:34:19.250037+0000","last_clean":"2024-04-02T13:34:19.250037+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:34:19.250037+0000","last_undegraded":"2024-04-02T13:34:19.250037+0000","last_fullsized":"2024-04-02T13:34:19.250037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":138,"num_read_kb":138,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557497+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:33:55.557497+0000","last_peered":"2024-04-02T13:33:55.557497+0000","last_clean":"2024-04-02T13:33:55.557497+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:33:55.557497+0000","last_undegraded":"2024-04-02T13:33:55.557497+0000","last_fullsized":"2024-04-02T13:33:55.557497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550685+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:33:55.550685+0000","last_peered":"2024-04-02T13:33:55.550685+0000","last_clean":"2024-04-02T13:33:55.550685+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:33:55.550685+0000","last_undegraded":"2024-04-02T13:33:55.550685+0000","last_fullsized":"2024-04-02T13:33:55.550685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555813+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:33:55.555813+0000","last_peered":"2024-04-02T13:33:55.555813+0000","last_clean":"2024-04-02T13:33:55.555813+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:33:55.555813+0000","last_undegraded":"2024-04-02T13:33:55.555813+0000","last_fullsized":"2024-04-02T13:33:55.555813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555235+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:33:55.555235+0000","last_peered":"2024-04-02T13:33:55.555235+0000","last_clean":"2024-04-02T13:33:55.555235+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:33:55.555235+0000","last_undegraded":"2024-04-02T13:33:55.555235+0000","last_fullsized":"2024-04-02T13:33:55.555235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551825+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:33:55.551825+0000","last_peered":"2024-04-02T13:33:55.551825+0000","last_clean":"2024-04-02T13:33:55.551825+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:33:55.551825+0000","last_undegraded":"2024-04-02T13:33:55.551825+0000","last_fullsized":"2024-04-02T13:33:55.551825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556321+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:33:55.556321+0000","last_peered":"2024-04-02T13:33:55.556321+0000","last_clean":"2024-04-02T13:33:55.556321+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:33:55.556321+0000","last_undegraded":"2024-04-02T13:33:55.556321+0000","last_fullsized":"2024-04-02T13:33:55.556321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557659+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:33:55.557659+0000","last_peered":"2024-04-02T13:33:55.557659+0000","last_clean":"2024-04-02T13:33:55.557659+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:33:55.557659+0000","last_undegraded":"2024-04-02T13:33:55.557659+0000","last_fullsized":"2024-04-02T13:33:55.557659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550714+0000","last_change":"2024-04-02T13:32:54.562030+0000","last_active":"2024-04-02T13:33:55.550714+0000","last_peered":"2024-04-02T13:33:55.550714+0000","last_clean":"2024-04-02T13:33:55.550714+0000","last_became_active":"2024-04-02T13:32:54.560872+0000","last_became_peered":"2024-04-02T13:32:54.560872+0000","last_unstale":"2024-04-02T13:33:55.550714+0000","last_undegraded":"2024-04-02T13:33:55.550714+0000","last_fullsized":"2024-04-02T13:33:55.550714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:25:08.994729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'36","reported_seq":207,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:19.249788+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:34:19.249788+0000","last_peered":"2024-04-02T13:34:19.249788+0000","last_clean":"2024-04-02T13:34:19.249788+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:34:19.249788+0000","last_undegraded":"2024-04-02T13:34:19.249788+0000","last_fullsized":"2024-04-02T13:34:19.249788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62333,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":145,"num_read_kb":133,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551765+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:33:55.551765+0000","last_peered":"2024-04-02T13:33:55.551765+0000","last_clean":"2024-04-02T13:33:55.551765+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:33:55.551765+0000","last_undegraded":"2024-04-02T13:33:55.551765+0000","last_fullsized":"2024-04-02T13:33:55.551765+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556196+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:33:55.556196+0000","last_peered":"2024-04-02T13:33:55.556196+0000","last_clean":"2024-04-02T13:33:55.556196+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:33:55.556196+0000","last_undegraded":"2024-04-02T13:33:55.556196+0000","last_fullsized":"2024-04-02T13:33:55.556196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558045+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:33:55.558045+0000","last_peered":"2024-04-02T13:33:55.558045+0000","last_clean":"2024-04-02T13:33:55.558045+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:33:55.558045+0000","last_undegraded":"2024-04-02T13:33:55.558045+0000","last_fullsized":"2024-04-02T13:33:55.558045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556260+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:33:55.556260+0000","last_peered":"2024-04-02T13:33:55.556260+0000","last_clean":"2024-04-02T13:33:55.556260+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:33:55.556260+0000","last_undegraded":"2024-04-02T13:33:55.556260+0000","last_fullsized":"2024-04-02T13:33:55.556260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553268+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:33:55.553268+0000","last_peered":"2024-04-02T13:33:55.553268+0000","last_clean":"2024-04-02T13:33:55.553268+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:33:55.553268+0000","last_undegraded":"2024-04-02T13:33:55.553268+0000","last_fullsized":"2024-04-02T13:33:55.553268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555434+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:33:55.555434+0000","last_peered":"2024-04-02T13:33:55.555434+0000","last_clean":"2024-04-02T13:33:55.555434+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:33:55.555434+0000","last_undegraded":"2024-04-02T13:33:55.555434+0000","last_fullsized":"2024-04-02T13:33:55.555434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556079+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:33:55.556079+0000","last_peered":"2024-04-02T13:33:55.556079+0000","last_clean":"2024-04-02T13:33:55.556079+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:33:55.556079+0000","last_undegraded":"2024-04-02T13:33:55.556079+0000","last_fullsized":"2024-04-02T13:33:55.556079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551106+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:33:55.551106+0000","last_peered":"2024-04-02T13:33:55.551106+0000","last_clean":"2024-04-02T13:33:55.551106+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:33:55.551106+0000","last_undegraded":"2024-04-02T13:33:55.551106+0000","last_fullsized":"2024-04-02T13:33:55.551106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:17.247050+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:34:17.247050+0000","last_peered":"2024-04-02T13:34:17.247050+0000","last_clean":"2024-04-02T13:34:17.247050+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:34:17.247050+0000","last_undegraded":"2024-04-02T13:34:17.247050+0000","last_fullsized":"2024-04-02T13:34:17.247050+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555138+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:33:55.555138+0000","last_peered":"2024-04-02T13:33:55.555138+0000","last_clean":"2024-04-02T13:33:55.555138+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:33:55.555138+0000","last_undegraded":"2024-04-02T13:33:55.555138+0000","last_fullsized":"2024-04-02T13:33:55.555138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556186+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:33:55.556186+0000","last_peered":"2024-04-02T13:33:55.556186+0000","last_clean":"2024-04-02T13:33:55.556186+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:33:55.556186+0000","last_undegraded":"2024-04-02T13:33:55.556186+0000","last_fullsized":"2024-04-02T13:33:55.556186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.574777+0000","last_change":"2024-04-02T13:33:55.574777+0000","last_active":"2024-04-02T13:33:55.574777+0000","last_peered":"2024-04-02T13:33:55.574777+0000","last_clean":"2024-04-02T13:33:55.574777+0000","last_became_active":"2024-04-02T13:33:55.561040+0000","last_became_peered":"2024-04-02T13:33:55.561040+0000","last_unstale":"2024-04-02T13:33:55.574777+0000","last_undegraded":"2024-04-02T13:33:55.574777+0000","last_fullsized":"2024-04-02T13:33:55.574777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:23.184540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560045+0000","last_change":"2024-04-02T13:33:55.560045+0000","last_active":"2024-04-02T13:33:55.560045+0000","last_peered":"2024-04-02T13:33:55.560045+0000","last_clean":"2024-04-02T13:33:55.560045+0000","last_became_active":"2024-04-02T13:33:55.559659+0000","last_became_peered":"2024-04-02T13:33:55.559659+0000","last_unstale":"2024-04-02T13:33:55.560045+0000","last_undegraded":"2024-04-02T13:33:55.560045+0000","last_fullsized":"2024-04-02T13:33:55.560045+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:10:42.028143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556260+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:33:55.556260+0000","last_peered":"2024-04-02T13:33:55.556260+0000","last_clean":"2024-04-02T13:33:55.556260+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:33:55.556260+0000","last_undegraded":"2024-04-02T13:33:55.556260+0000","last_fullsized":"2024-04-02T13:33:55.556260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551719+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:33:55.551719+0000","last_peered":"2024-04-02T13:33:55.551719+0000","last_clean":"2024-04-02T13:33:55.551719+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:33:55.551719+0000","last_undegraded":"2024-04-02T13:33:55.551719+0000","last_fullsized":"2024-04-02T13:33:55.551719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553535+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:33:55.553535+0000","last_peered":"2024-04-02T13:33:55.553535+0000","last_clean":"2024-04-02T13:33:55.553535+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:33:55.553535+0000","last_undegraded":"2024-04-02T13:33:55.553535+0000","last_fullsized":"2024-04-02T13:33:55.553535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551384+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:33:55.551384+0000","last_peered":"2024-04-02T13:33:55.551384+0000","last_clean":"2024-04-02T13:33:55.551384+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:33:55.551384+0000","last_undegraded":"2024-04-02T13:33:55.551384+0000","last_fullsized":"2024-04-02T13:33:55.551384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551007+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:33:55.551007+0000","last_peered":"2024-04-02T13:33:55.551007+0000","last_clean":"2024-04-02T13:33:55.551007+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:33:55.551007+0000","last_undegraded":"2024-04-02T13:33:55.551007+0000","last_fullsized":"2024-04-02T13:33:55.551007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555062+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:33:55.555062+0000","last_peered":"2024-04-02T13:33:55.555062+0000","last_clean":"2024-04-02T13:33:55.555062+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:33:55.555062+0000","last_undegraded":"2024-04-02T13:33:55.555062+0000","last_fullsized":"2024-04-02T13:33:55.555062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556228+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:33:55.556228+0000","last_peered":"2024-04-02T13:33:55.556228+0000","last_clean":"2024-04-02T13:33:55.556228+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:33:55.556228+0000","last_undegraded":"2024-04-02T13:33:55.556228+0000","last_fullsized":"2024-04-02T13:33:55.556228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:20.250652+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:34:20.250652+0000","last_peered":"2024-04-02T13:34:20.250652+0000","last_clean":"2024-04-02T13:34:20.250652+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:34:20.250652+0000","last_undegraded":"2024-04-02T13:34:20.250652+0000","last_fullsized":"2024-04-02T13:34:20.250652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554299+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:33:55.554299+0000","last_peered":"2024-04-02T13:33:55.554299+0000","last_clean":"2024-04-02T13:33:55.554299+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:33:55.554299+0000","last_undegraded":"2024-04-02T13:33:55.554299+0000","last_fullsized":"2024-04-02T13:33:55.554299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555936+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:33:55.555936+0000","last_peered":"2024-04-02T13:33:55.555936+0000","last_clean":"2024-04-02T13:33:55.555936+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:33:55.555936+0000","last_undegraded":"2024-04-02T13:33:55.555936+0000","last_fullsized":"2024-04-02T13:33:55.555936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555216+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:33:55.555216+0000","last_peered":"2024-04-02T13:33:55.555216+0000","last_clean":"2024-04-02T13:33:55.555216+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:33:55.555216+0000","last_undegraded":"2024-04-02T13:33:55.555216+0000","last_fullsized":"2024-04-02T13:33:55.555216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:20.250209+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:34:20.250209+0000","last_peered":"2024-04-02T13:34:20.250209+0000","last_clean":"2024-04-02T13:34:20.250209+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:34:20.250209+0000","last_undegraded":"2024-04-02T13:34:20.250209+0000","last_fullsized":"2024-04-02T13:34:20.250209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556151+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:33:55.556151+0000","last_peered":"2024-04-02T13:33:55.556151+0000","last_clean":"2024-04-02T13:33:55.556151+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:33:55.556151+0000","last_undegraded":"2024-04-02T13:33:55.556151+0000","last_fullsized":"2024-04-02T13:33:55.556151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657274+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:33:55.657274+0000","last_peered":"2024-04-02T13:33:55.657274+0000","last_clean":"2024-04-02T13:33:55.657274+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:33:55.657274+0000","last_undegraded":"2024-04-02T13:33:55.657274+0000","last_fullsized":"2024-04-02T13:33:55.657274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551675+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:33:55.551675+0000","last_peered":"2024-04-02T13:33:55.551675+0000","last_clean":"2024-04-02T13:33:55.551675+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:33:55.551675+0000","last_undegraded":"2024-04-02T13:33:55.551675+0000","last_fullsized":"2024-04-02T13:33:55.551675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554971+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:33:55.554971+0000","last_peered":"2024-04-02T13:33:55.554971+0000","last_clean":"2024-04-02T13:33:55.554971+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:33:55.554971+0000","last_undegraded":"2024-04-02T13:33:55.554971+0000","last_fullsized":"2024-04-02T13:33:55.554971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555638+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:33:55.555638+0000","last_peered":"2024-04-02T13:33:55.555638+0000","last_clean":"2024-04-02T13:33:55.555638+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:33:55.555638+0000","last_undegraded":"2024-04-02T13:33:55.555638+0000","last_fullsized":"2024-04-02T13:33:55.555638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556076+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:33:55.556076+0000","last_peered":"2024-04-02T13:33:55.556076+0000","last_clean":"2024-04-02T13:33:55.556076+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:33:55.556076+0000","last_undegraded":"2024-04-02T13:33:55.556076+0000","last_fullsized":"2024-04-02T13:33:55.556076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555030+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:33:55.555030+0000","last_peered":"2024-04-02T13:33:55.555030+0000","last_clean":"2024-04-02T13:33:55.555030+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:33:55.555030+0000","last_undegraded":"2024-04-02T13:33:55.555030+0000","last_fullsized":"2024-04-02T13:33:55.555030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553258+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:33:55.553258+0000","last_peered":"2024-04-02T13:33:55.553258+0000","last_clean":"2024-04-02T13:33:55.553258+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:33:55.553258+0000","last_undegraded":"2024-04-02T13:33:55.553258+0000","last_fullsized":"2024-04-02T13:33:55.553258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551795+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:33:55.551795+0000","last_peered":"2024-04-02T13:33:55.551795+0000","last_clean":"2024-04-02T13:33:55.551795+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:33:55.551795+0000","last_undegraded":"2024-04-02T13:33:55.551795+0000","last_fullsized":"2024-04-02T13:33:55.551795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556381+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:33:55.556381+0000","last_peered":"2024-04-02T13:33:55.556381+0000","last_clean":"2024-04-02T13:33:55.556381+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:33:55.556381+0000","last_undegraded":"2024-04-02T13:33:55.556381+0000","last_fullsized":"2024-04-02T13:33:55.556381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657124+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:33:55.657124+0000","last_peered":"2024-04-02T13:33:55.657124+0000","last_clean":"2024-04-02T13:33:55.657124+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:33:55.657124+0000","last_undegraded":"2024-04-02T13:33:55.657124+0000","last_fullsized":"2024-04-02T13:33:55.657124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551366+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:33:55.551366+0000","last_peered":"2024-04-02T13:33:55.551366+0000","last_clean":"2024-04-02T13:33:55.551366+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:33:55.551366+0000","last_undegraded":"2024-04-02T13:33:55.551366+0000","last_fullsized":"2024-04-02T13:33:55.551366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657417+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:33:55.657417+0000","last_peered":"2024-04-02T13:33:55.657417+0000","last_clean":"2024-04-02T13:33:55.657417+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:33:55.657417+0000","last_undegraded":"2024-04-02T13:33:55.657417+0000","last_fullsized":"2024-04-02T13:33:55.657417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558670+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:33:55.558670+0000","last_peered":"2024-04-02T13:33:55.558670+0000","last_clean":"2024-04-02T13:33:55.558670+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:33:55.558670+0000","last_undegraded":"2024-04-02T13:33:55.558670+0000","last_fullsized":"2024-04-02T13:33:55.558670+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.552138+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:33:55.552138+0000","last_peered":"2024-04-02T13:33:55.552138+0000","last_clean":"2024-04-02T13:33:55.552138+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:33:55.552138+0000","last_undegraded":"2024-04-02T13:33:55.552138+0000","last_fullsized":"2024-04-02T13:33:55.552138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555149+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:33:55.555149+0000","last_peered":"2024-04-02T13:33:55.555149+0000","last_clean":"2024-04-02T13:33:55.555149+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:33:55.555149+0000","last_undegraded":"2024-04-02T13:33:55.555149+0000","last_fullsized":"2024-04-02T13:33:55.555149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551989+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:33:55.551989+0000","last_peered":"2024-04-02T13:33:55.551989+0000","last_clean":"2024-04-02T13:33:55.551989+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:33:55.551989+0000","last_undegraded":"2024-04-02T13:33:55.551989+0000","last_fullsized":"2024-04-02T13:33:55.551989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.564335+0000","last_change":"2024-04-02T13:33:55.564335+0000","last_active":"2024-04-02T13:33:55.564335+0000","last_peered":"2024-04-02T13:33:55.564335+0000","last_clean":"2024-04-02T13:33:55.564335+0000","last_became_active":"2024-04-02T13:33:55.560370+0000","last_became_peered":"2024-04-02T13:33:55.560370+0000","last_unstale":"2024-04-02T13:33:55.564335+0000","last_undegraded":"2024-04-02T13:33:55.564335+0000","last_fullsized":"2024-04-02T13:33:55.564335+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:14.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551153+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:33:55.551153+0000","last_peered":"2024-04-02T13:33:55.551153+0000","last_clean":"2024-04-02T13:33:55.551153+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:33:55.551153+0000","last_undegraded":"2024-04-02T13:33:55.551153+0000","last_fullsized":"2024-04-02T13:33:55.551153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551206+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:33:55.551206+0000","last_peered":"2024-04-02T13:33:55.551206+0000","last_clean":"2024-04-02T13:33:55.551206+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:33:55.551206+0000","last_undegraded":"2024-04-02T13:33:55.551206+0000","last_fullsized":"2024-04-02T13:33:55.551206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657253+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:33:55.657253+0000","last_peered":"2024-04-02T13:33:55.657253+0000","last_clean":"2024-04-02T13:33:55.657253+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:33:55.657253+0000","last_undegraded":"2024-04-02T13:33:55.657253+0000","last_fullsized":"2024-04-02T13:33:55.657253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553707+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:33:55.553707+0000","last_peered":"2024-04-02T13:33:55.553707+0000","last_clean":"2024-04-02T13:33:55.553707+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:33:55.553707+0000","last_undegraded":"2024-04-02T13:33:55.553707+0000","last_fullsized":"2024-04-02T13:33:55.553707+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551496+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:33:55.551496+0000","last_peered":"2024-04-02T13:33:55.551496+0000","last_clean":"2024-04-02T13:33:55.551496+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:33:55.551496+0000","last_undegraded":"2024-04-02T13:33:55.551496+0000","last_fullsized":"2024-04-02T13:33:55.551496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557873+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:33:55.557873+0000","last_peered":"2024-04-02T13:33:55.557873+0000","last_clean":"2024-04-02T13:33:55.557873+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:33:55.557873+0000","last_undegraded":"2024-04-02T13:33:55.557873+0000","last_fullsized":"2024-04-02T13:33:55.557873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657361+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:33:55.657361+0000","last_peered":"2024-04-02T13:33:55.657361+0000","last_clean":"2024-04-02T13:33:55.657361+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:33:55.657361+0000","last_undegraded":"2024-04-02T13:33:55.657361+0000","last_fullsized":"2024-04-02T13:33:55.657361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551777+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:33:55.551777+0000","last_peered":"2024-04-02T13:33:55.551777+0000","last_clean":"2024-04-02T13:33:55.551777+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:33:55.551777+0000","last_undegraded":"2024-04-02T13:33:55.551777+0000","last_fullsized":"2024-04-02T13:33:55.551777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.565608+0000","last_change":"2024-04-02T13:33:55.565608+0000","last_active":"2024-04-02T13:33:55.565608+0000","last_peered":"2024-04-02T13:33:55.565608+0000","last_clean":"2024-04-02T13:33:55.565608+0000","last_became_active":"2024-04-02T13:33:55.557475+0000","last_became_peered":"2024-04-02T13:33:55.557475+0000","last_unstale":"2024-04-02T13:33:55.565608+0000","last_undegraded":"2024-04-02T13:33:55.565608+0000","last_fullsized":"2024-04-02T13:33:55.565608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:17:21.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555739+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:33:55.555739+0000","last_peered":"2024-04-02T13:33:55.555739+0000","last_clean":"2024-04-02T13:33:55.555739+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:33:55.555739+0000","last_undegraded":"2024-04-02T13:33:55.555739+0000","last_fullsized":"2024-04-02T13:33:55.555739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558108+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:33:55.558108+0000","last_peered":"2024-04-02T13:33:55.558108+0000","last_clean":"2024-04-02T13:33:55.558108+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:33:55.558108+0000","last_undegraded":"2024-04-02T13:33:55.558108+0000","last_fullsized":"2024-04-02T13:33:55.558108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:17.247756+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:34:17.247756+0000","last_peered":"2024-04-02T13:34:17.247756+0000","last_clean":"2024-04-02T13:34:17.247756+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:34:17.247756+0000","last_undegraded":"2024-04-02T13:34:17.247756+0000","last_fullsized":"2024-04-02T13:34:17.247756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556371+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:33:55.556371+0000","last_peered":"2024-04-02T13:33:55.556371+0000","last_clean":"2024-04-02T13:33:55.556371+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:33:55.556371+0000","last_undegraded":"2024-04-02T13:33:55.556371+0000","last_fullsized":"2024-04-02T13:33:55.556371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554680+0000","last_change":"2024-04-02T13:32:54.566345+0000","last_active":"2024-04-02T13:33:55.554680+0000","last_peered":"2024-04-02T13:33:55.554680+0000","last_clean":"2024-04-02T13:33:55.554680+0000","last_became_active":"2024-04-02T13:32:54.564927+0000","last_became_peered":"2024-04-02T13:32:54.564927+0000","last_unstale":"2024-04-02T13:33:55.554680+0000","last_undegraded":"2024-04-02T13:33:55.554680+0000","last_fullsized":"2024-04-02T13:33:55.554680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:34.439915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555903+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:33:55.555903+0000","last_peered":"2024-04-02T13:33:55.555903+0000","last_clean":"2024-04-02T13:33:55.555903+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:33:55.555903+0000","last_undegraded":"2024-04-02T13:33:55.555903+0000","last_fullsized":"2024-04-02T13:33:55.555903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555933+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:33:55.555933+0000","last_peered":"2024-04-02T13:33:55.555933+0000","last_clean":"2024-04-02T13:33:55.555933+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:33:55.555933+0000","last_undegraded":"2024-04-02T13:33:55.555933+0000","last_fullsized":"2024-04-02T13:33:55.555933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554572+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:33:55.554572+0000","last_peered":"2024-04-02T13:33:55.554572+0000","last_clean":"2024-04-02T13:33:55.554572+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:33:55.554572+0000","last_undegraded":"2024-04-02T13:33:55.554572+0000","last_fullsized":"2024-04-02T13:33:55.554572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556309+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:33:55.556309+0000","last_peered":"2024-04-02T13:33:55.556309+0000","last_clean":"2024-04-02T13:33:55.556309+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:33:55.556309+0000","last_undegraded":"2024-04-02T13:33:55.556309+0000","last_fullsized":"2024-04-02T13:33:55.556309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556190+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:33:55.556190+0000","last_peered":"2024-04-02T13:33:55.556190+0000","last_clean":"2024-04-02T13:33:55.556190+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:33:55.556190+0000","last_undegraded":"2024-04-02T13:33:55.556190+0000","last_fullsized":"2024-04-02T13:33:55.556190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553811+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:33:55.553811+0000","last_peered":"2024-04-02T13:33:55.553811+0000","last_clean":"2024-04-02T13:33:55.553811+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:33:55.553811+0000","last_undegraded":"2024-04-02T13:33:55.553811+0000","last_fullsized":"2024-04-02T13:33:55.553811+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.562541+0000","last_change":"2024-04-02T13:33:55.562541+0000","last_active":"2024-04-02T13:33:55.562541+0000","last_peered":"2024-04-02T13:33:55.562541+0000","last_clean":"2024-04-02T13:33:55.562541+0000","last_became_active":"2024-04-02T13:33:55.562167+0000","last_became_peered":"2024-04-02T13:33:55.562167+0000","last_unstale":"2024-04-02T13:33:55.562541+0000","last_undegraded":"2024-04-02T13:33:55.562541+0000","last_fullsized":"2024-04-02T13:33:55.562541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:29.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557710+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:33:55.557710+0000","last_peered":"2024-04-02T13:33:55.557710+0000","last_clean":"2024-04-02T13:33:55.557710+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:33:55.557710+0000","last_undegraded":"2024-04-02T13:33:55.557710+0000","last_fullsized":"2024-04-02T13:33:55.557710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554863+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:33:55.554863+0000","last_peered":"2024-04-02T13:33:55.554863+0000","last_clean":"2024-04-02T13:33:55.554863+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:33:55.554863+0000","last_undegraded":"2024-04-02T13:33:55.554863+0000","last_fullsized":"2024-04-02T13:33:55.554863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556327+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:33:55.556327+0000","last_peered":"2024-04-02T13:33:55.556327+0000","last_clean":"2024-04-02T13:33:55.556327+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:33:55.556327+0000","last_undegraded":"2024-04-02T13:33:55.556327+0000","last_fullsized":"2024-04-02T13:33:55.556327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553685+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:33:55.553685+0000","last_peered":"2024-04-02T13:33:55.553685+0000","last_clean":"2024-04-02T13:33:55.553685+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:33:55.553685+0000","last_undegraded":"2024-04-02T13:33:55.553685+0000","last_fullsized":"2024-04-02T13:33:55.553685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560478+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:33:55.560478+0000","last_peered":"2024-04-02T13:33:55.560478+0000","last_clean":"2024-04-02T13:33:55.560478+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:33:55.560478+0000","last_undegraded":"2024-04-02T13:33:55.560478+0000","last_fullsized":"2024-04-02T13:33:55.560478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657576+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:33:55.657576+0000","last_peered":"2024-04-02T13:33:55.657576+0000","last_clean":"2024-04-02T13:33:55.657576+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:33:55.657576+0000","last_undegraded":"2024-04-02T13:33:55.657576+0000","last_fullsized":"2024-04-02T13:33:55.657576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554911+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:33:55.554911+0000","last_peered":"2024-04-02T13:33:55.554911+0000","last_clean":"2024-04-02T13:33:55.554911+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:33:55.554911+0000","last_undegraded":"2024-04-02T13:33:55.554911+0000","last_fullsized":"2024-04-02T13:33:55.554911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657399+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:33:55.657399+0000","last_peered":"2024-04-02T13:33:55.657399+0000","last_clean":"2024-04-02T13:33:55.657399+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:33:55.657399+0000","last_undegraded":"2024-04-02T13:33:55.657399+0000","last_fullsized":"2024-04-02T13:33:55.657399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558369+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:33:55.558369+0000","last_peered":"2024-04-02T13:33:55.558369+0000","last_clean":"2024-04-02T13:33:55.558369+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:33:55.558369+0000","last_undegraded":"2024-04-02T13:33:55.558369+0000","last_fullsized":"2024-04-02T13:33:55.558369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556249+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:33:55.556249+0000","last_peered":"2024-04-02T13:33:55.556249+0000","last_clean":"2024-04-02T13:33:55.556249+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:33:55.556249+0000","last_undegraded":"2024-04-02T13:33:55.556249+0000","last_fullsized":"2024-04-02T13:33:55.556249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553888+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:33:55.553888+0000","last_peered":"2024-04-02T13:33:55.553888+0000","last_clean":"2024-04-02T13:33:55.553888+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:33:55.553888+0000","last_undegraded":"2024-04-02T13:33:55.553888+0000","last_fullsized":"2024-04-02T13:33:55.553888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68755,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":723,"num_read_kb":438,"num_write":241,"num_write_kb":140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":206265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":506039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1740800,"data_stored":1155076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":509953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037807,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":506123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1102341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":501361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":564724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":502817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411404,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":564944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182471,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":501069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920835,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":505435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724606,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1097923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:34:24.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph pg dump --format=json 2024-04-02T13:34:24.967 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:25 smithi122 ceph-mon[32114]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:25 smithi122 ceph-mon[32114]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T13:34:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:25 smithi187 ceph-mon[36064]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:25 smithi187 ceph-mon[36064]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T13:34:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:25 smithi188 ceph-mon[36117]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:25 smithi188 ceph-mon[36117]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 170 B/s wr, 11 op/s 2024-04-02T13:34:26.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 -- 172.21.15.122:0/1406285308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e81079b0 msgr2=0x7f37e810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:26.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 --2- 172.21.15.122:0/1406285308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e81079b0 0x7f37e810f530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f37e000b0d0 tx=0x7f37e000b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 -- 172.21.15.122:0/1406285308 shutdown_connections 2024-04-02T13:34:26.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 --2- 172.21.15.122:0/1406285308 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e81079b0 0x7f37e810f530 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 --2- 172.21.15.122:0/1406285308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101190 0x7f37e8101570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 --2- 172.21.15.122:0/1406285308 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e8101b40 0x7f37e8107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 -- 172.21.15.122:0/1406285308 >> 172.21.15.122:0/1406285308 conn(0x7f37e8076830 msgr2=0x7f37e8076c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 -- 172.21.15.122:0/1406285308 shutdown_connections 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.664+0000 7f37f0174700 1 -- 172.21.15.122:0/1406285308 wait complete. 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 Processor -- start 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 -- start start 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101b40 0x7f37e8199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e81079b0 0x7f37e819d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81129c0 con 0x7f37e8101190 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8112840 con 0x7f37e81079b0 2024-04-02T13:34:26.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37f0174700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8112b40 con 0x7f37e8101b40 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37edf10700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37edf10700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39324/0 (socket says 172.21.15.122:39324) 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37edf10700 1 -- 172.21.15.122:0/3684758975 learned_addr learned my addr 172.21.15.122:0/3684758975 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37ed70f700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101b40 0x7f37e8199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.665+0000 7f37ee711700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e81079b0 0x7f37e819d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 -- 172.21.15.122:0/3684758975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101b40 msgr2=0x7f37e8199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101b40 0x7f37e8199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 -- 172.21.15.122:0/3684758975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e81079b0 msgr2=0x7f37e819d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e81079b0 0x7f37e819d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 -- 172.21.15.122:0/3684758975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819ddb0 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37edf10700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b530 tx=0x7f37d800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8015020 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e819e0a0 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d8011d10 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d800f2e0 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.666+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a5920 con 0x7f37e8101190 2024-04-02T13:34:26.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.667+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e80692b0 con 0x7f37e8101190 2024-04-02T13:34:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.668+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f37d8018020 con 0x7f37e8101190 2024-04-02T13:34:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.669+0000 7f37deffd700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.669+0000 7f37ed70f700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.669+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f37d80cfe10 con 0x7f37e8101190 2024-04-02T13:34:26.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.669+0000 7f37ed70f700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f37e8108d70 tx=0x7f37e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:26.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.673+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f37d801f020 con 0x7f37e8101190 2024-04-02T13:34:26.676 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:26 smithi122 ceph-mon[32114]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-02T13:34:26.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.867+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37e8062a30 con 0x7f37d40a3c40 2024-04-02T13:34:26.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.876+0000 7f37deffd700 1 -- 172.21.15.122:0/3684758975 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340453 (secure 0 0 0) 0x7f37e8062a30 con 0x7f37d40a3c40 2024-04-02T13:34:26.883 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:26.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 msgr2=0x7f37d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:26.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 0x7f37d40a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f37e8108d70 tx=0x7f37e400b410 comp rx=0 tx=0).stop 2024-04-02T13:34:26.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 msgr2=0x7f37e8198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:26.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b530 tx=0x7f37d800b840 comp rx=0 tx=0).stop 2024-04-02T13:34:26.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 shutdown_connections 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37e81079b0 0x7f37e819d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f37d40a3c40 0x7f37d40a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37e8101190 0x7f37e8198df0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.879+0000 7f37f0174700 1 --2- 172.21.15.122:0/3684758975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37e8101b40 0x7f37e8199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.880+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 >> 172.21.15.122:0/3684758975 conn(0x7f37e8076830 msgr2=0x7f37e80fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.880+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 shutdown_connections 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:26.880+0000 7f37f0174700 1 -- 172.21.15.122:0/3684758975 wait complete. 2024-04-02T13:34:26.887 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:34:26.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:26 smithi187 ceph-mon[36064]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-02T13:34:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:26 smithi188 ceph-mon[36117]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-02T13:34:27.727 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-02T13:34:26.490298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659530,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":899,"num_read_kb":577,"num_write":426,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":423,"ondisk_log_size":423,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380252,"kb_used_data":14384,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561572,"statfs":{"total":1151940427776,"available":1151551049728,"internally_reserved":0,"allocated":14729216,"data_stored":8017805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":173,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012566"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.486828+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:34:24.486828+0000","last_peered":"2024-04-02T13:34:24.486828+0000","last_clean":"2024-04-02T13:34:24.486828+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:34:24.486828+0000","last_undegraded":"2024-04-02T13:34:24.486828+0000","last_fullsized":"2024-04-02T13:34:24.486828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555120+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:33:55.555120+0000","last_peered":"2024-04-02T13:33:55.555120+0000","last_clean":"2024-04-02T13:33:55.555120+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:33:55.555120+0000","last_undegraded":"2024-04-02T13:33:55.555120+0000","last_fullsized":"2024-04-02T13:33:55.555120+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554964+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:33:55.554964+0000","last_peered":"2024-04-02T13:33:55.554964+0000","last_clean":"2024-04-02T13:33:55.554964+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:33:55.554964+0000","last_undegraded":"2024-04-02T13:33:55.554964+0000","last_fullsized":"2024-04-02T13:33:55.554964+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551897+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:33:55.551897+0000","last_peered":"2024-04-02T13:33:55.551897+0000","last_clean":"2024-04-02T13:33:55.551897+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:33:55.551897+0000","last_undegraded":"2024-04-02T13:33:55.551897+0000","last_fullsized":"2024-04-02T13:33:55.551897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.468018+0000","last_change":"2024-04-02T13:33:55.560120+0000","last_active":"2024-04-02T13:34:24.468018+0000","last_peered":"2024-04-02T13:34:24.468018+0000","last_clean":"2024-04-02T13:34:24.468018+0000","last_became_active":"2024-04-02T13:33:55.559117+0000","last_became_peered":"2024-04-02T13:33:55.559117+0000","last_unstale":"2024-04-02T13:34:24.468018+0000","last_undegraded":"2024-04-02T13:34:24.468018+0000","last_fullsized":"2024-04-02T13:34:24.468018+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:03:42.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554581+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:33:55.554581+0000","last_peered":"2024-04-02T13:33:55.554581+0000","last_clean":"2024-04-02T13:33:55.554581+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:33:55.554581+0000","last_undegraded":"2024-04-02T13:33:55.554581+0000","last_fullsized":"2024-04-02T13:33:55.554581+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555398+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:33:55.555398+0000","last_peered":"2024-04-02T13:33:55.555398+0000","last_clean":"2024-04-02T13:33:55.555398+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:33:55.555398+0000","last_undegraded":"2024-04-02T13:33:55.555398+0000","last_fullsized":"2024-04-02T13:33:55.555398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657098+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:33:55.657098+0000","last_peered":"2024-04-02T13:33:55.657098+0000","last_clean":"2024-04-02T13:33:55.657098+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:33:55.657098+0000","last_undegraded":"2024-04-02T13:33:55.657098+0000","last_fullsized":"2024-04-02T13:33:55.657098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557525+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:33:55.557525+0000","last_peered":"2024-04-02T13:33:55.557525+0000","last_clean":"2024-04-02T13:33:55.557525+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:33:55.557525+0000","last_undegraded":"2024-04-02T13:33:55.557525+0000","last_fullsized":"2024-04-02T13:33:55.557525+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554912+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:33:55.554912+0000","last_peered":"2024-04-02T13:33:55.554912+0000","last_clean":"2024-04-02T13:33:55.554912+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:33:55.554912+0000","last_undegraded":"2024-04-02T13:33:55.554912+0000","last_fullsized":"2024-04-02T13:33:55.554912+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556315+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:33:55.556315+0000","last_peered":"2024-04-02T13:33:55.556315+0000","last_clean":"2024-04-02T13:33:55.556315+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:33:55.556315+0000","last_undegraded":"2024-04-02T13:33:55.556315+0000","last_fullsized":"2024-04-02T13:33:55.556315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556490+0000","last_change":"2024-04-02T13:33:55.555987+0000","last_active":"2024-04-02T13:33:55.556490+0000","last_peered":"2024-04-02T13:33:55.556490+0000","last_clean":"2024-04-02T13:33:55.556490+0000","last_became_active":"2024-04-02T13:33:55.555623+0000","last_became_peered":"2024-04-02T13:33:55.555623+0000","last_unstale":"2024-04-02T13:33:55.556490+0000","last_undegraded":"2024-04-02T13:33:55.556490+0000","last_fullsized":"2024-04-02T13:33:55.556490+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:32.669173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556489+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:33:55.556489+0000","last_peered":"2024-04-02T13:33:55.556489+0000","last_clean":"2024-04-02T13:33:55.556489+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:33:55.556489+0000","last_undegraded":"2024-04-02T13:33:55.556489+0000","last_fullsized":"2024-04-02T13:33:55.556489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554719+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:33:55.554719+0000","last_peered":"2024-04-02T13:33:55.554719+0000","last_clean":"2024-04-02T13:33:55.554719+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:33:55.554719+0000","last_undegraded":"2024-04-02T13:33:55.554719+0000","last_fullsized":"2024-04-02T13:33:55.554719+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554871+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:33:55.554871+0000","last_peered":"2024-04-02T13:33:55.554871+0000","last_clean":"2024-04-02T13:33:55.554871+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:33:55.554871+0000","last_undegraded":"2024-04-02T13:33:55.554871+0000","last_fullsized":"2024-04-02T13:33:55.554871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557286+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:33:55.557286+0000","last_peered":"2024-04-02T13:33:55.557286+0000","last_clean":"2024-04-02T13:33:55.557286+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:33:55.557286+0000","last_undegraded":"2024-04-02T13:33:55.557286+0000","last_fullsized":"2024-04-02T13:33:55.557286+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.487065+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:34:24.487065+0000","last_peered":"2024-04-02T13:34:24.487065+0000","last_clean":"2024-04-02T13:34:24.487065+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:34:24.487065+0000","last_undegraded":"2024-04-02T13:34:24.487065+0000","last_fullsized":"2024-04-02T13:34:24.487065+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551256+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:33:55.551256+0000","last_peered":"2024-04-02T13:33:55.551256+0000","last_clean":"2024-04-02T13:33:55.551256+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:33:55.551256+0000","last_undegraded":"2024-04-02T13:33:55.551256+0000","last_fullsized":"2024-04-02T13:33:55.551256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560137+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:33:55.560137+0000","last_peered":"2024-04-02T13:33:55.560137+0000","last_clean":"2024-04-02T13:33:55.560137+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:33:55.560137+0000","last_undegraded":"2024-04-02T13:33:55.560137+0000","last_fullsized":"2024-04-02T13:33:55.560137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558983+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:33:55.558983+0000","last_peered":"2024-04-02T13:33:55.558983+0000","last_clean":"2024-04-02T13:33:55.558983+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:33:55.558983+0000","last_undegraded":"2024-04-02T13:33:55.558983+0000","last_fullsized":"2024-04-02T13:33:55.558983+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'13","reported_seq":112,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:22.252681+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:34:22.252681+0000","last_peered":"2024-04-02T13:34:22.252681+0000","last_clean":"2024-04-02T13:34:22.252681+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:34:22.252681+0000","last_undegraded":"2024-04-02T13:34:22.252681+0000","last_fullsized":"2024-04-02T13:34:22.252681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657619+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:33:55.657619+0000","last_peered":"2024-04-02T13:33:55.657619+0000","last_clean":"2024-04-02T13:33:55.657619+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:33:55.657619+0000","last_undegraded":"2024-04-02T13:33:55.657619+0000","last_fullsized":"2024-04-02T13:33:55.657619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551401+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:33:55.551401+0000","last_peered":"2024-04-02T13:33:55.551401+0000","last_clean":"2024-04-02T13:33:55.551401+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:33:55.551401+0000","last_undegraded":"2024-04-02T13:33:55.551401+0000","last_fullsized":"2024-04-02T13:33:55.551401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657763+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:33:55.657763+0000","last_peered":"2024-04-02T13:33:55.657763+0000","last_clean":"2024-04-02T13:33:55.657763+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:33:55.657763+0000","last_undegraded":"2024-04-02T13:33:55.657763+0000","last_fullsized":"2024-04-02T13:33:55.657763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555002+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:33:55.555002+0000","last_peered":"2024-04-02T13:33:55.555002+0000","last_clean":"2024-04-02T13:33:55.555002+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:33:55.555002+0000","last_undegraded":"2024-04-02T13:33:55.555002+0000","last_fullsized":"2024-04-02T13:33:55.555002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551528+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:33:55.551528+0000","last_peered":"2024-04-02T13:33:55.551528+0000","last_clean":"2024-04-02T13:33:55.551528+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:33:55.551528+0000","last_undegraded":"2024-04-02T13:33:55.551528+0000","last_fullsized":"2024-04-02T13:33:55.551528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.552031+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:33:55.552031+0000","last_peered":"2024-04-02T13:33:55.552031+0000","last_clean":"2024-04-02T13:33:55.552031+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:33:55.552031+0000","last_undegraded":"2024-04-02T13:33:55.552031+0000","last_fullsized":"2024-04-02T13:33:55.552031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557467+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:33:55.557467+0000","last_peered":"2024-04-02T13:33:55.557467+0000","last_clean":"2024-04-02T13:33:55.557467+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:33:55.557467+0000","last_undegraded":"2024-04-02T13:33:55.557467+0000","last_fullsized":"2024-04-02T13:33:55.557467+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551483+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:33:55.551483+0000","last_peered":"2024-04-02T13:33:55.551483+0000","last_clean":"2024-04-02T13:33:55.551483+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:33:55.551483+0000","last_undegraded":"2024-04-02T13:33:55.551483+0000","last_fullsized":"2024-04-02T13:33:55.551483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551427+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:33:55.551427+0000","last_peered":"2024-04-02T13:33:55.551427+0000","last_clean":"2024-04-02T13:33:55.551427+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:33:55.551427+0000","last_undegraded":"2024-04-02T13:33:55.551427+0000","last_fullsized":"2024-04-02T13:33:55.551427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559746+0000","last_change":"2024-04-02T13:33:55.559231+0000","last_active":"2024-04-02T13:33:55.559746+0000","last_peered":"2024-04-02T13:33:55.559746+0000","last_clean":"2024-04-02T13:33:55.559746+0000","last_became_active":"2024-04-02T13:33:55.558856+0000","last_became_peered":"2024-04-02T13:33:55.558856+0000","last_unstale":"2024-04-02T13:33:55.559746+0000","last_undegraded":"2024-04-02T13:33:55.559746+0000","last_fullsized":"2024-04-02T13:33:55.559746+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:12:47.406198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559868+0000","last_change":"2024-04-02T13:32:54.560990+0000","last_active":"2024-04-02T13:33:55.559868+0000","last_peered":"2024-04-02T13:33:55.559868+0000","last_clean":"2024-04-02T13:33:55.559868+0000","last_became_active":"2024-04-02T13:32:54.560528+0000","last_became_peered":"2024-04-02T13:32:54.560528+0000","last_unstale":"2024-04-02T13:33:55.559868+0000","last_undegraded":"2024-04-02T13:33:55.559868+0000","last_fullsized":"2024-04-02T13:33:55.559868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:41:41.675349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'5","reported_seq":105,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:21.253018+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:34:21.253018+0000","last_peered":"2024-04-02T13:34:21.253018+0000","last_clean":"2024-04-02T13:34:21.253018+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:34:21.253018+0000","last_undegraded":"2024-04-02T13:34:21.253018+0000","last_fullsized":"2024-04-02T13:34:21.253018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.468419+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:34:24.468419+0000","last_peered":"2024-04-02T13:34:24.468419+0000","last_clean":"2024-04-02T13:34:24.468419+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:34:24.468419+0000","last_undegraded":"2024-04-02T13:34:24.468419+0000","last_fullsized":"2024-04-02T13:34:24.468419+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.559110+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:33:55.559110+0000","last_peered":"2024-04-02T13:33:55.559110+0000","last_clean":"2024-04-02T13:33:55.559110+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:33:55.559110+0000","last_undegraded":"2024-04-02T13:33:55.559110+0000","last_fullsized":"2024-04-02T13:33:55.559110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":47,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.565630+0000","last_change":"2024-04-02T13:33:55.565630+0000","last_active":"2024-04-02T13:33:55.565630+0000","last_peered":"2024-04-02T13:33:55.565630+0000","last_clean":"2024-04-02T13:33:55.565630+0000","last_became_active":"2024-04-02T13:33:55.559239+0000","last_became_peered":"2024-04-02T13:33:55.559239+0000","last_unstale":"2024-04-02T13:33:55.565630+0000","last_undegraded":"2024-04-02T13:33:55.565630+0000","last_fullsized":"2024-04-02T13:33:55.565630+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:46.161608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555606+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:33:55.555606+0000","last_peered":"2024-04-02T13:33:55.555606+0000","last_clean":"2024-04-02T13:33:55.555606+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:33:55.555606+0000","last_undegraded":"2024-04-02T13:33:55.555606+0000","last_fullsized":"2024-04-02T13:33:55.555606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558751+0000","last_change":"2024-04-02T13:33:55.558751+0000","last_active":"2024-04-02T13:33:55.558751+0000","last_peered":"2024-04-02T13:33:55.558751+0000","last_clean":"2024-04-02T13:33:55.558751+0000","last_became_active":"2024-04-02T13:33:55.558278+0000","last_became_peered":"2024-04-02T13:33:55.558278+0000","last_unstale":"2024-04-02T13:33:55.558751+0000","last_undegraded":"2024-04-02T13:33:55.558751+0000","last_fullsized":"2024-04-02T13:33:55.558751+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:15.164871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556460+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:33:55.556460+0000","last_peered":"2024-04-02T13:33:55.556460+0000","last_clean":"2024-04-02T13:33:55.556460+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:33:55.556460+0000","last_undegraded":"2024-04-02T13:33:55.556460+0000","last_fullsized":"2024-04-02T13:33:55.556460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657234+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:33:55.657234+0000","last_peered":"2024-04-02T13:33:55.657234+0000","last_clean":"2024-04-02T13:33:55.657234+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:33:55.657234+0000","last_undegraded":"2024-04-02T13:33:55.657234+0000","last_fullsized":"2024-04-02T13:33:55.657234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555029+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:33:55.555029+0000","last_peered":"2024-04-02T13:33:55.555029+0000","last_clean":"2024-04-02T13:33:55.555029+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:33:55.555029+0000","last_undegraded":"2024-04-02T13:33:55.555029+0000","last_fullsized":"2024-04-02T13:33:55.555029+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554795+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:33:55.554795+0000","last_peered":"2024-04-02T13:33:55.554795+0000","last_clean":"2024-04-02T13:33:55.554795+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:33:55.554795+0000","last_undegraded":"2024-04-02T13:33:55.554795+0000","last_fullsized":"2024-04-02T13:33:55.554795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550716+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:33:55.550716+0000","last_peered":"2024-04-02T13:33:55.550716+0000","last_clean":"2024-04-02T13:33:55.550716+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:33:55.550716+0000","last_undegraded":"2024-04-02T13:33:55.550716+0000","last_fullsized":"2024-04-02T13:33:55.550716+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657222+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:33:55.657222+0000","last_peered":"2024-04-02T13:33:55.657222+0000","last_clean":"2024-04-02T13:33:55.657222+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:33:55.657222+0000","last_undegraded":"2024-04-02T13:33:55.657222+0000","last_fullsized":"2024-04-02T13:33:55.657222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'4","reported_seq":178,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.256196+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:34:24.256196+0000","last_peered":"2024-04-02T13:34:24.256196+0000","last_clean":"2024-04-02T13:34:24.256196+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:34:24.256196+0000","last_undegraded":"2024-04-02T13:34:24.256196+0000","last_fullsized":"2024-04-02T13:34:24.256196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":148,"num_read_kb":148,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557497+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:33:55.557497+0000","last_peered":"2024-04-02T13:33:55.557497+0000","last_clean":"2024-04-02T13:33:55.557497+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:33:55.557497+0000","last_undegraded":"2024-04-02T13:33:55.557497+0000","last_fullsized":"2024-04-02T13:33:55.557497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550685+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:33:55.550685+0000","last_peered":"2024-04-02T13:33:55.550685+0000","last_clean":"2024-04-02T13:33:55.550685+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:33:55.550685+0000","last_undegraded":"2024-04-02T13:33:55.550685+0000","last_fullsized":"2024-04-02T13:33:55.550685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555813+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:33:55.555813+0000","last_peered":"2024-04-02T13:33:55.555813+0000","last_clean":"2024-04-02T13:33:55.555813+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:33:55.555813+0000","last_undegraded":"2024-04-02T13:33:55.555813+0000","last_fullsized":"2024-04-02T13:33:55.555813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555235+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:33:55.555235+0000","last_peered":"2024-04-02T13:33:55.555235+0000","last_clean":"2024-04-02T13:33:55.555235+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:33:55.555235+0000","last_undegraded":"2024-04-02T13:33:55.555235+0000","last_fullsized":"2024-04-02T13:33:55.555235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551825+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:33:55.551825+0000","last_peered":"2024-04-02T13:33:55.551825+0000","last_clean":"2024-04-02T13:33:55.551825+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:33:55.551825+0000","last_undegraded":"2024-04-02T13:33:55.551825+0000","last_fullsized":"2024-04-02T13:33:55.551825+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556321+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:33:55.556321+0000","last_peered":"2024-04-02T13:33:55.556321+0000","last_clean":"2024-04-02T13:33:55.556321+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:33:55.556321+0000","last_undegraded":"2024-04-02T13:33:55.556321+0000","last_fullsized":"2024-04-02T13:33:55.556321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557659+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:33:55.557659+0000","last_peered":"2024-04-02T13:33:55.557659+0000","last_clean":"2024-04-02T13:33:55.557659+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:33:55.557659+0000","last_undegraded":"2024-04-02T13:33:55.557659+0000","last_fullsized":"2024-04-02T13:33:55.557659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.550714+0000","last_change":"2024-04-02T13:32:54.562030+0000","last_active":"2024-04-02T13:33:55.550714+0000","last_peered":"2024-04-02T13:33:55.550714+0000","last_clean":"2024-04-02T13:33:55.550714+0000","last_became_active":"2024-04-02T13:32:54.560872+0000","last_became_peered":"2024-04-02T13:32:54.560872+0000","last_unstale":"2024-04-02T13:33:55.550714+0000","last_undegraded":"2024-04-02T13:33:55.550714+0000","last_fullsized":"2024-04-02T13:33:55.550714+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:25:08.994729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'38","reported_seq":219,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.435683+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:34:24.435683+0000","last_peered":"2024-04-02T13:34:24.435683+0000","last_clean":"2024-04-02T13:34:24.435683+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:34:24.435683+0000","last_undegraded":"2024-04-02T13:34:24.435683+0000","last_fullsized":"2024-04-02T13:34:24.435683+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62506,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":155,"num_read_kb":138,"num_write":40,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.465009+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:34:24.465009+0000","last_peered":"2024-04-02T13:34:24.465009+0000","last_clean":"2024-04-02T13:34:24.465009+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:34:24.465009+0000","last_undegraded":"2024-04-02T13:34:24.465009+0000","last_fullsized":"2024-04-02T13:34:24.465009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556196+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:33:55.556196+0000","last_peered":"2024-04-02T13:33:55.556196+0000","last_clean":"2024-04-02T13:33:55.556196+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:33:55.556196+0000","last_undegraded":"2024-04-02T13:33:55.556196+0000","last_fullsized":"2024-04-02T13:33:55.556196+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558045+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:33:55.558045+0000","last_peered":"2024-04-02T13:33:55.558045+0000","last_clean":"2024-04-02T13:33:55.558045+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:33:55.558045+0000","last_undegraded":"2024-04-02T13:33:55.558045+0000","last_fullsized":"2024-04-02T13:33:55.558045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556260+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:33:55.556260+0000","last_peered":"2024-04-02T13:33:55.556260+0000","last_clean":"2024-04-02T13:33:55.556260+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:33:55.556260+0000","last_undegraded":"2024-04-02T13:33:55.556260+0000","last_fullsized":"2024-04-02T13:33:55.556260+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553268+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:33:55.553268+0000","last_peered":"2024-04-02T13:33:55.553268+0000","last_clean":"2024-04-02T13:33:55.553268+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:33:55.553268+0000","last_undegraded":"2024-04-02T13:33:55.553268+0000","last_fullsized":"2024-04-02T13:33:55.553268+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555434+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:33:55.555434+0000","last_peered":"2024-04-02T13:33:55.555434+0000","last_clean":"2024-04-02T13:33:55.555434+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:33:55.555434+0000","last_undegraded":"2024-04-02T13:33:55.555434+0000","last_fullsized":"2024-04-02T13:33:55.555434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556079+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:33:55.556079+0000","last_peered":"2024-04-02T13:33:55.556079+0000","last_clean":"2024-04-02T13:33:55.556079+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:33:55.556079+0000","last_undegraded":"2024-04-02T13:33:55.556079+0000","last_fullsized":"2024-04-02T13:33:55.556079+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551106+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:33:55.551106+0000","last_peered":"2024-04-02T13:33:55.551106+0000","last_clean":"2024-04-02T13:33:55.551106+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:33:55.551106+0000","last_undegraded":"2024-04-02T13:33:55.551106+0000","last_fullsized":"2024-04-02T13:33:55.551106+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'5","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:22.253323+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:34:22.253323+0000","last_peered":"2024-04-02T13:34:22.253323+0000","last_clean":"2024-04-02T13:34:22.253323+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:34:22.253323+0000","last_undegraded":"2024-04-02T13:34:22.253323+0000","last_fullsized":"2024-04-02T13:34:22.253323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555138+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:33:55.555138+0000","last_peered":"2024-04-02T13:33:55.555138+0000","last_clean":"2024-04-02T13:33:55.555138+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:33:55.555138+0000","last_undegraded":"2024-04-02T13:33:55.555138+0000","last_fullsized":"2024-04-02T13:33:55.555138+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556186+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:33:55.556186+0000","last_peered":"2024-04-02T13:33:55.556186+0000","last_clean":"2024-04-02T13:33:55.556186+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:33:55.556186+0000","last_undegraded":"2024-04-02T13:33:55.556186+0000","last_fullsized":"2024-04-02T13:33:55.556186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.574777+0000","last_change":"2024-04-02T13:33:55.574777+0000","last_active":"2024-04-02T13:33:55.574777+0000","last_peered":"2024-04-02T13:33:55.574777+0000","last_clean":"2024-04-02T13:33:55.574777+0000","last_became_active":"2024-04-02T13:33:55.561040+0000","last_became_peered":"2024-04-02T13:33:55.561040+0000","last_unstale":"2024-04-02T13:33:55.574777+0000","last_undegraded":"2024-04-02T13:33:55.574777+0000","last_fullsized":"2024-04-02T13:33:55.574777+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:23.184540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560045+0000","last_change":"2024-04-02T13:33:55.560045+0000","last_active":"2024-04-02T13:33:55.560045+0000","last_peered":"2024-04-02T13:33:55.560045+0000","last_clean":"2024-04-02T13:33:55.560045+0000","last_became_active":"2024-04-02T13:33:55.559659+0000","last_became_peered":"2024-04-02T13:33:55.559659+0000","last_unstale":"2024-04-02T13:33:55.560045+0000","last_undegraded":"2024-04-02T13:33:55.560045+0000","last_fullsized":"2024-04-02T13:33:55.560045+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:10:42.028143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556260+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:33:55.556260+0000","last_peered":"2024-04-02T13:33:55.556260+0000","last_clean":"2024-04-02T13:33:55.556260+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:33:55.556260+0000","last_undegraded":"2024-04-02T13:33:55.556260+0000","last_fullsized":"2024-04-02T13:33:55.556260+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551719+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:33:55.551719+0000","last_peered":"2024-04-02T13:33:55.551719+0000","last_clean":"2024-04-02T13:33:55.551719+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:33:55.551719+0000","last_undegraded":"2024-04-02T13:33:55.551719+0000","last_fullsized":"2024-04-02T13:33:55.551719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.483304+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:34:24.483304+0000","last_peered":"2024-04-02T13:34:24.483304+0000","last_clean":"2024-04-02T13:34:24.483304+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:34:24.483304+0000","last_undegraded":"2024-04-02T13:34:24.483304+0000","last_fullsized":"2024-04-02T13:34:24.483304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551384+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:33:55.551384+0000","last_peered":"2024-04-02T13:33:55.551384+0000","last_clean":"2024-04-02T13:33:55.551384+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:33:55.551384+0000","last_undegraded":"2024-04-02T13:33:55.551384+0000","last_fullsized":"2024-04-02T13:33:55.551384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551007+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:33:55.551007+0000","last_peered":"2024-04-02T13:33:55.551007+0000","last_clean":"2024-04-02T13:33:55.551007+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:33:55.551007+0000","last_undegraded":"2024-04-02T13:33:55.551007+0000","last_fullsized":"2024-04-02T13:33:55.551007+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555062+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:33:55.555062+0000","last_peered":"2024-04-02T13:33:55.555062+0000","last_clean":"2024-04-02T13:33:55.555062+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:33:55.555062+0000","last_undegraded":"2024-04-02T13:33:55.555062+0000","last_fullsized":"2024-04-02T13:33:55.555062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556228+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:33:55.556228+0000","last_peered":"2024-04-02T13:33:55.556228+0000","last_clean":"2024-04-02T13:33:55.556228+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:33:55.556228+0000","last_undegraded":"2024-04-02T13:33:55.556228+0000","last_fullsized":"2024-04-02T13:33:55.556228+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'5","reported_seq":102,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:20.250652+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:34:20.250652+0000","last_peered":"2024-04-02T13:34:20.250652+0000","last_clean":"2024-04-02T13:34:20.250652+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:34:20.250652+0000","last_undegraded":"2024-04-02T13:34:20.250652+0000","last_fullsized":"2024-04-02T13:34:20.250652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554299+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:33:55.554299+0000","last_peered":"2024-04-02T13:33:55.554299+0000","last_clean":"2024-04-02T13:33:55.554299+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:33:55.554299+0000","last_undegraded":"2024-04-02T13:33:55.554299+0000","last_fullsized":"2024-04-02T13:33:55.554299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555936+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:33:55.555936+0000","last_peered":"2024-04-02T13:33:55.555936+0000","last_clean":"2024-04-02T13:33:55.555936+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:33:55.555936+0000","last_undegraded":"2024-04-02T13:33:55.555936+0000","last_fullsized":"2024-04-02T13:33:55.555936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555216+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:33:55.555216+0000","last_peered":"2024-04-02T13:33:55.555216+0000","last_clean":"2024-04-02T13:33:55.555216+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:33:55.555216+0000","last_undegraded":"2024-04-02T13:33:55.555216+0000","last_fullsized":"2024-04-02T13:33:55.555216+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'5","reported_seq":103,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:20.250209+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:34:20.250209+0000","last_peered":"2024-04-02T13:34:20.250209+0000","last_clean":"2024-04-02T13:34:20.250209+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:34:20.250209+0000","last_undegraded":"2024-04-02T13:34:20.250209+0000","last_fullsized":"2024-04-02T13:34:20.250209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556151+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:33:55.556151+0000","last_peered":"2024-04-02T13:33:55.556151+0000","last_clean":"2024-04-02T13:33:55.556151+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:33:55.556151+0000","last_undegraded":"2024-04-02T13:33:55.556151+0000","last_fullsized":"2024-04-02T13:33:55.556151+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657274+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:33:55.657274+0000","last_peered":"2024-04-02T13:33:55.657274+0000","last_clean":"2024-04-02T13:33:55.657274+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:33:55.657274+0000","last_undegraded":"2024-04-02T13:33:55.657274+0000","last_fullsized":"2024-04-02T13:33:55.657274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551675+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:33:55.551675+0000","last_peered":"2024-04-02T13:33:55.551675+0000","last_clean":"2024-04-02T13:33:55.551675+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:33:55.551675+0000","last_undegraded":"2024-04-02T13:33:55.551675+0000","last_fullsized":"2024-04-02T13:33:55.551675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554971+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:33:55.554971+0000","last_peered":"2024-04-02T13:33:55.554971+0000","last_clean":"2024-04-02T13:33:55.554971+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:33:55.554971+0000","last_undegraded":"2024-04-02T13:33:55.554971+0000","last_fullsized":"2024-04-02T13:33:55.554971+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555638+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:33:55.555638+0000","last_peered":"2024-04-02T13:33:55.555638+0000","last_clean":"2024-04-02T13:33:55.555638+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:33:55.555638+0000","last_undegraded":"2024-04-02T13:33:55.555638+0000","last_fullsized":"2024-04-02T13:33:55.555638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556076+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:33:55.556076+0000","last_peered":"2024-04-02T13:33:55.556076+0000","last_clean":"2024-04-02T13:33:55.556076+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:33:55.556076+0000","last_undegraded":"2024-04-02T13:33:55.556076+0000","last_fullsized":"2024-04-02T13:33:55.556076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555030+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:33:55.555030+0000","last_peered":"2024-04-02T13:33:55.555030+0000","last_clean":"2024-04-02T13:33:55.555030+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:33:55.555030+0000","last_undegraded":"2024-04-02T13:33:55.555030+0000","last_fullsized":"2024-04-02T13:33:55.555030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.553258+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:33:55.553258+0000","last_peered":"2024-04-02T13:33:55.553258+0000","last_clean":"2024-04-02T13:33:55.553258+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:33:55.553258+0000","last_undegraded":"2024-04-02T13:33:55.553258+0000","last_fullsized":"2024-04-02T13:33:55.553258+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551795+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:33:55.551795+0000","last_peered":"2024-04-02T13:33:55.551795+0000","last_clean":"2024-04-02T13:33:55.551795+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:33:55.551795+0000","last_undegraded":"2024-04-02T13:33:55.551795+0000","last_fullsized":"2024-04-02T13:33:55.551795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556381+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:33:55.556381+0000","last_peered":"2024-04-02T13:33:55.556381+0000","last_clean":"2024-04-02T13:33:55.556381+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:33:55.556381+0000","last_undegraded":"2024-04-02T13:33:55.556381+0000","last_fullsized":"2024-04-02T13:33:55.556381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657124+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:33:55.657124+0000","last_peered":"2024-04-02T13:33:55.657124+0000","last_clean":"2024-04-02T13:33:55.657124+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:33:55.657124+0000","last_undegraded":"2024-04-02T13:33:55.657124+0000","last_fullsized":"2024-04-02T13:33:55.657124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551366+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:33:55.551366+0000","last_peered":"2024-04-02T13:33:55.551366+0000","last_clean":"2024-04-02T13:33:55.551366+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:33:55.551366+0000","last_undegraded":"2024-04-02T13:33:55.551366+0000","last_fullsized":"2024-04-02T13:33:55.551366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657417+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:33:55.657417+0000","last_peered":"2024-04-02T13:33:55.657417+0000","last_clean":"2024-04-02T13:33:55.657417+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:33:55.657417+0000","last_undegraded":"2024-04-02T13:33:55.657417+0000","last_fullsized":"2024-04-02T13:33:55.657417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558670+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:33:55.558670+0000","last_peered":"2024-04-02T13:33:55.558670+0000","last_clean":"2024-04-02T13:33:55.558670+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:33:55.558670+0000","last_undegraded":"2024-04-02T13:33:55.558670+0000","last_fullsized":"2024-04-02T13:33:55.558670+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.552138+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:33:55.552138+0000","last_peered":"2024-04-02T13:33:55.552138+0000","last_clean":"2024-04-02T13:33:55.552138+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:33:55.552138+0000","last_undegraded":"2024-04-02T13:33:55.552138+0000","last_fullsized":"2024-04-02T13:33:55.552138+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555149+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:33:55.555149+0000","last_peered":"2024-04-02T13:33:55.555149+0000","last_clean":"2024-04-02T13:33:55.555149+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:33:55.555149+0000","last_undegraded":"2024-04-02T13:33:55.555149+0000","last_fullsized":"2024-04-02T13:33:55.555149+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551989+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:33:55.551989+0000","last_peered":"2024-04-02T13:33:55.551989+0000","last_clean":"2024-04-02T13:33:55.551989+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:33:55.551989+0000","last_undegraded":"2024-04-02T13:33:55.551989+0000","last_fullsized":"2024-04-02T13:33:55.551989+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.564335+0000","last_change":"2024-04-02T13:33:55.564335+0000","last_active":"2024-04-02T13:33:55.564335+0000","last_peered":"2024-04-02T13:33:55.564335+0000","last_clean":"2024-04-02T13:33:55.564335+0000","last_became_active":"2024-04-02T13:33:55.560370+0000","last_became_peered":"2024-04-02T13:33:55.560370+0000","last_unstale":"2024-04-02T13:33:55.564335+0000","last_undegraded":"2024-04-02T13:33:55.564335+0000","last_fullsized":"2024-04-02T13:33:55.564335+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:14.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.438412+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:34:24.438412+0000","last_peered":"2024-04-02T13:34:24.438412+0000","last_clean":"2024-04-02T13:34:24.438412+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:34:24.438412+0000","last_undegraded":"2024-04-02T13:34:24.438412+0000","last_fullsized":"2024-04-02T13:34:24.438412+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551206+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:33:55.551206+0000","last_peered":"2024-04-02T13:33:55.551206+0000","last_clean":"2024-04-02T13:33:55.551206+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:33:55.551206+0000","last_undegraded":"2024-04-02T13:33:55.551206+0000","last_fullsized":"2024-04-02T13:33:55.551206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657253+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:33:55.657253+0000","last_peered":"2024-04-02T13:33:55.657253+0000","last_clean":"2024-04-02T13:33:55.657253+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:33:55.657253+0000","last_undegraded":"2024-04-02T13:33:55.657253+0000","last_fullsized":"2024-04-02T13:33:55.657253+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.478146+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:34:24.478146+0000","last_peered":"2024-04-02T13:34:24.478146+0000","last_clean":"2024-04-02T13:34:24.478146+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:34:24.478146+0000","last_undegraded":"2024-04-02T13:34:24.478146+0000","last_fullsized":"2024-04-02T13:34:24.478146+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551496+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:33:55.551496+0000","last_peered":"2024-04-02T13:33:55.551496+0000","last_clean":"2024-04-02T13:33:55.551496+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:33:55.551496+0000","last_undegraded":"2024-04-02T13:33:55.551496+0000","last_fullsized":"2024-04-02T13:33:55.551496+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557873+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:33:55.557873+0000","last_peered":"2024-04-02T13:33:55.557873+0000","last_clean":"2024-04-02T13:33:55.557873+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:33:55.557873+0000","last_undegraded":"2024-04-02T13:33:55.557873+0000","last_fullsized":"2024-04-02T13:33:55.557873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657361+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:33:55.657361+0000","last_peered":"2024-04-02T13:33:55.657361+0000","last_clean":"2024-04-02T13:33:55.657361+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:33:55.657361+0000","last_undegraded":"2024-04-02T13:33:55.657361+0000","last_fullsized":"2024-04-02T13:33:55.657361+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.551777+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:33:55.551777+0000","last_peered":"2024-04-02T13:33:55.551777+0000","last_clean":"2024-04-02T13:33:55.551777+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:33:55.551777+0000","last_undegraded":"2024-04-02T13:33:55.551777+0000","last_fullsized":"2024-04-02T13:33:55.551777+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.565608+0000","last_change":"2024-04-02T13:33:55.565608+0000","last_active":"2024-04-02T13:33:55.565608+0000","last_peered":"2024-04-02T13:33:55.565608+0000","last_clean":"2024-04-02T13:33:55.565608+0000","last_became_active":"2024-04-02T13:33:55.557475+0000","last_became_peered":"2024-04-02T13:33:55.557475+0000","last_unstale":"2024-04-02T13:33:55.565608+0000","last_undegraded":"2024-04-02T13:33:55.565608+0000","last_fullsized":"2024-04-02T13:33:55.565608+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:17:21.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555739+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:33:55.555739+0000","last_peered":"2024-04-02T13:33:55.555739+0000","last_clean":"2024-04-02T13:33:55.555739+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:33:55.555739+0000","last_undegraded":"2024-04-02T13:33:55.555739+0000","last_fullsized":"2024-04-02T13:33:55.555739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558108+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:33:55.558108+0000","last_peered":"2024-04-02T13:33:55.558108+0000","last_clean":"2024-04-02T13:33:55.558108+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:33:55.558108+0000","last_undegraded":"2024-04-02T13:33:55.558108+0000","last_fullsized":"2024-04-02T13:33:55.558108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'5","reported_seq":107,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:22.254493+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:34:22.254493+0000","last_peered":"2024-04-02T13:34:22.254493+0000","last_clean":"2024-04-02T13:34:22.254493+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:34:22.254493+0000","last_undegraded":"2024-04-02T13:34:22.254493+0000","last_fullsized":"2024-04-02T13:34:22.254493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556371+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:33:55.556371+0000","last_peered":"2024-04-02T13:33:55.556371+0000","last_clean":"2024-04-02T13:33:55.556371+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:33:55.556371+0000","last_undegraded":"2024-04-02T13:33:55.556371+0000","last_fullsized":"2024-04-02T13:33:55.556371+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554680+0000","last_change":"2024-04-02T13:32:54.566345+0000","last_active":"2024-04-02T13:33:55.554680+0000","last_peered":"2024-04-02T13:33:55.554680+0000","last_clean":"2024-04-02T13:33:55.554680+0000","last_became_active":"2024-04-02T13:32:54.564927+0000","last_became_peered":"2024-04-02T13:32:54.564927+0000","last_unstale":"2024-04-02T13:33:55.554680+0000","last_undegraded":"2024-04-02T13:33:55.554680+0000","last_fullsized":"2024-04-02T13:33:55.554680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:34.439915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555903+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:33:55.555903+0000","last_peered":"2024-04-02T13:33:55.555903+0000","last_clean":"2024-04-02T13:33:55.555903+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:33:55.555903+0000","last_undegraded":"2024-04-02T13:33:55.555903+0000","last_fullsized":"2024-04-02T13:33:55.555903+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.555933+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:33:55.555933+0000","last_peered":"2024-04-02T13:33:55.555933+0000","last_clean":"2024-04-02T13:33:55.555933+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:33:55.555933+0000","last_undegraded":"2024-04-02T13:33:55.555933+0000","last_fullsized":"2024-04-02T13:33:55.555933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554572+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:33:55.554572+0000","last_peered":"2024-04-02T13:33:55.554572+0000","last_clean":"2024-04-02T13:33:55.554572+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:33:55.554572+0000","last_undegraded":"2024-04-02T13:33:55.554572+0000","last_fullsized":"2024-04-02T13:33:55.554572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556309+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:33:55.556309+0000","last_peered":"2024-04-02T13:33:55.556309+0000","last_clean":"2024-04-02T13:33:55.556309+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:33:55.556309+0000","last_undegraded":"2024-04-02T13:33:55.556309+0000","last_fullsized":"2024-04-02T13:33:55.556309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556190+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:33:55.556190+0000","last_peered":"2024-04-02T13:33:55.556190+0000","last_clean":"2024-04-02T13:33:55.556190+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:33:55.556190+0000","last_undegraded":"2024-04-02T13:33:55.556190+0000","last_fullsized":"2024-04-02T13:33:55.556190+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.479027+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:34:24.479027+0000","last_peered":"2024-04-02T13:34:24.479027+0000","last_clean":"2024-04-02T13:34:24.479027+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:34:24.479027+0000","last_undegraded":"2024-04-02T13:34:24.479027+0000","last_fullsized":"2024-04-02T13:34:24.479027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.562541+0000","last_change":"2024-04-02T13:33:55.562541+0000","last_active":"2024-04-02T13:33:55.562541+0000","last_peered":"2024-04-02T13:33:55.562541+0000","last_clean":"2024-04-02T13:33:55.562541+0000","last_became_active":"2024-04-02T13:33:55.562167+0000","last_became_peered":"2024-04-02T13:33:55.562167+0000","last_unstale":"2024-04-02T13:33:55.562541+0000","last_undegraded":"2024-04-02T13:33:55.562541+0000","last_fullsized":"2024-04-02T13:33:55.562541+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:29.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.557710+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:33:55.557710+0000","last_peered":"2024-04-02T13:33:55.557710+0000","last_clean":"2024-04-02T13:33:55.557710+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:33:55.557710+0000","last_undegraded":"2024-04-02T13:33:55.557710+0000","last_fullsized":"2024-04-02T13:33:55.557710+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554863+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:33:55.554863+0000","last_peered":"2024-04-02T13:33:55.554863+0000","last_clean":"2024-04-02T13:33:55.554863+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:33:55.554863+0000","last_undegraded":"2024-04-02T13:33:55.554863+0000","last_fullsized":"2024-04-02T13:33:55.554863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556327+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:33:55.556327+0000","last_peered":"2024-04-02T13:33:55.556327+0000","last_clean":"2024-04-02T13:33:55.556327+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:33:55.556327+0000","last_undegraded":"2024-04-02T13:33:55.556327+0000","last_fullsized":"2024-04-02T13:33:55.556327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.468607+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:34:24.468607+0000","last_peered":"2024-04-02T13:34:24.468607+0000","last_clean":"2024-04-02T13:34:24.468607+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:34:24.468607+0000","last_undegraded":"2024-04-02T13:34:24.468607+0000","last_fullsized":"2024-04-02T13:34:24.468607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.560478+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:33:55.560478+0000","last_peered":"2024-04-02T13:33:55.560478+0000","last_clean":"2024-04-02T13:33:55.560478+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:33:55.560478+0000","last_undegraded":"2024-04-02T13:33:55.560478+0000","last_fullsized":"2024-04-02T13:33:55.560478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657576+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:33:55.657576+0000","last_peered":"2024-04-02T13:33:55.657576+0000","last_clean":"2024-04-02T13:33:55.657576+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:33:55.657576+0000","last_undegraded":"2024-04-02T13:33:55.657576+0000","last_fullsized":"2024-04-02T13:33:55.657576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.554911+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:33:55.554911+0000","last_peered":"2024-04-02T13:33:55.554911+0000","last_clean":"2024-04-02T13:33:55.554911+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:33:55.554911+0000","last_undegraded":"2024-04-02T13:33:55.554911+0000","last_fullsized":"2024-04-02T13:33:55.554911+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.657399+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:33:55.657399+0000","last_peered":"2024-04-02T13:33:55.657399+0000","last_clean":"2024-04-02T13:33:55.657399+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:33:55.657399+0000","last_undegraded":"2024-04-02T13:33:55.657399+0000","last_fullsized":"2024-04-02T13:33:55.657399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.558369+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:33:55.558369+0000","last_peered":"2024-04-02T13:33:55.558369+0000","last_clean":"2024-04-02T13:33:55.558369+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:33:55.558369+0000","last_undegraded":"2024-04-02T13:33:55.558369+0000","last_fullsized":"2024-04-02T13:33:55.558369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:33:55.556249+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:33:55.556249+0000","last_peered":"2024-04-02T13:33:55.556249+0000","last_clean":"2024-04-02T13:33:55.556249+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:33:55.556249+0000","last_undegraded":"2024-04-02T13:33:55.556249+0000","last_fullsized":"2024-04-02T13:33:55.556249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-02T13:34:24.483409+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:34:24.483409+0000","last_peered":"2024-04-02T13:34:24.483409+0000","last_clean":"2024-04-02T13:34:24.483409+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:34:24.483409+0000","last_undegraded":"2024-04-02T13:34:24.483409+0000","last_fullsized":"2024-04-02T13:34:24.483409+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68928,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":763,"num_read_kb":458,"num_write":243,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":206438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":506039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612645,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37044,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708108,"statfs":{"total":95995035648,"available":95957102592,"internally_reserved":0,"allocated":1740800,"data_stored":1155076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776171,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":509953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":506123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1662976,"data_stored":1102341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":501361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1114112,"data_stored":564724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":502817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411405,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1134592,"data_stored":564944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36376,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708776,"statfs":{"total":95995035648,"available":95957786624,"internally_reserved":0,"allocated":1056768,"data_stored":501069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":505435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36976,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708176,"statfs":{"total":95995035648,"available":95957172224,"internally_reserved":0,"allocated":1671168,"data_stored":1097923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:34:27.733 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-02T13:34:27.734 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-02T13:34:27.734 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-02T13:34:27.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph health --format=json 2024-04-02T13:34:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:27 smithi122 ceph-mon[32114]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:34:27.981 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:27 smithi187 ceph-mon[36064]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:34:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:27 smithi188 ceph-mon[36117]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:34:28.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:28 smithi122 ceph-mon[32114]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:28.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:28 smithi187 ceph-mon[36064]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:29.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:28 smithi188 ceph-mon[36117]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 -- 172.21.15.122:0/3197676316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018101780 msgr2=0x7f0018101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 --2- 172.21.15.122:0/3197676316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018101780 0x7f0018101b60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f00140099c0 tx=0x7f0014009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 -- 172.21.15.122:0/3197676316 shutdown_connections 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 --2- 172.21.15.122:0/3197676316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f001810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 --2- 172.21.15.122:0/3197676316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018101780 0x7f0018101b60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 --2- 172.21.15.122:0/3197676316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018102130 0x7f0018106150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 -- 172.21.15.122:0/3197676316 >> 172.21.15.122:0/3197676316 conn(0x7f0018077510 msgr2=0x7f0018077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 -- 172.21.15.122:0/3197676316 shutdown_connections 2024-04-02T13:34:29.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.689+0000 7f002010b700 1 -- 172.21.15.122:0/3197676316 wait complete. 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 Processor -- start 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 -- start start 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018101780 0x7f0018103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f0018103be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00181117a0 con 0x7f0018102130 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0018111620 con 0x7f0018106880 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f002010b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0018111920 con 0x7f0018101780 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43960/0 (socket says 172.21.15.122:43960) 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 -- 172.21.15.122:0/3039421519 learned_addr learned my addr 172.21.15.122:0/3039421519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001e6a8700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f0018103be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001dea7700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018101780 0x7f0018103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:29.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 -- 172.21.15.122:0/3039421519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018101780 msgr2=0x7f0018103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018101780 0x7f0018103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 -- 172.21.15.122:0/3039421519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 msgr2=0x7f0018103be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f0018103be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001d6a6700 1 -- 172.21.15.122:0/3039421519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00181a7200 con 0x7f0018102130 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.690+0000 7f001e6a8700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f0018103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f001d6a6700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f000800c9b0 tx=0x7f000800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008007b00 con 0x7f0018102130 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0008007ca0 con 0x7f0018102130 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00181a74f0 con 0x7f0018102130 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0008018780 con 0x7f0018102130 2024-04-02T13:34:29.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.691+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00181a7a30 con 0x7f0018102130 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.692+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0018102c10 con 0x7f0018102130 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.693+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f000800f450 con 0x7f0018102130 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.694+0000 7f000effd700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 0x7f00040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.694+0000 7f001dea7700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 0x7f00040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.694+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f00080d07f0 con 0x7f0018102130 2024-04-02T13:34:29.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.694+0000 7f001dea7700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 0x7f00040a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0014009750 tx=0x7f0014005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:29.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.698+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0008014030 con 0x7f0018102130 2024-04-02T13:34:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:29 smithi122 ceph-mon[32114]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T13:34:29.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.939+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f00180692b0 con 0x7f0018102130 2024-04-02T13:34:29.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.940+0000 7f000effd700 1 -- 172.21.15.122:0/3039421519 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f00180692b0 con 0x7f0018102130 2024-04-02T13:34:29.943 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:29.943 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-02T13:34:29.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 msgr2=0x7f00040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:29.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 0x7f00040a6100 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0014009750 tx=0x7f0014005fd0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 msgr2=0x7f00181036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:29.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f000800c9b0 tx=0x7f000800cd70 comp rx=0 tx=0).stop 2024-04-02T13:34:29.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 shutdown_connections 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0018106880 0x7f0018103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0018101780 0x7f0018103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f00040a3c40 0x7f00040a6100 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 --2- 172.21.15.122:0/3039421519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0018102130 0x7f00181036a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 >> 172.21.15.122:0/3039421519 conn(0x7f0018077510 msgr2=0x7f00181053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:29.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 shutdown_connections 2024-04-02T13:34:29.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:29.942+0000 7f002010b700 1 -- 172.21.15.122:0/3039421519 wait complete. 2024-04-02T13:34:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:29 smithi187 ceph-mon[36064]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T13:34:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:29 smithi188 ceph-mon[36117]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-02T13:34:30.668 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-02T13:34:30.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:34:30.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- 172.21.15.122:0/256825144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 msgr2=0x7f0568102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:30.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- 172.21.15.122:0/256825144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f0568102090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0558009900 tx=0x7f055802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- 172.21.15.122:0/256825144 shutdown_connections 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- 172.21.15.122:0/256825144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f0568108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- 172.21.15.122:0/256825144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f0568102090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- 172.21.15.122:0/256825144 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 0x7f0568100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- 172.21.15.122:0/256825144 >> 172.21.15.122:0/256825144 conn(0x7f05680fbf90 msgr2=0x7f05680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- 172.21.15.122:0/256825144 shutdown_connections 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- 172.21.15.122:0/256825144 wait complete. 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 Processor -- start 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 -- start start 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 0x7f05680697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.771+0000 7f056fae4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f056819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f056810c700 con 0x7f0568101c10 2024-04-02T13:34:30.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f056810c580 con 0x7f0568100a20 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f056810c880 con 0x7f05681025d0 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43986/0 (socket says 172.21.15.122:43986) 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 -- 172.21.15.122:0/514581970 learned_addr learned my addr 172.21.15.122:0/514581970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d880640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 0x7f05680697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056e081640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f056819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 -- 172.21.15.122:0/514581970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 msgr2=0x7f056819f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f056819f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 -- 172.21.15.122:0/514581970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 msgr2=0x7f05680697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 0x7f05680697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 -- 172.21.15.122:0/514581970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056819fad0 con 0x7f0568101c10 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056e081640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f056819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056d07f640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558041070 con 0x7f0568101c10 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05580054e0 con 0x7f0568101c10 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056819fd60 con 0x7f0568101c10 2024-04-02T13:34:30.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580346a0 con 0x7f0568101c10 2024-04-02T13:34:30.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.772+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05681a01f0 con 0x7f0568101c10 2024-04-02T13:34:30.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.773+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0568107d40 con 0x7f0568101c10 2024-04-02T13:34:30.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.774+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f0558044070 con 0x7f0568101c10 2024-04-02T13:34:30.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.775+0000 7f055effd640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:30.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.775+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f05580f2860 con 0x7f0568101c10 2024-04-02T13:34:30.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.775+0000 7f056d880640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:30.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.775+0000 7f056d880640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 0x7f05540a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0568101a70 tx=0x7f056400a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:30.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.779+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f05580beee0 con 0x7f0568101c10 2024-04-02T13:34:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:30 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3039421519' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:34:30.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.959+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f056802d370 con 0x7f0568101c10 2024-04-02T13:34:30.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.961+0000 7f055effd640 1 -- 172.21.15.122:0/514581970 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24702 (secure 0 0 0) 0x7f05580be7e0 con 0x7f0568101c10 2024-04-02T13:34:30.963 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:30.964 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":92,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:33:55.547519+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:34:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 msgr2=0x7f05540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 0x7f05540a6070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0568101a70 tx=0x7f056400a590 comp rx=0 tx=0).stop 2024-04-02T13:34:30.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 msgr2=0x7f056819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0558009230 tx=0x7f0558004040 comp rx=0 tx=0).stop 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 shutdown_connections 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05681025d0 0x7f056819f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0568101c10 0x7f056819cbd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05540a3bb0 0x7f05540a6070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 --2- 172.21.15.122:0/514581970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0568100a20 0x7f05680697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 >> 172.21.15.122:0/514581970 conn(0x7f05680fbf90 msgr2=0x7f0568104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 shutdown_connections 2024-04-02T13:34:30.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:30.963+0000 7f056fae4640 1 -- 172.21.15.122:0/514581970 wait complete. 2024-04-02T13:34:30.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:34:30.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:34:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:30 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3039421519' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:34:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:30 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3039421519' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-02T13:34:31.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 -- 172.21.15.122:0/1943554603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4107800 msgr2=0x7f56f4109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/1943554603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4107800 0x7f56f4109c50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009ca0 tx=0x7f56e402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 -- 172.21.15.122:0/1943554603 shutdown_connections 2024-04-02T13:34:31.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/1943554603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4107800 0x7f56f4109c50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/1943554603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f4104ed0 0x7f56f41072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/1943554603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f41025a0 0x7f56f4104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 -- 172.21.15.122:0/1943554603 >> 172.21.15.122:0/1943554603 conn(0x7f56f40fbfb0 msgr2=0x7f56f40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 -- 172.21.15.122:0/1943554603 shutdown_connections 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.080+0000 7f56f8b7c640 1 -- 172.21.15.122:0/1943554603 wait complete. 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 Processor -- start 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 -- start start 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 0x7f56f4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f4107800 0x7f56f419f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410d7c0 con 0x7f56f4104ed0 2024-04-02T13:34:31.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410d640 con 0x7f56f41025a0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56f410d940 con 0x7f56f4107800 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44016/0 (socket says 172.21.15.122:44016) 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 -- 172.21.15.122:0/3108271424 learned_addr learned my addr 172.21.15.122:0/3108271424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f2d9d640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f4107800 0x7f56f419f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f259c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 0x7f56f4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 -- 172.21.15.122:0/3108271424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f4107800 msgr2=0x7f56f419f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f4107800 0x7f56f419f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 -- 172.21.15.122:0/3108271424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 msgr2=0x7f56f4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 0x7f56f4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 -- 172.21.15.122:0/3108271424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f419fa30 con 0x7f56f4104ed0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f259c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 0x7f56f4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f1d9b640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d770 tx=0x7f56dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc017070 con 0x7f56f4104ed0 2024-04-02T13:34:31.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56dc009c30 con 0x7f56f4104ed0 2024-04-02T13:34:31.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.081+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f419fd20 con 0x7f56f4104ed0 2024-04-02T13:34:31.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.082+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc00f620 con 0x7f56f4104ed0 2024-04-02T13:34:31.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.082+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f41a0210 con 0x7f56f4104ed0 2024-04-02T13:34:31.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.082+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f40fdaa0 con 0x7f56f4104ed0 2024-04-02T13:34:31.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.083+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f56dc01a070 con 0x7f56f4104ed0 2024-04-02T13:34:31.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.084+0000 7f56eb7fe640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 0x7f56e00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.084+0000 7f56f259c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 0x7f56e00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.084+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f56dc0d0270 con 0x7f56f4104ed0 2024-04-02T13:34:31.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.084+0000 7f56f259c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 0x7f56e00a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005fd0 tx=0x7f56ec005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.088+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f56dc009de0 con 0x7f56f4104ed0 2024-04-02T13:34:31.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.268+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f56f40627a0 con 0x7f56f4104ed0 2024-04-02T13:34:31.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.268+0000 7f56eb7fe640 1 -- 172.21.15.122:0/3108271424 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f56dc09c8f0 con 0x7f56f4104ed0 2024-04-02T13:34:31.271 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 msgr2=0x7f56e00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 0x7f56e00a6070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f56ec005fd0 tx=0x7f56ec005ea0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 msgr2=0x7f56f4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f56dc00d770 tx=0x7f56dc00dc30 comp rx=0 tx=0).stop 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 shutdown_connections 2024-04-02T13:34:31.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56f4107800 0x7f56f419f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56f4104ed0 0x7f56f4198350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56e00a3bb0 0x7f56e00a6070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 --2- 172.21.15.122:0/3108271424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f56f41025a0 0x7f56f4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 >> 172.21.15.122:0/3108271424 conn(0x7f56f40fbfb0 msgr2=0x7f56f4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 shutdown_connections 2024-04-02T13:34:31.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.270+0000 7f56f8b7c640 1 -- 172.21.15.122:0/3108271424 wait complete. 2024-04-02T13:34:31.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 -- 172.21.15.122:0/1505010538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c104ed0 msgr2=0x7f7f4c1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 --2- 172.21.15.122:0/1505010538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1072c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009900 tx=0x7f7f4002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 -- 172.21.15.122:0/1505010538 shutdown_connections 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 --2- 172.21.15.122:0/1505010538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107800 0x7f7f4c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 --2- 172.21.15.122:0/1505010538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1072c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 --2- 172.21.15.122:0/1505010538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 0x7f7f4c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 -- 172.21.15.122:0/1505010538 >> 172.21.15.122:0/1505010538 conn(0x7f7f4c0fbfb0 msgr2=0x7f7f4c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 -- 172.21.15.122:0/1505010538 shutdown_connections 2024-04-02T13:34:31.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.386+0000 7f7f510a5640 1 -- 172.21.15.122:0/1505010538 wait complete. 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 Processor -- start 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 -- start start 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 0x7f7f4c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c10d7b0 con 0x7f7f4c107800 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c10d630 con 0x7f7f4c104ed0 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f510a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c10d930 con 0x7f7f4c1025a0 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f4b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f4b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44034/0 (socket says 172.21.15.122:44034) 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f4b59e640 1 -- 172.21.15.122:0/2064160545 learned_addr learned my addr 172.21.15.122:0/2064160545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f4a59c640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.387+0000 7f7f4ad9d640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 0x7f7f4c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 -- 172.21.15.122:0/2064160545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 msgr2=0x7f7f4c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 0x7f7f4c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 -- 172.21.15.122:0/2064160545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c104ed0 msgr2=0x7f7f4c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 -- 172.21.15.122:0/2064160545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19faa0 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f4b59e640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400eaa0 tx=0x7f7f4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400cdb0 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f44005020 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19fd90 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440116b0 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.388+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a0200 con 0x7f7f4c107800 2024-04-02T13:34:31.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.389+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c0fdaa0 con 0x7f7f4c107800 2024-04-02T13:34:31.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.390+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7f44005560 con 0x7f7f4c107800 2024-04-02T13:34:31.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.391+0000 7f7f33fff640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 0x7f7f340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.391+0000 7f7f4ad9d640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 0x7f7f340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.391+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7f440d0ca0 con 0x7f7f4c107800 2024-04-02T13:34:31.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.391+0000 7f7f4ad9d640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 0x7f7f340a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0058c0 tx=0x7f7f3c00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.394+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7f44016030 con 0x7f7f4c107800 2024-04-02T13:34:31.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.573+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7f4c02d370 con 0x7f7f4c107800 2024-04-02T13:34:31.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.573+0000 7f7f33fff640 1 -- 172.21.15.122:0/2064160545 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f7f4409ccd0 con 0x7f7f4c107800 2024-04-02T13:34:31.576 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:31.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 msgr2=0x7f7f340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 0x7f7f340a6070 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c0058c0 tx=0x7f7f3c00a250 comp rx=0 tx=0).stop 2024-04-02T13:34:31.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 msgr2=0x7f7f4c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400eaa0 tx=0x7f7f4400ef60 comp rx=0 tx=0).stop 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 shutdown_connections 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c107800 0x7f7f4c19f430 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f7f340a3bb0 0x7f7f340a6070 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c104ed0 0x7f7f4c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 --2- 172.21.15.122:0/2064160545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c1025a0 0x7f7f4c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 >> 172.21.15.122:0/2064160545 conn(0x7f7f4c0fbfb0 msgr2=0x7f7f4c103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 shutdown_connections 2024-04-02T13:34:31.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.575+0000 7f7f510a5640 1 -- 172.21.15.122:0/2064160545 wait complete. 2024-04-02T13:34:31.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 -- 172.21.15.122:0/3182324247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f4100910 msgr2=0x7f40f4100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 --2- 172.21.15.122:0/3182324247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f4100910 0x7f40f4100d30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009900 tx=0x7f40e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 -- 172.21.15.122:0/3182324247 shutdown_connections 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 --2- 172.21.15.122:0/3182324247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f41024c0 0x7f40f4108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 --2- 172.21.15.122:0/3182324247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 0x7f40f4101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 --2- 172.21.15.122:0/3182324247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f4100910 0x7f40f4100d30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 -- 172.21.15.122:0/3182324247 >> 172.21.15.122:0/3182324247 conn(0x7f40f40fbea0 msgr2=0x7f40f40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 -- 172.21.15.122:0/3182324247 shutdown_connections 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.691+0000 7f40f9403640 1 -- 172.21.15.122:0/3182324247 wait complete. 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 Processor -- start 2024-04-02T13:34:31.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 -- start start 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f4100910 0x7f40f4196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 0x7f40f4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f40f410c590 con 0x7f40f41024c0 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40f410c410 con 0x7f40f4101b00 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f9403640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40f410c710 con 0x7f40f4100910 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.692+0000 7f40f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44054/0 (socket says 172.21.15.122:44054) 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 -- 172.21.15.122:0/2063904052 learned_addr learned my addr 172.21.15.122:0/2063904052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f2ffd640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f4100910 0x7f40f4196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f27fc640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 0x7f40f4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 -- 172.21.15.122:0/2063904052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f4100910 msgr2=0x7f40f4196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f4100910 0x7f40f4196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 -- 172.21.15.122:0/2063904052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 msgr2=0x7f40f4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 0x7f40f4197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 -- 172.21.15.122:0/2063904052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f419e830 con 0x7f40f41024c0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f37fe640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ebb0 tx=0x7f40ec007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec007ca0 con 0x7f40f41024c0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f4068c60 con 0x7f40f41024c0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec007e40 con 0x7f40f41024c0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0058a0 con 0x7f40f41024c0 2024-04-02T13:34:31.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.693+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f40691d0 con 0x7f40f41024c0 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.694+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4068810 con 0x7f40f41024c0 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.696+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f40ec004a40 con 0x7f40f41024c0 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.696+0000 7f40dbfff640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 0x7f40dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.697+0000 7f40f2ffd640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 0x7f40dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.697+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f40ec0d2100 con 0x7f40f41024c0 2024-04-02T13:34:31.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.697+0000 7f40f2ffd640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 0x7f40dc0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000c00 tx=0x7f40e8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:31.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.700+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f40ec09e130 con 0x7f40f41024c0 2024-04-02T13:34:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:31 smithi122 ceph-mon[32114]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:34:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:31 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/514581970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:31 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3108271424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:31.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.878+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40f4107830 con 0x7f40f41024c0 2024-04-02T13:34:31.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.878+0000 7f40dbfff640 1 -- 172.21.15.122:0/2063904052 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f40ec0a2430 con 0x7f40f41024c0 2024-04-02T13:34:31.881 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:34:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.880+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 msgr2=0x7f40dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.880+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 0x7f40dc0a6070 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000c00 tx=0x7f40e8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.880+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 msgr2=0x7f40f419e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:31.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.880+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f40ec00ebb0 tx=0x7f40ec007590 comp rx=0 tx=0).stop 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 shutdown_connections 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f40f41024c0 0x7f40f419e190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f40dc0a3bb0 0x7f40dc0a6070 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40f4101b00 0x7f40f4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 --2- 172.21.15.122:0/2063904052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40f4100910 0x7f40f4196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 >> 172.21.15.122:0/2063904052 conn(0x7f40f40fbea0 msgr2=0x7f40f4104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 shutdown_connections 2024-04-02T13:34:31.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.881+0000 7f40f9403640 1 -- 172.21.15.122:0/2063904052 wait complete. 2024-04-02T13:34:31.894 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:34:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:31 smithi187 ceph-mon[36064]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:34:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:31 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/514581970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:31 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3108271424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- 172.21.15.122:0/4095728724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 msgr2=0x7f3948102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 --2- 172.21.15.122:0/4095728724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948102090 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f393c009900 tx=0x7f393c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:32.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- 172.21.15.122:0/4095728724 shutdown_connections 2024-04-02T13:34:32.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 --2- 172.21.15.122:0/4095728724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 0x7f3948108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 --2- 172.21.15.122:0/4095728724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948102090 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 --2- 172.21.15.122:0/4095728724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 0x7f3948100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- 172.21.15.122:0/4095728724 >> 172.21.15.122:0/4095728724 conn(0x7f39480fbfb0 msgr2=0x7f39480fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- 172.21.15.122:0/4095728724 shutdown_connections 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- 172.21.15.122:0/4095728724 wait complete. 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 Processor -- start 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.997+0000 7f394e97b640 1 -- start start 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 0x7f3948197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 0x7f394819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f394810d820 con 0x7f3948101c10 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f394810d6a0 con 0x7f39481025d0 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394e97b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f394810d9a0 con 0x7f3948100a20 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44080/0 (socket says 172.21.15.122:44080) 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 -- 172.21.15.122:0/2360194128 learned_addr learned my addr 172.21.15.122:0/2360194128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f3947fff640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 0x7f3948197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f394cf18640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 0x7f394819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 -- 172.21.15.122:0/2360194128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 msgr2=0x7f3948197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 0x7f3948197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 -- 172.21.15.122:0/2360194128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 msgr2=0x7f394819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 0x7f394819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 -- 172.21.15.122:0/2360194128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394819fa50 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.998+0000 7f39477fe640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f393c000c00 tx=0x7f393c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.999+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c043020 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.999+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f393c0054e0 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.999+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394819fce0 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.999+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c003e60 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:31.999+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481a0220 con 0x7f3948101c10 2024-04-02T13:34:32.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.000+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3948107dc0 con 0x7f3948101c10 2024-04-02T13:34:32.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.001+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f393c046020 con 0x7f3948101c10 2024-04-02T13:34:32.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.001+0000 7f39457fa640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 0x7f39300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.001+0000 7f3947fff640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 0x7f39300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.001+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f393c0f28a0 con 0x7f3948101c10 2024-04-02T13:34:32.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.002+0000 7f3947fff640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 0x7f39300a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3948101a70 tx=0x7f393800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.006+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f393c04d020 con 0x7f3948101c10 2024-04-02T13:34:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:31 smithi188 ceph-mon[36117]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-02T13:34:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:31 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/514581970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:31 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3108271424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.183+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f39480627a0 con 0x7f3948101c10 2024-04-02T13:34:32.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.183+0000 7f39457fa640 1 -- 172.21.15.122:0/2360194128 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f393c0bef20 con 0x7f3948101c10 2024-04-02T13:34:32.186 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 msgr2=0x7f39300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 0x7f39300a60f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3948101a70 tx=0x7f393800a2d0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 msgr2=0x7f3948198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f393c000c00 tx=0x7f393c005dd0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 shutdown_connections 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f39300a3c30 0x7f39300a60f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39481025d0 0x7f394819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3948101c10 0x7f3948198370 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 --2- 172.21.15.122:0/2360194128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3948100a20 0x7f3948197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 >> 172.21.15.122:0/2360194128 conn(0x7f39480fbfb0 msgr2=0x7f3948104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 shutdown_connections 2024-04-02T13:34:32.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.185+0000 7f394e97b640 1 -- 172.21.15.122:0/2360194128 wait complete. 2024-04-02T13:34:32.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 -- 172.21.15.122:0/3324383062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc1025c0 msgr2=0x7f64fc108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 --2- 172.21.15.122:0/3324383062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc1025c0 0x7f64fc108e50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009ca0 tx=0x7f64f402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 -- 172.21.15.122:0/3324383062 shutdown_connections 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 --2- 172.21.15.122:0/3324383062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc1025c0 0x7f64fc108e50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 --2- 172.21.15.122:0/3324383062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc101c00 0x7f64fc102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 --2- 172.21.15.122:0/3324383062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.298+0000 7f6501350640 1 -- 172.21.15.122:0/3324383062 >> 172.21.15.122:0/3324383062 conn(0x7f64fc0fbf80 msgr2=0x7f64fc0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- 172.21.15.122:0/3324383062 shutdown_connections 2024-04-02T13:34:32.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- 172.21.15.122:0/3324383062 wait complete. 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 Processor -- start 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- start start 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc1025c0 0x7f64fc19e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10c700 con 0x7f64fc101c00 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10c580 con 0x7f64fc100a10 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f6501350640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc10c880 con 0x7f64fc1025c0 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f64f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44094/0 (socket says 172.21.15.122:44094) 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f64f3fff640 1 -- 172.21.15.122:0/2327328137 learned_addr learned my addr 172.21.15.122:0/2327328137 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.299+0000 7f64faffd640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64fb7fe640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc1025c0 0x7f64fc19e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 -- 172.21.15.122:0/2327328137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc1025c0 msgr2=0x7f64fc19e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc1025c0 0x7f64fc19e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 -- 172.21.15.122:0/2327328137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 msgr2=0x7f64fc196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 -- 172.21.15.122:0/2327328137 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc19e9d0 con 0x7f64fc101c00 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64faffd640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f3fff640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4017070 con 0x7f64fc101c00 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e4009c30 con 0x7f64fc101c00 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc19ecc0 con 0x7f64fc101c00 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e400f6b0 con 0x7f64fc101c00 2024-04-02T13:34:32.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.300+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc19f200 con 0x7f64fc101c00 2024-04-02T13:34:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.301+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64fc107cd0 con 0x7f64fc101c00 2024-04-02T13:34:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.302+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f64e40109d0 con 0x7f64fc101c00 2024-04-02T13:34:32.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.302+0000 7f64f8ff9640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 0x7f64e80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.303+0000 7f64faffd640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 0x7f64e80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.303+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f64e40d0300 con 0x7f64fc101c00 2024-04-02T13:34:32.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.303+0000 7f64faffd640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 0x7f64e80a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64fc101a60 tx=0x7f64ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.306+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f64e4009de0 con 0x7f64fc101c00 2024-04-02T13:34:32.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.522+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64fc02d5c0 con 0x7f64fc101c00 2024-04-02T13:34:32.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.522+0000 7f64f8ff9640 1 -- 172.21.15.122:0/2327328137 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7f64e409caa0 con 0x7f64fc101c00 2024-04-02T13:34:32.525 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:32.525 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:34:32.525 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 msgr2=0x7f64e80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 0x7f64e80a60f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f64fc101a60 tx=0x7f64ec008040 comp rx=0 tx=0).stop 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 msgr2=0x7f64fc1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64e400d950 tx=0x7f64e400de10 comp rx=0 tx=0).stop 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 shutdown_connections 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64fc1025c0 0x7f64fc19e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64fc101c00 0x7f64fc1972f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f64e80a3c30 0x7f64e80a60f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 --2- 172.21.15.122:0/2327328137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f64fc100a10 0x7f64fc196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 >> 172.21.15.122:0/2327328137 conn(0x7f64fc0fbf80 msgr2=0x7f64fc104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 shutdown_connections 2024-04-02T13:34:32.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.524+0000 7f6501350640 1 -- 172.21.15.122:0/2327328137 wait complete. 2024-04-02T13:34:32.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:34:32.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- 172.21.15.122:0/2172625463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e81025d0 msgr2=0x7f10e8108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/2172625463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e81025d0 0x7f10e8108e60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- 172.21.15.122:0/2172625463 shutdown_connections 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/2172625463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e81025d0 0x7f10e8108e60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/2172625463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e8101c10 0x7f10e8102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/2172625463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 0x7f10e8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- 172.21.15.122:0/2172625463 >> 172.21.15.122:0/2172625463 conn(0x7f10e80fbf90 msgr2=0x7f10e80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- 172.21.15.122:0/2172625463 shutdown_connections 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- 172.21.15.122:0/2172625463 wait complete. 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 Processor -- start 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- start start 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 0x7f10e8196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e81025d0 0x7f10e819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810c6a0 con 0x7f10e8101c10 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810c520 con 0x7f10e81025d0 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ef8f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10e810c820 con 0x7f10e8100a20 2024-04-02T13:34:32.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ece8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ece8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44114/0 (socket says 172.21.15.122:44114) 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.639+0000 7f10ece8d640 1 -- 172.21.15.122:0/1129947271 learned_addr learned my addr 172.21.15.122:0/1129947271 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ed68e640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 0x7f10e8196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ede8f640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e81025d0 0x7f10e819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 -- 172.21.15.122:0/1129947271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 msgr2=0x7f10e8196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 0x7f10e8196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 -- 172.21.15.122:0/1129947271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e81025d0 msgr2=0x7f10e819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e81025d0 0x7f10e819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 -- 172.21.15.122:0/1129947271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e819e940 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ece8d640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00d950 tx=0x7f10dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc017070 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e819ec30 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc009c30 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc00f6b0 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.640+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e819f180 con 0x7f10e8101c10 2024-04-02T13:34:32.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.641+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e8107d40 con 0x7f10e8101c10 2024-04-02T13:34:32.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.643+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f10dc0109d0 con 0x7f10e8101c10 2024-04-02T13:34:32.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.643+0000 7f10e67fc640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 0x7f10d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.644+0000 7f10ed68e640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 0x7f10d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.644+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f10dc0d04c0 con 0x7f10e8101c10 2024-04-02T13:34:32.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.644+0000 7f10ed68e640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 0x7f10d00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f10e8198200 tx=0x7f10e0009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.647+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f10dc009de0 con 0x7f10e8101c10 2024-04-02T13:34:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2064160545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2063904052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2360194128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:32 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2327328137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:32.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.825+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10e802d540 con 0x7f10e8101c10 2024-04-02T13:34:32.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.826+0000 7f10e67fc640 1 -- 172.21.15.122:0/1129947271 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24702 (secure 0 0 0) 0x7f10dc09cae0 con 0x7f10e8101c10 2024-04-02T13:34:32.829 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:32.829 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":92,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:33:55.547519+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 msgr2=0x7f10d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 0x7f10d00a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f10e8198200 tx=0x7f10e0009680 comp rx=0 tx=0).stop 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 msgr2=0x7f10e8197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00d950 tx=0x7f10dc00de10 comp rx=0 tx=0).stop 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 shutdown_connections 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f10d00a3bb0 0x7f10d00a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10e81025d0 0x7f10e819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f10e8101c10 0x7f10e8197270 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 --2- 172.21.15.122:0/1129947271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10e8100a20 0x7f10e8196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 >> 172.21.15.122:0/1129947271 conn(0x7f10e80fbf90 msgr2=0x7f10e8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.828+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 shutdown_connections 2024-04-02T13:34:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.829+0000 7f10ef8f2640 1 -- 172.21.15.122:0/1129947271 wait complete. 2024-04-02T13:34:32.843 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:34:32.844 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 -- 172.21.15.122:0/2074041575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714101bf0 msgr2=0x7f5714102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 --2- 172.21.15.122:0/2074041575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714101bf0 0x7f5714102070 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5708009900 tx=0x7f570802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 -- 172.21.15.122:0/2074041575 shutdown_connections 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 --2- 172.21.15.122:0/2074041575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57141025b0 0x7f5714108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 --2- 172.21.15.122:0/2074041575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714101bf0 0x7f5714102070 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 --2- 172.21.15.122:0/2074041575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5714100a00 0x7f5714100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.949+0000 7f571b54d640 1 -- 172.21.15.122:0/2074041575 >> 172.21.15.122:0/2074041575 conn(0x7f57140fbf90 msgr2=0x7f57140fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- 172.21.15.122:0/2074041575 shutdown_connections 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- 172.21.15.122:0/2074041575 wait complete. 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 Processor -- start 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- start start 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5714101bf0 0x7f5714198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57141025b0 0x7f571419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f571410c680 con 0x7f5714100a00 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f571410c500 con 0x7f57141025b0 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f571b54d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f571410c800 con 0x7f5714101bf0 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f57192e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f57192e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44126/0 (socket says 172.21.15.122:44126) 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f57192e9640 1 -- 172.21.15.122:0/2174452881 learned_addr learned my addr 172.21.15.122:0/2174452881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f5718ae8640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5714101bf0 0x7f5714198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.950+0000 7f5719aea640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57141025b0 0x7f571419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 -- 172.21.15.122:0/2174452881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5714101bf0 msgr2=0x7f5714198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5714101bf0 0x7f5714198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 -- 172.21.15.122:0/2174452881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57141025b0 msgr2=0x7f571419f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57141025b0 0x7f571419f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 -- 172.21.15.122:0/2174452881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419fa40 con 0x7f5714100a00 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57192e9640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f570400ea80 tx=0x7f570400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400ce40 con 0x7f5714100a00 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419fd30 con 0x7f5714100a00 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5704011950 con 0x7f5714100a00 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570401ab80 con 0x7f5714100a00 2024-04-02T13:34:32.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.951+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a0190 con 0x7f5714100a00 2024-04-02T13:34:32.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.952+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57141a0450 con 0x7f5714100a00 2024-04-02T13:34:32.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.953+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f5704011b00 con 0x7f5714100a00 2024-04-02T13:34:32.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.954+0000 7f57127fc640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:32.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.954+0000 7f5718ae8640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:32.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.954+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f57040d10a0 con 0x7f5714100a00 2024-04-02T13:34:32.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.954+0000 7f5718ae8640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5708006010 tx=0x7f5708000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:32.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:32.957+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5704016030 con 0x7f5714100a00 2024-04-02T13:34:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2064160545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2063904052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2360194128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:32 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2327328137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2064160545' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2063904052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2360194128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:32 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2327328137' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.134+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57141a0620 con 0x7f5714100a00 2024-04-02T13:34:33.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.135+0000 7f57127fc640 1 -- 172.21.15.122:0/2174452881 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f570409d720 con 0x7f5714100a00 2024-04-02T13:34:33.137 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 msgr2=0x7f56fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5708006010 tx=0x7f5708000bc0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 msgr2=0x7f5714197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f570400ea80 tx=0x7f570400ef40 comp rx=0 tx=0).stop 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 shutdown_connections 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f56fc0a3bb0 0x7f56fc0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57141025b0 0x7f571419f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5714101bf0 0x7f5714198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 --2- 172.21.15.122:0/2174452881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5714100a00 0x7f5714197e20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.136+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 >> 172.21.15.122:0/2174452881 conn(0x7f57140fbf90 msgr2=0x7f57140fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.137+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 shutdown_connections 2024-04-02T13:34:33.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.137+0000 7f571b54d640 1 -- 172.21.15.122:0/2174452881 wait complete. 2024-04-02T13:34:33.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:34:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.252+0000 7f3503be2640 1 -- 172.21.15.122:0/290979186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc1077e0 msgr2=0x7f34fc109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.252+0000 7f3503be2640 1 --2- 172.21.15.122:0/290979186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc1077e0 0x7f34fc109c30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009900 tx=0x7f34f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- 172.21.15.122:0/290979186 shutdown_connections 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- 172.21.15.122:0/290979186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc1077e0 0x7f34fc109c30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- 172.21.15.122:0/290979186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 0x7f34fc1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- 172.21.15.122:0/290979186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc102580 0x7f34fc104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- 172.21.15.122:0/290979186 >> 172.21.15.122:0/290979186 conn(0x7f34fc0fbf90 msgr2=0x7f34fc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- 172.21.15.122:0/290979186 shutdown_connections 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- 172.21.15.122:0/290979186 wait complete. 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 Processor -- start 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- start start 2024-04-02T13:34:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 0x7f34fc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc1077e0 0x7f34fc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10d7d0 con 0x7f34fc102580 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10d650 con 0x7f34fc1077e0 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f3503be2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34fc10d950 con 0x7f34fc104eb0 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f350197e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f350197e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44148/0 (socket says 172.21.15.122:44148) 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f350197e640 1 -- 172.21.15.122:0/3938427418 learned_addr learned my addr 172.21.15.122:0/3938427418 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.253+0000 7f350117d640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 0x7f34fc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350217f640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc1077e0 0x7f34fc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 -- 172.21.15.122:0/3938427418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 msgr2=0x7f34fc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 0x7f34fc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 -- 172.21.15.122:0/3938427418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc1077e0 msgr2=0x7f34fc19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc1077e0 0x7f34fc19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 -- 172.21.15.122:0/3938427418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34fc19fa30 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f350197e640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f34f400eab0 tx=0x7f34f400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f400ce40 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f4011950 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34fc19fd20 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4010ae0 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.254+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34fc1a01c0 con 0x7f34fc102580 2024-04-02T13:34:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.255+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34fc0fda80 con 0x7f34fc102580 2024-04-02T13:34:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.256+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f34f4011b00 con 0x7f34fc102580 2024-04-02T13:34:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.256+0000 7f34f2ffd640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 0x7f34e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.257+0000 7f350117d640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 0x7f34e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.257+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f34f40d0e20 con 0x7f34fc102580 2024-04-02T13:34:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.257+0000 7f350117d640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 0x7f34e80a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34fc199320 tx=0x7f34ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.261+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f34f4016030 con 0x7f34fc102580 2024-04-02T13:34:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.440+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34fc02d370 con 0x7f34fc102580 2024-04-02T13:34:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.441+0000 7f34f2ffd640 1 -- 172.21.15.122:0/3938427418 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f34fc02d370 con 0x7f34fc102580 2024-04-02T13:34:33.443 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 msgr2=0x7f34e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 0x7f34e80a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f34fc199320 tx=0x7f34ec00b390 comp rx=0 tx=0).stop 2024-04-02T13:34:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 msgr2=0x7f34fc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f34f400eab0 tx=0x7f34f400ef70 comp rx=0 tx=0).stop 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 shutdown_connections 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f34e80a3bb0 0x7f34e80a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34fc1077e0 0x7f34fc19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34fc104eb0 0x7f34fc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 --2- 172.21.15.122:0/3938427418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34fc102580 0x7f34fc197e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 >> 172.21.15.122:0/3938427418 conn(0x7f34fc0fbf90 msgr2=0x7f34fc102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 shutdown_connections 2024-04-02T13:34:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.443+0000 7f3503be2640 1 -- 172.21.15.122:0/3938427418 wait complete. 2024-04-02T13:34:33.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 -- 172.21.15.122:0/1278958103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 msgr2=0x7f4fbc104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/1278958103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc104980 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009900 tx=0x7f4fb002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 -- 172.21.15.122:0/1278958103 shutdown_connections 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/1278958103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc1077f0 0x7f4fbc109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/1278958103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc104ec0 0x7f4fbc1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/1278958103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc104980 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 -- 172.21.15.122:0/1278958103 >> 172.21.15.122:0/1278958103 conn(0x7f4fbc0fbfa0 msgr2=0x7f4fbc0fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 -- 172.21.15.122:0/1278958103 shutdown_connections 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 -- 172.21.15.122:0/1278958103 wait complete. 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.560+0000 7f4fc21ba640 1 Processor -- start 2024-04-02T13:34:33.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- start start 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc104ec0 0x7f4fbc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc1077f0 0x7f4fbc19f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d800 con 0x7f4fbc102590 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d680 con 0x7f4fbc1077f0 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc10d980 con 0x7f4fbc104ec0 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44160/0 (socket says 172.21.15.122:44160) 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 -- 172.21.15.122:0/2056498122 learned_addr learned my addr 172.21.15.122:0/2056498122 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbbfff640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc1077f0 0x7f4fbc19f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 -- 172.21.15.122:0/2056498122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc104ec0 msgr2=0x7f4fbc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc104ec0 0x7f4fbc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 -- 172.21.15.122:0/2056498122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc1077f0 msgr2=0x7f4fbc19f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc1077f0 0x7f4fbc19f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 -- 172.21.15.122:0/2056498122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc19faf0 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fbb7fe640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009230 tx=0x7f4fb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0041070 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc19fd80 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fb00054e0 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00346a0 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.561+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc1a0210 con 0x7f4fbc102590 2024-04-02T13:34:33.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.562+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc0fda90 con 0x7f4fbc102590 2024-04-02T13:34:33.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.564+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4fb0044070 con 0x7f4fbc102590 2024-04-02T13:34:33.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.564+0000 7f4fb8ff9640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 0x7f4fa40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.564+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f4fb00f2970 con 0x7f4fbc102590 2024-04-02T13:34:33.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.564+0000 7f4fbaffd640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 0x7f4fa40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.565+0000 7f4fbaffd640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 0x7f4fa40a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc1993e0 tx=0x7f4fac005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.568+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4fb00bef90 con 0x7f4fbc102590 2024-04-02T13:34:33.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.746+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4fbc02d5d0 con 0x7f4fbc102590 2024-04-02T13:34:33.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.747+0000 7f4fb8ff9640 1 -- 172.21.15.122:0/2056498122 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f4fb00a7e20 con 0x7f4fbc102590 2024-04-02T13:34:33.750 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 msgr2=0x7f4fa40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 0x7f4fa40a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc1993e0 tx=0x7f4fac005bf0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 msgr2=0x7f4fbc197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009230 tx=0x7f4fb0004040 comp rx=0 tx=0).stop 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 shutdown_connections 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fa40a3bb0 0x7f4fa40a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fbc1077f0 0x7f4fbc19f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fbc104ec0 0x7f4fbc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 --2- 172.21.15.122:0/2056498122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fbc102590 0x7f4fbc197ed0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 >> 172.21.15.122:0/2056498122 conn(0x7f4fbc0fbfa0 msgr2=0x7f4fbc102ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 shutdown_connections 2024-04-02T13:34:33.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.749+0000 7f4fc21ba640 1 -- 172.21.15.122:0/2056498122 wait complete. 2024-04-02T13:34:33.761 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:33 smithi122 ceph-mon[32114]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:34:33.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1129947271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2174452881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.762 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3938427418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:34:33.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 -- 172.21.15.122:0/3365870134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc1099d0 msgr2=0x7faddc10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 --2- 172.21.15.122:0/3365870134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc1099d0 0x7faddc10be60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009ca0 tx=0x7fadd402c180 comp rx=0 tx=0).stop 2024-04-02T13:34:33.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 -- 172.21.15.122:0/3365870134 shutdown_connections 2024-04-02T13:34:33.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 --2- 172.21.15.122:0/3365870134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc1099d0 0x7faddc10be60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 --2- 172.21.15.122:0/3365870134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 0x7faddc074940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 --2- 172.21.15.122:0/3365870134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc075ac0 0x7faddc073f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 -- 172.21.15.122:0/3365870134 >> 172.21.15.122:0/3365870134 conn(0x7faddc0fd090 msgr2=0x7faddc0ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 -- 172.21.15.122:0/3365870134 shutdown_connections 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.865+0000 7fade20b1640 1 -- 172.21.15.122:0/3365870134 wait complete. 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 Processor -- start 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 -- start start 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 0x7faddc199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc1099d0 0x7faddc1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10f9b0 con 0x7faddc075ac0 2024-04-02T13:34:33.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10f830 con 0x7faddc0744e0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7fade20b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faddc10fb30 con 0x7faddc1099d0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44170/0 (socket says 172.21.15.122:44170) 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 -- 172.21.15.122:0/4247775287 learned_addr learned my addr 172.21.15.122:0/4247775287 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddb7fe640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 0x7faddc199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddbfff640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc1099d0 0x7faddc1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 -- 172.21.15.122:0/4247775287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc1099d0 msgr2=0x7faddc1a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc1099d0 0x7faddc1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 -- 172.21.15.122:0/4247775287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 msgr2=0x7faddc199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 0x7faddc199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.866+0000 7faddaffd640 1 -- 172.21.15.122:0/4247775287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faddc1a1b60 con 0x7faddc075ac0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7faddaffd640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c920 tx=0x7fadd000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0007c70 con 0x7faddc075ac0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faddc1a1e50 con 0x7faddc075ac0 2024-04-02T13:34:33.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd0007e10 con 0x7faddc075ac0 2024-04-02T13:34:33.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0018890 con 0x7faddc075ac0 2024-04-02T13:34:33.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.867+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faddc1a2310 con 0x7faddc075ac0 2024-04-02T13:34:33.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.868+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faddc108790 con 0x7faddc075ac0 2024-04-02T13:34:33.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.869+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fadd0018a70 con 0x7faddc075ac0 2024-04-02T13:34:33.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.869+0000 7fadd8ff9640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 0x7fadc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:33.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.869+0000 7faddb7fe640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 0x7fadc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:33.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.869+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fadd00d0870 con 0x7faddc075ac0 2024-04-02T13:34:33.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.870+0000 7faddb7fe640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 0x7fadc40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005fd0 tx=0x7fadcc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:33.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:33.874+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fadd0014030 con 0x7faddc075ac0 2024-04-02T13:34:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:33 smithi187 ceph-mon[36064]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:34:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1129947271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2174452881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3938427418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:33 smithi188 ceph-mon[36117]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-02T13:34:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1129947271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2174452881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3938427418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.053+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faddc02d370 con 0x7faddc075ac0 2024-04-02T13:34:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.053+0000 7fadd8ff9640 1 -- 172.21.15.122:0/4247775287 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fadd009cef0 con 0x7faddc075ac0 2024-04-02T13:34:34.056 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 msgr2=0x7fadc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 0x7fadc40a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fadcc005fd0 tx=0x7fadcc00b390 comp rx=0 tx=0).stop 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 msgr2=0x7faddc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fadd000c920 tx=0x7fadd000cde0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 shutdown_connections 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faddc1099d0 0x7faddc1a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faddc075ac0 0x7faddc19a480 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fadc40a3bb0 0x7fadc40a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 --2- 172.21.15.122:0/4247775287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faddc0744e0 0x7faddc199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.055+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 >> 172.21.15.122:0/4247775287 conn(0x7faddc0fd090 msgr2=0x7faddc105960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.056+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 shutdown_connections 2024-04-02T13:34:34.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.056+0000 7fade20b1640 1 -- 172.21.15.122:0/4247775287 wait complete. 2024-04-02T13:34:34.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:34:34.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 -- 172.21.15.122:0/3129924564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6801036d0 msgr2=0x7fb680109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 --2- 172.21.15.122:0/3129924564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6801036d0 0x7fb680109f60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb678009ca0 tx=0x7fb67802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 -- 172.21.15.122:0/3129924564 shutdown_connections 2024-04-02T13:34:34.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 --2- 172.21.15.122:0/3129924564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6801036d0 0x7fb680109f60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 --2- 172.21.15.122:0/3129924564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb680102d10 0x7fb680103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 --2- 172.21.15.122:0/3129924564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 0x7fb680101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 -- 172.21.15.122:0/3129924564 >> 172.21.15.122:0/3129924564 conn(0x7fb6800fd0b0 msgr2=0x7fb6800ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 -- 172.21.15.122:0/3129924564 shutdown_connections 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 -- 172.21.15.122:0/3129924564 wait complete. 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.169+0000 7fb685dba640 1 Processor -- start 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 -- start start 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 0x7fb680197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6801036d0 0x7fb68019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb68010d800 con 0x7fb680102d10 2024-04-02T13:34:34.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb68010d680 con 0x7fb680101b20 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb685dba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb68010d980 con 0x7fb6801036d0 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44188/0 (socket says 172.21.15.122:44188) 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 -- 172.21.15.122:0/2873615832 learned_addr learned my addr 172.21.15.122:0/2873615832 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67ffff640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6801036d0 0x7fb68019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67f7fe640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 0x7fb680197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 -- 172.21.15.122:0/2873615832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6801036d0 msgr2=0x7fb68019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6801036d0 0x7fb68019f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 -- 172.21.15.122:0/2873615832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 msgr2=0x7fb680197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 0x7fb680197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.170+0000 7fb67effd640 1 -- 172.21.15.122:0/2873615832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb68019fa30 con 0x7fb680102d10 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb67effd640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b4a0 tx=0x7fb67400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674014070 con 0x7fb680102d10 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb674010af0 con 0x7fb680102d10 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68019fd20 con 0x7fb680102d10 2024-04-02T13:34:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674017d10 con 0x7fb680102d10 2024-04-02T13:34:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.171+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6801a01c0 con 0x7fb680102d10 2024-04-02T13:34:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.172+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb680108e40 con 0x7fb680102d10 2024-04-02T13:34:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.173+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fb674017070 con 0x7fb680102d10 2024-04-02T13:34:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.173+0000 7fb67cff9640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 0x7fb6680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.174+0000 7fb67f7fe640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 0x7fb6680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.174+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb6740cfb20 con 0x7fb680102d10 2024-04-02T13:34:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.174+0000 7fb67f7fe640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 0x7fb6680a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb670005fd0 tx=0x7fb670005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.178+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb67409c1a0 con 0x7fb680102d10 2024-04-02T13:34:34.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.393+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb68002d370 con 0x7fb680102d10 2024-04-02T13:34:34.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.394+0000 7fb67cff9640 1 -- 172.21.15.122:0/2873615832 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7fb6740bc0e0 con 0x7fb680102d10 2024-04-02T13:34:34.396 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:34.397 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:34:34.397 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 msgr2=0x7fb6680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 0x7fb6680a6070 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb670005fd0 tx=0x7fb670005850 comp rx=0 tx=0).stop 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 msgr2=0x7fb680198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb67400b4a0 tx=0x7fb67400b960 comp rx=0 tx=0).stop 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 shutdown_connections 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6801036d0 0x7fb68019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb680102d10 0x7fb680198350 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb6680a3bb0 0x7fb6680a6070 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 --2- 172.21.15.122:0/2873615832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb680101b20 0x7fb680197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 >> 172.21.15.122:0/2873615832 conn(0x7fb6800fd0b0 msgr2=0x7fb680106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 shutdown_connections 2024-04-02T13:34:34.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.396+0000 7fb685dba640 1 -- 172.21.15.122:0/2873615832 wait complete. 2024-04-02T13:34:34.410 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:34:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- 172.21.15.122:0/80953163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8101b40 msgr2=0x7f30b8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 --2- 172.21.15.122:0/80953163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8101b40 0x7f30b8101f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f30ac009900 tx=0x7f30ac02bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- 172.21.15.122:0/80953163 shutdown_connections 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 --2- 172.21.15.122:0/80953163 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 0x7f30b8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 --2- 172.21.15.122:0/80953163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8102d30 0x7f30b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 --2- 172.21.15.122:0/80953163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8101b40 0x7f30b8101f60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- 172.21.15.122:0/80953163 >> 172.21.15.122:0/80953163 conn(0x7f30b80fd0b0 msgr2=0x7f30b80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- 172.21.15.122:0/80953163 shutdown_connections 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- 172.21.15.122:0/80953163 wait complete. 2024-04-02T13:34:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 Processor -- start 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.510+0000 7f30bf260640 1 -- start start 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 0x7f30b80758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 0x7f30b8074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30b810d790 con 0x7f30b8102d30 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30b810d610 con 0x7f30b81036f0 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30b810d910 con 0x7f30b8101b40 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44198/0 (socket says 172.21.15.122:44198) 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 -- 172.21.15.122:0/119650756 learned_addr learned my addr 172.21.15.122:0/119650756 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bcffc640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 0x7f30b80758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 -- 172.21.15.122:0/119650756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 msgr2=0x7f30b80758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bd7fd640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 0x7f30b8074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 0x7f30b80758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 -- 172.21.15.122:0/119650756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 msgr2=0x7f30b8074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 0x7f30b8074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 -- 172.21.15.122:0/119650756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b8074cc0 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bcffc640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 0x7f30b80758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b7fff640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c890 tx=0x7f30a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4004220 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a40043c0 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b81a1fb0 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.511+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a40049e0 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.512+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b81a2420 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.512+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b8108fa0 con 0x7f30b8102d30 2024-04-02T13:34:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.513+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f30a40079b0 con 0x7f30b8102d30 2024-04-02T13:34:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.514+0000 7f30b5ffb640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 0x7f30a80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.514+0000 7f30bcffc640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 0x7f30a80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.514+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f30a40d07b0 con 0x7f30b8102d30 2024-04-02T13:34:34.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.514+0000 7f30bcffc640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 0x7f30a80a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02c7b0 tx=0x7f30ac005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.519+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f30a4014030 con 0x7f30b8102d30 2024-04-02T13:34:34.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.707+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30b80627a0 con 0x7f30b8102d30 2024-04-02T13:34:34.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.708+0000 7f30b5ffb640 1 -- 172.21.15.122:0/119650756 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24702 (secure 0 0 0) 0x7f30a409ce30 con 0x7f30b8102d30 2024-04-02T13:34:34.711 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:34.711 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":92,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:33:55.547519+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:34:34.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 msgr2=0x7f30a80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 0x7f30a80a60f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30ac02c7b0 tx=0x7f30ac005fd0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 msgr2=0x7f30b8073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c890 tx=0x7f30a400cd50 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 shutdown_connections 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f30a80a3c30 0x7f30a80a60f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30b81036f0 0x7f30b8074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8102d30 0x7f30b8073ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 --2- 172.21.15.122:0/119650756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30b8101b40 0x7f30b80758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.710+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 >> 172.21.15.122:0/119650756 conn(0x7f30b80fd0b0 msgr2=0x7f30b8106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.711+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 shutdown_connections 2024-04-02T13:34:34.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.711+0000 7f30bf260640 1 -- 172.21.15.122:0/119650756 wait complete. 2024-04-02T13:34:34.723 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:34 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2056498122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.723 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:34 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4247775287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.723 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:34 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2873615832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:34.724 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:34:34.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.823+0000 7fd26f214640 1 -- 172.21.15.122:0/1830695938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 msgr2=0x7fd268102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.823+0000 7fd26f214640 1 --2- 172.21.15.122:0/1830695938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd268102400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009900 tx=0x7fd25c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.823+0000 7fd26f214640 1 -- 172.21.15.122:0/1830695938 shutdown_connections 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- 172.21.15.122:0/1830695938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 0x7fd268108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- 172.21.15.122:0/1830695938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 0x7fd268101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- 172.21.15.122:0/1830695938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd268102400 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- 172.21.15.122:0/1830695938 >> 172.21.15.122:0/1830695938 conn(0x7fd2680fb740 msgr2=0x7fd2680fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- 172.21.15.122:0/1830695938 shutdown_connections 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- 172.21.15.122:0/1830695938 wait complete. 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 Processor -- start 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- start start 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 0x7fd268196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 0x7fd268197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810c6b0 con 0x7fd268101fe0 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810c530 con 0x7fd2681016d0 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26f214640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810c830 con 0x7fd268100d10 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26d7b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26d7b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44212/0 (socket says 172.21.15.122:44212) 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26d7b1640 1 -- 172.21.15.122:0/3783801838 learned_addr learned my addr 172.21.15.122:0/3783801838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd26cfb0640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 0x7fd268196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.824+0000 7fd267fff640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 0x7fd268197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 -- 172.21.15.122:0/3783801838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 msgr2=0x7fd268196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 0x7fd268196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 -- 172.21.15.122:0/3783801838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 msgr2=0x7fd268197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 0x7fd268197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 -- 172.21.15.122:0/3783801838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26819e970 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26d7b1640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd26000eaa0 tx=0x7fd26000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26000cdb0 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26819ec60 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260005020 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600116b0 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.825+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26819f1b0 con 0x7fd268101fe0 2024-04-02T13:34:34.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.826+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd268107e70 con 0x7fd268101fe0 2024-04-02T13:34:34.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.827+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd260005560 con 0x7fd268101fe0 2024-04-02T13:34:34.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.828+0000 7fd265ffb640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 0x7fd2580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:34.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.828+0000 7fd26cfb0640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 0x7fd2580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:34.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.828+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2600d08f0 con 0x7fd268101fe0 2024-04-02T13:34:34.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.828+0000 7fd26cfb0640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 0x7fd2580a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd25c02efd0 tx=0x7fd25c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:34.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:34.832+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd260016030 con 0x7fd268101fe0 2024-04-02T13:34:34.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:34 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2056498122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:34 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4247775287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:34.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:34 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2873615832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:34 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2056498122' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:35.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:34 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4247775287' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:35.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:34 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2873615832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.047+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd268062700 con 0x7fd268101fe0 2024-04-02T13:34:35.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.047+0000 7fd265ffb640 1 -- 172.21.15.122:0/3783801838 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7fd26009c920 con 0x7fd268101fe0 2024-04-02T13:34:35.050 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:35.050 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:34:35.051 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 msgr2=0x7fd2580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 0x7fd2580a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd25c02efd0 tx=0x7fd25c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 msgr2=0x7fd26819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd26000eaa0 tx=0x7fd26000ef60 comp rx=0 tx=0).stop 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 shutdown_connections 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd268101fe0 0x7fd26819e300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd2580a3bb0 0x7fd2580a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd2681016d0 0x7fd268197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 --2- 172.21.15.122:0/3783801838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd268100d10 0x7fd268196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.049+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 >> 172.21.15.122:0/3783801838 conn(0x7fd2680fb740 msgr2=0x7fd268105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.050+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 shutdown_connections 2024-04-02T13:34:35.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.050+0000 7fd26f214640 1 -- 172.21.15.122:0/3783801838 wait complete. 2024-04-02T13:34:35.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:32:49.884932+0000', 'modified': '2024-04-02T13:33:12.155913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24512, 'mds_1': 24487, 'mds_2': 24485, 'mds_3': 24460, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/1672540384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1672540384}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1672540384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.122:6841/385777794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6840', 'nonce': 385777794}, {'type': 'v1', 'addr': '172.21.15.122:6841', 'nonce': 385777794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/546075479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 546075479}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 546075479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6833/1223173792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1223173792}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1223173792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/179772221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 179772221}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 179772221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/4231455845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 4231455845}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 4231455845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6837/391935088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 391935088}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 391935088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.187:6837/453335780', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 453335780}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 453335780}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/180395175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 180395175}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 180395175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6839/2503909279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 2503909279}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 2503909279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24485, 24487, 24460, 24512, 14628]} 2024-04-02T13:34:35.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T13:34:35.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.164+0000 7f7588206640 1 -- 172.21.15.122:0/3741294279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 msgr2=0x7f7580102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- 172.21.15.122:0/3741294279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580102070 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7570009900 tx=0x7f757002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- 172.21.15.122:0/3741294279 shutdown_connections 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- 172.21.15.122:0/3741294279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75801025b0 0x7f7580108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- 172.21.15.122:0/3741294279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580102070 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- 172.21.15.122:0/3741294279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7580100a00 0x7f7580100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- 172.21.15.122:0/3741294279 >> 172.21.15.122:0/3741294279 conn(0x7f75800fbf90 msgr2=0x7f75800fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- 172.21.15.122:0/3741294279 shutdown_connections 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- 172.21.15.122:0/3741294279 wait complete. 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 Processor -- start 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- start start 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 0x7f7580196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 0x7f758019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f758010c660 con 0x7f7580101bf0 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f758010c4e0 con 0x7f75801025b0 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f7588206640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f758010c7e0 con 0x7f7580100a00 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.165+0000 7f75857a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44238/0 (socket says 172.21.15.122:44238) 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 -- 172.21.15.122:0/140417424 learned_addr learned my addr 172.21.15.122:0/140417424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7585fa2640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 0x7f7580196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75867a3640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 0x7f758019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 -- 172.21.15.122:0/140417424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 msgr2=0x7f7580196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 0x7f7580196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 -- 172.21.15.122:0/140417424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 msgr2=0x7f758019e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 0x7f758019e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 -- 172.21.15.122:0/140417424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758019e900 con 0x7f7580101bf0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7585fa2640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 0x7f7580196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75867a3640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 0x7f758019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f75857a1640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7570009230 tx=0x7f7570005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7576ffd640 1 -- 172.21.15.122:0/140417424 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570041070 con 0x7f7580101bf0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7576ffd640 1 -- 172.21.15.122:0/140417424 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75700054e0 con 0x7f7580101bf0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758019eb90 con 0x7f7580101bf0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7576ffd640 1 -- 172.21.15.122:0/140417424 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570034440 con 0x7f7580101bf0 2024-04-02T13:34:35.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.166+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758019f0d0 con 0x7f7580101bf0 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.168+0000 7f7576ffd640 1 -- 172.21.15.122:0/140417424 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f7570044070 con 0x7f7580101bf0 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 Processor -- start 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 -- start start 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561458918cf0 0x561458919110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f7576ffd640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 0x7f756c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458920ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56145887b060 con 0x5614589196e0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561458823650 con 0x56145891e850 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f758adef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561458877010 con 0x561458918cf0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f75857a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f7585fa2640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 0x7f756c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f75857a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44250/0 (socket says 172.21.15.122:44250) 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f75857a1640 1 -- 172.21.15.122:0/1588850650 learned_addr learned my addr 172.21.15.122:0/1588850650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f75867a3640 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458920ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f7576ffd640 1 -- 172.21.15.122:0/140417424 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f75700f2530 con 0x7f7580101bf0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.169+0000 7f75867a3640 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561458918cf0 0x561458919110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 -- 172.21.15.122:0/1588850650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561458918cf0 msgr2=0x561458919110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561458918cf0 0x561458919110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 -- 172.21.15.122:0/1588850650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 msgr2=0x561458920ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458920ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 -- 172.21.15.122:0/1588850650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145858f2d0 con 0x5614589196e0 2024-04-02T13:34:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f7585fa2640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 0x7f756c0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f75801981c0 tx=0x7f757c00b6b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f75857a1640 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75700badf0 tx=0x7f7570088a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567a6efea631f204 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f756bfff640 1 -- 172.21.15.122:0/1588850650 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757004b070 con 0x5614589196e0 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f756bfff640 1 -- 172.21.15.122:0/1588850650 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7570005690 con 0x5614589196e0 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.170+0000 7f756bfff640 1 -- 172.21.15.122:0/1588850650 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700bebb0 con 0x5614589196e0 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- 172.21.15.122:0/1588850650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 msgr2=0x56145891e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f75700badf0 tx=0x7f7570088a70 comp rx=0 tx=0).stop 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- 172.21.15.122:0/1588850650 shutdown_connections 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458920ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614589196e0 0x56145891e1e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 --2- 172.21.15.122:0/1588850650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561458918cf0 0x561458919110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- 172.21.15.122:0/1588850650 >> 172.21.15.122:0/1588850650 conn(0x561458912050 msgr2=0x561458912450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- 172.21.15.122:0/1588850650 shutdown_connections 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- 172.21.15.122:0/1588850650 wait complete. 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 -- start start 2024-04-02T13:34:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.171+0000 7f758adef740 1 Processor -- start 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561458918cf0 0x561458922f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458923470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5614589215c0 con 0x56145891e850 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561458836a10 con 0x561458918cf0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561458921740 con 0x5614589196e0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 10 client.0 _ll_register_callbacks cb 0x5614588730b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44260/0 (socket says 172.21.15.122:44260) 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 -- 172.21.15.122:0/2150445947 learned_addr learned my addr 172.21.15.122:0/2150445947 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75857a1640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458923470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561458918cf0 0x561458922f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 -- 172.21.15.122:0/2150445947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 msgr2=0x561458923470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458923470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 -- 172.21.15.122:0/2150445947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561458918cf0 msgr2=0x561458922f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561458918cf0 0x561458922f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 -- 172.21.15.122:0/2150445947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614586596b0 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75857a1640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458923470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75867a3640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7578013c20 tx=0x7f7578011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ae9e4041761b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75697fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578003b70 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7578003d10 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561458822cf0 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7578026050 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 10 client.15216 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.172+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56145892d890 con 0x56145891e850 2024-04-02T13:34:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561458822cf0 con 0x56145891e850 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f757801d070 con 0x56145891e850 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 10 client.15216 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 20 client.15216 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145892ff30 0x5614589313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 4 client.15216 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614588885a0 con 0x56145892ff30 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145892ff30 0x5614589313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f75697fa640 10 client.15216 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.173+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145892ff30 0x5614589313a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9ce60e3513f380b server_cookie=69539c6dbb38f46f in_seq=0 out_seq=0 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f75697fa640 1 -- 172.21.15.122:0/2150445947 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614588885a0 con 0x56145892ff30 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f75697fa640 10 client.15216 handle_command_reply: tid=0 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 client.15216 shutdown 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 msgr2=0x56145892a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7578013c20 tx=0x7f7578011810 comp rx=0 tx=0).stop 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 shutdown_connections 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145892ff30 0x5614589313a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145891e850 0x56145892a4b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458923470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 --2- 172.21.15.122:0/2150445947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561458918cf0 0x561458922f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f75697fa640 0 client.15216 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 >> 172.21.15.122:0/2150445947 conn(0x561458912050 msgr2=0x561458911190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 shutdown_connections 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 1 -- 172.21.15.122:0/2150445947 wait complete. 2024-04-02T13:34:35.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.174+0000 7f758adef740 20 client.15216 trim_cache size 0 max 16384 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 Processor -- start 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 -- start start 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614589196e0 0x56145891aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145891e850 0x56145891b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56145892f6e0 con 0x561458918cf0 2024-04-02T13:34:35.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56145892fdb0 con 0x5614589196e0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f758adef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561458930480 con 0x56145891e850 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f7585fa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f7585fa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44272/0 (socket says 172.21.15.122:44272) 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.192+0000 7f7585fa2640 1 -- 172.21.15.122:0/2388659840 learned_addr learned my addr 172.21.15.122:0/2388659840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f75867a3640 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145891e850 0x56145891b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 -- 172.21.15.122:0/2388659840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145891e850 msgr2=0x56145891b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145891e850 0x56145891b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 -- 172.21.15.122:0/2388659840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614589196e0 msgr2=0x56145891aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614589196e0 0x56145891aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 -- 172.21.15.122:0/2388659840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561458912aa0 con 0x561458918cf0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f7585fa2640 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f757c005fd0 tx=0x7f757c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b10b862a130b581 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f756b7fe640 1 -- 172.21.15.122:0/2388659840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c026d50 con 0x561458918cf0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f756b7fe640 1 -- 172.21.15.122:0/2388659840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757c026730 con 0x561458918cf0 2024-04-02T13:34:35.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.193+0000 7f756b7fe640 1 -- 172.21.15.122:0/2388659840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c01fc30 con 0x561458918cf0 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- 172.21.15.122:0/2388659840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 msgr2=0x56145891e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f757c005fd0 tx=0x7f757c015000 comp rx=0 tx=0).stop 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- 172.21.15.122:0/2388659840 shutdown_connections 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145891e850 0x56145891b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614589196e0 0x56145891aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- 172.21.15.122:0/2388659840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x56145891e210 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- 172.21.15.122:0/2388659840 >> 172.21.15.122:0/2388659840 conn(0x561458912050 msgr2=0x561458921a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- 172.21.15.122:0/2388659840 shutdown_connections 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- 172.21.15.122:0/2388659840 wait complete. 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- start start 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 Processor -- start 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458994410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458996960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561458931220 con 0x561458918cf0 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561458930b50 con 0x56145891e850 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5614589318f0 con 0x5614589196e0 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f758adef740 10 client.0 _ll_register_callbacks cb 0x7f75780138f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.194+0000 7f7585fa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44276/0 (socket says 172.21.15.122:44276) 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 -- 172.21.15.122:0/4032095375 learned_addr learned my addr 172.21.15.122:0/4032095375 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75857a1640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458994410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 -- 172.21.15.122:0/4032095375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 msgr2=0x561458994410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458994410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 -- 172.21.15.122:0/4032095375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 msgr2=0x561458996960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458996960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 -- 172.21.15.122:0/4032095375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561458996ea0 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f7585fa2640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f757c00be80 tx=0x7f757c01f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15c082bdcccaa6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75697fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c01f970 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f757c01fb10 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f757c026730 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f757c01fd70 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f758adef740 10 client.15222 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5614589981b0 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.195+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f757c026730 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f757c035070 con 0x561458918cf0 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 10 client.15222 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 10 client.15222 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 20 client.15222 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145899cc00 0x56145899b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 4 client.15222 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:34:35.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "status"}) -- 0x5614588828f0 con 0x56145899cc00 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75857a1640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145899cc00 0x56145899b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75697fa640 10 client.15222 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75857a1640 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145899cc00 0x56145899b910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b22fc983f617842 server_cookie=d8a0b4025a94beeb in_seq=0 out_seq=0 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75697fa640 1 -- 172.21.15.122:0/4032095375 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5614588828f0 con 0x56145899cc00 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75697fa640 10 client.15222 handle_command_reply: tid=0 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 client.15222 shutdown 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 msgr2=0x561458991ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f757c00be80 tx=0x7f757c01f4b0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 shutdown_connections 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56145899cc00 0x56145899b910 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56145891e850 0x561458996960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614589196e0 0x561458994410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 --2- 172.21.15.122:0/4032095375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561458918cf0 0x561458991ec0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f75697fa640 0 client.15222 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.196+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 >> 172.21.15.122:0/4032095375 conn(0x561458912050 msgr2=0x56145891aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 shutdown_connections 2024-04-02T13:34:35.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f758adef740 1 -- 172.21.15.122:0/4032095375 wait complete. 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f758adef740 20 client.15222 trim_cache size 0 max 16384 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 4, 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 14628, 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 84.046085665999996, 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:34:35.206 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 92, 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 111.81738066200001 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 msgr2=0x7f756c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 0x7f756c0a60f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f75801981c0 tx=0x7f757c00b6b0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 msgr2=0x7f7580197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7570009230 tx=0x7f7570005dd0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 shutdown_connections 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f756c0a3c30 0x7f756c0a60f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75801025b0 0x7f758019e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7580101bf0 0x7f7580197230 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 --2- 172.21.15.122:0/140417424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7580100a00 0x7f7580196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 >> 172.21.15.122:0/140417424 conn(0x7f75800fbf90 msgr2=0x7f7580104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 shutdown_connections 2024-04-02T13:34:35.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.197+0000 7f7588206640 1 -- 172.21.15.122:0/140417424 wait complete. 2024-04-02T13:34:35.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 -- 172.21.15.122:0/2499838162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 msgr2=0x7f75f81072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 --2- 172.21.15.122:0/2499838162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f81072a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009900 tx=0x7f75e402bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 -- 172.21.15.122:0/2499838162 shutdown_connections 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 --2- 172.21.15.122:0/2499838162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f81077e0 0x7f75f8109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 --2- 172.21.15.122:0/2499838162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f81072a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 --2- 172.21.15.122:0/2499838162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f8102580 0x7f75f8104970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 -- 172.21.15.122:0/2499838162 >> 172.21.15.122:0/2499838162 conn(0x7f75f80fbf90 msgr2=0x7f75f80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 -- 172.21.15.122:0/2499838162 shutdown_connections 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.310+0000 7f75fed40640 1 -- 172.21.15.122:0/2499838162 wait complete. 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 Processor -- start 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 -- start start 2024-04-02T13:34:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f8102580 0x7f75f8197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 0x7f75f819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810d780 con 0x7f75f8104eb0 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810d600 con 0x7f75f81077e0 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fed40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75f810d900 con 0x7f75f8102580 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44286/0 (socket says 172.21.15.122:44286) 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 -- 172.21.15.122:0/1120537912 learned_addr learned my addr 172.21.15.122:0/1120537912 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 0x7f75f819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 -- 172.21.15.122:0/1120537912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f8102580 msgr2=0x7f75f8197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fcadc640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f8102580 0x7f75f8197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f8102580 0x7f75f8197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 -- 172.21.15.122:0/1120537912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 msgr2=0x7f75f819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 0x7f75f819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75f7fff640 1 -- 172.21.15.122:0/1120537912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f819faf0 con 0x7f75f8104eb0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.311+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 0x7f75f819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75f7fff640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000c00 tx=0x7f75e4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75f5ffb640 1 -- 172.21.15.122:0/1120537912 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4041070 con 0x7f75f8104eb0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f819fd80 con 0x7f75f8104eb0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75f5ffb640 1 -- 172.21.15.122:0/1120537912 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e40044d0 con 0x7f75f8104eb0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75f5ffb640 1 -- 172.21.15.122:0/1120537912 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4034440 con 0x7f75f8104eb0 2024-04-02T13:34:35.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.312+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f81a0210 con 0x7f75f8104eb0 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.314+0000 7f75f5ffb640 1 -- 172.21.15.122:0/1120537912 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f75e4044070 con 0x7f75f8104eb0 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 Processor -- start 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 -- start start 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 0x55990be41ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55990bd9e060 con 0x55990be3a4d0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55990bd46650 con 0x55990be3f640 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f7601a06740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55990bd9a010 con 0x55990be39ae0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44300/0 (socket says 172.21.15.122:44300) 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 -- 172.21.15.122:0/136732953 learned_addr learned my addr 172.21.15.122:0/136732953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75f5ffb640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75f7fff640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 0x55990be41ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 -- 172.21.15.122:0/136732953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 msgr2=0x55990be39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 -- 172.21.15.122:0/136732953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 msgr2=0x55990be41ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 0x55990be41ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75fcadc640 1 -- 172.21.15.122:0/136732953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990bab22d0 con 0x55990be3a4d0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75f5ffb640 1 -- 172.21.15.122:0/1120537912 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f75e40f2400 con 0x7f75f8104eb0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.315+0000 7f75f7fff640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 0x55990be41ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f75fcadc640 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009be0 tx=0x7f75ec00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a168b17abb17afaa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 0x7f75e80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55990be3ad30 tx=0x7f75f0014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f75e2ffd640 1 -- 172.21.15.122:0/136732953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec016070 con 0x55990be3a4d0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f75e2ffd640 1 -- 172.21.15.122:0/136732953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec005500 con 0x55990be3a4d0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f75e2ffd640 1 -- 172.21.15.122:0/136732953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec011670 con 0x55990be3a4d0 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 -- 172.21.15.122:0/136732953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 msgr2=0x55990be3efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009be0 tx=0x7f75ec00e6c0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 -- 172.21.15.122:0/136732953 shutdown_connections 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3f640 0x55990be41ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3a4d0 0x55990be3efd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 --2- 172.21.15.122:0/136732953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be39f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 -- 172.21.15.122:0/136732953 >> 172.21.15.122:0/136732953 conn(0x55990be35050 msgr2=0x55990be35450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 -- 172.21.15.122:0/136732953 shutdown_connections 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.316+0000 7f7601a06740 1 -- 172.21.15.122:0/136732953 wait complete. 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 -- start start 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 Processor -- start 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 0x55990be44030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 0x55990be44570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55990be425f0 con 0x55990be3f640 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55990bd59a10 con 0x55990be39ae0 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55990be42770 con 0x55990be3a4d0 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f7601a06740 10 client.0 _ll_register_callbacks cb 0x55990bd960b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f75fd2dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f75fd2dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44310/0 (socket says 172.21.15.122:44310) 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1063074609 learned_addr learned my addr 172.21.15.122:0/1063074609 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f75fcadc640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 0x55990be44030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.317+0000 7f75f7fff640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 0x55990be44570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 msgr2=0x55990be44570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 0x55990be44570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1063074609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 msgr2=0x55990be44030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 0x55990be44030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1063074609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990bb7c6b0 con 0x55990be3f640 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fcadc640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 0x55990be44030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75f7fff640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 0x55990be44570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75f0014010 tx=0x7f75f0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f55157cb898a5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f002a060 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f75f001adb0 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75ec005500 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75f0031050 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f7601a06740 10 client.15234 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55990be4e8f0 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.318+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75ec005500 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f75f0024070 con 0x55990be3f640 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 20 client.15234 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990be52460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 4 client.15234 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55990bd45cf0 con 0x55990be50ff0 2024-04-02T13:34:35.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f75fcadc640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990be52460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f75d7fff640 10 client.15234 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.319+0000 7f75fcadc640 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990be52460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=346314fcd862412b server_cookie=5c387d99109b4011 in_seq=0 out_seq=0 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f75d7fff640 1 -- 172.21.15.122:0/1063074609 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55990bd45cf0 con 0x55990be50ff0 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f75d7fff640 10 client.15234 handle_command_reply: tid=0 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 client.15234 shutdown 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 msgr2=0x55990be4b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f75f0014010 tx=0x7f75f0009500 comp rx=0 tx=0).stop 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 shutdown_connections 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990be52460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be4b540 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f75d7fff640 0 client.15234 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3a4d0 0x55990be44570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 --2- 172.21.15.122:0/1063074609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be39ae0 0x55990be44030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.320+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 >> 172.21.15.122:0/1063074609 conn(0x55990be35050 msgr2=0x55990be34190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.321+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 shutdown_connections 2024-04-02T13:34:35.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.321+0000 7f7601a06740 1 -- 172.21.15.122:0/1063074609 wait complete. 2024-04-02T13:34:35.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.321+0000 7f7601a06740 20 client.15234 trim_cache size 0 max 16384 2024-04-02T13:34:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.338+0000 7f7601a06740 1 Processor -- start 2024-04-02T13:34:35.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.338+0000 7f7601a06740 1 -- start start 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be3af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990be3ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55990be507a0 con 0x55990be3f640 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55990be50e70 con 0x55990be3a4d0 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f7601a06740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55990be51540 con 0x55990be39ae0 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44314/0 (socket says 172.21.15.122:44314) 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1492927738 learned_addr learned my addr 172.21.15.122:0/1492927738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1492927738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 msgr2=0x55990be3af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be3af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1492927738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 msgr2=0x55990be3ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990be3ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 -- 172.21.15.122:0/1492927738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55990be3ad60 con 0x55990be3f640 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75fd2dd640 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009c40 tx=0x7f75f000ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec21842c6856beb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75e27fc640 1 -- 172.21.15.122:0/1492927738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f003a070 con 0x55990be3f640 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75e27fc640 1 -- 172.21.15.122:0/1492927738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f0024360 con 0x55990be3f640 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.339+0000 7f75e27fc640 1 -- 172.21.15.122:0/1492927738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f00358b0 con 0x55990be3f640 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- 172.21.15.122:0/1492927738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 msgr2=0x55990be3bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009c40 tx=0x7f75f000ff40 comp rx=0 tx=0).stop 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- 172.21.15.122:0/1492927738 shutdown_connections 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be3f640 0x55990be3bfa0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990be3ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 --2- 172.21.15.122:0/1492927738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be39ae0 0x55990be3af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- 172.21.15.122:0/1492927738 >> 172.21.15.122:0/1492927738 conn(0x55990be35050 msgr2=0x55990be35430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- 172.21.15.122:0/1492927738 shutdown_connections 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- 172.21.15.122:0/1492927738 wait complete. 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 -- start start 2024-04-02T13:34:35.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.340+0000 7f7601a06740 1 Processor -- start 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990beb76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3f640 0x55990beb9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55990be522e0 con 0x55990be39ae0 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55990be51c10 con 0x55990be3a4d0 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55990be529b0 con 0x55990be3f640 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 10 client.0 _ll_register_callbacks cb 0x7f75f0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44316/0 (socket says 172.21.15.122:44316) 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 -- 172.21.15.122:0/373090085 learned_addr learned my addr 172.21.15.122:0/373090085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 -- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3f640 msgr2=0x55990beb9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3f640 0x55990beb9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 -- 172.21.15.122:0/373090085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 msgr2=0x55990beb76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990beb76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 -- 172.21.15.122:0/373090085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55990beba500 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75fcadc640 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55990be3af40 tx=0x7f75ec009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0869fcf2ace2b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec016040 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f75ec029450 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f0024360 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f75ec034050 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 10 client.15240 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.341+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55990bebb8a0 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75f0024360 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f75ec00f040 con 0x55990be39ae0 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 10 client.15240 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:34:35.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 20 client.15240 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990bebf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 4 client.15240 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "status"}) -- 0x55990bdab5a0 con 0x55990be50ff0 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.342+0000 7f75f7fff640 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990bebf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f75d7fff640 10 client.15240 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f75f7fff640 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990bebf0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c64a60fb6123d570 server_cookie=801e264c765244ed in_seq=0 out_seq=0 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f75d7fff640 1 -- 172.21.15.122:0/373090085 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55990bdab5a0 con 0x55990be50ff0 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f75d7fff640 10 client.15240 handle_command_reply: tid=0 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 client.15240 shutdown 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 msgr2=0x55990beb5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55990be3af40 tx=0x7f75ec009d50 comp rx=0 tx=0).stop 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 shutdown_connections 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55990be3f640 0x55990beb9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55990be3a4d0 0x55990beb76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55990be50ff0 0x55990bebf0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 --2- 172.21.15.122:0/373090085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55990be39ae0 0x55990beb5180 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f75d7fff640 0 client.15240 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 >> 172.21.15.122:0/373090085 conn(0x55990be35050 msgr2=0x55990be3af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 shutdown_connections 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 1 -- 172.21.15.122:0/373090085 wait complete. 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.343+0000 7f7601a06740 20 client.15240 trim_cache size 0 max 16384 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 3, 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24460, 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:34:35.352 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 84.711092839000003, 2024-04-02T13:34:35.353 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:34:35.353 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 92, 2024-04-02T13:34:35.353 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:34:35.353 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 123.655321849 2024-04-02T13:34:35.353 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 msgr2=0x7f75e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 0x7f75e80a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55990be3ad30 tx=0x7f75f0014040 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 msgr2=0x7f75f8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f75e4000c00 tx=0x7f75e4005d50 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 shutdown_connections 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f75e80a3bb0 0x7f75e80a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75f81077e0 0x7f75f819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75f8104eb0 0x7f75f8198460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 --2- 172.21.15.122:0/1120537912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f8102580 0x7f75f8197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 >> 172.21.15.122:0/1120537912 conn(0x7f75f80fbf90 msgr2=0x7f75f81030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 shutdown_connections 2024-04-02T13:34:35.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.344+0000 7f75fed40640 1 -- 172.21.15.122:0/1120537912 wait complete. 2024-04-02T13:34:35.357 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:34:35.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 -- 172.21.15.122:0/1898867567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 msgr2=0x7fd564100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 --2- 172.21.15.122:0/1898867567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564100e20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd554009900 tx=0x7fd55402bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 -- 172.21.15.122:0/1898867567 shutdown_connections 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 --2- 172.21.15.122:0/1898867567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5641025b0 0x7fd564108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 --2- 172.21.15.122:0/1898867567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd564101bf0 0x7fd564102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 --2- 172.21.15.122:0/1898867567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564100e20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 -- 172.21.15.122:0/1898867567 >> 172.21.15.122:0/1898867567 conn(0x7fd5640fbf90 msgr2=0x7fd5640fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 -- 172.21.15.122:0/1898867567 shutdown_connections 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.457+0000 7fd569455640 1 -- 172.21.15.122:0/1898867567 wait complete. 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 Processor -- start 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 -- start start 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 0x7fd564198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 0x7fd56419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410c670 con 0x7fd564100a00 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410c4f0 con 0x7fd564101bf0 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd569455640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd56410c7f0 con 0x7fd5641025b0 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44334/0 (socket says 172.21.15.122:44334) 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 -- 172.21.15.122:0/1798648106 learned_addr learned my addr 172.21.15.122:0/1798648106 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd5637fe640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 0x7fd56419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 -- 172.21.15.122:0/1798648106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 msgr2=0x7fd56419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 0x7fd564198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 0x7fd56419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 -- 172.21.15.122:0/1798648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 msgr2=0x7fd564198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 0x7fd564198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 -- 172.21.15.122:0/1798648106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56419fb10 con 0x7fd564100a00 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd5637fe640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 0x7fd56419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 0x7fd564198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.458+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.459+0000 7fd54bfff640 1 -- 172.21.15.122:0/1798648106 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554041070 con 0x7fd564100a00 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.459+0000 7fd54bfff640 1 -- 172.21.15.122:0/1798648106 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5540044d0 con 0x7fd564100a00 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.459+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56419fda0 con 0x7fd564100a00 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.459+0000 7fd54bfff640 1 -- 172.21.15.122:0/1798648106 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554034440 con 0x7fd564100a00 2024-04-02T13:34:35.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.459+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a0220 con 0x7fd564100a00 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd54bfff640 1 -- 172.21.15.122:0/1798648106 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd554044070 con 0x7fd564100a00 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 Processor -- start 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 -- start start 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289df2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289df7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f289d57060 con 0x55f289df8640 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f289cff650 con 0x55f289df34d0 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd56c037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f289d53010 con 0x55f289df2ae0 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd54bfff640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 0x7fd54c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd5627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd5627fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44336/0 (socket says 172.21.15.122:44336) 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd5627fc640 1 -- 172.21.15.122:0/2609639326 learned_addr learned my addr 172.21.15.122:0/2609639326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289df7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.461+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289df2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5637fe640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 0x7fd54c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd54bfff640 1 -- 172.21.15.122:0/1798648106 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd5540f22f0 con 0x7fd564100a00 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 -- 172.21.15.122:0/2609639326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 msgr2=0x55f289df2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289df2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 -- 172.21.15.122:0/2609639326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 msgr2=0x55f289df7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289df7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 -- 172.21.15.122:0/2609639326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f289a6b2d0 con 0x55f289df8640 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5627fc640 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd558009c00 tx=0x7fd558010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78803fa075a349be server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5637fe640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 0x7fd54c0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55f289df3d30 tx=0x7fd55c004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5497fa640 1 -- 172.21.15.122:0/2609639326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558004030 con 0x55f289df8640 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5497fa640 1 -- 172.21.15.122:0/2609639326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5580041d0 con 0x55f289df8640 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.462+0000 7fd5497fa640 1 -- 172.21.15.122:0/2609639326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580126b0 con 0x55f289df8640 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- 172.21.15.122:0/2609639326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 msgr2=0x55f289dfaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd558009c00 tx=0x7fd558010740 comp rx=0 tx=0).stop 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- 172.21.15.122:0/2609639326 shutdown_connections 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289dfaad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289df7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- 172.21.15.122:0/2609639326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289df2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- 172.21.15.122:0/2609639326 >> 172.21.15.122:0/2609639326 conn(0x55f289dee050 msgr2=0x55f289dee450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- 172.21.15.122:0/2609639326 shutdown_connections 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- 172.21.15.122:0/2609639326 wait complete. 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- start start 2024-04-02T13:34:35.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 Processor -- start 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289dfd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289dfd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f289dfb5f0 con 0x55f289df8640 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f289d12a10 con 0x55f289df34d0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f289dfb770 con 0x55f289df2ae0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd56c037740 10 client.0 _ll_register_callbacks cb 0x55f289d4f0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd5637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44348/0 (socket says 172.21.15.122:44348) 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.463+0000 7fd5637fe640 1 -- 172.21.15.122:0/2531896386 learned_addr learned my addr 172.21.15.122:0/2531896386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289dfd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5627fc640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289dfd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 -- 172.21.15.122:0/2531896386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 msgr2=0x55f289dfd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289dfd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 -- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 msgr2=0x55f289dfd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289dfd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 -- 172.21.15.122:0/2531896386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f289b356b0 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009d70 tx=0x7fd55c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7324f66132ede2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c025070 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd55c004450 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5580041d0 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd55c031050 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd56c037740 10 client.15252 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f289e07910 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.464+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5580041d0 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd55c00f040 con 0x55f289df8640 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 10 client.15252 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:34:35.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 20 client.15252 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e0a030 0x55f289e0b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 4 client.15252 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f289cfecf0 con 0x55f289e0a030 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e0a030 0x55f289e0b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd546ff5640 10 client.15252 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.465+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e0a030 0x55f289e0b4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30a26f1f2dd1b125 server_cookie=e09bff6cc0d01364 in_seq=0 out_seq=0 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd546ff5640 1 -- 172.21.15.122:0/2531896386 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f289cfecf0 con 0x55f289e0a030 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd546ff5640 10 client.15252 handle_command_reply: tid=0 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 client.15252 shutdown 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 msgr2=0x55f289e04530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009d70 tx=0x7fd55c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 shutdown_connections 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df8640 0x55f289e04530 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e0a030 0x55f289e0b4a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289dfd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 --2- 172.21.15.122:0/2531896386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df2ae0 0x55f289dfd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd546ff5640 0 client.15252 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 >> 172.21.15.122:0/2531896386 conn(0x55f289dee050 msgr2=0x55f289ded190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 shutdown_connections 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 1 -- 172.21.15.122:0/2531896386 wait complete. 2024-04-02T13:34:35.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.466+0000 7fd56c037740 20 client.15252 trim_cache size 0 max 16384 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 Processor -- start 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 -- start start 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289dece40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f289e097e0 con 0x55f289df2ae0 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f289e09eb0 con 0x55f289df34d0 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd56c037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f289e0a580 con 0x55f289df8640 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44354/0 (socket says 172.21.15.122:44354) 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 -- 172.21.15.122:0/2665358459 learned_addr learned my addr 172.21.15.122:0/2665358459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289dece40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 -- 172.21.15.122:0/2665358459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 msgr2=0x55f289dece40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289dece40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd5627fc640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 -- 172.21.15.122:0/2665358459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 msgr2=0x55f289e044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 -- 172.21.15.122:0/2665358459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f289e0aa30 con 0x55f289df2ae0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd5637fe640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289dece40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd562ffd640 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd55402c7b0 tx=0x7fd554090db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ee3405457da984 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd548ff9640 1 -- 172.21.15.122:0/2665358459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55404b040 con 0x55f289df2ae0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd548ff9640 1 -- 172.21.15.122:0/2665358459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd554088dd0 con 0x55f289df2ae0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.485+0000 7fd548ff9640 1 -- 172.21.15.122:0/2665358459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5540c6ee0 con 0x55f289df2ae0 2024-04-02T13:34:35.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- 172.21.15.122:0/2665358459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 msgr2=0x55f289dec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd55402c7b0 tx=0x7fd554090db0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- 172.21.15.122:0/2665358459 shutdown_connections 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289dece40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 --2- 172.21.15.122:0/2665358459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dec900 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- 172.21.15.122:0/2665358459 >> 172.21.15.122:0/2665358459 conn(0x55f289dee050 msgr2=0x55f289df49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- 172.21.15.122:0/2665358459 shutdown_connections 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- 172.21.15.122:0/2665358459 wait complete. 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 -- start start 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.486+0000 7fd56c037740 1 Processor -- start 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e00290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289e03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f289e0b320 con 0x55f289df2ae0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f289e0ac50 con 0x55f289df34d0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f289e0b9f0 con 0x55f289df8640 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 10 client.0 _ll_register_callbacks cb 0x7fd55c013740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44360/0 (socket says 172.21.15.122:44360) 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 -- 172.21.15.122:0/1908500949 learned_addr learned my addr 172.21.15.122:0/1908500949 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e00290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd5637fe640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289e03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 -- 172.21.15.122:0/1908500949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 msgr2=0x55f289e03260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289e03260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 -- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 msgr2=0x55f289e00290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e00290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 -- 172.21.15.122:0/1908500949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f289e03850 con 0x55f289df2ae0 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e00290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd562ffd640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd5540036a0 tx=0x7fd554009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4ea521993fb32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd546ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55404b040 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd5540ba8e0 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd554088dd0 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5540be970 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 10 client.15258 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.487+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f289e700a0 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd554088dd0 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd5540bd6c0 con 0x55f289df2ae0 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 10 client.15258 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 10 client.15258 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 20 client.15258 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e74a30 0x55f289e73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 4 client.15258 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "status"}) -- 0x55f289d645a0 con 0x55f289e74a30 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e74a30 0x55f289e73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd5627fc640 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e74a30 0x55f289e73740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8098ef336ed08eb server_cookie=78116d8aa0198e96 in_seq=0 out_seq=0 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.488+0000 7fd546ff5640 10 client.15258 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd546ff5640 1 -- 172.21.15.122:0/1908500949 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f289d645a0 con 0x55f289e74a30 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd546ff5640 10 client.15258 handle_command_reply: tid=0 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 client.15258 shutdown 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 msgr2=0x55f289dffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd5540036a0 tx=0x7fd554009680 comp rx=0 tx=0).stop 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 shutdown_connections 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f289df8640 0x55f289e03260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55f289e74a30 0x55f289e73740 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f289df34d0 0x55f289e00290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 --2- 172.21.15.122:0/1908500949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f289df2ae0 0x55f289dffd50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd546ff5640 0 client.15258 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 >> 172.21.15.122:0/1908500949 conn(0x55f289dee050 msgr2=0x55f289dec900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 shutdown_connections 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 1 -- 172.21.15.122:0/1908500949 wait complete. 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd56c037740 20 client.15258 trim_cache size 0 max 16384 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24485, 2024-04-02T13:34:35.499 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 85.863521997999996, 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 92, 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 120.93883150800001 2024-04-02T13:34:35.500 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 msgr2=0x7fd54c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 0x7fd54c0a60f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55f289df3d30 tx=0x7fd55c004a40 comp rx=0 tx=0).stop 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 msgr2=0x7fd564197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd554000c00 tx=0x7fd554005d50 comp rx=0 tx=0).stop 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 shutdown_connections 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5641025b0 0x7fd56419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd54c0a3c30 0x7fd54c0a60f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd564101bf0 0x7fd564198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 --2- 172.21.15.122:0/1798648106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd564100a00 0x7fd564197ef0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.489+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 >> 172.21.15.122:0/1798648106 conn(0x7fd5640fbf90 msgr2=0x7fd564068cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.490+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 shutdown_connections 2024-04-02T13:34:35.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.490+0000 7fd569455640 1 -- 172.21.15.122:0/1798648106 wait complete. 2024-04-02T13:34:35.503 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T13:34:35.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- 172.21.15.122:0/514509746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 msgr2=0x7fd414102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/514509746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414102070 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd400009900 tx=0x7fd40002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:35.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- 172.21.15.122:0/514509746 shutdown_connections 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/514509746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4141025b0 0x7fd414108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/514509746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414102070 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/514509746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd414100a00 0x7fd414100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- 172.21.15.122:0/514509746 >> 172.21.15.122:0/514509746 conn(0x7fd4140fbf90 msgr2=0x7fd4140fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- 172.21.15.122:0/514509746 shutdown_connections 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- 172.21.15.122:0/514509746 wait complete. 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 Processor -- start 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 -- start start 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd414100a00 0x7fd414197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.605+0000 7fd41ad1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd41ad1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4141025b0 0x7fd41419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd41ad1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410c6f0 con 0x7fd414101bf0 2024-04-02T13:34:35.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd41ad1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410c570 con 0x7fd414100a00 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd41ad1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd41410c870 con 0x7fd4141025b0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44374/0 (socket says 172.21.15.122:44374) 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 -- 172.21.15.122:0/4038467802 learned_addr learned my addr 172.21.15.122:0/4038467802 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd418ab8640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd414100a00 0x7fd414197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd4192b9640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4141025b0 0x7fd41419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 -- 172.21.15.122:0/4038467802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4141025b0 msgr2=0x7fd41419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4141025b0 0x7fd41419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 -- 172.21.15.122:0/4038467802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd414100a00 msgr2=0x7fd414197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd414100a00 0x7fd414197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 -- 172.21.15.122:0/4038467802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41419fb30 con 0x7fd414101bf0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd413fff640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd400009230 tx=0x7fd400005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd411ffb640 1 -- 172.21.15.122:0/4038467802 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400041070 con 0x7fd414101bf0 2024-04-02T13:34:35.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41419fdc0 con 0x7fd414101bf0 2024-04-02T13:34:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd411ffb640 1 -- 172.21.15.122:0/4038467802 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4000054e0 con 0x7fd414101bf0 2024-04-02T13:34:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.606+0000 7fd411ffb640 1 -- 172.21.15.122:0/4038467802 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd400034440 con 0x7fd414101bf0 2024-04-02T13:34:35.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.607+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4141a0220 con 0x7fd414101bf0 2024-04-02T13:34:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd411ffb640 1 -- 172.21.15.122:0/4038467802 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd400044070 con 0x7fd414101bf0 2024-04-02T13:34:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 Processor -- start 2024-04-02T13:34:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 -- start start 2024-04-02T13:34:35.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a74d0 0x5621ba9abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621ba90b060 con 0x5621ba9a6ae0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621ba8b3650 con 0x5621ba9a74d0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621ba907010 con 0x5621ba9ac640 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd411ffb640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 0x7fd4040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44382/0 (socket says 172.21.15.122:44382) 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 -- 172.21.15.122:0/1139580443 learned_addr learned my addr 172.21.15.122:0/1139580443 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd418ab8640 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a74d0 0x5621ba9abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 0x7fd4040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd411ffb640 1 -- 172.21.15.122:0/4038467802 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd4000f27c0 con 0x7fd414101bf0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd413fff640 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 -- 172.21.15.122:0/1139580443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 msgr2=0x5621ba9aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 -- 172.21.15.122:0/1139580443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a74d0 msgr2=0x5621ba9abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a74d0 0x5621ba9abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.609+0000 7fd4192b9640 1 -- 172.21.15.122:0/1139580443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ba61f2d0 con 0x5621ba9a6ae0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0149e0 tx=0x7fd40c014ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde7a8bc10525e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd3feffd640 1 -- 172.21.15.122:0/1139580443 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c029070 con 0x5621ba9a6ae0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd3feffd640 1 -- 172.21.15.122:0/1139580443 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c01cc30 con 0x5621ba9a6ae0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd3feffd640 1 -- 172.21.15.122:0/1139580443 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c013af0 con 0x5621ba9a6ae0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1139580443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 msgr2=0x5621ba9a6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd4192b9640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 0x7fd4040a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd40c026000 tx=0x7fd40c022000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.610+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd40c0149e0 tx=0x7fd40c014ea0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1139580443 shutdown_connections 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a74d0 0x5621ba9abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1139580443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a6ae0 0x5621ba9a6f00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1139580443 >> 172.21.15.122:0/1139580443 conn(0x5621ba9a2050 msgr2=0x5621ba9a2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1139580443 shutdown_connections 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1139580443 wait complete. 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- start start 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 Processor -- start 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9ac640 0x5621ba9b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9af5f0 con 0x5621ba9a74d0 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621ba8c6a10 con 0x5621ba9ac640 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9af770 con 0x5621ba9a6ae0 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd41d9e2740 10 client.0 _ll_register_callbacks cb 0x5621ba9030b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44392/0 (socket says 172.21.15.122:44392) 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd413fff640 1 -- 172.21.15.122:0/476095387 learned_addr learned my addr 172.21.15.122:0/476095387 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.611+0000 7fd418ab8640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd4192b9640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9ac640 0x5621ba9b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 -- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a6ae0 msgr2=0x5621ba9b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 -- 172.21.15.122:0/476095387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9ac640 msgr2=0x5621ba9b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9ac640 0x5621ba9b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 -- 172.21.15.122:0/476095387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ba6e96b0 con 0x5621ba9a74d0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd413fff640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd400004ec0 tx=0x7fd40003cfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb3feef87810b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd3f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40004b040 con 0x5621ba9a74d0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd4000a90b0 con 0x5621ba9a74d0 2024-04-02T13:34:35.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40c01cc30 con 0x5621ba9a74d0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4000bee40 con 0x5621ba9a74d0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd41d9e2740 10 client.15270 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5621ba9bb920 con 0x5621ba9a74d0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.612+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c01cc30 con 0x5621ba9a74d0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd400003680 con 0x5621ba9a74d0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 20 client.15270 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621ba9be050 0x5621ba9bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 4 client.15270 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621ba8b2cf0 con 0x5621ba9be050 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.613+0000 7fd418ab8640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621ba9be050 0x5621ba9bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.614+0000 7fd418ab8640 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621ba9be050 0x5621ba9bf4c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1a8a4ab2705ddda server_cookie=b7c62f0009595fb2 in_seq=0 out_seq=0 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.614+0000 7fd3f3fff640 10 client.15270 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:34:35.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.614+0000 7fd3f3fff640 1 -- 172.21.15.122:0/476095387 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621ba8b2cf0 con 0x5621ba9be050 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.614+0000 7fd3f3fff640 10 client.15270 handle_command_reply: tid=0 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.614+0000 7fd41d9e2740 1 client.15270 shutdown 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 msgr2=0x5621ba9b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd400004ec0 tx=0x7fd40003cfe0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 shutdown_connections 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621ba9be050 0x5621ba9bf4c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9ac640 0x5621ba9b85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd3f3fff640 0 client.15270 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b1570 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/476095387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 >> 172.21.15.122:0/476095387 conn(0x5621ba9a2050 msgr2=0x5621ba9a1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 shutdown_connections 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 1 -- 172.21.15.122:0/476095387 wait complete. 2024-04-02T13:34:35.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.615+0000 7fd41d9e2740 20 client.15270 trim_cache size 0 max 16384 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.632+0000 7fd41d9e2740 1 Processor -- start 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.632+0000 7fd41d9e2740 1 -- start start 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 0x5621ba9bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9bd800 con 0x5621ba9ac640 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9bded0 con 0x5621ba9a6ae0 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9be5a0 con 0x5621ba9a74d0 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44400/0 (socket says 172.21.15.122:44400) 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 -- 172.21.15.122:0/1050669561 learned_addr learned my addr 172.21.15.122:0/1050669561 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd413fff640 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 0x5621ba9bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 -- 172.21.15.122:0/1050669561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 msgr2=0x5621ba9bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 0x5621ba9bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 -- 172.21.15.122:0/1050669561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 msgr2=0x5621ba9bcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 -- 172.21.15.122:0/1050669561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ba9b21b0 con 0x5621ba9ac640 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd413fff640 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 0x5621ba9bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd40c005c40 tx=0x7fd40c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04be477abdce940 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd3fe7fc640 1 -- 172.21.15.122:0/1050669561 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c029050 con 0x5621ba9ac640 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd3fe7fc640 1 -- 172.21.15.122:0/1050669561 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c01cc30 con 0x5621ba9ac640 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.633+0000 7fd3fe7fc640 1 -- 172.21.15.122:0/1050669561 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c013400 con 0x5621ba9ac640 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1050669561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 msgr2=0x5621ba9a0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd40c005c40 tx=0x7fd40c009b80 comp rx=0 tx=0).stop 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1050669561 shutdown_connections 2024-04-02T13:34:35.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9ac640 0x5621ba9a0900 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9a74d0 0x5621ba9bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1050669561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9bcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1050669561 >> 172.21.15.122:0/1050669561 conn(0x5621ba9a2050 msgr2=0x5621ba9a2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1050669561 shutdown_connections 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1050669561 wait complete. 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 -- start start 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.634+0000 7fd41d9e2740 1 Processor -- start 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9bf340 con 0x5621ba9a74d0 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9bec70 con 0x5621ba9a6ae0 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621ba9bfa10 con 0x5621ba9ac640 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 10 client.0 _ll_register_callbacks cb 0x7fd4000baa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44408/0 (socket says 172.21.15.122:44408) 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 -- 172.21.15.122:0/1303228363 learned_addr learned my addr 172.21.15.122:0/1303228363 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd4192b9640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd418ab8640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 -- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 msgr2=0x5621ba9b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 -- 172.21.15.122:0/1303228363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 msgr2=0x5621ba9b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 -- 172.21.15.122:0/1303228363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ba9b74f0 con 0x5621ba9a74d0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd413fff640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4000bc9f0 tx=0x7fd400004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9655b9375291564b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd3f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40004b070 con 0x5621ba9a74d0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd4000a90b0 con 0x5621ba9a74d0 2024-04-02T13:34:35.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40c01cc30 con 0x5621ba9a74d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd4000a2050 con 0x5621ba9a74d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 10 client.15276 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.635+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5621baa24780 con 0x5621ba9a74d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40c01cc30 con 0x5621ba9a74d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fd4000dd3d0 con 0x5621ba9a74d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 10 client.15276 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 10 client.15276 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 20 client.15276 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621baa291d0 0x5621baa27ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 4 client.15276 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "status"}) -- 0x5621ba9185a0 con 0x5621baa291d0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd418ab8640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621baa291d0 0x5621baa27ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd3f3fff640 10 client.15276 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:34:35.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.636+0000 7fd418ab8640 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621baa291d0 0x5621baa27ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8efe164a5adcef41 server_cookie=554ce3268077bab9 in_seq=0 out_seq=0 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd3f3fff640 1 -- 172.21.15.122:0/1303228363 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5621ba9185a0 con 0x5621baa291d0 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd3f3fff640 10 client.15276 handle_command_reply: tid=0 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 client.15276 shutdown 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 msgr2=0x5621ba9b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd4000bc9f0 tx=0x7fd400004550 comp rx=0 tx=0).stop 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 shutdown_connections 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5621baa291d0 0x5621baa27ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621ba9ac640 0x5621ba9b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd3f3fff640 0 client.15276 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621ba9a74d0 0x5621ba9b3ee0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 --2- 172.21.15.122:0/1303228363 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621ba9a6ae0 0x5621ba9b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 >> 172.21.15.122:0/1303228363 conn(0x5621ba9a2050 msgr2=0x5621ba9a0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 shutdown_connections 2024-04-02T13:34:35.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 1 -- 172.21.15.122:0/1303228363 wait complete. 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.637+0000 7fd41d9e2740 20 client.15276 trim_cache size 0 max 16384 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24487, 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 87.020855280000006, 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:34:35.647 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 92, 2024-04-02T13:34:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:34:35.648 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 118.01798788799999 2024-04-02T13:34:35.648 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:34:35.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 msgr2=0x7fd4040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 0x7fd4040a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd40c026000 tx=0x7fd40c022000 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 msgr2=0x7fd414198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd400009230 tx=0x7fd400005dd0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 shutdown_connections 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4141025b0 0x7fd41419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd414101bf0 0x7fd414198450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd4040a3bb0 0x7fd4040a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 --2- 172.21.15.122:0/4038467802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd414100a00 0x7fd414197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 >> 172.21.15.122:0/4038467802 conn(0x7fd4140fbf90 msgr2=0x7fd4140fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 shutdown_connections 2024-04-02T13:34:35.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.638+0000 7fd41ad1c640 1 -- 172.21.15.122:0/4038467802 wait complete. 2024-04-02T13:34:35.650 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:35 smithi122 ceph-mon[32114]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:34:35.650 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:35 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/119650756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.650 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:35 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3783801838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:34:35.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 -- 172.21.15.122:0/1524912168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae80ffb90 msgr2=0x7fdae8100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 --2- 172.21.15.122:0/1524912168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae80ffb90 0x7fdae8100010 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009900 tx=0x7fdad402bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:35.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 -- 172.21.15.122:0/1524912168 shutdown_connections 2024-04-02T13:34:35.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 --2- 172.21.15.122:0/1524912168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 0x7fdae810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 --2- 172.21.15.122:0/1524912168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae80ffb90 0x7fdae8100010 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 --2- 172.21.15.122:0/1524912168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae8103120 0x7fdae8103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 -- 172.21.15.122:0/1524912168 >> 172.21.15.122:0/1524912168 conn(0x7fdae80fb730 msgr2=0x7fdae80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 -- 172.21.15.122:0/1524912168 shutdown_connections 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.751+0000 7fdaeedba640 1 -- 172.21.15.122:0/1524912168 wait complete. 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 Processor -- start 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 -- start start 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 0x7fdae8197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 0x7fdae8198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810d780 con 0x7fdae8103120 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810d600 con 0x7fdae80ffb90 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaeedba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdae810d900 con 0x7fdae8100550 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44432/0 (socket says 172.21.15.122:44432) 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 -- 172.21.15.122:0/2096002468 learned_addr learned my addr 172.21.15.122:0/2096002468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaecb56640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 0x7fdae8197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdae7fff640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 0x7fdae8198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 -- 172.21.15.122:0/2096002468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 msgr2=0x7fdae8198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 0x7fdae8198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 -- 172.21.15.122:0/2096002468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 msgr2=0x7fdae8197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 0x7fdae8197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 -- 172.21.15.122:0/2096002468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae819fa50 con 0x7fdae8103120 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaecb56640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 0x7fdae8197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.752+0000 7fdaed357640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdae000cce0 tx=0x7fdae000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.753+0000 7fdae5ffb640 1 -- 172.21.15.122:0/2096002468 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae000f450 con 0x7fdae8103120 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.753+0000 7fdae5ffb640 1 -- 172.21.15.122:0/2096002468 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae000fad0 con 0x7fdae8103120 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.753+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae819fd40 con 0x7fdae8103120 2024-04-02T13:34:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.753+0000 7fdae5ffb640 1 -- 172.21.15.122:0/2096002468 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0018c10 con 0x7fdae8103120 2024-04-02T13:34:35.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.753+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae81a01e0 con 0x7fdae8103120 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdae5ffb640 1 -- 172.21.15.122:0/2096002468 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fdae000f5f0 con 0x7fdae8103120 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 Processor -- start 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 -- start start 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdae5ffb640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 0x7fdad80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e378f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e380ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e99e2dd060 con 0x55e99e3794d0 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e99e285650 con 0x55e99e37e640 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.755+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e99e2d9010 con 0x55e99e378ae0 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 0x7fdad80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae5ffb640 1 -- 172.21.15.122:0/2096002468 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdae00d0ee0 con 0x7fdae8103120 2024-04-02T13:34:35.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44442/0 (socket says 172.21.15.122:44442) 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 -- 172.21.15.122:0/313373582 learned_addr learned my addr 172.21.15.122:0/313373582 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdaed357640 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e380ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdaecb56640 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e378f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 -- 172.21.15.122:0/313373582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 msgr2=0x55e99e378f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e378f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 -- 172.21.15.122:0/313373582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 msgr2=0x55e99e380ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e380ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 -- 172.21.15.122:0/313373582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99dff12d0 con 0x55e99e3794d0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 0x7fdad80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdad402cfd0 tx=0x7fdad40095e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdae7fff640 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdad402c170 tx=0x7fdad404a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b9067acbd04ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdad2ffd640 1 -- 172.21.15.122:0/313373582 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4041940 con 0x55e99e3794d0 2024-04-02T13:34:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdad2ffd640 1 -- 172.21.15.122:0/313373582 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad40035f0 con 0x55e99e3794d0 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.756+0000 7fdad2ffd640 1 -- 172.21.15.122:0/313373582 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad40346a0 con 0x55e99e3794d0 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- 172.21.15.122:0/313373582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 msgr2=0x55e99e37dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdad402c170 tx=0x7fdad404a000 comp rx=0 tx=0).stop 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- 172.21.15.122:0/313373582 shutdown_connections 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e380ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e37dfd0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/313373582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e378f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- 172.21.15.122:0/313373582 >> 172.21.15.122:0/313373582 conn(0x55e99e374050 msgr2=0x55e99e374450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- 172.21.15.122:0/313373582 shutdown_connections 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- 172.21.15.122:0/313373582 wait complete. 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 -- start start 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.757+0000 7fdaf19a0740 1 Processor -- start 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e99e381590 con 0x55e99e37e640 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e99e298a10 con 0x55e99e3794d0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e99e381710 con 0x55e99e378ae0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 10 client.0 _ll_register_callbacks cb 0x55e99e2d50b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44458/0 (socket says 172.21.15.122:44458) 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 -- 172.21.15.122:0/3519546052 learned_addr learned my addr 172.21.15.122:0/3519546052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaecb56640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 -- 172.21.15.122:0/3519546052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 msgr2=0x55e99e383030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 -- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e3794d0 msgr2=0x55e99e383570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 -- 172.21.15.122:0/3519546052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99e0bb6b0 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaed357640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdae009cdf0 tx=0x7fdae009cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c521346d79708ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae0010e60 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdae009b160 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad40035f0 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.758+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdae00a1210 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 10 client.15288 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e99e38d9f0 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad40035f0 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdae0010a50 con 0x55e99e37e640 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:34:35.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 20 client.15288 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e390020 0x55e99e391490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 4 client.15288 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e99e284cf0 con 0x55e99e390020 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.759+0000 7fdaecb56640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e390020 0x55e99e391490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaecb56640 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e390020 0x55e99e391490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17dbe1b5ea0fbabc server_cookie=c2f317ba3f6c6239 in_seq=0 out_seq=0 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdac7fff640 10 client.15288 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdac7fff640 1 -- 172.21.15.122:0/3519546052 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e99e284cf0 con 0x55e99e390020 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdac7fff640 10 client.15288 handle_command_reply: tid=0 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 client.15288 shutdown 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 msgr2=0x55e99e38a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdae009cdf0 tx=0x7fdae009cad0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 shutdown_connections 2024-04-02T13:34:35.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e390020 0x55e99e391490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e37e640 0x55e99e38a5b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdac7fff640 0 client.15288 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/3519546052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.760+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 >> 172.21.15.122:0/3519546052 conn(0x55e99e374050 msgr2=0x55e99e373190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.761+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 shutdown_connections 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.761+0000 7fdaf19a0740 1 -- 172.21.15.122:0/3519546052 wait complete. 2024-04-02T13:34:35.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.761+0000 7fdaf19a0740 20 client.15288 trim_cache size 0 max 16384 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 Processor -- start 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 -- start start 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 0x55e99e38a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e99e38f7d0 con 0x55e99e3794d0 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e99e38fea0 con 0x55e99e378ae0 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e99e390570 con 0x55e99e37e640 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44470/0 (socket says 172.21.15.122:44470) 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 -- 172.21.15.122:0/671045505 learned_addr learned my addr 172.21.15.122:0/671045505 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaed357640 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 0x55e99e38a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 -- 172.21.15.122:0/671045505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 msgr2=0x55e99e38a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 0x55e99e38a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 -- 172.21.15.122:0/671045505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e378ae0 msgr2=0x55e99e383030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 -- 172.21.15.122:0/671045505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e99e37dc80 con 0x55e99e3794d0 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdaed357640 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 0x55e99e38a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdae7fff640 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55e99e379f40 tx=0x7fdad4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7d0f90711fdc4e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdad27fc640 1 -- 172.21.15.122:0/671045505 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad405c070 con 0x55e99e3794d0 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdad27fc640 1 -- 172.21.15.122:0/671045505 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4041e20 con 0x55e99e3794d0 2024-04-02T13:34:35.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.779+0000 7fdad27fc640 1 -- 172.21.15.122:0/671045505 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4034440 con 0x55e99e3794d0 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- 172.21.15.122:0/671045505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 msgr2=0x55e99e383570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x55e99e379f40 tx=0x7fdad4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- 172.21.15.122:0/671045505 shutdown_connections 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e37e640 0x55e99e38a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e3794d0 0x55e99e383570 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/671045505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e378ae0 0x55e99e383030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- 172.21.15.122:0/671045505 >> 172.21.15.122:0/671045505 conn(0x55e99e374050 msgr2=0x55e99e374430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- 172.21.15.122:0/671045505 shutdown_connections 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- 172.21.15.122:0/671045505 wait complete. 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 -- start start 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.780+0000 7fdaf19a0740 1 Processor -- start 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 0x55e99e3f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e3f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e99e391310 con 0x55e99e378ae0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e99e390c40 con 0x55e99e37e640 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e99e3919e0 con 0x55e99e3794d0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 10 client.0 _ll_register_callbacks cb 0x7fdae0098a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44472/0 (socket says 172.21.15.122:44472) 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 -- 172.21.15.122:0/552866368 learned_addr learned my addr 172.21.15.122:0/552866368 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdae7fff640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 0x55e99e3f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 -- 172.21.15.122:0/552866368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 msgr2=0x55e99e3f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 0x55e99e3f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 -- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 msgr2=0x55e99e3f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e3f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 -- 172.21.15.122:0/552866368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e99e3f4d70 con 0x55e99e378ae0 2024-04-02T13:34:35.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdae7fff640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 0x55e99e3f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaecb56640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdadc005fd0 tx=0x7fdadc01bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277e2008f4da8969 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdac7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc014d20 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdadc01b330 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.781+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad4041e20 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdadc02b050 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 10 client.15294 fetch_fsmap learned FSMap version 27 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e99e3f60b0 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdad4041e20 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdadc012aa0 con 0x55e99e378ae0 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 10 client.15294 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 20 client.15294 populate_metadata read hostname 'smithi122' 2024-04-02T13:34:35.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e3fad30 0x55e99e3f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 4 client.15294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "status"}) -- 0x55e99e2ea5a0 con 0x55e99e3fad30 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.782+0000 7fdae7fff640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e3fad30 0x55e99e3f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdac7fff640 10 client.15294 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdae7fff640 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e3fad30 0x55e99e3f9a40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc28ee4b55df254 server_cookie=34d77527873bcd6b in_seq=0 out_seq=0 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdac7fff640 1 -- 172.21.15.122:0/552866368 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e99e2ea5a0 con 0x55e99e3fad30 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdac7fff640 10 client.15294 handle_command_reply: tid=0 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 client.15294 shutdown 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 msgr2=0x55e99e388560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdadc005fd0 tx=0x7fdadc01bf90 comp rx=0 tx=0).stop 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 shutdown_connections 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e99e3fad30 0x55e99e3f9a40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e99e37e640 0x55e99e3f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdac7fff640 0 client.15294 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e99e3794d0 0x55e99e3f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 --2- 172.21.15.122:0/552866368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e99e378ae0 0x55e99e388560 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 >> 172.21.15.122:0/552866368 conn(0x55e99e374050 msgr2=0x55e99e383030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 shutdown_connections 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 1 -- 172.21.15.122:0/552866368 wait complete. 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.783+0000 7fdaf19a0740 20 client.15294 trim_cache size 0 max 16384 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24512, 2024-04-02T13:34:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 105.259708147, 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 92, 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 109.41028441900001 2024-04-02T13:34:35.794 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 msgr2=0x7fdad80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 0x7fdad80a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdad402cfd0 tx=0x7fdad40095e0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 msgr2=0x7fdae819f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fdae000cce0 tx=0x7fdae000a620 comp rx=0 tx=0).stop 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 shutdown_connections 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdae8103120 0x7fdae819f3e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdae8100550 0x7fdae8198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fdad80a3bb0 0x7fdad80a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 --2- 172.21.15.122:0/2096002468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdae80ffb90 0x7fdae8197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 >> 172.21.15.122:0/2096002468 conn(0x7fdae80fb730 msgr2=0x7fdae8106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 shutdown_connections 2024-04-02T13:34:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:35.784+0000 7fdaeedba640 1 -- 172.21.15.122:0/2096002468 wait complete. 2024-04-02T13:34:35.797 INFO:tasks.cephadm:Setup complete, yielding 2024-04-02T13:34:35.797 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:34:35.807 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:34:35.807 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch status' 2024-04-02T13:34:35.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:35 smithi188 ceph-mon[36117]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:34:35.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:35 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/119650756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.919 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:35 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3783801838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:35 smithi187 ceph-mon[36064]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:34:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:35 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/119650756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:35 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3783801838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:36.044 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:37.838 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:37 smithi122 ceph-mon[32114]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 -- 172.21.15.122:0/225754392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e4102720 msgr2=0x7f58e4102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 --2- 172.21.15.122:0/225754392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e4102720 0x7f58e4102b80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f58d40099c0 tx=0x7f58d4009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 -- 172.21.15.122:0/225754392 shutdown_connections 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 --2- 172.21.15.122:0/225754392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e41030c0 0x7f58e4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 --2- 172.21.15.122:0/225754392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 0x7f58e4101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 --2- 172.21.15.122:0/225754392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e4102720 0x7f58e4102b80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 -- 172.21.15.122:0/225754392 >> 172.21.15.122:0/225754392 conn(0x7f58e40fcac0 msgr2=0x7f58e40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 -- 172.21.15.122:0/225754392 shutdown_connections 2024-04-02T13:34:37.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.838+0000 7f58ec433700 1 -- 172.21.15.122:0/225754392 wait complete. 2024-04-02T13:34:37.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 Processor -- start 2024-04-02T13:34:37.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 -- start start 2024-04-02T13:34:37.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 0x7f58e406ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:37.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e4102720 0x7f58e419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:37.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410d200 con 0x7f58e41030c0 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410d080 con 0x7f58e4102720 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ec433700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410d380 con 0x7f58e4101520 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ea9d0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ea9d0700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44498/0 (socket says 172.21.15.122:44498) 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ea9d0700 1 -- 172.21.15.122:0/652096162 learned_addr learned my addr 172.21.15.122:0/652096162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58e99ce700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e4102720 0x7f58e419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.839+0000 7f58ea1cf700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 0x7f58e406ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 -- 172.21.15.122:0/652096162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 msgr2=0x7f58e406ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 0x7f58e406ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 -- 172.21.15.122:0/652096162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e4102720 msgr2=0x7f58e419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e4102720 0x7f58e419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:37.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 -- 172.21.15.122:0/652096162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e41a0650 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ea9d0700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b970 tx=0x7f58dc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00d860 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e4197520 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58dc00e430 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc017660 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.840+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e4197a90 con 0x7f58e41030c0 2024-04-02T13:34:37.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.841+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e40692b0 con 0x7f58e41030c0 2024-04-02T13:34:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.842+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f58dc00e5e0 con 0x7f58e41030c0 2024-04-02T13:34:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.843+0000 7f58db7fe700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 0x7f58d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.843+0000 7f58ea1cf700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 0x7f58d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.843+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f58dc0d0560 con 0x7f58e41030c0 2024-04-02T13:34:37.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.843+0000 7f58ea1cf700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 0x7f58d00a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58e4102580 tx=0x7f58e000f040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:37.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:37.846+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f58dc013030 con 0x7f58e41030c0 2024-04-02T13:34:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:37 smithi187 ceph-mon[36064]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:34:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:37 smithi188 ceph-mon[36117]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-02T13:34:38.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.066+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f58e4062a30 con 0x7f58d00a3c40 2024-04-02T13:34:38.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.067+0000 7f58db7fe700 1 -- 172.21.15.122:0/652096162 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f58e4062a30 con 0x7f58d00a3c40 2024-04-02T13:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout:Backend: cephadm 2024-04-02T13:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout:Available: Yes 2024-04-02T13:34:38.071 INFO:teuthology.orchestra.run.smithi122.stdout:Paused: No 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 msgr2=0x7f58d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 0x7f58d00a6100 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f58e4102580 tx=0x7f58e000f040 comp rx=0 tx=0).stop 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 msgr2=0x7f58e419fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b970 tx=0x7f58dc00bc80 comp rx=0 tx=0).stop 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 shutdown_connections 2024-04-02T13:34:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58d00a3c40 0x7f58d00a6100 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58e41030c0 0x7f58e419fd80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e4101520 0x7f58e406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 --2- 172.21.15.122:0/652096162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58e4102720 0x7f58e419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.069+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 >> 172.21.15.122:0/652096162 conn(0x7f58e40fcac0 msgr2=0x7f58e4105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.070+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 shutdown_connections 2024-04-02T13:34:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:38.070+0000 7f58ec433700 1 -- 172.21.15.122:0/652096162 wait complete. 2024-04-02T13:34:38.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ps' 2024-04-02T13:34:39.145 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:39 smithi122 ceph-mon[32114]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:39 smithi122 ceph-mon[32114]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:39 smithi187 ceph-mon[36064]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:39 smithi187 ceph-mon[36064]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:39 smithi188 ceph-mon[36117]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:39 smithi188 ceph-mon[36117]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:40.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- 172.21.15.122:0/3365734399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c1041c0 msgr2=0x7fb85c10aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 --2- 172.21.15.122:0/3365734399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c1041c0 0x7fb85c10aa70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb85400b0d0 tx=0x7fb85400b3e0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- 172.21.15.122:0/3365734399 shutdown_connections 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 --2- 172.21.15.122:0/3365734399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c1041c0 0x7fb85c10aa70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 --2- 172.21.15.122:0/3365734399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 0x7fb85c102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 --2- 172.21.15.122:0/3365734399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c103820 0x7fb85c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- 172.21.15.122:0/3365734399 >> 172.21.15.122:0/3365734399 conn(0x7fb85c0fdc00 msgr2=0x7fb85c100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- 172.21.15.122:0/3365734399 shutdown_connections 2024-04-02T13:34:40.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- 172.21.15.122:0/3365734399 wait complete. 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 Processor -- start 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.972+0000 7fb8621e2700 1 -- start start 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 0x7fb85c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 0x7fb85c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10e260 con 0x7fb85c103820 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10e0e0 con 0x7fb85c102620 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb8621e2700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb85c10e3e0 con 0x7fb85c1041c0 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43124/0 (socket says 172.21.15.122:43124) 2024-04-02T13:34:40.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 -- 172.21.15.122:0/1481901733 learned_addr learned my addr 172.21.15.122:0/1481901733 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb85b7fe700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 0x7fb85c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb85bfff700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 0x7fb85c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 -- 172.21.15.122:0/1481901733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 msgr2=0x7fb85c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 0x7fb85c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 -- 172.21.15.122:0/1481901733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 msgr2=0x7fb85c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 0x7fb85c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 -- 172.21.15.122:0/1481901733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb85c1a0480 con 0x7fb85c103820 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb85bfff700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 0x7fb85c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.973+0000 7fb852ffd700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb84400b560 tx=0x7fb84400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.974+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844015020 con 0x7fb85c103820 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.974+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb844011d10 con 0x7fb85c103820 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.974+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb85c1a0770 con 0x7fb85c103820 2024-04-02T13:34:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.974+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb844010e50 con 0x7fb85c103820 2024-04-02T13:34:40.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.974+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb85c1a0d30 con 0x7fb85c103820 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.975+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb85c0692b0 con 0x7fb85c103820 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.976+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fb844018020 con 0x7fb85c103820 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.976+0000 7fb8597fa700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 0x7fb8480a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.976+0000 7fb85b7fe700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 0x7fb8480a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.976+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb8440cfee0 con 0x7fb85c103820 2024-04-02T13:34:40.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.977+0000 7fb85b7fe700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 0x7fb8480a62a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb85c103680 tx=0x7fb84c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:40.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:40.981+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb84401f020 con 0x7fb85c103820 2024-04-02T13:34:41.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.189+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb85c062a30 con 0x7fb8480a3de0 2024-04-02T13:34:41.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.203+0000 7fb8597fa700 1 -- 172.21.15.122:0/1481901733 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7fb85c062a30 con 0x7fb8480a3de0 2024-04-02T13:34:41.206 INFO:teuthology.orchestra.run.smithi122.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-02T13:34:41.206 INFO:teuthology.orchestra.run.smithi122.stdout:mds.a smithi122 running (118s) 21s ago 117s 18.4M - 19.0.0-2609-g191d963a f7301e070545 9c52f184c1e2 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.b smithi187 running (115s) 48s ago 114s 26.7M - 19.0.0-2609-g191d963a f7301e070545 7db0b3aa5747 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.c smithi188 running (2m) 48s ago 2m 17.1M - 19.0.0-2609-g191d963a f7301e070545 1d13ad6701b0 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.d smithi122 running (2m) 21s ago 2m 15.3M - 19.0.0-2609-g191d963a f7301e070545 80318e11bbb1 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.e smithi187 running (2m) 48s ago 2m 16.4M - 19.0.0-2609-g191d963a f7301e070545 88c84384d0db 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.f smithi188 running (2m) 48s ago 2m 18.7M - 19.0.0-2609-g191d963a f7301e070545 0b7cff888dff 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.g smithi122 running (2m) 21s ago 2m 18.5M - 19.0.0-2609-g191d963a f7301e070545 9a85fd395f00 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.h smithi187 running (2m) 48s ago 2m 17.6M - 19.0.0-2609-g191d963a f7301e070545 891ac58885d0 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.i smithi188 running (2m) 48s ago 2m 18.1M - 19.0.0-2609-g191d963a f7301e070545 c13769789524 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.j smithi122 running (2m) 21s ago 2m 16.4M - 19.0.0-2609-g191d963a f7301e070545 9dfc2b0c1cfd 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.k smithi187 running (2m) 48s ago 2m 18.9M - 19.0.0-2609-g191d963a f7301e070545 179cdf463464 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mds.l smithi188 running (2m) 48s ago 2m 16.3M - 19.0.0-2609-g191d963a f7301e070545 e44ef441bb23 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.x smithi122 *:9283,8765 running (9m) 21s ago 9m 515M - 19.0.0-2609-g191d963a f7301e070545 b8451ced6522 2024-04-02T13:34:41.207 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.y smithi187 *:8443,8765 running (7m) 48s ago 7m 453M - 19.0.0-2609-g191d963a f7301e070545 740bc047a017 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:mgr.z smithi188 *:8443,8765 running (7m) 48s ago 7m 454M - 19.0.0-2609-g191d963a f7301e070545 60c096db2232 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:mon.a smithi122 running (9m) 21s ago 9m 49.4M 2048M 19.0.0-2609-g191d963a f7301e070545 ce047d83baa7 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:mon.b smithi187 running (7m) 48s ago 7m 39.7M 2048M 19.0.0-2609-g191d963a f7301e070545 2d596404099a 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:mon.c smithi188 running (8m) 48s ago 8m 39.1M 2048M 19.0.0-2609-g191d963a f7301e070545 0f49511e3d89 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 smithi122 running (7m) 21s ago 7m 53.9M 4272M 19.0.0-2609-g191d963a f7301e070545 287f9322fd0e 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 smithi187 running (6m) 48s ago 6m 47.0M 4270M 19.0.0-2609-g191d963a f7301e070545 bfec25de3dd0 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 smithi188 running (6m) 48s ago 6m 47.1M 4270M 19.0.0-2609-g191d963a f7301e070545 2439b0877a5d 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 smithi122 running (6m) 21s ago 6m 55.1M 4272M 19.0.0-2609-g191d963a f7301e070545 7052ec9c4608 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 smithi187 running (5m) 48s ago 5m 45.3M 4270M 19.0.0-2609-g191d963a f7301e070545 29c9a4704ace 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 smithi188 running (5m) 48s ago 5m 52.8M 4270M 19.0.0-2609-g191d963a f7301e070545 093aa9886f1b 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 smithi122 running (4m) 21s ago 4m 43.5M 4272M 19.0.0-2609-g191d963a f7301e070545 844c21a7ed11 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 smithi187 running (4m) 48s ago 4m 44.7M 4270M 19.0.0-2609-g191d963a f7301e070545 db582118161f 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 smithi188 running (4m) 48s ago 4m 46.2M 4270M 19.0.0-2609-g191d963a f7301e070545 91c299682684 2024-04-02T13:34:41.208 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 smithi122 running (3m) 21s ago 3m 45.6M 4272M 19.0.0-2609-g191d963a f7301e070545 31c40174ba35 2024-04-02T13:34:41.209 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 smithi187 running (3m) 48s ago 3m 50.6M 4270M 19.0.0-2609-g191d963a f7301e070545 f7ed1bcae85f 2024-04-02T13:34:41.209 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 smithi188 running (2m) 48s ago 2m 48.9M 4270M 19.0.0-2609-g191d963a f7301e070545 49d58663acd9 2024-04-02T13:34:41.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 msgr2=0x7fb8480a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 0x7fb8480a62a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb85c103680 tx=0x7fb84c00b410 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 msgr2=0x7fb85c198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb84400b560 tx=0x7fb84400b870 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 shutdown_connections 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb85c1041c0 0x7fb85c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb85c102620 0x7fb85c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb8480a3de0 0x7fb8480a62a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 --2- 172.21.15.122:0/1481901733 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb85c103820 0x7fb85c198d60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 >> 172.21.15.122:0/1481901733 conn(0x7fb85c0fdc00 msgr2=0x7fb85c106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 shutdown_connections 2024-04-02T13:34:41.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:41.205+0000 7fb8621e2700 1 -- 172.21.15.122:0/1481901733 wait complete. 2024-04-02T13:34:41.962 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:41 smithi122 ceph-mon[32114]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:41 smithi187 ceph-mon[36064]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:41.991 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch ls' 2024-04-02T13:34:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:41 smithi188 ceph-mon[36117]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-02T13:34:42.231 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:42.650 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:42 smithi122 ceph-mon[32114]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:42.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:42 smithi187 ceph-mon[36064]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:43.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:42 smithi188 ceph-mon[36117]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:43.766 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:43 smithi122 ceph-mon[32114]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:43.886 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:43 smithi187 ceph-mon[36064]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:43 smithi188 ceph-mon[36117]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 -- 172.21.15.122:0/3653155072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c101570 msgr2=0x7f519c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/3653155072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c101570 0x7f519c101990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f51980099c0 tx=0x7f5198009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 -- 172.21.15.122:0/3653155072 shutdown_connections 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/3653155072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c103110 0x7f519c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/3653155072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c101570 0x7f519c101990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/3653155072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c102770 0x7f519c102bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 -- 172.21.15.122:0/3653155072 >> 172.21.15.122:0/3653155072 conn(0x7f519c0fcb10 msgr2=0x7f519c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 -- 172.21.15.122:0/3653155072 shutdown_connections 2024-04-02T13:34:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.045+0000 7f51a3c3f700 1 -- 172.21.15.122:0/3653155072 wait complete. 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 Processor -- start 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 -- start start 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 0x7f519c105440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c103110 0x7f519c104090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d140 con 0x7f519c102770 2024-04-02T13:34:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10cfc0 con 0x7f519c103110 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a3c3f700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f519c10d2c0 con 0x7f519c101570 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43144/0 (socket says 172.21.15.122:43144) 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 -- 172.21.15.122:0/1922687781 learned_addr learned my addr 172.21.15.122:0/1922687781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a21dc700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c103110 0x7f519c104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a19db700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 0x7f519c105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 -- 172.21.15.122:0/1922687781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 msgr2=0x7f519c105440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 0x7f519c105440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 -- 172.21.15.122:0/1922687781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c103110 msgr2=0x7f519c104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c103110 0x7f519c104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a11da700 1 -- 172.21.15.122:0/1922687781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c1048a0 con 0x7f519c102770 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.046+0000 7f51a19db700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 0x7f519c105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f51a11da700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d950 tx=0x7f518c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c016070 con 0x7f519c102770 2024-04-02T13:34:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c009e20 con 0x7f519c102770 2024-04-02T13:34:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c1a29e0 con 0x7f519c102770 2024-04-02T13:34:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c00f620 con 0x7f519c102770 2024-04-02T13:34:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.047+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c1a2f40 con 0x7f519c102770 2024-04-02T13:34:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.048+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f519c0692b0 con 0x7f519c102770 2024-04-02T13:34:44.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.049+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f518c019070 con 0x7f519c102770 2024-04-02T13:34:44.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.049+0000 7f5192ffd700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 0x7f51880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:44.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.050+0000 7f51a19db700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 0x7f51880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:44.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.050+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f518c0d0280 con 0x7f519c102770 2024-04-02T13:34:44.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.050+0000 7f51a19db700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 0x7f51880a6180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f519800b580 tx=0x7f5198005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.053+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f518c09c900 con 0x7f519c102770 2024-04-02T13:34:44.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.262+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f519c0fecf0 con 0x7f51880a3cc0 2024-04-02T13:34:44.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.268+0000 7f5192ffd700 1 -- 172.21.15.122:0/1922687781 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f519c0fecf0 con 0x7f51880a3cc0 2024-04-02T13:34:44.271 INFO:teuthology.orchestra.run.smithi122.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-02T13:34:44.271 INFO:teuthology.orchestra.run.smithi122.stdout:mds.all 12/12 51s ago 2m smithi122=a;smithi122=d;smithi122=g;smithi122=j;smithi187=b;smithi187=e;smithi187=h;smithi187=k;smithi188=c;smithi188=f;smithi188=i;smithi188=l;count:12 2024-04-02T13:34:44.272 INFO:teuthology.orchestra.run.smithi122.stdout:mgr 3/3 51s ago 7m smithi122=x;smithi187=y;smithi188=z;count:3 2024-04-02T13:34:44.272 INFO:teuthology.orchestra.run.smithi122.stdout:mon 3/3 51s ago 8m smithi122:172.21.15.122=a;smithi187:172.21.15.187=b;smithi188:172.21.15.188=c;count:3 2024-04-02T13:34:44.272 INFO:teuthology.orchestra.run.smithi122.stdout:osd 12 51s ago - 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 msgr2=0x7f51880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 0x7f51880a6180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f519800b580 tx=0x7f5198005fb0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 msgr2=0x7f519c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d950 tx=0x7f518c00dc60 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 shutdown_connections 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f519c103110 0x7f519c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f519c101570 0x7f519c105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f51880a3cc0 0x7f51880a6180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 --2- 172.21.15.122:0/1922687781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f519c102770 0x7f519c103b50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 >> 172.21.15.122:0/1922687781 conn(0x7f519c0fcb10 msgr2=0x7f519c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:44.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 shutdown_connections 2024-04-02T13:34:44.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:44.270+0000 7f51a3c3f700 1 -- 172.21.15.122:0/1922687781 wait complete. 2024-04-02T13:34:45.075 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch host ls' 2024-04-02T13:34:45.317 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:45.768 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:45 smithi122 ceph-mon[32114]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:45.768 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:45 smithi122 ceph-mon[32114]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:45 smithi187 ceph-mon[36064]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:45 smithi187 ceph-mon[36064]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:45 smithi188 ceph-mon[36117]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:45 smithi188 ceph-mon[36117]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:47.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 -- 172.21.15.122:0/1476229965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f04108400 msgr2=0x7f1f0410a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:47.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/1476229965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f04108400 0x7f1f0410a7f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00b0a0 tx=0x7f1efc00b3b0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 -- 172.21.15.122:0/1476229965 shutdown_connections 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/1476229965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f04108400 0x7f1f0410a7f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/1476229965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f041031a0 0x7f1f04105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/1476229965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04105ad0 0x7f1f04107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 -- 172.21.15.122:0/1476229965 >> 172.21.15.122:0/1476229965 conn(0x7f1f040fcb10 msgr2=0x7f1f040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 -- 172.21.15.122:0/1476229965 shutdown_connections 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.039+0000 7f1f09dd9700 1 -- 172.21.15.122:0/1476229965 wait complete. 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 Processor -- start 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 -- start start 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f04105ad0 0x7f1f04198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04108400 0x7f1f0419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410e2a0 con 0x7f1f041031a0 2024-04-02T13:34:47.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410e120 con 0x7f1f04108400 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f09dd9700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0410e420 con 0x7f1f04105ad0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f037fe700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43168/0 (socket says 172.21.15.122:43168) 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f037fe700 1 -- 172.21.15.122:0/3778096156 learned_addr learned my addr 172.21.15.122:0/3778096156 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.040+0000 7f1f02ffd700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f04105ad0 0x7f1f04198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f03fff700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04108400 0x7f1f0419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 -- 172.21.15.122:0/3778096156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f04105ad0 msgr2=0x7f1f04198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f04105ad0 0x7f1f04198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 -- 172.21.15.122:0/3778096156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04108400 msgr2=0x7f1f0419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04108400 0x7f1f0419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 -- 172.21.15.122:0/3778096156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f041a03e0 con 0x7f1f041031a0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f037fe700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c9e0 tx=0x7f1ef400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4007ac0 con 0x7f1f041031a0 2024-04-02T13:34:47.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f041a06d0 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ef4007c60 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4018660 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.041+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a0c10 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.042+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f040692b0 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.043+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f1ef400f450 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.044+0000 7f1f00ff9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 0x7f1eec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.044+0000 7f1f02ffd700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 0x7f1eec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.044+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1ef40d0910 con 0x7f1f041031a0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.044+0000 7f1f02ffd700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 0x7f1eec0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009a10 tx=0x7f1ef8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:47.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.048+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1ef4014030 con 0x7f1f041031a0 2024-04-02T13:34:47.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.268+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1f04062a30 con 0x7f1eec0a3c40 2024-04-02T13:34:47.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.270+0000 7f1f00ff9700 1 -- 172.21.15.122:0/3778096156 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1f04062a30 con 0x7f1eec0a3c40 2024-04-02T13:34:47.273 INFO:teuthology.orchestra.run.smithi122.stdout:HOST ADDR LABELS STATUS 2024-04-02T13:34:47.274 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 172.21.15.122 2024-04-02T13:34:47.274 INFO:teuthology.orchestra.run.smithi122.stdout:smithi187 172.21.15.187 2024-04-02T13:34:47.274 INFO:teuthology.orchestra.run.smithi122.stdout:smithi188 172.21.15.188 2024-04-02T13:34:47.274 INFO:teuthology.orchestra.run.smithi122.stdout:3 hosts in cluster 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 msgr2=0x7f1eec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 0x7f1eec0a6100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009a10 tx=0x7f1ef8009190 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 msgr2=0x7f1f041987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400c9e0 tx=0x7f1ef400ccf0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 shutdown_connections 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f04108400 0x7f1f0419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f1eec0a3c40 0x7f1eec0a6100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f041031a0 0x7f1f041987f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 --2- 172.21.15.122:0/3778096156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f04105ad0 0x7f1f04198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:47.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.272+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 >> 172.21.15.122:0/3778096156 conn(0x7f1f040fcb10 msgr2=0x7f1f04103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.273+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 shutdown_connections 2024-04-02T13:34:47.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:47.273+0000 7f1f09dd9700 1 -- 172.21.15.122:0/3778096156 wait complete. 2024-04-02T13:34:47.936 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:47 smithi122 ceph-mon[32114]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:47 smithi187 ceph-mon[36064]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:47.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph orch device ls' 2024-04-02T13:34:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:47 smithi188 ceph-mon[36117]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:34:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:48.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:48 smithi122 ceph-mon[32114]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:48 smithi187 ceph-mon[36064]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:49.116 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:48 smithi188 ceph-mon[36117]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:49.871 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:49 smithi122 ceph-mon[32114]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:49 smithi187 ceph-mon[36064]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:49 smithi188 ceph-mon[36117]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:50.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 -- 172.21.15.122:0/4159856106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc094102620 msgr2=0x7fc094102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:50.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 --2- 172.21.15.122:0/4159856106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc094102620 0x7fc094102a40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc0900099c0 tx=0x7fc090009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 -- 172.21.15.122:0/4159856106 shutdown_connections 2024-04-02T13:34:50.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 --2- 172.21.15.122:0/4159856106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0941041c0 0x7fc09410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 --2- 172.21.15.122:0/4159856106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc094102620 0x7fc094102a40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 --2- 172.21.15.122:0/4159856106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094103820 0x7fc094103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.198+0000 7fc09b359700 1 -- 172.21.15.122:0/4159856106 >> 172.21.15.122:0/4159856106 conn(0x7fc0940fdc00 msgr2=0x7fc094100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:50.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 -- 172.21.15.122:0/4159856106 shutdown_connections 2024-04-02T13:34:50.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 -- 172.21.15.122:0/4159856106 wait complete. 2024-04-02T13:34:50.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 Processor -- start 2024-04-02T13:34:50.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 -- start start 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094102620 0x7fc094198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.199+0000 7fc09b359700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc094103820 0x7fc094198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410e200 con 0x7fc0941041c0 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410e080 con 0x7fc094103820 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc09410e380 con 0x7fc094102620 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33180/0 (socket says 172.21.15.122:33180) 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 -- 172.21.15.122:0/2559462127 learned_addr learned my addr 172.21.15.122:0/2559462127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0990f5700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094102620 0x7fc094198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0988f4700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc094103820 0x7fc094198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 -- 172.21.15.122:0/2559462127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094102620 msgr2=0x7fc094198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094102620 0x7fc094198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 -- 172.21.15.122:0/2559462127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc094103820 msgr2=0x7fc094198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:50.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc094103820 0x7fc094198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 -- 172.21.15.122:0/2559462127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0941a04d0 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc0998f6700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00add0 tx=0x7fc08c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c017020 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0941a07c0 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc08c00ce20 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0053e0 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.200+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0941a0cb0 con 0x7fc0941041c0 2024-04-02T13:34:50.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.201+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0940692b0 con 0x7fc0941041c0 2024-04-02T13:34:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.202+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc08c00c940 con 0x7fc0941041c0 2024-04-02T13:34:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.203+0000 7fc08a7fc700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 0x7fc0800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.203+0000 7fc0990f5700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 0x7fc0800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.203+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc08c0d0530 con 0x7fc0941041c0 2024-04-02T13:34:50.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.203+0000 7fc0990f5700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 0x7fc0800a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc09000b580 tx=0x7fc090005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:50.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.207+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc08c021020 con 0x7fc0941041c0 2024-04-02T13:34:50.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.417+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 --> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc094062a30 con 0x7fc0800a3c40 2024-04-02T13:34:50.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.418+0000 7fc08a7fc700 1 -- 172.21.15.122:0/2559462127 <== mgr.14152 v2:172.21.15.122:6800/3450525377 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc094062a30 con 0x7fc0800a3c40 2024-04-02T13:34:50.422 INFO:teuthology.orchestra.run.smithi122.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-02T13:34:50.422 INFO:teuthology.orchestra.run.smithi122.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:34:50.422 INFO:teuthology.orchestra.run.smithi122.stdout:smithi187 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MW400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-02T13:34:50.422 INFO:teuthology.orchestra.run.smithi122.stdout:smithi188 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500MP400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 msgr2=0x7fc0800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 0x7fc0800a6100 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc09000b580 tx=0x7fc090005fb0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 msgr2=0x7fc09419fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00add0 tx=0x7fc08c00c330 comp rx=0 tx=0).stop 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 shutdown_connections 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc0800a3c40 0x7fc0800a6100 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0941041c0 0x7fc09419fe30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc094102620 0x7fc094198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.420+0000 7fc09b359700 1 --2- 172.21.15.122:0/2559462127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc094103820 0x7fc094198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:50.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.421+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 >> 172.21.15.122:0/2559462127 conn(0x7fc0940fdc00 msgr2=0x7fc094106b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:50.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.421+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 shutdown_connections 2024-04-02T13:34:50.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:50.421+0000 7fc09b359700 1 -- 172.21.15.122:0/2559462127 wait complete. 2024-04-02T13:34:51.199 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-02T13:34:51.209 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:34:51.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph fs dump' 2024-04-02T13:34:51.453 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:51.852 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:51 smithi122 ceph-mon[32114]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:51.852 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:51 smithi122 ceph-mon[32114]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:51 smithi187 ceph-mon[36064]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:51 smithi187 ceph-mon[36064]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:51 smithi188 ceph-mon[36117]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:34:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:51 smithi188 ceph-mon[36117]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 -- 172.21.15.122:0/2521382392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 msgr2=0x7f0518102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 --2- 172.21.15.122:0/2521382392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518102b90 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f050c0099c0 tx=0x7f050c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 -- 172.21.15.122:0/2521382392 shutdown_connections 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 --2- 172.21.15.122:0/2521382392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05181030d0 0x7f0518109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 --2- 172.21.15.122:0/2521382392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0518101530 0x7f0518101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 --2- 172.21.15.122:0/2521382392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518102b90 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 -- 172.21.15.122:0/2521382392 >> 172.21.15.122:0/2521382392 conn(0x7f05180fcab0 msgr2=0x7f05180fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 -- 172.21.15.122:0/2521382392 shutdown_connections 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.306+0000 7f051d866700 1 -- 172.21.15.122:0/2521382392 wait complete. 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 Processor -- start 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 -- start start 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0518101530 0x7f0518198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05181030d0 0x7f051819feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f051810d190 con 0x7f0518102730 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f051810d010 con 0x7f0518101530 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.307+0000 7f051d866700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f051810d310 con 0x7f05181030d0 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33192/0 (socket says 172.21.15.122:33192) 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 -- 172.21.15.122:0/2518639540 learned_addr learned my addr 172.21.15.122:0/2518639540 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05177fe700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05181030d0 0x7f051819feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:53.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f0516ffd700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0518101530 0x7f0518198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 -- 172.21.15.122:0/2518639540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05181030d0 msgr2=0x7f051819feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05181030d0 0x7f051819feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 -- 172.21.15.122:0/2518639540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0518101530 msgr2=0x7f0518198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0518101530 0x7f0518198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 -- 172.21.15.122:0/2518639540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05181a0550 con 0x7f0518102730 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f05167fc700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f050c005910 tx=0x7f050c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c027070 con 0x7f0518102730 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05181a07e0 con 0x7f0518102730 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f050c0041d0 con 0x7f0518102730 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00f460 con 0x7f0518102730 2024-04-02T13:34:53.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.308+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a0d10 con 0x7f0518102730 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.309+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05180692b0 con 0x7f0518102730 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.311+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f050c02a070 con 0x7f0518102730 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.311+0000 7f051c864700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 0x7f05000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.311+0000 7f0516ffd700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 0x7f05000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.311+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f050c0d1b30 con 0x7f0518102730 2024-04-02T13:34:53.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.312+0000 7f0516ffd700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 0x7f05000a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0508006fd0 tx=0x7f0508008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:53.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.315+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f050c013070 con 0x7f0518102730 2024-04-02T13:34:53.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.557+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f05181a10e0 con 0x7f0518102730 2024-04-02T13:34:53.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.558+0000 7f051c864700 1 -- 172.21.15.122:0/2518639540 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3098 (secure 0 0 0) 0x7f050c09e240 con 0x7f0518102730 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:e27 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:legacy client fscid: 1 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:Filesystem 'cephfs' (1) 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:fs_name cephfs 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 27 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-04-02T13:32:49.884932+0000 2024-04-02T13:34:53.562 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-04-02T13:33:12.155913+0000 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:tableserver 0 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:root 0 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:session_timeout 300 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:session_autoclose 300 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:max_file_size 1099511627776 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:max_xattr_size 65536 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:required_client_features {} 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure 0 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:last_failure_osd_epoch 0 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:max_mds 5 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:in 0,1,2,3,4 2024-04-02T13:34:53.563 INFO:teuthology.orchestra.run.smithi122.stdout:up {0=24512,1=24487,2=24485,3=24460,4=14628} 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:failed 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:damaged 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:stopped 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:data_pools [3,4] 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:metadata_pool 2 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:inline_data disabled 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:balancer 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:bal_rank_mask -1 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:standby_count_wanted 0 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:qdb_cluster leader: 24512 members: 24485,24487,24460,24512,14628 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b{0:24512} state up:active seq 3 addr [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.564 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.e{0:24455} state up:standby-replay seq 1 addr [v2:172.21.15.187:6834/4231455845,v1:172.21.15.187:6835/4231455845] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.i{1:24487} state up:active seq 9 addr [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.f{1:24448} state up:standby-replay seq 1 addr [v2:172.21.15.188:6834/179772221,v1:172.21.15.188:6835/179772221] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.h{2:24485} state up:active seq 10 addr [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.k{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.187:6832/1223173792,v1:172.21.15.187:6833/1223173792] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c{3:24460} state up:active seq 11 addr [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.l{3:24400} state up:standby-replay seq 1 addr [v2:172.21.15.188:6832/546075479,v1:172.21.15.188:6833/546075479] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a{4:14628} state up:active seq 8 addr [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.122:6838/1672540384,v1:172.21.15.122:6839/1672540384] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:Standby daemons: 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:53.565 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.122:6834/2838975565,v1:172.21.15.122:6835/2838975565] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.566 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.122:6836/1110044990,v1:172.21.15.122:6837/1110044990] compat {c=[1],r=[1],i=[1fff]}] 2024-04-02T13:34:53.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 msgr2=0x7f05000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:53.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 0x7f05000a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0508006fd0 tx=0x7f0508008040 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 msgr2=0x7f0518198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f050c005910 tx=0x7f050c0047b0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 shutdown_connections 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05181030d0 0x7f051819feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0518101530 0x7f0518198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f05000a3c40 0x7f05000a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 --2- 172.21.15.122:0/2518639540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0518102730 0x7f0518198e80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 >> 172.21.15.122:0/2518639540 conn(0x7f05180fcab0 msgr2=0x7f05180fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 shutdown_connections 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:53.560+0000 7f051d866700 1 -- 172.21.15.122:0/2518639540 wait complete. 2024-04-02T13:34:53.567 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:34:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:53 smithi122 ceph-mon[32114]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:53 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2518639540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:34:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:53 smithi187 ceph-mon[36064]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:53 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2518639540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:34:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:53 smithi188 ceph-mon[36117]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-02T13:34:54.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:53 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2518639540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-02T13:34:54.206 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- bash -c 'ceph osd dump' 2024-04-02T13:34:54.450 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:34:54.840 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-02T13:34:54.840 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:34:54.840 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:54 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-04-02T13:34:54.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-02T13:34:54.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:34:54.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:54 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-04-02T13:34:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]: dispatch 2024-04-02T13:34:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-04-02T13:34:55.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:54 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]: dispatch 2024-04-02T13:34:55.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:55 smithi122 ceph-mon[32114]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:55.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-02T13:34:55.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:34:55.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:55 smithi122 ceph-mon[32114]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-04-02T13:34:55.933 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:55 smithi122 ceph-mon[32114]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:34:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:55 smithi187 ceph-mon[36064]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-02T13:34:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:34:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:55 smithi187 ceph-mon[36064]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-04-02T13:34:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:55 smithi187 ceph-mon[36064]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:34:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:55 smithi188 ceph-mon[36117]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-02T13:34:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [9, 8]}]': finished 2024-04-02T13:34:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-04-02T13:34:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:55 smithi188 ceph-mon[36117]: from='mgr.14152 172.21.15.122:0/1876772125' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [5, 8]}]': finished 2024-04-02T13:34:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:55 smithi188 ceph-mon[36117]: osdmap e93: 12 total, 12 up, 12 in 2024-04-02T13:34:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 -- 172.21.15.122:0/649908655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa184103850 msgr2=0x7fa184103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 --2- 172.21.15.122:0/649908655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa184103850 0x7fa184103cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa16c0099c0 tx=0x7fa16c009cd0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 -- 172.21.15.122:0/649908655 shutdown_connections 2024-04-02T13:34:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 --2- 172.21.15.122:0/649908655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1841041f0 0x7fa18410aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 --2- 172.21.15.122:0/649908655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184102650 0x7fa184102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 --2- 172.21.15.122:0/649908655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa184103850 0x7fa184103cb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 -- 172.21.15.122:0/649908655 >> 172.21.15.122:0/649908655 conn(0x7fa1840fdbf0 msgr2=0x7fa184100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 -- 172.21.15.122:0/649908655 shutdown_connections 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.216+0000 7fa189baf700 1 -- 172.21.15.122:0/649908655 wait complete. 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 Processor -- start 2024-04-02T13:34:56.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 -- start start 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa184102650 0x7fa184198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184103850 0x7fa184198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410e240 con 0x7fa1841041f0 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410e0c0 con 0x7fa184102650 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa189baf700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa18410e3c0 con 0x7fa184103850 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa183fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa183fff700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33210/0 (socket says 172.21.15.122:33210) 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.217+0000 7fa183fff700 1 -- 172.21.15.122:0/2367734886 learned_addr learned my addr 172.21.15.122:0/2367734886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa182ffd700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184103850 0x7fa184198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:56.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa1837fe700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa184102650 0x7fa184198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 -- 172.21.15.122:0/2367734886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184103850 msgr2=0x7fa184198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184103850 0x7fa184198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 -- 172.21.15.122:0/2367734886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa184102650 msgr2=0x7fa184198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa184102650 0x7fa184198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 -- 172.21.15.122:0/2367734886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1841a03c0 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa183fff700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c930 tx=0x7fa17400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174007ac0 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa174007c60 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1841a06b0 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa174018660 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.218+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1841a0bf0 con 0x7fa1841041f0 2024-04-02T13:34:56.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.219+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1840692b0 con 0x7fa1841041f0 2024-04-02T13:34:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.220+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fa17400f450 con 0x7fa1841041f0 2024-04-02T13:34:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.221+0000 7fa180ff9700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 0x7fa1700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.221+0000 7fa1837fe700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 0x7fa1700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.221+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fa1740d06c0 con 0x7fa1841041f0 2024-04-02T13:34:56.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.221+0000 7fa1837fe700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 0x7fa1700a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0060b0 tx=0x7fa17c007b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:56.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.225+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa174014030 con 0x7fa1841041f0 2024-04-02T13:34:56.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.422+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa18402dd60 con 0x7fa1841041f0 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.422+0000 7fa180ff9700 1 -- 172.21.15.122:0/2367734886 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5703 (secure 0 0 0) 0x7fa17409ccf0 con 0x7fa1841041f0 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 94 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-04-02T13:25:00.910314+0000 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-04-02T13:34:55.620203+0000 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:crush_version 27 2024-04-02T13:34:56.426 INFO:teuthology.orchestra.run.smithi122.stdout:full_ratio 0.95 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:backfillfull_ratio 0.9 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:nearfull_ratio 0.85 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:require_min_compat_client luminous 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:min_compat_client luminous 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:require_osd_release squid 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:stretch_mode_enabled false 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:max_osd 12 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6802/933440597,v1:172.21.15.122:6803/933440597] [v2:172.21.15.122:6804/933440597,v1:172.21.15.122:6805/933440597] exists,up 94d678b8-fbb0-485b-b98a-133e7a6de516 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6800/2690363076,v1:172.21.15.187:6801/2690363076] [v2:172.21.15.187:6802/2690363076,v1:172.21.15.187:6803/2690363076] exists,up 349c8fa1-e818-4c5e-862d-e30ff9fa14bc 2024-04-02T13:34:56.427 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6800/2840236580,v1:172.21.15.188:6801/2840236580] [v2:172.21.15.188:6802/2840236580,v1:172.21.15.188:6803/2840236580] exists,up e9bc5736-0e61-468b-b9a8-f988e2d36bf9 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6810/2685457026,v1:172.21.15.122:6811/2685457026] [v2:172.21.15.122:6812/2685457026,v1:172.21.15.122:6813/2685457026] exists,up e7608c20-4f88-434c-99ae-225f08577c91 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.4 up in weight 1 up_from 33 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6808/101695590,v1:172.21.15.187:6809/101695590] [v2:172.21.15.187:6810/101695590,v1:172.21.15.187:6811/101695590] exists,up 038779c8-ba3f-4165-a72d-61c31b3c3cbd 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.5 up in weight 1 up_from 39 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6808/1528823096,v1:172.21.15.188:6809/1528823096] [v2:172.21.15.188:6810/1528823096,v1:172.21.15.188:6811/1528823096] exists,up 266e5146-9b00-4cbe-8ced-85833fa9e959 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.6 up in weight 1 up_from 47 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6818/2482601616,v1:172.21.15.122:6819/2482601616] [v2:172.21.15.122:6820/2482601616,v1:172.21.15.122:6821/2482601616] exists,up 2961ced1-b269-44f8-ad13-dc4435028869 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6816/3961503181,v1:172.21.15.187:6817/3961503181] [v2:172.21.15.187:6818/3961503181,v1:172.21.15.187:6819/3961503181] exists,up 1fd83ac7-db42-40ec-aee5-c69f77239ce9 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.8 up in weight 1 up_from 60 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6816/2378797977,v1:172.21.15.188:6817/2378797977] [v2:172.21.15.188:6818/2378797977,v1:172.21.15.188:6819/2378797977] exists,up 5093b075-74f5-494e-afef-c42bda5e04b6 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.9 up in weight 1 up_from 68 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6826/3872606263,v1:172.21.15.122:6827/3872606263] [v2:172.21.15.122:6828/3872606263,v1:172.21.15.122:6829/3872606263] exists,up a26c2ba8-71db-4d24-8add-1e8ec85b3b70 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.10 up in weight 1 up_from 73 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.187:6824/442100564,v1:172.21.15.187:6825/442100564] [v2:172.21.15.187:6826/442100564,v1:172.21.15.187:6827/442100564] exists,up 9ee33545-53b3-428d-9b14-69e4bba42144 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:osd.11 up in weight 1 up_from 81 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.188:6824/993879353,v1:172.21.15.188:6825/993879353] [v2:172.21.15.188:6826/993879353,v1:172.21.15.188:6827/993879353] exists,up 14fa986c-a900-464f-a1d3-4bec844c1256 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.8 [2,11] 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.b [2,4] 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.10 [8,11] 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 2.1a [8,1] 2024-04-02T13:34:56.428 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.11 [4,8] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.1d [5,9] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.21 [4,6] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.34 [4,9] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 3.3b [3,9] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 4.b [9,8] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 4.12 [9,10] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 4.1d [5,8] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:pg_upmap_items 4.1f [9,8] 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/3421882822 expires 2024-04-03T13:25:54.372176+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/1783168866 expires 2024-04-03T13:25:54.372176+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6801/1645681773 expires 2024-04-03T13:25:27.690608+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/684561021 expires 2024-04-03T13:25:27.690608+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/776403907 expires 2024-04-03T13:25:54.372176+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/4063470519 expires 2024-04-03T13:25:54.372176+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/1783168866 expires 2024-04-03T13:25:54.372176+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:6800/1645681773 expires 2024-04-03T13:25:27.690608+0000 2024-04-02T13:34:56.429 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/1474593903 expires 2024-04-03T13:25:27.690608+0000 2024-04-02T13:34:56.430 INFO:teuthology.orchestra.run.smithi122.stdout:blocklist 172.21.15.122:0/1325797276 expires 2024-04-03T13:25:27.690608+0000 2024-04-02T13:34:56.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.424+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 msgr2=0x7fa1700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:56.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.424+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 0x7fa1700a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0060b0 tx=0x7fa17c007b80 comp rx=0 tx=0).stop 2024-04-02T13:34:56.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 msgr2=0x7fa18419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa17400c930 tx=0x7fa17400cc40 comp rx=0 tx=0).stop 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 shutdown_connections 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fa1700a3c40 0x7fa1700a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1841041f0 0x7fa18419fd20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa184102650 0x7fa184198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 --2- 172.21.15.122:0/2367734886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa184103850 0x7fa184198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 >> 172.21.15.122:0/2367734886 conn(0x7fa1840fdbf0 msgr2=0x7fa184106a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 shutdown_connections 2024-04-02T13:34:56.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:56.425+0000 7fa189baf700 1 -- 172.21.15.122:0/2367734886 wait complete. 2024-04-02T13:34:56.906 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:56 smithi122 ceph-mon[32114]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:34:56.906 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:56 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2367734886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:34:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:56 smithi187 ceph-mon[36064]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:34:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:56 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2367734886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:34:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:56 smithi188 ceph-mon[36117]: osdmap e94: 12 total, 12 up, 12 in 2024-04-02T13:34:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:56 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2367734886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-02T13:34:57.228 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-02T13:34:57.240 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:34:57.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 -- 172.21.15.122:0/2801274519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c101c10 msgr2=0x7f856c102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 --2- 172.21.15.122:0/2801274519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c101c10 0x7f856c102090 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8554009900 tx=0x7f855402bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 -- 172.21.15.122:0/2801274519 shutdown_connections 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 --2- 172.21.15.122:0/2801274519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f856c1025d0 0x7f856c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 --2- 172.21.15.122:0/2801274519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c101c10 0x7f856c102090 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 --2- 172.21.15.122:0/2801274519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c100a20 0x7f856c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 -- 172.21.15.122:0/2801274519 >> 172.21.15.122:0/2801274519 conn(0x7f856c0fbfb0 msgr2=0x7f856c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 -- 172.21.15.122:0/2801274519 shutdown_connections 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.344+0000 7f85729e9640 1 -- 172.21.15.122:0/2801274519 wait complete. 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 Processor -- start 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 -- start start 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f856c101c10 0x7f856c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 0x7f856c19f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f856c10c740 con 0x7f856c100a20 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f856c10c5c0 con 0x7f856c1025d0 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85729e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f856c10c8c0 con 0x7f856c101c10 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33222/0 (socket says 172.21.15.122:33222) 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 -- 172.21.15.122:0/325669973 learned_addr learned my addr 172.21.15.122:0/325669973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 -- 172.21.15.122:0/325669973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f856c101c10 msgr2=0x7f856c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f856c101c10 0x7f856c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 -- 172.21.15.122:0/325669973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 msgr2=0x7f856c19f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f8570f86640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 0x7f856c19f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 0x7f856c19f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 -- 172.21.15.122:0/325669973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856c19fac0 con 0x7f856c100a20 2024-04-02T13:34:57.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f8570f86640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 0x7f856c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f856bfff640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f856400b9c0 tx=0x7f856400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.345+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564019070 con 0x7f856c100a20 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.346+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856400cba0 con 0x7f856c100a20 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.346+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856c19fdb0 con 0x7f856c100a20 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.346+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564012640 con 0x7f856c100a20 2024-04-02T13:34:57.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.346+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f856c1a0270 con 0x7f856c100a20 2024-04-02T13:34:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.347+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856c107d40 con 0x7f856c100a20 2024-04-02T13:34:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.348+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f856401c070 con 0x7f856c100a20 2024-04-02T13:34:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.348+0000 7f85697fa640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 0x7f85580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.348+0000 7f856b7fe640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 0x7f85580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.348+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f85640cfda0 con 0x7f856c100a20 2024-04-02T13:34:57.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.349+0000 7f856b7fe640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 0x7f85580a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f855402c710 tx=0x7f8554000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.353+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f856409c3d0 con 0x7f856c100a20 2024-04-02T13:34:57.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.533+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f856c0627a0 con 0x7f856c100a20 2024-04-02T13:34:57.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.535+0000 7f85697fa640 1 -- 172.21.15.122:0/325669973 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f856409bcd0 con 0x7f856c100a20 2024-04-02T13:34:57.538 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:57.539 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":94,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:34:55.620203+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:34:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 msgr2=0x7f85580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 0x7f85580a6070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f855402c710 tx=0x7f8554000bc0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 msgr2=0x7f856c06ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f856400b9c0 tx=0x7f856400be80 comp rx=0 tx=0).stop 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 shutdown_connections 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f85580a3bb0 0x7f85580a6070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f856c1025d0 0x7f856c19f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f856c101c10 0x7f856c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 --2- 172.21.15.122:0/325669973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f856c100a20 0x7f856c06ace0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 >> 172.21.15.122:0/325669973 conn(0x7f856c0fbfb0 msgr2=0x7f856c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 shutdown_connections 2024-04-02T13:34:57.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.537+0000 7f85729e9640 1 -- 172.21.15.122:0/325669973 wait complete. 2024-04-02T13:34:57.552 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:34:57.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 -- 172.21.15.122:0/3929217576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c102db0 msgr2=0x7fd39c103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 --2- 172.21.15.122:0/3929217576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c102db0 0x7fd39c103230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd390009930 tx=0x7fd39002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 -- 172.21.15.122:0/3929217576 shutdown_connections 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 --2- 172.21.15.122:0/3929217576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c10a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 --2- 172.21.15.122:0/3929217576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c102db0 0x7fd39c103230 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 --2- 172.21.15.122:0/3929217576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c0ffde0 0x7fd39c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 -- 172.21.15.122:0/3929217576 >> 172.21.15.122:0/3929217576 conn(0x7fd39c0fb710 msgr2=0x7fd39c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 -- 172.21.15.122:0/3929217576 shutdown_connections 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.655+0000 7fd3a4055640 1 -- 172.21.15.122:0/3929217576 wait complete. 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 Processor -- start 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 -- start start 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c102db0 0x7fd39c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10d7a0 con 0x7fd39c0ffde0 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10d620 con 0x7fd39c103770 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a4055640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c10d920 con 0x7fd39c102db0 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33238/0 (socket says 172.21.15.122:33238) 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 -- 172.21.15.122:0/2477740055 learned_addr learned my addr 172.21.15.122:0/2477740055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a25f2640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a15f0640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c102db0 0x7fd39c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 -- 172.21.15.122:0/2477740055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c102db0 msgr2=0x7fd39c198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c102db0 0x7fd39c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 -- 172.21.15.122:0/2477740055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 msgr2=0x7fd39c19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.656+0000 7fd3a1df1640 1 -- 172.21.15.122:0/2477740055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c19fa00 con 0x7fd39c0ffde0 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd3a25f2640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd3a1df1640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d770 tx=0x7fd39400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394017070 con 0x7fd39c0ffde0 2024-04-02T13:34:57.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd394009c30 con 0x7fd39c0ffde0 2024-04-02T13:34:57.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd394003dd0 con 0x7fd39c0ffde0 2024-04-02T13:34:57.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c19fcf0 con 0x7fd39c0ffde0 2024-04-02T13:34:57.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.657+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c1a0190 con 0x7fd39c0ffde0 2024-04-02T13:34:57.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.658+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39c108ee0 con 0x7fd39c0ffde0 2024-04-02T13:34:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.659+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fd394003f70 con 0x7fd39c0ffde0 2024-04-02T13:34:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.660+0000 7fd39affd640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 0x7fd3840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.660+0000 7fd3a15f0640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 0x7fd3840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.660+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fd3940d0230 con 0x7fd39c0ffde0 2024-04-02T13:34:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.660+0000 7fd3a15f0640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 0x7fd3840a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd39002c170 tx=0x7fd390005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.664+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd39409c860 con 0x7fd39c0ffde0 2024-04-02T13:34:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:57 smithi122 ceph-mon[32114]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:57 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/325669973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:57.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.841+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd39c061f40 con 0x7fd39c0ffde0 2024-04-02T13:34:57.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.842+0000 7fd39affd640 1 -- 172.21.15.122:0/2477740055 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd394009de0 con 0x7fd39c0ffde0 2024-04-02T13:34:57.845 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:34:57.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 msgr2=0x7fd3840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 0x7fd3840a6070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd39002c170 tx=0x7fd390005fd0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 msgr2=0x7fd39c197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d770 tx=0x7fd39400dc30 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 shutdown_connections 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fd3840a3bb0 0x7fd3840a6070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd39c103770 0x7fd39c19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd39c102db0 0x7fd39c198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 --2- 172.21.15.122:0/2477740055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c0ffde0 0x7fd39c197de0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 >> 172.21.15.122:0/2477740055 conn(0x7fd39c0fb710 msgr2=0x7fd39c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 shutdown_connections 2024-04-02T13:34:57.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.844+0000 7fd3a4055640 1 -- 172.21.15.122:0/2477740055 wait complete. 2024-04-02T13:34:57.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 -- 172.21.15.122:0/1881149069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 msgr2=0x7f3524100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 --2- 172.21.15.122:0/1881149069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524100e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3518009900 tx=0x7f351802bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 -- 172.21.15.122:0/1881149069 shutdown_connections 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 --2- 172.21.15.122:0/1881149069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 --2- 172.21.15.122:0/1881149069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 0x7f35241020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 --2- 172.21.15.122:0/1881149069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524100e60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 -- 172.21.15.122:0/1881149069 >> 172.21.15.122:0/1881149069 conn(0x7f35240fbfb0 msgr2=0x7f35240fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 -- 172.21.15.122:0/1881149069 shutdown_connections 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.961+0000 7f352a1cd640 1 -- 172.21.15.122:0/1881149069 wait complete. 2024-04-02T13:34:57.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 Processor -- start 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 -- start start 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 0x7f3524103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c690 con 0x7f3524100a40 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c510 con 0x7f35241025f0 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f352a1cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f352410c810 con 0x7f3524101c30 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33256/0 (socket says 172.21.15.122:33256) 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 -- 172.21.15.122:0/3389124111 learned_addr learned my addr 172.21.15.122:0/3389124111 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f3523fff640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f3522ffd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 0x7f3524103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 -- 172.21.15.122:0/3389124111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 msgr2=0x7f3524103010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 0x7f3524103010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 -- 172.21.15.122:0/3389124111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 msgr2=0x7f3524103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.962+0000 7f35237fe640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f35237fe640 1 -- 172.21.15.122:0/3389124111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3524103e10 con 0x7f3524100a40 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f3523fff640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f35237fe640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3518000c00 tx=0x7f351802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518041070 con 0x7f3524100a40 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35241a2000 con 0x7f3524100a40 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35180044d0 con 0x7f3524100a40 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518034440 con 0x7f3524100a40 2024-04-02T13:34:57.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.963+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35241a2510 con 0x7f3524100a40 2024-04-02T13:34:57.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.964+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3524068060 con 0x7f3524100a40 2024-04-02T13:34:57.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.965+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f3518044070 con 0x7f3524100a40 2024-04-02T13:34:57.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.966+0000 7f3520ff9640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 0x7f350c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.966+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f35180f2600 con 0x7f3524100a40 2024-04-02T13:34:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.966+0000 7f3522ffd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 0x7f350c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:57.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.966+0000 7f3522ffd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 0x7f350c0a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3514005d00 tx=0x7f3514005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:57.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:57.970+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f35180bec30 con 0x7f3524100a40 2024-04-02T13:34:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:57 smithi187 ceph-mon[36064]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:57 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/325669973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:57 smithi188 ceph-mon[36117]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-02T13:34:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:57 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/325669973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:58.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.153+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f35241a2830 con 0x7f3524100a40 2024-04-02T13:34:58.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.153+0000 7f3520ff9640 1 -- 172.21.15.122:0/3389124111 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f35180be530 con 0x7f3524100a40 2024-04-02T13:34:58.157 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:58.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 msgr2=0x7f350c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 0x7f350c0a6070 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3514005d00 tx=0x7f3514005c50 comp rx=0 tx=0).stop 2024-04-02T13:34:58.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 msgr2=0x7f3524104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3518000c00 tx=0x7f351802c640 comp rx=0 tx=0).stop 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 shutdown_connections 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f350c0a3bb0 0x7f350c0a6070 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35241025f0 0x7f3524103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524101c30 0x7f3524103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 --2- 172.21.15.122:0/3389124111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3524100a40 0x7f3524104980 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 >> 172.21.15.122:0/3389124111 conn(0x7f35240fbfb0 msgr2=0x7f3524069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 shutdown_connections 2024-04-02T13:34:58.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.156+0000 7f352a1cd640 1 -- 172.21.15.122:0/3389124111 wait complete. 2024-04-02T13:34:58.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:34:58.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.271+0000 7f08afeb2640 1 -- 172.21.15.122:0/3476556795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 msgr2=0x7f08a810a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.271+0000 7f08afeb2640 1 --2- 172.21.15.122:0/3476556795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a810a0d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009930 tx=0x7f08a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:58.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- 172.21.15.122:0/3476556795 shutdown_connections 2024-04-02T13:34:58.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- 172.21.15.122:0/3476556795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a810a0d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- 172.21.15.122:0/3476556795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a8102e80 0x7f08a8103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- 172.21.15.122:0/3476556795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a80ffe10 0x7f08a8100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- 172.21.15.122:0/3476556795 >> 172.21.15.122:0/3476556795 conn(0x7f08a80fb700 msgr2=0x7f08a80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- 172.21.15.122:0/3476556795 shutdown_connections 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- 172.21.15.122:0/3476556795 wait complete. 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 Processor -- start 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- start start 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 0x7f08a8197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8102e80 0x7f08a8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810d790 con 0x7f08a8103840 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810d610 con 0x7f08a80ffe10 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08afeb2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810d910 con 0x7f08a8102e80 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.272+0000 7f08ae44f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33272/0 (socket says 172.21.15.122:33272) 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 -- 172.21.15.122:0/269771042 learned_addr learned my addr 172.21.15.122:0/269771042 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ad44d640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8102e80 0x7f08a8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08adc4e640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 0x7f08a8197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 -- 172.21.15.122:0/269771042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8102e80 msgr2=0x7f08a8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8102e80 0x7f08a8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 -- 172.21.15.122:0/269771042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 msgr2=0x7f08a8197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 0x7f08a8197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 -- 172.21.15.122:0/269771042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a819fa10 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08adc4e640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 0x7f08a8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08ae44f640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08a0036040 tx=0x7f08a0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0041070 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a819fca0 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a00054e0 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0034440 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.273+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a81a0190 con 0x7f08a8103840 2024-04-02T13:34:58.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.274+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a8108f50 con 0x7f08a8103840 2024-04-02T13:34:58.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.275+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f08a0044070 con 0x7f08a8103840 2024-04-02T13:34:58.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.276+0000 7f089effd640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 0x7f08940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.276+0000 7f08adc4e640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 0x7f08940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.276+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f08a00f2900 con 0x7f08a8103840 2024-04-02T13:34:58.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.276+0000 7f08adc4e640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 0x7f08940a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f08a8069e50 tx=0x7f08a4008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.279+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f08a00bef30 con 0x7f08a8103840 2024-04-02T13:34:58.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.457+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f08a802d520 con 0x7f08a8103840 2024-04-02T13:34:58.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.458+0000 7f089effd640 1 -- 172.21.15.122:0/269771042 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f08a00be830 con 0x7f08a8103840 2024-04-02T13:34:58.461 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:34:58.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 msgr2=0x7f08940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 0x7f08940a6070 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f08a8069e50 tx=0x7f08a4008370 comp rx=0 tx=0).stop 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 msgr2=0x7f08a819f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f08a0036040 tx=0x7f08a0005dd0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 shutdown_connections 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a8103840 0x7f08a819f3a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8102e80 0x7f08a8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f08940a3bb0 0x7f08940a6070 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 --2- 172.21.15.122:0/269771042 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a80ffe10 0x7f08a8197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 >> 172.21.15.122:0/269771042 conn(0x7f08a80fb700 msgr2=0x7f08a8106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 shutdown_connections 2024-04-02T13:34:58.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.460+0000 7f08afeb2640 1 -- 172.21.15.122:0/269771042 wait complete. 2024-04-02T13:34:58.474 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/3517094708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 msgr2=0x7fe7d80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/3517094708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d80749a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009900 tx=0x7fe7c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/3517094708 shutdown_connections 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/3517094708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 0x7fe7d810be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/3517094708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d80749a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/3517094708 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d8073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/3517094708 >> 172.21.15.122:0/3517094708 conn(0x7fe7d80fd0a0 msgr2=0x7fe7d80ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/3517094708 shutdown_connections 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.579+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/3517094708 wait complete. 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 Processor -- start 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 -- start start 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 0x7fe7d81a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810fa10 con 0x7fe7d8074520 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810f890 con 0x7fe7d8075b00 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7ddaf2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d810fb90 con 0x7fe7d81099e0 2024-04-02T13:34:58.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35656/0 (socket says 172.21.15.122:35656) 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.580+0000 7fe7d77fe640 1 -- 172.21.15.122:0/4090769803 learned_addr learned my addr 172.21.15.122:0/4090769803 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d7fff640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 0x7fe7d81a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7ceffd640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 -- 172.21.15.122:0/4090769803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 msgr2=0x7fe7d81a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 0x7fe7d81a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 -- 172.21.15.122:0/4090769803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 msgr2=0x7fe7d819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 -- 172.21.15.122:0/4090769803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d81a1cd0 con 0x7fe7d8074520 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7ceffd640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d77fe640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800d770 tx=0x7fe7c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8017070 con 0x7fe7d8074520 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7c8005500 con 0x7fe7d8074520 2024-04-02T13:34:58.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d81a1fc0 con 0x7fe7d8074520 2024-04-02T13:34:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8004e00 con 0x7fe7d8074520 2024-04-02T13:34:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.581+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d81a2420 con 0x7fe7d8074520 2024-04-02T13:34:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.582+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d8108760 con 0x7fe7d8074520 2024-04-02T13:34:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.583+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fe7c8004fa0 con 0x7fe7d8074520 2024-04-02T13:34:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.584+0000 7fe7d57fa640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 0x7fe7c40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.584+0000 7fe7ceffd640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 0x7fe7c40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.584+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe7c80d02d0 con 0x7fe7d8074520 2024-04-02T13:34:58.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.584+0000 7fe7ceffd640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 0x7fe7c40a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006010 tx=0x7fe7c0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.588+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe7c809c900 con 0x7fe7d8074520 2024-04-02T13:34:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.768+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe7d8062700 con 0x7fe7d8074520 2024-04-02T13:34:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.768+0000 7fe7d57fa640 1 -- 172.21.15.122:0/4090769803 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe7c809c200 con 0x7fe7d8074520 2024-04-02T13:34:58.772 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 msgr2=0x7fe7c40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 0x7fe7c40a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006010 tx=0x7fe7c0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 msgr2=0x7fe7d819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800d770 tx=0x7fe7c800dc30 comp rx=0 tx=0).stop 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 shutdown_connections 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7d81099e0 0x7fe7d81a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe7c40a3bb0 0x7fe7c40a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7d8075b00 0x7fe7d819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 --2- 172.21.15.122:0/4090769803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7d8074520 0x7fe7d819a0b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 >> 172.21.15.122:0/4090769803 conn(0x7fe7d80fd0a0 msgr2=0x7fe7d8105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 shutdown_connections 2024-04-02T13:34:58.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.770+0000 7fe7ddaf2640 1 -- 172.21.15.122:0/4090769803 wait complete. 2024-04-02T13:34:58.784 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:58 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2477740055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:58.784 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:58 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3389124111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:58.784 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:58 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/269771042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:58.785 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:34:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 -- 172.21.15.122:0/2009628910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 msgr2=0x7fc4b0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/2009628910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0100e40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0009900 tx=0x7fc4a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:34:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 -- 172.21.15.122:0/2009628910 shutdown_connections 2024-04-02T13:34:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/2009628910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b01025d0 0x7fc4b0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/2009628910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b0101c10 0x7fc4b0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/2009628910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0100e40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 -- 172.21.15.122:0/2009628910 >> 172.21.15.122:0/2009628910 conn(0x7fc4b00fbf90 msgr2=0x7fc4b00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 -- 172.21.15.122:0/2009628910 shutdown_connections 2024-04-02T13:34:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.883+0000 7fc4b5b15640 1 -- 172.21.15.122:0/2009628910 wait complete. 2024-04-02T13:34:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 Processor -- start 2024-04-02T13:34:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 -- start start 2024-04-02T13:34:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 0x7fc4b0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 0x7fc4b019e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b010c6a0 con 0x7fc4b0100a20 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b010c520 con 0x7fc4b01025d0 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4b5b15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b010c820 con 0x7fc4b0101c10 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35668/0 (socket says 172.21.15.122:35668) 2024-04-02T13:34:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 -- 172.21.15.122:0/1597225703 learned_addr learned my addr 172.21.15.122:0/1597225703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4affff640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 0x7fc4b019e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4aeffd640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 0x7fc4b0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 -- 172.21.15.122:0/1597225703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 msgr2=0x7fc4b0197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 0x7fc4b0197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 -- 172.21.15.122:0/1597225703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 msgr2=0x7fc4b019e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 0x7fc4b019e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4af7fe640 1 -- 172.21.15.122:0/1597225703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b019e960 con 0x7fc4b0100a20 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4aeffd640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 0x7fc4b0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.884+0000 7fc4affff640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 0x7fc4b019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4af7fe640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005ab0 tx=0x7fc4a0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0041070 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b019ebf0 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a00054e0 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0034440 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.885+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b019f0e0 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.886+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b0107d40 con 0x7fc4b0100a20 2024-04-02T13:34:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.887+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc4a0044070 con 0x7fc4b0100a20 2024-04-02T13:34:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.887+0000 7fc4acff9640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 0x7fc4980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.888+0000 7fc4aeffd640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 0x7fc4980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.888+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc4a00f26a0 con 0x7fc4b0100a20 2024-04-02T13:34:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.888+0000 7fc4aeffd640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 0x7fc4980a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005d00 tx=0x7fc4a4005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:58.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:58.891+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc4a00becd0 con 0x7fc4b0100a20 2024-04-02T13:34:58.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:58 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2477740055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:58.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:58 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3389124111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:58.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:58 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/269771042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:58 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2477740055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:58 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3389124111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:58 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/269771042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.106+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4b002d530 con 0x7fc4b0100a20 2024-04-02T13:34:59.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.107+0000 7fc4acff9640 1 -- 172.21.15.122:0/1597225703 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7fc4a00be5d0 con 0x7fc4b0100a20 2024-04-02T13:34:59.110 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:59.110 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:34:59.111 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:34:59.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 msgr2=0x7fc4980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 0x7fc4980a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005d00 tx=0x7fc4a4005ed0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 msgr2=0x7fc4b0196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0005ab0 tx=0x7fc4a0005dc0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 shutdown_connections 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc4980a3bb0 0x7fc4980a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4b01025d0 0x7fc4b019e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b0101c10 0x7fc4b0197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 --2- 172.21.15.122:0/1597225703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4b0100a20 0x7fc4b0196d40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 >> 172.21.15.122:0/1597225703 conn(0x7fc4b00fbf90 msgr2=0x7fc4b0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 shutdown_connections 2024-04-02T13:34:59.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.109+0000 7fc4b5b15640 1 -- 172.21.15.122:0/1597225703 wait complete. 2024-04-02T13:34:59.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:34:59.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 -- 172.21.15.122:0/576622013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48841025f0 msgr2=0x7f4884108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 --2- 172.21.15.122:0/576622013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48841025f0 0x7f4884108e80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4874009ca0 tx=0x7f487402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 -- 172.21.15.122:0/576622013 shutdown_connections 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 --2- 172.21.15.122:0/576622013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48841025f0 0x7f4884108e80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 --2- 172.21.15.122:0/576622013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4884101c30 0x7f48841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 --2- 172.21.15.122:0/576622013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884100a40 0x7f4884100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 -- 172.21.15.122:0/576622013 >> 172.21.15.122:0/576622013 conn(0x7f48840fbfb0 msgr2=0x7f48840fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 -- 172.21.15.122:0/576622013 shutdown_connections 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 -- 172.21.15.122:0/576622013 wait complete. 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.223+0000 7f4888eb8640 1 Processor -- start 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- start start 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 0x7f4884198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48841025f0 0x7f488419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f488410c6a0 con 0x7f4884100a40 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f488410c520 con 0x7f48841025f0 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f488410c820 con 0x7f4884101c30 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35692/0 (socket says 172.21.15.122:35692) 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 -- 172.21.15.122:0/824594447 learned_addr learned my addr 172.21.15.122:0/824594447 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4881d9b640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 0x7f4884198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 -- 172.21.15.122:0/824594447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 msgr2=0x7f4884198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 0x7f4884198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 -- 172.21.15.122:0/824594447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48841025f0 msgr2=0x7f488419f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48841025f0 0x7f488419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 -- 172.21.15.122:0/824594447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488419fa70 con 0x7f4884100a40 2024-04-02T13:34:59.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4881d9b640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 0x7f4884198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f488259c640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d770 tx=0x7f487c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c017070 con 0x7f4884100a40 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f487c009c30 con 0x7f4884100a40 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c003dd0 con 0x7f4884100a40 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488419fd60 con 0x7f4884100a40 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.224+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841a01a0 con 0x7f4884100a40 2024-04-02T13:34:59.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.225+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48841a04c0 con 0x7f4884100a40 2024-04-02T13:34:59.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.226+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f487c003f70 con 0x7f4884100a40 2024-04-02T13:34:59.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.227+0000 7f487b7fe640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 0x7f48700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.227+0000 7f4881d9b640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 0x7f48700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.227+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f487c0d05f0 con 0x7f4884100a40 2024-04-02T13:34:59.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.227+0000 7f4881d9b640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 0x7f48700a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4884199360 tx=0x7f486c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.230+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f487c009de0 con 0x7f4884100a40 2024-04-02T13:34:59.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.408+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48841a0690 con 0x7f4884100a40 2024-04-02T13:34:59.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.409+0000 7f487b7fe640 1 -- 172.21.15.122:0/824594447 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f487c09cc20 con 0x7f4884100a40 2024-04-02T13:34:59.413 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:34:59.413 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":94,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:34:55.620203+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 msgr2=0x7f48700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 0x7f48700a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4884199360 tx=0x7f486c005ec0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 msgr2=0x7f4884197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d770 tx=0x7f487c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 shutdown_connections 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f48700a3bb0 0x7f48700a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48841025f0 0x7f488419f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4884101c30 0x7f4884198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 --2- 172.21.15.122:0/824594447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4884100a40 0x7f4884197e50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 >> 172.21.15.122:0/824594447 conn(0x7f48840fbfb0 msgr2=0x7f488406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 shutdown_connections 2024-04-02T13:34:59.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.411+0000 7f4888eb8640 1 -- 172.21.15.122:0/824594447 wait complete. 2024-04-02T13:34:59.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:34:59.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:34:59.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.528+0000 7f247029a640 1 -- 172.21.15.122:0/4063182375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 msgr2=0x7f2468073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.528+0000 7f247029a640 1 --2- 172.21.15.122:0/4063182375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f2468073f30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2458009ca0 tx=0x7f245802c180 comp rx=0 tx=0).stop 2024-04-02T13:34:59.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- 172.21.15.122:0/4063182375 shutdown_connections 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- 172.21.15.122:0/4063182375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24681099c0 0x7f246810be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- 172.21.15.122:0/4063182375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2468074500 0x7f2468074960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- 172.21.15.122:0/4063182375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f2468073f30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- 172.21.15.122:0/4063182375 >> 172.21.15.122:0/4063182375 conn(0x7f24680fd0a0 msgr2=0x7f24680ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- 172.21.15.122:0/4063182375 shutdown_connections 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- 172.21.15.122:0/4063182375 wait complete. 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 Processor -- start 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- start start 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 0x7f246819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 0x7f24681a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f246810f9a0 con 0x7f2468075ae0 2024-04-02T13:34:59.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f246810f820 con 0x7f24681099c0 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f247029a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f246810fb20 con 0x7f2468074500 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f246d835640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f246d835640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35718/0 (socket says 172.21.15.122:35718) 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.529+0000 7f246d835640 1 -- 172.21.15.122:0/928281752 learned_addr learned my addr 172.21.15.122:0/928281752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246e036640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 0x7f246819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246e837640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 0x7f24681a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 -- 172.21.15.122:0/928281752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 msgr2=0x7f246819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 0x7f246819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 -- 172.21.15.122:0/928281752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 msgr2=0x7f24681a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 0x7f24681a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 -- 172.21.15.122:0/928281752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24681a1cc0 con 0x7f2468075ae0 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246e837640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 0x7f24681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246e036640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 0x7f246819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:34:59.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f246d835640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f245c00d950 tx=0x7f245c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c017070 con 0x7f2468075ae0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f245c009c30 con 0x7f2468075ae0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24681a1fb0 con 0x7f2468075ae0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c00f6b0 con 0x7f2468075ae0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.530+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24681a2470 con 0x7f2468075ae0 2024-04-02T13:34:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.531+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2468108780 con 0x7f2468075ae0 2024-04-02T13:34:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.532+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f245c0109d0 con 0x7f2468075ae0 2024-04-02T13:34:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.532+0000 7f24677fe640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 0x7f24500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.533+0000 7f246e036640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 0x7f24500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.533+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f245c0d03a0 con 0x7f2468075ae0 2024-04-02T13:34:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.533+0000 7f246e036640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 0x7f24500a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2458032500 tx=0x7f2458005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.536+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f245c009de0 con 0x7f2468075ae0 2024-04-02T13:34:59.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.714+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f246802d5c0 con 0x7f2468075ae0 2024-04-02T13:34:59.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.715+0000 7f24677fe640 1 -- 172.21.15.122:0/928281752 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f245c09ca70 con 0x7f2468075ae0 2024-04-02T13:34:59.718 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:34:59.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 msgr2=0x7f24500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 0x7f24500a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2458032500 tx=0x7f2458005c50 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 msgr2=0x7f246819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f245c00d950 tx=0x7f245c00de10 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 shutdown_connections 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f24500a3bb0 0x7f24500a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24681099c0 0x7f24681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2468075ae0 0x7f246819a630 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 --2- 172.21.15.122:0/928281752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2468074500 0x7f246819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 >> 172.21.15.122:0/928281752 conn(0x7f24680fd0a0 msgr2=0x7f2468105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 shutdown_connections 2024-04-02T13:34:59.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.717+0000 7f247029a640 1 -- 172.21.15.122:0/928281752 wait complete. 2024-04-02T13:34:59.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:59 smithi122 ceph-mon[32114]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:34:59.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4090769803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.730 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1597225703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.731 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:34:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/824594447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:34:59.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 -- 172.21.15.122:0/3718088935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 msgr2=0x7fcf24109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 --2- 172.21.15.122:0/3718088935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf24109c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009ca0 tx=0x7fcf1c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 -- 172.21.15.122:0/3718088935 shutdown_connections 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 --2- 172.21.15.122:0/3718088935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf24109c40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 --2- 172.21.15.122:0/3718088935 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 0x7fcf241072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 --2- 172.21.15.122:0/3718088935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 0x7fcf24104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.833+0000 7fcf2be63640 1 -- 172.21.15.122:0/3718088935 >> 172.21.15.122:0/3718088935 conn(0x7fcf240fbfa0 msgr2=0x7fcf240fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- 172.21.15.122:0/3718088935 shutdown_connections 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- 172.21.15.122:0/3718088935 wait complete. 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 Processor -- start 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- start start 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 0x7fcf24197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 0x7fcf24198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d7b0 con 0x7fcf241077f0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d630 con 0x7fcf24104ec0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2be63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2410d930 con 0x7fcf24102590 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2a400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35750/0 (socket says 172.21.15.122:35750) 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf2a400640 1 -- 172.21.15.122:0/888300150 learned_addr learned my addr 172.21.15.122:0/888300150 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf293fe640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 0x7fcf24198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.834+0000 7fcf29bff640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 0x7fcf24197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 -- 172.21.15.122:0/888300150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 msgr2=0x7fcf24197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 0x7fcf24197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 -- 172.21.15.122:0/888300150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 msgr2=0x7fcf24198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 0x7fcf24198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 -- 172.21.15.122:0/888300150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2419fa10 con 0x7fcf241077f0 2024-04-02T13:34:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2a400640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c037040 tx=0x7fcf1c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c041020 con 0x7fcf241077f0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c034d90 con 0x7fcf241077f0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2419fca0 con 0x7fcf241077f0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c039040 con 0x7fcf241077f0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.835+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241a0190 con 0x7fcf241077f0 2024-04-02T13:34:59.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.836+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf240fda90 con 0x7fcf241077f0 2024-04-02T13:34:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.837+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fcf1c044020 con 0x7fcf241077f0 2024-04-02T13:34:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.838+0000 7fcf1affd640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 0x7fcf100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:34:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.838+0000 7fcf29bff640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 0x7fcf100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:34:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.838+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcf1c0f2740 con 0x7fcf241077f0 2024-04-02T13:34:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.839+0000 7fcf29bff640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 0x7fcf100a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009b60 tx=0x7fcf20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:34:59.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:34:59.843+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcf1c04b020 con 0x7fcf241077f0 2024-04-02T13:34:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:59 smithi187 ceph-mon[36064]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:34:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4090769803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:34:59.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1597225703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:34:59.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:34:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/824594447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:59 smithi188 ceph-mon[36117]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-04-02T13:35:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4090769803' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1597225703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:00.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:34:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/824594447' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:00.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.018+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcf240627a0 con 0x7fcf241077f0 2024-04-02T13:35:00.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.019+0000 7fcf1affd640 1 -- 172.21.15.122:0/888300150 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fcf1c0bed70 con 0x7fcf241077f0 2024-04-02T13:35:00.022 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:00.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 msgr2=0x7fcf100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 0x7fcf100a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcf20009b60 tx=0x7fcf20005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:00.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 msgr2=0x7fcf2419f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c037040 tx=0x7fcf1c032f20 comp rx=0 tx=0).stop 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 shutdown_connections 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf241077f0 0x7fcf2419f3a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf100a3bb0 0x7fcf100a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf24104ec0 0x7fcf24198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 --2- 172.21.15.122:0/888300150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf24102590 0x7fcf24197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 >> 172.21.15.122:0/888300150 conn(0x7fcf240fbfa0 msgr2=0x7fcf24102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 shutdown_connections 2024-04-02T13:35:00.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.021+0000 7fcf2be63640 1 -- 172.21.15.122:0/888300150 wait complete. 2024-04-02T13:35:00.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 -- 172.21.15.122:0/1947526751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c101bf0 msgr2=0x7f6a0c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1947526751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c102070 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009900 tx=0x7f69fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 -- 172.21.15.122:0/1947526751 shutdown_connections 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1947526751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1947526751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c102070 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1947526751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c100a00 0x7f6a0c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 -- 172.21.15.122:0/1947526751 >> 172.21.15.122:0/1947526751 conn(0x7f6a0c0fbf90 msgr2=0x7f6a0c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 -- 172.21.15.122:0/1947526751 shutdown_connections 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 -- 172.21.15.122:0/1947526751 wait complete. 2024-04-02T13:35:00.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.137+0000 7f6a13f52640 1 Processor -- start 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 -- start start 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c100a00 0x7f6a0c19a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c19a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10c680 con 0x7f6a0c1025b0 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10c500 con 0x7f6a0c101bf0 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c10c800 con 0x7f6a0c100a00 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35772/0 (socket says 172.21.15.122:35772) 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 -- 172.21.15.122:0/1324456314 learned_addr learned my addr 172.21.15.122:0/1324456314 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a11cee640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c100a00 0x7f6a0c19a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a114ed640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c19a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 -- 172.21.15.122:0/1324456314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c100a00 msgr2=0x7f6a0c19a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c100a00 0x7f6a0c19a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 -- 172.21.15.122:0/1324456314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c101bf0 msgr2=0x7f6a0c19a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c19a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 -- 172.21.15.122:0/1324456314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c1a1c30 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a124ef640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400eaa0 tx=0x7f6a0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400cdb0 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c1a1f20 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a04005020 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.138+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a040116d0 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.139+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1a2430 con 0x7f6a0c1025b0 2024-04-02T13:35:00.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.139+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c1a26f0 con 0x7f6a0c1025b0 2024-04-02T13:35:00.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.141+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f6a040054d0 con 0x7f6a0c1025b0 2024-04-02T13:35:00.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.141+0000 7f6a02ffd640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 0x7f69f80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.141+0000 7f6a11cee640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 0x7f69f80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.141+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f6a040d0e00 con 0x7f6a0c1025b0 2024-04-02T13:35:00.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.142+0000 7f6a11cee640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 0x7f69f80a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a08005fd0 tx=0x7f6a08005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.145+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f6a04016030 con 0x7f6a0c1025b0 2024-04-02T13:35:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.324+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a0c1a28c0 con 0x7f6a0c1025b0 2024-04-02T13:35:00.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.324+0000 7f6a02ffd640 1 -- 172.21.15.122:0/1324456314 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f6a0409d430 con 0x7f6a0c1025b0 2024-04-02T13:35:00.328 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 msgr2=0x7f69f80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 0x7f69f80a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6a08005fd0 tx=0x7f6a08005850 comp rx=0 tx=0).stop 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 msgr2=0x7f6a0c1a15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400eaa0 tx=0x7f6a0400ef60 comp rx=0 tx=0).stop 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 shutdown_connections 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a0c1025b0 0x7f6a0c1a15c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f69f80a3bb0 0x7f69f80a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a0c101bf0 0x7f6a0c19a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 --2- 172.21.15.122:0/1324456314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a0c100a00 0x7f6a0c19a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 >> 172.21.15.122:0/1324456314 conn(0x7f6a0c0fbf90 msgr2=0x7f6a0c068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 shutdown_connections 2024-04-02T13:35:00.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.326+0000 7f6a13f52640 1 -- 172.21.15.122:0/1324456314 wait complete. 2024-04-02T13:35:00.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 -- 172.21.15.122:0/3490586946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a8101c10 msgr2=0x7f26a8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 --2- 172.21.15.122:0/3490586946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a8101c10 0x7f26a8102090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f269c009900 tx=0x7f269c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 -- 172.21.15.122:0/3490586946 shutdown_connections 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 --2- 172.21.15.122:0/3490586946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a81025d0 0x7f26a8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 --2- 172.21.15.122:0/3490586946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a8101c10 0x7f26a8102090 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 --2- 172.21.15.122:0/3490586946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8100a20 0x7f26a8100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 -- 172.21.15.122:0/3490586946 >> 172.21.15.122:0/3490586946 conn(0x7f26a80fbfb0 msgr2=0x7f26a80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 -- 172.21.15.122:0/3490586946 shutdown_connections 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.441+0000 7f26aeea5640 1 -- 172.21.15.122:0/3490586946 wait complete. 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 Processor -- start 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 -- start start 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a8100a20 0x7f26a8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8101c10 0x7f26a81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26a810c6a0 con 0x7f26a81025d0 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26a810c520 con 0x7f26a8101c10 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26aeea5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26a810c820 con 0x7f26a8100a20 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35794/0 (socket says 172.21.15.122:35794) 2024-04-02T13:35:00.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 -- 172.21.15.122:0/1964610789 learned_addr learned my addr 172.21.15.122:0/1964610789 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26acc41640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a8100a20 0x7f26a8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26a7fff640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8101c10 0x7f26a81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 -- 172.21.15.122:0/1964610789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a8100a20 msgr2=0x7f26a8197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a8100a20 0x7f26a8197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 -- 172.21.15.122:0/1964610789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8101c10 msgr2=0x7f26a81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8101c10 0x7f26a81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 -- 172.21.15.122:0/1964610789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a819fa80 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.442+0000 7f26ad442640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f26a000eaa0 tx=0x7f26a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.443+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a000cdb0 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.443+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a819fd70 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.443+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a0005020 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.443+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a00116d0 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.443+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a81a0230 con 0x7f26a81025d0 2024-04-02T13:35:00.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.444+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a81a04c0 con 0x7f26a81025d0 2024-04-02T13:35:00.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.445+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f26a00054d0 con 0x7f26a81025d0 2024-04-02T13:35:00.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.445+0000 7f26a5ffb640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 0x7f26900a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.445+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f26a00d0970 con 0x7f26a81025d0 2024-04-02T13:35:00.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.445+0000 7f26acc41640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 0x7f26900a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.446+0000 7f26acc41640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 0x7f26900a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2698005fd0 tx=0x7f269800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.450+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f26a0016030 con 0x7f26a81025d0 2024-04-02T13:35:00.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.628+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f26a81a0690 con 0x7f26a81025d0 2024-04-02T13:35:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.628+0000 7f26a5ffb640 1 -- 172.21.15.122:0/1964610789 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f26a009cfa0 con 0x7f26a81025d0 2024-04-02T13:35:00.632 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 msgr2=0x7f26900a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 0x7f26900a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2698005fd0 tx=0x7f269800b410 comp rx=0 tx=0).stop 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 msgr2=0x7f26a819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f26a000eaa0 tx=0x7f26a000ef60 comp rx=0 tx=0).stop 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 shutdown_connections 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26a81025d0 0x7f26a819f410 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f26900a3bb0 0x7f26900a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26a8101c10 0x7f26a81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 --2- 172.21.15.122:0/1964610789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a8100a20 0x7f26a8197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 >> 172.21.15.122:0/1964610789 conn(0x7f26a80fbfb0 msgr2=0x7f26a80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 shutdown_connections 2024-04-02T13:35:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.630+0000 7f26aeea5640 1 -- 172.21.15.122:0/1964610789 wait complete. 2024-04-02T13:35:00.646 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 -- 172.21.15.122:0/4057496392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0100a20 msgr2=0x7fc7d0100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 --2- 172.21.15.122:0/4057496392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0100a20 0x7fc7d0100e40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009900 tx=0x7fc7c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 -- 172.21.15.122:0/4057496392 shutdown_connections 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 --2- 172.21.15.122:0/4057496392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 0x7fc7d0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 --2- 172.21.15.122:0/4057496392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0101c10 0x7fc7d0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 --2- 172.21.15.122:0/4057496392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0100a20 0x7fc7d0100e40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 -- 172.21.15.122:0/4057496392 >> 172.21.15.122:0/4057496392 conn(0x7fc7d00fbf90 msgr2=0x7fc7d00fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 -- 172.21.15.122:0/4057496392 shutdown_connections 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.745+0000 7fc7d619d640 1 -- 172.21.15.122:0/4057496392 wait complete. 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 Processor -- start 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 -- start start 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 0x7fc7d00697d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 0x7fc7d01a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010c6a0 con 0x7fc7d0101c10 2024-04-02T13:35:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010c520 con 0x7fc7d0100a20 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7d619d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d010c820 con 0x7fc7d01025d0 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7c6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7c6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35810/0 (socket says 172.21.15.122:35810) 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.746+0000 7fc7c6ffd640 1 -- 172.21.15.122:0/1202426671 learned_addr learned my addr 172.21.15.122:0/1202426671 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cf7fe640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 0x7fc7d00697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cffff640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 0x7fc7d01a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 -- 172.21.15.122:0/1202426671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 msgr2=0x7fc7d01a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 0x7fc7d01a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 -- 172.21.15.122:0/1202426671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 msgr2=0x7fc7d00697d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 0x7fc7d00697d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 -- 172.21.15.122:0/1202426671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d01a1cd0 con 0x7fc7d0101c10 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cf7fe640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 0x7fc7d00697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7c6ffd640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c890 tx=0x7fc7b800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8004220 con 0x7fc7d0101c10 2024-04-02T13:35:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b80043c0 con 0x7fc7d0101c10 2024-04-02T13:35:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d01a1f60 con 0x7fc7d0101c10 2024-04-02T13:35:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80049e0 con 0x7fc7d0101c10 2024-04-02T13:35:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.747+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d01a2420 con 0x7fc7d0101c10 2024-04-02T13:35:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.748+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d0107dc0 con 0x7fc7d0101c10 2024-04-02T13:35:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.749+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fc7b80079b0 con 0x7fc7d0101c10 2024-04-02T13:35:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.750+0000 7fc7cd7fa640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 0x7fc7bc0a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.750+0000 7fc7cf7fe640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 0x7fc7bc0a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.750+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fc7b80d0830 con 0x7fc7d0101c10 2024-04-02T13:35:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.750+0000 7fc7cf7fe640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 0x7fc7bc0a60a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c002c7b0 tx=0x7fc7c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:00.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.754+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc7b8014030 con 0x7fc7d0101c10 2024-04-02T13:35:00.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.971+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7d00627a0 con 0x7fc7d0101c10 2024-04-02T13:35:00.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.972+0000 7fc7cd7fa640 1 -- 172.21.15.122:0/1202426671 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11942 (secure 0 0 0) 0x7fc7b809ce60 con 0x7fc7d0101c10 2024-04-02T13:35:00.975 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:00.975 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:33:12.155913+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:35:00.976 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 27 2024-04-02T13:35:00.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 msgr2=0x7fc7bc0a60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 0x7fc7bc0a60a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c002c7b0 tx=0x7fc7c0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 msgr2=0x7fc7d019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800c890 tx=0x7fc7b800cd50 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 shutdown_connections 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7d01025d0 0x7fc7d01a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7d0101c10 0x7fc7d019ef90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fc7bc0a3be0 0x7fc7bc0a60a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 --2- 172.21.15.122:0/1202426671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7d0100a20 0x7fc7d00697d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 >> 172.21.15.122:0/1202426671 conn(0x7fc7d00fbf90 msgr2=0x7fc7d0104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 shutdown_connections 2024-04-02T13:35:00.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:00.974+0000 7fc7d619d640 1 -- 172.21.15.122:0/1202426671 wait complete. 2024-04-02T13:35:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/928281752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/888300150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1324456314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1964610789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/928281752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/888300150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1324456314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.987 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1964610789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:00.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/928281752' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/888300150' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1324456314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:01.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1964610789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 -- 172.21.15.122:0/2508180786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c104ed0 msgr2=0x7f4c8c1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 --2- 172.21.15.122:0/2508180786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c1072c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009900 tx=0x7f4c7c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 -- 172.21.15.122:0/2508180786 shutdown_connections 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 --2- 172.21.15.122:0/2508180786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c107800 0x7f4c8c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 --2- 172.21.15.122:0/2508180786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c1072c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 --2- 172.21.15.122:0/2508180786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 -- 172.21.15.122:0/2508180786 >> 172.21.15.122:0/2508180786 conn(0x7f4c8c0fbf90 msgr2=0x7f4c8c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 -- 172.21.15.122:0/2508180786 shutdown_connections 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.085+0000 7f4c945a8640 1 -- 172.21.15.122:0/2508180786 wait complete. 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 Processor -- start 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 -- start start 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c107800 0x7f4c8c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10d870 con 0x7f4c8c1025a0 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10d6f0 con 0x7f4c8c107800 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c945a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10d9f0 con 0x7f4c8c104ed0 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35824/0 (socket says 172.21.15.122:35824) 2024-04-02T13:35:01.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 -- 172.21.15.122:0/897027644 learned_addr learned my addr 172.21.15.122:0/897027644 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c91b43640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92b45640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c107800 0x7f4c8c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 -- 172.21.15.122:0/897027644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c104ed0 msgr2=0x7f4c8c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 -- 172.21.15.122:0/897027644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c107800 msgr2=0x7f4c8c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c107800 0x7f4c8c19f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.086+0000 7f4c92344640 1 -- 172.21.15.122:0/897027644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19fb30 con 0x7f4c8c1025a0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c92344640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b4a0 tx=0x7f4c8800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88014070 con 0x7f4c8c1025a0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c88010af0 con 0x7f4c8c1025a0 2024-04-02T13:35:01.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c19fe20 con 0x7f4c8c1025a0 2024-04-02T13:35:01.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c88017d10 con 0x7f4c8c1025a0 2024-04-02T13:35:01.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.087+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a0280 con 0x7f4c8c1025a0 2024-04-02T13:35:01.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.088+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c0fda80 con 0x7f4c8c1025a0 2024-04-02T13:35:01.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.089+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4c88017070 con 0x7f4c8c1025a0 2024-04-02T13:35:01.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.089+0000 7f4c837fe640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 0x7f4c780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.089+0000 7f4c91b43640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 0x7f4c780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.089+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4c880cfbd0 con 0x7f4c8c1025a0 2024-04-02T13:35:01.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.090+0000 7f4c91b43640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 0x7f4c780a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c02c7b0 tx=0x7f4c7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.094+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4c8809c200 con 0x7f4c8c1025a0 2024-04-02T13:35:01.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.274+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8c062840 con 0x7f4c8c1025a0 2024-04-02T13:35:01.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.276+0000 7f4c837fe640 1 -- 172.21.15.122:0/897027644 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f4c8809bb00 con 0x7f4c8c1025a0 2024-04-02T13:35:01.280 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:01.280 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":94,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:34:55.620203+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:01.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 msgr2=0x7f4c780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 0x7f4c780a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c02c7b0 tx=0x7f4c7c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 msgr2=0x7f4c8c197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b4a0 tx=0x7f4c8800b960 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 shutdown_connections 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c780a3bb0 0x7f4c780a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c8c107800 0x7f4c8c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c8c104ed0 0x7f4c8c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 --2- 172.21.15.122:0/897027644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c8c1025a0 0x7f4c8c197f10 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 >> 172.21.15.122:0/897027644 conn(0x7f4c8c0fbf90 msgr2=0x7f4c8c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 shutdown_connections 2024-04-02T13:35:01.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.278+0000 7f4c945a8640 1 -- 172.21.15.122:0/897027644 wait complete. 2024-04-02T13:35:01.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-02T13:32:49.884932+0000', 'modified': '2024-04-02T13:33:12.155913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24512, 'mds_1': 24487, 'mds_2': 24485, 'mds_3': 24460, 'mds_4': 14628}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/1672540384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1672540384}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1672540384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.122:6841/385777794', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6840', 'nonce': 385777794}, {'type': 'v1', 'addr': '172.21.15.122:6841', 'nonce': 385777794}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6833/546075479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 546075479}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 546075479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6833/1223173792', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 1223173792}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 1223173792}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.188:6835/179772221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 179772221}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 179772221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.187:6835/4231455845', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6834', 'nonce': 4231455845}, {'type': 'v1', 'addr': '172.21.15.187:6835', 'nonce': 4231455845}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6837/391935088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6836', 'nonce': 391935088}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 391935088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.187:6837/453335780', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6836', 'nonce': 453335780}, {'type': 'v1', 'addr': '172.21.15.187:6837', 'nonce': 453335780}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.188:6839/180395175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6838', 'nonce': 180395175}, {'type': 'v1', 'addr': '172.21.15.188:6839', 'nonce': 180395175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6839/2503909279', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6838', 'nonce': 2503909279}, {'type': 'v1', 'addr': '172.21.15.187:6839', 'nonce': 2503909279}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24512, 'qdb_cluster': [24485, 24487, 24460, 24512, 14628]} 2024-04-02T13:35:01.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-02T13:35:01.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-02T13:35:01.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 -- 172.21.15.122:0/154258191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b80ffde0 msgr2=0x7fe0b8100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 --2- 172.21.15.122:0/154258191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8100200 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009930 tx=0x7fe0a802bc20 comp rx=0 tx=0).stop 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 -- 172.21.15.122:0/154258191 shutdown_connections 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 --2- 172.21.15.122:0/154258191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b8103770 0x7fe0b810a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 --2- 172.21.15.122:0/154258191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 0x7fe0b8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 --2- 172.21.15.122:0/154258191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8100200 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 -- 172.21.15.122:0/154258191 >> 172.21.15.122:0/154258191 conn(0x7fe0b80fb710 msgr2=0x7fe0b80fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 -- 172.21.15.122:0/154258191 shutdown_connections 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.393+0000 7fe0bca82640 1 -- 172.21.15.122:0/154258191 wait complete. 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 Processor -- start 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 -- start start 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 0x7fe0b8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b810d7b0 con 0x7fe0b8103770 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b810d630 con 0x7fe0b80ffde0 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0bca82640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b810d930 con 0x7fe0b8102db0 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35838/0 (socket says 172.21.15.122:35838) 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 -- 172.21.15.122:0/3724813627 learned_addr learned my addr 172.21.15.122:0/3724813627 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 0x7fe0b8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 -- 172.21.15.122:0/3724813627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 msgr2=0x7fe0b8198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 0x7fe0b8198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 -- 172.21.15.122:0/3724813627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b80ffde0 msgr2=0x7fe0b8197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 -- 172.21.15.122:0/3724813627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b819fa70 con 0x7fe0b8103770 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.394+0000 7fe0b6d9d640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000cce0 tx=0x7fe0b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.395+0000 7fe0a77fe640 1 -- 172.21.15.122:0/3724813627 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000f450 con 0x7fe0b8103770 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.395+0000 7fe0a77fe640 1 -- 172.21.15.122:0/3724813627 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b000fad0 con 0x7fe0b8103770 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.395+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b819fd60 con 0x7fe0b8103770 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.395+0000 7fe0a77fe640 1 -- 172.21.15.122:0/3724813627 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b0018c10 con 0x7fe0b8103770 2024-04-02T13:35:01.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.395+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b81a0250 con 0x7fe0b8103770 2024-04-02T13:35:01.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.397+0000 7fe0a77fe640 1 -- 172.21.15.122:0/3724813627 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fe0b000f5f0 con 0x7fe0b8103770 2024-04-02T13:35:01.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.397+0000 7fe0bf65f740 1 Processor -- start 2024-04-02T13:35:01.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 -- start start 2024-04-02T13:35:01.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beddf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a98be42060 con 0x55a98bee3640 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a98bdea650 con 0x55a98bede4d0 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a98be3e010 con 0x55a98beddae0 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35840/0 (socket says 172.21.15.122:35840) 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/1918603102 learned_addr learned my addr 172.21.15.122:0/1918603102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0a77fe640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 0x7fe0a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b659c640 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b6d9d640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 0x7fe0a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/1918603102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 msgr2=0x55a98beddf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beddf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/1918603102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 msgr2=0x55a98bee2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/1918603102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98bb562d0 con 0x55a98bee3640 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.398+0000 7fe0a77fe640 1 -- 172.21.15.122:0/3724813627 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe0b00d0eb0 con 0x7fe0b8103770 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00bcf0 tx=0x7fe0ac011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48c4099d59cf570 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0a4ff9640 1 -- 172.21.15.122:0/1918603102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac01a400 con 0x55a98bee3640 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0a4ff9640 1 -- 172.21.15.122:0/1918603102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac01aa80 con 0x55a98bee3640 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0a4ff9640 1 -- 172.21.15.122:0/1918603102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac012af0 con 0x55a98bee3640 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0b6d9d640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 0x7fe0a00a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00ce970 tx=0x7fe0b00922c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 -- 172.21.15.122:0/1918603102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 msgr2=0x55a98bee5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00bcf0 tx=0x7fe0ac011750 comp rx=0 tx=0).stop 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 -- 172.21.15.122:0/1918603102 shutdown_connections 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bee5ad0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/1918603102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beddf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.399+0000 7fe0bf65f740 1 -- 172.21.15.122:0/1918603102 >> 172.21.15.122:0/1918603102 conn(0x55a98bed9050 msgr2=0x55a98bed9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- 172.21.15.122:0/1918603102 shutdown_connections 2024-04-02T13:35:01.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- 172.21.15.122:0/1918603102 wait complete. 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- start start 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 Processor -- start 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98bee3640 0x55a98beef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a98bee65f0 con 0x55a98beddae0 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a98bdfda10 con 0x55a98bede4d0 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a98bee6770 con 0x55a98bee3640 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0bf65f740 10 client.0 _ll_register_callbacks cb 0x55a98be3a0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35850/0 (socket says 172.21.15.122:35850) 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3687273173 learned_addr learned my addr 172.21.15.122:0/3687273173 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3687273173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98bee3640 msgr2=0x55a98beef570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98bee3640 0x55a98beef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.400+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3687273173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 msgr2=0x55a98bee8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3687273173 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98bc206b0 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac01ac30 tx=0x7fe0ac01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965ecd8d7a993930 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac01a810 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe0ac012870 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac01aa80 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac030050 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 10 client.15426 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a98bef28e0 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac01aa80 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe0ac00fbc0 con 0x55a98beddae0 2024-04-02T13:35:01.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 10 client.15426 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 20 client.15426 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bef4ae0 0x55a98bef5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 4 client.15426 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.401+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a98bde9cf0 con 0x55a98bef4ae0 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bef4ae0 0x55a98bef5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bef4ae0 0x55a98bef5f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3388c0e9e069c4e server_cookie=fe3985a36c0722ab in_seq=0 out_seq=0 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe09dffb640 10 client.15426 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe09dffb640 1 -- 172.21.15.122:0/3687273173 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a98bde9cf0 con 0x55a98bef4ae0 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe09dffb640 10 client.15426 handle_command_reply: tid=0 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe0bf65f740 1 client.15426 shutdown 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 msgr2=0x55a98bee7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.402+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac01ac30 tx=0x7fe0ac01ac60 comp rx=0 tx=0).stop 2024-04-02T13:35:01.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 shutdown_connections 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bef4ae0 0x55a98bef5f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98bee3640 0x55a98beef570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe09dffb640 0 client.15426 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3687273173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98beddae0 0x55a98bee7ff0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 >> 172.21.15.122:0/3687273173 conn(0x55a98bed9050 msgr2=0x55a98bed8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 shutdown_connections 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3687273173 wait complete. 2024-04-02T13:35:01.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.403+0000 7fe0bf65f740 20 client.15426 trim_cache size 0 max 16384 2024-04-02T13:35:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 Processor -- start 2024-04-02T13:35:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 -- start start 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98bedeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef4290 con 0x55a98bee3640 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef4960 con 0x55a98bede4d0 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef5030 con 0x55a98beddae0 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35866/0 (socket says 172.21.15.122:35866) 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3090095221 learned_addr learned my addr 172.21.15.122:0/3090095221 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3090095221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 msgr2=0x55a98bedeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98bedeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3090095221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 msgr2=0x55a98bee5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/3090095221 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a98bede8b0 con 0x55a98bee3640 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b659c640 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0058c0 tx=0x7fe0ac00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4370ddb3c0143c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe09ffff640 1 -- 172.21.15.122:0/3090095221 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac01aa80 con 0x55a98bee3640 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe09ffff640 1 -- 172.21.15.122:0/3090095221 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac01acd0 con 0x55a98bee3640 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.421+0000 7fe09ffff640 1 -- 172.21.15.122:0/3090095221 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0277b0 con 0x55a98bee3640 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3090095221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 msgr2=0x55a98bed9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0058c0 tx=0x7fe0ac00b670 comp rx=0 tx=0).stop 2024-04-02T13:35:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3090095221 shutdown_connections 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98bed9430 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98bee5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/3090095221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98bedeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3090095221 >> 172.21.15.122:0/3090095221 conn(0x55a98bed9050 msgr2=0x55a98bee6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3090095221 shutdown_connections 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- 172.21.15.122:0/3090095221 wait complete. 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 -- start start 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.422+0000 7fe0bf65f740 1 Processor -- start 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98beeac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef5dd0 con 0x55a98bee3640 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef5700 con 0x55a98bede4d0 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a98bef64a0 con 0x55a98beddae0 2024-04-02T13:35:01.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 10 client.0 _ll_register_callbacks cb 0x7fe0ac00bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35870/0 (socket says 172.21.15.122:35870) 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/517990151 learned_addr learned my addr 172.21.15.122:0/517990151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b659c640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98beeac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/517990151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 msgr2=0x55a98beea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/517990151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 msgr2=0x55a98beeac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98beeac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 -- 172.21.15.122:0/517990151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a98beee260 con 0x55a98bee3640 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0b5d9b640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00b7e0 tx=0x7fe0ac01a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9beb5db703002760 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe09dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00db60 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe0ac00dd00 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac01acd0 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.423+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe0ac027530 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 10 client.15432 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a98bf5b1a0 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac01acd0 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe0ac033070 con 0x55a98bee3640 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 10 client.15432 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 10 client.15432 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 20 client.15432 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bf5f9a0 0x55a98bf5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 4 client.15432 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "status"}) -- 0x55a98be4f5a0 con 0x55a98bf5f9a0 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0b659c640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bf5f9a0 0x55a98bf5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe09dffb640 10 client.15432 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0b659c640 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bf5f9a0 0x55a98bf5c4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=659949fb4f2c8403 server_cookie=1076ca0a0bb9d76e in_seq=0 out_seq=0 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe09dffb640 1 -- 172.21.15.122:0/517990151 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55a98be4f5a0 con 0x55a98bf5f9a0 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe09dffb640 10 client.15432 handle_command_reply: tid=0 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.424+0000 7fe0bf65f740 1 client.15432 shutdown 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 msgr2=0x55a98beedc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00b7e0 tx=0x7fe0ac01a890 comp rx=0 tx=0).stop 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 shutdown_connections 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a98bf5f9a0 0x55a98bf5c4b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a98bee3640 0x55a98beedc70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe09dffb640 0 client.15432 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a98bede4d0 0x55a98beeac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 --2- 172.21.15.122:0/517990151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a98beddae0 0x55a98beea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 >> 172.21.15.122:0/517990151 conn(0x55a98bed9050 msgr2=0x55a98bee5a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 shutdown_connections 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 1 -- 172.21.15.122:0/517990151 wait complete. 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bf65f740 20 client.15432 trim_cache size 0 max 16384 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 4, 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 14628, 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 110.27525469, 2024-04-02T13:35:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:35:01.438 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 94, 2024-04-02T13:35:01.438 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:35:01.438 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 138.04654925599999 2024-04-02T13:35:01.438 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 msgr2=0x7fe0a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 0x7fe0a00a60f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00ce970 tx=0x7fe0b00922c0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 msgr2=0x7fe0b819f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000cce0 tx=0x7fe0b000a620 comp rx=0 tx=0).stop 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 shutdown_connections 2024-04-02T13:35:01.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0b8103770 0x7fe0b819f400 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0b8102db0 0x7fe0b8198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fe0a00a3c30 0x7fe0a00a60f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 --2- 172.21.15.122:0/3724813627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0b80ffde0 0x7fe0b8197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 >> 172.21.15.122:0/3724813627 conn(0x7fe0b80fb710 msgr2=0x7fe0b8106130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 shutdown_connections 2024-04-02T13:35:01.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.425+0000 7fe0bca82640 1 -- 172.21.15.122:0/3724813627 wait complete. 2024-04-02T13:35:01.440 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-02T13:35:01.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 -- 172.21.15.122:0/1014107567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58741025b0 msgr2=0x7f5874108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 --2- 172.21.15.122:0/1014107567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58741025b0 0x7f5874108e40 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f586c009ca0 tx=0x7f586c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 -- 172.21.15.122:0/1014107567 shutdown_connections 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 --2- 172.21.15.122:0/1014107567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58741025b0 0x7f5874108e40 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 --2- 172.21.15.122:0/1014107567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5874101bf0 0x7f5874102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.539+0000 7f587a4a5640 1 --2- 172.21.15.122:0/1014107567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874100a00 0x7f5874100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- 172.21.15.122:0/1014107567 >> 172.21.15.122:0/1014107567 conn(0x7f58740fbf90 msgr2=0x7f58740fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- 172.21.15.122:0/1014107567 shutdown_connections 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- 172.21.15.122:0/1014107567 wait complete. 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 Processor -- start 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- start start 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 0x7f5874198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58741025b0 0x7f587419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f587410c770 con 0x7f5874100a00 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f587410c5f0 con 0x7f58741025b0 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.540+0000 7f587a4a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f587410c8f0 con 0x7f5874101bf0 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35892/0 (socket says 172.21.15.122:35892) 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 -- 172.21.15.122:0/746042318 learned_addr learned my addr 172.21.15.122:0/746042318 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f58737fe640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 0x7f5874198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 -- 172.21.15.122:0/746042318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 msgr2=0x7f5874198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 0x7f5874198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 -- 172.21.15.122:0/746042318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58741025b0 msgr2=0x7f587419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5878a42640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58741025b0 0x7f587419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58741025b0 0x7f587419f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 -- 172.21.15.122:0/746042318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587419fb20 con 0x7f5874100a00 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f58737fe640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 0x7f5874198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f5873fff640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b450 tx=0x7f585c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f58717fa640 1 -- 172.21.15.122:0/746042318 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c015020 con 0x7f5874100a00 2024-04-02T13:35:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f58717fa640 1 -- 172.21.15.122:0/746042318 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c011d70 con 0x7f5874100a00 2024-04-02T13:35:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f58717fa640 1 -- 172.21.15.122:0/746042318 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c00f340 con 0x7f5874100a00 2024-04-02T13:35:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587419fe10 con 0x7f5874100a00 2024-04-02T13:35:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.541+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58741a02f0 con 0x7f5874100a00 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f58717fa640 1 -- 172.21.15.122:0/746042318 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f585c018020 con 0x7f5874100a00 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 Processor -- start 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 -- start start 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 0x559f89033fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89036ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f88f93060 con 0x559f8902eae0 2024-04-02T13:35:01.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f88f3b650 con 0x559f8902f4d0 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f587d16e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f88f8f010 con 0x559f89034640 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f5878a42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f5878a42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35894/0 (socket says 172.21.15.122:35894) 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.544+0000 7f5878a42640 1 -- 172.21.15.122:0/2112070323 learned_addr learned my addr 172.21.15.122:0/2112070323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5873fff640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 0x559f89033fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f58737fe640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89036ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 -- 172.21.15.122:0/2112070323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 msgr2=0x559f89036ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89036ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 -- 172.21.15.122:0/2112070323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 msgr2=0x559f89033fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f58717fa640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 0x7f58600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 0x559f89033fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 -- 172.21.15.122:0/2112070323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f88ca72d0 con 0x559f8902eae0 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f58737fe640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89036ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5873fff640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 0x559f89033fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f58717fa640 1 -- 172.21.15.122:0/746042318 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f585c0d02e0 con 0x7f5874100a00 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f586c009dc0 tx=0x7f586c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25cf999229dbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f5878a42640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 0x7f58600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f586a7fc640 1 -- 172.21.15.122:0/2112070323 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c042020 con 0x559f8902eae0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f586a7fc640 1 -- 172.21.15.122:0/2112070323 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586c003970 con 0x559f8902eae0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.545+0000 7f586a7fc640 1 -- 172.21.15.122:0/2112070323 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c039040 con 0x559f8902eae0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- 172.21.15.122:0/2112070323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 msgr2=0x559f8902ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f5878a42640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 0x7f58600a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f586c040000 tx=0x7f586c034440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f586c009dc0 tx=0x7f586c009800 comp rx=0 tx=0).stop 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- 172.21.15.122:0/2112070323 shutdown_connections 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89036ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902f4d0 0x559f89033fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 --2- 172.21.15.122:0/2112070323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f8902ef00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- 172.21.15.122:0/2112070323 >> 172.21.15.122:0/2112070323 conn(0x559f8902a050 msgr2=0x559f8902a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- 172.21.15.122:0/2112070323 shutdown_connections 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- 172.21.15.122:0/2112070323 wait complete. 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 -- start start 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.546+0000 7f587d16e740 1 Processor -- start 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902eae0 0x559f89038ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89040450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f89037590 con 0x559f8902f4d0 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f88f4ea10 con 0x559f8902eae0 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f89037710 con 0x559f89034640 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 10 client.0 _ll_register_callbacks cb 0x559f88f8b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35900/0 (socket says 172.21.15.122:35900) 2024-04-02T13:35:01.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 -- 172.21.15.122:0/709761358 learned_addr learned my addr 172.21.15.122:0/709761358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 -- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 msgr2=0x559f89040450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89040450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 -- 172.21.15.122:0/709761358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902eae0 msgr2=0x559f89038ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902eae0 0x559f89038ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 -- 172.21.15.122:0/709761358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f88d716b0 con 0x559f8902f4d0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58737fe640 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f586400d270 tx=0x7f586400d730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb3ac319b1155c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58537fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864016070 con 0x559f8902f4d0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f5864012870 con 0x559f8902f4d0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f586c003970 con 0x559f8902f4d0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5864022050 con 0x559f8902f4d0 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 10 client.15444 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.547+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f89043800 con 0x559f8902f4d0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f586c003970 con 0x559f8902f4d0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f5864011b90 con 0x559f8902f4d0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 10 client.15444 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 20 client.15444 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f89045f20 0x559f89047390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 4 client.15444 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f88f3acf0 con 0x559f89045f20 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f5873fff640 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f89045f20 0x559f89047390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f58537fe640 10 client.15444 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.548+0000 7f5873fff640 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f89045f20 0x559f89047390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=197521b9cc1e4988 server_cookie=2eeed83ac9b1657f in_seq=0 out_seq=0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f58537fe640 1 -- 172.21.15.122:0/709761358 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f88f3acf0 con 0x559f89045f20 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f58537fe640 10 client.15444 handle_command_reply: tid=0 2024-04-02T13:35:01.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 client.15444 shutdown 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 msgr2=0x559f89039410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f586400d270 tx=0x7f586400d730 comp rx=0 tx=0).stop 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 shutdown_connections 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f89034640 0x559f89040450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f89045f20 0x559f89047390 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f89039410 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 --2- 172.21.15.122:0/709761358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f8902eae0 0x559f89038ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f58537fe640 0 client.15444 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 >> 172.21.15.122:0/709761358 conn(0x559f8902a050 msgr2=0x559f89029010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.549+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 shutdown_connections 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.550+0000 7f587d16e740 1 -- 172.21.15.122:0/709761358 wait complete. 2024-04-02T13:35:01.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.550+0000 7f587d16e740 20 client.15444 trim_cache size 0 max 16384 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 Processor -- start 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 -- start start 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902f4d0 0x559f89040410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f89037ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f890456d0 con 0x559f8902eae0 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f89045da0 con 0x559f89034640 2024-04-02T13:35:01.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f587d16e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f89046470 con 0x559f8902f4d0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35904/0 (socket says 172.21.15.122:35904) 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 -- 172.21.15.122:0/700849239 learned_addr learned my addr 172.21.15.122:0/700849239 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 -- 172.21.15.122:0/700849239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902f4d0 msgr2=0x559f89040410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902f4d0 0x559f89040410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 -- 172.21.15.122:0/700849239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 msgr2=0x559f89037ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f89037ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 -- 172.21.15.122:0/700849239 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f89045500 con 0x559f8902eae0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5873fff640 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f585c098be0 tx=0x7f585c010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e187935b119be2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5869ffb640 1 -- 172.21.15.122:0/700849239 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0a4e80 con 0x559f8902eae0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5869ffb640 1 -- 172.21.15.122:0/700849239 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c009200 con 0x559f8902eae0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.568+0000 7f5869ffb640 1 -- 172.21.15.122:0/700849239 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0a0120 con 0x559f8902eae0 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- 172.21.15.122:0/700849239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 msgr2=0x559f89036a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f585c098be0 tx=0x7f585c010970 comp rx=0 tx=0).stop 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- 172.21.15.122:0/700849239 shutdown_connections 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f89037ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902f4d0 0x559f89040410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 --2- 172.21.15.122:0/700849239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902eae0 0x559f89036a30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- 172.21.15.122:0/700849239 >> 172.21.15.122:0/700849239 conn(0x559f8902a050 msgr2=0x559f89024e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- 172.21.15.122:0/700849239 shutdown_connections 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- 172.21.15.122:0/700849239 wait complete. 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.569+0000 7f587d16e740 1 -- start start 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 Processor -- start 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902eae0 0x559f8903ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f8903ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f89047210 con 0x559f8902f4d0 2024-04-02T13:35:01.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f89046b40 con 0x559f89034640 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f890478e0 con 0x559f8902eae0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 10 client.0 _ll_register_callbacks cb 0x7f586400b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35908/0 (socket says 172.21.15.122:35908) 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 -- 172.21.15.122:0/1670576679 learned_addr learned my addr 172.21.15.122:0/1670576679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 -- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902eae0 msgr2=0x559f8903ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902eae0 0x559f8903ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 -- 172.21.15.122:0/1670576679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 msgr2=0x559f8903ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f8903ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 -- 172.21.15.122:0/1670576679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f8903f580 con 0x559f8902f4d0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58737fe640 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f586400d240 tx=0x7f586401a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f5183e3c436f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58537fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864016040 con 0x559f8902f4d0 2024-04-02T13:35:01.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f586401ad80 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c009200 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f586400de80 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 10 client.15450 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.570+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559f890abed0 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c009200 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f586401b440 con 0x559f8902f4d0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 10 client.15450 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 10 client.15450 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 20 client.15450 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f890b0c00 0x559f890af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 4 client.15450 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "status"}) -- 0x559f88fa05a0 con 0x559f890b0c00 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f5873fff640 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f890b0c00 0x559f890af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f5873fff640 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f890b0c00 0x559f890af910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b75591b1c84cfcb3 server_cookie=781815df8a297489 in_seq=0 out_seq=0 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.571+0000 7f58537fe640 10 client.15450 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f58537fe640 1 -- 172.21.15.122:0/1670576679 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559f88fa05a0 con 0x559f890b0c00 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f58537fe640 10 client.15450 handle_command_reply: tid=0 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 client.15450 shutdown 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 msgr2=0x559f8903bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f586400d240 tx=0x7f586401a850 comp rx=0 tx=0).stop 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 shutdown_connections 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f89034640 0x559f8903ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559f890b0c00 0x559f890af910 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f8902f4d0 0x559f8903bf70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 --2- 172.21.15.122:0/1670576679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f8902eae0 0x559f8903ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f58537fe640 0 client.15450 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 >> 172.21.15.122:0/1670576679 conn(0x559f8902a050 msgr2=0x559f89024e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 shutdown_connections 2024-04-02T13:35:01.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 1 -- 172.21.15.122:0/1670576679 wait complete. 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587d16e740 20 client.15450 trim_cache size 0 max 16384 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 3, 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24460, 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 110.940855254, 2024-04-02T13:35:01.582 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:35:01.583 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 94, 2024-04-02T13:35:01.583 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:35:01.583 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 149.88507338400001 2024-04-02T13:35:01.583 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:01.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 msgr2=0x7f58600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 0x7f58600a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f586c040000 tx=0x7f586c034440 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 msgr2=0x7f5874197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.572+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f585c00b450 tx=0x7f585c00b910 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 shutdown_connections 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f58600a3bb0 0x7f58600a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58741025b0 0x7f587419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5874101bf0 0x7f5874198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 --2- 172.21.15.122:0/746042318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5874100a00 0x7f5874197f00 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 >> 172.21.15.122:0/746042318 conn(0x7f58740fbf90 msgr2=0x7f58740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 shutdown_connections 2024-04-02T13:35:01.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.573+0000 7f587a4a5640 1 -- 172.21.15.122:0/746042318 wait complete. 2024-04-02T13:35:01.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.687+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/3249088233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 msgr2=0x7fb0c4108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.687+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/3249088233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c4108e40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009900 tx=0x7fb0c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/3249088233 shutdown_connections 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/3249088233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c4108e40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/3249088233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4101bf0 0x7fb0c4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/3249088233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4100a00 0x7fb0c4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/3249088233 >> 172.21.15.122:0/3249088233 conn(0x7fb0c40fbf90 msgr2=0x7fb0c40fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/3249088233 shutdown_connections 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/3249088233 wait complete. 2024-04-02T13:35:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 Processor -- start 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- start start 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 0x7fb0c4196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4101bf0 0x7fb0c4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410c680 con 0x7fb0c41025b0 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410c500 con 0x7fb0c4101bf0 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.688+0000 7fb0cb9f6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410c800 con 0x7fb0c4100a00 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35922/0 (socket says 172.21.15.122:35922) 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2295214043 learned_addr learned my addr 172.21.15.122:0/2295214043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 0x7fb0c4196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2295214043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 msgr2=0x7fb0c4196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 0x7fb0c4196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2295214043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4101bf0 msgr2=0x7fb0c4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4101bf0 0x7fb0c4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4101bf0 0x7fb0c4197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2295214043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c419e920 con 0x7fb0c41025b0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 0x7fb0c4196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005ab0 tx=0x7fb0c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0ba7fc640 1 -- 172.21.15.122:0/2295214043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0005000 con 0x7fb0c41025b0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0ba7fc640 1 -- 172.21.15.122:0/2295214043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c00035f0 con 0x7fb0c41025b0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c4068c60 con 0x7fb0c41025b0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0ba7fc640 1 -- 172.21.15.122:0/2295214043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c003b730 con 0x7fb0c41025b0 2024-04-02T13:35:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.689+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c4069150 con 0x7fb0c41025b0 2024-04-02T13:35:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.691+0000 7fb0ba7fc640 1 -- 172.21.15.122:0/2295214043 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7fb0c00037a0 con 0x7fb0c41025b0 2024-04-02T13:35:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 Processor -- start 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 -- start start 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ba7fc640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 0x7fb0b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4552f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600e44b7060 con 0x5600e4558640 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5600e445f650 con 0x5600e4552ae0 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600e44b3010 con 0x5600e45534d0 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 0x7fb0b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35936/0 (socket says 172.21.15.122:35936) 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9f93640 1 -- 172.21.15.122:0/3935910769 learned_addr learned my addr 172.21.15.122:0/3935910769 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9792640 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4552f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0ba7fc640 1 -- 172.21.15.122:0/2295214043 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb0c00f4a30 con 0x7fb0c41025b0 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9f93640 1 -- 172.21.15.122:0/3935910769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 msgr2=0x5600e4557fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.692+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0c9f93640 1 -- 172.21.15.122:0/3935910769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 msgr2=0x5600e4552f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4552f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0c9f93640 1 -- 172.21.15.122:0/3935910769 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e41cb2d0 con 0x5600e4558640 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 0x7fb0b00a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40058c0 tx=0x7fb0b4014840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00c08f0 tx=0x7fb0c002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a685f83ab0d9be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3935910769 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00a9e60 con 0x5600e4558640 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3935910769 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c00c1060 con 0x5600e4558640 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.693+0000 7fb0af7fe640 1 -- 172.21.15.122:0/3935910769 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00c95d0 con 0x5600e4558640 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/3935910769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 msgr2=0x5600e455aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb0c00c08f0 tx=0x7fb0c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/3935910769 shutdown_connections 2024-04-02T13:35:01.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e455aad0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/3935910769 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4552f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/3935910769 >> 172.21.15.122:0/3935910769 conn(0x5600e454e050 msgr2=0x5600e454e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/3935910769 shutdown_connections 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/3935910769 wait complete. 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- start start 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 Processor -- start 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e455d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4558640 0x5600e4564150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600e44b7060 con 0x5600e4552ae0 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5600e4472a10 con 0x5600e4558640 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600e455b3b0 con 0x5600e45534d0 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0ce5d9740 10 client.0 _ll_register_callbacks cb 0x5600e44af0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0c9792640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0c9792640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35946/0 (socket says 172.21.15.122:35946) 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.694+0000 7fb0c9792640 1 -- 172.21.15.122:0/1399001515 learned_addr learned my addr 172.21.15.122:0/1399001515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e455d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 -- 172.21.15.122:0/1399001515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 msgr2=0x5600e455d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e455d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 -- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4558640 msgr2=0x5600e4564150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4558640 0x5600e4564150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 -- 172.21.15.122:0/1399001515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e42956b0 con 0x5600e4552ae0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0c9792640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc012750 tx=0x7fb0bc012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca391376a978a3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc012e60 con 0x5600e4552ae0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb0bc010e20 con 0x5600e4552ae0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c00c1060 con 0x5600e4552ae0 2024-04-02T13:35:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0bc020540 con 0x5600e4552ae0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0ce5d9740 10 client.15462 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600e4567590 con 0x5600e4552ae0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c00c1060 con 0x5600e4552ae0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.695+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb0bc019070 con 0x5600e4552ae0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 10 client.15462 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 20 client.15462 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e4569d50 0x5600e456b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 4 client.15462 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600e445ecf0 con 0x5600e4569d50 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e4569d50 0x5600e456b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e4569d50 0x5600e456b1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89bb2db0187e9db2 server_cookie=33e2743160eba354 in_seq=0 out_seq=0 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.696+0000 7fb0acff9640 10 client.15462 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0acff9640 1 -- 172.21.15.122:0/1399001515 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600e445ecf0 con 0x5600e4569d50 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0acff9640 10 client.15462 handle_command_reply: tid=0 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 client.15462 shutdown 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 msgr2=0x5600e455cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc012750 tx=0x7fb0bc012c10 comp rx=0 tx=0).stop 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 shutdown_connections 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e4569d50 0x5600e456b1c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4558640 0x5600e4564150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e455d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0acff9640 0 client.15462 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/1399001515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4552ae0 0x5600e455cbd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 >> 172.21.15.122:0/1399001515 conn(0x5600e454e050 msgr2=0x5600e454d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 shutdown_connections 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/1399001515 wait complete. 2024-04-02T13:35:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.697+0000 7fb0ce5d9740 20 client.15462 trim_cache size 0 max 16384 2024-04-02T13:35:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 Processor -- start 2024-04-02T13:35:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 -- start start 2024-04-02T13:35:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4553f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 0x5600e4564ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600e4569500 con 0x5600e45534d0 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5600e4569bd0 con 0x5600e4552ae0 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600e456a2a0 con 0x5600e4558640 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0c8f91640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0c8f91640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35960/0 (socket says 172.21.15.122:35960) 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.715+0000 7fb0c8f91640 1 -- 172.21.15.122:0/2011894525 learned_addr learned my addr 172.21.15.122:0/2011894525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 0x5600e4564ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4553f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 -- 172.21.15.122:0/2011894525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 msgr2=0x5600e4564ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 0x5600e4564ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 -- 172.21.15.122:0/2011894525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 msgr2=0x5600e4553f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4553f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 -- 172.21.15.122:0/2011894525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e4557cb0 con 0x5600e45534d0 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 0x5600e4564ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5600e45538d0 tx=0x7fb0b4014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8645de849143b1a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0aeffd640 1 -- 172.21.15.122:0/2011894525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4029030 con 0x5600e45534d0 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0aeffd640 1 -- 172.21.15.122:0/2011894525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b400b6c0 con 0x5600e45534d0 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0aeffd640 1 -- 172.21.15.122:0/2011894525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4012a80 con 0x5600e45534d0 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2011894525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 msgr2=0x5600e4564110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.716+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5600e45538d0 tx=0x7fb0b4014f90 comp rx=0 tx=0).stop 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2011894525 shutdown_connections 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e4558640 0x5600e4564ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e45534d0 0x5600e4564110 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2011894525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4553f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2011894525 >> 172.21.15.122:0/2011894525 conn(0x5600e454e050 msgr2=0x5600e454e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2011894525 shutdown_connections 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2011894525 wait complete. 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- start start 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 Processor -- start 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4556bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5600e456b040 con 0x5600e4558640 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5600e456a970 con 0x5600e4552ae0 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5600e456b710 con 0x5600e45534d0 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0ce5d9740 10 client.0 _ll_register_callbacks cb 0x7fb0bc00aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0c9f93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35968/0 (socket says 172.21.15.122:35968) 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2591097554 learned_addr learned my addr 172.21.15.122:0/2591097554 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.717+0000 7fb0c8f91640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4556bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2591097554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 msgr2=0x5600e4557130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 msgr2=0x5600e4556bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4556bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 -- 172.21.15.122:0/2591097554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e4562de0 con 0x5600e4558640 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4556bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0c9f93640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006010 tx=0x7fb0c00be970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33837fcd2ea0f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c00a9e60 con 0x5600e4558640 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fb0c0003ad0 con 0x5600e4558640 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0b400b6c0 con 0x5600e4558640 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb0c00beaa0 con 0x5600e4558640 2024-04-02T13:35:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 10 client.15468 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5600e4564010 con 0x5600e4558640 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0b400b6c0 con 0x5600e4558640 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb0c00df080 con 0x5600e4558640 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 10 client.15468 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 10 client.15468 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.718+0000 7fb0ce5d9740 20 client.15468 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e45d4270 0x5600e45d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 4 client.15468 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "status"}) -- 0x5600e44c45a0 con 0x5600e45d4270 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e45d4270 0x5600e45d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0acff9640 10 client.15468 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0c9792640 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e45d4270 0x5600e45d2f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d98dfaf40a78d62d server_cookie=7d5e0fef5f59f292 in_seq=0 out_seq=0 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0acff9640 1 -- 172.21.15.122:0/2591097554 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5600e44c45a0 con 0x5600e45d4270 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0acff9640 10 client.15468 handle_command_reply: tid=0 2024-04-02T13:35:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 client.15468 shutdown 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 msgr2=0x5600e45627a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0006010 tx=0x7fb0c00be970 comp rx=0 tx=0).stop 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 shutdown_connections 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5600e4558640 0x5600e45627a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5600e45534d0 0x5600e4557130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5600e45d4270 0x5600e45d2f80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 --2- 172.21.15.122:0/2591097554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5600e4552ae0 0x5600e4556bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0acff9640 0 client.15468 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.719+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 >> 172.21.15.122:0/2591097554 conn(0x5600e454e050 msgr2=0x5600e454e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 shutdown_connections 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0ce5d9740 1 -- 172.21.15.122:0/2591097554 wait complete. 2024-04-02T13:35:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0ce5d9740 20 client.15468 trim_cache size 0 max 16384 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 2, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24485, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 112.095222893, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 94, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 147.17053196699999 2024-04-02T13:35:01.731 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:01.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 msgr2=0x7fb0b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 0x7fb0b00a6070 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40058c0 tx=0x7fb0b4014840 comp rx=0 tx=0).stop 2024-04-02T13:35:01.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 msgr2=0x7fb0c419e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005ab0 tx=0x7fb0c0004270 comp rx=0 tx=0).stop 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 shutdown_connections 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0c41025b0 0x7fb0c419e280 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fb0b00a3bb0 0x7fb0b00a6070 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0c4101bf0 0x7fb0c4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 --2- 172.21.15.122:0/2295214043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0c4100a00 0x7fb0c4196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 >> 172.21.15.122:0/2295214043 conn(0x7fb0c40fbf90 msgr2=0x7fb0c4104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 shutdown_connections 2024-04-02T13:35:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.720+0000 7fb0cb9f6640 1 -- 172.21.15.122:0/2295214043 wait complete. 2024-04-02T13:35:01.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-02T13:35:01.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 -- 172.21.15.122:0/3346401077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b0102d00 msgr2=0x7f43b0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 --2- 172.21.15.122:0/3346401077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b0102d00 0x7f43b0103180 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009900 tx=0x7f43a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 -- 172.21.15.122:0/3346401077 shutdown_connections 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 --2- 172.21.15.122:0/3346401077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b01036c0 0x7f43b0109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 --2- 172.21.15.122:0/3346401077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b0102d00 0x7f43b0103180 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 --2- 172.21.15.122:0/3346401077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b0101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 -- 172.21.15.122:0/3346401077 >> 172.21.15.122:0/3346401077 conn(0x7f43b00fd0a0 msgr2=0x7f43b00ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 -- 172.21.15.122:0/3346401077 shutdown_connections 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.834+0000 7f43b7d50640 1 -- 172.21.15.122:0/3346401077 wait complete. 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 Processor -- start 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- start start 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b00758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b0102d00 0x7f43b0073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43b010d790 con 0x7f43b01036c0 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43b010d610 con 0x7f43b0101b10 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43b010d910 con 0x7f43b0102d00 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35992/0 (socket says 172.21.15.122:35992) 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 -- 172.21.15.122:0/1586400724 learned_addr learned my addr 172.21.15.122:0/1586400724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b5aec640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b00758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b52eb640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b0102d00 0x7f43b0073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 -- 172.21.15.122:0/1586400724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b0102d00 msgr2=0x7f43b0073ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b0102d00 0x7f43b0073ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 -- 172.21.15.122:0/1586400724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 msgr2=0x7f43b00758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b00758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 -- 172.21.15.122:0/1586400724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b0074cc0 con 0x7f43b01036c0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b5aec640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b00758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b62ed640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00eb10 tx=0x7f43ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43a6ffd640 1 -- 172.21.15.122:0/1586400724 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac004b20 con 0x7f43b01036c0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43a6ffd640 1 -- 172.21.15.122:0/1586400724 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43ac004cc0 con 0x7f43b01036c0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43a6ffd640 1 -- 172.21.15.122:0/1586400724 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac010760 con 0x7f43b01036c0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b01a1f60 con 0x7f43b01036c0 2024-04-02T13:35:01.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.835+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b01a2310 con 0x7f43b01036c0 2024-04-02T13:35:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.837+0000 7f43a6ffd640 1 -- 172.21.15.122:0/1586400724 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f43ac005020 con 0x7f43b01036c0 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 Processor -- start 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 -- start start 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1dae0 0x55b961b1df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 0x55b961b22fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b961a82060 con 0x55b961b23640 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b961a2a650 con 0x55b961b1e4d0 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43baa0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b961a7e010 con 0x55b961b1dae0 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.838+0000 7f43a6ffd640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 0x7f439c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35994/0 (socket says 172.21.15.122:35994) 2024-04-02T13:35:01.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b5aec640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 0x55b961b22fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 -- 172.21.15.122:0/2162956180 learned_addr learned my addr 172.21.15.122:0/2162956180 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b62ed640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 0x7f439c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1dae0 0x55b961b1df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43a6ffd640 1 -- 172.21.15.122:0/1586400724 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f43ac0d0fe0 con 0x7f43b01036c0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 -- 172.21.15.122:0/2162956180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1dae0 msgr2=0x55b961b1df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1dae0 0x55b961b1df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 -- 172.21.15.122:0/2162956180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 msgr2=0x55b961b22fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 0x55b961b22fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 -- 172.21.15.122:0/2162956180 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9617962d0 con 0x55b961b23640 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b5aec640 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 0x55b961b22fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b62ed640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 0x7f439c0a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0998e0 tx=0x7f43ac067a00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005940 tx=0x7f43a0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fcf36f77b1255b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f439bfff640 1 -- 172.21.15.122:0/2162956180 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0049070 con 0x55b961b23640 2024-04-02T13:35:01.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f439bfff640 1 -- 172.21.15.122:0/2162956180 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a0004230 con 0x55b961b23640 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.839+0000 7f439bfff640 1 -- 172.21.15.122:0/2162956180 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0042440 con 0x55b961b23640 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- 172.21.15.122:0/2162956180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 msgr2=0x55b961b25ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43a0005940 tx=0x7f43a0005a20 comp rx=0 tx=0).stop 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- 172.21.15.122:0/2162956180 shutdown_connections 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b25ad0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1e4d0 0x55b961b22fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2162956180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1dae0 0x55b961b1df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- 172.21.15.122:0/2162956180 >> 172.21.15.122:0/2162956180 conn(0x55b961b19050 msgr2=0x55b961b19450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- 172.21.15.122:0/2162956180 shutdown_connections 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- 172.21.15.122:0/2162956180 wait complete. 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 -- start start 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.840+0000 7f43baa0f740 1 Processor -- start 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b28000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b23640 0x55b961b2f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b961b26590 con 0x55b961b1e4d0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b961a3da10 con 0x55b961b1dae0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b961b26710 con 0x55b961b23640 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43baa0f740 10 client.0 _ll_register_callbacks cb 0x55b961a7a0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36008/0 (socket says 172.21.15.122:36008) 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 -- 172.21.15.122:0/2085163119 learned_addr learned my addr 172.21.15.122:0/2085163119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b28000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b23640 0x55b961b2f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 -- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b23640 msgr2=0x55b961b2f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b23640 0x55b961b2f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 -- 172.21.15.122:0/2085163119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 msgr2=0x55b961b28000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b28000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 -- 172.21.15.122:0/2085163119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9618606b0 con 0x55b961b1e4d0 2024-04-02T13:35:01.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43b5aec640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f43a800cda0 tx=0x7f43a80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cffe15752495661 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.841+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80119d0 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a0004230 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f43a8011b70 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43a8011d70 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 10 client.15480 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b961b32a10 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a0004230 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43a8018070 con 0x55b961b1e4d0 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 10 client.15480 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 20 client.15480 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b350c0 0x55b961b36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 4 client.15480 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b961a29cf0 con 0x55b961b350c0 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.842+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b350c0 0x55b961b36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.843+0000 7f43997fa640 10 client.15480 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.843+0000 7f43b52eb640 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b350c0 0x55b961b36530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61adc0c625e0b7ac server_cookie=c93354c96d3cfd9e in_seq=0 out_seq=0 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.843+0000 7f43997fa640 1 -- 172.21.15.122:0/2085163119 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b961a29cf0 con 0x55b961b350c0 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.843+0000 7f43997fa640 10 client.15480 handle_command_reply: tid=0 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.843+0000 7f43baa0f740 1 client.15480 shutdown 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 msgr2=0x55b961b28540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f43a800cda0 tx=0x7f43a80039c0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 shutdown_connections 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b350c0 0x55b961b36530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b23640 0x55b961b2f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b1e4d0 0x55b961b28540 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43997fa640 0 client.15480 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:01.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 --2- 172.21.15.122:0/2085163119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b28000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 >> 172.21.15.122:0/2085163119 conn(0x55b961b19050 msgr2=0x55b961b18190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 shutdown_connections 2024-04-02T13:35:01.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 1 -- 172.21.15.122:0/2085163119 wait complete. 2024-04-02T13:35:01.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.844+0000 7f43baa0f740 20 client.15480 trim_cache size 0 max 16384 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 Processor -- start 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 -- start start 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b2f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b961b34870 con 0x55b961b23640 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b961b34f40 con 0x55b961b1dae0 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43baa0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b961b35610 con 0x55b961b1e4d0 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36014/0 (socket says 172.21.15.122:36014) 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 -- 172.21.15.122:0/3924662688 learned_addr learned my addr 172.21.15.122:0/3924662688 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 -- 172.21.15.122:0/3924662688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 msgr2=0x55b961b2f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b2f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 -- 172.21.15.122:0/3924662688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 msgr2=0x55b961b2ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 -- 172.21.15.122:0/3924662688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b961b1e8b0 con 0x55b961b23640 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f43b52eb640 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c7b0 tx=0x7f43a00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdff96e2d61b2a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.862+0000 7f439b7fe640 1 -- 172.21.15.122:0/3924662688 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0056070 con 0x55b961b23640 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f439b7fe640 1 -- 172.21.15.122:0/3924662688 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a0051710 con 0x55b961b23640 2024-04-02T13:35:01.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f439b7fe640 1 -- 172.21.15.122:0/3924662688 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0049040 con 0x55b961b23640 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- 172.21.15.122:0/3924662688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 msgr2=0x55b961b19430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c7b0 tx=0x7f43a00057b0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- 172.21.15.122:0/3924662688 shutdown_connections 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b19430 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b2f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 --2- 172.21.15.122:0/3924662688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- 172.21.15.122:0/3924662688 >> 172.21.15.122:0/3924662688 conn(0x55b961b19050 msgr2=0x55b961b1eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- 172.21.15.122:0/3924662688 shutdown_connections 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- 172.21.15.122:0/3924662688 wait complete. 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.863+0000 7f43baa0f740 1 -- start start 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 Processor -- start 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b961b363b0 con 0x55b961b23640 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b961b35ce0 con 0x55b961b1dae0 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b961b36a80 con 0x55b961b1e4d0 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 10 client.0 _ll_register_callbacks cb 0x7f43a8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36016/0 (socket says 172.21.15.122:36016) 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 -- 172.21.15.122:0/1253760900 learned_addr learned my addr 172.21.15.122:0/1253760900 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 -- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 msgr2=0x55b961b97050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 -- 172.21.15.122:0/1253760900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 msgr2=0x55b961b2d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 -- 172.21.15.122:0/1253760900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b961b99cb0 con 0x55b961b23640 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43b52eb640 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c170 tx=0x7f43a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cba6655d040480a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a0056070 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f43a0005030 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.864+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a0051710 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f43a0046030 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 10 client.15486 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b961b9afb0 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a0051710 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f43a0004cd0 con 0x55b961b23640 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 10 client.15486 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 10 client.15486 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 20 client.15486 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b9f4b0 0x55b961b9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 4 client.15486 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:35:01.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "status"}) -- 0x55b961a8f5a0 con 0x55b961b9f4b0 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.865+0000 7f43b5aec640 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b9f4b0 0x55b961b9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43b5aec640 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b9f4b0 0x55b961b9e1c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eefd5cf73bd30bad server_cookie=f12257b446d28128 in_seq=0 out_seq=0 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43997fa640 10 client.15486 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43997fa640 1 -- 172.21.15.122:0/1253760900 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b961a8f5a0 con 0x55b961b9f4b0 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43997fa640 10 client.15486 handle_command_reply: tid=0 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 client.15486 shutdown 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 msgr2=0x55b961b995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f43a002c170 tx=0x7f43a0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 shutdown_connections 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b961b9f4b0 0x55b961b9e1c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b961b23640 0x55b961b995a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b961b1e4d0 0x55b961b97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43997fa640 0 client.15486 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:01.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 --2- 172.21.15.122:0/1253760900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b961b1dae0 0x55b961b2d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 >> 172.21.15.122:0/1253760900 conn(0x55b961b19050 msgr2=0x55b961b1fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 shutdown_connections 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 1 -- 172.21.15.122:0/1253760900 wait complete. 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.866+0000 7f43baa0f740 20 client.15486 trim_cache size 0 max 16384 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 1, 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24487, 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:35:01.876 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 113.250896157, 2024-04-02T13:35:01.877 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:35:01.877 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 94, 2024-04-02T13:35:01.877 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:35:01.877 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 144.24802820299999 2024-04-02T13:35:01.877 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 msgr2=0x7f439c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 0x7f439c0a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0998e0 tx=0x7f43ac067a00 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 msgr2=0x7f43b0074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00eb10 tx=0x7f43ac00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 shutdown_connections 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43b01036c0 0x7f43b0074430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b0102d00 0x7f43b0073ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f439c0a3bb0 0x7f439c0a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 --2- 172.21.15.122:0/1586400724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43b0101b10 0x7f43b00758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 >> 172.21.15.122:0/1586400724 conn(0x7f43b00fd0a0 msgr2=0x7f43b0106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 shutdown_connections 2024-04-02T13:35:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.867+0000 7f43b7d50640 1 -- 172.21.15.122:0/1586400724 wait complete. 2024-04-02T13:35:01.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:01 smithi122 ceph-mon[32114]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:35:01.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1202426671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:01.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/897027644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:01.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 -- 172.21.15.122:0/3765720075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98100a20 msgr2=0x7f4b98100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 --2- 172.21.15.122:0/3765720075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98100a20 0x7f4b98100e40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009900 tx=0x7f4b8802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 -- 172.21.15.122:0/3765720075 shutdown_connections 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 --2- 172.21.15.122:0/3765720075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 0x7f4b98108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 --2- 172.21.15.122:0/3765720075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98101c10 0x7f4b98102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 --2- 172.21.15.122:0/3765720075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98100a20 0x7f4b98100e40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 -- 172.21.15.122:0/3765720075 >> 172.21.15.122:0/3765720075 conn(0x7f4b980fbfb0 msgr2=0x7f4b980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 -- 172.21.15.122:0/3765720075 shutdown_connections 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.980+0000 7f4b9c947640 1 -- 172.21.15.122:0/3765720075 wait complete. 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 Processor -- start 2024-04-02T13:35:01.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 -- start start 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 0x7f4b98196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 0x7f4b9819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810c6a0 con 0x7f4b98101c10 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810c520 con 0x7f4b98100a20 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9c947640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9810c820 con 0x7f4b981025d0 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36032/0 (socket says 172.21.15.122:36032) 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 -- 172.21.15.122:0/2170303825 learned_addr learned my addr 172.21.15.122:0/2170303825 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 0x7f4b98196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b96d9d640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 0x7f4b9819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 -- 172.21.15.122:0/2170303825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 msgr2=0x7f4b9819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 0x7f4b9819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 -- 172.21.15.122:0/2170303825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 msgr2=0x7f4b98196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 0x7f4b98196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 -- 172.21.15.122:0/2170303825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9819e980 con 0x7f4b98101c10 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 0x7f4b98196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.981+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.982+0000 7f4b8f7fe640 1 -- 172.21.15.122:0/2170303825 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80017070 con 0x7f4b98101c10 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.982+0000 7f4b8f7fe640 1 -- 172.21.15.122:0/2170303825 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b80009c30 con 0x7f4b98101c10 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.982+0000 7f4b8f7fe640 1 -- 172.21.15.122:0/2170303825 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000f6b0 con 0x7f4b98101c10 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.982+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9819ec70 con 0x7f4b98101c10 2024-04-02T13:35:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.982+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9819f1c0 con 0x7f4b98101c10 2024-04-02T13:35:01.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:01 smithi187 ceph-mon[36064]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:35:01.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1202426671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:01.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/897027644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.984+0000 7f4b9f60d740 1 Processor -- start 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b8f7fe640 1 -- 172.21.15.122:0/2170303825 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f4b800109d0 con 0x7f4b98101c10 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 -- start start 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631dc41c060 con 0x5631dc4b7ae0 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631dc3c4650 con 0x5631dc4bd640 2024-04-02T13:35:01.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631dc418010 con 0x5631dc4b84d0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36048/0 (socket says 172.21.15.122:36048) 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 -- 172.21.15.122:0/1278530715 learned_addr learned my addr 172.21.15.122:0/1278530715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9659c640 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 -- 172.21.15.122:0/1278530715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 msgr2=0x5631dc4bcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4bcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 -- 172.21.15.122:0/1278530715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 msgr2=0x5631dc4bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b8f7fe640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 0x7f4b840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 -- 172.21.15.122:0/1278530715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dc1302d0 con 0x5631dc4b7ae0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 0x7f4b840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.985+0000 7f4b96d9d640 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009ee0 tx=0x7f4b90010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7f7683427fa479 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b8f7fe640 1 -- 172.21.15.122:0/2170303825 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4b800d0480 con 0x7f4b98101c10 2024-04-02T13:35:01.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b8cff9640 1 -- 172.21.15.122:0/1278530715 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90004d20 con 0x5631dc4b7ae0 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b8cff9640 1 -- 172.21.15.122:0/1278530715 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b90013ad0 con 0x5631dc4b7ae0 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b8cff9640 1 -- 172.21.15.122:0/1278530715 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90005670 con 0x5631dc4b7ae0 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 -- 172.21.15.122:0/1278530715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 msgr2=0x5631dc4b7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 0x7f4b840a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b8802c7b0 tx=0x7f4b88005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009ee0 tx=0x7f4b90010740 comp rx=0 tx=0).stop 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 -- 172.21.15.122:0/1278530715 shutdown_connections 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4bcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/1278530715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4b7f00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 -- 172.21.15.122:0/1278530715 >> 172.21.15.122:0/1278530715 conn(0x5631dc4b3050 msgr2=0x5631dc4b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.986+0000 7f4b9f60d740 1 -- 172.21.15.122:0/1278530715 shutdown_connections 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 -- 172.21.15.122:0/1278530715 wait complete. 2024-04-02T13:35:01.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 -- start start 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 Processor -- start 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4c05f0 con 0x5631dc4b7ae0 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631dc3d7a10 con 0x5631dc4bd640 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4c0770 con 0x5631dc4b84d0 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9f60d740 10 client.0 _ll_register_callbacks cb 0x5631dc4140b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:01.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36060/0 (socket says 172.21.15.122:36060) 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 -- 172.21.15.122:0/44994110 learned_addr learned my addr 172.21.15.122:0/44994110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b96d9d640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 -- 172.21.15.122:0/44994110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 msgr2=0x5631dc4c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 -- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 msgr2=0x5631dc4c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b9659c640 1 -- 172.21.15.122:0/44994110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dc1fa6b0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.987+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9659c640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a57b3fc5e758f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88043070 con 0x5631dc4b7ae0 2024-04-02T13:35:02.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4b88034640 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b90013ad0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b8805b050 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 10 client.15498 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631dc4cc950 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b90013ad0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b88052930 con 0x5631dc4b7ae0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 10 client.15498 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 10 client.15498 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 20 client.15498 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc4cefe0 0x5631dc4d0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 4 client.15498 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.988+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631dc3c3cf0 con 0x5631dc4cefe0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.989+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc4cefe0 0x5631dc4d0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.989+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc4cefe0 0x5631dc4d0450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5494292743e47946 server_cookie=653a7094aa823881 in_seq=0 out_seq=0 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.989+0000 7f4b7dffb640 10 client.15498 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b7dffb640 1 -- 172.21.15.122:0/44994110 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631dc3c3cf0 con 0x5631dc4cefe0 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b7dffb640 10 client.15498 handle_command_reply: tid=0 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 client.15498 shutdown 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 msgr2=0x5631dc4c2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b88009260 comp rx=0 tx=0).stop 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 shutdown_connections 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc4cefe0 0x5631dc4d0450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc4c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b7dffb640 0 client.15498 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc4c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/44994110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c2060 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 >> 172.21.15.122:0/44994110 conn(0x5631dc4b3050 msgr2=0x5631dc4b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 shutdown_connections 2024-04-02T13:35:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 1 -- 172.21.15.122:0/44994110 wait complete. 2024-04-02T13:35:02.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:01.990+0000 7f4b9f60d740 20 client.15498 trim_cache size 0 max 16384 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 Processor -- start 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 -- start start 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4b84d0 0x5631dc4b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4bd640 0x5631dc4c1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4ce790 con 0x5631dc4b7ae0 2024-04-02T13:35:02.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4cee60 con 0x5631dc4b84d0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4cf530 con 0x5631dc4bd640 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36064/0 (socket says 172.21.15.122:36064) 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 -- 172.21.15.122:0/2588581429 learned_addr learned my addr 172.21.15.122:0/2588581429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 -- 172.21.15.122:0/2588581429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4bd640 msgr2=0x5631dc4c1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4bd640 0x5631dc4c1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 -- 172.21.15.122:0/2588581429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4b84d0 msgr2=0x5631dc4b8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4b84d0 0x5631dc4b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.008+0000 7f4b9659c640 1 -- 172.21.15.122:0/2588581429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631dc4b3fc0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9659c640 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036040 tx=0x7f4b880058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e209825ab477b7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b7ffff640 1 -- 172.21.15.122:0/2588581429 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88063070 con 0x5631dc4b7ae0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b7ffff640 1 -- 172.21.15.122:0/2588581429 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8802c980 con 0x5631dc4b7ae0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b7ffff640 1 -- 172.21.15.122:0/2588581429 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8805fb10 con 0x5631dc4b7ae0 2024-04-02T13:35:02.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 -- 172.21.15.122:0/2588581429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 msgr2=0x5631dc4c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036040 tx=0x7f4b880058b0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 -- 172.21.15.122:0/2588581429 shutdown_connections 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4bd640 0x5631dc4c1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4b84d0 0x5631dc4b8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/2588581429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc4c04d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 -- 172.21.15.122:0/2588581429 >> 172.21.15.122:0/2588581429 conn(0x5631dc4b3050 msgr2=0x5631dc4c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 -- 172.21.15.122:0/2588581429 shutdown_connections 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.009+0000 7f4b9f60d740 1 -- 172.21.15.122:0/2588581429 wait complete. 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 -- start start 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 Processor -- start 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc531700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc5380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4d02d0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4cfc00 con 0x5631dc4bd640 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631dc4d09a0 con 0x5631dc4b84d0 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 10 client.0 _ll_register_callbacks cb 0x7f4b880056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36068/0 (socket says 172.21.15.122:36068) 2024-04-02T13:35:02.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 -- 172.21.15.122:0/3479698030 learned_addr learned my addr 172.21.15.122:0/3479698030 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:02.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 -- 172.21.15.122:0/3479698030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 msgr2=0x5631dc531700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc531700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 -- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 msgr2=0x5631dc5380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc5380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 -- 172.21.15.122:0/3479698030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631dc538720 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9659c640 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b8805fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3031ca4f68ebde37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b7dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88043070 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4b880523e0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.010+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b8802c980 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4b88063040 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 10 client.15504 fetch_fsmap learned FSMap version 27 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5631dc5399c0 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b8802c980 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4b88048740 con 0x5631dc4b7ae0 2024-04-02T13:35:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 10 client.15504 fetch_fsmap finished waiting for FSMap version 27 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 10 client.15504 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 20 client.15504 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc53e3e0 0x5631dc53d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 4 client.15504 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "status"}) -- 0x5631dc4295a0 con 0x5631dc53e3e0 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.011+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc53e3e0 0x5631dc53d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b8ffff640 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc53e3e0 0x5631dc53d0f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a1de1bdf73e78e server_cookie=ba9cfaaa45a88165 in_seq=0 out_seq=0 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b7dffb640 10 client.15504 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b7dffb640 1 -- 172.21.15.122:0/3479698030 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5631dc4295a0 con 0x5631dc53e3e0 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b7dffb640 10 client.15504 handle_command_reply: tid=0 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 client.15504 shutdown 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 msgr2=0x5631dc5311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4b880096c0 tx=0x7f4b8805fa40 comp rx=0 tx=0).stop 2024-04-02T13:35:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 shutdown_connections 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5631dc53e3e0 0x5631dc53d0f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631dc4bd640 0x5631dc5380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631dc4b84d0 0x5631dc531700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b7dffb640 0 client.15504 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 --2- 172.21.15.122:0/3479698030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631dc4b7ae0 0x5631dc5311c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 >> 172.21.15.122:0/3479698030 conn(0x5631dc4b3050 msgr2=0x5631dc4b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 shutdown_connections 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 1 -- 172.21.15.122:0/3479698030 wait complete. 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9f60d740 20 client.15504 trim_cache size 0 max 16384 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:02.024 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "3c818384-f0f4-11ee-b647-cb9ed24678a4", 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 24512, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 131.48972202900001, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 27, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 94, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 90, 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 135.64030004899999 2024-04-02T13:35:02.025 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 msgr2=0x7f4b840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 0x7f4b840a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4b8802c7b0 tx=0x7f4b88005a20 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 msgr2=0x7f4b981972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000d950 tx=0x7f4b8000de10 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 shutdown_connections 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.012+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b981025d0 0x7f4b9819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b98101c10 0x7f4b981972a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4b840a3bb0 0x7f4b840a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 --2- 172.21.15.122:0/2170303825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4b98100a20 0x7f4b98196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 >> 172.21.15.122:0/2170303825 conn(0x7f4b980fbfb0 msgr2=0x7f4b98104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 shutdown_connections 2024-04-02T13:35:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.013+0000 7f4b9c947640 1 -- 172.21.15.122:0/2170303825 wait complete. 2024-04-02T13:35:02.028 INFO:teuthology.run_tasks:Running task sequential... 2024-04-02T13:35:02.037 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-02T13:35:02.038 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-02T13:35:02.038 INFO:teuthology.task.print:Enabling mgr modules 2024-04-02T13:35:02.038 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-02T13:35:02.039 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:35:02.039 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:35:02.039 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-02T13:35:02.040 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:01 smithi188 ceph-mon[36117]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-02T13:35:02.040 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1202426671' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:02.040 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/897027644' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:02.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 -- 172.21.15.122:0/586930222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 msgr2=0x7f36ec1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 --2- 172.21.15.122:0/586930222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec1020c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009900 tx=0x7f36d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 -- 172.21.15.122:0/586930222 shutdown_connections 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 --2- 172.21.15.122:0/586930222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec102600 0x7f36ec108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 --2- 172.21.15.122:0/586930222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec1020c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 --2- 172.21.15.122:0/586930222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec100a50 0x7f36ec100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 -- 172.21.15.122:0/586930222 >> 172.21.15.122:0/586930222 conn(0x7f36ec0fbfe0 msgr2=0x7f36ec0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 -- 172.21.15.122:0/586930222 shutdown_connections 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 -- 172.21.15.122:0/586930222 wait complete. 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.138+0000 7f36f3088640 1 Processor -- start 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 -- start start 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec100a50 0x7f36ec06ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec102600 0x7f36ec19f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10c740 con 0x7f36ec101c40 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10c5c0 con 0x7f36ec100a50 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec10c8c0 con 0x7f36ec102600 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36084/0 (socket says 172.21.15.122:36084) 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 -- 172.21.15.122:0/1454798179 learned_addr learned my addr 172.21.15.122:0/1454798179 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f1625640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec102600 0x7f36ec19f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 -- 172.21.15.122:0/1454798179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec102600 msgr2=0x7f36ec19f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec102600 0x7f36ec19f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 -- 172.21.15.122:0/1454798179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec100a50 msgr2=0x7f36ec06ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f0e24640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec100a50 0x7f36ec06ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec100a50 0x7f36ec06ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 -- 172.21.15.122:0/1454798179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec19faa0 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36ebfff640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009230 tx=0x7f36d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8041070 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d80054e0 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec19fcd0 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.139+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d80346a0 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.140+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a01c0 con 0x7f36ec101c40 2024-04-02T13:35:02.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.141+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ec107d10 con 0x7f36ec101c40 2024-04-02T13:35:02.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.142+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 18) ==== 138647+0+0 (secure 0 0 0) 0x7f36d8044070 con 0x7f36ec101c40 2024-04-02T13:35:02.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.142+0000 7f36e9ffb640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 0x7f36dc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.142+0000 7f36f0e24640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 0x7f36dc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.142+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f36d80f25a0 con 0x7f36ec101c40 2024-04-02T13:35:02.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.143+0000 7f36f0e24640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 0x7f36dc0a6110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36ec101aa0 tx=0x7f36e0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.147+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f36d80bebd0 con 0x7f36ec101c40 2024-04-02T13:35:02.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.376+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f36ec0627a0 con 0x7f36ec101c40 2024-04-02T13:35:02.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.653+0000 7f36e9ffb640 1 -- 172.21.15.122:0/1454798179 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f36d80be4d0 con 0x7f36ec101c40 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 msgr2=0x7f36dc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 0x7f36dc0a6110 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f36ec101aa0 tx=0x7f36e0008040 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 msgr2=0x7f36ec19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009230 tx=0x7f36d8004040 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 shutdown_connections 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36ec102600 0x7f36ec19f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36ec101c40 0x7f36ec19cb70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f36dc0a3c50 0x7f36dc0a6110 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 --2- 172.21.15.122:0/1454798179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36ec100a50 0x7f36ec06ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 >> 172.21.15.122:0/1454798179 conn(0x7f36ec0fbfe0 msgr2=0x7f36ec104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.655+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 shutdown_connections 2024-04-02T13:35:02.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.656+0000 7f36f3088640 1 -- 172.21.15.122:0/1454798179 wait complete. 2024-04-02T13:35:02.677 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 -- 172.21.15.122:0/4175434270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf441025a0 msgr2=0x7fcf44104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/4175434270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf441025a0 0x7fcf44104990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009900 tx=0x7fcf4002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 -- 172.21.15.122:0/4175434270 shutdown_connections 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/4175434270 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf44107800 0x7fcf44109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/4175434270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44104ed0 0x7fcf441072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/4175434270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf441025a0 0x7fcf44104990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 -- 172.21.15.122:0/4175434270 >> 172.21.15.122:0/4175434270 conn(0x7fcf440fbf90 msgr2=0x7fcf440fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 -- 172.21.15.122:0/4175434270 shutdown_connections 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.781+0000 7fcf4b58f640 1 -- 172.21.15.122:0/4175434270 wait complete. 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 Processor -- start 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 -- start start 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf441025a0 0x7fcf44197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44107800 0x7fcf4419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410d7a0 con 0x7fcf44104ed0 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410d620 con 0x7fcf441025a0 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4b58f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410d920 con 0x7fcf44107800 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36118/0 (socket says 172.21.15.122:36118) 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 -- 172.21.15.122:0/3480464103 learned_addr learned my addr 172.21.15.122:0/3480464103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf4932b640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf441025a0 0x7fcf44197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf49b2c640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44107800 0x7fcf4419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 -- 172.21.15.122:0/3480464103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44107800 msgr2=0x7fcf4419f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44107800 0x7fcf4419f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 -- 172.21.15.122:0/3480464103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf441025a0 msgr2=0x7fcf44197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf441025a0 0x7fcf44197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 -- 172.21.15.122:0/3480464103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4419f9d0 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.782+0000 7fcf48b2a640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d770 tx=0x7fcf3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.783+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34017070 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.783+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34009c30 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.783+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4419fcc0 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.783+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400f620 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.783+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a01c0 con 0x7fcf44104ed0 2024-04-02T13:35:02.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.784+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf440fda80 con 0x7fcf44104ed0 2024-04-02T13:35:02.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.785+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7fcf3401a070 con 0x7fcf44104ed0 2024-04-02T13:35:02.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.785+0000 7fcf3a7fc640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 0x7fcf300a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:02.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.785+0000 7fcf4932b640 1 -- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 msgr2=0x7fcf300a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:02.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.786+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fcf340d0270 con 0x7fcf44104ed0 2024-04-02T13:35:02.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.786+0000 7fcf4932b640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 0x7fcf300a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:02.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.790+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcf34009de0 con 0x7fcf44104ed0 2024-04-02T13:35:02.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.974+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fcf44062700 con 0x7fcf44104ed0 2024-04-02T13:35:02.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.978+0000 7fcf3a7fc640 1 -- 172.21.15.122:0/3480464103 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fcf3409c250 con 0x7fcf44104ed0 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 msgr2=0x7fcf300a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 0x7fcf300a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 msgr2=0x7fcf441982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d770 tx=0x7fcf3400dc30 comp rx=0 tx=0).stop 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 shutdown_connections 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf44107800 0x7fcf4419f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf44104ed0 0x7fcf441982f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7fcf300a3ca0 0x7fcf300a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 --2- 172.21.15.122:0/3480464103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf441025a0 0x7fcf44197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 >> 172.21.15.122:0/3480464103 conn(0x7fcf440fbf90 msgr2=0x7fcf44102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 shutdown_connections 2024-04-02T13:35:02.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:02.980+0000 7fcf4b58f640 1 -- 172.21.15.122:0/3480464103 wait complete. 2024-04-02T13:35:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:02 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:35:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:02 smithi187 ceph-mon[36064]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:35:02.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:02 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: ignoring --setuser ceph since I am not root 2024-04-02T13:35:02.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:02 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: ignoring --setgroup ceph since I am not root 2024-04-02T13:35:02.986 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:02 smithi187 ceph-mgr[37451]: -- 172.21.15.187:0/3754361374 <== mon.2 v2:172.21.15.187:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560bf0cc25a0 con 0x560bf0c97000 2024-04-02T13:35:02.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:02 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:35:02.994 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:02 smithi122 ceph-mon[32114]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:35:02.994 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:02 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setuser ceph since I am not root 2024-04-02T13:35:02.994 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:02 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: ignoring --setgroup ceph since I am not root 2024-04-02T13:35:02.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-02T13:35:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:02 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-02T13:35:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:02 smithi188 ceph-mon[36117]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-02T13:35:03.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: ignoring --setuser ceph since I am not root 2024-04-02T13:35:03.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: ignoring --setgroup ceph since I am not root 2024-04-02T13:35:03.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:02 smithi188 ceph-mgr[37982]: -- 172.21.15.188:0/3404584715 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585b429c5a0 con 0x5585b4271000 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/1570099240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14101a50 msgr2=0x7f4c14101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/1570099240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14101a50 0x7f4c14101ed0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009900 tx=0x7f4c0402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/1570099240 shutdown_connections 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/1570099240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 0x7f4c14108ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/1570099240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14101a50 0x7f4c14101ed0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/1570099240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14100860 0x7f4c14100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.096+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/1570099240 >> 172.21.15.122:0/1570099240 conn(0x7f4c140fbdd0 msgr2=0x7f4c140fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:03.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/1570099240 shutdown_connections 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/1570099240 wait complete. 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 Processor -- start 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- start start 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 0x7f4c1419c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 0x7f4c1419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410c4e0 con 0x7f4c14100860 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410c360 con 0x7f4c14101a50 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1ab9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1410c660 con 0x7f4c14102410 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c19b9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c19b9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36132/0 (socket says 172.21.15.122:36132) 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c19b9b640 1 -- 172.21.15.122:0/3306914991 learned_addr learned my addr 172.21.15.122:0/3306914991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:03.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1a39c640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 0x7f4c1419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.097+0000 7f4c1939a640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 0x7f4c1419c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 -- 172.21.15.122:0/3306914991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 msgr2=0x7f4c1419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 0x7f4c1419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 -- 172.21.15.122:0/3306914991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 msgr2=0x7f4c1419c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 0x7f4c1419c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 -- 172.21.15.122:0/3306914991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1419f880 con 0x7f4c14100860 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c1939a640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 0x7f4c1419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c19b9b640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9c0 tx=0x7f4c1000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1000cd80 con 0x7f4c14100860 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c1419fb70 con 0x7f4c14100860 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c100035f0 con 0x7f4c14100860 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c10010690 con 0x7f4c14100860 2024-04-02T13:35:03.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.098+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c141a0090 con 0x7f4c14100860 2024-04-02T13:35:03.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.099+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c14107b80 con 0x7f4c14100860 2024-04-02T13:35:03.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.100+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f4c100037a0 con 0x7f4c14100860 2024-04-02T13:35:03.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.101+0000 7f4c0affd640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 0x7f4c000a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:03.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.101+0000 7f4c1939a640 1 -- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 msgr2=0x7f4c000a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:03.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.101+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4c100d0c70 con 0x7f4c14100860 2024-04-02T13:35:03.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.101+0000 7f4c1939a640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 0x7f4c000a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:03.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.107+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4c1009d2a0 con 0x7f4c14100860 2024-04-02T13:35:03.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.294+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f4c1402d370 con 0x7f4c14100860 2024-04-02T13:35:03.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.298+0000 7f4c0affd640 1 -- 172.21.15.122:0/3306914991 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f4c1009cba0 con 0x7f4c14100860 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 msgr2=0x7f4c000a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 0x7f4c000a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 msgr2=0x7f4c1406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000e9c0 tx=0x7f4c1000ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 shutdown_connections 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c14102410 0x7f4c1419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4c000a3bf0 0x7f4c000a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c14101a50 0x7f4c1419c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 --2- 172.21.15.122:0/3306914991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c14100860 0x7f4c1406a7a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:03.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 >> 172.21.15.122:0/3306914991 conn(0x7f4c140fbdd0 msgr2=0x7f4c14104d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:03.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 shutdown_connections 2024-04-02T13:35:03.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:03.300+0000 7f4c1ab9d640 1 -- 172.21.15.122:0/3306914991 wait complete. 2024-04-02T13:35:03.315 INFO:teuthology.run_tasks:Running task kclient... 2024-04-02T13:35:03.329 INFO:tasks.kclient:Mounting kernel clients... 2024-04-02T13:35:03.329 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-02T13:35:03.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:03.329 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-02T13:35:03.344 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-02T13:35:03.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:03.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-02T13:35:03.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:03.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-04-02T13:35:03.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:03.227+0000 7f995d527200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:35:03.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:03.321+0000 7f995d527200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:35:03.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:03.226+0000 7f5f5db3c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:35:03.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:03.320+0000 7f5f5db3c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:35:03.518 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:03.248+0000 7f573f99d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-02T13:35:03.518 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:03.347+0000 7f573f99d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-02T13:35:03.520 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-02T13:35:03.521 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-02T13:35:03.521 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:03.521 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph fs ls 2024-04-02T13:35:03.770 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:35:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:35:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-mon[32114]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:35:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3480464103' entity='client.admin' 2024-04-02T13:35:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3306914991' entity='client.admin' 2024-04-02T13:35:03.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:35:03.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-mon[36064]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:35:03.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3480464103' entity='client.admin' 2024-04-02T13:35:03.966 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3306914991' entity='client.admin' 2024-04-02T13:35:03.966 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1454798179' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-02T13:35:03.967 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-mon[36117]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-02T13:35:03.967 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3480464103' entity='client.admin' 2024-04-02T13:35:03.967 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3306914991' entity='client.admin' 2024-04-02T13:35:04.156 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.025+0000 7f573f99d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:35:04.156 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.122+0000 7f573f99d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:35:04.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:03.962+0000 7f995d527200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:35:04.236 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.056+0000 7f995d527200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:35:04.236 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.198+0000 7f995d527200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:35:04.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:03.962+0000 7f5f5db3c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-02T13:35:04.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.056+0000 7f5f5db3c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-02T13:35:04.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.199+0000 7f5f5db3c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:35:04.451 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.267+0000 7f573f99d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-02T13:35:04.451 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.364+0000 7f573f99d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:35:04.451 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.448+0000 7f573f99d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:35:04.645 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.294+0000 7f5f5db3c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:35:04.645 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.376+0000 7f5f5db3c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:35:04.646 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.294+0000 7f995d527200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-02T13:35:04.646 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.377+0000 7f995d527200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-02T13:35:04.646 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.642+0000 7f995d527200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:35:04.936 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.641+0000 7f5f5db3c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:35:04.936 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.752+0000 7f5f5db3c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:04.936 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.837+0000 7f5f5db3c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:35:04.939 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.753+0000 7f995d527200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:04.939 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.840+0000 7f995d527200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:35:05.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.718+0000 7f573f99d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-02T13:35:05.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.830+0000 7f573f99d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:05.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:04.918+0000 7f573f99d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-02T13:35:05.024 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:05.021+0000 7f573f99d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:35:05.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:04.935+0000 7f995d527200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:35:05.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:05.113+0000 7f995d527200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:35:05.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:04.932+0000 7f5f5db3c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-02T13:35:05.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:05.111+0000 7f5f5db3c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:35:05.281 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:05.200+0000 7f573f99d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-02T13:35:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- 172.21.15.122:0/3964101137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 msgr2=0x7f296c101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 --2- 172.21.15.122:0/3964101137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c101970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f295c009990 tx=0x7f295c009ca0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- 172.21.15.122:0/3964101137 shutdown_connections 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 --2- 172.21.15.122:0/3964101137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c101eb0 0x7f296c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 --2- 172.21.15.122:0/3964101137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c107500 0x7f296c1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 --2- 172.21.15.122:0/3964101137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c101970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- 172.21.15.122:0/3964101137 >> 172.21.15.122:0/3964101137 conn(0x7f296c078420 msgr2=0x7f296c078830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- 172.21.15.122:0/3964101137 shutdown_connections 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- 172.21.15.122:0/3964101137 wait complete. 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 Processor -- start 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.493+0000 7f297418b700 1 -- start start 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c101eb0 0x7f296c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c107500 0x7f296c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f296c111790 con 0x7f296c1014f0 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f296c111610 con 0x7f296c107500 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f296c111910 con 0x7f296c101eb0 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36148/0 (socket says 172.21.15.122:36148) 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 -- 172.21.15.122:0/3322898731 learned_addr learned my addr 172.21.15.122:0/3322898731 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2972728700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c107500 0x7f296c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971726700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c101eb0 0x7f296c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c101eb0 msgr2=0x7f296c19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c101eb0 0x7f296c19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c107500 msgr2=0x7f296c19f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c107500 0x7f296c19f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 -- 172.21.15.122:0/3322898731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f296c19ffa0 con 0x7f296c1014f0 2024-04-02T13:35:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2971f27700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f296800d980 tx=0x7f296800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968017070 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2968005020 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f296c1a0290 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.494+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296800fbf0 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.495+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f296c1a7b10 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.495+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f296c0692b0 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.497+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f29680051d0 con 0x7f296c1014f0 2024-04-02T13:35:05.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.497+0000 7f2962ffd700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 0x7f29580a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.497+0000 7f2971726700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 msgr2=0x7f29580a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.497+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f29680d0980 con 0x7f296c1014f0 2024-04-02T13:35:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.497+0000 7f2971726700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 0x7f29580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:05.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.501+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2968020020 con 0x7f296c1014f0 2024-04-02T13:35:05.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.698+0000 7f2971726700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 msgr2=0x7f29580a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:05.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.698+0000 7f2971726700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 0x7f29580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:35:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.792+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f296c1a7ce0 con 0x7f296c1014f0 2024-04-02T13:35:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.792+0000 7f2962ffd700 1 -- 172.21.15.122:0/3322898731 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+88 (secure 0 0 0) 0x7f296809cfb0 con 0x7f296c1014f0 2024-04-02T13:35:05.795 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.794+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 msgr2=0x7f29580a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.794+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 0x7f29580a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.794+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 msgr2=0x7f296c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.794+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f296800d980 tx=0x7f296800dc90 comp rx=0 tx=0).stop 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 shutdown_connections 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f296c107500 0x7f296c19f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f29580a3c90 0x7f29580a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f296c1014f0 0x7f296c19af90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 --2- 172.21.15.122:0/3322898731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f296c101eb0 0x7f296c19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 >> 172.21.15.122:0/3322898731 conn(0x7f296c078420 msgr2=0x7f296c06a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 shutdown_connections 2024-04-02T13:35:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:05.795+0000 7f297418b700 1 -- 172.21.15.122:0/3322898731 wait complete. 2024-04-02T13:35:06.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3322898731' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:35:06.215 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3322898731' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:35:06.216 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:06.081+0000 7f5f5db3c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:35:06.221 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3322898731' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-02T13:35:06.222 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:06.085+0000 7f995d527200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:35:06.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:06.217+0000 7f995d527200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:35:06.497 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:06.191+0000 7f573f99d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-02T13:35:06.498 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:06.322+0000 7f573f99d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:35:06.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:06.212+0000 7f5f5db3c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-02T13:35:06.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-02T13:35:06.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-02T13:35:06.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-04-02T13:35:06.528 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-02T13:35:06.529 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:06.529 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-02T13:35:06.529 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-02T13:35:06.529 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-02T13:35:06.529 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:35:06.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:06.529 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-04-02T13:35:06.547 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-02T13:35:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-02T13:35:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-04-02T13:35:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:35:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:35:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-02T13:35:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-04-02T13:35:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-04-02T13:35:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-04-02T13:35:06.551 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-02T13:35:06.551 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-02T13:35:06.552 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:35:06.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:06.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:06.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:06.703 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-02T13:35:06.730 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-04-02T13:35:06.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:06.732 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-04-02T13:35:06.775 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:06.673+0000 7f573f99d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:35:06.775 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:06.771+0000 7f573f99d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:35:06.787 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-04-02T13:35:06.787 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-02T13:35:06.787 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-02T13:35:06.788 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-02T13:35:06.788 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-02T13:35:06.788 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:35:06.840 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:06.549+0000 7f5f5db3c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:35:06.841 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:06.639+0000 7f5f5db3c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:35:06.847 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:06.556+0000 7f995d527200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-02T13:35:06.847 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:06.647+0000 7f995d527200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-02T13:35:06.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:06.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:06.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:06.926 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-02T13:35:06.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:06.940 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-04-02T13:35:06.997 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:35:06.997 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:35:06.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:35:06.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-02T13:35:06.998 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:35:07.070 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:06.982+0000 7f573f99d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.121 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-02T13:35:07.121 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:35:07.121 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:35:07.121 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-02T13:35:07.121 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-02T13:35:07.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-02T13:35:07.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-02T13:35:07.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-02T13:35:07.122 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:35:07.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:07.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:06.843+0000 7f995d527200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.235 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:07.028+0000 7f995d527200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:35:07.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:06.837+0000 7f5f5db3c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.267 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:07.025+0000 7f5f5db3c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:35:07.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.290 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-02T13:35:07.290 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-04-02T13:35:07.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-04-02T13:35:07.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-04-02T13:35:07.290 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-04-02T13:35:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:07.349 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:07.183+0000 7f573f99d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-02T13:35:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.374 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:35:07.374 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:07.390 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-02T13:35:07.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-02T13:35:07.391 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:07.447 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-02T13:35:07.447 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-02T13:35:07.447 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/7cc47f64-78ec-40c5-b8db-79497ae99943 2024-04-02T13:35:07.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:07.448 DEBUG:teuthology.orchestra.run.smithi122:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/7cc47f64-78ec-40c5-b8db-79497ae99943 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:35:07.532 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:35:07.532 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-04-02T13:35:07.532 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-04-02T13:35:07.578 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:07.312+0000 7f995d527200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.580 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:07.314+0000 7f5f5db3c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.619 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:07.492+0000 7f573f99d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:invalid new device string format 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: resolved to: "172.21.15.122:6789,172.21.15.187:6789,172.21.15.188:6789" 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.122:6789,172.21.15.187:6789,172.21.15.188:6789:/volumes/qa/sv_1/7cc47f64-78ec-40c5-b8db-79497ae99943 2024-04-02T13:35:07.666 INFO:teuthology.orchestra.run.smithi122.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=3c818384-f0f4-11ee-b647-cb9ed24678a4" will pass to kernel 2024-04-02T13:35:07.667 INFO:teuthology.orchestra.run.smithi122.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-02T13:35:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: You just mounted a file system that supports labels which does not 2024-04-02T13:35:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-02T13:35:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: applications will generate AVC messages and not be allowed access to 2024-04-02T13:35:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-02T13:35:07.669 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-02T13:35:07.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:07.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:07.697 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:35:07.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:35:07.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.794 INFO:teuthology.orchestra.run.smithi122.stdout:client25091 2024-04-02T13:35:07.794 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:35:07.795 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:35:07.811 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:35:07.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:35:07.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.908 INFO:teuthology.orchestra.run.smithi122.stdout:client25091 2024-04-02T13:35:07.908 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-02T13:35:07.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-02T13:35:07.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:07.964 INFO:teuthology.orchestra.run.smithi122.stdout:client25091 2024-04-02T13:35:07.964 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4.client25091/status') 2024-04-02T13:35:07.985 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:07.574+0000 7f995d527200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:07.985 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:07.815+0000 7f995d527200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:35:08.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-02T13:35:08.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:07.577+0000 7f5f5db3c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:08.017 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:07.816+0000 7f5f5db3c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:35:08.021 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:07.765+0000 7f573f99d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-02T13:35:08.021 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:08.018+0000 7f573f99d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-02T13:35:08.025 INFO:teuthology.orchestra.run.smithi122.stdout:instance: client.25091 (3)172.21.15.122:0/2225497597 2024-04-02T13:35:08.026 INFO:teuthology.orchestra.run.smithi122.stdout:blocklisted: false 2024-04-02T13:35:08.026 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records in 2024-04-02T13:35:08.026 INFO:teuthology.orchestra.run.smithi122.stderr:0+1 records out 2024-04-02T13:35:08.026 INFO:teuthology.orchestra.run.smithi122.stderr:72 bytes copied, 2.1174e-05 s, 3.4 MB/s 2024-04-02T13:35:08.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-02T13:35:08.033 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:35:08.042 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:35:08.042 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:35:08.043 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 -- 172.21.15.122:0/2837011357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c1078a0 msgr2=0x7f808c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 --2- 172.21.15.122:0/2837011357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c1078a0 0x7f808c109c90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8084009ca0 tx=0x7f808402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 -- 172.21.15.122:0/2837011357 shutdown_connections 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 --2- 172.21.15.122:0/2837011357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c1078a0 0x7f808c109c90 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 --2- 172.21.15.122:0/2837011357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c104f70 0x7f808c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 --2- 172.21.15.122:0/2837011357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c102640 0x7f808c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 -- 172.21.15.122:0/2837011357 >> 172.21.15.122:0/2837011357 conn(0x7f808c0fbf90 msgr2=0x7f808c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 -- 172.21.15.122:0/2837011357 shutdown_connections 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.146+0000 7f8093ba8640 1 -- 172.21.15.122:0/2837011357 wait complete. 2024-04-02T13:35:08.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 Processor -- start 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 -- start start 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c104f70 0x7f808c1005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c1078a0 0x7f808c100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10d7c0 con 0x7f808c102640 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10d640 con 0x7f808c104f70 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8093ba8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f808c10d940 con 0x7f808c1078a0 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36154/0 (socket says 172.21.15.122:36154) 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 -- 172.21.15.122:0/451266781 learned_addr learned my addr 172.21.15.122:0/451266781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091143640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c104f70 0x7f808c1005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8092145640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c1078a0 0x7f808c100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c1078a0 msgr2=0x7f808c100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c1078a0 0x7f808c100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c104f70 msgr2=0x7f808c1005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c104f70 0x7f808c1005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.147+0000 7f8091944640 1 -- 172.21.15.122:0/451266781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c1013c0 con 0x7f808c102640 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8091944640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f808800ea60 tx=0x7f808800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808800cde0 con 0x7f808c102640 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80880035f0 con 0x7f808c102640 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1a1ea0 con 0x7f808c102640 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088010670 con 0x7f808c102640 2024-04-02T13:35:08.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.148+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c1a22b0 con 0x7f808c102640 2024-04-02T13:35:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.149+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c0fda80 con 0x7f808c102640 2024-04-02T13:35:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.150+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f80880037a0 con 0x7f808c102640 2024-04-02T13:35:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.150+0000 7f8082ffd640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 0x7f80780a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.151+0000 7f8091143640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 msgr2=0x7f80780a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:08.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.151+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f80880d0af0 con 0x7f808c102640 2024-04-02T13:35:08.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.151+0000 7f8091143640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 0x7f80780a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:08.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.155+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f808809d120 con 0x7f808c102640 2024-04-02T13:35:08.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.352+0000 7f8091143640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 msgr2=0x7f80780a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:08.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.352+0000 7f8091143640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 0x7f80780a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-02T13:35:08.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.380+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f808c061f40 con 0x7f808c102640 2024-04-02T13:35:08.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.432+0000 7f8082ffd640 1 -- 172.21.15.122:0/451266781 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f808809ca20 con 0x7f808c102640 2024-04-02T13:35:08.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 msgr2=0x7f80780a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:08.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 0x7f80780a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 msgr2=0x7f808c101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f808800ea60 tx=0x7f808800ef20 comp rx=0 tx=0).stop 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 shutdown_connections 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f808c1078a0 0x7f808c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f80780a3ca0 0x7f80780a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f808c104f70 0x7f808c1005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 --2- 172.21.15.122:0/451266781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f808c102640 0x7f808c101fb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 >> 172.21.15.122:0/451266781 conn(0x7f808c0fbf90 msgr2=0x7f808c102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 shutdown_connections 2024-04-02T13:35:08.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.435+0000 7f8093ba8640 1 -- 172.21.15.122:0/451266781 wait complete. 2024-04-02T13:35:08.449 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-02T13:35:08.461 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-02T13:35:08.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/3480542633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc101c10 msgr2=0x7f4fcc102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/3480542633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc101c10 0x7f4fcc102090 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009900 tx=0x7f4fbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/3480542633 shutdown_connections 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/3480542633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 0x7f4fcc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/3480542633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc101c10 0x7f4fcc102090 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/3480542633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc100a20 0x7f4fcc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.630+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/3480542633 >> 172.21.15.122:0/3480542633 conn(0x7f4fcc0fbf90 msgr2=0x7f4fcc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/3480542633 shutdown_connections 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/3480542633 wait complete. 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 Processor -- start 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- start start 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 0x7f4fcc197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 0x7f4fcc19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc10c680 con 0x7f4fcc100a20 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc10c500 con 0x7f4fcc1025d0 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.631+0000 7f4fd1d8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc10c800 con 0x7f4fcc101c10 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37050/0 (socket says 172.21.15.122:37050) 2024-04-02T13:35:08.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 -- 172.21.15.122:0/665410149 learned_addr learned my addr 172.21.15.122:0/665410149 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcbfff640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 0x7f4fcc19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcaffd640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 0x7f4fcc197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 -- 172.21.15.122:0/665410149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 msgr2=0x7f4fcc197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 0x7f4fcc197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 -- 172.21.15.122:0/665410149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 msgr2=0x7f4fcc19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 0x7f4fcc19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 -- 172.21.15.122:0/665410149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc19e950 con 0x7f4fcc100a20 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcaffd640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 0x7f4fcc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fcb7fe640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e9c0 tx=0x7f4fc000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc000cde0 con 0x7f4fcc100a20 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fc0010910 con 0x7f4fcc100a20 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc0018a00 con 0x7f4fcc100a20 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc19ec40 con 0x7f4fcc100a20 2024-04-02T13:35:08.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.632+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc19f190 con 0x7f4fcc100a20 2024-04-02T13:35:08.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.633+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc107dc0 con 0x7f4fcc100a20 2024-04-02T13:35:08.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.635+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f4fc001f020 con 0x7f4fcc100a20 2024-04-02T13:35:08.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.635+0000 7f4fc8ff9640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 0x7f4fb40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.635+0000 7f4fcaffd640 1 -- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 msgr2=0x7f4fb40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:08.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.635+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4fc00d1050 con 0x7f4fcc100a20 2024-04-02T13:35:08.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.635+0000 7f4fcaffd640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 0x7f4fb40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:08.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.639+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4fc009d680 con 0x7f4fcc100a20 2024-04-02T13:35:08.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T13:35:08.735 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:08.623+0000 7f995d527200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:35:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T13:35:08.767 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:08.625+0000 7f5f5db3c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:35:08.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-02T13:35:08.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.826+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4fcc02d560 con 0x7f4fcc100a20 2024-04-02T13:35:08.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.828+0000 7f4fc8ff9640 1 -- 172.21.15.122:0/665410149 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24843 (secure 0 0 0) 0x7f4fc009cf80 con 0x7f4fcc100a20 2024-04-02T13:35:08.831 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:08.831 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":94,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:34:55.620203+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 msgr2=0x7f4fb40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 0x7f4fb40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 msgr2=0x7f4fcc196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4fc000e9c0 tx=0x7f4fc000ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 shutdown_connections 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f4fb40a3c00 0x7f4fb40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4fcc1025d0 0x7f4fcc19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fcc101c10 0x7f4fcc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 --2- 172.21.15.122:0/665410149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fcc100a20 0x7f4fcc196d40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 >> 172.21.15.122:0/665410149 conn(0x7f4fcc0fbf90 msgr2=0x7f4fcc104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 shutdown_connections 2024-04-02T13:35:08.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.830+0000 7f4fd1d8d640 1 -- 172.21.15.122:0/665410149 wait complete. 2024-04-02T13:35:08.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:35:08.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/685004002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec102d00 msgr2=0x7f3cec103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/685004002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec102d00 0x7f3cec103180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009900 tx=0x7f3cd802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/685004002 shutdown_connections 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/685004002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec1036c0 0x7f3cec109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/685004002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec102d00 0x7f3cec103180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/685004002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec101b10 0x7f3cec101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/685004002 >> 172.21.15.122:0/685004002 conn(0x7f3cec0fd060 msgr2=0x7f3cec0ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:08.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/685004002 shutdown_connections 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/685004002 wait complete. 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 Processor -- start 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- start start 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 0x7f3cec19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec102d00 0x7f3cec19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d790 con 0x7f3cec1036c0 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d610 con 0x7f3cec101b10 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.950+0000 7f3cf2ed5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d910 con 0x7f3cec102d00 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37064/0 (socket says 172.21.15.122:37064) 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 -- 172.21.15.122:0/2844210337 learned_addr learned my addr 172.21.15.122:0/2844210337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf0c71640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 0x7f3cec19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cebfff640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec102d00 0x7f3cec19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 -- 172.21.15.122:0/2844210337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec102d00 msgr2=0x7f3cec19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec102d00 0x7f3cec19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 -- 172.21.15.122:0/2844210337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 msgr2=0x7f3cec19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 0x7f3cec19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 -- 172.21.15.122:0/2844210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec1a1c70 con 0x7f3cec1036c0 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf0c71640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 0x7f3cec19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf1472640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400eb10 tx=0x7f3ce400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:08.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4004b20 con 0x7f3cec1036c0 2024-04-02T13:35:08.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce4004cc0 con 0x7f3cec1036c0 2024-04-02T13:35:08.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec1a1f60 con 0x7f3cec1036c0 2024-04-02T13:35:08.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.951+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4010760 con 0x7f3cec1036c0 2024-04-02T13:35:08.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.952+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a23d0 con 0x7f3cec1036c0 2024-04-02T13:35:08.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.952+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec1a2660 con 0x7f3cec1036c0 2024-04-02T13:35:08.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.954+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f3ce4005020 con 0x7f3cec1036c0 2024-04-02T13:35:08.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.954+0000 7f3ce9ffb640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 0x7f3cdc0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:08.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.955+0000 7f3cf0c71640 1 -- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 msgr2=0x7f3cdc0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:08.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.955+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3ce40d0d80 con 0x7f3cec1036c0 2024-04-02T13:35:08.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.955+0000 7f3cf0c71640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 0x7f3cdc0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:08.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:08.959+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3ce409d3b0 con 0x7f3cec1036c0 2024-04-02T13:35:09.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.134+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3cec1a28d0 con 0x7f3cec1036c0 2024-04-02T13:35:09.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.134+0000 7f3ce9ffb640 1 -- 172.21.15.122:0/2844210337 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f3ce409ccb0 con 0x7f3cec1036c0 2024-04-02T13:35:09.137 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:35:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.136+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 msgr2=0x7f3cdc0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.136+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 0x7f3cdc0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 msgr2=0x7f3cec1a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3ce400eb10 tx=0x7f3ce400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 shutdown_connections 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3cec1036c0 0x7f3cec1a1600 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cec102d00 0x7f3cec19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f3cdc0a3c00 0x7f3cdc0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 --2- 172.21.15.122:0/2844210337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3cec101b10 0x7f3cec19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 >> 172.21.15.122:0/2844210337 conn(0x7f3cec0fd060 msgr2=0x7f3cec073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 shutdown_connections 2024-04-02T13:35:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.137+0000 7f3cf2ed5640 1 -- 172.21.15.122:0/2844210337 wait complete. 2024-04-02T13:35:09.149 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:08.878+0000 7f573f99d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-02T13:35:09.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 -- 172.21.15.122:0/3359283547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c1078a0 msgr2=0x7f721c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 --2- 172.21.15.122:0/3359283547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c1078a0 0x7f721c109c90 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7214009ca0 tx=0x7f721402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 -- 172.21.15.122:0/3359283547 shutdown_connections 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 --2- 172.21.15.122:0/3359283547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c1078a0 0x7f721c109c90 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 --2- 172.21.15.122:0/3359283547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c104f70 0x7f721c107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 --2- 172.21.15.122:0/3359283547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 0x7f721c104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 -- 172.21.15.122:0/3359283547 >> 172.21.15.122:0/3359283547 conn(0x7f721c0fbf90 msgr2=0x7f721c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 -- 172.21.15.122:0/3359283547 shutdown_connections 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.254+0000 7f7220c55640 1 -- 172.21.15.122:0/3359283547 wait complete. 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 Processor -- start 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 -- start start 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 0x7f721c101fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c1078a0 0x7f721c100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10d790 con 0x7f721c104f70 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10d610 con 0x7f721c1078a0 2024-04-02T13:35:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7220c55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10d910 con 0x7f721c102640 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37092/0 (socket says 172.21.15.122:37092) 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 -- 172.21.15.122:0/685811415 learned_addr learned my addr 172.21.15.122:0/685811415 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f721a59c640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 0x7f721c101fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f721ad9d640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c1078a0 0x7f721c100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 -- 172.21.15.122:0/685811415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 msgr2=0x7f721c101fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 0x7f721c101fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 -- 172.21.15.122:0/685811415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c1078a0 msgr2=0x7f721c100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c1078a0 0x7f721c100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 -- 172.21.15.122:0/685811415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c1013c0 con 0x7f721c104f70 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.255+0000 7f7219d9b640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9c0 tx=0x7f721000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.256+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721000cd80 con 0x7f721c104f70 2024-04-02T13:35:09.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.256+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c1a1f20 con 0x7f721c104f70 2024-04-02T13:35:09.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.256+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7210005500 con 0x7f721c104f70 2024-04-02T13:35:09.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.256+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210010690 con 0x7f721c104f70 2024-04-02T13:35:09.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.256+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c1a2390 con 0x7f721c104f70 2024-04-02T13:35:09.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.257+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f721c0fda80 con 0x7f721c104f70 2024-04-02T13:35:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.258+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f72100046c0 con 0x7f721c104f70 2024-04-02T13:35:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.258+0000 7f720b7fe640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 0x7f72040a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.258+0000 7f721a59c640 1 -- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 msgr2=0x7f72040a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.259+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f72100d0b30 con 0x7f721c104f70 2024-04-02T13:35:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.259+0000 7f721a59c640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 0x7f72040a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.263+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f721009d160 con 0x7f721c104f70 2024-04-02T13:35:09.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.443+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f721c0627a0 con 0x7f721c104f70 2024-04-02T13:35:09.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.444+0000 7f720b7fe640 1 -- 172.21.15.122:0/685811415 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f721009ca60 con 0x7f721c104f70 2024-04-02T13:35:09.446 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:09.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 msgr2=0x7f72040a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 0x7f72040a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 msgr2=0x7f721c1005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9c0 tx=0x7f721000ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 shutdown_connections 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f72040a3c00 0x7f72040a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f721c1078a0 0x7f721c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f721c104f70 0x7f721c1005f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 --2- 172.21.15.122:0/685811415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721c102640 0x7f721c101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 >> 172.21.15.122:0/685811415 conn(0x7f721c0fbf90 msgr2=0x7f721c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 shutdown_connections 2024-04-02T13:35:09.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.446+0000 7f7220c55640 1 -- 172.21.15.122:0/685811415 wait complete. 2024-04-02T13:35:09.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:35:09.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:09.198+0000 7f995d527200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:35:09.485 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:09.297+0000 7f995d527200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:35:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T13:35:09.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:35:09.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/665410149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:09.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2844210337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:09.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:09.200+0000 7f5f5db3c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:35:09.517 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:09.298+0000 7f5f5db3c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:35:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T13:35:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:35:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/665410149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:09.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2844210337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.564+0000 7f04ef3a8640 1 -- 172.21.15.122:0/3472268173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 msgr2=0x7f04e8108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.564+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/3472268173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8108e40 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04e0009ca0 tx=0x7f04e002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.564+0000 7f04ef3a8640 1 -- 172.21.15.122:0/3472268173 shutdown_connections 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/3472268173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8108e40 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/3472268173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/3472268173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 0x7f04e8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- 172.21.15.122:0/3472268173 >> 172.21.15.122:0/3472268173 conn(0x7f04e80fbf90 msgr2=0x7f04e80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- 172.21.15.122:0/3472268173 shutdown_connections 2024-04-02T13:35:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- 172.21.15.122:0/3472268173 wait complete. 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 Processor -- start 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- start start 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 0x7f04e8104940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810c680 con 0x7f04e81025b0 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810c500 con 0x7f04e8100a00 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ef3a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04e810c800 con 0x7f04e8101bf0 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.565+0000 7f04ed945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37116/0 (socket says 172.21.15.122:37116) 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 -- 172.21.15.122:0/1207705724 learned_addr learned my addr 172.21.15.122:0/1207705724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed144640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 0x7f04e8104940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ec943640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 -- 172.21.15.122:0/1207705724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 msgr2=0x7f04e8102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 -- 172.21.15.122:0/1207705724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 msgr2=0x7f04e8104940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 0x7f04e8104940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 -- 172.21.15.122:0/1207705724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e8103dd0 con 0x7f04e81025b0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ec943640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ed945640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04e0037040 tx=0x7f04e0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0041020 con 0x7f04e81025b0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04e0034cd0 con 0x7f04e81025b0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e81a1f40 con 0x7f04e81025b0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0039040 con 0x7f04e81025b0 2024-04-02T13:35:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.566+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a23a0 con 0x7f04e81025b0 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.567+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e81a2690 con 0x7f04e81025b0 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.568+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138664+0+0 (secure 0 0 0) 0x7f04e0044020 con 0x7f04e81025b0 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.569+0000 7f04de7fc640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 0x7f04d40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.569+0000 7f04ed144640 1 -- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 msgr2=0x7f04d40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6800/3450525377 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.569+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f04e00f2610 con 0x7f04e81025b0 2024-04-02T13:35:09.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.569+0000 7f04ed144640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 0x7f04d40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-02T13:35:09.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.573+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f04e004b020 con 0x7f04e81025b0 2024-04-02T13:35:09.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.758+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04e81a29f0 con 0x7f04e81025b0 2024-04-02T13:35:09.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.759+0000 7f04de7fc640 1 -- 172.21.15.122:0/1207705724 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f04e00becd0 con 0x7f04e81025b0 2024-04-02T13:35:09.761 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:35:09.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 msgr2=0x7f04d40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:35:09.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 0x7f04d40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 msgr2=0x7f04e8103510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04e0037040 tx=0x7f04e0032f40 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 shutdown_connections 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e81025b0 0x7f04e8103510 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04e8101bf0 0x7f04e8102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.122:6800/3450525377,v1:172.21.15.122:6801/3450525377] conn(0x7f04d40a3c00 0x7f04d40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 --2- 172.21.15.122:0/1207705724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04e8100a00 0x7f04e8104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 >> 172.21.15.122:0/1207705724 conn(0x7f04e80fbf90 msgr2=0x7f04e80fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 shutdown_connections 2024-04-02T13:35:09.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.761+0000 7f04ef3a8640 1 -- 172.21.15.122:0/1207705724 wait complete. 2024-04-02T13:35:09.773 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:09.470+0000 7f573f99d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-02T13:35:09.773 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:09.580+0000 7f573f99d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-02T13:35:09.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/451266781' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-02T13:35:09.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:35:09.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/665410149' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:09.774 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2844210337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:09.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 -- 172.21.15.122:0/1829386225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8101bf0 msgr2=0x7f22f8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1829386225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8101bf0 0x7f22f8102070 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009900 tx=0x7f22e402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 -- 172.21.15.122:0/1829386225 shutdown_connections 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1829386225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f81025b0 0x7f22f8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1829386225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8101bf0 0x7f22f8102070 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1829386225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f8100a00 0x7f22f8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.888+0000 7f22feaeb640 1 -- 172.21.15.122:0/1829386225 >> 172.21.15.122:0/1829386225 conn(0x7f22f80fbf90 msgr2=0x7f22f80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 -- 172.21.15.122:0/1829386225 shutdown_connections 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 -- 172.21.15.122:0/1829386225 wait complete. 2024-04-02T13:35:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 Processor -- start 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 -- start start 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f8101bf0 0x7f22f819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.889+0000 7f22feaeb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f81025b0 0x7f22f81a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22feaeb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810c660 con 0x7f22f8100a00 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22feaeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810c4e0 con 0x7f22f8101bf0 2024-04-02T13:35:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22feaeb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810c7e0 con 0x7f22f81025b0 2024-04-02T13:35:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37148/0 (socket says 172.21.15.122:37148) 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 -- 172.21.15.122:0/1392905151 learned_addr learned my addr 172.21.15.122:0/1392905151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fd088640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f81025b0 0x7f22f81a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22f7fff640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f8101bf0 0x7f22f819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 -- 172.21.15.122:0/1392905151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f81025b0 msgr2=0x7f22f81a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f81025b0 0x7f22f81a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 -- 172.21.15.122:0/1392905151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f8101bf0 msgr2=0x7f22f819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f8101bf0 0x7f22f819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 -- 172.21.15.122:0/1392905151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f81a60a0 con 0x7f22f8100a00 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22fc887640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d770 tx=0x7f22ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec028070 con 0x7f22f8100a00 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f806b580 con 0x7f22f8100a00 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.890+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ec009c30 con 0x7f22f8100a00 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.891+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec028070 con 0x7f22f8100a00 2024-04-02T13:35:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.891+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f806bb20 con 0x7f22f8100a00 2024-04-02T13:35:09.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.892+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f806bde0 con 0x7f22f8100a00 2024-04-02T13:35:09.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.894+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f22ec005030 con 0x7f22f8100a00 2024-04-02T13:35:09.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.894+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f22ec0e0880 con 0x7f22f8100a00 2024-04-02T13:35:09.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:09.897+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f22ec0ad1b0 con 0x7f22f8100a00 2024-04-02T13:35:09.964 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:35:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y[37447]: 2024-04-02T13:35:09.514+0000 7f995d527200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:35:09.965 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:35:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z[37978]: 2024-04-02T13:35:09.514+0000 7f5f5db3c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:35:10.053 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:09.811+0000 7f573f99d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-02T13:35:10.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.165+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f22f806c240 con 0x7f22f8100a00 2024-04-02T13:35:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.165+0000 7f22f5ffb640 1 -- 172.21.15.122:0/1392905151 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f22ec0acab0 con 0x7f22f8100a00 2024-04-02T13:35:10.168 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 msgr2=0x7f22f819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00d770 tx=0x7f22ec00dc30 comp rx=0 tx=0).stop 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 shutdown_connections 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22f81025b0 0x7f22f81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22f8101bf0 0x7f22f819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 --2- 172.21.15.122:0/1392905151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22f8100a00 0x7f22f819eb80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.167+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 >> 172.21.15.122:0/1392905151 conn(0x7f22f80fbf90 msgr2=0x7f22f8068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.168+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 shutdown_connections 2024-04-02T13:35:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.168+0000 7f22feaeb640 1 -- 172.21.15.122:0/1392905151 wait complete. 2024-04-02T13:35:10.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 -- 172.21.15.122:0/1420558773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 msgr2=0x7f4a98104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/1420558773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98104990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009900 tx=0x7f4a8802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 -- 172.21.15.122:0/1420558773 shutdown_connections 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/1420558773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a98109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/1420558773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 0x7f4a981072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/1420558773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98104990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 -- 172.21.15.122:0/1420558773 >> 172.21.15.122:0/1420558773 conn(0x7f4a980fbf90 msgr2=0x7f4a980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 -- 172.21.15.122:0/1420558773 shutdown_connections 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.286+0000 7f4a9d24a640 1 -- 172.21.15.122:0/1420558773 wait complete. 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 Processor -- start 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 -- start start 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 0x7f4a981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a9819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810d7a0 con 0x7f4a981025a0 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810d620 con 0x7f4a98107800 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9d24a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9810d920 con 0x7f4a98104ed0 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a96d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a9819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a96d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37246/0 (socket says 172.21.15.122:37246) 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a96d9d640 1 -- 172.21.15.122:0/3942810333 learned_addr learned my addr 172.21.15.122:0/3942810333 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.287+0000 7f4a9659c640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 0x7f4a981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 -- 172.21.15.122:0/3942810333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 msgr2=0x7f4a981983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 0x7f4a981983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 -- 172.21.15.122:0/3942810333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 msgr2=0x7f4a9819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a9819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 -- 172.21.15.122:0/3942810333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9819fa90 con 0x7f4a981025a0 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a9759e640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a9819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a96d9d640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009230 tx=0x7f4a8802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88041070 con 0x7f4a981025a0 2024-04-02T13:35:10.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9819fd20 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a981a01b0 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a88004450 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.288+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8804b060 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.289+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a840052c0 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.290+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f4a88044070 con 0x7f4a981025a0 2024-04-02T13:35:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.291+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4a880f1cb0 con 0x7f4a981025a0 2024-04-02T13:35:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.295+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f4a88003e20 con 0x7f4a981025a0 2024-04-02T13:35:10.320 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f569f3af700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f569f3af700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f569f3af700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f569f3af700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f569f3af700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f56a23b5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f56a23b5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f56a23b5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f56a23b5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.321 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:35:10.057+0000 7f56a23b5700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:35:10.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.485+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a84005880 con 0x7f4a981025a0 2024-04-02T13:35:10.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.488+0000 7f4a7ffff640 1 -- 172.21.15.122:0/3942810333 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25227 (secure 0 0 0) 0x7f4a880bdf90 con 0x7f4a981025a0 2024-04-02T13:35:10.491 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:10.491 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":95,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:35:09.813014+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 msgr2=0x7f4a98197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a88009230 tx=0x7f4a8802c650 comp rx=0 tx=0).stop 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 shutdown_connections 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a98107800 0x7f4a9819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a98104ed0 0x7f4a981983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 --2- 172.21.15.122:0/3942810333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a981025a0 0x7f4a98197e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 >> 172.21.15.122:0/3942810333 conn(0x7f4a980fbf90 msgr2=0x7f4a98103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 shutdown_connections 2024-04-02T13:35:10.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.490+0000 7f4a9d24a640 1 -- 172.21.15.122:0/3942810333 wait complete. 2024-04-02T13:35:10.504 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:35:10.505 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:35:10.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 -- 172.21.15.122:0/782193872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc107840 msgr2=0x7fd6fc109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 --2- 172.21.15.122:0/782193872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc107840 0x7fd6fc109c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd6f4009ca0 tx=0x7fd6f402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 -- 172.21.15.122:0/782193872 shutdown_connections 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 --2- 172.21.15.122:0/782193872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc107840 0x7fd6fc109c30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 --2- 172.21.15.122:0/782193872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 --2- 172.21.15.122:0/782193872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc1049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 -- 172.21.15.122:0/782193872 >> 172.21.15.122:0/782193872 conn(0x7fd6fc0fbf90 msgr2=0x7fd6fc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 -- 172.21.15.122:0/782193872 shutdown_connections 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.612+0000 7fd703b0e640 1 -- 172.21.15.122:0/782193872 wait complete. 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 Processor -- start 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 -- start start 2024-04-02T13:35:10.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc107840 0x7fd6fc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc10d7a0 con 0x7fd6fc1025e0 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc10d620 con 0x7fd6fc107840 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd703b0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6fc10d920 con 0x7fd6fc104f10 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd7018aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd7010a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd7018aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37264/0 (socket says 172.21.15.122:37264) 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.613+0000 7fd7018aa640 1 -- 172.21.15.122:0/248007213 learned_addr learned my addr 172.21.15.122:0/248007213 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7020ab640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc107840 0x7fd6fc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 -- 172.21.15.122:0/248007213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc107840 msgr2=0x7fd6fc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc107840 0x7fd6fc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 -- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 msgr2=0x7fd6fc197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 -- 172.21.15.122:0/248007213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc19fa40 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7018aa640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd7010a9640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc198360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e9c0 tx=0x7fd6ec00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00ccc0 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec005020 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc19fd30 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec004a00 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.614+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6fc1a01d0 con 0x7fd6fc104f10 2024-04-02T13:35:10.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.615+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6fc0fda80 con 0x7fd6fc104f10 2024-04-02T13:35:10.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.617+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7fd6ec0051d0 con 0x7fd6fc104f10 2024-04-02T13:35:10.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.618+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd6ec0d0460 con 0x7fd6fc104f10 2024-04-02T13:35:10.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.620+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd6ec09ce40 con 0x7fd6fc104f10 2024-04-02T13:35:10.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/685811415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Standby manager daemon y restarted 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Standby manager daemon y started 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Standby manager daemon z restarted 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Standby manager daemon z started 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:35:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1207705724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Active manager daemon x restarted 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Activating manager daemon x 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: mgrmap e20: x(active, starting, since 0.0115737s), standbys: z, y 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:10.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:35:10.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: Manager daemon x is now available 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:10.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:35:10.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:35:10.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:10 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1392905151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/685811415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Standby manager daemon y restarted 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Standby manager daemon y started 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Standby manager daemon z restarted 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Standby manager daemon z started 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1207705724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Active manager daemon x restarted 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Activating manager daemon x 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:35:10.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: mgrmap e20: x(active, starting, since 0.0115737s), standbys: z, y 2024-04-02T13:35:10.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:35:10.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:35:10.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:35:10.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:10.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:35:10.771 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:35:10.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: Manager daemon x is now available 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:35:10.773 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:10 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1392905151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/685811415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Standby manager daemon y restarted 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Standby manager daemon y started 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Standby manager daemon z restarted 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Standby manager daemon z started 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.187:0/2896140231' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-02T13:35:10.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.? 172.21.15.188:0/394665708' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1207705724' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Active manager daemon x restarted 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Activating manager daemon x 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: osdmap e95: 12 total, 12 up, 12 in 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: mgrmap e20: x(active, starting, since 0.0115737s), standbys: z, y 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:35:10.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-02T13:35:10.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-02T13:35:10.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: Manager daemon x is now available 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-02T13:35:10.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:10 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1392905151' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:10.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.830+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6fc02d5c0 con 0x7fd6fc104f10 2024-04-02T13:35:10.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.831+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fd6ec098490 con 0x7fd6fc104f10 2024-04-02T13:35:10.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.832+0000 7fd6f2ffd640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 0x7fd6e80ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.833+0000 7fd7018aa640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 0x7fd6e80ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.834+0000 7fd6f2ffd640 1 -- 172.21.15.122:0/248007213 <== mon.1 v2:172.21.15.188:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7fd6ec048530 con 0x7fd6fc104f10 2024-04-02T13:35:10.837 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:35:10.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.837+0000 7fd7018aa640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 0x7fd6e80ae480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8011550 tx=0x7fd6f8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:10.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 msgr2=0x7fd6e80ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 0x7fd6e80ae480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8011550 tx=0x7fd6f8005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:10.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 msgr2=0x7fd6fc198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc198360 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e9c0 tx=0x7fd6ec00ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 shutdown_connections 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6e80ac090 0x7fd6e80ae480 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd6f8011550 tx=0x7fd6f8005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6fc107840 0x7fd6fc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6fc104f10 0x7fd6fc198360 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00e9c0 tx=0x7fd6ec00ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 --2- 172.21.15.122:0/248007213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6fc1025e0 0x7fd6fc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 >> 172.21.15.122:0/248007213 conn(0x7fd6fc0fbf90 msgr2=0x7fd6fc102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 shutdown_connections 2024-04-02T13:35:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.839+0000 7fd703b0e640 1 -- 172.21.15.122:0/248007213 wait complete. 2024-04-02T13:35:10.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:35:10.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- 172.21.15.122:0/661447983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 msgr2=0x7fd15c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- 172.21.15.122:0/661447983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c100e20 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd14c009900 tx=0x7fd14c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:10.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- 172.21.15.122:0/661447983 shutdown_connections 2024-04-02T13:35:10.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- 172.21.15.122:0/661447983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 0x7fd15c108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- 172.21.15.122:0/661447983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 0x7fd15c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- 172.21.15.122:0/661447983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c100e20 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- 172.21.15.122:0/661447983 >> 172.21.15.122:0/661447983 conn(0x7fd15c0fbf90 msgr2=0x7fd15c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- 172.21.15.122:0/661447983 shutdown_connections 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- 172.21.15.122:0/661447983 wait complete. 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 Processor -- start 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 -- start start 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.970+0000 7fd1628d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 0x7fd15c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 0x7fd15c19e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c10c6d0 con 0x7fd15c100a00 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c10c550 con 0x7fd15c101bf0 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c10c850 con 0x7fd15c1025b0 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37280/0 (socket says 172.21.15.122:37280) 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 -- 172.21.15.122:0/3862806005 learned_addr learned my addr 172.21.15.122:0/3862806005 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15b7fe640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 0x7fd15c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd160e73640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 0x7fd15c19e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 -- 172.21.15.122:0/3862806005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 msgr2=0x7fd15c19e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 0x7fd15c19e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 -- 172.21.15.122:0/3862806005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 msgr2=0x7fd15c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 0x7fd15c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 -- 172.21.15.122:0/3862806005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c19e9f0 con 0x7fd15c100a00 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd15bfff640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005ab0 tx=0x7fd14c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c041070 con 0x7fd15c100a00 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c19ec80 con 0x7fd15c100a00 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c0054e0 con 0x7fd15c100a00 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c034440 con 0x7fd15c100a00 2024-04-02T13:35:10.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.971+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c19f170 con 0x7fd15c100a00 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.972+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c107d90 con 0x7fd15c100a00 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.973+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fd14c044070 con 0x7fd15c100a00 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.974+0000 7fd1597fa640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 0x7fd1440a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.974+0000 7fd15b7fe640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 0x7fd1440a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.974+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd14c0f2350 con 0x7fd15c100a00 2024-04-02T13:35:10.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.974+0000 7fd15b7fe640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 0x7fd1440a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd15c1982e0 tx=0x7fd15000b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:10.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:10.982+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd14c0beb70 con 0x7fd15c100a00 2024-04-02T13:35:11.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.172+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd15c02d500 con 0x7fd15c100a00 2024-04-02T13:35:11.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.172+0000 7fd1597fa640 1 -- 172.21.15.122:0/3862806005 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7fd14c0be470 con 0x7fd15c100a00 2024-04-02T13:35:11.175 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:11.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 msgr2=0x7fd1440a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 0x7fd1440a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd15c1982e0 tx=0x7fd15000b390 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 msgr2=0x7fd15c196dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005ab0 tx=0x7fd14c005dc0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 shutdown_connections 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd1440a39b0 0x7fd1440a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd15c1025b0 0x7fd15c19e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd15c101bf0 0x7fd15c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 --2- 172.21.15.122:0/3862806005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd15c100a00 0x7fd15c196dd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 >> 172.21.15.122:0/3862806005 conn(0x7fd15c0fbf90 msgr2=0x7fd15c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 shutdown_connections 2024-04-02T13:35:11.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.174+0000 7fd1628d6640 1 -- 172.21.15.122:0/3862806005 wait complete. 2024-04-02T13:35:11.188 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:35:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.295+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2604017699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d81036c0 msgr2=0x7fa9d8109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.295+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2604017699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d81036c0 0x7fa9d8109f50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0009ca0 tx=0x7fa9d002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2604017699 shutdown_connections 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2604017699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d81036c0 0x7fa9d8109f50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2604017699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d8102d00 0x7fa9d8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2604017699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2604017699 >> 172.21.15.122:0/2604017699 conn(0x7fa9d80fd0a0 msgr2=0x7fa9d80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2604017699 shutdown_connections 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2604017699 wait complete. 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 Processor -- start 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- start start 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d81036c0 0x7fa9d819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d810d790 con 0x7fa9d8102d00 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d810d610 con 0x7fa9d8101b10 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.296+0000 7fa9dca6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d810d910 con 0x7fa9d81036c0 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37288/0 (socket says 172.21.15.122:37288) 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 -- 172.21.15.122:0/2025621281 learned_addr learned my addr 172.21.15.122:0/2025621281 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9d6d9d640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d81036c0 0x7fa9d819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9d659c640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 -- 172.21.15.122:0/2025621281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d81036c0 msgr2=0x7fa9d819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d81036c0 0x7fa9d819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 -- 172.21.15.122:0/2025621281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 msgr2=0x7fa9d8197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 -- 172.21.15.122:0/2025621281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d819f9f0 con 0x7fa9d8102d00 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9d659c640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:11.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cffff640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c970 tx=0x7fa9c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0007bb0 con 0x7fa9d8102d00 2024-04-02T13:35:11.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c0007d50 con 0x7fa9d8102d00 2024-04-02T13:35:11.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d819fce0 con 0x7fa9d8102d00 2024-04-02T13:35:11.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000f450 con 0x7fa9d8102d00 2024-04-02T13:35:11.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.297+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d81a0180 con 0x7fa9d8102d00 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.298+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d8108e30 con 0x7fa9d8102d00 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.299+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fa9c0005020 con 0x7fa9d8102d00 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.300+0000 7fa9cf7fe640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 0x7fa9c40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.300+0000 7fa9d659c640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 0x7fa9c40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.300+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa9c00d02b0 con 0x7fa9d8102d00 2024-04-02T13:35:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.301+0000 7fa9d659c640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 0x7fa9c40a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058c0 tx=0x7fa9c8005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.305+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa9c0014030 con 0x7fa9d8102d00 2024-04-02T13:35:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.488+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9d802d370 con 0x7fa9d8102d00 2024-04-02T13:35:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.489+0000 7fa9cf7fe640 1 -- 172.21.15.122:0/2025621281 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7fa9c009cad0 con 0x7fa9d8102d00 2024-04-02T13:35:11.491 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:35:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 msgr2=0x7fa9c40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 0x7fa9c40a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80058c0 tx=0x7fa9c8005850 comp rx=0 tx=0).stop 2024-04-02T13:35:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 msgr2=0x7fa9d8198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c970 tx=0x7fa9c000ce30 comp rx=0 tx=0).stop 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 shutdown_connections 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9c40a39b0 0x7fa9c40a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9d81036c0 0x7fa9d819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9d8102d00 0x7fa9d8198310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 --2- 172.21.15.122:0/2025621281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9d8101b10 0x7fa9d8197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 >> 172.21.15.122:0/2025621281 conn(0x7fa9d80fd0a0 msgr2=0x7fa9d8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 shutdown_connections 2024-04-02T13:35:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.491+0000 7fa9dca6a640 1 -- 172.21.15.122:0/2025621281 wait complete. 2024-04-02T13:35:11.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:35:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 -- 172.21.15.122:0/1026521433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 msgr2=0x7f9f14103250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 --2- 172.21.15.122:0/1026521433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14103250 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9f08009930 tx=0x7f9f0802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 -- 172.21.15.122:0/1026521433 shutdown_connections 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 --2- 172.21.15.122:0/1026521433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 0x7f9f1410a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 --2- 172.21.15.122:0/1026521433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14103250 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 --2- 172.21.15.122:0/1026521433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 -- 172.21.15.122:0/1026521433 >> 172.21.15.122:0/1026521433 conn(0x7f9f140fb730 msgr2=0x7f9f140fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 -- 172.21.15.122:0/1026521433 shutdown_connections 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.608+0000 7f9f18a88640 1 -- 172.21.15.122:0/1026521433 wait complete. 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 Processor -- start 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 -- start start 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 0x7f9f1419f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1410d7e0 con 0x7f9f14102dd0 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1410d660 con 0x7f9f14103790 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f18a88640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f1410d960 con 0x7f9f140ffe00 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f11d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f11d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37304/0 (socket says 172.21.15.122:37304) 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f12d9d640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 0x7f9f1419f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f11d9b640 1 -- 172.21.15.122:0/3363000180 learned_addr learned my addr 172.21.15.122:0/3363000180 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 -- 172.21.15.122:0/3363000180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 msgr2=0x7f9f1419f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 0x7f9f1419f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 -- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 msgr2=0x7f9f14198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.609+0000 7f9f1259c640 1 -- 172.21.15.122:0/3363000180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1419faf0 con 0x7f9f140ffe00 2024-04-02T13:35:11.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f1259c640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14197ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9c0 tx=0x7f9f0400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400ccc0 con 0x7f9f140ffe00 2024-04-02T13:35:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f040035f0 con 0x7f9f140ffe00 2024-04-02T13:35:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400ce60 con 0x7f9f140ffe00 2024-04-02T13:35:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1419fde0 con 0x7f9f140ffe00 2024-04-02T13:35:11.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.610+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f141a0240 con 0x7f9f140ffe00 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.611+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f14108f00 con 0x7f9f140ffe00 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.612+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f9f040037a0 con 0x7f9f140ffe00 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.613+0000 7f9f037fe640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 0x7f9efc0a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.613+0000 7f9f11d9b640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 0x7f9efc0a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.613+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9f040d0a30 con 0x7f9f140ffe00 2024-04-02T13:35:11.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.613+0000 7f9f11d9b640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 0x7f9efc0a5ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f141993e0 tx=0x7f9f0802c610 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.616+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9f0409d250 con 0x7f9f140ffe00 2024-04-02T13:35:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3942810333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:11 smithi187 ceph-mon[36064]: mgrmap e21: x(active, since 1.01821s), standbys: z, y 2024-04-02T13:35:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/248007213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3862806005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3942810333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:11 smithi188 ceph-mon[36117]: mgrmap e21: x(active, since 1.01821s), standbys: z, y 2024-04-02T13:35:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/248007213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3862806005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3942810333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:11 smithi122 ceph-mon[32114]: mgrmap e21: x(active, since 1.01821s), standbys: z, y 2024-04-02T13:35:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/248007213' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3862806005' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:11.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.819+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9f1402d5d0 con 0x7f9f140ffe00 2024-04-02T13:35:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.819+0000 7f9f037fe640 1 -- 172.21.15.122:0/3363000180 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7f9f0409cb50 con 0x7f9f140ffe00 2024-04-02T13:35:11.822 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 msgr2=0x7f9efc0a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 0x7f9efc0a5ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9f141993e0 tx=0x7f9f0802c610 comp rx=0 tx=0).stop 2024-04-02T13:35:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 msgr2=0x7f9f14197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14197ed0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9f0400e9c0 tx=0x7f9f0400ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 shutdown_connections 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9efc0a3a30 0x7f9efc0a5ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f14103790 0x7f9f1419f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f14102dd0 0x7f9f14198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 --2- 172.21.15.122:0/3363000180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f140ffe00 0x7f9f14197ed0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 >> 172.21.15.122:0/3363000180 conn(0x7f9f140fb730 msgr2=0x7f9f141060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 shutdown_connections 2024-04-02T13:35:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.822+0000 7f9f18a88640 1 -- 172.21.15.122:0/3363000180 wait complete. 2024-04-02T13:35:11.836 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-02T13:35:11.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 -- 172.21.15.122:0/3796241938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c1036e0 msgr2=0x7f2a4c109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 --2- 172.21.15.122:0/3796241938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c109f70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009ca0 tx=0x7f2a4402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 -- 172.21.15.122:0/3796241938 shutdown_connections 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 --2- 172.21.15.122:0/3796241938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c109f70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 --2- 172.21.15.122:0/3796241938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c102d20 0x7f2a4c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 --2- 172.21.15.122:0/3796241938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c101b30 0x7f2a4c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 -- 172.21.15.122:0/3796241938 >> 172.21.15.122:0/3796241938 conn(0x7f2a4c0fd0a0 msgr2=0x7f2a4c0ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 -- 172.21.15.122:0/3796241938 shutdown_connections 2024-04-02T13:35:11.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.950+0000 7f2a52847640 1 -- 172.21.15.122:0/3796241938 wait complete. 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 Processor -- start 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 -- start start 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c101b30 0x7f2a4c0758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c074430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10d7b0 con 0x7f2a4c102d20 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10d630 con 0x7f2a4c1036e0 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a52847640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a4c10d930 con 0x7f2a4c101b30 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37332/0 (socket says 172.21.15.122:37332) 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 -- 172.21.15.122:0/2917802193 learned_addr learned my addr 172.21.15.122:0/2917802193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4bfff640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c101b30 0x7f2a4c0758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a50de4640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c074430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 -- 172.21.15.122:0/2917802193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c101b30 msgr2=0x7f2a4c0758b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c101b30 0x7f2a4c0758b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 -- 172.21.15.122:0/2917802193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c1036e0 msgr2=0x7f2a4c074430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c074430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 -- 172.21.15.122:0/2917802193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4c074cc0 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.951+0000 7f2a4b7fe640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9f0 tx=0x7f2a4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.952+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000cd80 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.952+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4c1a1f80 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.952+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a40005500 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.952+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40010690 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.952+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a4c1a2390 con 0x7f2a4c102d20 2024-04-02T13:35:11.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.953+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4c108df0 con 0x7f2a4c102d20 2024-04-02T13:35:11.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.954+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f2a400046c0 con 0x7f2a4c102d20 2024-04-02T13:35:11.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.954+0000 7f2a497fa640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 0x7f2a340a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:11.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.955+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2a400d0a70 con 0x7f2a4c102d20 2024-04-02T13:35:11.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.955+0000 7f2a4bfff640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 0x7f2a340a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:11.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.955+0000 7f2a4bfff640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 0x7f2a340a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c102b80 tx=0x7f2a3c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:11.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:11.959+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2a40053f20 con 0x7f2a4c102d20 2024-04-02T13:35:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.210+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a4c02d370 con 0x7f2a4c102d20 2024-04-02T13:35:12.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.211+0000 7f2a497fa640 1 -- 172.21.15.122:0/2917802193 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f2a4009d290 con 0x7f2a4c102d20 2024-04-02T13:35:12.213 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:12.213 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:35:12.214 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 msgr2=0x7f2a340a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 0x7f2a340a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c102b80 tx=0x7f2a3c005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:12.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 msgr2=0x7f2a4c073ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2a4000e9f0 tx=0x7f2a4000edf0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 shutdown_connections 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a340a39b0 0x7f2a340a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a4c1036e0 0x7f2a4c074430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a4c102d20 0x7f2a4c073ef0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 --2- 172.21.15.122:0/2917802193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4c101b30 0x7f2a4c0758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 >> 172.21.15.122:0/2917802193 conn(0x7f2a4c0fd0a0 msgr2=0x7f2a4c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 shutdown_connections 2024-04-02T13:35:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.213+0000 7f2a52847640 1 -- 172.21.15.122:0/2917802193 wait complete. 2024-04-02T13:35:12.227 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-02T13:35:12.227 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-02T13:35:12.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 -- 172.21.15.122:0/2138791931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa81025a0 msgr2=0x7fcfa8104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/2138791931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa81025a0 0x7fcfa8104990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c009900 tx=0x7fcf9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 -- 172.21.15.122:0/2138791931 shutdown_connections 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/2138791931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa8107800 0x7fcfa8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/2138791931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 0x7fcfa81072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/2138791931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa81025a0 0x7fcfa8104990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 -- 172.21.15.122:0/2138791931 >> 172.21.15.122:0/2138791931 conn(0x7fcfa80fbf90 msgr2=0x7fcfa80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 -- 172.21.15.122:0/2138791931 shutdown_connections 2024-04-02T13:35:12.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.329+0000 7fcfaef1c640 1 -- 172.21.15.122:0/2138791931 wait complete. 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 Processor -- start 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 -- start start 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa81025a0 0x7fcfa8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 0x7fcfa8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d7a0 con 0x7fcfa8107800 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d620 con 0x7fcfa8104ed0 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaef1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfa810d920 con 0x7fcfa81025a0 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37344/0 (socket says 172.21.15.122:37344) 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 -- 172.21.15.122:0/1712215436 learned_addr learned my addr 172.21.15.122:0/1712215436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfa7fff640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 0x7fcfa8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfaccb8640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa81025a0 0x7fcfa8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 -- 172.21.15.122:0/1712215436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa81025a0 msgr2=0x7fcfa8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa81025a0 0x7fcfa8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 -- 172.21.15.122:0/1712215436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 msgr2=0x7fcfa8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 0x7fcfa8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 -- 172.21.15.122:0/1712215436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfa819fa60 con 0x7fcfa8107800 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.330+0000 7fcfad4b9640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000eaa0 tx=0x7fcfa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.331+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000cdb0 con 0x7fcfa8107800 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.331+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfa819fd50 con 0x7fcfa8107800 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.331+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfa0005020 con 0x7fcfa8107800 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.331+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa00116b0 con 0x7fcfa8107800 2024-04-02T13:35:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.331+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfa81a0240 con 0x7fcfa8107800 2024-04-02T13:35:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.332+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfa80fda80 con 0x7fcfa8107800 2024-04-02T13:35:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.333+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fcfa0005560 con 0x7fcfa8107800 2024-04-02T13:35:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.333+0000 7fcfa5ffb640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 0x7fcf980a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.333+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcfa00d05f0 con 0x7fcfa8107800 2024-04-02T13:35:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.333+0000 7fcfaccb8640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 0x7fcf980a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.334+0000 7fcfaccb8640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 0x7fcf980a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c02c7b0 tx=0x7fcf9c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:12.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.338+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcfa0016030 con 0x7fcfa8107800 2024-04-02T13:35:12.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.458+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7fcfa0098e30 con 0x7fcfa8107800 2024-04-02T13:35:12.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.543+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcfa8061f40 con 0x7fcf980a39b0 2024-04-02T13:35:12.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.553+0000 7fcfa5ffb640 1 -- 172.21.15.122:0/1712215436 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339429 (secure 0 0 0) 0x7fcfa8061f40 con 0x7fcf980a39b0 2024-04-02T13:35:12.561 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:12.563 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 msgr2=0x7fcf980a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 0x7fcf980a5e70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c02c7b0 tx=0x7fcf9c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 msgr2=0x7fcfa819f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000eaa0 tx=0x7fcfa000ef60 comp rx=0 tx=0).stop 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 shutdown_connections 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf980a39b0 0x7fcf980a5e70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfa8107800 0x7fcfa819f3f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfa8104ed0 0x7fcfa8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 --2- 172.21.15.122:0/1712215436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfa81025a0 0x7fcfa8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 >> 172.21.15.122:0/1712215436 conn(0x7fcfa80fbf90 msgr2=0x7fcfa8102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 shutdown_connections 2024-04-02T13:35:12.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.558+0000 7fcfaef1c640 1 -- 172.21.15.122:0/1712215436 wait complete. 2024-04-02T13:35:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: [02/Apr/2024:13:35:11] ENGINE Bus STARTING 2024-04-02T13:35:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: [02/Apr/2024:13:35:11] ENGINE Serving on https://172.21.15.122:7150 2024-04-02T13:35:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: [02/Apr/2024:13:35:11] ENGINE Serving on http://172.21.15.122:8765 2024-04-02T13:35:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: [02/Apr/2024:13:35:11] ENGINE Bus STARTED 2024-04-02T13:35:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2025621281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3363000180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:12 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2917802193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:12.576 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-02T13:35:11.831805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193539,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1470,"num_read_kb":904,"num_write":558,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382296,"kb_used_data":16468,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559528,"statfs":{"total":1151940427776,"available":1151548956672,"internally_reserved":0,"allocated":16863232,"data_stored":9926551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987906"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.071042+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:35:10.071042+0000","last_peered":"2024-04-02T13:35:10.071042+0000","last_clean":"2024-04-02T13:35:10.071042+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:35:10.071042+0000","last_undegraded":"2024-04-02T13:35:10.071042+0000","last_fullsized":"2024-04-02T13:35:10.071042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065645+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:35:10.065645+0000","last_peered":"2024-04-02T13:35:10.065645+0000","last_clean":"2024-04-02T13:35:10.065645+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:35:10.065645+0000","last_undegraded":"2024-04-02T13:35:10.065645+0000","last_fullsized":"2024-04-02T13:35:10.065645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065404+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:35:10.065404+0000","last_peered":"2024-04-02T13:35:10.065404+0000","last_clean":"2024-04-02T13:35:10.065404+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:35:10.065404+0000","last_undegraded":"2024-04-02T13:35:10.065404+0000","last_fullsized":"2024-04-02T13:35:10.065404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062647+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:35:10.062647+0000","last_peered":"2024-04-02T13:35:10.062647+0000","last_clean":"2024-04-02T13:35:10.062647+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:35:10.062647+0000","last_undegraded":"2024-04-02T13:35:10.062647+0000","last_fullsized":"2024-04-02T13:35:10.062647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818970+0000","last_change":"2024-04-02T13:33:55.560120+0000","last_active":"2024-04-02T13:35:09.818970+0000","last_peered":"2024-04-02T13:35:09.818970+0000","last_clean":"2024-04-02T13:35:09.818970+0000","last_became_active":"2024-04-02T13:33:55.559117+0000","last_became_peered":"2024-04-02T13:33:55.559117+0000","last_unstale":"2024-04-02T13:35:09.818970+0000","last_undegraded":"2024-04-02T13:35:09.818970+0000","last_fullsized":"2024-04-02T13:35:09.818970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:03:42.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820256+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:09.820256+0000","last_peered":"2024-04-02T13:35:09.820256+0000","last_clean":"2024-04-02T13:35:09.820256+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:35:09.820256+0000","last_undegraded":"2024-04-02T13:35:09.820256+0000","last_fullsized":"2024-04-02T13:35:09.820256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063497+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:35:10.063497+0000","last_peered":"2024-04-02T13:35:10.063497+0000","last_clean":"2024-04-02T13:35:10.063497+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:35:10.063497+0000","last_undegraded":"2024-04-02T13:35:10.063497+0000","last_fullsized":"2024-04-02T13:35:10.063497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.817945+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:35:09.817945+0000","last_peered":"2024-04-02T13:35:09.817945+0000","last_clean":"2024-04-02T13:35:09.817945+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:35:09.817945+0000","last_undegraded":"2024-04-02T13:35:09.817945+0000","last_fullsized":"2024-04-02T13:35:09.817945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068204+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:35:10.068204+0000","last_peered":"2024-04-02T13:35:10.068204+0000","last_clean":"2024-04-02T13:35:10.068204+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:35:10.068204+0000","last_undegraded":"2024-04-02T13:35:10.068204+0000","last_fullsized":"2024-04-02T13:35:10.068204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820407+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:35:09.820407+0000","last_peered":"2024-04-02T13:35:09.820407+0000","last_clean":"2024-04-02T13:35:09.820407+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:35:09.820407+0000","last_undegraded":"2024-04-02T13:35:09.820407+0000","last_fullsized":"2024-04-02T13:35:09.820407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063961+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:35:10.063961+0000","last_peered":"2024-04-02T13:35:10.063961+0000","last_clean":"2024-04-02T13:35:10.063961+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:35:10.063961+0000","last_undegraded":"2024-04-02T13:35:10.063961+0000","last_fullsized":"2024-04-02T13:35:10.063961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919462+0000","last_change":"2024-04-02T13:33:55.555987+0000","last_active":"2024-04-02T13:35:09.919462+0000","last_peered":"2024-04-02T13:35:09.919462+0000","last_clean":"2024-04-02T13:35:09.919462+0000","last_became_active":"2024-04-02T13:33:55.555623+0000","last_became_peered":"2024-04-02T13:33:55.555623+0000","last_unstale":"2024-04-02T13:35:09.919462+0000","last_undegraded":"2024-04-02T13:35:09.919462+0000","last_fullsized":"2024-04-02T13:35:09.919462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:32.669173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062249+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:35:10.062249+0000","last_peered":"2024-04-02T13:35:10.062249+0000","last_clean":"2024-04-02T13:35:10.062249+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:35:10.062249+0000","last_undegraded":"2024-04-02T13:35:10.062249+0000","last_fullsized":"2024-04-02T13:35:10.062249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064625+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:35:10.064625+0000","last_peered":"2024-04-02T13:35:10.064625+0000","last_clean":"2024-04-02T13:35:10.064625+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:35:10.064625+0000","last_undegraded":"2024-04-02T13:35:10.064625+0000","last_fullsized":"2024-04-02T13:35:10.064625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064990+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:35:10.064990+0000","last_peered":"2024-04-02T13:35:10.064990+0000","last_clean":"2024-04-02T13:35:10.064990+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:35:10.064990+0000","last_undegraded":"2024-04-02T13:35:10.064990+0000","last_fullsized":"2024-04-02T13:35:10.064990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067588+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:35:10.067588+0000","last_peered":"2024-04-02T13:35:10.067588+0000","last_clean":"2024-04-02T13:35:10.067588+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:35:10.067588+0000","last_undegraded":"2024-04-02T13:35:10.067588+0000","last_fullsized":"2024-04-02T13:35:10.067588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070971+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:35:10.070971+0000","last_peered":"2024-04-02T13:35:10.070971+0000","last_clean":"2024-04-02T13:35:10.070971+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:35:10.070971+0000","last_undegraded":"2024-04-02T13:35:10.070971+0000","last_fullsized":"2024-04-02T13:35:10.070971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919799+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:35:09.919799+0000","last_peered":"2024-04-02T13:35:09.919799+0000","last_clean":"2024-04-02T13:35:09.919799+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:35:09.919799+0000","last_undegraded":"2024-04-02T13:35:09.919799+0000","last_fullsized":"2024-04-02T13:35:09.919799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063291+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:35:10.063291+0000","last_peered":"2024-04-02T13:35:10.063291+0000","last_clean":"2024-04-02T13:35:10.063291+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:35:10.063291+0000","last_undegraded":"2024-04-02T13:35:10.063291+0000","last_fullsized":"2024-04-02T13:35:10.063291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067551+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:35:10.067551+0000","last_peered":"2024-04-02T13:35:10.067551+0000","last_clean":"2024-04-02T13:35:10.067551+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:35:10.067551+0000","last_undegraded":"2024-04-02T13:35:10.067551+0000","last_fullsized":"2024-04-02T13:35:10.067551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818258+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:35:09.818258+0000","last_peered":"2024-04-02T13:35:09.818258+0000","last_clean":"2024-04-02T13:35:09.818258+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:35:09.818258+0000","last_undegraded":"2024-04-02T13:35:09.818258+0000","last_fullsized":"2024-04-02T13:35:09.818258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061716+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:35:10.061716+0000","last_peered":"2024-04-02T13:35:10.061716+0000","last_clean":"2024-04-02T13:35:10.061716+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:35:10.061716+0000","last_undegraded":"2024-04-02T13:35:10.061716+0000","last_fullsized":"2024-04-02T13:35:10.061716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818857+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:35:09.818857+0000","last_peered":"2024-04-02T13:35:09.818857+0000","last_clean":"2024-04-02T13:35:09.818857+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:35:09.818857+0000","last_undegraded":"2024-04-02T13:35:09.818857+0000","last_fullsized":"2024-04-02T13:35:09.818857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065791+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:35:10.065791+0000","last_peered":"2024-04-02T13:35:10.065791+0000","last_clean":"2024-04-02T13:35:10.065791+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:35:10.065791+0000","last_undegraded":"2024-04-02T13:35:10.065791+0000","last_fullsized":"2024-04-02T13:35:10.065791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061979+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:35:10.061979+0000","last_peered":"2024-04-02T13:35:10.061979+0000","last_clean":"2024-04-02T13:35:10.061979+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:35:10.061979+0000","last_undegraded":"2024-04-02T13:35:10.061979+0000","last_fullsized":"2024-04-02T13:35:10.061979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819095+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:35:09.819095+0000","last_peered":"2024-04-02T13:35:09.819095+0000","last_clean":"2024-04-02T13:35:09.819095+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:35:09.819095+0000","last_undegraded":"2024-04-02T13:35:09.819095+0000","last_fullsized":"2024-04-02T13:35:09.819095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067884+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:35:10.067884+0000","last_peered":"2024-04-02T13:35:10.067884+0000","last_clean":"2024-04-02T13:35:10.067884+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:35:10.067884+0000","last_undegraded":"2024-04-02T13:35:10.067884+0000","last_fullsized":"2024-04-02T13:35:10.067884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061881+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:35:10.061881+0000","last_peered":"2024-04-02T13:35:10.061881+0000","last_clean":"2024-04-02T13:35:10.061881+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:35:10.061881+0000","last_undegraded":"2024-04-02T13:35:10.061881+0000","last_fullsized":"2024-04-02T13:35:10.061881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819043+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:35:09.819043+0000","last_peered":"2024-04-02T13:35:09.819043+0000","last_clean":"2024-04-02T13:35:09.819043+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:35:09.819043+0000","last_undegraded":"2024-04-02T13:35:09.819043+0000","last_fullsized":"2024-04-02T13:35:09.819043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919412+0000","last_change":"2024-04-02T13:33:55.559231+0000","last_active":"2024-04-02T13:35:09.919412+0000","last_peered":"2024-04-02T13:35:09.919412+0000","last_clean":"2024-04-02T13:35:09.919412+0000","last_became_active":"2024-04-02T13:33:55.558856+0000","last_became_peered":"2024-04-02T13:33:55.558856+0000","last_unstale":"2024-04-02T13:35:09.919412+0000","last_undegraded":"2024-04-02T13:35:09.919412+0000","last_fullsized":"2024-04-02T13:35:09.919412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:12:47.406198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063082+0000","last_change":"2024-04-02T13:34:56.175049+0000","last_active":"2024-04-02T13:35:10.063082+0000","last_peered":"2024-04-02T13:35:10.063082+0000","last_clean":"2024-04-02T13:35:10.063082+0000","last_became_active":"2024-04-02T13:34:56.174477+0000","last_became_peered":"2024-04-02T13:34:56.174477+0000","last_unstale":"2024-04-02T13:35:10.063082+0000","last_undegraded":"2024-04-02T13:35:10.063082+0000","last_fullsized":"2024-04-02T13:35:10.063082+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:30.726406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":162,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315244+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:35:10.315244+0000","last_peered":"2024-04-02T13:35:10.315244+0000","last_clean":"2024-04-02T13:35:10.315244+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:35:10.315244+0000","last_undegraded":"2024-04-02T13:35:10.315244+0000","last_fullsized":"2024-04-02T13:35:10.315244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818973+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:35:09.818973+0000","last_peered":"2024-04-02T13:35:09.818973+0000","last_clean":"2024-04-02T13:35:09.818973+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:35:09.818973+0000","last_undegraded":"2024-04-02T13:35:09.818973+0000","last_fullsized":"2024-04-02T13:35:09.818973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063063+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:35:10.063063+0000","last_peered":"2024-04-02T13:35:10.063063+0000","last_clean":"2024-04-02T13:35:10.063063+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:35:10.063063+0000","last_undegraded":"2024-04-02T13:35:10.063063+0000","last_fullsized":"2024-04-02T13:35:10.063063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820123+0000","last_change":"2024-04-02T13:33:55.565630+0000","last_active":"2024-04-02T13:35:09.820123+0000","last_peered":"2024-04-02T13:35:09.820123+0000","last_clean":"2024-04-02T13:35:09.820123+0000","last_became_active":"2024-04-02T13:33:55.559239+0000","last_became_peered":"2024-04-02T13:33:55.559239+0000","last_unstale":"2024-04-02T13:35:09.820123+0000","last_undegraded":"2024-04-02T13:35:09.820123+0000","last_fullsized":"2024-04-02T13:35:09.820123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:46.161608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928087+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:35:09.928087+0000","last_peered":"2024-04-02T13:35:09.928087+0000","last_clean":"2024-04-02T13:35:09.928087+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:35:09.928087+0000","last_undegraded":"2024-04-02T13:35:09.928087+0000","last_fullsized":"2024-04-02T13:35:09.928087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070614+0000","last_change":"2024-04-02T13:33:55.558751+0000","last_active":"2024-04-02T13:35:10.070614+0000","last_peered":"2024-04-02T13:35:10.070614+0000","last_clean":"2024-04-02T13:35:10.070614+0000","last_became_active":"2024-04-02T13:33:55.558278+0000","last_became_peered":"2024-04-02T13:33:55.558278+0000","last_unstale":"2024-04-02T13:35:10.070614+0000","last_undegraded":"2024-04-02T13:35:10.070614+0000","last_fullsized":"2024-04-02T13:35:10.070614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:15.164871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062024+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:35:10.062024+0000","last_peered":"2024-04-02T13:35:10.062024+0000","last_clean":"2024-04-02T13:35:10.062024+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:35:10.062024+0000","last_undegraded":"2024-04-02T13:35:10.062024+0000","last_fullsized":"2024-04-02T13:35:10.062024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818108+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:35:09.818108+0000","last_peered":"2024-04-02T13:35:09.818108+0000","last_clean":"2024-04-02T13:35:09.818108+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:35:09.818108+0000","last_undegraded":"2024-04-02T13:35:09.818108+0000","last_fullsized":"2024-04-02T13:35:09.818108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065607+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:35:10.065607+0000","last_peered":"2024-04-02T13:35:10.065607+0000","last_clean":"2024-04-02T13:35:10.065607+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:35:10.065607+0000","last_undegraded":"2024-04-02T13:35:10.065607+0000","last_fullsized":"2024-04-02T13:35:10.065607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927765+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:35:09.927765+0000","last_peered":"2024-04-02T13:35:09.927765+0000","last_clean":"2024-04-02T13:35:09.927765+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:35:09.927765+0000","last_undegraded":"2024-04-02T13:35:09.927765+0000","last_fullsized":"2024-04-02T13:35:09.927765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919045+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:35:09.919045+0000","last_peered":"2024-04-02T13:35:09.919045+0000","last_clean":"2024-04-02T13:35:09.919045+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:35:09.919045+0000","last_undegraded":"2024-04-02T13:35:09.919045+0000","last_fullsized":"2024-04-02T13:35:09.919045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818007+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:35:09.818007+0000","last_peered":"2024-04-02T13:35:09.818007+0000","last_clean":"2024-04-02T13:35:09.818007+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:35:09.818007+0000","last_undegraded":"2024-04-02T13:35:09.818007+0000","last_fullsized":"2024-04-02T13:35:09.818007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":278,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.314531+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:35:10.314531+0000","last_peered":"2024-04-02T13:35:10.314531+0000","last_clean":"2024-04-02T13:35:10.314531+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:35:10.314531+0000","last_undegraded":"2024-04-02T13:35:10.314531+0000","last_fullsized":"2024-04-02T13:35:10.314531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068022+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:10.068022+0000","last_peered":"2024-04-02T13:35:10.068022+0000","last_clean":"2024-04-02T13:35:10.068022+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:35:10.068022+0000","last_undegraded":"2024-04-02T13:35:10.068022+0000","last_fullsized":"2024-04-02T13:35:10.068022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.918900+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:35:09.918900+0000","last_peered":"2024-04-02T13:35:09.918900+0000","last_clean":"2024-04-02T13:35:09.918900+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:35:09.918900+0000","last_undegraded":"2024-04-02T13:35:09.918900+0000","last_fullsized":"2024-04-02T13:35:09.918900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063718+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:35:10.063718+0000","last_peered":"2024-04-02T13:35:10.063718+0000","last_clean":"2024-04-02T13:35:10.063718+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:35:10.063718+0000","last_undegraded":"2024-04-02T13:35:10.063718+0000","last_fullsized":"2024-04-02T13:35:10.063718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065934+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:35:10.065934+0000","last_peered":"2024-04-02T13:35:10.065934+0000","last_clean":"2024-04-02T13:35:10.065934+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:35:10.065934+0000","last_undegraded":"2024-04-02T13:35:10.065934+0000","last_fullsized":"2024-04-02T13:35:10.065934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062853+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:35:10.062853+0000","last_peered":"2024-04-02T13:35:10.062853+0000","last_clean":"2024-04-02T13:35:10.062853+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:35:10.062853+0000","last_undegraded":"2024-04-02T13:35:10.062853+0000","last_fullsized":"2024-04-02T13:35:10.062853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928052+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:35:09.928052+0000","last_peered":"2024-04-02T13:35:09.928052+0000","last_clean":"2024-04-02T13:35:09.928052+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:35:09.928052+0000","last_undegraded":"2024-04-02T13:35:09.928052+0000","last_fullsized":"2024-04-02T13:35:09.928052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068046+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:35:10.068046+0000","last_peered":"2024-04-02T13:35:10.068046+0000","last_clean":"2024-04-02T13:35:10.068046+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:35:10.068046+0000","last_undegraded":"2024-04-02T13:35:10.068046+0000","last_fullsized":"2024-04-02T13:35:10.068046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819835+0000","last_change":"2024-04-02T13:34:56.181716+0000","last_active":"2024-04-02T13:35:09.819835+0000","last_peered":"2024-04-02T13:35:09.819835+0000","last_clean":"2024-04-02T13:35:09.819835+0000","last_became_active":"2024-04-02T13:34:56.181233+0000","last_became_peered":"2024-04-02T13:34:56.181233+0000","last_unstale":"2024-04-02T13:35:09.819835+0000","last_undegraded":"2024-04-02T13:35:09.819835+0000","last_fullsized":"2024-04-02T13:35:09.819835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:24:55.347041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315476+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:35:10.315476+0000","last_peered":"2024-04-02T13:35:10.315476+0000","last_clean":"2024-04-02T13:35:10.315476+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:35:10.315476+0000","last_undegraded":"2024-04-02T13:35:10.315476+0000","last_fullsized":"2024-04-02T13:35:10.315476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":193,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819358+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:35:09.819358+0000","last_peered":"2024-04-02T13:35:09.819358+0000","last_clean":"2024-04-02T13:35:09.819358+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:35:09.819358+0000","last_undegraded":"2024-04-02T13:35:09.819358+0000","last_fullsized":"2024-04-02T13:35:09.819358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928355+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:35:09.928355+0000","last_peered":"2024-04-02T13:35:09.928355+0000","last_clean":"2024-04-02T13:35:09.928355+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:35:09.928355+0000","last_undegraded":"2024-04-02T13:35:09.928355+0000","last_fullsized":"2024-04-02T13:35:09.928355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062970+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:35:10.062970+0000","last_peered":"2024-04-02T13:35:10.062970+0000","last_clean":"2024-04-02T13:35:10.062970+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:35:10.062970+0000","last_undegraded":"2024-04-02T13:35:10.062970+0000","last_fullsized":"2024-04-02T13:35:10.062970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062183+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:35:10.062183+0000","last_peered":"2024-04-02T13:35:10.062183+0000","last_clean":"2024-04-02T13:35:10.062183+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:35:10.062183+0000","last_undegraded":"2024-04-02T13:35:10.062183+0000","last_fullsized":"2024-04-02T13:35:10.062183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070601+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:35:10.070601+0000","last_peered":"2024-04-02T13:35:10.070601+0000","last_clean":"2024-04-02T13:35:10.070601+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:35:10.070601+0000","last_undegraded":"2024-04-02T13:35:10.070601+0000","last_fullsized":"2024-04-02T13:35:10.070601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928018+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:35:09.928018+0000","last_peered":"2024-04-02T13:35:09.928018+0000","last_clean":"2024-04-02T13:35:09.928018+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:35:09.928018+0000","last_undegraded":"2024-04-02T13:35:09.928018+0000","last_fullsized":"2024-04-02T13:35:09.928018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063434+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:35:10.063434+0000","last_peered":"2024-04-02T13:35:10.063434+0000","last_clean":"2024-04-02T13:35:10.063434+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:35:10.063434+0000","last_undegraded":"2024-04-02T13:35:10.063434+0000","last_fullsized":"2024-04-02T13:35:10.063434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919844+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:35:09.919844+0000","last_peered":"2024-04-02T13:35:09.919844+0000","last_clean":"2024-04-02T13:35:09.919844+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:35:09.919844+0000","last_undegraded":"2024-04-02T13:35:09.919844+0000","last_fullsized":"2024-04-02T13:35:09.919844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":163,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315744+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:35:10.315744+0000","last_peered":"2024-04-02T13:35:10.315744+0000","last_clean":"2024-04-02T13:35:10.315744+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:35:10.315744+0000","last_undegraded":"2024-04-02T13:35:10.315744+0000","last_fullsized":"2024-04-02T13:35:10.315744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063690+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:35:10.063690+0000","last_peered":"2024-04-02T13:35:10.063690+0000","last_clean":"2024-04-02T13:35:10.063690+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:35:10.063690+0000","last_undegraded":"2024-04-02T13:35:10.063690+0000","last_fullsized":"2024-04-02T13:35:10.063690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070705+0000","last_change":"2024-04-02T13:33:55.574777+0000","last_active":"2024-04-02T13:35:10.070705+0000","last_peered":"2024-04-02T13:35:10.070705+0000","last_clean":"2024-04-02T13:35:10.070705+0000","last_became_active":"2024-04-02T13:33:55.561040+0000","last_became_peered":"2024-04-02T13:33:55.561040+0000","last_unstale":"2024-04-02T13:35:10.070705+0000","last_undegraded":"2024-04-02T13:35:10.070705+0000","last_fullsized":"2024-04-02T13:35:10.070705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:23.184540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927913+0000","last_change":"2024-04-02T13:33:55.560045+0000","last_active":"2024-04-02T13:35:09.927913+0000","last_peered":"2024-04-02T13:35:09.927913+0000","last_clean":"2024-04-02T13:35:09.927913+0000","last_became_active":"2024-04-02T13:33:55.559659+0000","last_became_peered":"2024-04-02T13:33:55.559659+0000","last_unstale":"2024-04-02T13:35:09.927913+0000","last_undegraded":"2024-04-02T13:35:09.927913+0000","last_fullsized":"2024-04-02T13:35:09.927913+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:10:42.028143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927883+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:35:09.927883+0000","last_peered":"2024-04-02T13:35:09.927883+0000","last_clean":"2024-04-02T13:35:09.927883+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:35:09.927883+0000","last_undegraded":"2024-04-02T13:35:09.927883+0000","last_fullsized":"2024-04-02T13:35:09.927883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819290+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:35:09.819290+0000","last_peered":"2024-04-02T13:35:09.819290+0000","last_clean":"2024-04-02T13:35:09.819290+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:35:09.819290+0000","last_undegraded":"2024-04-02T13:35:09.819290+0000","last_fullsized":"2024-04-02T13:35:09.819290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070776+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:35:10.070776+0000","last_peered":"2024-04-02T13:35:10.070776+0000","last_clean":"2024-04-02T13:35:10.070776+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:35:10.070776+0000","last_undegraded":"2024-04-02T13:35:10.070776+0000","last_fullsized":"2024-04-02T13:35:10.070776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818918+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:35:09.818918+0000","last_peered":"2024-04-02T13:35:09.818918+0000","last_clean":"2024-04-02T13:35:09.818918+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:35:09.818918+0000","last_undegraded":"2024-04-02T13:35:09.818918+0000","last_fullsized":"2024-04-02T13:35:09.818918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919605+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:35:09.919605+0000","last_peered":"2024-04-02T13:35:09.919605+0000","last_clean":"2024-04-02T13:35:09.919605+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:35:09.919605+0000","last_undegraded":"2024-04-02T13:35:09.919605+0000","last_fullsized":"2024-04-02T13:35:09.919605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820178+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:35:09.820178+0000","last_peered":"2024-04-02T13:35:09.820178+0000","last_clean":"2024-04-02T13:35:09.820178+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:35:09.820178+0000","last_undegraded":"2024-04-02T13:35:09.820178+0000","last_fullsized":"2024-04-02T13:35:09.820178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062428+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:35:10.062428+0000","last_peered":"2024-04-02T13:35:10.062428+0000","last_clean":"2024-04-02T13:35:10.062428+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:35:10.062428+0000","last_undegraded":"2024-04-02T13:35:10.062428+0000","last_fullsized":"2024-04-02T13:35:10.062428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.323339+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:35:10.323339+0000","last_peered":"2024-04-02T13:35:10.323339+0000","last_clean":"2024-04-02T13:35:10.323339+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:35:10.323339+0000","last_undegraded":"2024-04-02T13:35:10.323339+0000","last_fullsized":"2024-04-02T13:35:10.323339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820157+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:35:09.820157+0000","last_peered":"2024-04-02T13:35:09.820157+0000","last_clean":"2024-04-02T13:35:09.820157+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:35:09.820157+0000","last_undegraded":"2024-04-02T13:35:09.820157+0000","last_fullsized":"2024-04-02T13:35:09.820157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062937+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:35:10.062937+0000","last_peered":"2024-04-02T13:35:10.062937+0000","last_clean":"2024-04-02T13:35:10.062937+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:35:10.062937+0000","last_undegraded":"2024-04-02T13:35:10.062937+0000","last_fullsized":"2024-04-02T13:35:10.062937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928098+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:35:09.928098+0000","last_peered":"2024-04-02T13:35:09.928098+0000","last_clean":"2024-04-02T13:35:09.928098+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:35:09.928098+0000","last_undegraded":"2024-04-02T13:35:09.928098+0000","last_fullsized":"2024-04-02T13:35:09.928098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.066596+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:35:10.066596+0000","last_peered":"2024-04-02T13:35:10.066596+0000","last_clean":"2024-04-02T13:35:10.066596+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:35:10.066596+0000","last_undegraded":"2024-04-02T13:35:10.066596+0000","last_fullsized":"2024-04-02T13:35:10.066596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063688+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:35:10.063688+0000","last_peered":"2024-04-02T13:35:10.063688+0000","last_clean":"2024-04-02T13:35:10.063688+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:35:10.063688+0000","last_undegraded":"2024-04-02T13:35:10.063688+0000","last_fullsized":"2024-04-02T13:35:10.063688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818230+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:35:09.818230+0000","last_peered":"2024-04-02T13:35:09.818230+0000","last_clean":"2024-04-02T13:35:09.818230+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:35:09.818230+0000","last_undegraded":"2024-04-02T13:35:09.818230+0000","last_fullsized":"2024-04-02T13:35:09.818230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062201+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:35:10.062201+0000","last_peered":"2024-04-02T13:35:10.062201+0000","last_clean":"2024-04-02T13:35:10.062201+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:35:10.062201+0000","last_undegraded":"2024-04-02T13:35:10.062201+0000","last_fullsized":"2024-04-02T13:35:10.062201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063125+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:35:10.063125+0000","last_peered":"2024-04-02T13:35:10.063125+0000","last_clean":"2024-04-02T13:35:10.063125+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:35:10.063125+0000","last_undegraded":"2024-04-02T13:35:10.063125+0000","last_fullsized":"2024-04-02T13:35:10.063125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928211+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:35:09.928211+0000","last_peered":"2024-04-02T13:35:09.928211+0000","last_clean":"2024-04-02T13:35:09.928211+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:35:09.928211+0000","last_undegraded":"2024-04-02T13:35:09.928211+0000","last_fullsized":"2024-04-02T13:35:09.928211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061853+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:35:10.061853+0000","last_peered":"2024-04-02T13:35:10.061853+0000","last_clean":"2024-04-02T13:35:10.061853+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:35:10.061853+0000","last_undegraded":"2024-04-02T13:35:10.061853+0000","last_fullsized":"2024-04-02T13:35:10.061853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063479+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:35:10.063479+0000","last_peered":"2024-04-02T13:35:10.063479+0000","last_clean":"2024-04-02T13:35:10.063479+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:35:10.063479+0000","last_undegraded":"2024-04-02T13:35:10.063479+0000","last_fullsized":"2024-04-02T13:35:10.063479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070541+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:35:10.070541+0000","last_peered":"2024-04-02T13:35:10.070541+0000","last_clean":"2024-04-02T13:35:10.070541+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:35:10.070541+0000","last_undegraded":"2024-04-02T13:35:10.070541+0000","last_fullsized":"2024-04-02T13:35:10.070541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819228+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:35:09.819228+0000","last_peered":"2024-04-02T13:35:09.819228+0000","last_clean":"2024-04-02T13:35:09.819228+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:35:09.819228+0000","last_undegraded":"2024-04-02T13:35:09.819228+0000","last_fullsized":"2024-04-02T13:35:09.819228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062665+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:35:10.062665+0000","last_peered":"2024-04-02T13:35:10.062665+0000","last_clean":"2024-04-02T13:35:10.062665+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:35:10.062665+0000","last_undegraded":"2024-04-02T13:35:10.062665+0000","last_fullsized":"2024-04-02T13:35:10.062665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818025+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:35:09.818025+0000","last_peered":"2024-04-02T13:35:09.818025+0000","last_clean":"2024-04-02T13:35:09.818025+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:35:09.818025+0000","last_undegraded":"2024-04-02T13:35:09.818025+0000","last_fullsized":"2024-04-02T13:35:09.818025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'49","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.547985+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:35:10.547985+0000","last_peered":"2024-04-02T13:35:10.547985+0000","last_clean":"2024-04-02T13:35:10.547985+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:35:10.547985+0000","last_undegraded":"2024-04-02T13:35:10.547985+0000","last_fullsized":"2024-04-02T13:35:10.547985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818962+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:35:09.818962+0000","last_peered":"2024-04-02T13:35:09.818962+0000","last_clean":"2024-04-02T13:35:09.818962+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:35:09.818962+0000","last_undegraded":"2024-04-02T13:35:09.818962+0000","last_fullsized":"2024-04-02T13:35:09.818962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062662+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:35:10.062662+0000","last_peered":"2024-04-02T13:35:10.062662+0000","last_clean":"2024-04-02T13:35:10.062662+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:35:10.062662+0000","last_undegraded":"2024-04-02T13:35:10.062662+0000","last_fullsized":"2024-04-02T13:35:10.062662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062851+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:35:10.062851+0000","last_peered":"2024-04-02T13:35:10.062851+0000","last_clean":"2024-04-02T13:35:10.062851+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:35:10.062851+0000","last_undegraded":"2024-04-02T13:35:10.062851+0000","last_fullsized":"2024-04-02T13:35:10.062851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065708+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:35:10.065708+0000","last_peered":"2024-04-02T13:35:10.065708+0000","last_clean":"2024-04-02T13:35:10.065708+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:35:10.065708+0000","last_undegraded":"2024-04-02T13:35:10.065708+0000","last_fullsized":"2024-04-02T13:35:10.065708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061934+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:35:10.061934+0000","last_peered":"2024-04-02T13:35:10.061934+0000","last_clean":"2024-04-02T13:35:10.061934+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:35:10.061934+0000","last_undegraded":"2024-04-02T13:35:10.061934+0000","last_fullsized":"2024-04-02T13:35:10.061934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067569+0000","last_change":"2024-04-02T13:33:55.564335+0000","last_active":"2024-04-02T13:35:10.067569+0000","last_peered":"2024-04-02T13:35:10.067569+0000","last_clean":"2024-04-02T13:35:10.067569+0000","last_became_active":"2024-04-02T13:33:55.560370+0000","last_became_peered":"2024-04-02T13:33:55.560370+0000","last_unstale":"2024-04-02T13:35:10.067569+0000","last_undegraded":"2024-04-02T13:35:10.067569+0000","last_fullsized":"2024-04-02T13:35:10.067569+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:14.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818822+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:35:09.818822+0000","last_peered":"2024-04-02T13:35:09.818822+0000","last_clean":"2024-04-02T13:35:09.818822+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:35:09.818822+0000","last_undegraded":"2024-04-02T13:35:09.818822+0000","last_fullsized":"2024-04-02T13:35:09.818822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919950+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:35:09.919950+0000","last_peered":"2024-04-02T13:35:09.919950+0000","last_clean":"2024-04-02T13:35:09.919950+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:35:09.919950+0000","last_undegraded":"2024-04-02T13:35:09.919950+0000","last_fullsized":"2024-04-02T13:35:09.919950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818121+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:35:09.818121+0000","last_peered":"2024-04-02T13:35:09.818121+0000","last_clean":"2024-04-02T13:35:09.818121+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:35:09.818121+0000","last_undegraded":"2024-04-02T13:35:09.818121+0000","last_fullsized":"2024-04-02T13:35:09.818121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070490+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:35:10.070490+0000","last_peered":"2024-04-02T13:35:10.070490+0000","last_clean":"2024-04-02T13:35:10.070490+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:35:10.070490+0000","last_undegraded":"2024-04-02T13:35:10.070490+0000","last_fullsized":"2024-04-02T13:35:10.070490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062642+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:35:10.062642+0000","last_peered":"2024-04-02T13:35:10.062642+0000","last_clean":"2024-04-02T13:35:10.062642+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:35:10.062642+0000","last_undegraded":"2024-04-02T13:35:10.062642+0000","last_fullsized":"2024-04-02T13:35:10.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062817+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:35:10.062817+0000","last_peered":"2024-04-02T13:35:10.062817+0000","last_clean":"2024-04-02T13:35:10.062817+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:35:10.062817+0000","last_undegraded":"2024-04-02T13:35:10.062817+0000","last_fullsized":"2024-04-02T13:35:10.062817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818375+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:35:09.818375+0000","last_peered":"2024-04-02T13:35:09.818375+0000","last_clean":"2024-04-02T13:35:09.818375+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:35:09.818375+0000","last_undegraded":"2024-04-02T13:35:09.818375+0000","last_fullsized":"2024-04-02T13:35:09.818375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062023+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:35:10.062023+0000","last_peered":"2024-04-02T13:35:10.062023+0000","last_clean":"2024-04-02T13:35:10.062023+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:35:10.062023+0000","last_undegraded":"2024-04-02T13:35:10.062023+0000","last_fullsized":"2024-04-02T13:35:10.062023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818964+0000","last_change":"2024-04-02T13:33:55.565608+0000","last_active":"2024-04-02T13:35:09.818964+0000","last_peered":"2024-04-02T13:35:09.818964+0000","last_clean":"2024-04-02T13:35:09.818964+0000","last_became_active":"2024-04-02T13:33:55.557475+0000","last_became_peered":"2024-04-02T13:33:55.557475+0000","last_unstale":"2024-04-02T13:35:09.818964+0000","last_undegraded":"2024-04-02T13:35:09.818964+0000","last_fullsized":"2024-04-02T13:35:09.818964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:17:21.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063486+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:35:10.063486+0000","last_peered":"2024-04-02T13:35:10.063486+0000","last_clean":"2024-04-02T13:35:10.063486+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:35:10.063486+0000","last_undegraded":"2024-04-02T13:35:10.063486+0000","last_fullsized":"2024-04-02T13:35:10.063486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067660+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:35:10.067660+0000","last_peered":"2024-04-02T13:35:10.067660+0000","last_clean":"2024-04-02T13:35:10.067660+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:35:10.067660+0000","last_undegraded":"2024-04-02T13:35:10.067660+0000","last_fullsized":"2024-04-02T13:35:10.067660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'8","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315848+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:35:10.315848+0000","last_peered":"2024-04-02T13:35:10.315848+0000","last_clean":"2024-04-02T13:35:10.315848+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:35:10.315848+0000","last_undegraded":"2024-04-02T13:35:10.315848+0000","last_fullsized":"2024-04-02T13:35:10.315848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064153+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:35:10.064153+0000","last_peered":"2024-04-02T13:35:10.064153+0000","last_clean":"2024-04-02T13:35:10.064153+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:35:10.064153+0000","last_undegraded":"2024-04-02T13:35:10.064153+0000","last_fullsized":"2024-04-02T13:35:10.064153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064951+0000","last_change":"2024-04-02T13:34:56.181555+0000","last_active":"2024-04-02T13:35:10.064951+0000","last_peered":"2024-04-02T13:35:10.064951+0000","last_clean":"2024-04-02T13:35:10.064951+0000","last_became_active":"2024-04-02T13:34:56.181156+0000","last_became_peered":"2024-04-02T13:34:56.181156+0000","last_unstale":"2024-04-02T13:35:10.064951+0000","last_undegraded":"2024-04-02T13:35:10.064951+0000","last_fullsized":"2024-04-02T13:35:10.064951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:23.387417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":82,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.027635+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:35:10.027635+0000","last_peered":"2024-04-02T13:35:10.027635+0000","last_clean":"2024-04-02T13:35:10.027635+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:35:10.027635+0000","last_undegraded":"2024-04-02T13:35:10.027635+0000","last_fullsized":"2024-04-02T13:35:10.027635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063750+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:35:10.063750+0000","last_peered":"2024-04-02T13:35:10.063750+0000","last_clean":"2024-04-02T13:35:10.063750+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:35:10.063750+0000","last_undegraded":"2024-04-02T13:35:10.063750+0000","last_fullsized":"2024-04-02T13:35:10.063750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927710+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:35:09.927710+0000","last_peered":"2024-04-02T13:35:09.927710+0000","last_clean":"2024-04-02T13:35:09.927710+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:35:09.927710+0000","last_undegraded":"2024-04-02T13:35:09.927710+0000","last_fullsized":"2024-04-02T13:35:09.927710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062701+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:35:10.062701+0000","last_peered":"2024-04-02T13:35:10.062701+0000","last_clean":"2024-04-02T13:35:10.062701+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:35:10.062701+0000","last_undegraded":"2024-04-02T13:35:10.062701+0000","last_fullsized":"2024-04-02T13:35:10.062701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063549+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:35:10.063549+0000","last_peered":"2024-04-02T13:35:10.063549+0000","last_clean":"2024-04-02T13:35:10.063549+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:35:10.063549+0000","last_undegraded":"2024-04-02T13:35:10.063549+0000","last_fullsized":"2024-04-02T13:35:10.063549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070648+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:35:10.070648+0000","last_peered":"2024-04-02T13:35:10.070648+0000","last_clean":"2024-04-02T13:35:10.070648+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:35:10.070648+0000","last_undegraded":"2024-04-02T13:35:10.070648+0000","last_fullsized":"2024-04-02T13:35:10.070648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062874+0000","last_change":"2024-04-02T13:33:55.562541+0000","last_active":"2024-04-02T13:35:10.062874+0000","last_peered":"2024-04-02T13:35:10.062874+0000","last_clean":"2024-04-02T13:35:10.062874+0000","last_became_active":"2024-04-02T13:33:55.562167+0000","last_became_peered":"2024-04-02T13:33:55.562167+0000","last_unstale":"2024-04-02T13:35:10.062874+0000","last_undegraded":"2024-04-02T13:35:10.062874+0000","last_fullsized":"2024-04-02T13:35:10.062874+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:29.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062695+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:35:10.062695+0000","last_peered":"2024-04-02T13:35:10.062695+0000","last_clean":"2024-04-02T13:35:10.062695+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:35:10.062695+0000","last_undegraded":"2024-04-02T13:35:10.062695+0000","last_fullsized":"2024-04-02T13:35:10.062695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065156+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:35:10.065156+0000","last_peered":"2024-04-02T13:35:10.065156+0000","last_clean":"2024-04-02T13:35:10.065156+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:35:10.065156+0000","last_undegraded":"2024-04-02T13:35:10.065156+0000","last_fullsized":"2024-04-02T13:35:10.065156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062536+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:35:10.062536+0000","last_peered":"2024-04-02T13:35:10.062536+0000","last_clean":"2024-04-02T13:35:10.062536+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:35:10.062536+0000","last_undegraded":"2024-04-02T13:35:10.062536+0000","last_fullsized":"2024-04-02T13:35:10.062536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070908+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:35:10.070908+0000","last_peered":"2024-04-02T13:35:10.070908+0000","last_clean":"2024-04-02T13:35:10.070908+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:35:10.070908+0000","last_undegraded":"2024-04-02T13:35:10.070908+0000","last_fullsized":"2024-04-02T13:35:10.070908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063259+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:35:10.063259+0000","last_peered":"2024-04-02T13:35:10.063259+0000","last_clean":"2024-04-02T13:35:10.063259+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:35:10.063259+0000","last_undegraded":"2024-04-02T13:35:10.063259+0000","last_fullsized":"2024-04-02T13:35:10.063259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819077+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:35:09.819077+0000","last_peered":"2024-04-02T13:35:09.819077+0000","last_clean":"2024-04-02T13:35:09.819077+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:35:09.819077+0000","last_undegraded":"2024-04-02T13:35:09.819077+0000","last_fullsized":"2024-04-02T13:35:09.819077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927773+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:35:09.927773+0000","last_peered":"2024-04-02T13:35:09.927773+0000","last_clean":"2024-04-02T13:35:09.927773+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:35:09.927773+0000","last_undegraded":"2024-04-02T13:35:09.927773+0000","last_fullsized":"2024-04-02T13:35:09.927773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818159+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:35:09.818159+0000","last_peered":"2024-04-02T13:35:09.818159+0000","last_clean":"2024-04-02T13:35:09.818159+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:35:09.818159+0000","last_undegraded":"2024-04-02T13:35:09.818159+0000","last_fullsized":"2024-04-02T13:35:09.818159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067987+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:35:10.067987+0000","last_peered":"2024-04-02T13:35:10.067987+0000","last_clean":"2024-04-02T13:35:10.067987+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:35:10.067987+0000","last_undegraded":"2024-04-02T13:35:10.067987+0000","last_fullsized":"2024-04-02T13:35:10.067987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062204+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:35:10.062204+0000","last_peered":"2024-04-02T13:35:10.062204+0000","last_clean":"2024-04-02T13:35:10.062204+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:35:10.062204+0000","last_undegraded":"2024-04-02T13:35:10.062204+0000","last_fullsized":"2024-04-02T13:35:10.062204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070777+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:35:10.070777+0000","last_peered":"2024-04-02T13:35:10.070777+0000","last_clean":"2024-04-02T13:35:10.070777+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:35:10.070777+0000","last_undegraded":"2024-04-02T13:35:10.070777+0000","last_fullsized":"2024-04-02T13:35:10.070777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602937,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1334,"num_read_kb":785,"num_write":375,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1808811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":525216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1322543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1515520,"data_stored":931933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1499136,"data_stored":928103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1132669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":530516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":600591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":584678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1054649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":534590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37148,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708004,"statfs":{"total":95995035648,"available":95956996096,"internally_reserved":0,"allocated":1855488,"data_stored":1259069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:35:12.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 -- 172.21.15.122:0/2686100119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941003f0 msgr2=0x7fe794100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 --2- 172.21.15.122:0/2686100119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941003f0 0x7fe794100810 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe788009900 tx=0x7fe78802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 -- 172.21.15.122:0/2686100119 shutdown_connections 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 --2- 172.21.15.122:0/2686100119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7941016f0 0x7fe794101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 --2- 172.21.15.122:0/2686100119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe794100d50 0x7fe7941011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 --2- 172.21.15.122:0/2686100119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941003f0 0x7fe794100810 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 -- 172.21.15.122:0/2686100119 >> 172.21.15.122:0/2686100119 conn(0x7fe7940fbf90 msgr2=0x7fe7940fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 -- 172.21.15.122:0/2686100119 shutdown_connections 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.719+0000 7fe79abe2640 1 -- 172.21.15.122:0/2686100119 wait complete. 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 Processor -- start 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 -- start start 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7941003f0 0x7fe79419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 0x7fe79419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410f960 con 0x7fe7941016f0 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410f7e0 con 0x7fe7941003f0 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79abe2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe79410fae0 con 0x7fe794100d50 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37360/0 (socket says 172.21.15.122:37360) 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe793fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 0x7fe79419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 -- 172.21.15.122:0/1188532446 learned_addr learned my addr 172.21.15.122:0/1188532446 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79897e640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7941003f0 0x7fe79419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 -- 172.21.15.122:0/1188532446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 msgr2=0x7fe79419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 0x7fe79419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 -- 172.21.15.122:0/1188532446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7941003f0 msgr2=0x7fe79419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7941003f0 0x7fe79419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.720+0000 7fe79917f640 1 -- 172.21.15.122:0/1188532446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7941a1c50 con 0x7fe7941016f0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe793fff640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 0x7fe79419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe79917f640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00eb10 tx=0x7fe78c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c004b20 con 0x7fe7941016f0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78c004cc0 con 0x7fe7941016f0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7941a1f40 con 0x7fe7941016f0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c010760 con 0x7fe7941016f0 2024-04-02T13:35:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.721+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7941a23b0 con 0x7fe7941016f0 2024-04-02T13:35:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.723+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79410b340 con 0x7fe7941016f0 2024-04-02T13:35:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.724+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7fe78c005020 con 0x7fe7941016f0 2024-04-02T13:35:12.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.724+0000 7fe791ffb640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.724+0000 7fe79897e640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.724+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe78c0d1100 con 0x7fe7941016f0 2024-04-02T13:35:12.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.725+0000 7fe79897e640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 0x7fe7840a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c7b0 tx=0x7fe788005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:12.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.729+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe78c09d410 con 0x7fe7941016f0 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: [02/Apr/2024:13:35:11] ENGINE Bus STARTING 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: [02/Apr/2024:13:35:11] ENGINE Serving on https://172.21.15.122:7150 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: [02/Apr/2024:13:35:11] ENGINE Serving on http://172.21.15.122:8765 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: [02/Apr/2024:13:35:11] ENGINE Bus STARTED 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2025621281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3363000180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:12 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2917802193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: [02/Apr/2024:13:35:11] ENGINE Bus STARTING 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: [02/Apr/2024:13:35:11] ENGINE Serving on https://172.21.15.122:7150 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: [02/Apr/2024:13:35:11] ENGINE Serving on http://172.21.15.122:8765 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: [02/Apr/2024:13:35:11] ENGINE Bus STARTED 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2025621281' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3363000180' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:12 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2917802193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:12.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.912+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe794061f40 con 0x7fe7840a3d20 2024-04-02T13:35:12.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.921+0000 7fe791ffb640 1 -- 172.21.15.122:0/1188532446 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339429 (secure 0 0 0) 0x7fe794061f40 con 0x7fe7840a3d20 2024-04-02T13:35:12.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:12.930 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:35:12.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 msgr2=0x7fe7840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 0x7fe7840a61e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe78802c7b0 tx=0x7fe788005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 msgr2=0x7fe7941a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00eb10 tx=0x7fe78c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 shutdown_connections 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe7840a3d20 0x7fe7840a61e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7941016f0 0x7fe7941a15e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe794100d50 0x7fe79419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 --2- 172.21.15.122:0/1188532446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe7941003f0 0x7fe79419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 >> 172.21.15.122:0/1188532446 conn(0x7fe7940fbf90 msgr2=0x7fe7940fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 shutdown_connections 2024-04-02T13:35:12.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:12.926+0000 7fe79abe2640 1 -- 172.21.15.122:0/1188532446 wait complete. 2024-04-02T13:35:12.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-02T13:35:11.831805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193539,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1470,"num_read_kb":904,"num_write":558,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382296,"kb_used_data":16468,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559528,"statfs":{"total":1151940427776,"available":1151548956672,"internally_reserved":0,"allocated":16863232,"data_stored":9926551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987906"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.071042+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:35:10.071042+0000","last_peered":"2024-04-02T13:35:10.071042+0000","last_clean":"2024-04-02T13:35:10.071042+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:35:10.071042+0000","last_undegraded":"2024-04-02T13:35:10.071042+0000","last_fullsized":"2024-04-02T13:35:10.071042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065645+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:35:10.065645+0000","last_peered":"2024-04-02T13:35:10.065645+0000","last_clean":"2024-04-02T13:35:10.065645+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:35:10.065645+0000","last_undegraded":"2024-04-02T13:35:10.065645+0000","last_fullsized":"2024-04-02T13:35:10.065645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065404+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:35:10.065404+0000","last_peered":"2024-04-02T13:35:10.065404+0000","last_clean":"2024-04-02T13:35:10.065404+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:35:10.065404+0000","last_undegraded":"2024-04-02T13:35:10.065404+0000","last_fullsized":"2024-04-02T13:35:10.065404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062647+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:35:10.062647+0000","last_peered":"2024-04-02T13:35:10.062647+0000","last_clean":"2024-04-02T13:35:10.062647+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:35:10.062647+0000","last_undegraded":"2024-04-02T13:35:10.062647+0000","last_fullsized":"2024-04-02T13:35:10.062647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818970+0000","last_change":"2024-04-02T13:33:55.560120+0000","last_active":"2024-04-02T13:35:09.818970+0000","last_peered":"2024-04-02T13:35:09.818970+0000","last_clean":"2024-04-02T13:35:09.818970+0000","last_became_active":"2024-04-02T13:33:55.559117+0000","last_became_peered":"2024-04-02T13:33:55.559117+0000","last_unstale":"2024-04-02T13:35:09.818970+0000","last_undegraded":"2024-04-02T13:35:09.818970+0000","last_fullsized":"2024-04-02T13:35:09.818970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:03:42.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820256+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:09.820256+0000","last_peered":"2024-04-02T13:35:09.820256+0000","last_clean":"2024-04-02T13:35:09.820256+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:35:09.820256+0000","last_undegraded":"2024-04-02T13:35:09.820256+0000","last_fullsized":"2024-04-02T13:35:09.820256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063497+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:35:10.063497+0000","last_peered":"2024-04-02T13:35:10.063497+0000","last_clean":"2024-04-02T13:35:10.063497+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:35:10.063497+0000","last_undegraded":"2024-04-02T13:35:10.063497+0000","last_fullsized":"2024-04-02T13:35:10.063497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.817945+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:35:09.817945+0000","last_peered":"2024-04-02T13:35:09.817945+0000","last_clean":"2024-04-02T13:35:09.817945+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:35:09.817945+0000","last_undegraded":"2024-04-02T13:35:09.817945+0000","last_fullsized":"2024-04-02T13:35:09.817945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068204+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:35:10.068204+0000","last_peered":"2024-04-02T13:35:10.068204+0000","last_clean":"2024-04-02T13:35:10.068204+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:35:10.068204+0000","last_undegraded":"2024-04-02T13:35:10.068204+0000","last_fullsized":"2024-04-02T13:35:10.068204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820407+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:35:09.820407+0000","last_peered":"2024-04-02T13:35:09.820407+0000","last_clean":"2024-04-02T13:35:09.820407+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:35:09.820407+0000","last_undegraded":"2024-04-02T13:35:09.820407+0000","last_fullsized":"2024-04-02T13:35:09.820407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063961+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:35:10.063961+0000","last_peered":"2024-04-02T13:35:10.063961+0000","last_clean":"2024-04-02T13:35:10.063961+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:35:10.063961+0000","last_undegraded":"2024-04-02T13:35:10.063961+0000","last_fullsized":"2024-04-02T13:35:10.063961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919462+0000","last_change":"2024-04-02T13:33:55.555987+0000","last_active":"2024-04-02T13:35:09.919462+0000","last_peered":"2024-04-02T13:35:09.919462+0000","last_clean":"2024-04-02T13:35:09.919462+0000","last_became_active":"2024-04-02T13:33:55.555623+0000","last_became_peered":"2024-04-02T13:33:55.555623+0000","last_unstale":"2024-04-02T13:35:09.919462+0000","last_undegraded":"2024-04-02T13:35:09.919462+0000","last_fullsized":"2024-04-02T13:35:09.919462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:32.669173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062249+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:35:10.062249+0000","last_peered":"2024-04-02T13:35:10.062249+0000","last_clean":"2024-04-02T13:35:10.062249+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:35:10.062249+0000","last_undegraded":"2024-04-02T13:35:10.062249+0000","last_fullsized":"2024-04-02T13:35:10.062249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064625+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:35:10.064625+0000","last_peered":"2024-04-02T13:35:10.064625+0000","last_clean":"2024-04-02T13:35:10.064625+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:35:10.064625+0000","last_undegraded":"2024-04-02T13:35:10.064625+0000","last_fullsized":"2024-04-02T13:35:10.064625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064990+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:35:10.064990+0000","last_peered":"2024-04-02T13:35:10.064990+0000","last_clean":"2024-04-02T13:35:10.064990+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:35:10.064990+0000","last_undegraded":"2024-04-02T13:35:10.064990+0000","last_fullsized":"2024-04-02T13:35:10.064990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067588+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:35:10.067588+0000","last_peered":"2024-04-02T13:35:10.067588+0000","last_clean":"2024-04-02T13:35:10.067588+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:35:10.067588+0000","last_undegraded":"2024-04-02T13:35:10.067588+0000","last_fullsized":"2024-04-02T13:35:10.067588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070971+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:35:10.070971+0000","last_peered":"2024-04-02T13:35:10.070971+0000","last_clean":"2024-04-02T13:35:10.070971+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:35:10.070971+0000","last_undegraded":"2024-04-02T13:35:10.070971+0000","last_fullsized":"2024-04-02T13:35:10.070971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919799+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:35:09.919799+0000","last_peered":"2024-04-02T13:35:09.919799+0000","last_clean":"2024-04-02T13:35:09.919799+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:35:09.919799+0000","last_undegraded":"2024-04-02T13:35:09.919799+0000","last_fullsized":"2024-04-02T13:35:09.919799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063291+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:35:10.063291+0000","last_peered":"2024-04-02T13:35:10.063291+0000","last_clean":"2024-04-02T13:35:10.063291+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:35:10.063291+0000","last_undegraded":"2024-04-02T13:35:10.063291+0000","last_fullsized":"2024-04-02T13:35:10.063291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067551+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:35:10.067551+0000","last_peered":"2024-04-02T13:35:10.067551+0000","last_clean":"2024-04-02T13:35:10.067551+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:35:10.067551+0000","last_undegraded":"2024-04-02T13:35:10.067551+0000","last_fullsized":"2024-04-02T13:35:10.067551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818258+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:35:09.818258+0000","last_peered":"2024-04-02T13:35:09.818258+0000","last_clean":"2024-04-02T13:35:09.818258+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:35:09.818258+0000","last_undegraded":"2024-04-02T13:35:09.818258+0000","last_fullsized":"2024-04-02T13:35:09.818258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061716+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:35:10.061716+0000","last_peered":"2024-04-02T13:35:10.061716+0000","last_clean":"2024-04-02T13:35:10.061716+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:35:10.061716+0000","last_undegraded":"2024-04-02T13:35:10.061716+0000","last_fullsized":"2024-04-02T13:35:10.061716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818857+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:35:09.818857+0000","last_peered":"2024-04-02T13:35:09.818857+0000","last_clean":"2024-04-02T13:35:09.818857+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:35:09.818857+0000","last_undegraded":"2024-04-02T13:35:09.818857+0000","last_fullsized":"2024-04-02T13:35:09.818857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065791+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:35:10.065791+0000","last_peered":"2024-04-02T13:35:10.065791+0000","last_clean":"2024-04-02T13:35:10.065791+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:35:10.065791+0000","last_undegraded":"2024-04-02T13:35:10.065791+0000","last_fullsized":"2024-04-02T13:35:10.065791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061979+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:35:10.061979+0000","last_peered":"2024-04-02T13:35:10.061979+0000","last_clean":"2024-04-02T13:35:10.061979+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:35:10.061979+0000","last_undegraded":"2024-04-02T13:35:10.061979+0000","last_fullsized":"2024-04-02T13:35:10.061979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819095+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:35:09.819095+0000","last_peered":"2024-04-02T13:35:09.819095+0000","last_clean":"2024-04-02T13:35:09.819095+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:35:09.819095+0000","last_undegraded":"2024-04-02T13:35:09.819095+0000","last_fullsized":"2024-04-02T13:35:09.819095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067884+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:35:10.067884+0000","last_peered":"2024-04-02T13:35:10.067884+0000","last_clean":"2024-04-02T13:35:10.067884+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:35:10.067884+0000","last_undegraded":"2024-04-02T13:35:10.067884+0000","last_fullsized":"2024-04-02T13:35:10.067884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061881+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:35:10.061881+0000","last_peered":"2024-04-02T13:35:10.061881+0000","last_clean":"2024-04-02T13:35:10.061881+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:35:10.061881+0000","last_undegraded":"2024-04-02T13:35:10.061881+0000","last_fullsized":"2024-04-02T13:35:10.061881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819043+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:35:09.819043+0000","last_peered":"2024-04-02T13:35:09.819043+0000","last_clean":"2024-04-02T13:35:09.819043+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:35:09.819043+0000","last_undegraded":"2024-04-02T13:35:09.819043+0000","last_fullsized":"2024-04-02T13:35:09.819043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919412+0000","last_change":"2024-04-02T13:33:55.559231+0000","last_active":"2024-04-02T13:35:09.919412+0000","last_peered":"2024-04-02T13:35:09.919412+0000","last_clean":"2024-04-02T13:35:09.919412+0000","last_became_active":"2024-04-02T13:33:55.558856+0000","last_became_peered":"2024-04-02T13:33:55.558856+0000","last_unstale":"2024-04-02T13:35:09.919412+0000","last_undegraded":"2024-04-02T13:35:09.919412+0000","last_fullsized":"2024-04-02T13:35:09.919412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:12:47.406198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063082+0000","last_change":"2024-04-02T13:34:56.175049+0000","last_active":"2024-04-02T13:35:10.063082+0000","last_peered":"2024-04-02T13:35:10.063082+0000","last_clean":"2024-04-02T13:35:10.063082+0000","last_became_active":"2024-04-02T13:34:56.174477+0000","last_became_peered":"2024-04-02T13:34:56.174477+0000","last_unstale":"2024-04-02T13:35:10.063082+0000","last_undegraded":"2024-04-02T13:35:10.063082+0000","last_fullsized":"2024-04-02T13:35:10.063082+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:30.726406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":162,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315244+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:35:10.315244+0000","last_peered":"2024-04-02T13:35:10.315244+0000","last_clean":"2024-04-02T13:35:10.315244+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:35:10.315244+0000","last_undegraded":"2024-04-02T13:35:10.315244+0000","last_fullsized":"2024-04-02T13:35:10.315244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818973+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:35:09.818973+0000","last_peered":"2024-04-02T13:35:09.818973+0000","last_clean":"2024-04-02T13:35:09.818973+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:35:09.818973+0000","last_undegraded":"2024-04-02T13:35:09.818973+0000","last_fullsized":"2024-04-02T13:35:09.818973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063063+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:35:10.063063+0000","last_peered":"2024-04-02T13:35:10.063063+0000","last_clean":"2024-04-02T13:35:10.063063+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:35:10.063063+0000","last_undegraded":"2024-04-02T13:35:10.063063+0000","last_fullsized":"2024-04-02T13:35:10.063063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820123+0000","last_change":"2024-04-02T13:33:55.565630+0000","last_active":"2024-04-02T13:35:09.820123+0000","last_peered":"2024-04-02T13:35:09.820123+0000","last_clean":"2024-04-02T13:35:09.820123+0000","last_became_active":"2024-04-02T13:33:55.559239+0000","last_became_peered":"2024-04-02T13:33:55.559239+0000","last_unstale":"2024-04-02T13:35:09.820123+0000","last_undegraded":"2024-04-02T13:35:09.820123+0000","last_fullsized":"2024-04-02T13:35:09.820123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:46.161608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928087+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:35:09.928087+0000","last_peered":"2024-04-02T13:35:09.928087+0000","last_clean":"2024-04-02T13:35:09.928087+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:35:09.928087+0000","last_undegraded":"2024-04-02T13:35:09.928087+0000","last_fullsized":"2024-04-02T13:35:09.928087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070614+0000","last_change":"2024-04-02T13:33:55.558751+0000","last_active":"2024-04-02T13:35:10.070614+0000","last_peered":"2024-04-02T13:35:10.070614+0000","last_clean":"2024-04-02T13:35:10.070614+0000","last_became_active":"2024-04-02T13:33:55.558278+0000","last_became_peered":"2024-04-02T13:33:55.558278+0000","last_unstale":"2024-04-02T13:35:10.070614+0000","last_undegraded":"2024-04-02T13:35:10.070614+0000","last_fullsized":"2024-04-02T13:35:10.070614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:15.164871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062024+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:35:10.062024+0000","last_peered":"2024-04-02T13:35:10.062024+0000","last_clean":"2024-04-02T13:35:10.062024+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:35:10.062024+0000","last_undegraded":"2024-04-02T13:35:10.062024+0000","last_fullsized":"2024-04-02T13:35:10.062024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818108+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:35:09.818108+0000","last_peered":"2024-04-02T13:35:09.818108+0000","last_clean":"2024-04-02T13:35:09.818108+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:35:09.818108+0000","last_undegraded":"2024-04-02T13:35:09.818108+0000","last_fullsized":"2024-04-02T13:35:09.818108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065607+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:35:10.065607+0000","last_peered":"2024-04-02T13:35:10.065607+0000","last_clean":"2024-04-02T13:35:10.065607+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:35:10.065607+0000","last_undegraded":"2024-04-02T13:35:10.065607+0000","last_fullsized":"2024-04-02T13:35:10.065607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927765+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:35:09.927765+0000","last_peered":"2024-04-02T13:35:09.927765+0000","last_clean":"2024-04-02T13:35:09.927765+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:35:09.927765+0000","last_undegraded":"2024-04-02T13:35:09.927765+0000","last_fullsized":"2024-04-02T13:35:09.927765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919045+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:35:09.919045+0000","last_peered":"2024-04-02T13:35:09.919045+0000","last_clean":"2024-04-02T13:35:09.919045+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:35:09.919045+0000","last_undegraded":"2024-04-02T13:35:09.919045+0000","last_fullsized":"2024-04-02T13:35:09.919045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818007+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:35:09.818007+0000","last_peered":"2024-04-02T13:35:09.818007+0000","last_clean":"2024-04-02T13:35:09.818007+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:35:09.818007+0000","last_undegraded":"2024-04-02T13:35:09.818007+0000","last_fullsized":"2024-04-02T13:35:09.818007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":278,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.314531+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:35:10.314531+0000","last_peered":"2024-04-02T13:35:10.314531+0000","last_clean":"2024-04-02T13:35:10.314531+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:35:10.314531+0000","last_undegraded":"2024-04-02T13:35:10.314531+0000","last_fullsized":"2024-04-02T13:35:10.314531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068022+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:10.068022+0000","last_peered":"2024-04-02T13:35:10.068022+0000","last_clean":"2024-04-02T13:35:10.068022+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:35:10.068022+0000","last_undegraded":"2024-04-02T13:35:10.068022+0000","last_fullsized":"2024-04-02T13:35:10.068022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.918900+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:35:09.918900+0000","last_peered":"2024-04-02T13:35:09.918900+0000","last_clean":"2024-04-02T13:35:09.918900+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:35:09.918900+0000","last_undegraded":"2024-04-02T13:35:09.918900+0000","last_fullsized":"2024-04-02T13:35:09.918900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063718+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:35:10.063718+0000","last_peered":"2024-04-02T13:35:10.063718+0000","last_clean":"2024-04-02T13:35:10.063718+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:35:10.063718+0000","last_undegraded":"2024-04-02T13:35:10.063718+0000","last_fullsized":"2024-04-02T13:35:10.063718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065934+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:35:10.065934+0000","last_peered":"2024-04-02T13:35:10.065934+0000","last_clean":"2024-04-02T13:35:10.065934+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:35:10.065934+0000","last_undegraded":"2024-04-02T13:35:10.065934+0000","last_fullsized":"2024-04-02T13:35:10.065934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062853+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:35:10.062853+0000","last_peered":"2024-04-02T13:35:10.062853+0000","last_clean":"2024-04-02T13:35:10.062853+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:35:10.062853+0000","last_undegraded":"2024-04-02T13:35:10.062853+0000","last_fullsized":"2024-04-02T13:35:10.062853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928052+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:35:09.928052+0000","last_peered":"2024-04-02T13:35:09.928052+0000","last_clean":"2024-04-02T13:35:09.928052+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:35:09.928052+0000","last_undegraded":"2024-04-02T13:35:09.928052+0000","last_fullsized":"2024-04-02T13:35:09.928052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068046+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:35:10.068046+0000","last_peered":"2024-04-02T13:35:10.068046+0000","last_clean":"2024-04-02T13:35:10.068046+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:35:10.068046+0000","last_undegraded":"2024-04-02T13:35:10.068046+0000","last_fullsized":"2024-04-02T13:35:10.068046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819835+0000","last_change":"2024-04-02T13:34:56.181716+0000","last_active":"2024-04-02T13:35:09.819835+0000","last_peered":"2024-04-02T13:35:09.819835+0000","last_clean":"2024-04-02T13:35:09.819835+0000","last_became_active":"2024-04-02T13:34:56.181233+0000","last_became_peered":"2024-04-02T13:34:56.181233+0000","last_unstale":"2024-04-02T13:35:09.819835+0000","last_undegraded":"2024-04-02T13:35:09.819835+0000","last_fullsized":"2024-04-02T13:35:09.819835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:24:55.347041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315476+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:35:10.315476+0000","last_peered":"2024-04-02T13:35:10.315476+0000","last_clean":"2024-04-02T13:35:10.315476+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:35:10.315476+0000","last_undegraded":"2024-04-02T13:35:10.315476+0000","last_fullsized":"2024-04-02T13:35:10.315476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":193,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819358+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:35:09.819358+0000","last_peered":"2024-04-02T13:35:09.819358+0000","last_clean":"2024-04-02T13:35:09.819358+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:35:09.819358+0000","last_undegraded":"2024-04-02T13:35:09.819358+0000","last_fullsized":"2024-04-02T13:35:09.819358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928355+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:35:09.928355+0000","last_peered":"2024-04-02T13:35:09.928355+0000","last_clean":"2024-04-02T13:35:09.928355+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:35:09.928355+0000","last_undegraded":"2024-04-02T13:35:09.928355+0000","last_fullsized":"2024-04-02T13:35:09.928355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062970+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:35:10.062970+0000","last_peered":"2024-04-02T13:35:10.062970+0000","last_clean":"2024-04-02T13:35:10.062970+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:35:10.062970+0000","last_undegraded":"2024-04-02T13:35:10.062970+0000","last_fullsized":"2024-04-02T13:35:10.062970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062183+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:35:10.062183+0000","last_peered":"2024-04-02T13:35:10.062183+0000","last_clean":"2024-04-02T13:35:10.062183+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:35:10.062183+0000","last_undegraded":"2024-04-02T13:35:10.062183+0000","last_fullsized":"2024-04-02T13:35:10.062183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070601+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:35:10.070601+0000","last_peered":"2024-04-02T13:35:10.070601+0000","last_clean":"2024-04-02T13:35:10.070601+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:35:10.070601+0000","last_undegraded":"2024-04-02T13:35:10.070601+0000","last_fullsized":"2024-04-02T13:35:10.070601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928018+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:35:09.928018+0000","last_peered":"2024-04-02T13:35:09.928018+0000","last_clean":"2024-04-02T13:35:09.928018+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:35:09.928018+0000","last_undegraded":"2024-04-02T13:35:09.928018+0000","last_fullsized":"2024-04-02T13:35:09.928018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063434+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:35:10.063434+0000","last_peered":"2024-04-02T13:35:10.063434+0000","last_clean":"2024-04-02T13:35:10.063434+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:35:10.063434+0000","last_undegraded":"2024-04-02T13:35:10.063434+0000","last_fullsized":"2024-04-02T13:35:10.063434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919844+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:35:09.919844+0000","last_peered":"2024-04-02T13:35:09.919844+0000","last_clean":"2024-04-02T13:35:09.919844+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:35:09.919844+0000","last_undegraded":"2024-04-02T13:35:09.919844+0000","last_fullsized":"2024-04-02T13:35:09.919844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":163,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315744+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:35:10.315744+0000","last_peered":"2024-04-02T13:35:10.315744+0000","last_clean":"2024-04-02T13:35:10.315744+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:35:10.315744+0000","last_undegraded":"2024-04-02T13:35:10.315744+0000","last_fullsized":"2024-04-02T13:35:10.315744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063690+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:35:10.063690+0000","last_peered":"2024-04-02T13:35:10.063690+0000","last_clean":"2024-04-02T13:35:10.063690+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:35:10.063690+0000","last_undegraded":"2024-04-02T13:35:10.063690+0000","last_fullsized":"2024-04-02T13:35:10.063690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070705+0000","last_change":"2024-04-02T13:33:55.574777+0000","last_active":"2024-04-02T13:35:10.070705+0000","last_peered":"2024-04-02T13:35:10.070705+0000","last_clean":"2024-04-02T13:35:10.070705+0000","last_became_active":"2024-04-02T13:33:55.561040+0000","last_became_peered":"2024-04-02T13:33:55.561040+0000","last_unstale":"2024-04-02T13:35:10.070705+0000","last_undegraded":"2024-04-02T13:35:10.070705+0000","last_fullsized":"2024-04-02T13:35:10.070705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:23.184540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927913+0000","last_change":"2024-04-02T13:33:55.560045+0000","last_active":"2024-04-02T13:35:09.927913+0000","last_peered":"2024-04-02T13:35:09.927913+0000","last_clean":"2024-04-02T13:35:09.927913+0000","last_became_active":"2024-04-02T13:33:55.559659+0000","last_became_peered":"2024-04-02T13:33:55.559659+0000","last_unstale":"2024-04-02T13:35:09.927913+0000","last_undegraded":"2024-04-02T13:35:09.927913+0000","last_fullsized":"2024-04-02T13:35:09.927913+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:10:42.028143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927883+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:35:09.927883+0000","last_peered":"2024-04-02T13:35:09.927883+0000","last_clean":"2024-04-02T13:35:09.927883+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:35:09.927883+0000","last_undegraded":"2024-04-02T13:35:09.927883+0000","last_fullsized":"2024-04-02T13:35:09.927883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819290+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:35:09.819290+0000","last_peered":"2024-04-02T13:35:09.819290+0000","last_clean":"2024-04-02T13:35:09.819290+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:35:09.819290+0000","last_undegraded":"2024-04-02T13:35:09.819290+0000","last_fullsized":"2024-04-02T13:35:09.819290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070776+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:35:10.070776+0000","last_peered":"2024-04-02T13:35:10.070776+0000","last_clean":"2024-04-02T13:35:10.070776+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:35:10.070776+0000","last_undegraded":"2024-04-02T13:35:10.070776+0000","last_fullsized":"2024-04-02T13:35:10.070776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818918+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:35:09.818918+0000","last_peered":"2024-04-02T13:35:09.818918+0000","last_clean":"2024-04-02T13:35:09.818918+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:35:09.818918+0000","last_undegraded":"2024-04-02T13:35:09.818918+0000","last_fullsized":"2024-04-02T13:35:09.818918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919605+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:35:09.919605+0000","last_peered":"2024-04-02T13:35:09.919605+0000","last_clean":"2024-04-02T13:35:09.919605+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:35:09.919605+0000","last_undegraded":"2024-04-02T13:35:09.919605+0000","last_fullsized":"2024-04-02T13:35:09.919605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820178+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:35:09.820178+0000","last_peered":"2024-04-02T13:35:09.820178+0000","last_clean":"2024-04-02T13:35:09.820178+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:35:09.820178+0000","last_undegraded":"2024-04-02T13:35:09.820178+0000","last_fullsized":"2024-04-02T13:35:09.820178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062428+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:35:10.062428+0000","last_peered":"2024-04-02T13:35:10.062428+0000","last_clean":"2024-04-02T13:35:10.062428+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:35:10.062428+0000","last_undegraded":"2024-04-02T13:35:10.062428+0000","last_fullsized":"2024-04-02T13:35:10.062428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.323339+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:35:10.323339+0000","last_peered":"2024-04-02T13:35:10.323339+0000","last_clean":"2024-04-02T13:35:10.323339+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:35:10.323339+0000","last_undegraded":"2024-04-02T13:35:10.323339+0000","last_fullsized":"2024-04-02T13:35:10.323339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820157+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:35:09.820157+0000","last_peered":"2024-04-02T13:35:09.820157+0000","last_clean":"2024-04-02T13:35:09.820157+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:35:09.820157+0000","last_undegraded":"2024-04-02T13:35:09.820157+0000","last_fullsized":"2024-04-02T13:35:09.820157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062937+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:35:10.062937+0000","last_peered":"2024-04-02T13:35:10.062937+0000","last_clean":"2024-04-02T13:35:10.062937+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:35:10.062937+0000","last_undegraded":"2024-04-02T13:35:10.062937+0000","last_fullsized":"2024-04-02T13:35:10.062937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928098+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:35:09.928098+0000","last_peered":"2024-04-02T13:35:09.928098+0000","last_clean":"2024-04-02T13:35:09.928098+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:35:09.928098+0000","last_undegraded":"2024-04-02T13:35:09.928098+0000","last_fullsized":"2024-04-02T13:35:09.928098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.066596+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:35:10.066596+0000","last_peered":"2024-04-02T13:35:10.066596+0000","last_clean":"2024-04-02T13:35:10.066596+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:35:10.066596+0000","last_undegraded":"2024-04-02T13:35:10.066596+0000","last_fullsized":"2024-04-02T13:35:10.066596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063688+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:35:10.063688+0000","last_peered":"2024-04-02T13:35:10.063688+0000","last_clean":"2024-04-02T13:35:10.063688+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:35:10.063688+0000","last_undegraded":"2024-04-02T13:35:10.063688+0000","last_fullsized":"2024-04-02T13:35:10.063688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818230+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:35:09.818230+0000","last_peered":"2024-04-02T13:35:09.818230+0000","last_clean":"2024-04-02T13:35:09.818230+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:35:09.818230+0000","last_undegraded":"2024-04-02T13:35:09.818230+0000","last_fullsized":"2024-04-02T13:35:09.818230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062201+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:35:10.062201+0000","last_peered":"2024-04-02T13:35:10.062201+0000","last_clean":"2024-04-02T13:35:10.062201+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:35:10.062201+0000","last_undegraded":"2024-04-02T13:35:10.062201+0000","last_fullsized":"2024-04-02T13:35:10.062201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063125+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:35:10.063125+0000","last_peered":"2024-04-02T13:35:10.063125+0000","last_clean":"2024-04-02T13:35:10.063125+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:35:10.063125+0000","last_undegraded":"2024-04-02T13:35:10.063125+0000","last_fullsized":"2024-04-02T13:35:10.063125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928211+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:35:09.928211+0000","last_peered":"2024-04-02T13:35:09.928211+0000","last_clean":"2024-04-02T13:35:09.928211+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:35:09.928211+0000","last_undegraded":"2024-04-02T13:35:09.928211+0000","last_fullsized":"2024-04-02T13:35:09.928211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061853+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:35:10.061853+0000","last_peered":"2024-04-02T13:35:10.061853+0000","last_clean":"2024-04-02T13:35:10.061853+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:35:10.061853+0000","last_undegraded":"2024-04-02T13:35:10.061853+0000","last_fullsized":"2024-04-02T13:35:10.061853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063479+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:35:10.063479+0000","last_peered":"2024-04-02T13:35:10.063479+0000","last_clean":"2024-04-02T13:35:10.063479+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:35:10.063479+0000","last_undegraded":"2024-04-02T13:35:10.063479+0000","last_fullsized":"2024-04-02T13:35:10.063479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070541+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:35:10.070541+0000","last_peered":"2024-04-02T13:35:10.070541+0000","last_clean":"2024-04-02T13:35:10.070541+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:35:10.070541+0000","last_undegraded":"2024-04-02T13:35:10.070541+0000","last_fullsized":"2024-04-02T13:35:10.070541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819228+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:35:09.819228+0000","last_peered":"2024-04-02T13:35:09.819228+0000","last_clean":"2024-04-02T13:35:09.819228+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:35:09.819228+0000","last_undegraded":"2024-04-02T13:35:09.819228+0000","last_fullsized":"2024-04-02T13:35:09.819228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062665+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:35:10.062665+0000","last_peered":"2024-04-02T13:35:10.062665+0000","last_clean":"2024-04-02T13:35:10.062665+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:35:10.062665+0000","last_undegraded":"2024-04-02T13:35:10.062665+0000","last_fullsized":"2024-04-02T13:35:10.062665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818025+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:35:09.818025+0000","last_peered":"2024-04-02T13:35:09.818025+0000","last_clean":"2024-04-02T13:35:09.818025+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:35:09.818025+0000","last_undegraded":"2024-04-02T13:35:09.818025+0000","last_fullsized":"2024-04-02T13:35:09.818025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'49","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.547985+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:35:10.547985+0000","last_peered":"2024-04-02T13:35:10.547985+0000","last_clean":"2024-04-02T13:35:10.547985+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:35:10.547985+0000","last_undegraded":"2024-04-02T13:35:10.547985+0000","last_fullsized":"2024-04-02T13:35:10.547985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818962+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:35:09.818962+0000","last_peered":"2024-04-02T13:35:09.818962+0000","last_clean":"2024-04-02T13:35:09.818962+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:35:09.818962+0000","last_undegraded":"2024-04-02T13:35:09.818962+0000","last_fullsized":"2024-04-02T13:35:09.818962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062662+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:35:10.062662+0000","last_peered":"2024-04-02T13:35:10.062662+0000","last_clean":"2024-04-02T13:35:10.062662+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:35:10.062662+0000","last_undegraded":"2024-04-02T13:35:10.062662+0000","last_fullsized":"2024-04-02T13:35:10.062662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062851+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:35:10.062851+0000","last_peered":"2024-04-02T13:35:10.062851+0000","last_clean":"2024-04-02T13:35:10.062851+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:35:10.062851+0000","last_undegraded":"2024-04-02T13:35:10.062851+0000","last_fullsized":"2024-04-02T13:35:10.062851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065708+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:35:10.065708+0000","last_peered":"2024-04-02T13:35:10.065708+0000","last_clean":"2024-04-02T13:35:10.065708+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:35:10.065708+0000","last_undegraded":"2024-04-02T13:35:10.065708+0000","last_fullsized":"2024-04-02T13:35:10.065708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061934+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:35:10.061934+0000","last_peered":"2024-04-02T13:35:10.061934+0000","last_clean":"2024-04-02T13:35:10.061934+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:35:10.061934+0000","last_undegraded":"2024-04-02T13:35:10.061934+0000","last_fullsized":"2024-04-02T13:35:10.061934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067569+0000","last_change":"2024-04-02T13:33:55.564335+0000","last_active":"2024-04-02T13:35:10.067569+0000","last_peered":"2024-04-02T13:35:10.067569+0000","last_clean":"2024-04-02T13:35:10.067569+0000","last_became_active":"2024-04-02T13:33:55.560370+0000","last_became_peered":"2024-04-02T13:33:55.560370+0000","last_unstale":"2024-04-02T13:35:10.067569+0000","last_undegraded":"2024-04-02T13:35:10.067569+0000","last_fullsized":"2024-04-02T13:35:10.067569+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:14.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818822+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:35:09.818822+0000","last_peered":"2024-04-02T13:35:09.818822+0000","last_clean":"2024-04-02T13:35:09.818822+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:35:09.818822+0000","last_undegraded":"2024-04-02T13:35:09.818822+0000","last_fullsized":"2024-04-02T13:35:09.818822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919950+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:35:09.919950+0000","last_peered":"2024-04-02T13:35:09.919950+0000","last_clean":"2024-04-02T13:35:09.919950+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:35:09.919950+0000","last_undegraded":"2024-04-02T13:35:09.919950+0000","last_fullsized":"2024-04-02T13:35:09.919950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818121+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:35:09.818121+0000","last_peered":"2024-04-02T13:35:09.818121+0000","last_clean":"2024-04-02T13:35:09.818121+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:35:09.818121+0000","last_undegraded":"2024-04-02T13:35:09.818121+0000","last_fullsized":"2024-04-02T13:35:09.818121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070490+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:35:10.070490+0000","last_peered":"2024-04-02T13:35:10.070490+0000","last_clean":"2024-04-02T13:35:10.070490+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:35:10.070490+0000","last_undegraded":"2024-04-02T13:35:10.070490+0000","last_fullsized":"2024-04-02T13:35:10.070490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062642+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:35:10.062642+0000","last_peered":"2024-04-02T13:35:10.062642+0000","last_clean":"2024-04-02T13:35:10.062642+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:35:10.062642+0000","last_undegraded":"2024-04-02T13:35:10.062642+0000","last_fullsized":"2024-04-02T13:35:10.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062817+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:35:10.062817+0000","last_peered":"2024-04-02T13:35:10.062817+0000","last_clean":"2024-04-02T13:35:10.062817+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:35:10.062817+0000","last_undegraded":"2024-04-02T13:35:10.062817+0000","last_fullsized":"2024-04-02T13:35:10.062817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818375+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:35:09.818375+0000","last_peered":"2024-04-02T13:35:09.818375+0000","last_clean":"2024-04-02T13:35:09.818375+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:35:09.818375+0000","last_undegraded":"2024-04-02T13:35:09.818375+0000","last_fullsized":"2024-04-02T13:35:09.818375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062023+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:35:10.062023+0000","last_peered":"2024-04-02T13:35:10.062023+0000","last_clean":"2024-04-02T13:35:10.062023+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:35:10.062023+0000","last_undegraded":"2024-04-02T13:35:10.062023+0000","last_fullsized":"2024-04-02T13:35:10.062023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818964+0000","last_change":"2024-04-02T13:33:55.565608+0000","last_active":"2024-04-02T13:35:09.818964+0000","last_peered":"2024-04-02T13:35:09.818964+0000","last_clean":"2024-04-02T13:35:09.818964+0000","last_became_active":"2024-04-02T13:33:55.557475+0000","last_became_peered":"2024-04-02T13:33:55.557475+0000","last_unstale":"2024-04-02T13:35:09.818964+0000","last_undegraded":"2024-04-02T13:35:09.818964+0000","last_fullsized":"2024-04-02T13:35:09.818964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:17:21.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063486+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:35:10.063486+0000","last_peered":"2024-04-02T13:35:10.063486+0000","last_clean":"2024-04-02T13:35:10.063486+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:35:10.063486+0000","last_undegraded":"2024-04-02T13:35:10.063486+0000","last_fullsized":"2024-04-02T13:35:10.063486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067660+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:35:10.067660+0000","last_peered":"2024-04-02T13:35:10.067660+0000","last_clean":"2024-04-02T13:35:10.067660+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:35:10.067660+0000","last_undegraded":"2024-04-02T13:35:10.067660+0000","last_fullsized":"2024-04-02T13:35:10.067660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'8","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315848+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:35:10.315848+0000","last_peered":"2024-04-02T13:35:10.315848+0000","last_clean":"2024-04-02T13:35:10.315848+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:35:10.315848+0000","last_undegraded":"2024-04-02T13:35:10.315848+0000","last_fullsized":"2024-04-02T13:35:10.315848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064153+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:35:10.064153+0000","last_peered":"2024-04-02T13:35:10.064153+0000","last_clean":"2024-04-02T13:35:10.064153+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:35:10.064153+0000","last_undegraded":"2024-04-02T13:35:10.064153+0000","last_fullsized":"2024-04-02T13:35:10.064153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064951+0000","last_change":"2024-04-02T13:34:56.181555+0000","last_active":"2024-04-02T13:35:10.064951+0000","last_peered":"2024-04-02T13:35:10.064951+0000","last_clean":"2024-04-02T13:35:10.064951+0000","last_became_active":"2024-04-02T13:34:56.181156+0000","last_became_peered":"2024-04-02T13:34:56.181156+0000","last_unstale":"2024-04-02T13:35:10.064951+0000","last_undegraded":"2024-04-02T13:35:10.064951+0000","last_fullsized":"2024-04-02T13:35:10.064951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:23.387417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":82,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.027635+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:35:10.027635+0000","last_peered":"2024-04-02T13:35:10.027635+0000","last_clean":"2024-04-02T13:35:10.027635+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:35:10.027635+0000","last_undegraded":"2024-04-02T13:35:10.027635+0000","last_fullsized":"2024-04-02T13:35:10.027635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063750+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:35:10.063750+0000","last_peered":"2024-04-02T13:35:10.063750+0000","last_clean":"2024-04-02T13:35:10.063750+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:35:10.063750+0000","last_undegraded":"2024-04-02T13:35:10.063750+0000","last_fullsized":"2024-04-02T13:35:10.063750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927710+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:35:09.927710+0000","last_peered":"2024-04-02T13:35:09.927710+0000","last_clean":"2024-04-02T13:35:09.927710+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:35:09.927710+0000","last_undegraded":"2024-04-02T13:35:09.927710+0000","last_fullsized":"2024-04-02T13:35:09.927710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062701+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:35:10.062701+0000","last_peered":"2024-04-02T13:35:10.062701+0000","last_clean":"2024-04-02T13:35:10.062701+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:35:10.062701+0000","last_undegraded":"2024-04-02T13:35:10.062701+0000","last_fullsized":"2024-04-02T13:35:10.062701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063549+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:35:10.063549+0000","last_peered":"2024-04-02T13:35:10.063549+0000","last_clean":"2024-04-02T13:35:10.063549+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:35:10.063549+0000","last_undegraded":"2024-04-02T13:35:10.063549+0000","last_fullsized":"2024-04-02T13:35:10.063549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070648+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:35:10.070648+0000","last_peered":"2024-04-02T13:35:10.070648+0000","last_clean":"2024-04-02T13:35:10.070648+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:35:10.070648+0000","last_undegraded":"2024-04-02T13:35:10.070648+0000","last_fullsized":"2024-04-02T13:35:10.070648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062874+0000","last_change":"2024-04-02T13:33:55.562541+0000","last_active":"2024-04-02T13:35:10.062874+0000","last_peered":"2024-04-02T13:35:10.062874+0000","last_clean":"2024-04-02T13:35:10.062874+0000","last_became_active":"2024-04-02T13:33:55.562167+0000","last_became_peered":"2024-04-02T13:33:55.562167+0000","last_unstale":"2024-04-02T13:35:10.062874+0000","last_undegraded":"2024-04-02T13:35:10.062874+0000","last_fullsized":"2024-04-02T13:35:10.062874+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:29.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062695+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:35:10.062695+0000","last_peered":"2024-04-02T13:35:10.062695+0000","last_clean":"2024-04-02T13:35:10.062695+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:35:10.062695+0000","last_undegraded":"2024-04-02T13:35:10.062695+0000","last_fullsized":"2024-04-02T13:35:10.062695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065156+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:35:10.065156+0000","last_peered":"2024-04-02T13:35:10.065156+0000","last_clean":"2024-04-02T13:35:10.065156+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:35:10.065156+0000","last_undegraded":"2024-04-02T13:35:10.065156+0000","last_fullsized":"2024-04-02T13:35:10.065156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062536+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:35:10.062536+0000","last_peered":"2024-04-02T13:35:10.062536+0000","last_clean":"2024-04-02T13:35:10.062536+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:35:10.062536+0000","last_undegraded":"2024-04-02T13:35:10.062536+0000","last_fullsized":"2024-04-02T13:35:10.062536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070908+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:35:10.070908+0000","last_peered":"2024-04-02T13:35:10.070908+0000","last_clean":"2024-04-02T13:35:10.070908+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:35:10.070908+0000","last_undegraded":"2024-04-02T13:35:10.070908+0000","last_fullsized":"2024-04-02T13:35:10.070908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063259+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:35:10.063259+0000","last_peered":"2024-04-02T13:35:10.063259+0000","last_clean":"2024-04-02T13:35:10.063259+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:35:10.063259+0000","last_undegraded":"2024-04-02T13:35:10.063259+0000","last_fullsized":"2024-04-02T13:35:10.063259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819077+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:35:09.819077+0000","last_peered":"2024-04-02T13:35:09.819077+0000","last_clean":"2024-04-02T13:35:09.819077+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:35:09.819077+0000","last_undegraded":"2024-04-02T13:35:09.819077+0000","last_fullsized":"2024-04-02T13:35:09.819077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927773+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:35:09.927773+0000","last_peered":"2024-04-02T13:35:09.927773+0000","last_clean":"2024-04-02T13:35:09.927773+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:35:09.927773+0000","last_undegraded":"2024-04-02T13:35:09.927773+0000","last_fullsized":"2024-04-02T13:35:09.927773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818159+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:35:09.818159+0000","last_peered":"2024-04-02T13:35:09.818159+0000","last_clean":"2024-04-02T13:35:09.818159+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:35:09.818159+0000","last_undegraded":"2024-04-02T13:35:09.818159+0000","last_fullsized":"2024-04-02T13:35:09.818159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067987+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:35:10.067987+0000","last_peered":"2024-04-02T13:35:10.067987+0000","last_clean":"2024-04-02T13:35:10.067987+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:35:10.067987+0000","last_undegraded":"2024-04-02T13:35:10.067987+0000","last_fullsized":"2024-04-02T13:35:10.067987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062204+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:35:10.062204+0000","last_peered":"2024-04-02T13:35:10.062204+0000","last_clean":"2024-04-02T13:35:10.062204+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:35:10.062204+0000","last_undegraded":"2024-04-02T13:35:10.062204+0000","last_fullsized":"2024-04-02T13:35:10.062204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070777+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:35:10.070777+0000","last_peered":"2024-04-02T13:35:10.070777+0000","last_clean":"2024-04-02T13:35:10.070777+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:35:10.070777+0000","last_undegraded":"2024-04-02T13:35:10.070777+0000","last_fullsized":"2024-04-02T13:35:10.070777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602937,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1334,"num_read_kb":785,"num_write":375,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1808811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":525216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1322543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1515520,"data_stored":931933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1499136,"data_stored":928103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1132669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":530516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":600591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":584678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1054649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":534590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37148,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708004,"statfs":{"total":95995035648,"available":95956996096,"internally_reserved":0,"allocated":1855488,"data_stored":1259069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:35:12.946 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-02T13:35:12.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-02T13:35:13.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 -- 172.21.15.122:0/1536118527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb100102dc0 msgr2=0x7fb100103240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 --2- 172.21.15.122:0/1536118527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb100102dc0 0x7fb100103240 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009930 tx=0x7fb0f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 -- 172.21.15.122:0/1536118527 shutdown_connections 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 --2- 172.21.15.122:0/1536118527 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 0x7fb10010a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 --2- 172.21.15.122:0/1536118527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb100102dc0 0x7fb100103240 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 --2- 172.21.15.122:0/1536118527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1000ffdf0 0x7fb100100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 -- 172.21.15.122:0/1536118527 >> 172.21.15.122:0/1536118527 conn(0x7fb1000fb6e0 msgr2=0x7fb1000fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 -- 172.21.15.122:0/1536118527 shutdown_connections 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 -- 172.21.15.122:0/1536118527 wait complete. 2024-04-02T13:35:13.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.082+0000 7fb105515640 1 Processor -- start 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 -- start start 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb100102dc0 0x7fb100198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 0x7fb10019f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010d7b0 con 0x7fb1000ffdf0 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010d630 con 0x7fb100102dc0 2024-04-02T13:35:13.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb105515640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb10010d930 con 0x7fb100103780 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0fe7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb100102dc0 0x7fb100198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37374/0 (socket says 172.21.15.122:37374) 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 -- 172.21.15.122:0/2125929180 learned_addr learned my addr 172.21.15.122:0/2125929180 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0ff7fe640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 0x7fb10019f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 -- 172.21.15.122:0/2125929180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 msgr2=0x7fb10019f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 0x7fb10019f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 -- 172.21.15.122:0/2125929180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb100102dc0 msgr2=0x7fb100198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb100102dc0 0x7fb100198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.083+0000 7fb0feffd640 1 -- 172.21.15.122:0/2125929180 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10019fa50 con 0x7fb1000ffdf0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb0feffd640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9c0 tx=0x7fb0f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000cd80 con 0x7fb1000ffdf0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f00035f0 con 0x7fb1000ffdf0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10019fd40 con 0x7fb1000ffdf0 2024-04-02T13:35:13.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0010690 con 0x7fb1000ffdf0 2024-04-02T13:35:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.084+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1001a01b0 con 0x7fb1000ffdf0 2024-04-02T13:35:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.085+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100108fc0 con 0x7fb1000ffdf0 2024-04-02T13:35:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.086+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7fb0f00037a0 con 0x7fb1000ffdf0 2024-04-02T13:35:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.086+0000 7fb0e7fff640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 0x7fb0e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.087+0000 7fb0fe7fc640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 0x7fb0e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.087+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb0f00d0f70 con 0x7fb1000ffdf0 2024-04-02T13:35:13.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.087+0000 7fb0fe7fc640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 0x7fb0e80a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402efd0 tx=0x7fb0f4000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.091+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb0f009d280 con 0x7fb1000ffdf0 2024-04-02T13:35:13.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.280+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb100061f40 con 0x7fb0e80a3d20 2024-04-02T13:35:13.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.290+0000 7fb0e7fff640 1 -- 172.21.15.122:0/2125929180 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339429 (secure 0 0 0) 0x7fb100061f40 con 0x7fb0e80a3d20 2024-04-02T13:35:13.297 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:13.298 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-04-02T13:35:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 msgr2=0x7fb0e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 0x7fb0e80a61e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb0f402efd0 tx=0x7fb0f4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 msgr2=0x7fb100197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb0f000e9c0 tx=0x7fb0f000ee80 comp rx=0 tx=0).stop 2024-04-02T13:35:13.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 shutdown_connections 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0e80a3d20 0x7fb0e80a61e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb100103780 0x7fb10019f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb100102dc0 0x7fb100198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 --2- 172.21.15.122:0/2125929180 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1000ffdf0 0x7fb100197e30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 >> 172.21.15.122:0/2125929180 conn(0x7fb1000fb6e0 msgr2=0x7fb100106190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 shutdown_connections 2024-04-02T13:35:13.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.294+0000 7fb105515640 1 -- 172.21.15.122:0/2125929180 wait complete. 2024-04-02T13:35:13.308 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-02T13:35:11.831805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1193539,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1470,"num_read_kb":904,"num_write":558,"num_write_kb":4412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":943,"ondisk_log_size":943,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382296,"kb_used_data":16468,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559528,"statfs":{"total":1151940427776,"available":1151548956672,"internally_reserved":0,"allocated":16863232,"data_stored":9926551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.987906"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.071042+0000","last_change":"2024-04-02T13:32:50.527254+0000","last_active":"2024-04-02T13:35:10.071042+0000","last_peered":"2024-04-02T13:35:10.071042+0000","last_clean":"2024-04-02T13:35:10.071042+0000","last_became_active":"2024-04-02T13:32:50.526825+0000","last_became_peered":"2024-04-02T13:32:50.526825+0000","last_unstale":"2024-04-02T13:35:10.071042+0000","last_undegraded":"2024-04-02T13:35:10.071042+0000","last_fullsized":"2024-04-02T13:35:10.071042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:45:10.604294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065645+0000","last_change":"2024-04-02T13:32:50.532917+0000","last_active":"2024-04-02T13:35:10.065645+0000","last_peered":"2024-04-02T13:35:10.065645+0000","last_clean":"2024-04-02T13:35:10.065645+0000","last_became_active":"2024-04-02T13:32:50.532592+0000","last_became_peered":"2024-04-02T13:32:50.532592+0000","last_unstale":"2024-04-02T13:35:10.065645+0000","last_undegraded":"2024-04-02T13:35:10.065645+0000","last_fullsized":"2024-04-02T13:35:10.065645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:00:15.164588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065404+0000","last_change":"2024-04-02T13:32:50.527936+0000","last_active":"2024-04-02T13:35:10.065404+0000","last_peered":"2024-04-02T13:35:10.065404+0000","last_clean":"2024-04-02T13:35:10.065404+0000","last_became_active":"2024-04-02T13:32:50.527609+0000","last_became_peered":"2024-04-02T13:32:50.527609+0000","last_unstale":"2024-04-02T13:35:10.065404+0000","last_undegraded":"2024-04-02T13:35:10.065404+0000","last_fullsized":"2024-04-02T13:35:10.065404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:07.794027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062647+0000","last_change":"2024-04-02T13:32:50.540406+0000","last_active":"2024-04-02T13:35:10.062647+0000","last_peered":"2024-04-02T13:35:10.062647+0000","last_clean":"2024-04-02T13:35:10.062647+0000","last_became_active":"2024-04-02T13:32:50.540117+0000","last_became_peered":"2024-04-02T13:32:50.540117+0000","last_unstale":"2024-04-02T13:35:10.062647+0000","last_undegraded":"2024-04-02T13:35:10.062647+0000","last_fullsized":"2024-04-02T13:35:10.062647+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:40:04.579566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818970+0000","last_change":"2024-04-02T13:33:55.560120+0000","last_active":"2024-04-02T13:35:09.818970+0000","last_peered":"2024-04-02T13:35:09.818970+0000","last_clean":"2024-04-02T13:35:09.818970+0000","last_became_active":"2024-04-02T13:33:55.559117+0000","last_became_peered":"2024-04-02T13:33:55.559117+0000","last_unstale":"2024-04-02T13:35:09.818970+0000","last_undegraded":"2024-04-02T13:35:09.818970+0000","last_fullsized":"2024-04-02T13:35:09.818970+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:03:42.006013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820256+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:09.820256+0000","last_peered":"2024-04-02T13:35:09.820256+0000","last_clean":"2024-04-02T13:35:09.820256+0000","last_became_active":"2024-04-02T13:32:50.530758+0000","last_became_peered":"2024-04-02T13:32:50.530758+0000","last_unstale":"2024-04-02T13:35:09.820256+0000","last_undegraded":"2024-04-02T13:35:09.820256+0000","last_fullsized":"2024-04-02T13:35:09.820256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:00.039551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063497+0000","last_change":"2024-04-02T13:32:50.537409+0000","last_active":"2024-04-02T13:35:10.063497+0000","last_peered":"2024-04-02T13:35:10.063497+0000","last_clean":"2024-04-02T13:35:10.063497+0000","last_became_active":"2024-04-02T13:32:50.537113+0000","last_became_peered":"2024-04-02T13:32:50.537113+0000","last_unstale":"2024-04-02T13:35:10.063497+0000","last_undegraded":"2024-04-02T13:35:10.063497+0000","last_fullsized":"2024-04-02T13:35:10.063497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:26:42.175244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.817945+0000","last_change":"2024-04-02T13:32:50.534090+0000","last_active":"2024-04-02T13:35:09.817945+0000","last_peered":"2024-04-02T13:35:09.817945+0000","last_clean":"2024-04-02T13:35:09.817945+0000","last_became_active":"2024-04-02T13:32:50.533807+0000","last_became_peered":"2024-04-02T13:32:50.533807+0000","last_unstale":"2024-04-02T13:35:09.817945+0000","last_undegraded":"2024-04-02T13:35:09.817945+0000","last_fullsized":"2024-04-02T13:35:09.817945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:26:10.769879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068204+0000","last_change":"2024-04-02T13:32:50.531268+0000","last_active":"2024-04-02T13:35:10.068204+0000","last_peered":"2024-04-02T13:35:10.068204+0000","last_clean":"2024-04-02T13:35:10.068204+0000","last_became_active":"2024-04-02T13:32:50.530965+0000","last_became_peered":"2024-04-02T13:32:50.530965+0000","last_unstale":"2024-04-02T13:35:10.068204+0000","last_undegraded":"2024-04-02T13:35:10.068204+0000","last_fullsized":"2024-04-02T13:35:10.068204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:05:20.283808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820407+0000","last_change":"2024-04-02T13:32:50.527612+0000","last_active":"2024-04-02T13:35:09.820407+0000","last_peered":"2024-04-02T13:35:09.820407+0000","last_clean":"2024-04-02T13:35:09.820407+0000","last_became_active":"2024-04-02T13:32:50.525390+0000","last_became_peered":"2024-04-02T13:32:50.525390+0000","last_unstale":"2024-04-02T13:35:09.820407+0000","last_undegraded":"2024-04-02T13:35:09.820407+0000","last_fullsized":"2024-04-02T13:35:09.820407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:42:32.093625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063961+0000","last_change":"2024-04-02T13:32:50.533940+0000","last_active":"2024-04-02T13:35:10.063961+0000","last_peered":"2024-04-02T13:35:10.063961+0000","last_clean":"2024-04-02T13:35:10.063961+0000","last_became_active":"2024-04-02T13:32:50.533577+0000","last_became_peered":"2024-04-02T13:32:50.533577+0000","last_unstale":"2024-04-02T13:35:10.063961+0000","last_undegraded":"2024-04-02T13:35:10.063961+0000","last_fullsized":"2024-04-02T13:35:10.063961+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:04:34.126222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919462+0000","last_change":"2024-04-02T13:33:55.555987+0000","last_active":"2024-04-02T13:35:09.919462+0000","last_peered":"2024-04-02T13:35:09.919462+0000","last_clean":"2024-04-02T13:35:09.919462+0000","last_became_active":"2024-04-02T13:33:55.555623+0000","last_became_peered":"2024-04-02T13:33:55.555623+0000","last_unstale":"2024-04-02T13:35:09.919462+0000","last_undegraded":"2024-04-02T13:35:09.919462+0000","last_fullsized":"2024-04-02T13:35:09.919462+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:32.669173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062249+0000","last_change":"2024-04-02T13:32:50.529558+0000","last_active":"2024-04-02T13:35:10.062249+0000","last_peered":"2024-04-02T13:35:10.062249+0000","last_clean":"2024-04-02T13:35:10.062249+0000","last_became_active":"2024-04-02T13:32:50.528965+0000","last_became_peered":"2024-04-02T13:32:50.528965+0000","last_unstale":"2024-04-02T13:35:10.062249+0000","last_undegraded":"2024-04-02T13:35:10.062249+0000","last_fullsized":"2024-04-02T13:35:10.062249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:14:09.264854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064625+0000","last_change":"2024-04-02T13:32:50.531192+0000","last_active":"2024-04-02T13:35:10.064625+0000","last_peered":"2024-04-02T13:35:10.064625+0000","last_clean":"2024-04-02T13:35:10.064625+0000","last_became_active":"2024-04-02T13:32:50.530879+0000","last_became_peered":"2024-04-02T13:32:50.530879+0000","last_unstale":"2024-04-02T13:35:10.064625+0000","last_undegraded":"2024-04-02T13:35:10.064625+0000","last_fullsized":"2024-04-02T13:35:10.064625+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:55:58.537930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064990+0000","last_change":"2024-04-02T13:32:50.537565+0000","last_active":"2024-04-02T13:35:10.064990+0000","last_peered":"2024-04-02T13:35:10.064990+0000","last_clean":"2024-04-02T13:35:10.064990+0000","last_became_active":"2024-04-02T13:32:50.537204+0000","last_became_peered":"2024-04-02T13:32:50.537204+0000","last_unstale":"2024-04-02T13:35:10.064990+0000","last_undegraded":"2024-04-02T13:35:10.064990+0000","last_fullsized":"2024-04-02T13:35:10.064990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:29:37.364712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067588+0000","last_change":"2024-04-02T13:32:50.525705+0000","last_active":"2024-04-02T13:35:10.067588+0000","last_peered":"2024-04-02T13:35:10.067588+0000","last_clean":"2024-04-02T13:35:10.067588+0000","last_became_active":"2024-04-02T13:32:50.525434+0000","last_became_peered":"2024-04-02T13:32:50.525434+0000","last_unstale":"2024-04-02T13:35:10.067588+0000","last_undegraded":"2024-04-02T13:35:10.067588+0000","last_fullsized":"2024-04-02T13:35:10.067588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:39:12.814271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070971+0000","last_change":"2024-04-02T13:32:50.526800+0000","last_active":"2024-04-02T13:35:10.070971+0000","last_peered":"2024-04-02T13:35:10.070971+0000","last_clean":"2024-04-02T13:35:10.070971+0000","last_became_active":"2024-04-02T13:32:50.523244+0000","last_became_peered":"2024-04-02T13:32:50.523244+0000","last_unstale":"2024-04-02T13:35:10.070971+0000","last_undegraded":"2024-04-02T13:35:10.070971+0000","last_fullsized":"2024-04-02T13:35:10.070971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:08:45.685138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919799+0000","last_change":"2024-04-02T13:32:50.529319+0000","last_active":"2024-04-02T13:35:09.919799+0000","last_peered":"2024-04-02T13:35:09.919799+0000","last_clean":"2024-04-02T13:35:09.919799+0000","last_became_active":"2024-04-02T13:32:50.528985+0000","last_became_peered":"2024-04-02T13:32:50.528985+0000","last_unstale":"2024-04-02T13:35:09.919799+0000","last_undegraded":"2024-04-02T13:35:09.919799+0000","last_fullsized":"2024-04-02T13:35:09.919799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:28:51.248808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063291+0000","last_change":"2024-04-02T13:32:50.537817+0000","last_active":"2024-04-02T13:35:10.063291+0000","last_peered":"2024-04-02T13:35:10.063291+0000","last_clean":"2024-04-02T13:35:10.063291+0000","last_became_active":"2024-04-02T13:32:50.537476+0000","last_became_peered":"2024-04-02T13:32:50.537476+0000","last_unstale":"2024-04-02T13:35:10.063291+0000","last_undegraded":"2024-04-02T13:35:10.063291+0000","last_fullsized":"2024-04-02T13:35:10.063291+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:21:31.184632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.565907+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.565546+0000","last_became_peered":"2024-04-02T13:32:54.565546+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:21:43.534260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"95'14","reported_seq":167,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067551+0000","last_change":"2024-04-02T13:32:49.527154+0000","last_active":"2024-04-02T13:35:10.067551+0000","last_peered":"2024-04-02T13:35:10.067551+0000","last_clean":"2024-04-02T13:35:10.067551+0000","last_became_active":"2024-04-02T13:32:49.526674+0000","last_became_peered":"2024-04-02T13:32:49.526674+0000","last_unstale":"2024-04-02T13:35:10.067551+0000","last_undegraded":"2024-04-02T13:35:10.067551+0000","last_fullsized":"2024-04-02T13:35:10.067551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:08.567098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818258+0000","last_change":"2024-04-02T13:32:50.536481+0000","last_active":"2024-04-02T13:35:09.818258+0000","last_peered":"2024-04-02T13:35:09.818258+0000","last_clean":"2024-04-02T13:35:09.818258+0000","last_became_active":"2024-04-02T13:32:50.535577+0000","last_became_peered":"2024-04-02T13:32:50.535577+0000","last_unstale":"2024-04-02T13:35:09.818258+0000","last_undegraded":"2024-04-02T13:35:09.818258+0000","last_fullsized":"2024-04-02T13:35:09.818258+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:03:42.307880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061716+0000","last_change":"2024-04-02T13:32:54.570006+0000","last_active":"2024-04-02T13:35:10.061716+0000","last_peered":"2024-04-02T13:35:10.061716+0000","last_clean":"2024-04-02T13:35:10.061716+0000","last_became_active":"2024-04-02T13:32:54.569345+0000","last_became_peered":"2024-04-02T13:32:54.569345+0000","last_unstale":"2024-04-02T13:35:10.061716+0000","last_undegraded":"2024-04-02T13:35:10.061716+0000","last_fullsized":"2024-04-02T13:35:10.061716+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:44:12.811098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818857+0000","last_change":"2024-04-02T13:32:49.530862+0000","last_active":"2024-04-02T13:35:09.818857+0000","last_peered":"2024-04-02T13:35:09.818857+0000","last_clean":"2024-04-02T13:35:09.818857+0000","last_became_active":"2024-04-02T13:32:49.530083+0000","last_became_peered":"2024-04-02T13:32:49.530083+0000","last_unstale":"2024-04-02T13:35:09.818857+0000","last_undegraded":"2024-04-02T13:35:09.818857+0000","last_fullsized":"2024-04-02T13:35:09.818857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:21:46.628050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065791+0000","last_change":"2024-04-02T13:32:50.532864+0000","last_active":"2024-04-02T13:35:10.065791+0000","last_peered":"2024-04-02T13:35:10.065791+0000","last_clean":"2024-04-02T13:35:10.065791+0000","last_became_active":"2024-04-02T13:32:50.532547+0000","last_became_peered":"2024-04-02T13:32:50.532547+0000","last_unstale":"2024-04-02T13:35:10.065791+0000","last_undegraded":"2024-04-02T13:35:10.065791+0000","last_fullsized":"2024-04-02T13:35:10.065791+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:58:42.794322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061979+0000","last_change":"2024-04-02T13:32:54.571091+0000","last_active":"2024-04-02T13:35:10.061979+0000","last_peered":"2024-04-02T13:35:10.061979+0000","last_clean":"2024-04-02T13:35:10.061979+0000","last_became_active":"2024-04-02T13:32:54.570627+0000","last_became_peered":"2024-04-02T13:32:54.570627+0000","last_unstale":"2024-04-02T13:35:10.061979+0000","last_undegraded":"2024-04-02T13:35:10.061979+0000","last_fullsized":"2024-04-02T13:35:10.061979+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:05:57.815262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'12","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819095+0000","last_change":"2024-04-02T13:32:49.528022+0000","last_active":"2024-04-02T13:35:09.819095+0000","last_peered":"2024-04-02T13:35:09.819095+0000","last_clean":"2024-04-02T13:35:09.819095+0000","last_became_active":"2024-04-02T13:32:49.527720+0000","last_became_peered":"2024-04-02T13:32:49.527720+0000","last_unstale":"2024-04-02T13:35:09.819095+0000","last_undegraded":"2024-04-02T13:35:09.819095+0000","last_fullsized":"2024-04-02T13:35:09.819095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:59:31.227993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067884+0000","last_change":"2024-04-02T13:32:50.526656+0000","last_active":"2024-04-02T13:35:10.067884+0000","last_peered":"2024-04-02T13:35:10.067884+0000","last_clean":"2024-04-02T13:35:10.067884+0000","last_became_active":"2024-04-02T13:32:50.526347+0000","last_became_peered":"2024-04-02T13:32:50.526347+0000","last_unstale":"2024-04-02T13:35:10.067884+0000","last_undegraded":"2024-04-02T13:35:10.067884+0000","last_fullsized":"2024-04-02T13:35:10.067884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:54:49.833142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061881+0000","last_change":"2024-04-02T13:32:54.569970+0000","last_active":"2024-04-02T13:35:10.061881+0000","last_peered":"2024-04-02T13:35:10.061881+0000","last_clean":"2024-04-02T13:35:10.061881+0000","last_became_active":"2024-04-02T13:32:54.569490+0000","last_became_peered":"2024-04-02T13:32:54.569490+0000","last_unstale":"2024-04-02T13:35:10.061881+0000","last_undegraded":"2024-04-02T13:35:10.061881+0000","last_fullsized":"2024-04-02T13:35:10.061881+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:50:36.503265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819043+0000","last_change":"2024-04-02T13:32:49.524716+0000","last_active":"2024-04-02T13:35:09.819043+0000","last_peered":"2024-04-02T13:35:09.819043+0000","last_clean":"2024-04-02T13:35:09.819043+0000","last_became_active":"2024-04-02T13:32:49.524428+0000","last_became_peered":"2024-04-02T13:32:49.524428+0000","last_unstale":"2024-04-02T13:35:09.819043+0000","last_undegraded":"2024-04-02T13:35:09.819043+0000","last_fullsized":"2024-04-02T13:35:09.819043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:01:33.624106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919412+0000","last_change":"2024-04-02T13:33:55.559231+0000","last_active":"2024-04-02T13:35:09.919412+0000","last_peered":"2024-04-02T13:35:09.919412+0000","last_clean":"2024-04-02T13:35:09.919412+0000","last_became_active":"2024-04-02T13:33:55.558856+0000","last_became_peered":"2024-04-02T13:33:55.558856+0000","last_unstale":"2024-04-02T13:35:09.919412+0000","last_undegraded":"2024-04-02T13:35:09.919412+0000","last_fullsized":"2024-04-02T13:35:09.919412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:12:47.406198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063082+0000","last_change":"2024-04-02T13:34:56.175049+0000","last_active":"2024-04-02T13:35:10.063082+0000","last_peered":"2024-04-02T13:35:10.063082+0000","last_clean":"2024-04-02T13:35:10.063082+0000","last_became_active":"2024-04-02T13:34:56.174477+0000","last_became_peered":"2024-04-02T13:34:56.174477+0000","last_unstale":"2024-04-02T13:35:10.063082+0000","last_undegraded":"2024-04-02T13:35:10.063082+0000","last_fullsized":"2024-04-02T13:35:10.063082+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:33:30.726406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,7,11],"acting":[2,8,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":162,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315244+0000","last_change":"2024-04-02T13:32:49.529198+0000","last_active":"2024-04-02T13:35:10.315244+0000","last_peered":"2024-04-02T13:35:10.315244+0000","last_clean":"2024-04-02T13:35:10.315244+0000","last_became_active":"2024-04-02T13:32:49.528912+0000","last_became_peered":"2024-04-02T13:32:49.528912+0000","last_unstale":"2024-04-02T13:35:10.315244+0000","last_undegraded":"2024-04-02T13:35:10.315244+0000","last_fullsized":"2024-04-02T13:35:10.315244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:45:14.273424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1404,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818973+0000","last_change":"2024-04-02T13:32:50.529837+0000","last_active":"2024-04-02T13:35:09.818973+0000","last_peered":"2024-04-02T13:35:09.818973+0000","last_clean":"2024-04-02T13:35:09.818973+0000","last_became_active":"2024-04-02T13:32:50.529536+0000","last_became_peered":"2024-04-02T13:32:50.529536+0000","last_unstale":"2024-04-02T13:35:09.818973+0000","last_undegraded":"2024-04-02T13:35:09.818973+0000","last_fullsized":"2024-04-02T13:35:09.818973+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:21:38.576401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063063+0000","last_change":"2024-04-02T13:32:54.559930+0000","last_active":"2024-04-02T13:35:10.063063+0000","last_peered":"2024-04-02T13:35:10.063063+0000","last_clean":"2024-04-02T13:35:10.063063+0000","last_became_active":"2024-04-02T13:32:54.559529+0000","last_became_peered":"2024-04-02T13:32:54.559529+0000","last_unstale":"2024-04-02T13:35:10.063063+0000","last_undegraded":"2024-04-02T13:35:10.063063+0000","last_fullsized":"2024-04-02T13:35:10.063063+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.708026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'7","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820123+0000","last_change":"2024-04-02T13:33:55.565630+0000","last_active":"2024-04-02T13:35:09.820123+0000","last_peered":"2024-04-02T13:35:09.820123+0000","last_clean":"2024-04-02T13:35:09.820123+0000","last_became_active":"2024-04-02T13:33:55.559239+0000","last_became_peered":"2024-04-02T13:33:55.559239+0000","last_unstale":"2024-04-02T13:35:09.820123+0000","last_undegraded":"2024-04-02T13:35:09.820123+0000","last_fullsized":"2024-04-02T13:35:09.820123+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:29:46.161608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928087+0000","last_change":"2024-04-02T13:32:50.545453+0000","last_active":"2024-04-02T13:35:09.928087+0000","last_peered":"2024-04-02T13:35:09.928087+0000","last_clean":"2024-04-02T13:35:09.928087+0000","last_became_active":"2024-04-02T13:32:50.545136+0000","last_became_peered":"2024-04-02T13:32:50.545136+0000","last_unstale":"2024-04-02T13:35:09.928087+0000","last_undegraded":"2024-04-02T13:35:09.928087+0000","last_fullsized":"2024-04-02T13:35:09.928087+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:35:15.187905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070614+0000","last_change":"2024-04-02T13:33:55.558751+0000","last_active":"2024-04-02T13:35:10.070614+0000","last_peered":"2024-04-02T13:35:10.070614+0000","last_clean":"2024-04-02T13:35:10.070614+0000","last_became_active":"2024-04-02T13:33:55.558278+0000","last_became_peered":"2024-04-02T13:33:55.558278+0000","last_unstale":"2024-04-02T13:35:10.070614+0000","last_undegraded":"2024-04-02T13:35:10.070614+0000","last_fullsized":"2024-04-02T13:35:10.070614+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:15.164871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062024+0000","last_change":"2024-04-02T13:32:49.516979+0000","last_active":"2024-04-02T13:35:10.062024+0000","last_peered":"2024-04-02T13:35:10.062024+0000","last_clean":"2024-04-02T13:35:10.062024+0000","last_became_active":"2024-04-02T13:32:49.516620+0000","last_became_peered":"2024-04-02T13:32:49.516620+0000","last_unstale":"2024-04-02T13:35:10.062024+0000","last_undegraded":"2024-04-02T13:35:10.062024+0000","last_fullsized":"2024-04-02T13:35:10.062024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:11:22.081316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818108+0000","last_change":"2024-04-02T13:32:50.530321+0000","last_active":"2024-04-02T13:35:09.818108+0000","last_peered":"2024-04-02T13:35:09.818108+0000","last_clean":"2024-04-02T13:35:09.818108+0000","last_became_active":"2024-04-02T13:32:50.530023+0000","last_became_peered":"2024-04-02T13:32:50.530023+0000","last_unstale":"2024-04-02T13:35:09.818108+0000","last_undegraded":"2024-04-02T13:35:09.818108+0000","last_fullsized":"2024-04-02T13:35:09.818108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:15:18.191059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065607+0000","last_change":"2024-04-02T13:32:54.564818+0000","last_active":"2024-04-02T13:35:10.065607+0000","last_peered":"2024-04-02T13:35:10.065607+0000","last_clean":"2024-04-02T13:35:10.065607+0000","last_became_active":"2024-04-02T13:32:54.564445+0000","last_became_peered":"2024-04-02T13:32:54.564445+0000","last_unstale":"2024-04-02T13:35:10.065607+0000","last_undegraded":"2024-04-02T13:35:10.065607+0000","last_fullsized":"2024-04-02T13:35:10.065607+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:22:13.505049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"90'3","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927765+0000","last_change":"2024-04-02T13:32:49.526857+0000","last_active":"2024-04-02T13:35:09.927765+0000","last_peered":"2024-04-02T13:35:09.927765+0000","last_clean":"2024-04-02T13:35:09.927765+0000","last_became_active":"2024-04-02T13:32:49.525752+0000","last_became_peered":"2024-04-02T13:32:49.525752+0000","last_unstale":"2024-04-02T13:35:09.927765+0000","last_undegraded":"2024-04-02T13:35:09.927765+0000","last_fullsized":"2024-04-02T13:35:09.927765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:26:56.861300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919045+0000","last_change":"2024-04-02T13:32:50.531155+0000","last_active":"2024-04-02T13:35:09.919045+0000","last_peered":"2024-04-02T13:35:09.919045+0000","last_clean":"2024-04-02T13:35:09.919045+0000","last_became_active":"2024-04-02T13:32:50.530832+0000","last_became_peered":"2024-04-02T13:32:50.530832+0000","last_unstale":"2024-04-02T13:35:09.919045+0000","last_undegraded":"2024-04-02T13:35:09.919045+0000","last_fullsized":"2024-04-02T13:35:09.919045+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:21:43.615534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818007+0000","last_change":"2024-04-02T13:32:54.560107+0000","last_active":"2024-04-02T13:35:09.818007+0000","last_peered":"2024-04-02T13:35:09.818007+0000","last_clean":"2024-04-02T13:35:09.818007+0000","last_became_active":"2024-04-02T13:32:54.559724+0000","last_became_peered":"2024-04-02T13:32:54.559724+0000","last_unstale":"2024-04-02T13:35:09.818007+0000","last_undegraded":"2024-04-02T13:35:09.818007+0000","last_fullsized":"2024-04-02T13:35:09.818007+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:39.630793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"95'6","reported_seq":278,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.314531+0000","last_change":"2024-04-02T13:32:49.521290+0000","last_active":"2024-04-02T13:35:10.314531+0000","last_peered":"2024-04-02T13:35:10.314531+0000","last_clean":"2024-04-02T13:35:10.314531+0000","last_became_active":"2024-04-02T13:32:49.520662+0000","last_became_peered":"2024-04-02T13:32:49.520662+0000","last_unstale":"2024-04-02T13:35:10.314531+0000","last_undegraded":"2024-04-02T13:35:10.314531+0000","last_fullsized":"2024-04-02T13:35:10.314531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:30:40.545239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":240,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068022+0000","last_change":"2024-04-02T13:32:50.531073+0000","last_active":"2024-04-02T13:35:10.068022+0000","last_peered":"2024-04-02T13:35:10.068022+0000","last_clean":"2024-04-02T13:35:10.068022+0000","last_became_active":"2024-04-02T13:32:50.530769+0000","last_became_peered":"2024-04-02T13:32:50.530769+0000","last_unstale":"2024-04-02T13:35:10.068022+0000","last_undegraded":"2024-04-02T13:35:10.068022+0000","last_fullsized":"2024-04-02T13:35:10.068022+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:12.680329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.918900+0000","last_change":"2024-04-02T13:32:54.561061+0000","last_active":"2024-04-02T13:35:09.918900+0000","last_peered":"2024-04-02T13:35:09.918900+0000","last_clean":"2024-04-02T13:35:09.918900+0000","last_became_active":"2024-04-02T13:32:54.560653+0000","last_became_peered":"2024-04-02T13:32:54.560653+0000","last_unstale":"2024-04-02T13:35:09.918900+0000","last_undegraded":"2024-04-02T13:35:09.918900+0000","last_fullsized":"2024-04-02T13:35:09.918900+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:09:49.455750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'4","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063718+0000","last_change":"2024-04-02T13:32:49.523700+0000","last_active":"2024-04-02T13:35:10.063718+0000","last_peered":"2024-04-02T13:35:10.063718+0000","last_clean":"2024-04-02T13:35:10.063718+0000","last_became_active":"2024-04-02T13:32:49.523218+0000","last_became_peered":"2024-04-02T13:32:49.523218+0000","last_unstale":"2024-04-02T13:35:10.063718+0000","last_undegraded":"2024-04-02T13:35:10.063718+0000","last_fullsized":"2024-04-02T13:35:10.063718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:23:26.742731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065934+0000","last_change":"2024-04-02T13:32:50.535858+0000","last_active":"2024-04-02T13:35:10.065934+0000","last_peered":"2024-04-02T13:35:10.065934+0000","last_clean":"2024-04-02T13:35:10.065934+0000","last_became_active":"2024-04-02T13:32:50.535170+0000","last_became_peered":"2024-04-02T13:32:50.535170+0000","last_unstale":"2024-04-02T13:35:10.065934+0000","last_undegraded":"2024-04-02T13:35:10.065934+0000","last_fullsized":"2024-04-02T13:35:10.065934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:57:51.274857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062853+0000","last_change":"2024-04-02T13:32:54.571857+0000","last_active":"2024-04-02T13:35:10.062853+0000","last_peered":"2024-04-02T13:35:10.062853+0000","last_clean":"2024-04-02T13:35:10.062853+0000","last_became_active":"2024-04-02T13:32:54.570887+0000","last_became_peered":"2024-04-02T13:32:54.570887+0000","last_unstale":"2024-04-02T13:35:10.062853+0000","last_undegraded":"2024-04-02T13:35:10.062853+0000","last_fullsized":"2024-04-02T13:35:10.062853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:48:29.290935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928052+0000","last_change":"2024-04-02T13:32:49.522978+0000","last_active":"2024-04-02T13:35:09.928052+0000","last_peered":"2024-04-02T13:35:09.928052+0000","last_clean":"2024-04-02T13:35:09.928052+0000","last_became_active":"2024-04-02T13:32:49.522630+0000","last_became_peered":"2024-04-02T13:32:49.522630+0000","last_unstale":"2024-04-02T13:35:09.928052+0000","last_undegraded":"2024-04-02T13:35:09.928052+0000","last_fullsized":"2024-04-02T13:35:09.928052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:13:43.257700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.068046+0000","last_change":"2024-04-02T13:32:50.525187+0000","last_active":"2024-04-02T13:35:10.068046+0000","last_peered":"2024-04-02T13:35:10.068046+0000","last_clean":"2024-04-02T13:35:10.068046+0000","last_became_active":"2024-04-02T13:32:50.524892+0000","last_became_peered":"2024-04-02T13:32:50.524892+0000","last_unstale":"2024-04-02T13:35:10.068046+0000","last_undegraded":"2024-04-02T13:35:10.068046+0000","last_fullsized":"2024-04-02T13:35:10.068046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:23:51.273152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819835+0000","last_change":"2024-04-02T13:34:56.181716+0000","last_active":"2024-04-02T13:35:09.819835+0000","last_peered":"2024-04-02T13:35:09.819835+0000","last_clean":"2024-04-02T13:35:09.819835+0000","last_became_active":"2024-04-02T13:34:56.181233+0000","last_became_peered":"2024-04-02T13:34:56.181233+0000","last_unstale":"2024-04-02T13:35:09.819835+0000","last_undegraded":"2024-04-02T13:35:09.819835+0000","last_fullsized":"2024-04-02T13:35:09.819835+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:24:55.347041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"95'43","reported_seq":326,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315476+0000","last_change":"2024-04-02T13:32:49.520261+0000","last_active":"2024-04-02T13:35:10.315476+0000","last_peered":"2024-04-02T13:35:10.315476+0000","last_clean":"2024-04-02T13:35:10.315476+0000","last_became_active":"2024-04-02T13:32:49.519900+0000","last_became_peered":"2024-04-02T13:32:49.519900+0000","last_unstale":"2024-04-02T13:35:10.315476+0000","last_undegraded":"2024-04-02T13:35:10.315476+0000","last_fullsized":"2024-04-02T13:35:10.315476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:19:30.150714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69045,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":250,"num_read_kb":193,"num_write":45,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819358+0000","last_change":"2024-04-02T13:32:50.538227+0000","last_active":"2024-04-02T13:35:09.819358+0000","last_peered":"2024-04-02T13:35:09.819358+0000","last_clean":"2024-04-02T13:35:09.819358+0000","last_became_active":"2024-04-02T13:32:50.536667+0000","last_became_peered":"2024-04-02T13:32:50.536667+0000","last_unstale":"2024-04-02T13:35:09.819358+0000","last_undegraded":"2024-04-02T13:35:09.819358+0000","last_fullsized":"2024-04-02T13:35:09.819358+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:00:49.454582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928355+0000","last_change":"2024-04-02T13:32:54.569205+0000","last_active":"2024-04-02T13:35:09.928355+0000","last_peered":"2024-04-02T13:35:09.928355+0000","last_clean":"2024-04-02T13:35:09.928355+0000","last_became_active":"2024-04-02T13:32:54.568737+0000","last_became_peered":"2024-04-02T13:32:54.568737+0000","last_unstale":"2024-04-02T13:35:09.928355+0000","last_undegraded":"2024-04-02T13:35:09.928355+0000","last_fullsized":"2024-04-02T13:35:09.928355+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:57:53.641854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062970+0000","last_change":"2024-04-02T13:32:49.524454+0000","last_active":"2024-04-02T13:35:10.062970+0000","last_peered":"2024-04-02T13:35:10.062970+0000","last_clean":"2024-04-02T13:35:10.062970+0000","last_became_active":"2024-04-02T13:32:49.524138+0000","last_became_peered":"2024-04-02T13:32:49.524138+0000","last_unstale":"2024-04-02T13:35:10.062970+0000","last_undegraded":"2024-04-02T13:35:10.062970+0000","last_fullsized":"2024-04-02T13:35:10.062970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:47:56.416786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062183+0000","last_change":"2024-04-02T13:32:50.532643+0000","last_active":"2024-04-02T13:35:10.062183+0000","last_peered":"2024-04-02T13:35:10.062183+0000","last_clean":"2024-04-02T13:35:10.062183+0000","last_became_active":"2024-04-02T13:32:50.532324+0000","last_became_peered":"2024-04-02T13:32:50.532324+0000","last_unstale":"2024-04-02T13:35:10.062183+0000","last_undegraded":"2024-04-02T13:35:10.062183+0000","last_fullsized":"2024-04-02T13:35:10.062183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:58:01.997505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070601+0000","last_change":"2024-04-02T13:32:54.568020+0000","last_active":"2024-04-02T13:35:10.070601+0000","last_peered":"2024-04-02T13:35:10.070601+0000","last_clean":"2024-04-02T13:35:10.070601+0000","last_became_active":"2024-04-02T13:32:54.567557+0000","last_became_peered":"2024-04-02T13:32:54.567557+0000","last_unstale":"2024-04-02T13:35:10.070601+0000","last_undegraded":"2024-04-02T13:35:10.070601+0000","last_fullsized":"2024-04-02T13:35:10.070601+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:36:05.605531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"90'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928018+0000","last_change":"2024-04-02T13:32:49.529049+0000","last_active":"2024-04-02T13:35:09.928018+0000","last_peered":"2024-04-02T13:35:09.928018+0000","last_clean":"2024-04-02T13:35:09.928018+0000","last_became_active":"2024-04-02T13:32:49.528735+0000","last_became_peered":"2024-04-02T13:32:49.528735+0000","last_unstale":"2024-04-02T13:35:09.928018+0000","last_undegraded":"2024-04-02T13:35:09.928018+0000","last_fullsized":"2024-04-02T13:35:09.928018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:32:19.148769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063434+0000","last_change":"2024-04-02T13:32:50.528219+0000","last_active":"2024-04-02T13:35:10.063434+0000","last_peered":"2024-04-02T13:35:10.063434+0000","last_clean":"2024-04-02T13:35:10.063434+0000","last_became_active":"2024-04-02T13:32:50.527903+0000","last_became_peered":"2024-04-02T13:32:50.527903+0000","last_unstale":"2024-04-02T13:35:10.063434+0000","last_undegraded":"2024-04-02T13:35:10.063434+0000","last_fullsized":"2024-04-02T13:35:10.063434+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:37.593557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919844+0000","last_change":"2024-04-02T13:32:54.565760+0000","last_active":"2024-04-02T13:35:09.919844+0000","last_peered":"2024-04-02T13:35:09.919844+0000","last_clean":"2024-04-02T13:35:09.919844+0000","last_became_active":"2024-04-02T13:32:54.564629+0000","last_became_peered":"2024-04-02T13:32:54.564629+0000","last_unstale":"2024-04-02T13:35:09.919844+0000","last_undegraded":"2024-04-02T13:35:09.919844+0000","last_fullsized":"2024-04-02T13:35:09.919844+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:13:46.842955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":163,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315744+0000","last_change":"2024-04-02T13:32:49.517391+0000","last_active":"2024-04-02T13:35:10.315744+0000","last_peered":"2024-04-02T13:35:10.315744+0000","last_clean":"2024-04-02T13:35:10.315744+0000","last_became_active":"2024-04-02T13:32:49.517071+0000","last_became_peered":"2024-04-02T13:32:49.517071+0000","last_unstale":"2024-04-02T13:35:10.315744+0000","last_undegraded":"2024-04-02T13:35:10.315744+0000","last_fullsized":"2024-04-02T13:35:10.315744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:02:39.852245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063690+0000","last_change":"2024-04-02T13:32:50.528669+0000","last_active":"2024-04-02T13:35:10.063690+0000","last_peered":"2024-04-02T13:35:10.063690+0000","last_clean":"2024-04-02T13:35:10.063690+0000","last_became_active":"2024-04-02T13:32:50.528377+0000","last_became_peered":"2024-04-02T13:32:50.528377+0000","last_unstale":"2024-04-02T13:35:10.063690+0000","last_undegraded":"2024-04-02T13:35:10.063690+0000","last_fullsized":"2024-04-02T13:35:10.063690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:25:41.849948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062778+0000","last_change":"2024-04-02T13:32:54.572006+0000","last_active":"2024-04-02T13:35:10.062778+0000","last_peered":"2024-04-02T13:35:10.062778+0000","last_clean":"2024-04-02T13:35:10.062778+0000","last_became_active":"2024-04-02T13:32:54.571635+0000","last_became_peered":"2024-04-02T13:32:54.571635+0000","last_unstale":"2024-04-02T13:35:10.062778+0000","last_undegraded":"2024-04-02T13:35:10.062778+0000","last_fullsized":"2024-04-02T13:35:10.062778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:23:10.747402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'17","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070705+0000","last_change":"2024-04-02T13:33:55.574777+0000","last_active":"2024-04-02T13:35:10.070705+0000","last_peered":"2024-04-02T13:35:10.070705+0000","last_clean":"2024-04-02T13:35:10.070705+0000","last_became_active":"2024-04-02T13:33:55.561040+0000","last_became_peered":"2024-04-02T13:33:55.561040+0000","last_unstale":"2024-04-02T13:35:10.070705+0000","last_undegraded":"2024-04-02T13:35:10.070705+0000","last_fullsized":"2024-04-02T13:35:10.070705+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:07:23.184540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927913+0000","last_change":"2024-04-02T13:33:55.560045+0000","last_active":"2024-04-02T13:35:09.927913+0000","last_peered":"2024-04-02T13:35:09.927913+0000","last_clean":"2024-04-02T13:35:09.927913+0000","last_became_active":"2024-04-02T13:33:55.559659+0000","last_became_peered":"2024-04-02T13:33:55.559659+0000","last_unstale":"2024-04-02T13:35:09.927913+0000","last_undegraded":"2024-04-02T13:35:09.927913+0000","last_fullsized":"2024-04-02T13:35:09.927913+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:10:42.028143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927883+0000","last_change":"2024-04-02T13:32:54.565423+0000","last_active":"2024-04-02T13:35:09.927883+0000","last_peered":"2024-04-02T13:35:09.927883+0000","last_clean":"2024-04-02T13:35:09.927883+0000","last_became_active":"2024-04-02T13:32:54.565003+0000","last_became_peered":"2024-04-02T13:32:54.565003+0000","last_unstale":"2024-04-02T13:35:09.927883+0000","last_undegraded":"2024-04-02T13:35:09.927883+0000","last_fullsized":"2024-04-02T13:35:09.927883+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:46:00.482197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819290+0000","last_change":"2024-04-02T13:32:49.524687+0000","last_active":"2024-04-02T13:35:09.819290+0000","last_peered":"2024-04-02T13:35:09.819290+0000","last_clean":"2024-04-02T13:35:09.819290+0000","last_became_active":"2024-04-02T13:32:49.524380+0000","last_became_peered":"2024-04-02T13:32:49.524380+0000","last_unstale":"2024-04-02T13:35:09.819290+0000","last_undegraded":"2024-04-02T13:35:09.819290+0000","last_fullsized":"2024-04-02T13:35:09.819290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:45:29.833819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070776+0000","last_change":"2024-04-02T13:32:50.533700+0000","last_active":"2024-04-02T13:35:10.070776+0000","last_peered":"2024-04-02T13:35:10.070776+0000","last_clean":"2024-04-02T13:35:10.070776+0000","last_became_active":"2024-04-02T13:32:50.532209+0000","last_became_peered":"2024-04-02T13:32:50.532209+0000","last_unstale":"2024-04-02T13:35:10.070776+0000","last_undegraded":"2024-04-02T13:35:10.070776+0000","last_fullsized":"2024-04-02T13:35:10.070776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:20:53.611140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818918+0000","last_change":"2024-04-02T13:32:54.560195+0000","last_active":"2024-04-02T13:35:09.818918+0000","last_peered":"2024-04-02T13:35:09.818918+0000","last_clean":"2024-04-02T13:35:09.818918+0000","last_became_active":"2024-04-02T13:32:54.559800+0000","last_became_peered":"2024-04-02T13:32:54.559800+0000","last_unstale":"2024-04-02T13:35:09.818918+0000","last_undegraded":"2024-04-02T13:35:09.818918+0000","last_fullsized":"2024-04-02T13:35:09.818918+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:14:40.816410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"90'1","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919605+0000","last_change":"2024-04-02T13:32:49.527956+0000","last_active":"2024-04-02T13:35:09.919605+0000","last_peered":"2024-04-02T13:35:09.919605+0000","last_clean":"2024-04-02T13:35:09.919605+0000","last_became_active":"2024-04-02T13:32:49.527618+0000","last_became_peered":"2024-04-02T13:32:49.527618+0000","last_unstale":"2024-04-02T13:35:09.919605+0000","last_undegraded":"2024-04-02T13:35:09.919605+0000","last_fullsized":"2024-04-02T13:35:09.919605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:58:10.980526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820178+0000","last_change":"2024-04-02T13:32:50.522253+0000","last_active":"2024-04-02T13:35:09.820178+0000","last_peered":"2024-04-02T13:35:09.820178+0000","last_clean":"2024-04-02T13:35:09.820178+0000","last_became_active":"2024-04-02T13:32:50.521932+0000","last_became_peered":"2024-04-02T13:32:50.521932+0000","last_unstale":"2024-04-02T13:35:09.820178+0000","last_undegraded":"2024-04-02T13:35:09.820178+0000","last_fullsized":"2024-04-02T13:35:09.820178+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:35:41.100223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062428+0000","last_change":"2024-04-02T13:32:54.563695+0000","last_active":"2024-04-02T13:35:10.062428+0000","last_peered":"2024-04-02T13:35:10.062428+0000","last_clean":"2024-04-02T13:35:10.062428+0000","last_became_active":"2024-04-02T13:32:54.563323+0000","last_became_peered":"2024-04-02T13:32:54.563323+0000","last_unstale":"2024-04-02T13:35:10.062428+0000","last_undegraded":"2024-04-02T13:35:10.062428+0000","last_fullsized":"2024-04-02T13:35:10.062428+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:24:57.696770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.323339+0000","last_change":"2024-04-02T13:32:49.526659+0000","last_active":"2024-04-02T13:35:10.323339+0000","last_peered":"2024-04-02T13:35:10.323339+0000","last_clean":"2024-04-02T13:35:10.323339+0000","last_became_active":"2024-04-02T13:32:49.526352+0000","last_became_peered":"2024-04-02T13:32:49.526352+0000","last_unstale":"2024-04-02T13:35:10.323339+0000","last_undegraded":"2024-04-02T13:35:10.323339+0000","last_fullsized":"2024-04-02T13:35:10.323339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:55:15.273844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.820157+0000","last_change":"2024-04-02T13:32:50.527264+0000","last_active":"2024-04-02T13:35:09.820157+0000","last_peered":"2024-04-02T13:35:09.820157+0000","last_clean":"2024-04-02T13:35:09.820157+0000","last_became_active":"2024-04-02T13:32:50.525600+0000","last_became_peered":"2024-04-02T13:32:50.525600+0000","last_unstale":"2024-04-02T13:35:09.820157+0000","last_undegraded":"2024-04-02T13:35:09.820157+0000","last_fullsized":"2024-04-02T13:35:09.820157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:51:21.313904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062937+0000","last_change":"2024-04-02T13:32:54.572522+0000","last_active":"2024-04-02T13:35:10.062937+0000","last_peered":"2024-04-02T13:35:10.062937+0000","last_clean":"2024-04-02T13:35:10.062937+0000","last_became_active":"2024-04-02T13:32:54.571124+0000","last_became_peered":"2024-04-02T13:32:54.571124+0000","last_unstale":"2024-04-02T13:35:10.062937+0000","last_undegraded":"2024-04-02T13:35:10.062937+0000","last_fullsized":"2024-04-02T13:35:10.062937+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:16:22.309738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928098+0000","last_change":"2024-04-02T13:32:50.527342+0000","last_active":"2024-04-02T13:35:09.928098+0000","last_peered":"2024-04-02T13:35:09.928098+0000","last_clean":"2024-04-02T13:35:09.928098+0000","last_became_active":"2024-04-02T13:32:50.527038+0000","last_became_peered":"2024-04-02T13:32:50.527038+0000","last_unstale":"2024-04-02T13:35:09.928098+0000","last_undegraded":"2024-04-02T13:35:09.928098+0000","last_fullsized":"2024-04-02T13:35:09.928098+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:06:19.807925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":160,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.066596+0000","last_change":"2024-04-02T13:32:49.529447+0000","last_active":"2024-04-02T13:35:10.066596+0000","last_peered":"2024-04-02T13:35:10.066596+0000","last_clean":"2024-04-02T13:35:10.066596+0000","last_became_active":"2024-04-02T13:32:49.529138+0000","last_became_peered":"2024-04-02T13:32:49.529138+0000","last_unstale":"2024-04-02T13:35:10.066596+0000","last_undegraded":"2024-04-02T13:35:10.066596+0000","last_fullsized":"2024-04-02T13:35:10.066596+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:32:51.676065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063688+0000","last_change":"2024-04-02T13:32:54.562709+0000","last_active":"2024-04-02T13:35:10.063688+0000","last_peered":"2024-04-02T13:35:10.063688+0000","last_clean":"2024-04-02T13:35:10.063688+0000","last_became_active":"2024-04-02T13:32:54.562209+0000","last_became_peered":"2024-04-02T13:32:54.562209+0000","last_unstale":"2024-04-02T13:35:10.063688+0000","last_undegraded":"2024-04-02T13:35:10.063688+0000","last_fullsized":"2024-04-02T13:35:10.063688+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:31:02.530080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818230+0000","last_change":"2024-04-02T13:32:50.535913+0000","last_active":"2024-04-02T13:35:09.818230+0000","last_peered":"2024-04-02T13:35:09.818230+0000","last_clean":"2024-04-02T13:35:09.818230+0000","last_became_active":"2024-04-02T13:32:50.535619+0000","last_became_peered":"2024-04-02T13:32:50.535619+0000","last_unstale":"2024-04-02T13:35:09.818230+0000","last_undegraded":"2024-04-02T13:35:09.818230+0000","last_fullsized":"2024-04-02T13:35:09.818230+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:34:19.910430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062201+0000","last_change":"2024-04-02T13:32:49.515946+0000","last_active":"2024-04-02T13:35:10.062201+0000","last_peered":"2024-04-02T13:35:10.062201+0000","last_clean":"2024-04-02T13:35:10.062201+0000","last_became_active":"2024-04-02T13:32:49.515624+0000","last_became_peered":"2024-04-02T13:32:49.515624+0000","last_unstale":"2024-04-02T13:35:10.062201+0000","last_undegraded":"2024-04-02T13:35:10.062201+0000","last_fullsized":"2024-04-02T13:35:10.062201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:35:36.228546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":367,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063125+0000","last_change":"2024-04-02T13:31:37.130024+0000","last_active":"2024-04-02T13:35:10.063125+0000","last_peered":"2024-04-02T13:35:10.063125+0000","last_clean":"2024-04-02T13:35:10.063125+0000","last_became_active":"2024-04-02T13:31:37.129405+0000","last_became_peered":"2024-04-02T13:31:37.129405+0000","last_unstale":"2024-04-02T13:35:10.063125+0000","last_undegraded":"2024-04-02T13:35:10.063125+0000","last_fullsized":"2024-04-02T13:35:10.063125+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:28:14.686008+0000","last_clean_scrub_stamp":"2024-04-02T13:28:14.686008+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:50:47.425782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.928211+0000","last_change":"2024-04-02T13:32:54.564169+0000","last_active":"2024-04-02T13:35:09.928211+0000","last_peered":"2024-04-02T13:35:09.928211+0000","last_clean":"2024-04-02T13:35:09.928211+0000","last_became_active":"2024-04-02T13:32:54.563687+0000","last_became_peered":"2024-04-02T13:32:54.563687+0000","last_unstale":"2024-04-02T13:35:09.928211+0000","last_undegraded":"2024-04-02T13:35:09.928211+0000","last_fullsized":"2024-04-02T13:35:09.928211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:56:22.123841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061853+0000","last_change":"2024-04-02T13:32:50.526116+0000","last_active":"2024-04-02T13:35:10.061853+0000","last_peered":"2024-04-02T13:35:10.061853+0000","last_clean":"2024-04-02T13:35:10.061853+0000","last_became_active":"2024-04-02T13:32:50.525077+0000","last_became_peered":"2024-04-02T13:32:50.525077+0000","last_unstale":"2024-04-02T13:35:10.061853+0000","last_undegraded":"2024-04-02T13:35:10.061853+0000","last_fullsized":"2024-04-02T13:35:10.061853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:50:27.375016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'4","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063479+0000","last_change":"2024-04-02T13:32:49.529064+0000","last_active":"2024-04-02T13:35:10.063479+0000","last_peered":"2024-04-02T13:35:10.063479+0000","last_clean":"2024-04-02T13:35:10.063479+0000","last_became_active":"2024-04-02T13:32:49.528689+0000","last_became_peered":"2024-04-02T13:32:49.528689+0000","last_unstale":"2024-04-02T13:35:10.063479+0000","last_undegraded":"2024-04-02T13:35:10.063479+0000","last_fullsized":"2024-04-02T13:35:10.063479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:25:30.583923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070541+0000","last_change":"2024-04-02T13:32:54.567166+0000","last_active":"2024-04-02T13:35:10.070541+0000","last_peered":"2024-04-02T13:35:10.070541+0000","last_clean":"2024-04-02T13:35:10.070541+0000","last_became_active":"2024-04-02T13:32:54.566693+0000","last_became_peered":"2024-04-02T13:32:54.566693+0000","last_unstale":"2024-04-02T13:35:10.070541+0000","last_undegraded":"2024-04-02T13:35:10.070541+0000","last_fullsized":"2024-04-02T13:35:10.070541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:31:41.993868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"90'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819228+0000","last_change":"2024-04-02T13:32:49.529277+0000","last_active":"2024-04-02T13:35:09.819228+0000","last_peered":"2024-04-02T13:35:09.819228+0000","last_clean":"2024-04-02T13:35:09.819228+0000","last_became_active":"2024-04-02T13:32:49.528927+0000","last_became_peered":"2024-04-02T13:32:49.528927+0000","last_unstale":"2024-04-02T13:35:09.819228+0000","last_undegraded":"2024-04-02T13:35:09.819228+0000","last_fullsized":"2024-04-02T13:35:09.819228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:23:16.360358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062665+0000","last_change":"2024-04-02T13:32:50.528477+0000","last_active":"2024-04-02T13:35:10.062665+0000","last_peered":"2024-04-02T13:35:10.062665+0000","last_clean":"2024-04-02T13:35:10.062665+0000","last_became_active":"2024-04-02T13:32:50.528144+0000","last_became_peered":"2024-04-02T13:32:50.528144+0000","last_unstale":"2024-04-02T13:35:10.062665+0000","last_undegraded":"2024-04-02T13:35:10.062665+0000","last_fullsized":"2024-04-02T13:35:10.062665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:56:22.731241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818025+0000","last_change":"2024-04-02T13:32:54.571928+0000","last_active":"2024-04-02T13:35:09.818025+0000","last_peered":"2024-04-02T13:35:09.818025+0000","last_clean":"2024-04-02T13:35:09.818025+0000","last_became_active":"2024-04-02T13:32:54.570794+0000","last_became_peered":"2024-04-02T13:32:54.570794+0000","last_unstale":"2024-04-02T13:35:09.818025+0000","last_undegraded":"2024-04-02T13:35:09.818025+0000","last_fullsized":"2024-04-02T13:35:09.818025+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:24:10.768538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'49","reported_seq":105,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.547985+0000","last_change":"2024-04-02T13:32:49.525786+0000","last_active":"2024-04-02T13:35:10.547985+0000","last_peered":"2024-04-02T13:35:10.547985+0000","last_clean":"2024-04-02T13:35:10.547985+0000","last_became_active":"2024-04-02T13:32:49.525271+0000","last_became_peered":"2024-04-02T13:32:49.525271+0000","last_unstale":"2024-04-02T13:35:10.547985+0000","last_undegraded":"2024-04-02T13:35:10.547985+0000","last_fullsized":"2024-04-02T13:35:10.547985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:45.029460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818962+0000","last_change":"2024-04-02T13:32:50.525118+0000","last_active":"2024-04-02T13:35:09.818962+0000","last_peered":"2024-04-02T13:35:09.818962+0000","last_clean":"2024-04-02T13:35:09.818962+0000","last_became_active":"2024-04-02T13:32:50.524807+0000","last_became_peered":"2024-04-02T13:32:50.524807+0000","last_unstale":"2024-04-02T13:35:09.818962+0000","last_undegraded":"2024-04-02T13:35:09.818962+0000","last_fullsized":"2024-04-02T13:35:09.818962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:49:13.953469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062662+0000","last_change":"2024-04-02T13:32:54.562268+0000","last_active":"2024-04-02T13:35:10.062662+0000","last_peered":"2024-04-02T13:35:10.062662+0000","last_clean":"2024-04-02T13:35:10.062662+0000","last_became_active":"2024-04-02T13:32:54.561188+0000","last_became_peered":"2024-04-02T13:32:54.561188+0000","last_unstale":"2024-04-02T13:35:10.062662+0000","last_undegraded":"2024-04-02T13:35:10.062662+0000","last_fullsized":"2024-04-02T13:35:10.062662+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:14:09.230373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"90'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062851+0000","last_change":"2024-04-02T13:32:49.520853+0000","last_active":"2024-04-02T13:35:10.062851+0000","last_peered":"2024-04-02T13:35:10.062851+0000","last_clean":"2024-04-02T13:35:10.062851+0000","last_became_active":"2024-04-02T13:32:49.520545+0000","last_became_peered":"2024-04-02T13:32:49.520545+0000","last_unstale":"2024-04-02T13:35:10.062851+0000","last_undegraded":"2024-04-02T13:35:10.062851+0000","last_fullsized":"2024-04-02T13:35:10.062851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:48:15.709817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065708+0000","last_change":"2024-04-02T13:32:50.536530+0000","last_active":"2024-04-02T13:35:10.065708+0000","last_peered":"2024-04-02T13:35:10.065708+0000","last_clean":"2024-04-02T13:35:10.065708+0000","last_became_active":"2024-04-02T13:32:50.536232+0000","last_became_peered":"2024-04-02T13:32:50.536232+0000","last_unstale":"2024-04-02T13:35:10.065708+0000","last_undegraded":"2024-04-02T13:35:10.065708+0000","last_fullsized":"2024-04-02T13:35:10.065708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:38:50.720401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.061934+0000","last_change":"2024-04-02T13:32:54.567420+0000","last_active":"2024-04-02T13:35:10.061934+0000","last_peered":"2024-04-02T13:35:10.061934+0000","last_clean":"2024-04-02T13:35:10.061934+0000","last_became_active":"2024-04-02T13:32:54.567007+0000","last_became_peered":"2024-04-02T13:32:54.567007+0000","last_unstale":"2024-04-02T13:35:10.061934+0000","last_undegraded":"2024-04-02T13:35:10.061934+0000","last_fullsized":"2024-04-02T13:35:10.061934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:15:58.688956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067569+0000","last_change":"2024-04-02T13:33:55.564335+0000","last_active":"2024-04-02T13:35:10.067569+0000","last_peered":"2024-04-02T13:35:10.067569+0000","last_clean":"2024-04-02T13:35:10.067569+0000","last_became_active":"2024-04-02T13:33:55.560370+0000","last_became_peered":"2024-04-02T13:33:55.560370+0000","last_unstale":"2024-04-02T13:35:10.067569+0000","last_undegraded":"2024-04-02T13:35:10.067569+0000","last_fullsized":"2024-04-02T13:35:10.067569+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:14.069597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818822+0000","last_change":"2024-04-02T13:32:50.530315+0000","last_active":"2024-04-02T13:35:09.818822+0000","last_peered":"2024-04-02T13:35:09.818822+0000","last_clean":"2024-04-02T13:35:09.818822+0000","last_became_active":"2024-04-02T13:32:50.530001+0000","last_became_peered":"2024-04-02T13:32:50.530001+0000","last_unstale":"2024-04-02T13:35:09.818822+0000","last_undegraded":"2024-04-02T13:35:09.818822+0000","last_fullsized":"2024-04-02T13:35:09.818822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:36:08.789880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.919950+0000","last_change":"2024-04-02T13:32:54.559931+0000","last_active":"2024-04-02T13:35:09.919950+0000","last_peered":"2024-04-02T13:35:09.919950+0000","last_clean":"2024-04-02T13:35:09.919950+0000","last_became_active":"2024-04-02T13:32:54.559504+0000","last_became_peered":"2024-04-02T13:32:54.559504+0000","last_unstale":"2024-04-02T13:35:09.919950+0000","last_undegraded":"2024-04-02T13:35:09.919950+0000","last_fullsized":"2024-04-02T13:35:09.919950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:44:58.755339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"90'7","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818121+0000","last_change":"2024-04-02T13:32:49.530593+0000","last_active":"2024-04-02T13:35:09.818121+0000","last_peered":"2024-04-02T13:35:09.818121+0000","last_clean":"2024-04-02T13:35:09.818121+0000","last_became_active":"2024-04-02T13:32:49.529470+0000","last_became_peered":"2024-04-02T13:32:49.529470+0000","last_unstale":"2024-04-02T13:35:09.818121+0000","last_undegraded":"2024-04-02T13:35:09.818121+0000","last_fullsized":"2024-04-02T13:35:09.818121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:19:50.022188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070490+0000","last_change":"2024-04-02T13:32:50.522630+0000","last_active":"2024-04-02T13:35:10.070490+0000","last_peered":"2024-04-02T13:35:10.070490+0000","last_clean":"2024-04-02T13:35:10.070490+0000","last_became_active":"2024-04-02T13:32:50.522314+0000","last_became_peered":"2024-04-02T13:32:50.522314+0000","last_unstale":"2024-04-02T13:35:10.070490+0000","last_undegraded":"2024-04-02T13:35:10.070490+0000","last_fullsized":"2024-04-02T13:35:10.070490+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:57:15.573666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":26,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062642+0000","last_change":"2024-04-02T13:32:54.571148+0000","last_active":"2024-04-02T13:35:10.062642+0000","last_peered":"2024-04-02T13:35:10.062642+0000","last_clean":"2024-04-02T13:35:10.062642+0000","last_became_active":"2024-04-02T13:32:54.570712+0000","last_became_peered":"2024-04-02T13:32:54.570712+0000","last_unstale":"2024-04-02T13:35:10.062642+0000","last_undegraded":"2024-04-02T13:35:10.062642+0000","last_fullsized":"2024-04-02T13:35:10.062642+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T13:54:56.944473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"95'4","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062817+0000","last_change":"2024-04-02T13:32:49.525809+0000","last_active":"2024-04-02T13:35:10.062817+0000","last_peered":"2024-04-02T13:35:10.062817+0000","last_clean":"2024-04-02T13:35:10.062817+0000","last_became_active":"2024-04-02T13:32:49.525358+0000","last_became_peered":"2024-04-02T13:32:49.525358+0000","last_unstale":"2024-04-02T13:35:10.062817+0000","last_undegraded":"2024-04-02T13:35:10.062817+0000","last_fullsized":"2024-04-02T13:35:10.062817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:41:48.307652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818375+0000","last_change":"2024-04-02T13:32:50.536904+0000","last_active":"2024-04-02T13:35:09.818375+0000","last_peered":"2024-04-02T13:35:09.818375+0000","last_clean":"2024-04-02T13:35:09.818375+0000","last_became_active":"2024-04-02T13:32:50.533737+0000","last_became_peered":"2024-04-02T13:32:50.533737+0000","last_unstale":"2024-04-02T13:35:09.818375+0000","last_undegraded":"2024-04-02T13:35:09.818375+0000","last_fullsized":"2024-04-02T13:35:09.818375+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T22:13:05.023721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062023+0000","last_change":"2024-04-02T13:32:54.570077+0000","last_active":"2024-04-02T13:35:10.062023+0000","last_peered":"2024-04-02T13:35:10.062023+0000","last_clean":"2024-04-02T13:35:10.062023+0000","last_became_active":"2024-04-02T13:32:54.569367+0000","last_became_peered":"2024-04-02T13:32:54.569367+0000","last_unstale":"2024-04-02T13:35:10.062023+0000","last_undegraded":"2024-04-02T13:35:10.062023+0000","last_fullsized":"2024-04-02T13:35:10.062023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:55:56.796422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818964+0000","last_change":"2024-04-02T13:33:55.565608+0000","last_active":"2024-04-02T13:35:09.818964+0000","last_peered":"2024-04-02T13:35:09.818964+0000","last_clean":"2024-04-02T13:35:09.818964+0000","last_became_active":"2024-04-02T13:33:55.557475+0000","last_became_peered":"2024-04-02T13:33:55.557475+0000","last_unstale":"2024-04-02T13:35:09.818964+0000","last_undegraded":"2024-04-02T13:35:09.818964+0000","last_fullsized":"2024-04-02T13:35:09.818964+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:17:21.644195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063486+0000","last_change":"2024-04-02T13:32:50.524220+0000","last_active":"2024-04-02T13:35:10.063486+0000","last_peered":"2024-04-02T13:35:10.063486+0000","last_clean":"2024-04-02T13:35:10.063486+0000","last_became_active":"2024-04-02T13:32:50.523773+0000","last_became_peered":"2024-04-02T13:32:50.523773+0000","last_unstale":"2024-04-02T13:35:10.063486+0000","last_undegraded":"2024-04-02T13:35:10.063486+0000","last_fullsized":"2024-04-02T13:35:10.063486+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T19:27:15.487340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067660+0000","last_change":"2024-04-02T13:32:54.560438+0000","last_active":"2024-04-02T13:35:10.067660+0000","last_peered":"2024-04-02T13:35:10.067660+0000","last_clean":"2024-04-02T13:35:10.067660+0000","last_became_active":"2024-04-02T13:32:54.559989+0000","last_became_peered":"2024-04-02T13:32:54.559989+0000","last_unstale":"2024-04-02T13:35:10.067660+0000","last_undegraded":"2024-04-02T13:35:10.067660+0000","last_fullsized":"2024-04-02T13:35:10.067660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:15:07.584013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"95'8","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.315848+0000","last_change":"2024-04-02T13:32:49.526315+0000","last_active":"2024-04-02T13:35:10.315848+0000","last_peered":"2024-04-02T13:35:10.315848+0000","last_clean":"2024-04-02T13:35:10.315848+0000","last_became_active":"2024-04-02T13:32:49.526000+0000","last_became_peered":"2024-04-02T13:32:49.526000+0000","last_unstale":"2024-04-02T13:35:10.315848+0000","last_undegraded":"2024-04-02T13:35:10.315848+0000","last_fullsized":"2024-04-02T13:35:10.315848+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:30:41.898675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064153+0000","last_change":"2024-04-02T13:32:50.526739+0000","last_active":"2024-04-02T13:35:10.064153+0000","last_peered":"2024-04-02T13:35:10.064153+0000","last_clean":"2024-04-02T13:35:10.064153+0000","last_became_active":"2024-04-02T13:32:50.526215+0000","last_became_peered":"2024-04-02T13:32:50.526215+0000","last_unstale":"2024-04-02T13:35:10.064153+0000","last_undegraded":"2024-04-02T13:35:10.064153+0000","last_fullsized":"2024-04-02T13:35:10.064153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T14:20:57.802804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.064951+0000","last_change":"2024-04-02T13:34:56.181555+0000","last_active":"2024-04-02T13:35:10.064951+0000","last_peered":"2024-04-02T13:35:10.064951+0000","last_clean":"2024-04-02T13:35:10.064951+0000","last_became_active":"2024-04-02T13:34:56.181156+0000","last_became_peered":"2024-04-02T13:34:56.181156+0000","last_unstale":"2024-04-02T13:35:10.064951+0000","last_undegraded":"2024-04-02T13:35:10.064951+0000","last_fullsized":"2024-04-02T13:35:10.064951+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:08:23.387417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,1,4],"acting":[0,8,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":82,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.027635+0000","last_change":"2024-04-02T13:32:49.528516+0000","last_active":"2024-04-02T13:35:10.027635+0000","last_peered":"2024-04-02T13:35:10.027635+0000","last_clean":"2024-04-02T13:35:10.027635+0000","last_became_active":"2024-04-02T13:32:49.528182+0000","last_became_peered":"2024-04-02T13:32:49.528182+0000","last_unstale":"2024-04-02T13:35:10.027635+0000","last_undegraded":"2024-04-02T13:35:10.027635+0000","last_fullsized":"2024-04-02T13:35:10.027635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:13:09.056123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063750+0000","last_change":"2024-04-02T13:32:50.527519+0000","last_active":"2024-04-02T13:35:10.063750+0000","last_peered":"2024-04-02T13:35:10.063750+0000","last_clean":"2024-04-02T13:35:10.063750+0000","last_became_active":"2024-04-02T13:32:50.527227+0000","last_became_peered":"2024-04-02T13:32:50.527227+0000","last_unstale":"2024-04-02T13:35:10.063750+0000","last_undegraded":"2024-04-02T13:35:10.063750+0000","last_fullsized":"2024-04-02T13:35:10.063750+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:25:33.790924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927710+0000","last_change":"2024-04-02T13:32:54.560616+0000","last_active":"2024-04-02T13:35:09.927710+0000","last_peered":"2024-04-02T13:35:09.927710+0000","last_clean":"2024-04-02T13:35:09.927710+0000","last_became_active":"2024-04-02T13:32:54.560206+0000","last_became_peered":"2024-04-02T13:32:54.560206+0000","last_unstale":"2024-04-02T13:35:09.927710+0000","last_undegraded":"2024-04-02T13:35:09.927710+0000","last_fullsized":"2024-04-02T13:35:09.927710+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:53.533445+0000","last_clean_scrub_stamp":"2024-04-02T13:32:53.533445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T16:03:06.924673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"90'5","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062701+0000","last_change":"2024-04-02T13:32:49.532122+0000","last_active":"2024-04-02T13:35:10.062701+0000","last_peered":"2024-04-02T13:35:10.062701+0000","last_clean":"2024-04-02T13:35:10.062701+0000","last_became_active":"2024-04-02T13:32:49.531823+0000","last_became_peered":"2024-04-02T13:32:49.531823+0000","last_unstale":"2024-04-02T13:35:10.062701+0000","last_undegraded":"2024-04-02T13:35:10.062701+0000","last_fullsized":"2024-04-02T13:35:10.062701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:48.493294+0000","last_clean_scrub_stamp":"2024-04-02T13:32:48.493294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:16:21.454786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063549+0000","last_change":"2024-04-02T13:32:50.532480+0000","last_active":"2024-04-02T13:35:10.063549+0000","last_peered":"2024-04-02T13:35:10.063549+0000","last_clean":"2024-04-02T13:35:10.063549+0000","last_became_active":"2024-04-02T13:32:50.532173+0000","last_became_peered":"2024-04-02T13:32:50.532173+0000","last_unstale":"2024-04-02T13:35:10.063549+0000","last_undegraded":"2024-04-02T13:35:10.063549+0000","last_fullsized":"2024-04-02T13:35:10.063549+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T00:53:22.296245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070648+0000","last_change":"2024-04-02T13:32:50.529609+0000","last_active":"2024-04-02T13:35:10.070648+0000","last_peered":"2024-04-02T13:35:10.070648+0000","last_clean":"2024-04-02T13:35:10.070648+0000","last_became_active":"2024-04-02T13:32:50.529282+0000","last_became_peered":"2024-04-02T13:32:50.529282+0000","last_unstale":"2024-04-02T13:35:10.070648+0000","last_undegraded":"2024-04-02T13:35:10.070648+0000","last_fullsized":"2024-04-02T13:35:10.070648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:26:53.760387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062874+0000","last_change":"2024-04-02T13:33:55.562541+0000","last_active":"2024-04-02T13:35:10.062874+0000","last_peered":"2024-04-02T13:35:10.062874+0000","last_clean":"2024-04-02T13:35:10.062874+0000","last_became_active":"2024-04-02T13:33:55.562167+0000","last_became_peered":"2024-04-02T13:33:55.562167+0000","last_unstale":"2024-04-02T13:35:10.062874+0000","last_undegraded":"2024-04-02T13:35:10.062874+0000","last_fullsized":"2024-04-02T13:35:10.062874+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:18:29.215070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062695+0000","last_change":"2024-04-02T13:32:50.536883+0000","last_active":"2024-04-02T13:35:10.062695+0000","last_peered":"2024-04-02T13:35:10.062695+0000","last_clean":"2024-04-02T13:35:10.062695+0000","last_became_active":"2024-04-02T13:32:50.536569+0000","last_became_peered":"2024-04-02T13:32:50.536569+0000","last_unstale":"2024-04-02T13:35:10.062695+0000","last_undegraded":"2024-04-02T13:35:10.062695+0000","last_fullsized":"2024-04-02T13:35:10.062695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T23:30:27.419565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.065156+0000","last_change":"2024-04-02T13:32:50.537785+0000","last_active":"2024-04-02T13:35:10.065156+0000","last_peered":"2024-04-02T13:35:10.065156+0000","last_clean":"2024-04-02T13:35:10.065156+0000","last_became_active":"2024-04-02T13:32:50.537475+0000","last_became_peered":"2024-04-02T13:32:50.537475+0000","last_unstale":"2024-04-02T13:35:10.065156+0000","last_undegraded":"2024-04-02T13:35:10.065156+0000","last_fullsized":"2024-04-02T13:35:10.065156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:25:01.710866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062536+0000","last_change":"2024-04-02T13:32:50.527678+0000","last_active":"2024-04-02T13:35:10.062536+0000","last_peered":"2024-04-02T13:35:10.062536+0000","last_clean":"2024-04-02T13:35:10.062536+0000","last_became_active":"2024-04-02T13:32:50.527343+0000","last_became_peered":"2024-04-02T13:32:50.527343+0000","last_unstale":"2024-04-02T13:35:10.062536+0000","last_undegraded":"2024-04-02T13:35:10.062536+0000","last_fullsized":"2024-04-02T13:35:10.062536+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:27:55.806101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070908+0000","last_change":"2024-04-02T13:32:50.534312+0000","last_active":"2024-04-02T13:35:10.070908+0000","last_peered":"2024-04-02T13:35:10.070908+0000","last_clean":"2024-04-02T13:35:10.070908+0000","last_became_active":"2024-04-02T13:32:50.530063+0000","last_became_peered":"2024-04-02T13:32:50.530063+0000","last_unstale":"2024-04-02T13:35:10.070908+0000","last_undegraded":"2024-04-02T13:35:10.070908+0000","last_fullsized":"2024-04-02T13:35:10.070908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T20:15:46.517981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.063259+0000","last_change":"2024-04-02T13:32:50.525584+0000","last_active":"2024-04-02T13:35:10.063259+0000","last_peered":"2024-04-02T13:35:10.063259+0000","last_clean":"2024-04-02T13:35:10.063259+0000","last_became_active":"2024-04-02T13:32:50.525247+0000","last_became_peered":"2024-04-02T13:32:50.525247+0000","last_unstale":"2024-04-02T13:35:10.063259+0000","last_undegraded":"2024-04-02T13:35:10.063259+0000","last_fullsized":"2024-04-02T13:35:10.063259+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:49:57.008528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.819077+0000","last_change":"2024-04-02T13:32:50.540696+0000","last_active":"2024-04-02T13:35:09.819077+0000","last_peered":"2024-04-02T13:35:09.819077+0000","last_clean":"2024-04-02T13:35:09.819077+0000","last_became_active":"2024-04-02T13:32:50.539103+0000","last_became_peered":"2024-04-02T13:32:50.539103+0000","last_unstale":"2024-04-02T13:35:09.819077+0000","last_undegraded":"2024-04-02T13:35:09.819077+0000","last_fullsized":"2024-04-02T13:35:09.819077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T21:49:56.296814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.927773+0000","last_change":"2024-04-02T13:32:50.543364+0000","last_active":"2024-04-02T13:35:09.927773+0000","last_peered":"2024-04-02T13:35:09.927773+0000","last_clean":"2024-04-02T13:35:09.927773+0000","last_became_active":"2024-04-02T13:32:50.543017+0000","last_became_peered":"2024-04-02T13:32:50.543017+0000","last_unstale":"2024-04-02T13:35:09.927773+0000","last_undegraded":"2024-04-02T13:35:09.927773+0000","last_fullsized":"2024-04-02T13:35:09.927773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T17:36:48.699664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:09.818159+0000","last_change":"2024-04-02T13:32:50.528559+0000","last_active":"2024-04-02T13:35:09.818159+0000","last_peered":"2024-04-02T13:35:09.818159+0000","last_clean":"2024-04-02T13:35:09.818159+0000","last_became_active":"2024-04-02T13:32:50.528233+0000","last_became_peered":"2024-04-02T13:32:50.528233+0000","last_unstale":"2024-04-02T13:35:09.818159+0000","last_undegraded":"2024-04-02T13:35:09.818159+0000","last_fullsized":"2024-04-02T13:35:09.818159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T15:18:08.351851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.067987+0000","last_change":"2024-04-02T13:32:50.529462+0000","last_active":"2024-04-02T13:35:10.067987+0000","last_peered":"2024-04-02T13:35:10.067987+0000","last_clean":"2024-04-02T13:35:10.067987+0000","last_became_active":"2024-04-02T13:32:50.528584+0000","last_became_peered":"2024-04-02T13:32:50.528584+0000","last_unstale":"2024-04-02T13:35:10.067987+0000","last_undegraded":"2024-04-02T13:35:10.067987+0000","last_fullsized":"2024-04-02T13:35:10.067987+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-04T01:22:43.042981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.062204+0000","last_change":"2024-04-02T13:32:50.528688+0000","last_active":"2024-04-02T13:35:10.062204+0000","last_peered":"2024-04-02T13:35:10.062204+0000","last_clean":"2024-04-02T13:35:10.062204+0000","last_became_active":"2024-04-02T13:32:50.528370+0000","last_became_peered":"2024-04-02T13:32:50.528370+0000","last_unstale":"2024-04-02T13:35:10.062204+0000","last_undegraded":"2024-04-02T13:35:10.062204+0000","last_fullsized":"2024-04-02T13:35:10.062204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:35:34.210577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-02T13:35:10.070777+0000","last_change":"2024-04-02T13:32:50.527365+0000","last_active":"2024-04-02T13:35:10.070777+0000","last_peered":"2024-04-02T13:35:10.070777+0000","last_clean":"2024-04-02T13:35:10.070777+0000","last_became_active":"2024-04-02T13:32:50.527035+0000","last_became_peered":"2024-04-02T13:32:50.527035+0000","last_unstale":"2024-04-02T13:35:10.070777+0000","last_undegraded":"2024-04-02T13:35:10.070777+0000","last_fullsized":"2024-04-02T13:35:10.070777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-02T13:32:49.499761+0000","last_clean_scrub_stamp":"2024-04-02T13:32:49.499761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-03T18:00:52.336392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602937,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1334,"num_read_kb":785,"num_write":375,"num_write_kb":1472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1808811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":304,"ondisk_log_size":304,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351016,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1085440,"data_stored":525216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612653,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37220,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707932,"statfs":{"total":95995035648,"available":95956922368,"internally_reserved":0,"allocated":1929216,"data_stored":1322543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776179,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28500,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716652,"statfs":{"total":95995035648,"available":95965851648,"internally_reserved":0,"allocated":1515520,"data_stored":931933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037816,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28484,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716668,"statfs":{"total":95995035648,"available":95965868032,"internally_reserved":0,"allocated":1499136,"data_stored":928103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299453,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1712128,"data_stored":1132669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1093632,"data_stored":530516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28160,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716992,"statfs":{"total":95995035648,"available":95966199808,"internally_reserved":0,"allocated":1167360,"data_stored":600591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":521994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1167360,"data_stored":584678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36932,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708220,"statfs":{"total":95995035648,"available":95957217280,"internally_reserved":0,"allocated":1634304,"data_stored":1054649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":534590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37148,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708004,"statfs":{"total":95995035648,"available":95956996096,"internally_reserved":0,"allocated":1855488,"data_stored":1259069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":394604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":202772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-02T13:35:13.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 -- 172.21.15.122:0/3586462572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 msgr2=0x7ff12c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 --2- 172.21.15.122:0/3586462572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c103220 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff114009930 tx=0x7ff11402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 -- 172.21.15.122:0/3586462572 shutdown_connections 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 --2- 172.21.15.122:0/3586462572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 --2- 172.21.15.122:0/3586462572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c103220 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 --2- 172.21.15.122:0/3586462572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 0x7ff12c1001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 -- 172.21.15.122:0/3586462572 >> 172.21.15.122:0/3586462572 conn(0x7ff12c0fb700 msgr2=0x7ff12c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 -- 172.21.15.122:0/3586462572 shutdown_connections 2024-04-02T13:35:13.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.454+0000 7ff130bf6640 1 -- 172.21.15.122:0/3586462572 wait complete. 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 Processor -- start 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 -- start start 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 0x7ff12c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c10d800 con 0x7ff12c102da0 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c10d680 con 0x7ff12c103760 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff130bf6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c10d980 con 0x7ff12c0ffdd0 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff123fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff123fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37406/0 (socket says 172.21.15.122:37406) 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff123fff640 1 -- 172.21.15.122:0/358205429 learned_addr learned my addr 172.21.15.122:0/358205429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff12ad9d640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff12a59c640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 0x7ff12c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff123fff640 1 -- 172.21.15.122:0/358205429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 msgr2=0x7ff12c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.455+0000 7ff123fff640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 0x7ff12c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff123fff640 1 -- 172.21.15.122:0/358205429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 msgr2=0x7ff12c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff123fff640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff123fff640 1 -- 172.21.15.122:0/358205429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c19fa60 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff12ad9d640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff123fff640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff114000c00 tx=0x7ff11402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114043020 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1140054e0 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c19fcf0 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114003c40 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.456+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c1a01e0 con 0x7ff12c102da0 2024-04-02T13:35:13.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.457+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c108e70 con 0x7ff12c102da0 2024-04-02T13:35:13.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.458+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7ff114046020 con 0x7ff12c102da0 2024-04-02T13:35:13.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.459+0000 7ff1237fe640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 0x7ff1180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.459+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff1140f2f50 con 0x7ff12c102da0 2024-04-02T13:35:13.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.459+0000 7ff12a59c640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 0x7ff1180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.459+0000 7ff12a59c640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 0x7ff1180a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff12c068a60 tx=0x7ff11c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.463+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff11404d020 con 0x7ff12c102da0 2024-04-02T13:35:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-02T13:35:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:13.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:13 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:13.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.660+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff12c02d5c0 con 0x7ff12c102da0 2024-04-02T13:35:13.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.662+0000 7ff1237fe640 1 -- 172.21.15.122:0/358205429 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25227 (secure 0 0 0) 0x7ff1140a9020 con 0x7ff12c102da0 2024-04-02T13:35:13.665 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:13.665 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":95,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:35:09.813014+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 msgr2=0x7ff1180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 0x7ff1180a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff12c068a60 tx=0x7ff11c005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 msgr2=0x7ff12c198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff114000c00 tx=0x7ff11402c650 comp rx=0 tx=0).stop 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 shutdown_connections 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff1180a3d20 0x7ff1180a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff12c103760 0x7ff12c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c102da0 0x7ff12c198380 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 --2- 172.21.15.122:0/358205429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff12c0ffdd0 0x7ff12c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 >> 172.21.15.122:0/358205429 conn(0x7ff12c0fb700 msgr2=0x7ff12c106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 shutdown_connections 2024-04-02T13:35:13.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.664+0000 7ff130bf6640 1 -- 172.21.15.122:0/358205429 wait complete. 2024-04-02T13:35:13.678 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:35:13.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:13.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:13 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='client.15666 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:13.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:13 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:13.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 -- 172.21.15.122:0/2002744387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 msgr2=0x7f8338103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 --2- 172.21.15.122:0/2002744387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338103180 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8328009900 tx=0x7f832802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 -- 172.21.15.122:0/2002744387 shutdown_connections 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 --2- 172.21.15.122:0/2002744387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 0x7f8338109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 --2- 172.21.15.122:0/2002744387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338103180 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 --2- 172.21.15.122:0/2002744387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 0x7f8338101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 -- 172.21.15.122:0/2002744387 >> 172.21.15.122:0/2002744387 conn(0x7f83380fd080 msgr2=0x7f83380ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 -- 172.21.15.122:0/2002744387 shutdown_connections 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 -- 172.21.15.122:0/2002744387 wait complete. 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.784+0000 7f834052b640 1 Processor -- start 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- start start 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 0x7f8338197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 0x7f833819f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d790 con 0x7f8338102d00 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d610 con 0x7f8338101b10 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f833810d910 con 0x7f83381036c0 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37428/0 (socket says 172.21.15.122:37428) 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 -- 172.21.15.122:0/3225028175 learned_addr learned my addr 172.21.15.122:0/3225028175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833e2c7640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 0x7f8338197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833eac8640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 0x7f833819f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 -- 172.21.15.122:0/3225028175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 msgr2=0x7f833819f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 0x7f833819f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 -- 172.21.15.122:0/3225028175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 msgr2=0x7f8338197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 0x7f8338197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 -- 172.21.15.122:0/3225028175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833819fa50 con 0x7f8338102d00 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f833dac6640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8328009230 tx=0x7f832802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328041070 con 0x7f8338102d00 2024-04-02T13:35:13.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833819fce0 con 0x7f8338102d00 2024-04-02T13:35:13.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8328004450 con 0x7f8338102d00 2024-04-02T13:35:13.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8328003e20 con 0x7f8338102d00 2024-04-02T13:35:13.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.785+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a01d0 con 0x7f8338102d00 2024-04-02T13:35:13.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.786+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338108e30 con 0x7f8338102d00 2024-04-02T13:35:13.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.787+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7f8328044070 con 0x7f8338102d00 2024-04-02T13:35:13.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.788+0000 7f832f7fe640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:13.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.788+0000 7f833e2c7640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:13.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.788+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f83280f2580 con 0x7f8338102d00 2024-04-02T13:35:13.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.788+0000 7f833e2c7640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 0x7f83240a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8338102b60 tx=0x7f8334005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:13.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.792+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f83280be350 con 0x7f8338102d00 2024-04-02T13:35:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.980+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f833802d540 con 0x7f8338102d00 2024-04-02T13:35:13.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.980+0000 7f832f7fe640 1 -- 172.21.15.122:0/3225028175 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f83280c2650 con 0x7f8338102d00 2024-04-02T13:35:13.983 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 msgr2=0x7f83240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 0x7f83240a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8338102b60 tx=0x7f8334005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 msgr2=0x7f8338198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8328009230 tx=0x7f832802c650 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 shutdown_connections 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83240a3d20 0x7f83240a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83381036c0 0x7f833819f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8338102d00 0x7f8338198370 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 --2- 172.21.15.122:0/3225028175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8338101b10 0x7f8338197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 >> 172.21.15.122:0/3225028175 conn(0x7f83380fd080 msgr2=0x7f8338106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 shutdown_connections 2024-04-02T13:35:13.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:13.982+0000 7f834052b640 1 -- 172.21.15.122:0/3225028175 wait complete. 2024-04-02T13:35:13.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- 172.21.15.122:0/2168039668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950100970 msgr2=0x7f7950100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 --2- 172.21.15.122:0/2168039668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950100970 0x7f7950100df0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7944009930 tx=0x7f794402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- 172.21.15.122:0/2168039668 shutdown_connections 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 --2- 172.21.15.122:0/2168039668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950101330 0x7f795010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 --2- 172.21.15.122:0/2168039668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950100970 0x7f7950100df0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 --2- 172.21.15.122:0/2168039668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 0x7f7950106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- 172.21.15.122:0/2168039668 >> 172.21.15.122:0/2168039668 conn(0x7f7950077b00 msgr2=0x7f7950077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- 172.21.15.122:0/2168039668 shutdown_connections 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- 172.21.15.122:0/2168039668 wait complete. 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 Processor -- start 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.102+0000 7f79566c7640 1 -- start start 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950100970 0x7f79501972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 0x7f795019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7950110d60 con 0x7f7950101330 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7950110be0 con 0x7f7950106980 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f79566c7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7950110ee0 con 0x7f7950100970 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37440/0 (socket says 172.21.15.122:37440) 2024-04-02T13:35:14.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 -- 172.21.15.122:0/2823296003 learned_addr learned my addr 172.21.15.122:0/2823296003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794ffff640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950100970 0x7f79501972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f7954c64640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 0x7f795019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 -- 172.21.15.122:0/2823296003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950100970 msgr2=0x7f79501972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950100970 0x7f79501972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 -- 172.21.15.122:0/2823296003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 msgr2=0x7f795019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 0x7f795019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 -- 172.21.15.122:0/2823296003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795019c2b0 con 0x7f7950101330 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794f7fe640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f794402cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.103+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944041070 con 0x7f7950101330 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.104+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7944004450 con 0x7f7950101330 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.104+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795019c540 con 0x7f7950101330 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.104+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7944034440 con 0x7f7950101330 2024-04-02T13:35:14.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.104+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79501a3e30 con 0x7f7950101330 2024-04-02T13:35:14.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.105+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79500688f0 con 0x7f7950101330 2024-04-02T13:35:14.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.106+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7f7944044070 con 0x7f7950101330 2024-04-02T13:35:14.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.107+0000 7f794d7fa640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 0x7f79380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.107+0000 7f794ffff640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 0x7f79380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.107+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f79440f2740 con 0x7f7950101330 2024-04-02T13:35:14.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.108+0000 7f794ffff640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 0x7f79380a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7940005d00 tx=0x7f794000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.110+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f79440be510 con 0x7f7950101330 2024-04-02T13:35:14.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.299+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7950108970 con 0x7f7950101330 2024-04-02T13:35:14.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.300+0000 7f794d7fa640 1 -- 172.21.15.122:0/2823296003 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f79440c2810 con 0x7f7950101330 2024-04-02T13:35:14.302 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:14.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 msgr2=0x7f79380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 0x7f79380a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7940005d00 tx=0x7f794000b410 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 msgr2=0x7f7950197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7944009230 tx=0x7f794402cba0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 shutdown_connections 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79380a3d20 0x7f79380a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7950106980 0x7f795019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7950101330 0x7f7950197800 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 --2- 172.21.15.122:0/2823296003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7950100970 0x7f79501972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 >> 172.21.15.122:0/2823296003 conn(0x7f7950077b00 msgr2=0x7f79501031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 shutdown_connections 2024-04-02T13:35:14.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.302+0000 7f79566c7640 1 -- 172.21.15.122:0/2823296003 wait complete. 2024-04-02T13:35:14.317 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:35:14.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 -- 172.21.15.122:0/1023280768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c101740 msgr2=0x7ff42c101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 --2- 172.21.15.122:0/1023280768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c101740 0x7ff42c101b20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff420009900 tx=0x7ff42002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 -- 172.21.15.122:0/1023280768 shutdown_connections 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 --2- 172.21.15.122:0/1023280768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c105dc0 0x7ff42c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 --2- 172.21.15.122:0/1023280768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c1020f0 0x7ff42c105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 --2- 172.21.15.122:0/1023280768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c101740 0x7ff42c101b20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 -- 172.21.15.122:0/1023280768 >> 172.21.15.122:0/1023280768 conn(0x7ff42c075d40 msgr2=0x7ff42c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 -- 172.21.15.122:0/1023280768 shutdown_connections 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 -- 172.21.15.122:0/1023280768 wait complete. 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.423+0000 7ff431d15640 1 Processor -- start 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 -- start start 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c101740 0x7ff42c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c19baa0 0x7ff42c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c110de0 con 0x7ff42c105dc0 2024-04-02T13:35:14.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c110c60 con 0x7ff42c101740 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff431d15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff42c110f60 con 0x7ff42c19baa0 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37466/0 (socket says 172.21.15.122:37466) 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 -- 172.21.15.122:0/4057477945 learned_addr learned my addr 172.21.15.122:0/4057477945 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42b7fe640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c101740 0x7ff42c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42bfff640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c19baa0 0x7ff42c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 -- 172.21.15.122:0/4057477945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c19baa0 msgr2=0x7ff42c19bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c19baa0 0x7ff42c19bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 -- 172.21.15.122:0/4057477945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c101740 msgr2=0x7ff42c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c101740 0x7ff42c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 -- 172.21.15.122:0/4057477945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42c19c590 con 0x7ff42c105dc0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.424+0000 7ff42affd640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00eb00 tx=0x7ff41c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.425+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c01c070 con 0x7ff42c105dc0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.425+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42c19c880 con 0x7ff42c105dc0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.425+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff41c00bbd0 con 0x7ff42c105dc0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.425+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41c016610 con 0x7ff42c105dc0 2024-04-02T13:35:14.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.425+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42c1006e0 con 0x7ff42c105dc0 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.426+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42c068d40 con 0x7ff42c105dc0 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.427+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 22) ==== 138767+0+0 (secure 0 0 0) 0x7ff41c01f070 con 0x7ff42c105dc0 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.427+0000 7ff428ff9640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 0x7ff4140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.427+0000 7ff42b7fe640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 0x7ff4140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.427+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff41c0d2640 con 0x7ff42c105dc0 2024-04-02T13:35:14.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.428+0000 7ff42b7fe640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 0x7ff4140a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff420006010 tx=0x7ff420005dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.431+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff41c09e300 con 0x7ff42c105dc0 2024-04-02T13:35:14.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.467+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7ff41c0cfc80 con 0x7ff42c105dc0 2024-04-02T13:35:14.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/358205429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3225028175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2823296003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:14 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.626+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff42c1020f0 con 0x7ff42c105dc0 2024-04-02T13:35:14.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.627+0000 7ff428ff9640 1 -- 172.21.15.122:0/4057477945 <== mon.0 v2:172.21.15.122:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7ff41c0cb770 con 0x7ff42c105dc0 2024-04-02T13:35:14.629 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:35:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 msgr2=0x7ff4140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 0x7ff4140a61e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff420006010 tx=0x7ff420005dd0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 msgr2=0x7ff42c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00eb00 tx=0x7ff41c009ec0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 shutdown_connections 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff4140a3d20 0x7ff4140a61e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff42c19baa0 0x7ff42c19bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff42c105dc0 0x7ff42c197720 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 --2- 172.21.15.122:0/4057477945 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff42c101740 0x7ff42c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 >> 172.21.15.122:0/4057477945 conn(0x7ff42c075d40 msgr2=0x7ff42c0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 shutdown_connections 2024-04-02T13:35:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.629+0000 7ff431d15640 1 -- 172.21.15.122:0/4057477945 wait complete. 2024-04-02T13:35:14.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:35:14.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:35:14.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/358205429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3225028175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2823296003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:14 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 -- 172.21.15.122:0/4010576112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d0103690 msgr2=0x7fd9d010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/4010576112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d0103690 0x7fd9d010bb70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009900 tx=0x7fd9c402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 -- 172.21.15.122:0/4010576112 shutdown_connections 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/4010576112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d010c140 0x7fd9d010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/4010576112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d0103690 0x7fd9d010bb70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/4010576112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0102d70 0x7fd9d0103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 -- 172.21.15.122:0/4010576112 >> 172.21.15.122:0/4010576112 conn(0x7fd9d0077b60 msgr2=0x7fd9d0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 -- 172.21.15.122:0/4010576112 shutdown_connections 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.744+0000 7fd9d60eb640 1 -- 172.21.15.122:0/4010576112 wait complete. 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 Processor -- start 2024-04-02T13:35:14.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 -- start start 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0102d70 0x7fd9d0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0103690 0x7fd9d0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111f10 con 0x7fd9d010c140 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111d90 con 0x7fd9d0103690 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9d60eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0112090 con 0x7fd9d0102d70 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37486/0 (socket says 172.21.15.122:37486) 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 -- 172.21.15.122:0/538204695 learned_addr learned my addr 172.21.15.122:0/538204695 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9ceffd640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0103690 0x7fd9d0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cf7fe640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0102d70 0x7fd9d0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 -- 172.21.15.122:0/538204695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0102d70 msgr2=0x7fd9d0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0102d70 0x7fd9d0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 -- 172.21.15.122:0/538204695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0103690 msgr2=0x7fd9d0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0103690 0x7fd9d0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.745+0000 7fd9cffff640 1 -- 172.21.15.122:0/538204695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d019d420 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9cffff640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800cad0 tx=0x7fd9c800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800ae20 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8005020 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800f6f0 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d019d710 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.746+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d01a4fa0 con 0x7fd9d010c140 2024-04-02T13:35:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.747+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d0068d40 con 0x7fd9d010c140 2024-04-02T13:35:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.748+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7fd9c80051d0 con 0x7fd9d010c140 2024-04-02T13:35:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.749+0000 7fd9ccff9640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 0x7fd9b80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.749+0000 7fd9cf7fe640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 0x7fd9b80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.749+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9c80d0f80 con 0x7fd9d010c140 2024-04-02T13:35:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.749+0000 7fd9cf7fe640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 0x7fd9b80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009bd0 tx=0x7fd9c0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.753+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd9c8014030 con 0x7fd9d010c140 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:35:14.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi122:/etc/ceph/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi187:/etc/ceph/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi188:/etc/ceph/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='client.15678 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.conf 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/358205429' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3225028175' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2823296003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:14 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.929+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd9d0106ca0 con 0x7fd9d010c140 2024-04-02T13:35:14.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.930+0000 7fd9ccff9640 1 -- 172.21.15.122:0/538204695 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fd9c809cad0 con 0x7fd9d010c140 2024-04-02T13:35:14.932 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:35:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 msgr2=0x7fd9b80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 0x7fd9b80a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009bd0 tx=0x7fd9c0005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 msgr2=0x7fd9d019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800cad0 tx=0x7fd9c800cf90 comp rx=0 tx=0).stop 2024-04-02T13:35:14.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 shutdown_connections 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9b80a3db0 0x7fd9b80a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9d010c140 0x7fd9d019ccf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9d0103690 0x7fd9d0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 --2- 172.21.15.122:0/538204695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9d0102d70 0x7fd9d0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 >> 172.21.15.122:0/538204695 conn(0x7fd9d0077b60 msgr2=0x7fd9d00fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 shutdown_connections 2024-04-02T13:35:14.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:14.932+0000 7fd9d60eb640 1 -- 172.21.15.122:0/538204695 wait complete. 2024-04-02T13:35:14.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:15.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 -- 172.21.15.122:0/283727908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4100040 msgr2=0x7f67d4100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 --2- 172.21.15.122:0/283727908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4100040 0x7f67d4100480 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009930 tx=0x7f67c802bc20 comp rx=0 tx=0).stop 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 -- 172.21.15.122:0/283727908 shutdown_connections 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 --2- 172.21.15.122:0/283727908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d41009c0 0x7f67d410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 --2- 172.21.15.122:0/283727908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4100040 0x7f67d4100480 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 --2- 172.21.15.122:0/283727908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d4107c50 0x7f67d4108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 -- 172.21.15.122:0/283727908 >> 172.21.15.122:0/283727908 conn(0x7f67d4076f90 msgr2=0x7f67d40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 -- 172.21.15.122:0/283727908 shutdown_connections 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.044+0000 7f67d9da8640 1 -- 172.21.15.122:0/283727908 wait complete. 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 Processor -- start 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 -- start start 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d4100040 0x7f67d4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 0x7f67d4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4111fa0 con 0x7f67d4107c50 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4111e20 con 0x7f67d41009c0 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d9da8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4112120 con 0x7f67d4100040 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37504/0 (socket says 172.21.15.122:37504) 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 -- 172.21.15.122:0/2272162082 learned_addr learned my addr 172.21.15.122:0/2272162082 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d37fe640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d4100040 0x7f67d4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d2ffd640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 0x7f67d4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 -- 172.21.15.122:0/2272162082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d4100040 msgr2=0x7f67d4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d4100040 0x7f67d4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 -- 172.21.15.122:0/2272162082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 msgr2=0x7f67d4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 0x7f67d4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 -- 172.21.15.122:0/2272162082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d419d3e0 con 0x7f67d4107c50 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d2ffd640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 0x7f67d4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:15.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.045+0000 7f67d3fff640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00cce0 tx=0x7f67cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.046+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00f450 con 0x7f67d4107c50 2024-04-02T13:35:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.046+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67cc00fad0 con 0x7f67d4107c50 2024-04-02T13:35:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.046+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d419d6d0 con 0x7f67d4107c50 2024-04-02T13:35:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.046+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc018b60 con 0x7f67d4107c50 2024-04-02T13:35:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.046+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41a4f60 con 0x7f67d4107c50 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.047+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d4068d40 con 0x7f67d4107c50 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.048+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f67cc00fc80 con 0x7f67d4107c50 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.048+0000 7f67d0ff9640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 0x7f67bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.048+0000 7f67d37fe640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 0x7f67bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.048+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f67cc0d16c0 con 0x7f67d4107c50 2024-04-02T13:35:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.049+0000 7f67d37fe640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 0x7f67bc0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67c400ab60 tx=0x7f67c400a2d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.052+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f67cc014030 con 0x7f67d4107c50 2024-04-02T13:35:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.270+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67d4199740 con 0x7f67d4107c50 2024-04-02T13:35:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.270+0000 7f67d0ff9640 1 -- 172.21.15.122:0/2272162082 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f67d4199740 con 0x7f67d4107c50 2024-04-02T13:35:15.273 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:15.273 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:35:15.273 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 msgr2=0x7f67bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 0x7f67bc0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67c400ab60 tx=0x7f67c400a2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 msgr2=0x7f67d419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00cce0 tx=0x7f67cc00a620 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 shutdown_connections 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f67bc0a3db0 0x7f67bc0a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67d4107c50 0x7f67d419ccb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f67d41009c0 0x7f67d4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 --2- 172.21.15.122:0/2272162082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67d4100040 0x7f67d4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.272+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 >> 172.21.15.122:0/2272162082 conn(0x7f67d4076f90 msgr2=0x7f67d4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.273+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 shutdown_connections 2024-04-02T13:35:15.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.273+0000 7f67d9da8640 1 -- 172.21.15.122:0/2272162082 wait complete. 2024-04-02T13:35:15.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- 172.21.15.122:0/1680544134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a401013c0 msgr2=0x7f5a4010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 --2- 172.21.15.122:0/1680544134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a401013c0 0x7f5a4010d8d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5a38009cd0 tx=0x7f5a3802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- 172.21.15.122:0/1680544134 shutdown_connections 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 --2- 172.21.15.122:0/1680544134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a401013c0 0x7f5a4010d8d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 --2- 172.21.15.122:0/1680544134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 --2- 172.21.15.122:0/1680544134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a40106a10 0x7f5a40106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- 172.21.15.122:0/1680544134 >> 172.21.15.122:0/1680544134 conn(0x7f5a40077b00 msgr2=0x7f5a40077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- 172.21.15.122:0/1680544134 shutdown_connections 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- 172.21.15.122:0/1680544134 wait complete. 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 Processor -- start 2024-04-02T13:35:15.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.389+0000 7f5a46677640 1 -- start start 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a401013c0 0x7f5a401977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40110e20 con 0x7f5a40106a10 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40110ca0 con 0x7f5a40100a00 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a46677640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a40110fa0 con 0x7f5a401013c0 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37522/0 (socket says 172.21.15.122:37522) 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 -- 172.21.15.122:0/1218110994 learned_addr learned my addr 172.21.15.122:0/1218110994 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a3ffff640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a3f7fe640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a401013c0 0x7f5a401977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 -- 172.21.15.122:0/1218110994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a401013c0 msgr2=0x7f5a401977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a401013c0 0x7f5a401977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 -- 172.21.15.122:0/1218110994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 msgr2=0x7f5a40197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a44c14640 1 -- 172.21.15.122:0/1218110994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4019c250 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.390+0000 7f5a3ffff640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a44c14640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a38032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38041020 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a38034cd0 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4019c4e0 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a38039040 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.391+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a401a3dd0 con 0x7f5a40106a10 2024-04-02T13:35:15.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.392+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a400688f0 con 0x7f5a40106a10 2024-04-02T13:35:15.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.394+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f5a38044020 con 0x7f5a40106a10 2024-04-02T13:35:15.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.394+0000 7f5a3d7fa640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 0x7f5a280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.395+0000 7f5a3ffff640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 0x7f5a280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.395+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5a380f2920 con 0x7f5a40106a10 2024-04-02T13:35:15.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.395+0000 7f5a3ffff640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 0x7f5a280a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a34005fd0 tx=0x7f5a3400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.399+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5a3804b020 con 0x7f5a40106a10 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:15.567 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:15.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4057477945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/538204695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.568 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:15 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2272162082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.593+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a40108f70 con 0x7f5a40106a10 2024-04-02T13:35:15.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.594+0000 7f5a3d7fa640 1 -- 172.21.15.122:0/1218110994 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25227 (secure 0 0 0) 0x7f5a380bebd0 con 0x7f5a40106a10 2024-04-02T13:35:15.597 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:15.597 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":95,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:35:09.813014+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:35:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 msgr2=0x7f5a280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 0x7f5a280a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a34005fd0 tx=0x7f5a3400b390 comp rx=0 tx=0).stop 2024-04-02T13:35:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 msgr2=0x7f5a4019bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5a38004af0 tx=0x7f5a38032ec0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 shutdown_connections 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a280a3db0 0x7f5a280a6270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a40106a10 0x7f5a4019bb20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a401013c0 0x7f5a401977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 --2- 172.21.15.122:0/1218110994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a40100a00 0x7f5a40197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 >> 172.21.15.122:0/1218110994 conn(0x7f5a40077b00 msgr2=0x7f5a40103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 shutdown_connections 2024-04-02T13:35:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.597+0000 7f5a46677640 1 -- 172.21.15.122:0/1218110994 wait complete. 2024-04-02T13:35:15.610 DEBUG:tasks.fwd_scrub:Yielding 2024-04-02T13:35:15.610 INFO:teuthology.run_tasks:Running task exec... 2024-04-02T13:35:15.619 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:35:15.619 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:35:15.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-02T13:35:15.619 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-02T13:35:15.619 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:35:15.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:15.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:35:15.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 -- 172.21.15.122:0/3084852157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 msgr2=0x7fe45c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 --2- 172.21.15.122:0/3084852157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c10d8b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe454009cd0 tx=0x7fe45402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/168570939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 msgr2=0x7f4ae410e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/168570939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae410e950 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009cd0 tx=0x7f4adc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/168570939 shutdown_connections 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/168570939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae410e950 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/168570939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ae4101a80 0x7f4ae4101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/168570939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4107a90 0x7f4ae4107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/168570939 >> 172.21.15.122:0/168570939 conn(0x7f4ae40fd590 msgr2=0x7f4ae40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 -- 172.21.15.122:0/3084852157 shutdown_connections 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 --2- 172.21.15.122:0/3084852157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c10d8b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 --2- 172.21.15.122:0/3084852157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1009e0 0x7fe45c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 --2- 172.21.15.122:0/3084852157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1069f0 0x7fe45c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 -- 172.21.15.122:0/3084852157 >> 172.21.15.122:0/3084852157 conn(0x7fe45c077b20 msgr2=0x7fe45c077f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/168570939 shutdown_connections 2024-04-02T13:35:15.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 -- 172.21.15.122:0/3084852157 shutdown_connections 2024-04-02T13:35:15.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/168570939 wait complete. 2024-04-02T13:35:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 -- 172.21.15.122:0/3084852157 wait complete. 2024-04-02T13:35:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7fe462c77640 1 Processor -- start 2024-04-02T13:35:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.726+0000 7f4aeaaeb640 1 Processor -- start 2024-04-02T13:35:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 -- start start 2024-04-02T13:35:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 -- start start 2024-04-02T13:35:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 0x7fe45c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4101a80 0x7f4ae4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ae4107a90 0x7f4ae419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111e70 con 0x7f4ae4102440 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111cf0 con 0x7f4ae4101a80 2024-04-02T13:35:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4aeaaeb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae4111ff0 con 0x7f4ae4107a90 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37556/0 (socket says 172.21.15.122:37556) 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 -- 172.21.15.122:0/353483643 learned_addr learned my addr 172.21.15.122:0/353483643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe460a13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 0x7fe45c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe460a13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 0x7fe45c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60870/0 (socket says 172.21.15.122:60870) 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe460a13640 1 -- 172.21.15.122:0/1508327411 learned_addr learned my addr 172.21.15.122:0/1508327411 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 0x7fe45c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c110e00 con 0x7fe45c1013a0 2024-04-02T13:35:15.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae8887640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4101a80 0x7f4ae4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c110c80 con 0x7fe45c1009e0 2024-04-02T13:35:15.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe45c110f80 con 0x7fe45c1069f0 2024-04-02T13:35:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 -- 172.21.15.122:0/353483643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ae4107a90 msgr2=0x7f4ae419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ae4107a90 0x7f4ae419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 -- 172.21.15.122:0/353483643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4101a80 msgr2=0x7f4ae4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4101a80 0x7f4ae4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 -- 172.21.15.122:0/353483643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae419d360 con 0x7f4ae4102440 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 -- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 msgr2=0x7fe45c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe461214640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 0x7fe45c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 0x7fe45c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 -- 172.21.15.122:0/1508327411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 msgr2=0x7fe45c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 0x7fe45c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe45bfff640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45c19d420 con 0x7fe45c1013a0 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7fe461214640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 0x7fe45c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:15.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.727+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000d950 tx=0x7f4ad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe45bfff640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c198970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe44800d720 tx=0x7fe44800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7f4ae1ffb640 1 -- 172.21.15.122:0/353483643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0017070 con 0x7f4ae4102440 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:15.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:15.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4057477945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/538204695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.741 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:15 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2272162082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448017070 con 0x7fe45c1013a0 2024-04-02T13:35:15.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe44800de30 con 0x7fe45c1013a0 2024-04-02T13:35:15.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae419d650 con 0x7f4ae4102440 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7f4ae1ffb640 1 -- 172.21.15.122:0/353483643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad0009c30 con 0x7f4ae4102440 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7f4ae1ffb640 1 -- 172.21.15.122:0/353483643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad000f6b0 con 0x7f4ae4102440 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae41a4ee0 con 0x7f4ae4102440 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480047a0 con 0x7fe45c1013a0 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45c19d710 con 0x7fe45c1013a0 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.728+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe45c0fd8d0 con 0x7fe45c1013a0 2024-04-02T13:35:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.729+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45c102120 con 0x7fe45c1013a0 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.730+0000 7f4ae1ffb640 1 -- 172.21.15.122:0/353483643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f4ad00109d0 con 0x7f4ae4102440 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 Processor -- start 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 -- start start 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae1ffb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 0x7f4ad40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6c92770 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6c3a960 con 0x55a8d6d377c0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6c8e8c0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae1ffb640 1 -- 172.21.15.122:0/353483643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4ad00d0960 con 0x7f4ae4102440 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae9088640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 0x7f4ad40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae8887640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae8887640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d2e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:56856/0 (socket says 172.21.15.122:56856) 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37568/0 (socket says 172.21.15.122:37568) 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae8887640 1 -- 172.21.15.122:0/3839575192 learned_addr learned my addr 172.21.15.122:0/3839575192 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae9088640 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 -- 172.21.15.122:0/3839575192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 msgr2=0x55a8d6d2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7fe4480099d0 con 0x7fe45c1013a0 2024-04-02T13:35:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 -- 172.21.15.122:0/3839575192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 msgr2=0x55a8d6d39bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d39bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae3fff640 1 -- 172.21.15.122:0/3839575192 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d6a6e7e0 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.731+0000 7f4ae9088640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 0x7f4ad40a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4adc037040 tx=0x7f4adc009800 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.732+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00cdc60 tx=0x7f4ad009c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f354ac3caaa4a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.732+0000 7fe459ffb640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 0x7fe44c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.732+0000 7fe460a13640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 0x7fe44c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.732+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe4480d0de0 con 0x7fe45c1013a0 2024-04-02T13:35:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.732+0000 7fe460a13640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 0x7fe44c0a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe45000b480 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.736+0000 7f4aceffd640 1 -- 172.21.15.122:0/3839575192 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0020070 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.736+0000 7f4aceffd640 1 -- 172.21.15.122:0/3839575192 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad009c400 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.736+0000 7f4aceffd640 1 -- 172.21.15.122:0/3839575192 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad009b850 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.736+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fe448020020 con 0x7fe45c1013a0 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- 172.21.15.122:0/3839575192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 msgr2=0x55a8d6d37280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00cdc60 tx=0x7f4ad009c060 comp rx=0 tx=0).stop 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- 172.21.15.122:0/3839575192 shutdown_connections 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d39bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d37280 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/3839575192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- 172.21.15.122:0/3839575192 >> 172.21.15.122:0/3839575192 conn(0x55a8d6d289f0 msgr2=0x55a8d6d28dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- 172.21.15.122:0/3839575192 shutdown_connections 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- 172.21.15.122:0/3839575192 wait complete. 2024-04-02T13:35:15.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 -- start start 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.737+0000 7f4aed7ab740 1 Processor -- start 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d34d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d35240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6d3a4f0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6c4dd20 con 0x55a8d6d377c0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a8d6d3a670 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 10 client.0 _ll_register_callbacks cb 0x55a8d6c1aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37576/0 (socket says 172.21.15.122:37576) 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 -- 172.21.15.122:0/4243326114 learned_addr learned my addr 172.21.15.122:0/4243326114 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d34d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae9088640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d35240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 -- 172.21.15.122:0/4243326114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 msgr2=0x55a8d6d34d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d34d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 -- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 msgr2=0x55a8d6d35240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d35240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 -- 172.21.15.122:0/4243326114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d6acef70 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ae8887640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ba90 tx=0x7f4ad800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab049c464b4fbad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8011440 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad009c400 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.738+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4ad80115e0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ad80117e0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 10 client.15738 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8d6966170 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad009c400 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4ad801b9f0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 10 client.15738 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 10 client.15738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 20 client.15738 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6d47a90 0x55a8d6d49ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 4 client.15738 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d6c9fcb0 con 0x55a8d6d47a90 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.739+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6d47a90 0x55a8d6d49ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.740+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6d47a90 0x55a8d6d49ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8be12b5ca849395 server_cookie=858bf7ad3a3531f in_seq=0 out_seq=0 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.740+0000 7f4ac3fff640 10 client.15738 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.740+0000 7f4ac3fff640 1 -- 172.21.15.122:0/4243326114 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a8d6c9fcb0 con 0x55a8d6d47a90 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.740+0000 7f4ac3fff640 10 client.15738 handle_command_reply: tid=0 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.740+0000 7f4aed7ab740 1 client.15738 shutdown 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 msgr2=0x55a8d6d347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800ba90 tx=0x7f4ad800bf50 comp rx=0 tx=0).stop 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 shutdown_connections 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6d47a90 0x55a8d6d49ef0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d377c0 0x55a8d6d35240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d34d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/4243326114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d347c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4ac3fff640 0 client.15738 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 >> 172.21.15.122:0/4243326114 conn(0x55a8d6d289f0 msgr2=0x55a8d6d2a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 shutdown_connections 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 1 -- 172.21.15.122:0/4243326114 wait complete. 2024-04-02T13:35:15.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.741+0000 7f4aed7ab740 20 client.15738 trim_cache size 0 max 16384 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.758+0000 7f4aed7ab740 1 Processor -- start 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.758+0000 7f4aed7ab740 1 -- start start 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d36ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d377c0 0x55a8d6d4f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41af110 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41b0c50 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41af7e0 con 0x55a8d6d377c0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37582/0 (socket says 172.21.15.122:37582) 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 -- 172.21.15.122:0/2702245344 learned_addr learned my addr 172.21.15.122:0/2702245344 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae8887640 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d36ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae9088640 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d377c0 0x55a8d6d4f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 -- 172.21.15.122:0/2702245344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d377c0 msgr2=0x55a8d6d4f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d377c0 0x55a8d6d4f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 -- 172.21.15.122:0/2702245344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 msgr2=0x55a8d6d36ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d36ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 -- 172.21.15.122:0/2702245344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d6d35720 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.759+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce320 tx=0x7f4ad000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ba621b70176b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4ace7fc640 1 -- 172.21.15.122:0/2702245344 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0020040 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4ace7fc640 1 -- 172.21.15.122:0/2702245344 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad0098a70 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4ace7fc640 1 -- 172.21.15.122:0/2702245344 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0010460 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 -- 172.21.15.122:0/2702245344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 msgr2=0x55a8d6d433b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce320 tx=0x7f4ad000bf60 comp rx=0 tx=0).stop 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 -- 172.21.15.122:0/2702245344 shutdown_connections 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d377c0 0x55a8d6d4f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d433b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/2702245344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d36ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 -- 172.21.15.122:0/2702245344 >> 172.21.15.122:0/2702245344 conn(0x55a8d6d289f0 msgr2=0x55a8d6d35010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 -- 172.21.15.122:0/2702245344 shutdown_connections 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.760+0000 7f4aed7ab740 1 -- 172.21.15.122:0/2702245344 wait complete. 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 -- start start 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 Processor -- start 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d3bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d3c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41b0580 con 0x55a8d6d377c0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41afeb0 con 0x55a8d6d2e2c0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae41ad0f0 con 0x55a8d6d2ebe0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4aed7ab740 10 client.0 _ll_register_callbacks cb 0x7f4ad8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4ae9088640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4ae9088640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37598/0 (socket says 172.21.15.122:37598) 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4ae9088640 1 -- 172.21.15.122:0/1532150650 learned_addr learned my addr 172.21.15.122:0/1532150650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.761+0000 7f4ae8887640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d3bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae3fff640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d3c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 -- 172.21.15.122:0/1532150650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 msgr2=0x55a8d6d3c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d3c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 -- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 msgr2=0x55a8d6d3bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d3bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 -- 172.21.15.122:0/1532150650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d6dad620 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ae9088640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4adc036000 tx=0x7f4adc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a23ed6151174d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ac3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc051020 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4adc003df0 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad0098a70 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4adc05c050 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4aed7ab740 10 client.15744 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a8d6dae940 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.762+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad0098a70 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4adc04d840 con 0x55a8d6d377c0 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 10 client.15744 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 20 client.15744 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:15.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6daf110 0x55a8d6db1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 4 client.15744 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8d6c9a000 con 0x55a8d6daf110 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4ae8887640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6daf110 0x55a8d6db1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4ac3fff640 10 client.15744 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.763+0000 7f4ae8887640 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6daf110 0x55a8d6db1530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e17ab06975c072c5 server_cookie=4ef8d1f82763052 in_seq=0 out_seq=0 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4ac3fff640 1 -- 172.21.15.122:0/1532150650 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8d6c9a000 con 0x55a8d6daf110 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4ac3fff640 10 client.15744 handle_command_reply: tid=0 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 client.15744 shutdown 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 msgr2=0x55a8d6dad030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4adc036000 tx=0x7f4adc0095c0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 shutdown_connections 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a8d6daf110 0x55a8d6db1530 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8d6d377c0 0x55a8d6dad030 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8d6d2ebe0 0x55a8d6d3c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 --2- 172.21.15.122:0/1532150650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8d6d2e2c0 0x55a8d6d3bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 >> 172.21.15.122:0/1532150650 conn(0x55a8d6d289f0 msgr2=0x55a8d6d27220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4ac3fff640 0 client.15744 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 shutdown_connections 2024-04-02T13:35:15.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 1 -- 172.21.15.122:0/1532150650 wait complete. 2024-04-02T13:35:15.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.765+0000 7f4aed7ab740 20 client.15744 trim_cache size 0 max 16384 2024-04-02T13:35:15.774 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:35:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "9d355402-c38f-45cd-9b94-5a2a759ab54e", 2024-04-02T13:35:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:35:15.774 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 msgr2=0x7f4ad40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 0x7f4ad40a6270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4adc037040 tx=0x7f4adc009800 comp rx=0 tx=0).stop 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 msgr2=0x7f4ae41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4ad000d950 tx=0x7f4ad000de10 comp rx=0 tx=0).stop 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 shutdown_connections 2024-04-02T13:35:15.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4ad40a3db0 0x7f4ad40a6270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ae4107a90 0x7f4ae419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ae4102440 0x7f4ae41988b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 --2- 172.21.15.122:0/353483643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ae4101a80 0x7f4ae4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 >> 172.21.15.122:0/353483643 conn(0x7f4ae40fd590 msgr2=0x7f4ae40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 shutdown_connections 2024-04-02T13:35:15.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.766+0000 7f4aeaaeb640 1 -- 172.21.15.122:0/353483643 wait complete. 2024-04-02T13:35:15.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi188:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi187:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-04-02T13:35:15.776 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: Updating smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/config/ceph.client.admin.keyring 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4057477945' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/538204695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:35:15.777 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:15 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2272162082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:15.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:35:15.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:35:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- 172.21.15.122:0/1119036079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 msgr2=0x7f7164107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 --2- 172.21.15.122:0/1119036079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f7164107ea0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7160009930 tx=0x7f716002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- 172.21.15.122:0/1119036079 shutdown_connections 2024-04-02T13:35:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 --2- 172.21.15.122:0/1119036079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164102470 0x7f716410e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 --2- 172.21.15.122:0/1119036079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164101ab0 0x7f7164101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 --2- 172.21.15.122:0/1119036079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f7164107ea0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- 172.21.15.122:0/1119036079 >> 172.21.15.122:0/1119036079 conn(0x7f71640fd540 msgr2=0x7f71640ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- 172.21.15.122:0/1119036079 shutdown_connections 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- 172.21.15.122:0/1119036079 wait complete. 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 Processor -- start 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.885+0000 7f716b9af640 1 -- start start 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 0x7f71641983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164102470 0x7f7164198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7164111ea0 con 0x7f7164107ac0 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7164111d20 con 0x7f7164101ab0 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7164112020 con 0x7f7164102470 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37616/0 (socket says 172.21.15.122:37616) 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 -- 172.21.15.122:0/3193372019 learned_addr learned my addr 172.21.15.122:0/3193372019 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716974b640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 0x7f71641983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7168f4a640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164102470 0x7f7164198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 -- 172.21.15.122:0/3193372019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164102470 msgr2=0x7f7164198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164102470 0x7f7164198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 -- 172.21.15.122:0/3193372019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 msgr2=0x7f71641983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 0x7f71641983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 -- 172.21.15.122:0/3193372019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716419d3d0 con 0x7f7164107ac0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716974b640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 0x7f71641983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f7169f4c640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f715c00eb10 tx=0x7f715c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f715a7fc640 1 -- 172.21.15.122:0/3193372019 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c004b20 con 0x7f7164107ac0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f715a7fc640 1 -- 172.21.15.122:0/3193372019 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f715c004cc0 con 0x7f7164107ac0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.886+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716419d6c0 con 0x7f7164107ac0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.887+0000 7f715a7fc640 1 -- 172.21.15.122:0/3193372019 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c010760 con 0x7f7164107ac0 2024-04-02T13:35:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.887+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71641a4f50 con 0x7f7164107ac0 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.889+0000 7f715a7fc640 1 -- 172.21.15.122:0/3193372019 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f715c005020 con 0x7f7164107ac0 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.889+0000 7f715a7fc640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 0x7f71500a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.889+0000 7f715a7fc640 1 -- 172.21.15.122:0/3193372019 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f715c0d11e0 con 0x7f7164107ac0 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.889+0000 7f716974b640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 0x7f71500a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716974b640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 0x7f71500a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7160005ab0 tx=0x7f7160005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 Processor -- start 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 -- start start 2024-04-02T13:35:15.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a28d0 0x55b7c82a2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82abdd0 0x55b7c82ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b7c8206830 con 0x55b7c82a31f0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b7c81ae820 con 0x55b7c82abdd0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.890+0000 7f716e597740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7c82027e0 con 0x55b7c82a28d0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f716974b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a28d0 0x55b7c82a2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37628/0 (socket says 172.21.15.122:37628) 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 -- 172.21.15.122:0/4087212709 learned_addr learned my addr 172.21.15.122:0/4087212709 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7169f4c640 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82abdd0 0x55b7c82ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 -- 172.21.15.122:0/4087212709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a28d0 msgr2=0x55b7c82a2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a28d0 0x55b7c82a2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 -- 172.21.15.122:0/4087212709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82abdd0 msgr2=0x55b7c82ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82abdd0 0x55b7c82ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 -- 172.21.15.122:0/4087212709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c7fe47d0 con 0x55b7c82a31f0 2024-04-02T13:35:15.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f7168f4a640 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7154009c90 tx=0x7f7154011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c99a237826bfb97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f714f7fe640 1 -- 172.21.15.122:0/4087212709 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715401e070 con 0x55b7c82a31f0 2024-04-02T13:35:15.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f714f7fe640 1 -- 172.21.15.122:0/4087212709 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7154019b10 con 0x55b7c82a31f0 2024-04-02T13:35:15.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.891+0000 7f714f7fe640 1 -- 172.21.15.122:0/4087212709 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715400ea70 con 0x55b7c82a31f0 2024-04-02T13:35:15.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 -- 172.21.15.122:0/4087212709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 msgr2=0x55b7c82ab890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7154009c90 tx=0x7f7154011810 comp rx=0 tx=0).stop 2024-04-02T13:35:15.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 -- 172.21.15.122:0/4087212709 shutdown_connections 2024-04-02T13:35:15.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82abdd0 0x55b7c82ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a31f0 0x55b7c82ab890 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 --2- 172.21.15.122:0/4087212709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a28d0 0x55b7c82a2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 -- 172.21.15.122:0/4087212709 >> 172.21.15.122:0/4087212709 conn(0x55b7c829d000 msgr2=0x55b7c829d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 -- 172.21.15.122:0/4087212709 shutdown_connections 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.892+0000 7f716e597740 1 -- 172.21.15.122:0/4087212709 wait complete. 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.893+0000 7f716e597740 1 -- start start 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.893+0000 7f716e597740 1 Processor -- start 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.893+0000 7f716e597740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.893+0000 7f716e597740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.893+0000 7f716e597740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b7c82aebc0 con 0x55b7c82abdd0 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b7c81c1d30 con 0x55b7c82a28d0 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b7c82aed40 con 0x55b7c82a31f0 2024-04-02T13:35:15.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37644/0 (socket says 172.21.15.122:37644) 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 -- 172.21.15.122:0/1281540108 learned_addr learned my addr 172.21.15.122:0/1281540108 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 10 client.0 _ll_register_callbacks cb 0x55b7c81c1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7168f4a640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716974b640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 -- 172.21.15.122:0/1281540108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 msgr2=0x55b7c82a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 -- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 msgr2=0x55b7c82ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 -- 172.21.15.122:0/1281540108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c7f1a3f0 con 0x55b7c82abdd0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f7169f4c640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f715c0ce5c0 tx=0x7f715c09c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9f60066a3876bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f714cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c0a1030 con 0x55b7c82abdd0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f715c0ce790 con 0x55b7c82abdd0 2024-04-02T13:35:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7154019b10 con 0x55b7c82abdd0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f715c0ce990 con 0x55b7c82abdd0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.894+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b7c7eda0f0 con 0x55b7c82abdd0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7154019b10 con 0x55b7c82abdd0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f715c0a5460 con 0x55b7c82abdd0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 10 client.15756 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 20 client.15756 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c82bbd70 0x55b7c82be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 4 client.15756 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c8213d70 con 0x55b7c82bbd70 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.895+0000 7f716974b640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c82bbd70 0x55b7c82be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.896+0000 7f714cff9640 10 client.15756 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.896+0000 7f716974b640 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c82bbd70 0x55b7c82be160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d48bbc40f1def1f server_cookie=877adf1989b0e062 in_seq=0 out_seq=0 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.896+0000 7f714cff9640 1 -- 172.21.15.122:0/1281540108 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7c8213d70 con 0x55b7c82bbd70 2024-04-02T13:35:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.896+0000 7f714cff9640 10 client.15756 handle_command_reply: tid=0 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.896+0000 7f716e597740 1 client.15756 shutdown 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 msgr2=0x55b7c82a8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f715c0ce5c0 tx=0x7f715c09c880 comp rx=0 tx=0).stop 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 shutdown_connections 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c82bbd70 0x55b7c82be160 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82a8f20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f714cff9640 0 client.15756 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 --2- 172.21.15.122:0/1281540108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 >> 172.21.15.122:0/1281540108 conn(0x55b7c829d000 msgr2=0x55b7c829ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 shutdown_connections 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 1 -- 172.21.15.122:0/1281540108 wait complete. 2024-04-02T13:35:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.897+0000 7f716e597740 20 client.15756 trim_cache size 0 max 16384 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 Processor -- start 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 -- start start 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 0x55b7c829b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82abdd0 0x55b7c82ab110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71641af010 con 0x55b7c82a28d0 2024-04-02T13:35:15.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b0b50 con 0x55b7c82a31f0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.914+0000 7f716e597740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71641af6e0 con 0x55b7c82abdd0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37656/0 (socket says 172.21.15.122:37656) 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 -- 172.21.15.122:0/978377142 learned_addr learned my addr 172.21.15.122:0/978377142 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f7168f4a640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 0x55b7c829b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f7169f4c640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82abdd0 0x55b7c82ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 -- 172.21.15.122:0/978377142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82abdd0 msgr2=0x55b7c82ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82abdd0 0x55b7c82ab110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 -- 172.21.15.122:0/978377142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 msgr2=0x55b7c829b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 0x55b7c829b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 -- 172.21.15.122:0/978377142 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c829c860 con 0x55b7c82a28d0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f7168f4a640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 0x55b7c829b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f716974b640 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f716002cfd0 tx=0x7f716003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2380bc83271d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f714effd640 1 -- 172.21.15.122:0/978377142 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71600041c0 con 0x55b7c82a28d0 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f714effd640 1 -- 172.21.15.122:0/978377142 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f716005a070 con 0x55b7c82a28d0 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.915+0000 7f714effd640 1 -- 172.21.15.122:0/978377142 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71600054f0 con 0x55b7c82a28d0 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- 172.21.15.122:0/978377142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 msgr2=0x55b7c82b7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f716002cfd0 tx=0x7f716003f000 comp rx=0 tx=0).stop 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- 172.21.15.122:0/978377142 shutdown_connections 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82abdd0 0x55b7c82ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a31f0 0x55b7c829b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 --2- 172.21.15.122:0/978377142 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82a28d0 0x55b7c82b7ab0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- 172.21.15.122:0/978377142 >> 172.21.15.122:0/978377142 conn(0x55b7c829d000 msgr2=0x55b7c82a7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- 172.21.15.122:0/978377142 shutdown_connections 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- 172.21.15.122:0/978377142 wait complete. 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 -- start start 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.916+0000 7f716e597740 1 Processor -- start 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82afa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82affc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71641b0480 con 0x55b7c82abdd0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71641afdb0 con 0x55b7c82a28d0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71641acff0 con 0x55b7c82a31f0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716e597740 10 client.0 _ll_register_callbacks cb 0x7f715c098f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37664/0 (socket says 172.21.15.122:37664) 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 -- 172.21.15.122:0/2309691346 learned_addr learned my addr 172.21.15.122:0/2309691346 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716974b640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 -- 172.21.15.122:0/2309691346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 msgr2=0x55b7c82affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 -- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 msgr2=0x55b7c82afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f7169f4c640 1 -- 172.21.15.122:0/2309691346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c83212b0 con 0x55b7c82abdd0 2024-04-02T13:35:15.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.917+0000 7f716974b640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f7169f4c640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f715c0049d0 tx=0x7f715c0bd280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2798795c7c1361 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c09ca50 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f715c0ce790 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f716005a070 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f715c0055c0 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 10 client.15762 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b7c83225c0 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f716005a070 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f715c0a5b40 con 0x55b7c82abdd0 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 10 client.15762 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 20 client.15762 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c8322cc0 0x55b7c83250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 4 client.15762 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:35:15.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.918+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7c820e0c0 con 0x55b7c8322cc0 2024-04-02T13:35:15.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716974b640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c8322cc0 0x55b7c83250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:15.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716974b640 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c8322cc0 0x55b7c83250b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db81bb3e56548d37 server_cookie=29282b87f3e3dad4 in_seq=0 out_seq=0 2024-04-02T13:35:15.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f714cff9640 10 client.15762 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f714cff9640 1 -- 172.21.15.122:0/2309691346 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b7c820e0c0 con 0x55b7c8322cc0 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f714cff9640 10 client.15762 handle_command_reply: tid=0 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 client.15762 shutdown 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 msgr2=0x55b7c82b0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f715c0049d0 tx=0x7f715c0bd280 comp rx=0 tx=0).stop 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 shutdown_connections 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b7c8322cc0 0x55b7c83250b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b7c82abdd0 0x55b7c82b0500 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b7c82a31f0 0x55b7c82affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 --2- 172.21.15.122:0/2309691346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b7c82a28d0 0x55b7c82afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 >> 172.21.15.122:0/2309691346 conn(0x55b7c829d000 msgr2=0x55b7c82a7180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.919+0000 7f714cff9640 0 client.15762 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 shutdown_connections 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716e597740 1 -- 172.21.15.122:0/2309691346 wait complete. 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716e597740 20 client.15762 trim_cache size 0 max 16384 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stdout: "9d355402-c38f-45cd-9b94-5a2a759ab54e": { 2024-04-02T13:35:15.928 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "9d355402-c38f-45cd-9b94-5a2a759ab54e", 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:15.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 msgr2=0x7f71500a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 0x7f71500a62f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7160005ab0 tx=0x7f7160005fd0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 msgr2=0x7f716419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f715c00eb10 tx=0x7f715c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 shutdown_connections 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71500a3e30 0x7f71500a62f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7164107ac0 0x7f716419cca0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7164102470 0x7f7164198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 --2- 172.21.15.122:0/3193372019 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7164101ab0 0x7f71641983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 >> 172.21.15.122:0/3193372019 conn(0x7f71640fd540 msgr2=0x7f71640fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 shutdown_connections 2024-04-02T13:35:15.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.920+0000 7f716b9af640 1 -- 172.21.15.122:0/3193372019 wait complete. 2024-04-02T13:35:15.933 INFO:tasks.cephfs.filesystem:scrub status for tag:9d355402-c38f-45cd-9b94-5a2a759ab54e - {'path': '/', 'tag': '9d355402-c38f-45cd-9b94-5a2a759ab54e', 'options': 'recursive,force'} 2024-04-02T13:35:15.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:15.993+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fe45c061f40 con 0x7fe44c0a3ee0 2024-04-02T13:35:16.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.059+0000 7fe459ffb640 1 -- 172.21.15.122:0/1508327411 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe45c061f40 con 0x7fe44c0a3ee0 2024-04-02T13:35:16.062 INFO:teuthology.orchestra.run.smithi122.stdout:Schedule set for path / 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 msgr2=0x7fe44c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 0x7fe44c0a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe45000b480 comp rx=0 tx=0).stop 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 msgr2=0x7fe45c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c198970 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe44800d720 tx=0x7fe44800dbe0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 shutdown_connections 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe44c0a3ee0 0x7fe44c0a63a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe45c1069f0 0x7fe45c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe45c1013a0 0x7fe45c198970 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 --2- 172.21.15.122:0/1508327411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe45c1009e0 0x7fe45c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.061+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 >> 172.21.15.122:0/1508327411 conn(0x7fe45c077b20 msgr2=0x7fe45c06a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.062+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 shutdown_connections 2024-04-02T13:35:16.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.062+0000 7fe462c77640 1 -- 172.21.15.122:0/1508327411 wait complete. 2024-04-02T13:35:16.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 -- 172.21.15.122:0/1855528562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069b20 msgr2=0x7f48a010e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 --2- 172.21.15.122:0/1855528562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069b20 0x7f48a010e990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4898009cd0 tx=0x7f489802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 -- 172.21.15.122:0/1855528562 shutdown_connections 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 --2- 172.21.15.122:0/1855528562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069b20 0x7f48a010e990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 --2- 172.21.15.122:0/1855528562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0069180 0x7f48a00695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 --2- 172.21.15.122:0/1855528562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0107bd0 0x7f48a0107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 -- 172.21.15.122:0/1855528562 >> 172.21.15.122:0/1855528562 conn(0x7f48a0077850 msgr2=0x7f48a0077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 -- 172.21.15.122:0/1855528562 shutdown_connections 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.177+0000 7f48a4b15640 1 -- 172.21.15.122:0/1855528562 wait complete. 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 Processor -- start 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 -- start start 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0069b20 0x7f48a0101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0107bd0 0x7f48a01017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0111ec0 con 0x7f48a0069180 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0111d40 con 0x7f48a0069b20 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f48a4b15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f48a0112040 con 0x7f48a0107bd0 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.178+0000 7f489e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37694/0 (socket says 172.21.15.122:37694) 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 -- 172.21.15.122:0/3479864422 learned_addr learned my addr 172.21.15.122:0/3479864422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:16.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489ed9d640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0107bd0 0x7f48a01017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489dd9b640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0069b20 0x7f48a0101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 -- 172.21.15.122:0/3479864422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0107bd0 msgr2=0x7f48a01017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0107bd0 0x7f48a01017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 -- 172.21.15.122:0/3479864422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0069b20 msgr2=0x7f48a0101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0069b20 0x7f48a0101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 -- 172.21.15.122:0/3479864422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a01a6680 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f489e59c640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9f0 tx=0x7f489400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489400ccc0 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a01a6970 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4894010910 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48940188a0 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.179+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a01a6ec0 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.180+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a004f860 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.181+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f4894010ac0 con 0x7f48a0069180 2024-04-02T13:35:16.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.182+0000 7f488f7fe640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 0x7f48880a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.182+0000 7f489dd9b640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 0x7f48880a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.182+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f48940d1530 con 0x7f48a0069180 2024-04-02T13:35:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.183+0000 7f489dd9b640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 0x7f48880a6350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48a0102760 tx=0x7f489000a2d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.186+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f489409d6d0 con 0x7f48a0069180 2024-04-02T13:35:16.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.392+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f48a00ff3b0 con 0x7f48880a3e90 2024-04-02T13:35:16.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.448+0000 7f488f7fe640 1 -- 172.21.15.122:0/3479864422 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f48a00ff3b0 con 0x7f48880a3e90 2024-04-02T13:35:16.451 INFO:teuthology.orchestra.run.smithi122.stdout:Retention added to path / 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 msgr2=0x7f48880a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 0x7f48880a6350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f48a0102760 tx=0x7f489000a2d0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 msgr2=0x7f48a0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9f0 tx=0x7f489400edf0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 shutdown_connections 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48880a3e90 0x7f48880a6350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48a0107bd0 0x7f48a01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48a0069b20 0x7f48a0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 --2- 172.21.15.122:0/3479864422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48a0069180 0x7f48a0100d20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.450+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 >> 172.21.15.122:0/3479864422 conn(0x7f48a0077850 msgr2=0x7f48a00fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.451+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 shutdown_connections 2024-04-02T13:35:16.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.451+0000 7f48a4b15640 1 -- 172.21.15.122:0/3479864422 wait complete. 2024-04-02T13:35:16.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1601636478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0069180 msgr2=0x7fb8b00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1601636478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0069180 0x7fb8b00695e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009900 tx=0x7fb8a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1601636478 shutdown_connections 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1601636478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069b20 0x7fb8b010e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1601636478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0069180 0x7fb8b00695e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1601636478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0107bf0 0x7fb8b0107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1601636478 >> 172.21.15.122:0/1601636478 conn(0x7fb8b0077850 msgr2=0x7fb8b0077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1601636478 shutdown_connections 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1601636478 wait complete. 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 Processor -- start 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.565+0000 7fb8b65e9640 1 -- start start 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069180 0x7fb8b0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0069b20 0x7fb8b0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0111f70 con 0x7fb8b0107bf0 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0111df0 con 0x7fb8b0069b20 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b65e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b01120f0 con 0x7fb8b0069180 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37704/0 (socket says 172.21.15.122:37704) 2024-04-02T13:35:16.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 -- 172.21.15.122:0/1571297209 learned_addr learned my addr 172.21.15.122:0/1571297209 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8affff640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069180 0x7fb8b0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8af7fe640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0069b20 0x7fb8b0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 -- 172.21.15.122:0/1571297209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069180 msgr2=0x7fb8b0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069180 0x7fb8b0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 -- 172.21.15.122:0/1571297209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0069b20 msgr2=0x7fb8b0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0069b20 0x7fb8b0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 -- 172.21.15.122:0/1571297209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b019d3f0 con 0x7fb8b0107bf0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8b4b86640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800da40 tx=0x7fb8a800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.566+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8017070 con 0x7fb8b0107bf0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.567+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b019d6e0 con 0x7fb8b0107bf0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.567+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8a8003e50 con 0x7fb8b0107bf0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.567+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8010460 con 0x7fb8b0107bf0 2024-04-02T13:35:16.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.567+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b01a4f70 con 0x7fb8b0107bf0 2024-04-02T13:35:16.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.568+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b004f860 con 0x7fb8b0107bf0 2024-04-02T13:35:16.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.569+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7fb8a8003970 con 0x7fb8b0107bf0 2024-04-02T13:35:16.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.569+0000 7fb8ad7fa640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 0x7fb8980a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.570+0000 7fb8affff640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 0x7fb8980a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.570+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb8a80d0950 con 0x7fb8b0107bf0 2024-04-02T13:35:16.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.570+0000 7fb8affff640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 0x7fb8980a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009b50 tx=0x7fb8a4005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.578+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb8a809caf0 con 0x7fb8b0107bf0 2024-04-02T13:35:16.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:16 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1218110994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:16.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:16 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:35:16.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:16 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:16 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:35:16.720 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:16 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:16 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1218110994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:16 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:35:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:16 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:16 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:35:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:16 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:16 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1218110994' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:35:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:16 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:35:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:16 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:16 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:35:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:16 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:16.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.795+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb8b0061f40 con 0x7fb8980a3e60 2024-04-02T13:35:16.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.803+0000 7fb8ad7fa640 1 -- 172.21.15.122:0/1571297209 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb8b0061f40 con 0x7fb8980a3e60 2024-04-02T13:35:16.805 INFO:teuthology.orchestra.run.smithi122.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:35:15", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-02T13:35:16.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 msgr2=0x7fb8980a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 0x7fb8980a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009b50 tx=0x7fb8a4005f90 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 msgr2=0x7fb8b019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800da40 tx=0x7fb8a800df00 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 shutdown_connections 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8980a3e60 0x7fb8980a6320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8b0107bf0 0x7fb8b019ccc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8b0069b20 0x7fb8b0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 --2- 172.21.15.122:0/1571297209 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8b0069180 0x7fb8b0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 >> 172.21.15.122:0/1571297209 conn(0x7fb8b0077850 msgr2=0x7fb8b00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 shutdown_connections 2024-04-02T13:35:16.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.805+0000 7fb8b65e9640 1 -- 172.21.15.122:0/1571297209 wait complete. 2024-04-02T13:35:16.818 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 -- 172.21.15.122:0/3258793309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8069b20 msgr2=0x7f69b810e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/3258793309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8069b20 0x7f69b810e990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009930 tx=0x7f69b002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 -- 172.21.15.122:0/3258793309 shutdown_connections 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/3258793309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8069b20 0x7f69b810e990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/3258793309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069180 0x7f69b80695e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/3258793309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8107bd0 0x7f69b8107fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.919+0000 7f69bfa7b640 1 -- 172.21.15.122:0/3258793309 >> 172.21.15.122:0/3258793309 conn(0x7f69b8077850 msgr2=0x7f69b8077c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- 172.21.15.122:0/3258793309 shutdown_connections 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- 172.21.15.122:0/3258793309 wait complete. 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 Processor -- start 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- start start 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8069180 0x7f69b8198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069b20 0x7f69b8198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8111ec0 con 0x7f69b8107bd0 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8111d40 con 0x7f69b8069b20 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bfa7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69b8112040 con 0x7f69b8069180 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69be018640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69be018640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37720/0 (socket says 172.21.15.122:37720) 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69be018640 1 -- 172.21.15.122:0/4037019389 learned_addr learned my addr 172.21.15.122:0/4037019389 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:16.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.920+0000 7f69bd817640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8069180 0x7f69b8198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69bd016640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069b20 0x7f69b8198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 -- 172.21.15.122:0/4037019389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8069180 msgr2=0x7f69b8198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8069180 0x7f69b8198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 -- 172.21.15.122:0/4037019389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069b20 msgr2=0x7f69b8198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069b20 0x7f69b8198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 -- 172.21.15.122:0/4037019389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b819d1f0 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69be018640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f69b002efd0 tx=0x7f69b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0041070 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69b002ccc0 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69b0003bd0 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b819d480 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.921+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b81a4d70 con 0x7f69b8107bd0 2024-04-02T13:35:16.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.922+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b804f860 con 0x7f69b8107bd0 2024-04-02T13:35:16.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.923+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f69b0044070 con 0x7f69b8107bd0 2024-04-02T13:35:16.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.924+0000 7f69aeffd640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 0x7f69a40a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:16.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.924+0000 7f69bd817640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 0x7f69a40a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:16.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.924+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f69b00f2980 con 0x7f69b8107bd0 2024-04-02T13:35:16.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.924+0000 7f69bd817640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 0x7f69a40a63a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69b4005fd0 tx=0x7f69b4009680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:16.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:16.928+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f69b00bec30 con 0x7f69b8107bd0 2024-04-02T13:35:17.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.129+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f69b8061f40 con 0x7f69a40a3ee0 2024-04-02T13:35:17.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.136+0000 7f69aeffd640 1 -- 172.21.15.122:0/4037019389 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f69b8061f40 con 0x7f69a40a3ee0 2024-04-02T13:35:17.138 INFO:teuthology.orchestra.run.smithi122.stdout:/ 1m 6m3h 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 msgr2=0x7f69a40a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 0x7f69a40a63a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69b4005fd0 tx=0x7f69b4009680 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 msgr2=0x7f69b819cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f69b002efd0 tx=0x7f69b00042f0 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 shutdown_connections 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69a40a3ee0 0x7f69a40a63a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69b8107bd0 0x7f69b819cac0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69b8069b20 0x7f69b8198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 --2- 172.21.15.122:0/4037019389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69b8069180 0x7f69b8198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 >> 172.21.15.122:0/4037019389 conn(0x7f69b8077850 msgr2=0x7f69b80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 shutdown_connections 2024-04-02T13:35:17.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:17.138+0000 7f69bfa7b640 1 -- 172.21.15.122:0/4037019389 wait complete. 2024-04-02T13:35:17.151 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-02T13:35:17.179 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-02T13:35:17.192 INFO:teuthology.run_tasks:Running task workunit... 2024-04-02T13:35:17.248 INFO:tasks.workunit:Pulling workunits from ref 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:35:17.248 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-02T13:35:17.248 INFO:tasks.workunit:timeout=3h 2024-04-02T13:35:17.248 INFO:tasks.workunit:cleanup=True 2024-04-02T13:35:17.248 DEBUG:teuthology.orchestra.run.smithi122:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-04-02 13:33:12.036199612 +0000 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-04-02 13:33:12.036199612 +0000 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-04-02 13:35:07.692260505 +0000 2024-04-02T13:35:17.264 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-04-02 13:33:12.036199612 +0000 2024-04-02T13:35:17.265 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-02T13:35:17.265 DEBUG:teuthology.orchestra.run.smithi122:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-02T13:35:17.337 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 191d963ac9de7322da8aff20b567b3fa458e2ef6 2024-04-02T13:35:17.392 INFO:tasks.workunit.client.0.smithi122.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-02T13:35:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:17 smithi187 ceph-mon[36064]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:17 smithi187 ceph-mon[36064]: from='client.15732 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:17 smithi187 ceph-mon[36064]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:17 smithi188 ceph-mon[36117]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:17 smithi188 ceph-mon[36117]: from='client.15732 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:17 smithi188 ceph-mon[36117]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:17 smithi122 ceph-mon[32114]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:35:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:17 smithi122 ceph-mon[32114]: from='client.15732 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:17 smithi122 ceph-mon[32114]: from='client.15768 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:18 smithi187 ceph-mon[36064]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:18 smithi187 ceph-mon[36064]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:18 smithi188 ceph-mon[36117]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:18 smithi188 ceph-mon[36117]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:18 smithi122 ceph-mon[32114]: from='client.15774 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:18 smithi122 ceph-mon[32114]: from='client.15780 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:35:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:19 smithi187 ceph-mon[36064]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-02T13:35:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:19 smithi188 ceph-mon[36117]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-02T13:35:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:19 smithi122 ceph-mon[32114]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-02T13:35:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:21 smithi187 ceph-mon[36064]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-02T13:35:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:21 smithi188 ceph-mon[36117]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-02T13:35:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:21 smithi122 ceph-mon[32114]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-02T13:35:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:23 smithi187 ceph-mon[36064]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-02T13:35:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:23 smithi188 ceph-mon[36117]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-02T13:35:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:23 smithi122 ceph-mon[32114]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 84 KiB/s wr, 55 op/s 2024-04-02T13:35:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:25 smithi188 ceph-mon[36117]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:25 smithi122 ceph-mon[32114]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:25 smithi187 ceph-mon[36064]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:26 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:35:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:26 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:35:26.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:26 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:35:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:27 smithi188 ceph-mon[36117]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:27 smithi122 ceph-mon[32114]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:27 smithi187 ceph-mon[36064]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-02T13:35:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:29 smithi188 ceph-mon[36117]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-04-02T13:35:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:29 smithi122 ceph-mon[32114]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-04-02T13:35:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:29 smithi187 ceph-mon[36064]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 77 KiB/s wr, 54 op/s 2024-04-02T13:35:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:31 smithi188 ceph-mon[36117]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:35:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:31 smithi122 ceph-mon[32114]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:35:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:31 smithi187 ceph-mon[36064]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-04-02T13:35:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:33 smithi188 ceph-mon[36117]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T13:35:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:33 smithi122 ceph-mon[32114]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T13:35:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:33 smithi187 ceph-mon[36064]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-04-02T13:35:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:35 smithi188 ceph-mon[36117]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:35 smithi122 ceph-mon[32114]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:35 smithi187 ceph-mon[36064]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:37 smithi188 ceph-mon[36117]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:38.367 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:37 smithi187 ceph-mon[36064]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:38.368 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:37 smithi122 ceph-mon[32114]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:39 smithi188 ceph-mon[36117]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:39 smithi122 ceph-mon[32114]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:39 smithi187 ceph-mon[36064]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:41 smithi188 ceph-mon[36117]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:41 smithi122 ceph-mon[32114]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:41 smithi187 ceph-mon[36064]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:43.787 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:43 smithi187 ceph-mon[36064]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:43 smithi122 ceph-mon[32114]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:43 smithi188 ceph-mon[36117]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-02T13:35:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:45 smithi122 ceph-mon[32114]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:45.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:35:45.934 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:35:45.982 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:45 smithi187 ceph-mon[36064]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:45 smithi188 ceph-mon[36117]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.050+0000 7f6a43455640 1 -- 172.21.15.122:0/2169489030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c1069c0 msgr2=0x7f6a3c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.050+0000 7f6a43455640 1 --2- 172.21.15.122:0/2169489030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c106da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009cd0 tx=0x7f6a3402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- 172.21.15.122:0/2169489030 shutdown_connections 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- 172.21.15.122:0/2169489030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c101370 0x7f6a3c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- 172.21.15.122:0/2169489030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 0x7f6a3c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- 172.21.15.122:0/2169489030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c106da0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- 172.21.15.122:0/2169489030 >> 172.21.15.122:0/2169489030 conn(0x7f6a3c076d80 msgr2=0x7f6a3c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- 172.21.15.122:0/2169489030 shutdown_connections 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- 172.21.15.122:0/2169489030 wait complete. 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 Processor -- start 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- start start 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 0x7f6a3c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c110da0 con 0x7f6a3c101370 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c110c20 con 0x7f6a3c1009b0 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.051+0000 7f6a43455640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c110f20 con 0x7f6a3c1069c0 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37730/0 (socket says 172.21.15.122:37730) 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 -- 172.21.15.122:0/477821237 learned_addr learned my addr 172.21.15.122:0/477821237 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a411f1640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 0x7f6a3c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a419f2640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 -- 172.21.15.122:0/477821237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c1069c0 msgr2=0x7f6a3c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 -- 172.21.15.122:0/477821237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 msgr2=0x7f6a3c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 0x7f6a3c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 -- 172.21.15.122:0/477821237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3c19d400 con 0x7f6a3c101370 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a409f0640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9f0 tx=0x7f6a2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a327fc640 1 -- 172.21.15.122:0/477821237 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00cd80 con 0x7f6a3c101370 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3c19d6f0 con 0x7f6a3c101370 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a327fc640 1 -- 172.21.15.122:0/477821237 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c010910 con 0x7f6a3c101370 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a327fc640 1 -- 172.21.15.122:0/477821237 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0188f0 con 0x7f6a3c101370 2024-04-02T13:35:46.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.052+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3c1a4f80 con 0x7f6a3c101370 2024-04-02T13:35:46.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a327fc640 1 -- 172.21.15.122:0/477821237 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f6a2c010ac0 con 0x7f6a3c101370 2024-04-02T13:35:46.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 Processor -- start 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 -- start start 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56057f830830 con 0x56057f8ccf20 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56057f7d8820 con 0x56057f8da1e0 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a46036740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56057f82c7e0 con 0x56057f8cc600 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a327fc640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 0x7f6a280a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a411f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a411f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37740/0 (socket says 172.21.15.122:37740) 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a411f1640 1 -- 172.21.15.122:0/2156160934 learned_addr learned my addr 172.21.15.122:0/2156160934 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a419f2640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 0x7f6a280a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.055+0000 7f6a419f2640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 -- 172.21.15.122:0/2156160934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 msgr2=0x56057f8cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 -- 172.21.15.122:0/2156160934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 msgr2=0x56057f8dc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a327fc640 1 -- 172.21.15.122:0/477821237 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f6a2c0d11f0 con 0x7f6a3c101370 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8dc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 -- 172.21.15.122:0/2156160934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56057f60e7d0 con 0x56057f8ccf20 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a419f2640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009ca0 tx=0x7f6a34009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f448b477d325fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a419f2640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 0x7f6a280a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005e00 tx=0x7f6a38005d50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a277fe640 1 -- 172.21.15.122:0/2156160934 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34043070 con 0x56057f8ccf20 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a277fe640 1 -- 172.21.15.122:0/2156160934 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3402cca0 con 0x56057f8ccf20 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.056+0000 7f6a277fe640 1 -- 172.21.15.122:0/2156160934 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34039040 con 0x56057f8ccf20 2024-04-02T13:35:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- 172.21.15.122:0/2156160934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 msgr2=0x56057f8d9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6a34009ca0 tx=0x7f6a34009880 comp rx=0 tx=0).stop 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- 172.21.15.122:0/2156160934 shutdown_connections 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d9ca0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 --2- 172.21.15.122:0/2156160934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- 172.21.15.122:0/2156160934 >> 172.21.15.122:0/2156160934 conn(0x56057f8c7000 msgr2=0x56057f8c73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- 172.21.15.122:0/2156160934 shutdown_connections 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- 172.21.15.122:0/2156160934 wait complete. 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 -- start start 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.057+0000 7f6a46036740 1 Processor -- start 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8d7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56057f8dcf40 con 0x56057f8ccf20 2024-04-02T13:35:46.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56057f7ebd30 con 0x56057f8da1e0 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56057f8dd0c0 con 0x56057f8cc600 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 10 client.0 _ll_register_callbacks cb 0x56057f7eb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37752/0 (socket says 172.21.15.122:37752) 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 -- 172.21.15.122:0/2088637899 learned_addr learned my addr 172.21.15.122:0/2088637899 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a419f2640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8d7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 -- 172.21.15.122:0/2088637899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 msgr2=0x56057f8d99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 -- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 msgr2=0x56057f8d7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8d7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 -- 172.21.15.122:0/2088637899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057f5443f0 con 0x56057f8ccf20 2024-04-02T13:35:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c098910 tx=0x7f6a2c09c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffe100f81aba6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0b6060 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f6a2c0bd0b0 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3402cca0 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c087070 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 10 client.15792 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.058+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56057f5040f0 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3402cca0 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a2c0ce740 con 0x56057f8ccf20 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 20 client.15792 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f8ea0f0 0x56057f8ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 4 client.15792 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56057f83dd70 con 0x56057f8ea0f0 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.059+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f8ea0f0 0x56057f8ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a24ff9640 10 client.15792 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f8ea0f0 0x56057f8ec580 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937f7f76e117203d server_cookie=7e5522c615696ab6 in_seq=0 out_seq=0 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a24ff9640 1 -- 172.21.15.122:0/2088637899 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56057f83dd70 con 0x56057f8ea0f0 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a24ff9640 10 client.15792 handle_command_reply: tid=0 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 client.15792 shutdown 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 msgr2=0x56057f8d6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c098910 tx=0x7f6a2c09c830 comp rx=0 tx=0).stop 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 shutdown_connections 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f8ea0f0 0x56057f8ec580 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8da1e0 0x56057f8d7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8ccf20 0x56057f8d6d40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 --2- 172.21.15.122:0/2088637899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a24ff9640 0 client.15792 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.060+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 >> 172.21.15.122:0/2088637899 conn(0x56057f8c7000 msgr2=0x56057f8c8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.061+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 shutdown_connections 2024-04-02T13:35:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.061+0000 7f6a46036740 1 -- 172.21.15.122:0/2088637899 wait complete. 2024-04-02T13:35:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.061+0000 7f6a46036740 20 client.15792 trim_cache size 0 max 16384 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.078+0000 7f6a46036740 1 Processor -- start 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.078+0000 7f6a46036740 1 -- start start 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 0x56057f8d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8ccf20 0x56057f8c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1acd30 con 0x56057f8da1e0 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1ae870 con 0x56057f8cc600 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a46036740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1ad400 con 0x56057f8ccf20 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a419f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a419f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37760/0 (socket says 172.21.15.122:37760) 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a419f2640 1 -- 172.21.15.122:0/2242354669 learned_addr learned my addr 172.21.15.122:0/2242354669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 0x56057f8d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a409f0640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8ccf20 0x56057f8c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 -- 172.21.15.122:0/2242354669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8ccf20 msgr2=0x56057f8c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8ccf20 0x56057f8c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 -- 172.21.15.122:0/2242354669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 msgr2=0x56057f8d9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 -- 172.21.15.122:0/2242354669 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56057f8c6e30 con 0x56057f8cc600 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a419f2640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a411f1640 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 0x56057f8d24d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a3402c630 tx=0x7f6a34005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1154b2abd9e9fd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a26ffd640 1 -- 172.21.15.122:0/2242354669 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34039040 con 0x56057f8cc600 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a26ffd640 1 -- 172.21.15.122:0/2242354669 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3402ce50 con 0x56057f8cc600 2024-04-02T13:35:46.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.079+0000 7f6a26ffd640 1 -- 172.21.15.122:0/2242354669 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a34043040 con 0x56057f8cc600 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- 172.21.15.122:0/2242354669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 msgr2=0x56057f8d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 0x56057f8d24d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6a3402c630 tx=0x7f6a34005c50 comp rx=0 tx=0).stop 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- 172.21.15.122:0/2242354669 shutdown_connections 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8ccf20 0x56057f8c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 --2- 172.21.15.122:0/2242354669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8cc600 0x56057f8d24d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- 172.21.15.122:0/2242354669 >> 172.21.15.122:0/2242354669 conn(0x56057f8c7000 msgr2=0x56057f8c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- 172.21.15.122:0/2242354669 shutdown_connections 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- 172.21.15.122:0/2242354669 wait complete. 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.080+0000 7f6a46036740 1 -- start start 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 Processor -- start 2024-04-02T13:35:46.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 0x56057f8d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1ae1a0 con 0x56057f8da1e0 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1adad0 con 0x56057f8ccf20 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a3c1aad10 con 0x56057f8cc600 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a46036740 10 client.0 _ll_register_callbacks cb 0x7f6a2c09ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37776/0 (socket says 172.21.15.122:37776) 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 -- 172.21.15.122:0/3937676578 learned_addr learned my addr 172.21.15.122:0/3937676578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a409f0640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 0x56057f8d40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a411f1640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 -- 172.21.15.122:0/3937676578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 msgr2=0x56057f8d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 -- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 msgr2=0x56057f8d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 0x56057f8d40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a419f2640 1 -- 172.21.15.122:0/3937676578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56057f94f820 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.081+0000 7f6a409f0640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 0x56057f8d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a419f2640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6a38021c40 tx=0x7f6a38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4420d84c802ef818 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3802e060 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f6a38011640 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3402ce50 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6a38035050 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 10 client.15798 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56057f950b70 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3402ce50 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6a38028030 con 0x56057f8da1e0 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:46.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 20 client.15798 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f951320 0x56057f953740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 4 client.15798 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.082+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56057f8380c0 con 0x56057f951320 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a411f1640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f951320 0x56057f953740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a24ff9640 10 client.15798 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a411f1640 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f951320 0x56057f953740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf3f3eb1c5609a87 server_cookie=e6adaac18c981f30 in_seq=0 out_seq=0 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a24ff9640 1 -- 172.21.15.122:0/3937676578 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56057f8380c0 con 0x56057f951320 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a24ff9640 10 client.15798 handle_command_reply: tid=0 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 client.15798 shutdown 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 msgr2=0x56057f8d45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6a38021c40 tx=0x7f6a38004730 comp rx=0 tx=0).stop 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 shutdown_connections 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56057f951320 0x56057f953740 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56057f8da1e0 0x56057f8d45f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56057f8ccf20 0x56057f8d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a24ff9640 0 client.15798 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 --2- 172.21.15.122:0/3937676578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56057f8cc600 0x56057f8d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 >> 172.21.15.122:0/3937676578 conn(0x56057f8c7000 msgr2=0x56057f8c6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 shutdown_connections 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 1 -- 172.21.15.122:0/3937676578 wait complete. 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.083+0000 7f6a46036740 20 client.15798 trim_cache size 0 max 16384 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:35:46.093 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:46.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 msgr2=0x7f6a280a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 0x7f6a280a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a38005e00 tx=0x7f6a38005d50 comp rx=0 tx=0).stop 2024-04-02T13:35:46.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 msgr2=0x7f6a3c198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c00e9f0 tx=0x7f6a2c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 shutdown_connections 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6a280a3e30 0x7f6a280a62f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a3c1069c0 0x7f6a3c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a3c101370 0x7f6a3c198950 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 --2- 172.21.15.122:0/477821237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6a3c1009b0 0x7f6a3c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 >> 172.21.15.122:0/477821237 conn(0x7f6a3c076d80 msgr2=0x7f6a3c06a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 shutdown_connections 2024-04-02T13:35:46.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.084+0000 7f6a43455640 1 -- 172.21.15.122:0/477821237 wait complete. 2024-04-02T13:35:46.098 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:35:46.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:35:46.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 -- 172.21.15.122:0/32489122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb401013c0 msgr2=0x7feb4010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 --2- 172.21.15.122:0/32489122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb401013c0 0x7feb4010d8d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7feb38009cd0 tx=0x7feb3802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 -- 172.21.15.122:0/32489122 shutdown_connections 2024-04-02T13:35:46.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 --2- 172.21.15.122:0/32489122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb401013c0 0x7feb4010d8d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 --2- 172.21.15.122:0/32489122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb40100a00 0x7feb40100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 --2- 172.21.15.122:0/32489122 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 0x7feb40106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 -- 172.21.15.122:0/32489122 >> 172.21.15.122:0/32489122 conn(0x7feb40077b00 msgr2=0x7feb40077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 -- 172.21.15.122:0/32489122 shutdown_connections 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.198+0000 7feb4502a640 1 -- 172.21.15.122:0/32489122 wait complete. 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 Processor -- start 2024-04-02T13:35:46.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 -- start start 2024-04-02T13:35:46.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 0x7feb401996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 0x7feb401a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb40110df0 con 0x7feb40100a00 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb40110c70 con 0x7feb40106a10 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.199+0000 7feb4502a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb40110f70 con 0x7feb401013c0 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37798/0 (socket says 172.21.15.122:37798) 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 -- 172.21.15.122:0/389867908 learned_addr learned my addr 172.21.15.122:0/389867908 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3f59e640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 0x7feb401a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3e59c640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 0x7feb401996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 -- 172.21.15.122:0/389867908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 msgr2=0x7feb401996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 0x7feb401996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 -- 172.21.15.122:0/389867908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 msgr2=0x7feb401a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 0x7feb401a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 -- 172.21.15.122:0/389867908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb401a4770 con 0x7feb40100a00 2024-04-02T13:35:46.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3e59c640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 0x7feb401996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb3ed9d640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9f0 tx=0x7feb3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3400cd80 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb34010910 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401a4940 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34018990 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.200+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a4e60 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.201+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb400688f0 con 0x7feb40100a00 2024-04-02T13:35:46.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.202+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7feb34010ac0 con 0x7feb40100a00 2024-04-02T13:35:46.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.203+0000 7feb27fff640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 0x7feb280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.203+0000 7feb3e59c640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 0x7feb280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.203+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7feb340d1060 con 0x7feb40100a00 2024-04-02T13:35:46.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.204+0000 7feb3e59c640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 0x7feb280a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb300058c0 tx=0x7feb30005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.208+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7feb3409d200 con 0x7feb40100a00 2024-04-02T13:35:46.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.429+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb40108f90 con 0x7feb40100a00 2024-04-02T13:35:46.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.430+0000 7feb27fff640 1 -- 172.21.15.122:0/389867908 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7feb3409cb00 con 0x7feb40100a00 2024-04-02T13:35:46.433 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:35:46.433 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:35:46.434 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 msgr2=0x7feb280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 0x7feb280a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7feb300058c0 tx=0x7feb30005850 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 msgr2=0x7feb401991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7feb3400e9f0 tx=0x7feb3400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 shutdown_connections 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb280a3db0 0x7feb280a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb40106a10 0x7feb401a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb401013c0 0x7feb401996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 --2- 172.21.15.122:0/389867908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb40100a00 0x7feb401991b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 >> 172.21.15.122:0/389867908 conn(0x7feb40077b00 msgr2=0x7feb401026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 shutdown_connections 2024-04-02T13:35:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.432+0000 7feb4502a640 1 -- 172.21.15.122:0/389867908 wait complete. 2024-04-02T13:35:46.447 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 -- 172.21.15.122:0/3191950306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4107b50 msgr2=0x7fb0d4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 --2- 172.21.15.122:0/3191950306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4107b50 0x7fb0d4107f30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009cd0 tx=0x7fb0c402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 -- 172.21.15.122:0/3191950306 shutdown_connections 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 --2- 172.21.15.122:0/3191950306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d40696c0 0x7fb0d410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 --2- 172.21.15.122:0/3191950306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 0x7fb0d4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 --2- 172.21.15.122:0/3191950306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d4107b50 0x7fb0d4107f30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.552+0000 7fb0db78d640 1 -- 172.21.15.122:0/3191950306 >> 172.21.15.122:0/3191950306 conn(0x7fb0d4077b00 msgr2=0x7fb0d4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- 172.21.15.122:0/3191950306 shutdown_connections 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- 172.21.15.122:0/3191950306 wait complete. 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 Processor -- start 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- start start 2024-04-02T13:35:46.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 0x7fb0d41009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 0x7fb0d4101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4111e70 con 0x7fb0d40696c0 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4111cf0 con 0x7fb0d4107b50 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0db78d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4111ff0 con 0x7fb0d4068d00 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0d8d28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0d8d28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37824/0 (socket says 172.21.15.122:37824) 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.553+0000 7fb0d8d28640 1 -- 172.21.15.122:0/2852719502 learned_addr learned my addr 172.21.15.122:0/2852719502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 0x7fb0d4101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d9529640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 0x7fb0d41009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 -- 172.21.15.122:0/2852719502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 msgr2=0x7fb0d41009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 0x7fb0d41009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 -- 172.21.15.122:0/2852719502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 msgr2=0x7fb0d4101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 0x7fb0d4101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 -- 172.21.15.122:0/2852719502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d41a68d0 con 0x7fb0d40696c0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 0x7fb0d4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d27fc640 1 -- 172.21.15.122:0/2852719502 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c800cd80 con 0x7fb0d40696c0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d41a6bc0 con 0x7fb0d40696c0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d27fc640 1 -- 172.21.15.122:0/2852719502 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c8010910 con 0x7fb0d40696c0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0d27fc640 1 -- 172.21.15.122:0/2852719502 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c80188f0 con 0x7fb0d40696c0 2024-04-02T13:35:46.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.554+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a7160 con 0x7fb0d40696c0 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d27fc640 1 -- 172.21.15.122:0/2852719502 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7fb0c8010ac0 con 0x7fb0d40696c0 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 Processor -- start 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 -- start start 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a4015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a401eca0 0x55c4a4021090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4a3f797b0 con 0x55c4a40160c0 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4a3f18570 con 0x55c4a401eca0 2024-04-02T13:35:46.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0de374740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4a3f75760 con 0x55c4a40157a0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37830/0 (socket says 172.21.15.122:37830) 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 -- 172.21.15.122:0/119454896 learned_addr learned my addr 172.21.15.122:0/119454896 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d27fc640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 0x7fb0bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a4015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a401eca0 0x55c4a4021090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 0x7fb0bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 -- 172.21.15.122:0/119454896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 msgr2=0x55c4a4015b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a4015b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 -- 172.21.15.122:0/119454896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a401eca0 msgr2=0x55c4a4021090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a401eca0 0x55c4a4021090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9529640 1 -- 172.21.15.122:0/119454896 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a3c8d3f0 con 0x55c4a40160c0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.557+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a4015b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0d27fc640 1 -- 172.21.15.122:0/2852719502 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb0c80d1220 con 0x7fb0d40696c0 2024-04-02T13:35:46.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0d9529640 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000f80 tx=0x7fb0c40323c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c609652c11771b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 0x7fb0bc0a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009f10 tx=0x7fb0cc004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0bb7fe640 1 -- 172.21.15.122:0/119454896 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4044020 con 0x55c4a40160c0 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0bb7fe640 1 -- 172.21.15.122:0/119454896 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4032c90 con 0x55c4a40160c0 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.558+0000 7fb0bb7fe640 1 -- 172.21.15.122:0/119454896 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4039040 con 0x55c4a40160c0 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- 172.21.15.122:0/119454896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 msgr2=0x55c4a401e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4000f80 tx=0x7fb0c40323c0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- 172.21.15.122:0/119454896 shutdown_connections 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a401eca0 0x55c4a4021090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40160c0 0x55c4a401e760 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 --2- 172.21.15.122:0/119454896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a4015b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- 172.21.15.122:0/119454896 >> 172.21.15.122:0/119454896 conn(0x55c4a400fed0 msgr2=0x55c4a40102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- 172.21.15.122:0/119454896 shutdown_connections 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- 172.21.15.122:0/119454896 wait complete. 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 -- start start 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 Processor -- start 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.559+0000 7fb0de374740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a401ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 0x55c4a401c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4021a30 con 0x55c4a40157a0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4a3f34e80 con 0x55c4a40160c0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4a4021bb0 con 0x55c4a401eca0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 10 client.0 _ll_register_callbacks cb 0x55c4a3f70aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37840/0 (socket says 172.21.15.122:37840) 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 -- 172.21.15.122:0/2962252449 learned_addr learned my addr 172.21.15.122:0/2962252449 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 0x55c4a401c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a401ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 -- 172.21.15.122:0/2962252449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 msgr2=0x55c4a401c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 0x55c4a401c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 -- 172.21.15.122:0/2962252449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 msgr2=0x55c4a401ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a401ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 -- 172.21.15.122:0/2962252449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a3d577d0 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a401ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 0x55c4a401c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0d9529640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4037040 tx=0x7fb0c4003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914702b654ec8cd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c404d070 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb0c4032790 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0c4032c90 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.560+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0c4058050 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 10 client.15816 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4a402daa0 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0c4032c90 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0c4049690 con 0x55c4a40157a0 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 10 client.15816 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 10 client.15816 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 20 client.15816 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a402edf0 0x55c4a40311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 4 client.15816 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a3f86cf0 con 0x55c4a402edf0 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a402edf0 0x55c4a40311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.561+0000 7fb0b8ff9640 10 client.15816 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a402edf0 0x55c4a40311e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=355dbab663a4580 server_cookie=ceb50f9167515914 in_seq=0 out_seq=0 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/2962252449 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4a3f86cf0 con 0x55c4a402edf0 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0b8ff9640 10 client.15816 handle_command_reply: tid=0 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 client.15816 shutdown 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 msgr2=0x55c4a401e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4037040 tx=0x7fb0c4003e40 comp rx=0 tx=0).stop 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 shutdown_connections 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a402edf0 0x55c4a40311e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a401eca0 0x55c4a401c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a401ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0b8ff9640 0 client.15816 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 --2- 172.21.15.122:0/2962252449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a40157a0 0x55c4a401e700 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 >> 172.21.15.122:0/2962252449 conn(0x55c4a400fed0 msgr2=0x55c4a4011c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 shutdown_connections 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.562+0000 7fb0de374740 1 -- 172.21.15.122:0/2962252449 wait complete. 2024-04-02T13:35:46.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.563+0000 7fb0de374740 20 client.15816 trim_cache size 0 max 16384 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 Processor -- start 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 -- start start 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a402a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a40367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41b1160 con 0x55c4a401eca0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41b2ca0 con 0x55c4a40160c0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0de374740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41b1830 con 0x55c4a40157a0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37852/0 (socket says 172.21.15.122:37852) 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/2237231367 learned_addr learned my addr 172.21.15.122:0/2237231367 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a40367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9529640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a402a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/2237231367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 msgr2=0x55c4a402a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a402a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/2237231367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 msgr2=0x55c4a40367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a40367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/2237231367 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a4015ef0 con 0x55c4a401eca0 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.576+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a40367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc021620 tx=0x7fb0cc009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d54b1d7f17ea8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0baffd640 1 -- 172.21.15.122:0/2237231367 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc019070 con 0x55c4a401eca0 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0baffd640 1 -- 172.21.15.122:0/2237231367 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0cc0047a0 con 0x55c4a401eca0 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0baffd640 1 -- 172.21.15.122:0/2237231367 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc01d890 con 0x55c4a401eca0 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0de374740 1 -- 172.21.15.122:0/2237231367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 msgr2=0x55c4a4036d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0de374740 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc021620 tx=0x7fb0cc009610 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0de374740 1 -- 172.21.15.122:0/2237231367 shutdown_connections 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.577+0000 7fb0de374740 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4036d20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40160c0 0x55c4a40367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 --2- 172.21.15.122:0/2237231367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40157a0 0x55c4a402a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- 172.21.15.122:0/2237231367 >> 172.21.15.122:0/2237231367 conn(0x55c4a400fed0 msgr2=0x55c4a401a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- 172.21.15.122:0/2237231367 shutdown_connections 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- 172.21.15.122:0/2237231367 wait complete. 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- start start 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 Processor -- start 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40157a0 0x55c4a40186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40160c0 0x55c4a4018c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41b25d0 con 0x55c4a401eca0 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41b1f00 con 0x55c4a40157a0 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41af140 con 0x55c4a40160c0 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0de374740 10 client.0 _ll_register_callbacks cb 0x7fb0c80ce3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0d9d2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0d9d2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37854/0 (socket says 172.21.15.122:37854) 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/710422835 learned_addr learned my addr 172.21.15.122:0/710422835 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.578+0000 7fb0d9529640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40157a0 0x55c4a40186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d8d28640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40160c0 0x55c4a4018c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/710422835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40160c0 msgr2=0x55c4a4018c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40160c0 0x55c4a4018c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/710422835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40157a0 msgr2=0x55c4a40186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40157a0 0x55c4a40186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 -- 172.21.15.122:0/710422835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a4024f40 con 0x55c4a401eca0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0d9d2a640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc004700 tx=0x7fb0cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ae656de4536045 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc019070 con 0x55c4a401eca0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb0cc0052d0 con 0x55c4a401eca0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0cc0047a0 con 0x55c4a401eca0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb0cc03f050 con 0x55c4a401eca0 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0de374740 10 client.15822 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c4a40952c0 con 0x55c4a401eca0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.579+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0cc0047a0 con 0x55c4a401eca0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb0cc028070 con 0x55c4a401eca0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 10 client.15822 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 20 client.15822 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a40959d0 0x55c4a4097df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 4 client.15822 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4a3f81040 con 0x55c4a40959d0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0d9529640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a40959d0 0x55c4a4097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0b8ff9640 10 client.15822 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0d9529640 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a40959d0 0x55c4a4097df0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3ff7d58b4feae1bd server_cookie=ceb93e3c0c6815e1 in_seq=0 out_seq=0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0b8ff9640 1 -- 172.21.15.122:0/710422835 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4a3f81040 con 0x55c4a40959d0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0b8ff9640 10 client.15822 handle_command_reply: tid=0 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 client.15822 shutdown 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 msgr2=0x55c4a4019170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc004700 tx=0x7fb0cc009b50 comp rx=0 tx=0).stop 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 shutdown_connections 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c4a40959d0 0x55c4a4097df0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4a401eca0 0x55c4a4019170 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4a40160c0 0x55c4a4018c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0b8ff9640 0 client.15822 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 --2- 172.21.15.122:0/710422835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4a40157a0 0x55c4a40186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.580+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 >> 172.21.15.122:0/710422835 conn(0x55c4a400fed0 msgr2=0x55c4a400e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 shutdown_connections 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0de374740 1 -- 172.21.15.122:0/710422835 wait complete. 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0de374740 20 client.15822 trim_cache size 0 max 16384 2024-04-02T13:35:46.589 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:35:46.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 msgr2=0x7fb0bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 0x7fb0bc0a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009f10 tx=0x7fb0cc004730 comp rx=0 tx=0).stop 2024-04-02T13:35:46.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 msgr2=0x7fb0d4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb0c800e9f0 tx=0x7fb0c800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 shutdown_connections 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0bc0a3db0 0x7fb0bc0a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0d4107b50 0x7fb0d4101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0d40696c0 0x7fb0d4100f00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 --2- 172.21.15.122:0/2852719502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0d4068d00 0x7fb0d41009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 >> 172.21.15.122:0/2852719502 conn(0x7fb0d4077b00 msgr2=0x7fb0d40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 shutdown_connections 2024-04-02T13:35:46.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.581+0000 7fb0db78d640 1 -- 172.21.15.122:0/2852719502 wait complete. 2024-04-02T13:35:46.594 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.696+0000 7ff0252dd640 1 -- 172.21.15.122:0/3013316415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 msgr2=0x7ff020100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 --2- 172.21.15.122:0/3013316415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020100df0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff010009930 tx=0x7ff01002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 -- 172.21.15.122:0/3013316415 shutdown_connections 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 --2- 172.21.15.122:0/3013316415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020101330 0x7ff02010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 --2- 172.21.15.122:0/3013316415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020100df0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 --2- 172.21.15.122:0/3013316415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020106980 0x7ff020106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 -- 172.21.15.122:0/3013316415 >> 172.21.15.122:0/3013316415 conn(0x7ff020077b00 msgr2=0x7ff020077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 -- 172.21.15.122:0/3013316415 shutdown_connections 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 -- 172.21.15.122:0/3013316415 wait complete. 2024-04-02T13:35:46.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 Processor -- start 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.697+0000 7ff0252dd640 1 -- start start 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020101330 0x7ff0201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020106980 0x7ff02019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff020110d90 con 0x7ff020100970 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff020110c10 con 0x7ff020101330 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff020110f10 con 0x7ff020106980 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37878/0 (socket says 172.21.15.122:37878) 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 -- 172.21.15.122:0/162358857 learned_addr learned my addr 172.21.15.122:0/162358857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020101330 0x7ff0201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020106980 0x7ff02019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 -- 172.21.15.122:0/162358857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020106980 msgr2=0x7ff02019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020106980 0x7ff02019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 -- 172.21.15.122:0/162358857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020101330 msgr2=0x7ff0201988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020101330 0x7ff0201988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 -- 172.21.15.122:0/162358857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02019d380 con 0x7ff020100970 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff01effd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff01400ea60 tx=0x7ff01400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff007fff640 1 -- 172.21.15.122:0/162358857 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01400cde0 con 0x7ff020100970 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff007fff640 1 -- 172.21.15.122:0/162358857 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0140035f0 con 0x7ff020100970 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02019d670 con 0x7ff020100970 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff007fff640 1 -- 172.21.15.122:0/162358857 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014010670 con 0x7ff020100970 2024-04-02T13:35:46.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.698+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0201a4f00 con 0x7ff020100970 2024-04-02T13:35:46.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.701+0000 7ff007fff640 1 -- 172.21.15.122:0/162358857 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7ff0140037a0 con 0x7ff020100970 2024-04-02T13:35:46.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.701+0000 7ff027f9d740 1 Processor -- start 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.701+0000 7ff027f9d740 1 -- start start 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df41760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df41ca0 0x55a08df44090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a08de9c7b0 con 0x55a08df387a0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a08de3b570 con 0x55a08df390c0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff027f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a08de98760 con 0x55a08df41ca0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff007fff640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 0x7ff0080a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df41760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37888/0 (socket says 172.21.15.122:37888) 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 -- 172.21.15.122:0/25402900 learned_addr learned my addr 172.21.15.122:0/25402900 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 0x7ff0080a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df41ca0 0x55a08df44090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff007fff640 1 -- 172.21.15.122:0/162358857 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff0140d10b0 con 0x7ff020100970 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 -- 172.21.15.122:0/25402900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df41ca0 msgr2=0x55a08df44090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df41ca0 0x55a08df44090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 -- 172.21.15.122:0/25402900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 msgr2=0x55a08df41760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df41760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 -- 172.21.15.122:0/25402900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08dbb03f0 con 0x55a08df387a0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01effd640 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.702+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff018012da0 tx=0x7ff0180107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69596cfdb6a0ee15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 0x7ff0080a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff018005c40 tx=0x7ff01801d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff0057fa640 1 -- 172.21.15.122:0/25402900 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01802a070 con 0x55a08df387a0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff0057fa640 1 -- 172.21.15.122:0/25402900 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018003e50 con 0x55a08df387a0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff0057fa640 1 -- 172.21.15.122:0/25402900 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018004290 con 0x55a08df387a0 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 -- 172.21.15.122:0/25402900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 msgr2=0x55a08df38b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff018012da0 tx=0x7ff0180107a0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 -- 172.21.15.122:0/25402900 shutdown_connections 2024-04-02T13:35:46.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df41ca0 0x55a08df44090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df41760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 --2- 172.21.15.122:0/25402900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df387a0 0x55a08df38b80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 -- 172.21.15.122:0/25402900 >> 172.21.15.122:0/25402900 conn(0x55a08df32ed0 msgr2=0x55a08df332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 -- 172.21.15.122:0/25402900 shutdown_connections 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.703+0000 7ff027f9d740 1 -- 172.21.15.122:0/25402900 wait complete. 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 -- start start 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 Processor -- start 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df387a0 0x55a08df416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 0x55a08df3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a08df44a30 con 0x55a08df41ca0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a08de57e80 con 0x55a08df387a0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a08df44bb0 con 0x55a08df390c0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff027f9d740 10 client.0 _ll_register_callbacks cb 0x55a08de93aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37902/0 (socket says 172.21.15.122:37902) 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 -- 172.21.15.122:0/3280464285 learned_addr learned my addr 172.21.15.122:0/3280464285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01effd640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df387a0 0x55a08df416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 0x55a08df3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 -- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 msgr2=0x55a08df3ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 0x55a08df3ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 -- 172.21.15.122:0/3280464285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df387a0 msgr2=0x55a08df416d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df387a0 0x55a08df416d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01f7fe640 1 -- 172.21.15.122:0/3280464285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08dc7a7d0 con 0x55a08df41ca0 2024-04-02T13:35:46.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.704+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 0x55a08df3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff01800dcf0 tx=0x7ff018003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c05c1960bc07f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01802a070 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff018009dc0 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018003e50 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0180052d0 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 10 client.15834 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a08df50a50 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018003e50 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff018024030 con 0x55a08df41ca0 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 10 client.15834 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 10 client.15834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 20 client.15834 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08df51f10 0x55a08df54300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 4 client.15834 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:46.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.705+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a08dea9cf0 con 0x55a08df51f10 2024-04-02T13:35:46.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.706+0000 7ff01effd640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08df51f10 0x55a08df54300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.706+0000 7ff002ff5640 10 client.15834 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:46.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.706+0000 7ff01effd640 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08df51f10 0x55a08df54300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9f74b404d0194af server_cookie=7d2405ab1b522eaf in_seq=0 out_seq=0 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff002ff5640 1 -- 172.21.15.122:0/3280464285 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a08dea9cf0 con 0x55a08df51f10 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff002ff5640 10 client.15834 handle_command_reply: tid=0 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 client.15834 shutdown 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 msgr2=0x55a08df3f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff01800dcf0 tx=0x7ff018003ba0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 shutdown_connections 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08df51f10 0x55a08df54300 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df3f1a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff002ff5640 0 client.15834 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df390c0 0x55a08df3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 --2- 172.21.15.122:0/3280464285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df387a0 0x55a08df416d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 >> 172.21.15.122:0/3280464285 conn(0x55a08df32ed0 msgr2=0x55a08df34c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 shutdown_connections 2024-04-02T13:35:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 1 -- 172.21.15.122:0/3280464285 wait complete. 2024-04-02T13:35:46.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.707+0000 7ff027f9d740 20 client.15834 trim_cache size 0 max 16384 2024-04-02T13:35:46.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.720+0000 7ff027f9d740 1 Processor -- start 2024-04-02T13:35:46.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.720+0000 7ff027f9d740 1 -- start start 2024-04-02T13:35:46.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201af090 con 0x55a08df41ca0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201b0bd0 con 0x55a08df390c0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff027f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201af760 con 0x55a08df387a0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37906/0 (socket says 172.21.15.122:37906) 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 -- 172.21.15.122:0/1562266555 learned_addr learned my addr 172.21.15.122:0/1562266555 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01effd640 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 -- 172.21.15.122:0/1562266555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 msgr2=0x55a08df4d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 -- 172.21.15.122:0/1562266555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 msgr2=0x55a08df31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 -- 172.21.15.122:0/1562266555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a08df38ef0 con 0x55a08df41ca0 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff0180094c0 tx=0x7ff018005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d59dd28d839f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff004ff9640 1 -- 172.21.15.122:0/1562266555 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01802a040 con 0x55a08df41ca0 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff004ff9640 1 -- 172.21.15.122:0/1562266555 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018012910 con 0x55a08df41ca0 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.721+0000 7ff004ff9640 1 -- 172.21.15.122:0/1562266555 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01802e450 con 0x55a08df41ca0 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- 172.21.15.122:0/1562266555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 msgr2=0x55a08df59960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff0180094c0 tx=0x7ff018005050 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- 172.21.15.122:0/1562266555 shutdown_connections 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df59960 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 --2- 172.21.15.122:0/1562266555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- 172.21.15.122:0/1562266555 >> 172.21.15.122:0/1562266555 conn(0x55a08df32ed0 msgr2=0x55a08df316a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- 172.21.15.122:0/1562266555 shutdown_connections 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- 172.21.15.122:0/1562266555 wait complete. 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 -- start start 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.722+0000 7ff027f9d740 1 Processor -- start 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df453a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201b0500 con 0x55a08df41ca0 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201afe30 con 0x55a08df390c0 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0201ad070 con 0x55a08df387a0 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 10 client.0 _ll_register_callbacks cb 0x7ff018028070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37916/0 (socket says 172.21.15.122:37916) 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2339138791 learned_addr learned my addr 172.21.15.122:0/2339138791 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01e7fc640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01effd640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 msgr2=0x55a08df4a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2339138791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 msgr2=0x55a08df453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2339138791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a08df45f20 con 0x55a08df41ca0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01effd640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff018011010 tx=0x7ff018005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce777658501ae35b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff002ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018040070 con 0x55a08df41ca0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff01802ec50 con 0x55a08df41ca0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.723+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff018012910 con 0x55a08df41ca0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff018049050 con 0x55a08df41ca0 2024-04-02T13:35:46.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 10 client.15840 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55a08dfb8040 con 0x55a08df41ca0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018012910 con 0x55a08df41ca0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff018043020 con 0x55a08df41ca0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 10 client.15840 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 10 client.15840 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 20 client.15840 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08dfb86f0 0x55a08dfbab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 4 client.15840 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a08dea4040 con 0x55a08dfb86f0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.724+0000 7ff01effd640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08dfb86f0 0x55a08dfbab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff002ff5640 10 client.15840 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff01effd640 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08dfb86f0 0x55a08dfbab10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=be5d638d50f2ffdc server_cookie=f54a4f366e0796ff in_seq=0 out_seq=0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff002ff5640 1 -- 172.21.15.122:0/2339138791 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a08dea4040 con 0x55a08dfb86f0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff002ff5640 10 client.15840 handle_command_reply: tid=0 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 client.15840 shutdown 2024-04-02T13:35:46.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 msgr2=0x55a08df458e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff018011010 tx=0x7ff018005f50 comp rx=0 tx=0).stop 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 shutdown_connections 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a08dfb86f0 0x55a08dfbab10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a08df41ca0 0x55a08df458e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff002ff5640 0 client.15840 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a08df390c0 0x55a08df453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 --2- 172.21.15.122:0/2339138791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a08df387a0 0x55a08df4a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 >> 172.21.15.122:0/2339138791 conn(0x55a08df32ed0 msgr2=0x55a08df31f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 shutdown_connections 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 1 -- 172.21.15.122:0/2339138791 wait complete. 2024-04-02T13:35:46.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.725+0000 7ff027f9d740 20 client.15840 trim_cache size 0 max 16384 2024-04-02T13:35:46.735 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:35:46.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 msgr2=0x7ff0080a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 0x7ff0080a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff018005c40 tx=0x7ff01801d000 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 msgr2=0x7ff020198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff01400ea60 tx=0x7ff01400ef20 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 shutdown_connections 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0080a3db0 0x7ff0080a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff020106980 0x7ff02019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff020101330 0x7ff0201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 --2- 172.21.15.122:0/162358857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff020100970 0x7ff020198390 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 >> 172.21.15.122:0/162358857 conn(0x7ff020077b00 msgr2=0x7ff0200ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 shutdown_connections 2024-04-02T13:35:46.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.726+0000 7ff0252dd640 1 -- 172.21.15.122:0/162358857 wait complete. 2024-04-02T13:35:46.738 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:46 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/389867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:46.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.841+0000 7f321b424640 1 -- 172.21.15.122:0/2937263634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214106980 msgr2=0x7f3214106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.841+0000 7f321b424640 1 --2- 172.21.15.122:0/2937263634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214106980 0x7f3214106d60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3210009930 tx=0x7f321002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- 172.21.15.122:0/2937263634 shutdown_connections 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- 172.21.15.122:0/2937263634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 0x7f321410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- 172.21.15.122:0/2937263634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214100970 0x7f3214100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- 172.21.15.122:0/2937263634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214106980 0x7f3214106d60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- 172.21.15.122:0/2937263634 >> 172.21.15.122:0/2937263634 conn(0x7f3214077b00 msgr2=0x7f3214077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- 172.21.15.122:0/2937263634 shutdown_connections 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- 172.21.15.122:0/2937263634 wait complete. 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 Processor -- start 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- start start 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 0x7f3214197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214106980 0x7f321419bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3214110d60 con 0x7f3214100970 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3214110be0 con 0x7f3214101330 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.842+0000 7f321b424640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3214110ee0 con 0x7f3214106980 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37926/0 (socket says 172.21.15.122:37926) 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 -- 172.21.15.122:0/3163372893 learned_addr learned my addr 172.21.15.122:0/3163372893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32189bf640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 0x7f3214197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 -- 172.21.15.122:0/3163372893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214106980 msgr2=0x7f321419bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32199c1640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214106980 0x7f321419bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214106980 0x7f321419bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 -- 172.21.15.122:0/3163372893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 msgr2=0x7f3214197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 0x7f3214197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 -- 172.21.15.122:0/3163372893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321419c2b0 con 0x7f3214100970 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f32191c0640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3210009230 tx=0x7f321002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f320a7fc640 1 -- 172.21.15.122:0/3163372893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210041070 con 0x7f3214100970 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321419c540 con 0x7f3214100970 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f320a7fc640 1 -- 172.21.15.122:0/3163372893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3210004450 con 0x7f3214100970 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f320a7fc640 1 -- 172.21.15.122:0/3163372893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210034440 con 0x7f3214100970 2024-04-02T13:35:46.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.843+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32141a3e30 con 0x7f3214100970 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.846+0000 7f320a7fc640 1 -- 172.21.15.122:0/3163372893 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f3210044070 con 0x7f3214100970 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.846+0000 7f321e005740 1 Processor -- start 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.846+0000 7f321e005740 1 -- start start 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ef1eeb87b0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ef1ee57570 con 0x55ef1ef547a0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f321e005740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef1eeb4760 con 0x55ef1ef550c0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32189bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38370/0 (socket says 172.21.15.122:38370) 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 -- 172.21.15.122:0/4137697239 learned_addr learned my addr 172.21.15.122:0/4137697239 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32191c0640 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f320a7fc640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 0x7f32000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 -- 172.21.15.122:0/4137697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef550c0 msgr2=0x55ef1ef5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 -- 172.21.15.122:0/4137697239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 msgr2=0x55ef1ef60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 -- 172.21.15.122:0/4137697239 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1ebcc3f0 con 0x55ef1ef547a0 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32189bf640 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 0x7f32000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f320a7fc640 1 -- 172.21.15.122:0/3163372893 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f32100f2660 con 0x7f3214100970 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.847+0000 7f32199c1640 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f320c009fb0 tx=0x7f320c00f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad77482c2e983b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f31ff7fe640 1 -- 172.21.15.122:0/4137697239 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c012e40 con 0x55ef1ef547a0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f32199c1640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 0x7f32000a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f320c020000 tx=0x7f320c01d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f31ff7fe640 1 -- 172.21.15.122:0/4137697239 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c01c8e0 con 0x55ef1ef547a0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f31ff7fe640 1 -- 172.21.15.122:0/4137697239 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c010aa0 con 0x55ef1ef547a0 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 -- 172.21.15.122:0/4137697239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 msgr2=0x55ef1ef54b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f320c009fb0 tx=0x7f320c00f7f0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 -- 172.21.15.122:0/4137697239 shutdown_connections 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 --2- 172.21.15.122:0/4137697239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef547a0 0x55ef1ef54b80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 -- 172.21.15.122:0/4137697239 >> 172.21.15.122:0/4137697239 conn(0x55ef1ef4eed0 msgr2=0x55ef1ef4f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 -- 172.21.15.122:0/4137697239 shutdown_connections 2024-04-02T13:35:46.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.848+0000 7f321e005740 1 -- 172.21.15.122:0/4137697239 wait complete. 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 -- start start 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 Processor -- start 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ef1ef60a90 con 0x55ef1ef5dca0 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ef1ee73e80 con 0x55ef1ef550c0 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ef1ef60c10 con 0x55ef1ef547a0 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f321e005740 10 client.0 _ll_register_callbacks cb 0x55ef1eeafaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f32199c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f32199c1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37944/0 (socket says 172.21.15.122:37944) 2024-04-02T13:35:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f32199c1640 1 -- 172.21.15.122:0/3001480165 learned_addr learned my addr 172.21.15.122:0/3001480165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f32189bf640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.849+0000 7f32191c0640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 -- 172.21.15.122:0/3001480165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 msgr2=0x55ef1ef5ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 -- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef550c0 msgr2=0x55ef1ef5b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 -- 172.21.15.122:0/3001480165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1ec967d0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f32199c1640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f320c010770 tx=0x7f320c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a122635d304afc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f31fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c010a90 con 0x55ef1ef5dca0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320c01c8e0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f320c01c400 con 0x55ef1ef5dca0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f320c038050 con 0x55ef1ef5dca0 2024-04-02T13:35:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f321e005740 10 client.15852 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef1ef70fa0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.850+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320c01c8e0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f320c003970 con 0x55ef1ef5dca0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 10 client.15852 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 10 client.15852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 20 client.15852 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1ef722f0 0x55ef1ef746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 4 client.15852 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef1eec5cf0 con 0x55ef1ef722f0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f32191c0640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1ef722f0 0x55ef1ef746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f31fcff9640 10 client.15852 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.851+0000 7f32191c0640 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1ef722f0 0x55ef1ef746e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b41f11bc77115bf0 server_cookie=b15379d7797d41e0 in_seq=0 out_seq=0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f31fcff9640 1 -- 172.21.15.122:0/3001480165 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef1eec5cf0 con 0x55ef1ef722f0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f31fcff9640 10 client.15852 handle_command_reply: tid=0 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 client.15852 shutdown 2024-04-02T13:35:46.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 msgr2=0x55ef1ef5b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f320c010770 tx=0x7f320c0107a0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 shutdown_connections 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b580 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1ef722f0 0x55ef1ef746e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef550c0 0x55ef1ef5b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 --2- 172.21.15.122:0/3001480165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f31fcff9640 0 client.15852 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 >> 172.21.15.122:0/3001480165 conn(0x55ef1ef4eed0 msgr2=0x55ef1ef50c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 shutdown_connections 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 1 -- 172.21.15.122:0/3001480165 wait complete. 2024-04-02T13:35:46.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.852+0000 7f321e005740 20 client.15852 trim_cache size 0 max 16384 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.865+0000 7f321e005740 1 Processor -- start 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 -- start start 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32141ace90 con 0x55ef1ef550c0 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32141ae9d0 con 0x55ef1ef5dca0 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f321e005740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32141ad560 con 0x55ef1ef547a0 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37960/0 (socket says 172.21.15.122:37960) 2024-04-02T13:35:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 -- 172.21.15.122:0/1820184041 learned_addr learned my addr 172.21.15.122:0/1820184041 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32191c0640 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32199c1640 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 -- 172.21.15.122:0/1820184041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 msgr2=0x55ef1ef5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 -- 172.21.15.122:0/1820184041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 msgr2=0x55ef1ef5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 -- 172.21.15.122:0/1820184041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef1ef54ef0 con 0x55ef1ef550c0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.866+0000 7f32189bf640 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f320400b6e0 tx=0x7f320400bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d41f62cd60ae1bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f31feffd640 1 -- 172.21.15.122:0/1820184041 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204003b50 con 0x55ef1ef550c0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f31feffd640 1 -- 172.21.15.122:0/1820184041 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3204020070 con 0x55ef1ef550c0 2024-04-02T13:35:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f31feffd640 1 -- 172.21.15.122:0/1820184041 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320401b660 con 0x55ef1ef550c0 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 -- 172.21.15.122:0/1820184041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 msgr2=0x55ef1ef59d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f320400b6e0 tx=0x7f320400bba0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 -- 172.21.15.122:0/1820184041 shutdown_connections 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1ef5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef59d70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 --2- 172.21.15.122:0/1820184041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.867+0000 7f321e005740 1 -- 172.21.15.122:0/1820184041 >> 172.21.15.122:0/1820184041 conn(0x55ef1ef4eed0 msgr2=0x55ef1ef4df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- 172.21.15.122:0/1820184041 shutdown_connections 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- 172.21.15.122:0/1820184041 wait complete. 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- start start 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 Processor -- start 2024-04-02T13:35:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1efd67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32141ae300 con 0x55ef1ef550c0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32141adc30 con 0x55ef1ef5dca0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32141aae70 con 0x55ef1ef547a0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f321e005740 10 client.0 _ll_register_callbacks cb 0x7f320c01f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f32189bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f32189bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37964/0 (socket says 172.21.15.122:37964) 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f32189bf640 1 -- 172.21.15.122:0/377072494 learned_addr learned my addr 172.21.15.122:0/377072494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f32199c1640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1efd67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.868+0000 7f32191c0640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 -- 172.21.15.122:0/377072494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 msgr2=0x55ef1ef654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 -- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 msgr2=0x55ef1efd67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1efd67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 -- 172.21.15.122:0/377072494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef1efd6d30 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32199c1640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1efd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f32189bf640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3204004320 tx=0x7f3204004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f92c8fddd7a876 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f31fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204011e30 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f3204005030 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3204020070 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3204037050 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f321e005740 10 client.15858 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.869+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ef1efd8000 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3204020070 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f32040046c0 con 0x55ef1ef550c0 2024-04-02T13:35:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 10 client.15858 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 10 client.15858 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 20 client.15858 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1efd86a0 0x55ef1efdaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 4 client.15858 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef1eec0040 con 0x55ef1efd86a0 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f32191c0640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1efd86a0 0x55ef1efdaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f31fcff9640 10 client.15858 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f32191c0640 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1efd86a0 0x55ef1efdaac0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5778924a7bb7c95b server_cookie=178755418dbf8338 in_seq=0 out_seq=0 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f31fcff9640 1 -- 172.21.15.122:0/377072494 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef1eec0040 con 0x55ef1efd86a0 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.870+0000 7f31fcff9640 10 client.15858 handle_command_reply: tid=0 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 client.15858 shutdown 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 msgr2=0x55ef1ef65a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3204004320 tx=0x7f3204004380 comp rx=0 tx=0).stop 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 shutdown_connections 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55ef1efd86a0 0x55ef1efdaac0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ef1ef5dca0 0x55ef1efd67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f31fcff9640 0 client.15858 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef1ef550c0 0x55ef1ef65a10 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 --2- 172.21.15.122:0/377072494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ef1ef547a0 0x55ef1ef654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 >> 172.21.15.122:0/377072494 conn(0x55ef1ef4eed0 msgr2=0x55ef1ef50270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 shutdown_connections 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 1 -- 172.21.15.122:0/377072494 wait complete. 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321e005740 20 client.15858 trim_cache size 0 max 16384 2024-04-02T13:35:46.879 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:35:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 msgr2=0x7f32000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 0x7f32000a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f320c020000 tx=0x7f320c01d000 comp rx=0 tx=0).stop 2024-04-02T13:35:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 msgr2=0x7f32141972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3210009230 tx=0x7f321002cba0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 shutdown_connections 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32000a3db0 0x7f32000a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214106980 0x7f321419bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3214101330 0x7f3214197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.871+0000 7f321b424640 1 --2- 172.21.15.122:0/3163372893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3214100970 0x7f32141972c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.872+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 >> 172.21.15.122:0/3163372893 conn(0x7f3214077b00 msgr2=0x7f3214103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.872+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 shutdown_connections 2024-04-02T13:35:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.872+0000 7f321b424640 1 -- 172.21.15.122:0/3163372893 wait complete. 2024-04-02T13:35:46.885 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:35:46.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:46 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/389867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/1442315364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 msgr2=0x7fd0f810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/1442315364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f810d880 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009930 tx=0x7fd0f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/1442315364 shutdown_connections 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/1442315364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f810d880 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/1442315364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81009b0 0x7fd0f8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/1442315364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81069c0 0x7fd0f8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/1442315364 >> 172.21.15.122:0/1442315364 conn(0x7fd0f8076d80 msgr2=0x7fd0f8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/1442315364 shutdown_connections 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.986+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/1442315364 wait complete. 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 Processor -- start 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 -- start start 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 0x7fd0f8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 0x7fd0f819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110dd0 con 0x7fd0f8101370 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110c50 con 0x7fd0f81009b0 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0ffaa5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8110f50 con 0x7fd0f81069c0 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37988/0 (socket says 172.21.15.122:37988) 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 -- 172.21.15.122:0/3267651350 learned_addr learned my addr 172.21.15.122:0/3267651350 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 0x7fd0f819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 0x7fd0f8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 -- 172.21.15.122:0/3267651350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 msgr2=0x7fd0f819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 0x7fd0f819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 -- 172.21.15.122:0/3267651350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 msgr2=0x7fd0f8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 0x7fd0f8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 -- 172.21.15.122:0/3267651350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f819d370 con 0x7fd0f8101370 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 0x7fd0f819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:46.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 0x7fd0f8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0fd040640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800e9f0 tx=0x7fd0e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.987+0000 7fd0eeffd640 1 -- 172.21.15.122:0/3267651350 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e800cd80 con 0x7fd0f8101370 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.988+0000 7fd0eeffd640 1 -- 172.21.15.122:0/3267651350 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0e8005500 con 0x7fd0f8101370 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.988+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f819d660 con 0x7fd0f8101370 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.988+0000 7fd0eeffd640 1 -- 172.21.15.122:0/3267651350 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8010690 con 0x7fd0f8101370 2024-04-02T13:35:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.988+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f81a4ef0 con 0x7fd0f8101370 2024-04-02T13:35:46.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd0eeffd640 1 -- 172.21.15.122:0/3267651350 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7fd0e80046c0 con 0x7fd0f8101370 2024-04-02T13:35:46.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 Processor -- start 2024-04-02T13:35:46.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 -- start start 2024-04-02T13:35:46.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475039b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd0eeffd640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 0x7fd0e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558474f9d7b0 con 0x558475042ca0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558474f3c570 con 0x5584750397a0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.990+0000 7fd10276c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558474f99760 con 0x55847503a0c0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fd040640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 0x7fd0e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38000/0 (socket says 172.21.15.122:38000) 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 -- 172.21.15.122:0/3408457583 learned_addr learned my addr 172.21.15.122:0/3408457583 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0eeffd640 1 -- 172.21.15.122:0/3267651350 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd0e80d1220 con 0x7fd0f8101370 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475039b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 -- 172.21.15.122:0/3408457583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 msgr2=0x558475042760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 -- 172.21.15.122:0/3408457583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 msgr2=0x558475039b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475039b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 -- 172.21.15.122:0/3408457583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558474cb13f0 con 0x558475042ca0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fd040640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 0x7fd0e40a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80ce580 tx=0x7fd0e801bb30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009a50 tx=0x7fd0f40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af66aa023cbb3ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0e3fff640 1 -- 172.21.15.122:0/3408457583 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4047070 con 0x558475042ca0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0e3fff640 1 -- 172.21.15.122:0/3408457583 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4004030 con 0x558475042ca0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.991+0000 7fd0e3fff640 1 -- 172.21.15.122:0/3408457583 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4003af0 con 0x558475042ca0 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- 172.21.15.122:0/3408457583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 msgr2=0x558475045090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4009a50 tx=0x7fd0f40056d0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- 172.21.15.122:0/3408457583 shutdown_connections 2024-04-02T13:35:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475045090 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 --2- 172.21.15.122:0/3408457583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475039b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- 172.21.15.122:0/3408457583 >> 172.21.15.122:0/3408457583 conn(0x558475033ed0 msgr2=0x5584750342b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- 172.21.15.122:0/3408457583 shutdown_connections 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- 172.21.15.122:0/3408457583 wait complete. 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.992+0000 7fd10276c740 1 -- start start 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 Processor -- start 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584750397a0 0x558475042730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55847503a0c0 0x55847503fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558475045a30 con 0x558475042ca0 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558474f58e80 con 0x55847503a0c0 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558475045bb0 con 0x5584750397a0 2024-04-02T13:35:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd10276c740 10 client.0 _ll_register_callbacks cb 0x558474f94aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38008/0 (socket says 172.21.15.122:38008) 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 -- 172.21.15.122:0/265333081 learned_addr learned my addr 172.21.15.122:0/265333081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fd841640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55847503a0c0 0x55847503fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584750397a0 0x558475042730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 -- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584750397a0 msgr2=0x558475042730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584750397a0 0x558475042730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 -- 172.21.15.122:0/265333081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55847503a0c0 msgr2=0x55847503fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55847503a0c0 0x55847503fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.993+0000 7fd0fe042640 1 -- 172.21.15.122:0/265333081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558474d7b7d0 con 0x558475042ca0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0fe042640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c170 tx=0x7fd0f4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e02e4d327eec013 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4047070 con 0x558475042ca0 2024-04-02T13:35:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd0f402cb00 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4004030 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0f4040460 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 10 client.15870 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558475051af0 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4004030 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0f4034440 con 0x558475042ca0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 10 client.15870 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 10 client.15870 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 20 client.15870 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x558475052e40 0x558475055230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 4 client.15870 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558474faacf0 con 0x558475052e40 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.994+0000 7fd0fd841640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x558475052e40 0x558475055230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.995+0000 7fd0e17fa640 10 client.15870 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.995+0000 7fd0fd841640 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x558475052e40 0x558475055230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9731501bf041e2b9 server_cookie=af0bdda02bd9b0fe in_seq=0 out_seq=0 2024-04-02T13:35:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd0e17fa640 1 -- 172.21.15.122:0/265333081 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558474faacf0 con 0x558475052e40 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd0e17fa640 10 client.15870 handle_command_reply: tid=0 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 client.15870 shutdown 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 msgr2=0x5584750401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402c170 tx=0x7fd0f4005d50 comp rx=0 tx=0).stop 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 shutdown_connections 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x558475052e40 0x558475055230 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x5584750401b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55847503a0c0 0x55847503fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 --2- 172.21.15.122:0/265333081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584750397a0 0x558475042730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd0e17fa640 0 client.15870 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 >> 172.21.15.122:0/265333081 conn(0x558475033ed0 msgr2=0x558475035c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 shutdown_connections 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 1 -- 172.21.15.122:0/265333081 wait complete. 2024-04-02T13:35:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:46.996+0000 7fd10276c740 20 client.15870 trim_cache size 0 max 16384 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 Processor -- start 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 -- start start 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x5584750421f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81aef40 con 0x558475042ca0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81b0a80 con 0x5584750397a0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd10276c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81af610 con 0x55847503a0c0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38014/0 (socket says 172.21.15.122:38014) 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.009+0000 7fd0fe042640 1 -- 172.21.15.122:0/2963482849 learned_addr learned my addr 172.21.15.122:0/2963482849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fd841640 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x5584750421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 -- 172.21.15.122:0/2963482849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 msgr2=0x558475042730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 -- 172.21.15.122:0/2963482849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 msgr2=0x5584750421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x5584750421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 -- 172.21.15.122:0/2963482849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558475039ef0 con 0x558475042ca0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0fe042640 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4004630 tx=0x7fd0f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eee2396cd0efd37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0e37fe640 1 -- 172.21.15.122:0/2963482849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4047030 con 0x558475042ca0 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0e37fe640 1 -- 172.21.15.122:0/2963482849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4048040 con 0x558475042ca0 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.010+0000 7fd0e37fe640 1 -- 172.21.15.122:0/2963482849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f402cae0 con 0x558475042ca0 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- 172.21.15.122:0/2963482849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 msgr2=0x558475058250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd0f4004630 tx=0x7fd0f4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- 172.21.15.122:0/2963482849 shutdown_connections 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475058250 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x558475042730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- 172.21.15.122:0/2963482849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x5584750421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- 172.21.15.122:0/2963482849 >> 172.21.15.122:0/2963482849 conn(0x558475033ed0 msgr2=0x55847503e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- 172.21.15.122:0/2963482849 shutdown_connections 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- 172.21.15.122:0/2963482849 wait complete. 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- start start 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 Processor -- start 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475058420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x5584750557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81b03b0 con 0x558475042ca0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81afce0 con 0x5584750397a0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81acf20 con 0x55847503a0c0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd10276c740 10 client.0 _ll_register_callbacks cb 0x7fd0f4045070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd0fe042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38028/0 (socket says 172.21.15.122:38028) 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.011+0000 7fd0fe042640 1 -- 172.21.15.122:0/3064221563 learned_addr learned my addr 172.21.15.122:0/3064221563 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475058420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x5584750557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 -- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 msgr2=0x5584750557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x5584750557b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 -- 172.21.15.122:0/3064221563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 msgr2=0x558475058420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475058420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 -- 172.21.15.122:0/3064221563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584750562e0 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0fe042640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40045f0 tx=0x7fd0f4004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8811389824f4c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4047030 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd0f402ca80 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f4048040 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0f4034a80 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd10276c740 10 client.15876 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5584750bd230 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.012+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f4048040 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0f4042430 con 0x558475042ca0 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 10 client.15876 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 10 client.15876 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:35:47.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 20 client.15876 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5584750bd6d0 0x5584750bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 4 client.15876 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558474fa5040 con 0x5584750bd6d0 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5584750bd6d0 0x5584750bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0e17fa640 10 client.15876 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0fd841640 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5584750bd6d0 0x5584750bfaf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10994dde3f29b07f server_cookie=4e7925952d417329 in_seq=0 out_seq=0 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0e17fa640 1 -- 172.21.15.122:0/3064221563 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558474fa5040 con 0x5584750bd6d0 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd0e17fa640 10 client.15876 handle_command_reply: tid=0 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 1 client.15876 shutdown 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 msgr2=0x558475055cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.013+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40045f0 tx=0x7fd0f4004190 comp rx=0 tx=0).stop 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 shutdown_connections 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5584750bd6d0 0x5584750bfaf0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558475042ca0 0x558475055cf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55847503a0c0 0x5584750557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0e17fa640 0 client.15876 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 --2- 172.21.15.122:0/3064221563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584750397a0 0x558475058420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 >> 172.21.15.122:0/3064221563 conn(0x558475033ed0 msgr2=0x55847504de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 shutdown_connections 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 1 -- 172.21.15.122:0/3064221563 wait complete. 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd10276c740 20 client.15876 trim_cache size 0 max 16384 2024-04-02T13:35:47.023 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 msgr2=0x7fd0e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 0x7fd0e40a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80ce580 tx=0x7fd0e801bb30 comp rx=0 tx=0).stop 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 msgr2=0x7fd0f81988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd0e800e9f0 tx=0x7fd0e800edf0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 shutdown_connections 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd0e40a3e30 0x7fd0e40a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0f81069c0 0x7fd0f819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0f8101370 0x7fd0f81988c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 --2- 172.21.15.122:0/3267651350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd0f81009b0 0x7fd0f8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 >> 172.21.15.122:0/3267651350 conn(0x7fd0f8076d80 msgr2=0x7fd0f806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 shutdown_connections 2024-04-02T13:35:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.014+0000 7fd0ffaa5640 1 -- 172.21.15.122:0/3267651350 wait complete. 2024-04-02T13:35:47.025 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:46 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/389867908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:35:47.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 -- 172.21.15.122:0/2569363002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 msgr2=0x7f8e5c069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 --2- 172.21.15.122:0/2569363002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c069160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009930 tx=0x7f8e5802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 -- 172.21.15.122:0/2569363002 shutdown_connections 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 --2- 172.21.15.122:0/2569363002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 0x7f8e5c10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 --2- 172.21.15.122:0/2569363002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c069160 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 --2- 172.21.15.122:0/2569363002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 0x7f8e5c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 -- 172.21.15.122:0/2569363002 >> 172.21.15.122:0/2569363002 conn(0x7f8e5c077b00 msgr2=0x7f8e5c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 -- 172.21.15.122:0/2569363002 shutdown_connections 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.129+0000 7f8e6443c640 1 -- 172.21.15.122:0/2569363002 wait complete. 2024-04-02T13:35:47.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 Processor -- start 2024-04-02T13:35:47.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 -- start start 2024-04-02T13:35:47.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 0x7f8e5c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 0x7f8e5c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c111e90 con 0x7f8e5c068ce0 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c111d10 con 0x7f8e5c0696a0 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e6443c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c112010 con 0x7f8e5c107a80 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38052/0 (socket says 172.21.15.122:38052) 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 -- 172.21.15.122:0/1348894552 learned_addr learned my addr 172.21.15.122:0/1348894552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e619d7640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 0x7f8e5c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e629d9640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 0x7f8e5c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 -- 172.21.15.122:0/1348894552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 msgr2=0x7f8e5c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 0x7f8e5c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 -- 172.21.15.122:0/1348894552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 msgr2=0x7f8e5c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 0x7f8e5c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.130+0000 7f8e621d8640 1 -- 172.21.15.122:0/1348894552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c19d3e0 con 0x7f8e5c068ce0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e621d8640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00d770 tx=0x7f8e4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e537fe640 1 -- 172.21.15.122:0/1348894552 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c017070 con 0x7f8e5c068ce0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c19d6d0 con 0x7f8e5c068ce0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e537fe640 1 -- 172.21.15.122:0/1348894552 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4c005500 con 0x7f8e5c068ce0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e537fe640 1 -- 172.21.15.122:0/1348894552 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c004e00 con 0x7f8e5c068ce0 2024-04-02T13:35:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.131+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c1a4f60 con 0x7f8e5c068ce0 2024-04-02T13:35:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.133+0000 7f8e537fe640 1 -- 172.21.15.122:0/1348894552 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f8e4c004fa0 con 0x7f8e5c068ce0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 Processor -- start 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 -- start start 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e537fe640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 0x7f8e480a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eaf2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eafe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56043ea567b0 con 0x56043eafbca0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56043e9f5570 con 0x56043eaf30c0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e67026740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56043ea52760 con 0x56043eaf27a0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e629d9640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 0x7f8e480a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e537fe640 1 -- 172.21.15.122:0/1348894552 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8e4c0d0700 con 0x7f8e5c068ce0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eaf2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e619d7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e629d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eafe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eaf2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:40818/0 (socket says 172.21.15.122:40818) 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e619d7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38490/0 (socket says 172.21.15.122:38490) 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.134+0000 7f8e621d8640 1 -- 172.21.15.122:0/3200696775 learned_addr learned my addr 172.21.15.122:0/3200696775 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 -- 172.21.15.122:0/3200696775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 msgr2=0x56043eaf2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eaf2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 -- 172.21.15.122:0/3200696775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 msgr2=0x56043eafe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eafe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 -- 172.21.15.122:0/3200696775 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043e76a3f0 con 0x56043eaf30c0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e629d9640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 0x7f8e480a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54012af0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009230 tx=0x7f8e58005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bee22054e3fb8ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e50bfe640 1 -- 172.21.15.122:0/3200696775 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58042070 con 0x56043eaf30c0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e50bfe640 1 -- 172.21.15.122:0/3200696775 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e5802c690 con 0x56043eaf30c0 2024-04-02T13:35:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.135+0000 7f8e50bfe640 1 -- 172.21.15.122:0/3200696775 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58004ba0 con 0x56043eaf30c0 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- 172.21.15.122:0/3200696775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 msgr2=0x56043eafb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e58009230 tx=0x7f8e58005bf0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- 172.21.15.122:0/3200696775 shutdown_connections 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eafe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eafb760 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- 172.21.15.122:0/3200696775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eaf2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- 172.21.15.122:0/3200696775 >> 172.21.15.122:0/3200696775 conn(0x56043eaeced0 msgr2=0x56043eaed2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- 172.21.15.122:0/3200696775 shutdown_connections 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- 172.21.15.122:0/3200696775 wait complete. 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- start start 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 Processor -- start 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eafbca0 0x56043eaf91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56043eafea30 con 0x56043eaf27a0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56043ea11e80 con 0x56043eafbca0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56043eafebb0 con 0x56043eaf30c0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e67026740 10 client.0 _ll_register_callbacks cb 0x56043ea4daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.136+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38074/0 (socket says 172.21.15.122:38074) 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e629d9640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eafbca0 0x56043eaf91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 -- 172.21.15.122:0/3411060760 learned_addr learned my addr 172.21.15.122:0/3411060760 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 -- 172.21.15.122:0/3411060760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 msgr2=0x56043eaf8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 -- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eafbca0 msgr2=0x56043eaf91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eafbca0 0x56043eaf91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 -- 172.21.15.122:0/3411060760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043e8347d0 con 0x56043eaf27a0 2024-04-02T13:35:47.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e621d8640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c098b70 tx=0x7f8e4c09bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7b433dd90e0011 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c020040 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8e4c098810 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56043ea63cf0 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e4c09c1a0 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e67026740 10 client.15888 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56043eb0aa60 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.137+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56043ea63cf0 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e4c0a0120 con 0x56043eaf27a0 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 10 client.15888 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 20 client.15888 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb0be70 0x56043eb0e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 4 client.15888 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56043ea5e040 con 0x56043eb0be70 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb0be70 0x56043eb0e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e45ffb640 10 client.15888 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.138+0000 7f8e619d7640 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb0be70 0x56043eb0e280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bceb3d80fc94bad server_cookie=5a81ba8d8bf7b8d6 in_seq=0 out_seq=0 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e45ffb640 1 -- 172.21.15.122:0/3411060760 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56043ea5e040 con 0x56043eb0be70 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e45ffb640 10 client.15888 handle_command_reply: tid=0 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 client.15888 shutdown 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 msgr2=0x56043eafb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c098b70 tx=0x7f8e4c09bbf0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 shutdown_connections 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb0be70 0x56043eb0e280 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eafbca0 0x56043eaf91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 --2- 172.21.15.122:0/3411060760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eaf27a0 0x56043eafb6e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e45ffb640 0 client.15888 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:47.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 >> 172.21.15.122:0/3411060760 conn(0x56043eaeced0 msgr2=0x56043eaeec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 shutdown_connections 2024-04-02T13:35:47.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 1 -- 172.21.15.122:0/3411060760 wait complete. 2024-04-02T13:35:47.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.139+0000 7f8e67026740 20 client.15888 trim_cache size 0 max 16384 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 Processor -- start 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 -- start start 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eb0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eb078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1aeee0 con 0x56043eafbca0 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b0a20 con 0x56043eaf30c0 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e67026740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1af5b0 con 0x56043eaf27a0 2024-04-02T13:35:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e629d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e629d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38090/0 (socket says 172.21.15.122:38090) 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.152+0000 7f8e629d9640 1 -- 172.21.15.122:0/542357895 learned_addr learned my addr 172.21.15.122:0/542357895 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e619d7640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eb078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e621d8640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eb0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 -- 172.21.15.122:0/542357895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 msgr2=0x56043eb0e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eb0e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 -- 172.21.15.122:0/542357895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 msgr2=0x56043eb078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eb078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 -- 172.21.15.122:0/542357895 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56043eaf2ef0 con 0x56043eafbca0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e621d8640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eb0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e629d9640 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e54003c60 tx=0x7f8e5401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12199f1dd7d4ace4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e47fff640 1 -- 172.21.15.122:0/542357895 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004240 con 0x56043eafbca0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e47fff640 1 -- 172.21.15.122:0/542357895 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e5402c070 con 0x56043eafbca0 2024-04-02T13:35:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e47fff640 1 -- 172.21.15.122:0/542357895 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54027d80 con 0x56043eafbca0 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e67026740 1 -- 172.21.15.122:0/542357895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 msgr2=0x56043eb13b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.153+0000 7f8e67026740 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8e54003c60 tx=0x7f8e5401e000 comp rx=0 tx=0).stop 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- 172.21.15.122:0/542357895 shutdown_connections 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eb13b70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf30c0 0x56043eb078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- 172.21.15.122:0/542357895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf27a0 0x56043eb0e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- 172.21.15.122:0/542357895 >> 172.21.15.122:0/542357895 conn(0x56043eaeced0 msgr2=0x56043eaf7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- 172.21.15.122:0/542357895 shutdown_connections 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- 172.21.15.122:0/542357895 wait complete. 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- start start 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 Processor -- start 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 0x56043eaf4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1b0350 con 0x56043eafbca0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1afc80 con 0x56043eaf27a0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c1acec0 con 0x56043eaf30c0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e67026740 10 client.0 _ll_register_callbacks cb 0x7f8e4c0cd9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38104/0 (socket says 172.21.15.122:38104) 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 -- 172.21.15.122:0/2664790298 learned_addr learned my addr 172.21.15.122:0/2664790298 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e621d8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 0x56043eaf4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e619d7640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 -- 172.21.15.122:0/2664790298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 msgr2=0x56043eaf5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 -- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 msgr2=0x56043eaf4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 0x56043eaf4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.154+0000 7f8e629d9640 1 -- 172.21.15.122:0/2664790298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56043eb03070 con 0x56043eafbca0 2024-04-02T13:35:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e619d7640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e621d8640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 0x56043eaf4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e629d9640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009810 tx=0x7f8e54027c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a31dbc175dfe3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004e50 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8e54003f90 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e5402c070 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8e54004190 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 10 client.15894 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56043eb042e0 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e5402c070 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8e5401e070 con 0x56043eafbca0 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 10 client.15894 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 20 client.15894 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:47.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb724a0 0x56043eb74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 4 client.15894 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56043e9db6b0 con 0x56043eb724a0 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.155+0000 7f8e621d8640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb724a0 0x56043eb74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e45ffb640 10 client.15894 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e621d8640 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb724a0 0x56043eb74890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eca991752e76afb6 server_cookie=5247393d3ae25f46 in_seq=0 out_seq=0 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e45ffb640 1 -- 172.21.15.122:0/2664790298 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56043e9db6b0 con 0x56043eb724a0 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e45ffb640 10 client.15894 handle_command_reply: tid=0 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 client.15894 shutdown 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 msgr2=0x56043eaf5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8e54009810 tx=0x7f8e54027c90 comp rx=0 tx=0).stop 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 shutdown_connections 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56043eb724a0 0x56043eb74890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56043eafbca0 0x56043eaf5940 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56043eaf30c0 0x56043eaf5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e45ffb640 0 client.15894 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 --2- 172.21.15.122:0/2664790298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56043eaf27a0 0x56043eaf4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 >> 172.21.15.122:0/2664790298 conn(0x56043eaeced0 msgr2=0x56043eaeb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 shutdown_connections 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 1 -- 172.21.15.122:0/2664790298 wait complete. 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.156+0000 7f8e67026740 20 client.15894 trim_cache size 0 max 16384 2024-04-02T13:35:47.166 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 msgr2=0x7f8e480a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 0x7f8e480a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8e54005fd0 tx=0x7f8e54012af0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 msgr2=0x7f8e5c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00d770 tx=0x7f8e4c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 shutdown_connections 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e480a3db0 0x7f8e480a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5c107a80 0x7f8e5c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5c0696a0 0x7f8e5c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 --2- 172.21.15.122:0/1348894552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5c068ce0 0x7f8e5c1983f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:47.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 >> 172.21.15.122:0/1348894552 conn(0x7f8e5c077b00 msgr2=0x7f8e5c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:47.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 shutdown_connections 2024-04-02T13:35:47.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:47.157+0000 7f8e6443c640 1 -- 172.21.15.122:0/1348894552 wait complete. 2024-04-02T13:35:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:47 smithi122 ceph-mon[32114]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:47.843 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:47 smithi188 ceph-mon[36117]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:47 smithi187 ceph-mon[36064]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-04-02T13:35:48.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:35:48.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:35:48.170 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:35:48.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 -- 172.21.15.122:0/1408593852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688107ab0 msgr2=0x7f5688107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 --2- 172.21.15.122:0/1408593852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688107ab0 0x7f5688107e90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5678009930 tx=0x7f567802bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:48.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 -- 172.21.15.122:0/1408593852 shutdown_connections 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 --2- 172.21.15.122:0/1408593852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688102460 0x7f568810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 --2- 172.21.15.122:0/1408593852 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688101aa0 0x7f5688101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 --2- 172.21.15.122:0/1408593852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688107ab0 0x7f5688107e90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 -- 172.21.15.122:0/1408593852 >> 172.21.15.122:0/1408593852 conn(0x7f56880fd530 msgr2=0x7f56880ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.286+0000 7f568d8b7640 1 -- 172.21.15.122:0/1408593852 shutdown_connections 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.287+0000 7f568d8b7640 1 -- 172.21.15.122:0/1408593852 wait complete. 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.287+0000 7f568d8b7640 1 Processor -- start 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.287+0000 7f568d8b7640 1 -- start start 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 0x7f5688074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688107ab0 0x7f5688074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5688111e90 con 0x7f5688101aa0 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5688111d10 con 0x7f5688102460 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5688112010 con 0x7f5688107ab0 2024-04-02T13:35:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38128/0 (socket says 172.21.15.122:38128) 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 -- 172.21.15.122:0/200071813 learned_addr learned my addr 172.21.15.122:0/200071813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f56877fe640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688107ab0 0x7f5688074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f56867fc640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 0x7f5688074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 -- 172.21.15.122:0/200071813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688107ab0 msgr2=0x7f5688074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688107ab0 0x7f5688074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 -- 172.21.15.122:0/200071813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 msgr2=0x7f5688074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 0x7f5688074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 -- 172.21.15.122:0/200071813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56881a6850 con 0x7f5688101aa0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f56867fc640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 0x7f5688074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f5686ffd640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5678009230 tx=0x7f567802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568c8b5640 1 -- 172.21.15.122:0/200071813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678041070 con 0x7f5688101aa0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56881a6ae0 con 0x7f5688101aa0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568c8b5640 1 -- 172.21.15.122:0/200071813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5678004450 con 0x7f5688101aa0 2024-04-02T13:35:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568c8b5640 1 -- 172.21.15.122:0/200071813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678034440 con 0x7f5688101aa0 2024-04-02T13:35:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.288+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56881a6fd0 con 0x7f5688101aa0 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.290+0000 7f568c8b5640 1 -- 172.21.15.122:0/200071813 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f5678044070 con 0x7f5688101aa0 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 Processor -- start 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f568c8b5640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 0x7f56700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 -- start start 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f56877fe640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 0x7f56700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f568c8b5640 1 -- 172.21.15.122:0/200071813 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f56780f2800 con 0x7f5688101aa0 2024-04-02T13:35:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b80bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578e5ad9770 con 0x5578e5b75be0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5578e5a81960 con 0x5578e5b7e7c0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f569057b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578e5ad58c0 con 0x5578e5b752c0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f56867fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f56867fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38142/0 (socket says 172.21.15.122:38142) 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f56867fc640 1 -- 172.21.15.122:0/1226108819 learned_addr learned my addr 172.21.15.122:0/1226108819 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f5686ffd640 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.291+0000 7f56877fe640 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b80bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 -- 172.21.15.122:0/1226108819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 msgr2=0x5578e5b756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56877fe640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 0x7f56700a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f568000ad10 tx=0x7f568000a2b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 -- 172.21.15.122:0/1226108819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 msgr2=0x5578e5b80bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b80bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 -- 172.21.15.122:0/1226108819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e58b57e0 con 0x5578e5b75be0 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f56867fc640 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f567c009c20 tx=0x7f567c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e06f9b05fca3108 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f566dffb640 1 -- 172.21.15.122:0/1226108819 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c016070 con 0x5578e5b75be0 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f566dffb640 1 -- 172.21.15.122:0/1226108819 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c011950 con 0x5578e5b75be0 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.292+0000 7f566dffb640 1 -- 172.21.15.122:0/1226108819 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c010870 con 0x5578e5b75be0 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- 172.21.15.122:0/1226108819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 msgr2=0x5578e5b7e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f567c009c20 tx=0x7f567c00e6c0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- 172.21.15.122:0/1226108819 shutdown_connections 2024-04-02T13:35:48.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b80bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b7e280 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 --2- 172.21.15.122:0/1226108819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- 172.21.15.122:0/1226108819 >> 172.21.15.122:0/1226108819 conn(0x5578e5b6f9f0 msgr2=0x5578e5b6fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- 172.21.15.122:0/1226108819 shutdown_connections 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- 172.21.15.122:0/1226108819 wait complete. 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 -- start start 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.293+0000 7f569057b740 1 Processor -- start 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 0x5578e5b7bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b7e7c0 0x5578e5b7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578e5b815b0 con 0x5578e5b752c0 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5578e5a94d20 con 0x5578e5b75be0 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578e5b81730 con 0x5578e5b7e7c0 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 10 client.0 _ll_register_callbacks cb 0x5578e5a61a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:48.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38152/0 (socket says 172.21.15.122:38152) 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 -- 172.21.15.122:0/3062781510 learned_addr learned my addr 172.21.15.122:0/3062781510 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 -- 172.21.15.122:0/3062781510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b7e7c0 msgr2=0x5578e5b7c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b7e7c0 0x5578e5b7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 -- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 msgr2=0x5578e5b7bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f56867fc640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 0x5578e5b7bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 0x5578e5b7bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 -- 172.21.15.122:0/3062781510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e5915f70 con 0x5578e5b752c0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f56867fc640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 0x5578e5b7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5686ffd640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f56780be2e0 tx=0x7f56780bea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49b48beab1f4903 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567804b040 con 0x5578e5b752c0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f56780f23c0 con 0x5578e5b752c0 2024-04-02T13:35:48.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c011950 con 0x5578e5b752c0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f56780eff50 con 0x5578e5b752c0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 10 client.15906 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.294+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578e57ad170 con 0x5578e5b752c0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c011950 con 0x5578e5b752c0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f56780c6b30 con 0x5578e5b752c0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 10 client.15906 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 20 client.15906 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5b8ea30 0x5578e5b90e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 4 client.15906 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578e5ae6cb0 con 0x5578e5b8ea30 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.295+0000 7f56867fc640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5b8ea30 0x5578e5b90e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.296+0000 7f5666ffd640 10 client.15906 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.296+0000 7f56867fc640 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5b8ea30 0x5578e5b90e80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2456b0da5d8f5596 server_cookie=7114c25ecf7888b3 in_seq=0 out_seq=0 2024-04-02T13:35:48.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.296+0000 7f5666ffd640 1 -- 172.21.15.122:0/3062781510 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578e5ae6cb0 con 0x5578e5b8ea30 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.296+0000 7f5666ffd640 10 client.15906 handle_command_reply: tid=0 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.296+0000 7f569057b740 1 client.15906 shutdown 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 msgr2=0x5578e5b7b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f56780be2e0 tx=0x7f56780bea50 comp rx=0 tx=0).stop 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 shutdown_connections 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5b8ea30 0x5578e5b90e80 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b7e7c0 0x5578e5b7c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b75be0 0x5578e5b7bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f5666ffd640 0 client.15906 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 --2- 172.21.15.122:0/3062781510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b752c0 0x5578e5b7b750 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 >> 172.21.15.122:0/3062781510 conn(0x5578e5b6f9f0 msgr2=0x5578e5b71740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 shutdown_connections 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 1 -- 172.21.15.122:0/3062781510 wait complete. 2024-04-02T13:35:48.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.297+0000 7f569057b740 20 client.15906 trim_cache size 0 max 16384 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 Processor -- start 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 -- start start 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 0x5578e5b7de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 0x5578e5b8a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b1110 con 0x5578e5b7e7c0 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b2c50 con 0x5578e5b752c0 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f569057b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b17e0 con 0x5578e5b75be0 2024-04-02T13:35:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f56877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f56877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38164/0 (socket says 172.21.15.122:38164) 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f56877fe640 1 -- 172.21.15.122:0/322984712 learned_addr learned my addr 172.21.15.122:0/322984712 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f5686ffd640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 0x5578e5b7de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f56867fc640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 0x5578e5b8a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.314+0000 7f56877fe640 1 -- 172.21.15.122:0/322984712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 msgr2=0x5578e5b8a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56877fe640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 0x5578e5b8a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56877fe640 1 -- 172.21.15.122:0/322984712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 msgr2=0x5578e5b7de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56877fe640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 0x5578e5b7de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56877fe640 1 -- 172.21.15.122:0/322984712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e5b7a720 con 0x5578e5b7e7c0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56867fc640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 0x5578e5b8a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f5686ffd640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 0x5578e5b7de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f56877fe640 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f56800104e0 tx=0x7f5680004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e1ce27a9c77c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f566d7fa640 1 -- 172.21.15.122:0/322984712 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568001d850 con 0x5578e5b7e7c0 2024-04-02T13:35:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f566d7fa640 1 -- 172.21.15.122:0/322984712 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5680021070 con 0x5578e5b7e7c0 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f566d7fa640 1 -- 172.21.15.122:0/322984712 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568001c440 con 0x5578e5b7e7c0 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 -- 172.21.15.122:0/322984712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 msgr2=0x5578e5b96790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f56800104e0 tx=0x7f5680004ab0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 -- 172.21.15.122:0/322984712 shutdown_connections 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b7e7c0 0x5578e5b96790 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b75be0 0x5578e5b8a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 --2- 172.21.15.122:0/322984712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b752c0 0x5578e5b7de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.315+0000 7f569057b740 1 -- 172.21.15.122:0/322984712 >> 172.21.15.122:0/322984712 conn(0x5578e5b6f9f0 msgr2=0x5578e5b6eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- 172.21.15.122:0/322984712 shutdown_connections 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- 172.21.15.122:0/322984712 wait complete. 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- start start 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 Processor -- start 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b2580 con 0x5578e5b75be0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b1eb0 con 0x5578e5b7e7c0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56881af0f0 con 0x5578e5b752c0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f569057b740 10 client.0 _ll_register_callbacks cb 0x7f56780bde90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f56867fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f56867fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38172/0 (socket says 172.21.15.122:38172) 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.316+0000 7f56867fc640 1 -- 172.21.15.122:0/1854608477 learned_addr learned my addr 172.21.15.122:0/1854608477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5686ffd640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56877fe640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b83a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 -- 172.21.15.122:0/1854608477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 msgr2=0x5578e5b869e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b869e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 -- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 msgr2=0x5578e5b83a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b83a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 -- 172.21.15.122:0/1854608477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e5b84240 con 0x5578e5b75be0 2024-04-02T13:35:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f56867fc640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f567c005940 tx=0x7f567c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176b91f3d0c86635 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c02d060 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f567c0283c0 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5680021070 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f567c034050 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f569057b740 10 client.15912 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578e5bf5ba0 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5680021070 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.317+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f567c00f040 con 0x5578e5b75be0 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 10 client.15912 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 20 client.15912 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5bf5ff0 0x5578e5bf83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 4 client.15912 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:35:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5578e5ae1000 con 0x5578e5bf5ff0 2024-04-02T13:35:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f5686ffd640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5bf5ff0 0x5578e5bf83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f5666ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.318+0000 7f5686ffd640 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5bf5ff0 0x5578e5bf83e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7611cd133d558f2 server_cookie=66569ccd67482d1f in_seq=0 out_seq=0 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f5666ffd640 1 -- 172.21.15.122:0/1854608477 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5578e5ae1000 con 0x5578e5bf5ff0 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f5666ffd640 10 client.15912 handle_command_reply: tid=0 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 client.15912 shutdown 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 msgr2=0x5578e5b86f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f567c005940 tx=0x7f567c009500 comp rx=0 tx=0).stop 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 shutdown_connections 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578e5bf5ff0 0x5578e5bf83e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578e5b7e7c0 0x5578e5b83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578e5b75be0 0x5578e5b86f20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 --2- 172.21.15.122:0/1854608477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578e5b752c0 0x5578e5b869e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 >> 172.21.15.122:0/1854608477 conn(0x5578e5b6f9f0 msgr2=0x5578e5b7b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.319+0000 7f5666ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 shutdown_connections 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f569057b740 1 -- 172.21.15.122:0/1854608477 wait complete. 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f569057b740 20 client.15912 trim_cache size 0 max 16384 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "c88bc4e0-798a-4b5e-96eb-bacbfb06df5b", 2024-04-02T13:35:48.328 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:35:48.329 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 msgr2=0x7f56700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 0x7f56700a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f568000ad10 tx=0x7f568000a2b0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 msgr2=0x7f5688073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5678009230 tx=0x7f567802cc70 comp rx=0 tx=0).stop 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 shutdown_connections 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3db0 0x7f56700a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688107ab0 0x7f5688074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688102460 0x7f5688074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 --2- 172.21.15.122:0/200071813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5688101aa0 0x7f5688073e00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 >> 172.21.15.122:0/200071813 conn(0x7f56880fd530 msgr2=0x7f56880fdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 shutdown_connections 2024-04-02T13:35:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.320+0000 7f568d8b7640 1 -- 172.21.15.122:0/200071813 wait complete. 2024-04-02T13:35:48.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:35:48.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:35:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 -- 172.21.15.122:0/2651401650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420107a80 msgr2=0x7f2420107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 --2- 172.21.15.122:0/2651401650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420107a80 0x7f2420107e60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2410009930 tx=0x7f241002bc30 comp rx=0 tx=0).stop 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 -- 172.21.15.122:0/2651401650 shutdown_connections 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 --2- 172.21.15.122:0/2651401650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24200696a0 0x7f242010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 --2- 172.21.15.122:0/2651401650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2420068ce0 0x7f2420069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 --2- 172.21.15.122:0/2651401650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420107a80 0x7f2420107e60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 -- 172.21.15.122:0/2651401650 >> 172.21.15.122:0/2651401650 conn(0x7f2420077b00 msgr2=0x7f2420077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 -- 172.21.15.122:0/2651401650 shutdown_connections 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.435+0000 7f2426241640 1 -- 172.21.15.122:0/2651401650 wait complete. 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 Processor -- start 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 -- start start 2024-04-02T13:35:48.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24200696a0 0x7f24201988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2420107a80 0x7f242019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2420111e90 con 0x7f2420068ce0 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2420111d10 con 0x7f24200696a0 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f2426241640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2420112010 con 0x7f2420107a80 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38184/0 (socket says 172.21.15.122:38184) 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 -- 172.21.15.122:0/941066441 learned_addr learned my addr 172.21.15.122:0/941066441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241effd640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24200696a0 0x7f24201988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 -- 172.21.15.122:0/941066441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2420107a80 msgr2=0x7f242019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2420107a80 0x7f242019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 -- 172.21.15.122:0/941066441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24200696a0 msgr2=0x7f24201988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24200696a0 0x7f24201988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.436+0000 7f241f7fe640 1 -- 172.21.15.122:0/941066441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242019d380 con 0x7f2420068ce0 2024-04-02T13:35:48.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f241f7fe640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2410009230 tx=0x7f241002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f241cff9640 1 -- 172.21.15.122:0/941066441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410041070 con 0x7f2420068ce0 2024-04-02T13:35:48.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f241cff9640 1 -- 172.21.15.122:0/941066441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410004450 con 0x7f2420068ce0 2024-04-02T13:35:48.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242019d610 con 0x7f2420068ce0 2024-04-02T13:35:48.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f241cff9640 1 -- 172.21.15.122:0/941066441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410034440 con 0x7f2420068ce0 2024-04-02T13:35:48.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.437+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a4f00 con 0x7f2420068ce0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.439+0000 7f241cff9640 1 -- 172.21.15.122:0/941066441 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 23) ==== 138823+0+0 (secure 0 0 0) 0x7f2410044070 con 0x7f2420068ce0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.439+0000 7f2428e22740 1 Processor -- start 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.439+0000 7f2428e22740 1 -- start start 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b41f0 0x5645733bc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241cff9640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 0x7f24080a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564573317830 con 0x5645733b38d0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5645732bf820 con 0x5645733bcdd0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2428e22740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645733137e0 con 0x5645733b41f0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38194/0 (socket says 172.21.15.122:38194) 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241effd640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 0x7f24080a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 -- 172.21.15.122:0/1025923594 learned_addr learned my addr 172.21.15.122:0/1025923594 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241cff9640 1 -- 172.21.15.122:0/941066441 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f24100f2790 con 0x7f2420068ce0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241f7fe640 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b41f0 0x5645733bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 -- 172.21.15.122:0/1025923594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b41f0 msgr2=0x5645733bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b41f0 0x5645733bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 -- 172.21.15.122:0/1025923594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 msgr2=0x5645733bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 -- 172.21.15.122:0/1025923594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645730f57d0 con 0x5645733b38d0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241effd640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 0x7f24080a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2420100440 tx=0x7f241400b590 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f241ffff640 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2418009fd0 tx=0x7f2418009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe277f59adbfea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2405ffb640 1 -- 172.21.15.122:0/1025923594 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180150f0 con 0x5645733b38d0 2024-04-02T13:35:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2405ffb640 1 -- 172.21.15.122:0/1025923594 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418010500 con 0x5645733b38d0 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.440+0000 7f2405ffb640 1 -- 172.21.15.122:0/1025923594 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418005260 con 0x5645733b38d0 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- 172.21.15.122:0/1025923594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 msgr2=0x5645733b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2418009fd0 tx=0x7f2418009710 comp rx=0 tx=0).stop 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- 172.21.15.122:0/1025923594 shutdown_connections 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b41f0 0x5645733bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 --2- 172.21.15.122:0/1025923594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b38d0 0x5645733b3cb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- 172.21.15.122:0/1025923594 >> 172.21.15.122:0/1025923594 conn(0x5645733ae000 msgr2=0x5645733ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- 172.21.15.122:0/1025923594 shutdown_connections 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- 172.21.15.122:0/1025923594 wait complete. 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.441+0000 7f2428e22740 1 -- start start 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 Processor -- start 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733b9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645733bfbc0 con 0x5645733b41f0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5645732d2d30 con 0x5645733bcdd0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645733bfd40 con 0x5645733b38d0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f2428e22740 10 client.0 _ll_register_callbacks cb 0x5645732d2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38202/0 (socket says 172.21.15.122:38202) 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 -- 172.21.15.122:0/1663742529 learned_addr learned my addr 172.21.15.122:0/1663742529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241f7fe640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241ffff640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733b9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 -- 172.21.15.122:0/1663742529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 msgr2=0x5645733bc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733bc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 -- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 msgr2=0x5645733b9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733b9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 -- 172.21.15.122:0/1663742529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457302b3f0 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241ffff640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241f7fe640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733bc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f241effd640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f241400ffc0 tx=0x7f24140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d77beb16b33ac57 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.442+0000 7f23feffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414022070 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f241401e860 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2418010500 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f241402e050 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 10 client.15924 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564572feb0f0 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418010500 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2414010b10 con 0x5645733b41f0 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 10 client.15924 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 20 client.15924 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645733ccc80 0x5645733cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 4 client.15924 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.443+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564573324d70 con 0x5645733ccc80 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.444+0000 7f241f7fe640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645733ccc80 0x5645733cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.444+0000 7f241f7fe640 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645733ccc80 0x5645733cf100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3de51f2c26db65 server_cookie=96ca7bedb3f50304 in_seq=0 out_seq=0 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.444+0000 7f23feffd640 10 client.15924 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f23feffd640 1 -- 172.21.15.122:0/1663742529 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564573324d70 con 0x5645733ccc80 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f23feffd640 10 client.15924 handle_command_reply: tid=0 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 client.15924 shutdown 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 msgr2=0x5645733b9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f241400ffc0 tx=0x7f24140046c0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 shutdown_connections 2024-04-02T13:35:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645733ccc80 0x5645733cf100 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733b9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f23feffd640 0 client.15924 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733b9940 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 --2- 172.21.15.122:0/1663742529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733bc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 >> 172.21.15.122:0/1663742529 conn(0x5645733ae000 msgr2=0x5645733afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 shutdown_connections 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 1 -- 172.21.15.122:0/1663742529 wait complete. 2024-04-02T13:35:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.445+0000 7f2428e22740 20 client.15924 trim_cache size 0 max 16384 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 Processor -- start 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 -- start start 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24201aefd0 con 0x5645733b41f0 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24201b0b10 con 0x5645733bcdd0 2024-04-02T13:35:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.462+0000 7f2428e22740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24201af6a0 con 0x5645733b38d0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47270/0 (socket says 172.21.15.122:47270) 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 -- 172.21.15.122:0/2353963999 learned_addr learned my addr 172.21.15.122:0/2353963999 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241f7fe640 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 -- 172.21.15.122:0/2353963999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 msgr2=0x5645733c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 -- 172.21.15.122:0/2353963999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 msgr2=0x5645733bc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 -- 172.21.15.122:0/2353963999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645733ade30 con 0x5645733b41f0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241f7fe640 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f241effd640 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2414005780 tx=0x7f2414005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a0c4838d0b77d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f24057fa640 1 -- 172.21.15.122:0/2353963999 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414036030 con 0x5645733b41f0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f24057fa640 1 -- 172.21.15.122:0/2353963999 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2414005500 con 0x5645733b41f0 2024-04-02T13:35:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.463+0000 7f24057fa640 1 -- 172.21.15.122:0/2353963999 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414022050 con 0x5645733b41f0 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- 172.21.15.122:0/2353963999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 msgr2=0x5645733bc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2414005780 tx=0x7f2414005b90 comp rx=0 tx=0).stop 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- 172.21.15.122:0/2353963999 shutdown_connections 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733bcdd0 0x5645733bc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733b41f0 0x5645733bc070 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- 172.21.15.122:0/2353963999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- 172.21.15.122:0/2353963999 >> 172.21.15.122:0/2353963999 conn(0x5645733ae000 msgr2=0x5645733ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- 172.21.15.122:0/2353963999 shutdown_connections 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- 172.21.15.122:0/2353963999 wait complete. 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- start start 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 Processor -- start 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 0x5645733b7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24201b0440 con 0x5645733bcdd0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24201afd70 con 0x5645733b41f0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24201acfb0 con 0x5645733b38d0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f2428e22740 10 client.0 _ll_register_callbacks cb 0x7f241400fc70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f241ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f241ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47278/0 (socket says 172.21.15.122:47278) 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.464+0000 7f241ffff640 1 -- 172.21.15.122:0/738413661 learned_addr learned my addr 172.21.15.122:0/738413661 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241f7fe640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241effd640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 0x5645733b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 -- 172.21.15.122:0/738413661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 msgr2=0x5645733b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 -- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 msgr2=0x5645733b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 0x5645733b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 -- 172.21.15.122:0/738413661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564573432780 con 0x5645733bcdd0 2024-04-02T13:35:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241effd640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 0x5645733b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f241ffff640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f24180092f0 tx=0x7f2418009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff34c087c402ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418004b20 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2418004cc0 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2414005500 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2418030050 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 10 client.15930 fetch_fsmap learned FSMap version 28 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564573433960 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2414005500 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f241800c070 con 0x5645733bcdd0 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 10 client.15930 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.465+0000 7f2428e22740 20 client.15930 populate_metadata read hostname 'smithi122' 2024-04-02T13:35:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564573433dd0 0x5645734361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f2428e22740 4 client.15930 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56457331f0c0 con 0x564573433dd0 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f241f7fe640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564573433dd0 0x5645734361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f23feffd640 10 client.15930 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f241f7fe640 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564573433dd0 0x5645734361f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec433dedb42433a2 server_cookie=b87c0fc2cff021cb in_seq=0 out_seq=0 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f23feffd640 1 -- 172.21.15.122:0/738413661 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56457331f0c0 con 0x564573433dd0 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f23feffd640 10 client.15930 handle_command_reply: tid=0 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.466+0000 7f2428e22740 1 client.15930 shutdown 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 msgr2=0x564573432240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f24180092f0 tx=0x7f2418009560 comp rx=0 tx=0).stop 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 shutdown_connections 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564573433dd0 0x5645734361f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645733bcdd0 0x564573432240 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f23feffd640 0 client.15930 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:35:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645733b41f0 0x5645733b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 --2- 172.21.15.122:0/738413661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645733b38d0 0x5645733b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 >> 172.21.15.122:0/738413661 conn(0x5645733ae000 msgr2=0x5645733ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 shutdown_connections 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 1 -- 172.21.15.122:0/738413661 wait complete. 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2428e22740 20 client.15930 trim_cache size 0 max 16384 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:35:48.476 INFO:teuthology.orchestra.run.smithi122.stdout: "c88bc4e0-798a-4b5e-96eb-bacbfb06df5b": { 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "c88bc4e0-798a-4b5e-96eb-bacbfb06df5b", 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:35:48.477 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 msgr2=0x7f24080a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 0x7f24080a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2420100440 tx=0x7f241400b590 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 msgr2=0x7f2420198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2410009230 tx=0x7f241002cc70 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 shutdown_connections 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f24080a3db0 0x7f24080a6270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2420107a80 0x7f242019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f24200696a0 0x7f24201988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 --2- 172.21.15.122:0/941066441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2420068ce0 0x7f2420198390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 >> 172.21.15.122:0/941066441 conn(0x7f2420077b00 msgr2=0x7f24200fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 shutdown_connections 2024-04-02T13:35:48.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:35:48.467+0000 7f2426241640 1 -- 172.21.15.122:0/941066441 wait complete. 2024-04-02T13:35:48.480 INFO:tasks.cephfs.filesystem:scrub status for tag:c88bc4e0-798a-4b5e-96eb-bacbfb06df5b - {'path': '/', 'tag': 'c88bc4e0-798a-4b5e-96eb-bacbfb06df5b', 'options': 'recursive,force'} 2024-04-02T13:35:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:48 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:35:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:48 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:48 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:35:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:48 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:48 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:35:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:48 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:48 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:35:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:48 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:49.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:48 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:35:49.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:48 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:49.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:48 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:35:49.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:48 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:35:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:49 smithi122 ceph-mon[32114]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T13:35:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:49 smithi187 ceph-mon[36064]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T13:35:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:49 smithi188 ceph-mon[36117]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-04-02T13:35:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:51 smithi122 ceph-mon[32114]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:51 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:35:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:51 smithi187 ceph-mon[36064]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:51 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:35:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:51 smithi188 ceph-mon[36117]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:35:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:51 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:35:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:53 smithi122 ceph-mon[32114]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T13:35:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:53 smithi187 ceph-mon[36064]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T13:35:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:53 smithi188 ceph-mon[36117]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-02T13:35:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:55 smithi187 ceph-mon[36064]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-02T13:35:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:55 smithi122 ceph-mon[32114]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-02T13:35:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:55 smithi188 ceph-mon[36117]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-02T13:35:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:57 smithi122 ceph-mon[32114]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T13:35:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:57 smithi187 ceph-mon[36064]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T13:35:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:57 smithi188 ceph-mon[36117]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-02T13:35:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:35:59 smithi122 ceph-mon[32114]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-02T13:35:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:35:59 smithi187 ceph-mon[36064]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-02T13:36:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:35:59 smithi188 ceph-mon[36117]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-04-02T13:36:00.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:36:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:36:00.457+0000 7f567be29700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:00.820 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:36:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:36:00.457+0000 7f567be29700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:00.821 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:36:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:36:00.457+0000 7f567be29700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:00.821 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:36:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:36:00.457+0000 7f567be29700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:00.821 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:36:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:36:00.457+0000 7f567be29700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-02T13:36:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:01 smithi122 ceph-mon[32114]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-02T13:36:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:01 smithi187 ceph-mon[36064]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-02T13:36:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:01 smithi188 ceph-mon[36117]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-02T13:36:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:03 smithi122 ceph-mon[32114]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:36:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:03 smithi122 ceph-mon[32114]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-04-02T13:36:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:03 smithi187 ceph-mon[36064]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:36:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:03 smithi187 ceph-mon[36064]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-04-02T13:36:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:03 smithi188 ceph-mon[36117]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-02T13:36:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:03 smithi188 ceph-mon[36117]: mgrmap e24: x(active, since 52s), standbys: z, y 2024-04-02T13:36:05.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:05 smithi188 ceph-mon[36117]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:05 smithi122 ceph-mon[32114]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:05 smithi187 ceph-mon[36064]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:07 smithi122 ceph-mon[32114]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T13:36:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:07 smithi187 ceph-mon[36064]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T13:36:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:07 smithi188 ceph-mon[36117]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-04-02T13:36:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:09 smithi122 ceph-mon[32114]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-02T13:36:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:09 smithi187 ceph-mon[36064]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-02T13:36:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:09 smithi188 ceph-mon[36117]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 26 op/s 2024-04-02T13:36:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:11 smithi122 ceph-mon[32114]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:11 smithi187 ceph-mon[36064]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:11 smithi188 ceph-mon[36117]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-02T13:36:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:13 smithi187 ceph-mon[36064]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:36:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:13 smithi188 ceph-mon[36117]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:36:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:13 smithi122 ceph-mon[32114]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-02T13:36:15.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:15 smithi122 ceph-mon[32114]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:36:15.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:15.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:15.873 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:15 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:15 smithi187 ceph-mon[36064]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:36:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:15 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:15 smithi188 ceph-mon[36117]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-02T13:36:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:36:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:15 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:36:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:36:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:36:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:17 smithi187 ceph-mon[36064]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:36:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:17 smithi188 ceph-mon[36117]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:36:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:17 smithi122 ceph-mon[32114]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-02T13:36:18.481 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:36:18.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 -- 172.21.15.122:0/1843463360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c106980 msgr2=0x7f5b8c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 --2- 172.21.15.122:0/1843463360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c106980 0x7f5b8c106d60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009930 tx=0x7f5b8802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 -- 172.21.15.122:0/1843463360 shutdown_connections 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 --2- 172.21.15.122:0/1843463360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c101330 0x7f5b8c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 --2- 172.21.15.122:0/1843463360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c100970 0x7f5b8c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 --2- 172.21.15.122:0/1843463360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c106980 0x7f5b8c106d60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 -- 172.21.15.122:0/1843463360 >> 172.21.15.122:0/1843463360 conn(0x7f5b8c077b00 msgr2=0x7f5b8c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 -- 172.21.15.122:0/1843463360 shutdown_connections 2024-04-02T13:36:18.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.641+0000 7f5b944d3640 1 -- 172.21.15.122:0/1843463360 wait complete. 2024-04-02T13:36:18.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.642+0000 7f5b944d3640 1 Processor -- start 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.642+0000 7f5b944d3640 1 -- start start 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c101330 0x7f5b8c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 0x7f5b8c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c110de0 con 0x7f5b8c100970 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c110c60 con 0x7f5b8c101330 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b944d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c110f60 con 0x7f5b8c106980 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59182/0 (socket says 172.21.15.122:59182) 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 -- 172.21.15.122:0/3240690307 learned_addr learned my addr 172.21.15.122:0/3240690307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b92a70640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 0x7f5b8c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c101330 0x7f5b8c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 -- 172.21.15.122:0/3240690307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 msgr2=0x7f5b8c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 0x7f5b8c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 -- 172.21.15.122:0/3240690307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c101330 msgr2=0x7f5b8c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c101330 0x7f5b8c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b9226f640 1 -- 172.21.15.122:0/3240690307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8c19c290 con 0x7f5b8c100970 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.643+0000 7f5b92a70640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 0x7f5b8c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009900 tx=0x7f5b88004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b837fe640 1 -- 172.21.15.122:0/3240690307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88046070 con 0x7f5b8c100970 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8c19c520 con 0x7f5b8c100970 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b837fe640 1 -- 172.21.15.122:0/3240690307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b8802cc90 con 0x7f5b8c100970 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b837fe640 1 -- 172.21.15.122:0/3240690307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880035f0 con 0x7f5b8c100970 2024-04-02T13:36:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.644+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b8c068c60 con 0x7f5b8c100970 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.646+0000 7f5b837fe640 1 -- 172.21.15.122:0/3240690307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f5b88049070 con 0x7f5b8c100970 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.647+0000 7f5b837fe640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 0x7f5b780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.647+0000 7f5b837fe640 1 -- 172.21.15.122:0/3240690307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5b880f69c0 con 0x7f5b8c100970 2024-04-02T13:36:18.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.647+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 0x7f5b780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 0x7f5b780a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c104ff0 tx=0x7f5b7c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 Processor -- start 2024-04-02T13:36:18.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 -- start start 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be74890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56164bdcf830 con 0x56164be6b8d0 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56164bd77820 con 0x56164be74dd0 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.648+0000 7f5b97190740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56164bdcb7e0 con 0x56164be6c1f0 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59192/0 (socket says 172.21.15.122:59192) 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 -- 172.21.15.122:0/3526303427 learned_addr learned my addr 172.21.15.122:0/3526303427 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be74890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b92a70640 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 -- 172.21.15.122:0/3526303427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 msgr2=0x56164be74890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be74890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 -- 172.21.15.122:0/3526303427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 msgr2=0x56164be771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 -- 172.21.15.122:0/3526303427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164bbad7d0 con 0x56164be6b8d0 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b9226f640 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b880c2190 tx=0x7f5b88040b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc69bc9492299e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b80bfe640 1 -- 172.21.15.122:0/3526303427 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88050040 con 0x56164be6b8d0 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b80bfe640 1 -- 172.21.15.122:0/3526303427 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b880c2b90 con 0x56164be6b8d0 2024-04-02T13:36:18.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.649+0000 7f5b80bfe640 1 -- 172.21.15.122:0/3526303427 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b88040950 con 0x56164be6b8d0 2024-04-02T13:36:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 -- 172.21.15.122:0/3526303427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 msgr2=0x56164be6bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5b880c2190 tx=0x7f5b88040b20 comp rx=0 tx=0).stop 2024-04-02T13:36:18.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 -- 172.21.15.122:0/3526303427 shutdown_connections 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be74890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 --2- 172.21.15.122:0/3526303427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6b8d0 0x56164be6bcb0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.650+0000 7f5b97190740 1 -- 172.21.15.122:0/3526303427 >> 172.21.15.122:0/3526303427 conn(0x56164be66000 msgr2=0x56164be663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 -- 172.21.15.122:0/3526303427 shutdown_connections 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 -- 172.21.15.122:0/3526303427 wait complete. 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 -- start start 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 Processor -- start 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.651+0000 7f5b97190740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 0x56164be80d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b97190740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56164be77980 con 0x56164be6c1f0 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b97190740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56164bd8ad30 con 0x56164be74dd0 2024-04-02T13:36:18.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b97190740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56164be77b00 con 0x56164be72630 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59204/0 (socket says 172.21.15.122:59204) 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b92a70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 0x56164be80d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 -- 172.21.15.122:0/1250163843 learned_addr learned my addr 172.21.15.122:0/1250163843 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b97190740 10 client.0 _ll_register_callbacks cb 0x56164bd8a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 -- 172.21.15.122:0/1250163843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 msgr2=0x56164be80d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 0x56164be80d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 -- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 msgr2=0x56164be720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 -- 172.21.15.122:0/1250163843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164bae33f0 con 0x56164be6c1f0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b92a70640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 0x56164be80d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c820 tx=0x7f5b88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbc85cbb0f99551 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.652+0000 7f5b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:18.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880db060 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b880c2b90 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5b880caea0 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b88050070 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b97190740 10 client.15948 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56164baa30f0 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.653+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b880c2b90 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b880f4000 con 0x56164be6c1f0 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 10 client.15948 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 20 client.15948 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be84f60 0x56164be873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 4 client.15948 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56164bddcd70 con 0x56164be84f60 2024-04-02T13:36:18.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.654+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be84f60 0x56164be873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.655+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be84f60 0x56164be873d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342b344e30420301 server_cookie=69cd8d3ec4b38456 in_seq=0 out_seq=0 2024-04-02T13:36:18.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.655+0000 7f5b75ffb640 10 client.15948 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1250163843 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56164bddcd70 con 0x56164be84f60 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b75ffb640 10 client.15948 handle_command_reply: tid=0 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 client.15948 shutdown 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 msgr2=0x56164be71bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5b8802c820 tx=0x7f5b88009260 comp rx=0 tx=0).stop 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 shutdown_connections 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be84f60 0x56164be873d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be72630 0x56164be80d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be74dd0 0x56164be720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 --2- 172.21.15.122:0/1250163843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be71bb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 >> 172.21.15.122:0/1250163843 conn(0x56164be66000 msgr2=0x56164be67d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b75ffb640 0 client.15948 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:18.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 shutdown_connections 2024-04-02T13:36:18.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 1 -- 172.21.15.122:0/1250163843 wait complete. 2024-04-02T13:36:18.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.656+0000 7f5b97190740 20 client.15948 trim_cache size 0 max 16384 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.690+0000 7f5b97190740 1 Processor -- start 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.690+0000 7f5b97190740 1 -- start start 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be72630 0x56164be8d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1acbe0 con 0x56164be6c1f0 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1ae720 con 0x56164be72630 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b97190740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1ad2b0 con 0x56164be66000 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:51858/0 (socket says 172.21.15.122:51858) 2024-04-02T13:36:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 -- 172.21.15.122:0/1735427545 learned_addr learned my addr 172.21.15.122:0/1735427545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 -- 172.21.15.122:0/1735427545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be72630 msgr2=0x56164be8d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be72630 0x56164be8d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 -- 172.21.15.122:0/1735427545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 msgr2=0x56164be8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be8ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.691+0000 7f5b9226f640 1 -- 172.21.15.122:0/1735427545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56164be81860 con 0x56164be66000 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.692+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.692+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b880bee60 tx=0x7f5b88004ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8307db78e5c07aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.692+0000 7f5b77fff640 1 -- 172.21.15.122:0/1735427545 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880db040 con 0x56164be66000 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.692+0000 7f5b77fff640 1 -- 172.21.15.122:0/1735427545 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b880c2b90 con 0x56164be66000 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.692+0000 7f5b77fff640 1 -- 172.21.15.122:0/1735427545 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b880cb100 con 0x56164be66000 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 -- 172.21.15.122:0/1735427545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 msgr2=0x56164be70480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b880bee60 tx=0x7f5b88004ee0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 -- 172.21.15.122:0/1735427545 shutdown_connections 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be72630 0x56164be8d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be6c1f0 0x56164be8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 --2- 172.21.15.122:0/1735427545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be66000 0x56164be70480 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 -- 172.21.15.122:0/1735427545 >> 172.21.15.122:0/1735427545 conn(0x56164be72250 msgr2=0x56164be65240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 -- 172.21.15.122:0/1735427545 shutdown_connections 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.693+0000 7f5b97190740 1 -- 172.21.15.122:0/1735427545 wait complete. 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 -- start start 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 Processor -- start 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be66000 0x56164be7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 0x56164be787e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1ae050 con 0x56164be72630 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.694+0000 7f5b97190740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1ad980 con 0x56164be66000 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b97190740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8c1aac70 con 0x56164be6c1f0 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:51868/0 (socket says 172.21.15.122:51868) 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 -- 172.21.15.122:0/1663547956 learned_addr learned my addr 172.21.15.122:0/1663547956 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b97190740 10 client.0 _ll_register_callbacks cb 0x7f5b880f3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b92a70640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 0x56164be787e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 -- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be66000 msgr2=0x56164be7cf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be66000 0x56164be7cf30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 -- 172.21.15.122:0/1663547956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 msgr2=0x56164be787e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 0x56164be787e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 -- 172.21.15.122:0/1663547956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56164be78fb0 con 0x56164be6c1f0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b92a70640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 0x56164be787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b91a6e640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009a20 tx=0x7f5b7c014070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8395cfbab29ccded server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7c02b900 con 0x56164be6c1f0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b880c2b90 con 0x56164be6c1f0 2024-04-02T13:36:18.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.695+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5b7c020550 con 0x56164be6c1f0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5b7c039050 con 0x56164be6c1f0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 10 client.25375 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56164beeb540 con 0x56164be6c1f0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b880c2b90 con 0x56164be6c1f0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5b7c0349a0 con 0x56164be6c1f0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 10 client.25375 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 20 client.25375 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be74dd0 0x56164beedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 4 client.25375 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56164bdd70c0 con 0x56164be74dd0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.696+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be74dd0 0x56164beedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b75ffb640 10 client.25375 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b9226f640 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be74dd0 0x56164beedd30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c9ba471ea3ea7c server_cookie=b38d64c4417273ab in_seq=0 out_seq=0 2024-04-02T13:36:18.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b75ffb640 1 -- 172.21.15.122:0/1663547956 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56164bdd70c0 con 0x56164be74dd0 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b75ffb640 10 client.25375 handle_command_reply: tid=0 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 client.25375 shutdown 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 msgr2=0x56164be782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c009a20 tx=0x7f5b7c014070 comp rx=0 tx=0).stop 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 shutdown_connections 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56164be74dd0 0x56164beedd30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56164be72630 0x56164be787e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56164be6c1f0 0x56164be782a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 --2- 172.21.15.122:0/1663547956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56164be66000 0x56164be7cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 >> 172.21.15.122:0/1663547956 conn(0x56164be72250 msgr2=0x56164be6c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b75ffb640 0 client.25375 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 shutdown_connections 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 1 -- 172.21.15.122:0/1663547956 wait complete. 2024-04-02T13:36:18.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.697+0000 7f5b97190740 20 client.25375 trim_cache size 0 max 16384 2024-04-02T13:36:18.706 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:18.706 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:36:18.706 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:36:18.706 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 msgr2=0x7f5b780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 0x7f5b780a6300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c104ff0 tx=0x7f5b7c005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 msgr2=0x7f5b8c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5b88009900 tx=0x7f5b88004360 comp rx=0 tx=0).stop 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 shutdown_connections 2024-04-02T13:36:18.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5b780a3e40 0x7f5b780a6300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b8c106980 0x7f5b8c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b8c101330 0x7f5b8c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 --2- 172.21.15.122:0/3240690307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5b8c100970 0x7f5b8c1972a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 >> 172.21.15.122:0/3240690307 conn(0x7f5b8c077b00 msgr2=0x7f5b8c1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 shutdown_connections 2024-04-02T13:36:18.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.698+0000 7f5b944d3640 1 -- 172.21.15.122:0/3240690307 wait complete. 2024-04-02T13:36:18.712 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:36:18.712 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:18.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4178919943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 msgr2=0x7fbf5810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4178919943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf5810bb70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009900 tx=0x7fbf4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:18.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4178919943 shutdown_connections 2024-04-02T13:36:18.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4178919943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 0x7fbf5810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4178919943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf5810bb70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4178919943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 0x7fbf58103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4178919943 >> 172.21.15.122:0/4178919943 conn(0x7fbf58077b00 msgr2=0x7fbf58077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:18.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4178919943 shutdown_connections 2024-04-02T13:36:18.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.813+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4178919943 wait complete. 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 Processor -- start 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 -- start start 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 0x7fbf58198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 0x7fbf5819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111f40 con 0x7fbf58103690 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58111dc0 con 0x7fbf5810c140 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5c98d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf581120c0 con 0x7fbf58102d70 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59236/0 (socket says 172.21.15.122:59236) 2024-04-02T13:36:18.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 -- 172.21.15.122:0/4011670689 learned_addr learned my addr 172.21.15.122:0/4011670689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf56d9d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 0x7fbf5819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf5659c640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 0x7fbf58198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 -- 172.21.15.122:0/4011670689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 msgr2=0x7fbf58198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 0x7fbf58198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 -- 172.21.15.122:0/4011670689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 msgr2=0x7fbf5819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 0x7fbf5819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.814+0000 7fbf55d9b640 1 -- 172.21.15.122:0/4011670689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5819d410 con 0x7fbf58103690 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.815+0000 7fbf55d9b640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.815+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c041070 con 0x7fbf58103690 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.815+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5819d6a0 con 0x7fbf58103690 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.815+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c004450 con 0x7fbf58103690 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.815+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c003d00 con 0x7fbf58103690 2024-04-02T13:36:18.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.816+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a4f90 con 0x7fbf58103690 2024-04-02T13:36:18.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.817+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf580688f0 con 0x7fbf58103690 2024-04-02T13:36:18.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.817+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fbf4c044070 con 0x7fbf58103690 2024-04-02T13:36:18.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.818+0000 7fbf477fe640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 0x7fbf400a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:18.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.818+0000 7fbf5659c640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 0x7fbf400a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:18.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.818+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf4c0f27c0 con 0x7fbf58103690 2024-04-02T13:36:18.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.819+0000 7fbf5659c640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 0x7fbf400a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009b60 tx=0x7fbf48005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:18.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:18.823+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbf4c0be990 con 0x7fbf58103690 2024-04-02T13:36:19.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.272+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf58107230 con 0x7fbf58103690 2024-04-02T13:36:19.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.273+0000 7fbf477fe640 1 -- 172.21.15.122:0/4011670689 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fbf4c0be290 con 0x7fbf58103690 2024-04-02T13:36:19.277 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:36:19.277 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:36:19.277 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 msgr2=0x7fbf400a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 0x7fbf400a6380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009b60 tx=0x7fbf48005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 msgr2=0x7fbf58198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c006010 tx=0x7fbf4c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 shutdown_connections 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf400a3ec0 0x7fbf400a6380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf5810c140 0x7fbf5819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf58103690 0x7fbf58198960 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 --2- 172.21.15.122:0/4011670689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf58102d70 0x7fbf58198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 >> 172.21.15.122:0/4011670689 conn(0x7fbf58077b00 msgr2=0x7fbf580fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 shutdown_connections 2024-04-02T13:36:19.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.276+0000 7fbf5c98d640 1 -- 172.21.15.122:0/4011670689 wait complete. 2024-04-02T13:36:19.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.438+0000 7f5f19e55640 1 -- 172.21.15.122:0/742319296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f140696c0 msgr2=0x7f5f1410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.438+0000 7f5f19e55640 1 --2- 172.21.15.122:0/742319296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f140696c0 0x7f5f1410e960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009cd0 tx=0x7f5f0c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 -- 172.21.15.122:0/742319296 shutdown_connections 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 --2- 172.21.15.122:0/742319296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f140696c0 0x7f5f1410e960 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 --2- 172.21.15.122:0/742319296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f14068d00 0x7f5f14069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 --2- 172.21.15.122:0/742319296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14107b50 0x7f5f14107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 -- 172.21.15.122:0/742319296 >> 172.21.15.122:0/742319296 conn(0x7f5f14077b00 msgr2=0x7f5f14077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 -- 172.21.15.122:0/742319296 shutdown_connections 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 -- 172.21.15.122:0/742319296 wait complete. 2024-04-02T13:36:19.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.439+0000 7f5f19e55640 1 Processor -- start 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 -- start start 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14068d00 0x7f5f14198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 0x7f5f14198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14111ea0 con 0x7f5f14107b50 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14111d20 con 0x7f5f14068d00 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f19e55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f14112020 con 0x7f5f140696c0 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59258/0 (socket says 172.21.15.122:59258) 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 -- 172.21.15.122:0/2678332448 learned_addr learned my addr 172.21.15.122:0/2678332448 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14068d00 0x7f5f14198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 0x7f5f14198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 -- 172.21.15.122:0/2678332448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 msgr2=0x7f5f14198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 0x7f5f14198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 -- 172.21.15.122:0/2678332448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14068d00 msgr2=0x7f5f14198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.440+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14068d00 0x7f5f14198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f13fff640 1 -- 172.21.15.122:0/2678332448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1419d310 con 0x7f5f14107b50 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 0x7f5f14198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004af0 tx=0x7f5f0c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f10ff9640 1 -- 172.21.15.122:0/2678332448 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c042020 con 0x7f5f14107b50 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1419d5a0 con 0x7f5f14107b50 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f10ff9640 1 -- 172.21.15.122:0/2678332448 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0c032e80 con 0x7f5f14107b50 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f10ff9640 1 -- 172.21.15.122:0/2678332448 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c039040 con 0x7f5f14107b50 2024-04-02T13:36:19.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.441+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f141a4e90 con 0x7f5f14107b50 2024-04-02T13:36:19.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f10ff9640 1 -- 172.21.15.122:0/2678332448 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f5f0c045020 con 0x7f5f14107b50 2024-04-02T13:36:19.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f10ff9640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 0x7f5f000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f10ff9640 1 -- 172.21.15.122:0/2678332448 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5f0c0f2960 con 0x7f5f14107b50 2024-04-02T13:36:19.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 0x7f5f000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f1ca3b740 1 Processor -- start 2024-04-02T13:36:19.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.444+0000 7f5f1ca3b740 1 -- start start 2024-04-02T13:36:19.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 0x55de73318980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de732827b0 con 0x55de73318fa0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55de73221570 con 0x55de7332c7f0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55de7327e760 con 0x55de7332be40 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 0x7f5f000a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009bd0 tx=0x7f5f04005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59270/0 (socket says 172.21.15.122:59270) 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 -- 172.21.15.122:0/1617451739 learned_addr learned my addr 172.21.15.122:0/1617451739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 0x55de73318980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 -- 172.21.15.122:0/1617451739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 msgr2=0x55de7332c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 -- 172.21.15.122:0/1617451739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 msgr2=0x55de73318980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 0x55de73318980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 -- 172.21.15.122:0/1617451739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de72f963f0 con 0x55de73318fa0 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 0x55de73318980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.445+0000 7f5f13fff640 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009ca0 tx=0x7f5f0c0be200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea24979e686201f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.446+0000 7f5f09ffb640 1 -- 172.21.15.122:0/1617451739 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0beb30 con 0x55de73318fa0 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.446+0000 7f5f09ffb640 1 -- 172.21.15.122:0/1617451739 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0c0be430 con 0x55de73318fa0 2024-04-02T13:36:19.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.446+0000 7f5f09ffb640 1 -- 172.21.15.122:0/1617451739 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0c6c10 con 0x55de73318fa0 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1617451739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 msgr2=0x55de73319400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009ca0 tx=0x7f5f0c0be200 comp rx=0 tx=0).stop 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1617451739 shutdown_connections 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de73318fa0 0x55de73319400 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de7332c7f0 0x55de73318980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1617451739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1617451739 >> 172.21.15.122:0/1617451739 conn(0x55de7331b440 msgr2=0x55de7331b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1617451739 shutdown_connections 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1617451739 wait complete. 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.447+0000 7f5f1ca3b740 1 -- start start 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 Processor -- start 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de7332fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f12ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f12ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59278/0 (socket says 172.21.15.122:59278) 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73330690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f12ffd640 1 -- 172.21.15.122:0/2394864797 learned_addr learned my addr 172.21.15.122:0/2394864797 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de7332fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55de73333040 con 0x55de7332be40 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55de7323de80 con 0x55de73318fa0 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55de733331c0 con 0x55de7332c7f0 2024-04-02T13:36:19.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.448+0000 7f5f1ca3b740 10 client.0 _ll_register_callbacks cb 0x55de73279aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 -- 172.21.15.122:0/2394864797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 msgr2=0x55de73330690 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73330690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 -- 172.21.15.122:0/2394864797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 msgr2=0x55de7332fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de7332fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de730607d0 con 0x55de7332be40 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de7332fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5efc012720 tx=0x7f5efc012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633506b84fc5c731 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5ef2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/2394864797 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc004460 con 0x55de7332be40 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f0c0be430 con 0x55de7332be40 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/2394864797 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5efc010950 con 0x55de7332be40 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/2394864797 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5efc010b50 con 0x55de7332be40 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f1ca3b740 10 client.15972 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.449+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55de7333f090 con 0x55de7332be40 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/2394864797 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5efc019070 con 0x55de7332be40 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 10 client.15972 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 10 client.15972 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 20 client.15972 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733403e0 0x55de733427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 4 client.15972 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5f0c0be430 con 0x55de733403e0 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.450+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733403e0 0x55de733427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.451+0000 7f5ef2ffd640 10 client.15972 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.451+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733403e0 0x55de733427f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da1caf22b9ecf956 server_cookie=2d8403c8361921cc in_seq=0 out_seq=0 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/2394864797 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f5f0c0be430 con 0x55de733403e0 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5ef2ffd640 10 client.15972 handle_command_reply: tid=0 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 client.15972 shutdown 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 msgr2=0x55de73330150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f5efc012720 tx=0x7f5efc012b20 comp rx=0 tx=0).stop 2024-04-02T13:36:19.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 shutdown_connections 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733403e0 0x55de733427f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5ef2ffd640 0 client.15972 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73330690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73330150 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2394864797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de7332fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 >> 172.21.15.122:0/2394864797 conn(0x55de7331b440 msgr2=0x55de7331aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 shutdown_connections 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2394864797 wait complete. 2024-04-02T13:36:19.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.452+0000 7f5f1ca3b740 20 client.15972 trim_cache size 0 max 16384 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.478+0000 7f5f1ca3b740 1 Processor -- start 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 -- start start 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de73332340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141acce0 con 0x55de7332c7f0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141ae820 con 0x55de73318fa0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f1ca3b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141ad3b0 con 0x55de7332be40 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59290/0 (socket says 172.21.15.122:59290) 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 -- 172.21.15.122:0/2232346194 learned_addr learned my addr 172.21.15.122:0/2232346194 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f137fe640 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de73332340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 -- 172.21.15.122:0/2232346194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 msgr2=0x55de7332b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 -- 172.21.15.122:0/2232346194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 msgr2=0x55de73332340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de73332340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.479+0000 7f5f13fff640 1 -- 172.21.15.122:0/2232346194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7332e160 con 0x55de7332c7f0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.480+0000 7f5f13fff640 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c006010 tx=0x7f5f0c035f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f18fd6c66e98111 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.480+0000 7f5f097fa640 1 -- 172.21.15.122:0/2232346194 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0be430 con 0x55de7332c7f0 2024-04-02T13:36:19.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.480+0000 7f5f097fa640 1 -- 172.21.15.122:0/2232346194 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f0c0c27e0 con 0x55de7332c7f0 2024-04-02T13:36:19.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.480+0000 7f5f097fa640 1 -- 172.21.15.122:0/2232346194 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0c6c10 con 0x55de7332c7f0 2024-04-02T13:36:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2232346194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 msgr2=0x55de733304b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c006010 tx=0x7f5f0c035f90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2232346194 shutdown_connections 2024-04-02T13:36:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332c7f0 0x55de733304b0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332be40 0x55de7332b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/2232346194 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de73332340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2232346194 >> 172.21.15.122:0/2232346194 conn(0x55de7331b440 msgr2=0x55de7332e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2232346194 shutdown_connections 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.481+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/2232346194 wait complete. 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.482+0000 7f5f1ca3b740 1 -- start start 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.482+0000 7f5f1ca3b740 1 Processor -- start 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.482+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de733474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.482+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59292/0 (socket says 172.21.15.122:59292) 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f1ca3b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73347f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 -- 172.21.15.122:0/1887614285 learned_addr learned my addr 172.21.15.122:0/1887614285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141ae150 con 0x55de7332be40 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141ada80 con 0x55de73318fa0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f141aacc0 con 0x55de7332c7f0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de733474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f13fff640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73347f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 -- 172.21.15.122:0/1887614285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 msgr2=0x55de73347f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73347f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 -- 172.21.15.122:0/1887614285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 msgr2=0x55de733474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de733474c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de733a5930 con 0x55de7332be40 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f13fff640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73347f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de733474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f1ca3b740 10 client.0 _ll_register_callbacks cb 0x7f5efc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5f12ffd640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55de733311e0 tx=0x7f5efc014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a9577088b5a938 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.483+0000 7f5ef2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/1887614285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc02e070 con 0x55de7332be40 2024-04-02T13:36:19.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/1887614285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5efc003de0 con 0x55de7332be40 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f0c0c27e0 con 0x55de7332be40 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/1887614285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5efc03a050 con 0x55de7332be40 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 10 client.15978 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55de733a6960 con 0x55de7332be40 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/1887614285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f5efc019040 con 0x55de7332be40 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 10 client.15978 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 10 client.15978 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.484+0000 7f5f1ca3b740 20 client.15978 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733a6fc0 0x55de733a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f1ca3b740 4 client.15978 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5f0c0c27e0 con 0x55de733a6fc0 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733a6fc0 0x55de733a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5ef2ffd640 10 client.15978 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f137fe640 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733a6fc0 0x55de733a93e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0cbd8204a878b60 server_cookie=3e85946a6d9029b7 in_seq=0 out_seq=0 2024-04-02T13:36:19.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5ef2ffd640 1 -- 172.21.15.122:0/1887614285 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5f0c0c27e0 con 0x55de733a6fc0 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5ef2ffd640 10 client.15978 handle_command_reply: tid=0 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.485+0000 7f5f1ca3b740 1 client.15978 shutdown 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 msgr2=0x55de73347a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55de733311e0 tx=0x7f5efc014040 comp rx=0 tx=0).stop 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 shutdown_connections 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55de733a6fc0 0x55de733a93e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55de7332c7f0 0x55de73347f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5ef2ffd640 0 client.15978 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55de7332be40 0x55de73347a00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 --2- 172.21.15.122:0/1887614285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55de73318fa0 0x55de733474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 >> 172.21.15.122:0/1887614285 conn(0x55de7331b440 msgr2=0x55de7332b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 shutdown_connections 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 1 -- 172.21.15.122:0/1887614285 wait complete. 2024-04-02T13:36:19.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f1ca3b740 20 client.15978 trim_cache size 0 max 16384 2024-04-02T13:36:19.494 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:19.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 msgr2=0x7f5f000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 0x7f5f000a6300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009bd0 tx=0x7f5f04005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 msgr2=0x7f5f1419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.486+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c004af0 tx=0x7f5f0c032c90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 shutdown_connections 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f000a3e40 0x7f5f000a6300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f14107b50 0x7f5f1419cbe0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f140696c0 0x7f5f14198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 --2- 172.21.15.122:0/2678332448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f14068d00 0x7f5f14198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 >> 172.21.15.122:0/2678332448 conn(0x7f5f14077b00 msgr2=0x7f5f140fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 shutdown_connections 2024-04-02T13:36:19.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.487+0000 7f5f19e55640 1 -- 172.21.15.122:0/2678332448 wait complete. 2024-04-02T13:36:19.501 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 -- 172.21.15.122:0/54376539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 msgr2=0x7fd9f4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 --2- 172.21.15.122:0/54376539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f4107f30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009930 tx=0x7fd9f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 -- 172.21.15.122:0/54376539 shutdown_connections 2024-04-02T13:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 --2- 172.21.15.122:0/54376539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f40696c0 0x7fd9f410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 --2- 172.21.15.122:0/54376539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f4068d00 0x7fd9f4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 --2- 172.21.15.122:0/54376539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f4107f30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 -- 172.21.15.122:0/54376539 >> 172.21.15.122:0/54376539 conn(0x7fd9f4077b00 msgr2=0x7fd9f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 -- 172.21.15.122:0/54376539 shutdown_connections 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 -- 172.21.15.122:0/54376539 wait complete. 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.605+0000 7fd9fc360640 1 Processor -- start 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 -- start start 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f4068d00 0x7fd9f4198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 0x7fd9f4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4111ea0 con 0x7fd9f4107b50 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4111d20 con 0x7fd9f40696c0 2024-04-02T13:36:19.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fc360640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f4112020 con 0x7fd9f4068d00 2024-04-02T13:36:19.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59324/0 (socket says 172.21.15.122:59324) 2024-04-02T13:36:19.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 -- 172.21.15.122:0/3109720833 learned_addr learned my addr 172.21.15.122:0/3109720833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f4068d00 0x7fd9f4198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 0x7fd9f4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 -- 172.21.15.122:0/3109720833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f4068d00 msgr2=0x7fd9f4198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f4068d00 0x7fd9f4198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 -- 172.21.15.122:0/3109720833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 msgr2=0x7fd9f4198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 0x7fd9f4198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 -- 172.21.15.122:0/3109720833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f419d3a0 con 0x7fd9f4107b50 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 0x7fd9f4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.606+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009fd0 tx=0x7fd9ec00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.607+0000 7fd9eb7fe640 1 -- 172.21.15.122:0/3109720833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec017070 con 0x7fd9f4107b50 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.607+0000 7fd9eb7fe640 1 -- 172.21.15.122:0/3109720833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9ec003e50 con 0x7fd9f4107b50 2024-04-02T13:36:19.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.607+0000 7fd9eb7fe640 1 -- 172.21.15.122:0/3109720833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0106c0 con 0x7fd9f4107b50 2024-04-02T13:36:19.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.607+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f419d690 con 0x7fd9f4107b50 2024-04-02T13:36:19.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.607+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f41a4f20 con 0x7fd9f4107b50 2024-04-02T13:36:19.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.609+0000 7fd9eb7fe640 1 -- 172.21.15.122:0/3109720833 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fd9ec003970 con 0x7fd9f4107b50 2024-04-02T13:36:19.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.610+0000 7fd9ff029740 1 Processor -- start 2024-04-02T13:36:19.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.610+0000 7fd9eb7fe640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 0x7fd9e00a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.610+0000 7fd9eb7fe640 1 -- 172.21.15.122:0/3109720833 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9ec0d0c00 con 0x7fd9f4107b50 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.610+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 0x7fd9e00a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 -- start start 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d64760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d67090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d81cbf7b0 con 0x564d81d5b7a0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d81c5e570 con 0x564d81d5c0c0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9ff029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d81cbb760 con 0x564d81d64ca0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59326/0 (socket says 172.21.15.122:59326) 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1597520679 learned_addr learned my addr 172.21.15.122:0/1597520679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d67090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1597520679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 msgr2=0x564d81d67090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 0x7fd9e00a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4005d00 tx=0x7fd9e4005c70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d67090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1597520679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 msgr2=0x564d81d64760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d64760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1597520679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d819d33f0 con 0x564d81d5b7a0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.611+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d67090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.612+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002efd0 tx=0x7fd9f0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6414199f3aa289 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.612+0000 7fd9e8bfe640 1 -- 172.21.15.122:0/1597520679 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0042070 con 0x564d81d5b7a0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.612+0000 7fd9e8bfe640 1 -- 172.21.15.122:0/1597520679 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f0004240 con 0x564d81d5b7a0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.612+0000 7fd9e8bfe640 1 -- 172.21.15.122:0/1597520679 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00046c0 con 0x564d81d5b7a0 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- 172.21.15.122:0/1597520679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 msgr2=0x564d81d5bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002efd0 tx=0x7fd9f0005e80 comp rx=0 tx=0).stop 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- 172.21.15.122:0/1597520679 shutdown_connections 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d67090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d64760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1597520679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5b7a0 0x564d81d5bb80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- 172.21.15.122:0/1597520679 >> 172.21.15.122:0/1597520679 conn(0x564d81d55ed0 msgr2=0x564d81d562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- 172.21.15.122:0/1597520679 shutdown_connections 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- 172.21.15.122:0/1597520679 wait complete. 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.613+0000 7fd9ff029740 1 -- start start 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 Processor -- start 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d61b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9f98fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9f98fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59340/0 (socket says 172.21.15.122:59340) 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9f98fb640 1 -- 172.21.15.122:0/2222524081 learned_addr learned my addr 172.21.15.122:0/2222524081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d81d6be40 con 0x564d81d5c0c0 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d81c7ae80 con 0x564d81d5b7a0 2024-04-02T13:36:19.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d61b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d81d6bfc0 con 0x564d81d64ca0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9ff029740 10 client.0 _ll_register_callbacks cb 0x564d81cb6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 msgr2=0x564d81d625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2222524081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 msgr2=0x564d81d620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d81a9d7d0 con 0x564d81d5b7a0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.614+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f0004240 con 0x564d81d5b7a0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d61b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002f020 tx=0x7fd9f0005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0190c3eb8854dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ddffb640 10 client.25496 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2222524081 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f004f060 con 0x564d81d5b7a0 2024-04-02T13:36:19.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2222524081 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9f0047860 con 0x564d81d5b7a0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2222524081 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9f0042070 con 0x564d81d5b7a0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ff029740 10 client.25496 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.615+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=28}) -- 0x564d81d67710 con 0x564d81d5b7a0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2222524081 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9f0047aa0 con 0x564d81d5b7a0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 10 client.25496 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 10 client.25496 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 20 client.25496 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81d68a60 0x564d81d6aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 4 client.25496 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d81ccccf0 con 0x564d81d68a60 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81d68a60 0x564d81d6aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9ddffb640 10 client.25496 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.616+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81d68a60 0x564d81d6aed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a1179abfa5192b4 server_cookie=b67530bdb5d41945 in_seq=0 out_seq=0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2222524081 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d81ccccf0 con 0x564d81d68a60 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ddffb640 10 client.25496 handle_command_reply: tid=0 2024-04-02T13:36:19.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 client.25496 shutdown 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 msgr2=0x564d81d61b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d61b60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002f020 tx=0x7fd9f0005730 comp rx=0 tx=0).stop 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 shutdown_connections 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81d68a60 0x564d81d6aed0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ddffb640 0 client.25496 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2222524081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d61b60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 >> 172.21.15.122:0/2222524081 conn(0x564d81d55ed0 msgr2=0x564d81d57c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 shutdown_connections 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.617+0000 7fd9ff029740 1 -- 172.21.15.122:0/2222524081 wait complete. 2024-04-02T13:36:19.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.618+0000 7fd9ff029740 20 client.25496 trim_cache size 0 max 16384 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 Processor -- start 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 -- start start 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d7cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41acd30 con 0x564d81d5c0c0 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41ae870 con 0x564d81d5b7a0 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9ff029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41ad400 con 0x564d81d64ca0 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9f98fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9f98fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59350/0 (socket says 172.21.15.122:59350) 2024-04-02T13:36:19.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9f98fb640 1 -- 172.21.15.122:0/1571422494 learned_addr learned my addr 172.21.15.122:0/1571422494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.631+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1571422494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 msgr2=0x564d81d7cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d7cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1571422494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 msgr2=0x564d81d74d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/1571422494 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d81d517a0 con 0x564d81d5b7a0 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.632+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d64290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0049010 tx=0x7fd9f004a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e1b0340667b4f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9dffff640 1 -- 172.21.15.122:0/1571422494 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f002cbc0 con 0x564d81d5b7a0 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9dffff640 1 -- 172.21.15.122:0/1571422494 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f0004240 con 0x564d81d5b7a0 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9dffff640 1 -- 172.21.15.122:0/1571422494 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f003da10 con 0x564d81d5b7a0 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 -- 172.21.15.122:0/1571422494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 msgr2=0x564d81d64290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d64290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0049010 tx=0x7fd9f004a000 comp rx=0 tx=0).stop 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 -- 172.21.15.122:0/1571422494 shutdown_connections 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d64ca0 0x564d81d7cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d5c0c0 0x564d81d74d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 --2- 172.21.15.122:0/1571422494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5b7a0 0x564d81d64290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 -- 172.21.15.122:0/1571422494 >> 172.21.15.122:0/1571422494 conn(0x564d81d55ed0 msgr2=0x564d81d60b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 -- 172.21.15.122:0/1571422494 shutdown_connections 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.633+0000 7fd9ff029740 1 -- 172.21.15.122:0/1571422494 wait complete. 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 -- start start 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 Processor -- start 2024-04-02T13:36:19.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d6c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 0x564d81dd95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41ae1a0 con 0x564d81d64ca0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41adad0 con 0x564d81d5c0c0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f41aad10 con 0x564d81d5b7a0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9ff029740 10 client.0 _ll_register_callbacks cb 0x7fd9f0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9fa8fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 0x564d81dd95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9fa0fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9f98fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d6c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9fa0fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52022/0 (socket says 172.21.15.122:52022) 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.634+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2802290420 learned_addr learned my addr 172.21.15.122:0/2802290420 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2802290420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 msgr2=0x564d81d6c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d6c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2802290420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 msgr2=0x564d81dd95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 0x564d81dd95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 -- 172.21.15.122:0/2802290420 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d81dd9b30 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa8fd640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 0x564d81dd95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9fa0fc640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002c170 tx=0x7fd9f002c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56370f1398a2b52b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0047a70 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f0004240 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9f003d690 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9f0047c10 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ff029740 10 client.25399 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.635+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564d81ddad30 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f0004240 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd9f0065030 con 0x564d81d5b7a0 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 10 client.25399 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 10 client.25399 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:19.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 20 client.25399 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81ddb440 0x564d81ddd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 4 client.25399 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:19.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.636+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d81d5c950 con 0x564d81ddb440 2024-04-02T13:36:19.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81ddb440 0x564d81ddd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ddffb640 10 client.25399 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9f98fb640 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81ddb440 0x564d81ddd830 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3942a6b8289558de server_cookie=aa462f2b226fe2a in_seq=0 out_seq=0 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ddffb640 1 -- 172.21.15.122:0/2802290420 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d81d5c950 con 0x564d81ddb440 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ddffb640 10 client.25399 handle_command_reply: tid=0 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 client.25399 shutdown 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 msgr2=0x564d81d6c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9f002c170 tx=0x7fd9f002c7e0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 shutdown_connections 2024-04-02T13:36:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x564d81ddb440 0x564d81ddd830 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d81d64ca0 0x564d81dd95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d81d5c0c0 0x564d81d6c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 --2- 172.21.15.122:0/2802290420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d81d5b7a0 0x564d81d6c390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ddffb640 0 client.25399 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 >> 172.21.15.122:0/2802290420 conn(0x564d81d55ed0 msgr2=0x564d81d61b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.637+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 shutdown_connections 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9ff029740 1 -- 172.21.15.122:0/2802290420 wait complete. 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9ff029740 20 client.25399 trim_cache size 0 max 16384 2024-04-02T13:36:19.645 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 msgr2=0x7fd9e00a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 0x7fd9e00a6300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4005d00 tx=0x7fd9e4005c70 comp rx=0 tx=0).stop 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 msgr2=0x7fd9f419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009fd0 tx=0x7fd9ec00b260 comp rx=0 tx=0).stop 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 shutdown_connections 2024-04-02T13:36:19.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9e00a3e40 0x7fd9e00a6300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9f4107b50 0x7fd9f419cc70 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9f40696c0 0x7fd9f4198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 --2- 172.21.15.122:0/3109720833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f4068d00 0x7fd9f4198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 >> 172.21.15.122:0/3109720833 conn(0x7fd9f4077b00 msgr2=0x7fd9f40fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 shutdown_connections 2024-04-02T13:36:19.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.638+0000 7fd9fc360640 1 -- 172.21.15.122:0/3109720833 wait complete. 2024-04-02T13:36:19.653 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 -- 172.21.15.122:0/1539733041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a40696c0 msgr2=0x7f41a410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 --2- 172.21.15.122:0/1539733041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a40696c0 0x7f41a410e960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f419c009cd0 tx=0x7f419c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 -- 172.21.15.122:0/1539733041 shutdown_connections 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 --2- 172.21.15.122:0/1539733041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a40696c0 0x7f41a410e960 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 --2- 172.21.15.122:0/1539733041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 0x7f41a4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 --2- 172.21.15.122:0/1539733041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a4107b50 0x7f41a4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 -- 172.21.15.122:0/1539733041 >> 172.21.15.122:0/1539733041 conn(0x7f41a4077b00 msgr2=0x7f41a4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 -- 172.21.15.122:0/1539733041 shutdown_connections 2024-04-02T13:36:19.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.757+0000 7f41abc47640 1 -- 172.21.15.122:0/1539733041 wait complete. 2024-04-02T13:36:19.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.758+0000 7f41abc47640 1 Processor -- start 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.758+0000 7f41abc47640 1 -- start start 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 0x7f41a4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a40696c0 0x7f41a4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4111ea0 con 0x7f41a4107b50 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4111d20 con 0x7f41a4068d00 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41abc47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41a4112020 con 0x7f41a40696c0 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59392/0 (socket says 172.21.15.122:59392) 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 -- 172.21.15.122:0/4258653541 learned_addr learned my addr 172.21.15.122:0/4258653541 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41a99e3640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 0x7f41a4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41a91e2640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a40696c0 0x7f41a4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 -- 172.21.15.122:0/4258653541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a40696c0 msgr2=0x7f41a4198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a40696c0 0x7f41a4198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 -- 172.21.15.122:0/4258653541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 msgr2=0x7f41a4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 0x7f41a4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 -- 172.21.15.122:0/4258653541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a419d2e0 con 0x7f41a4107b50 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f419c004af0 tx=0x7f419c032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.759+0000 7f419affd640 1 -- 172.21.15.122:0/4258653541 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c042020 con 0x7f41a4107b50 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.760+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a419d570 con 0x7f41a4107b50 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.760+0000 7f419affd640 1 -- 172.21.15.122:0/4258653541 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419c032e80 con 0x7f41a4107b50 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.760+0000 7f419affd640 1 -- 172.21.15.122:0/4258653541 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c039040 con 0x7f41a4107b50 2024-04-02T13:36:19.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.760+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a41a4e60 con 0x7f41a4107b50 2024-04-02T13:36:19.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.762+0000 7f419affd640 1 -- 172.21.15.122:0/4258653541 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f419c045020 con 0x7f41a4107b50 2024-04-02T13:36:19.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.763+0000 7f419affd640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 0x7f41900a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.763+0000 7f41a99e3640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 0x7f41900a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.763+0000 7f419affd640 1 -- 172.21.15.122:0/4258653541 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f419c0f2c50 con 0x7f41a4107b50 2024-04-02T13:36:19.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 Processor -- start 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 -- start start 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41a99e3640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 0x7f41900a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009bd0 tx=0x7f41a0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8db0 0x5578266e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55782663c7b0 con 0x5578266e60b0 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5578265db570 con 0x5578266d8490 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41ae82b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557826638760 con 0x5578266d8db0 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41a99e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.764+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59408/0 (socket says 172.21.15.122:59408) 2024-04-02T13:36:19.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41aa1e4640 1 -- 172.21.15.122:0/3032627332 learned_addr learned my addr 172.21.15.122:0/3032627332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a91e2640 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8db0 0x5578266e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 -- 172.21.15.122:0/3032627332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8db0 msgr2=0x5578266e5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8db0 0x5578266e5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 -- 172.21.15.122:0/3032627332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 msgr2=0x5578266e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 -- 172.21.15.122:0/3032627332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578263503f0 con 0x5578266d8490 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f41a99e3640 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266d8870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009fd0 tx=0x7f41a0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0ec1e4bf7c468a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f418ffff640 1 -- 172.21.15.122:0/3032627332 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0005020 con 0x5578266d8490 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f418ffff640 1 -- 172.21.15.122:0/3032627332 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a00047a0 con 0x5578266d8490 2024-04-02T13:36:19.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.765+0000 7f418ffff640 1 -- 172.21.15.122:0/3032627332 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a001e6f0 con 0x5578266d8490 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 -- 172.21.15.122:0/3032627332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 msgr2=0x5578266d8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266d8870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009fd0 tx=0x7f41a0014000 comp rx=0 tx=0).stop 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 -- 172.21.15.122:0/3032627332 shutdown_connections 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8db0 0x5578266e5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 --2- 172.21.15.122:0/3032627332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266d8870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 -- 172.21.15.122:0/3032627332 >> 172.21.15.122:0/3032627332 conn(0x5578266d2ed0 msgr2=0x5578266d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 -- 172.21.15.122:0/3032627332 shutdown_connections 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.766+0000 7f41ae82b740 1 -- 172.21.15.122:0/3032627332 wait complete. 2024-04-02T13:36:19.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 -- start start 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 Processor -- start 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266e60b0 0x5578266e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.767+0000 7f41ae82b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5578266e8ef0 con 0x5578266d8db0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41ae82b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5578265f7e80 con 0x5578266d8490 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41ae82b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578266e9070 con 0x5578266e60b0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41ae82b740 10 client.0 _ll_register_callbacks cb 0x557826633aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59416/0 (socket says 172.21.15.122:59416) 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 -- 172.21.15.122:0/2729242798 learned_addr learned my addr 172.21.15.122:0/2729242798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a99e3640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266e60b0 0x5578266e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 -- 172.21.15.122:0/2729242798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266e60b0 msgr2=0x5578266e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266e60b0 0x5578266e3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 -- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 msgr2=0x5578266e5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266e5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 -- 172.21.15.122:0/2729242798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55782641a7d0 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a99e3640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41a91e2640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4194013710 tx=0x7f4194013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1096cec1f7996b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f418d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4194011dd0 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4194004030 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.768+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a00047a0 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4194004230 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 10 client.16008 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578266f4f80 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a00047a0 con 0x5578266d8db0 2024-04-02T13:36:19.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f419401a070 con 0x5578266d8db0 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 10 client.16008 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 10 client.16008 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 20 client.16008 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266f6490 0x5578266f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 4 client.16008 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557826649cf0 con 0x5578266f6490 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.769+0000 7f41a99e3640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266f6490 0x5578266f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.770+0000 7f418d7fa640 10 client.16008 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.770+0000 7f41a99e3640 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266f6490 0x5578266f88f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffecb38d9a5169b3 server_cookie=97b5a53bce11de81 in_seq=0 out_seq=0 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.770+0000 7f418d7fa640 1 -- 172.21.15.122:0/2729242798 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557826649cf0 con 0x5578266f6490 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.771+0000 7f418d7fa640 10 client.16008 handle_command_reply: tid=0 2024-04-02T13:36:19.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.771+0000 7f41ae82b740 1 client.16008 shutdown 2024-04-02T13:36:19.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 msgr2=0x5578266e2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4194013710 tx=0x7f4194013bd0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 shutdown_connections 2024-04-02T13:36:19.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266e60b0 0x5578266e3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e2db0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266f6490 0x5578266f88f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 --2- 172.21.15.122:0/2729242798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8490 0x5578266e5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f418d7fa640 0 client.16008 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 >> 172.21.15.122:0/2729242798 conn(0x5578266d2ed0 msgr2=0x5578266d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 shutdown_connections 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 1 -- 172.21.15.122:0/2729242798 wait complete. 2024-04-02T13:36:19.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.772+0000 7f41ae82b740 20 client.16008 trim_cache size 0 max 16384 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.785+0000 7f41ae82b740 1 Processor -- start 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.785+0000 7f41ae82b740 1 -- start start 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.785+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.785+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8db0 0x5578266e1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.785+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41ae82b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41acd70 con 0x5578266e60b0 2024-04-02T13:36:19.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41ae82b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41ae8b0 con 0x5578266d8db0 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41ae82b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41ad440 con 0x5578266d8490 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41aa1e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59426/0 (socket says 172.21.15.122:59426) 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41aa1e4640 1 -- 172.21.15.122:0/837189261 learned_addr learned my addr 172.21.15.122:0/837189261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a91e2640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8db0 0x5578266e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 -- 172.21.15.122:0/837189261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8db0 msgr2=0x5578266e1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8db0 0x5578266e1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 -- 172.21.15.122:0/837189261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 msgr2=0x5578266e54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 -- 172.21.15.122:0/837189261 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578266e3680 con 0x5578266d8490 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f41a99e3640 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266f1e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41a0004fa0 tx=0x7f41a00119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1156620bb7248d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.786+0000 7f418f7fe640 1 -- 172.21.15.122:0/837189261 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0004620 con 0x5578266d8490 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f418f7fe640 1 -- 172.21.15.122:0/837189261 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0035930 con 0x5578266d8490 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f418f7fe640 1 -- 172.21.15.122:0/837189261 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0005190 con 0x5578266d8490 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 -- 172.21.15.122:0/837189261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 msgr2=0x5578266f1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266f1e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41a0004fa0 tx=0x7f41a00119f0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 -- 172.21.15.122:0/837189261 shutdown_connections 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266e60b0 0x5578266e54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266d8db0 0x5578266e1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 --2- 172.21.15.122:0/837189261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266f1e70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 -- 172.21.15.122:0/837189261 >> 172.21.15.122:0/837189261 conn(0x5578266d2ed0 msgr2=0x5578266e1680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 -- 172.21.15.122:0/837189261 shutdown_connections 2024-04-02T13:36:19.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.787+0000 7f41ae82b740 1 -- 172.21.15.122:0/837189261 wait complete. 2024-04-02T13:36:19.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 -- start start 2024-04-02T13:36:19.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 Processor -- start 2024-04-02T13:36:19.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 0x5578266ed380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41ae1e0 con 0x5578266d8db0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41adb10 con 0x5578266e60b0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41a41aad50 con 0x5578266d8490 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41ae82b740 10 client.0 _ll_register_callbacks cb 0x7f4194009690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41a91e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41a91e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59436/0 (socket says 172.21.15.122:59436) 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41a91e2640 1 -- 172.21.15.122:0/1160760963 learned_addr learned my addr 172.21.15.122:0/1160760963 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 0x5578266ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.788+0000 7f41a99e3640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 -- 172.21.15.122:0/1160760963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 msgr2=0x5578266e6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266e6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 -- 172.21.15.122:0/1160760963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 msgr2=0x5578266e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 -- 172.21.15.122:0/1160760963 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578266ed8c0 con 0x5578266e60b0 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41a91e2640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41aa1e4640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 0x5578266ed380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f419c0f0610 tx=0x7f419c0be600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21fdda7b3a2ebeb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f418d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c0be720 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a0035930 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f419c0347d0 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f419c0c2ad0 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41ae82b740 10 client.25520 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.789+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578266eec20 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419c0c7390 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f419c004b40 con 0x5578266e60b0 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 10 client.25520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 20 client.25520 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266d9c20 0x5578266dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 4 client.25520 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f41a0035930 con 0x5578266d9c20 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.790+0000 7f41a99e3640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266d9c20 0x5578266dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41a99e3640 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266d9c20 0x5578266dc040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b0ea4801069cce0 server_cookie=c7062b4d2ff447e5 in_seq=0 out_seq=0 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f418d7fa640 10 client.25520 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f418d7fa640 1 -- 172.21.15.122:0/1160760963 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f41a0035930 con 0x5578266d9c20 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f418d7fa640 10 client.25520 handle_command_reply: tid=0 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 client.25520 shutdown 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 msgr2=0x5578266ed380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 0x5578266ed380 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f419c0f0610 tx=0x7f419c0be600 comp rx=0 tx=0).stop 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 shutdown_connections 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5578266d9c20 0x5578266dc040 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578266e60b0 0x5578266ed380 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578266d8db0 0x5578266e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 --2- 172.21.15.122:0/1160760963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578266d8490 0x5578266e6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f418d7fa640 0 client.25520 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:19.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 >> 172.21.15.122:0/1160760963 conn(0x5578266d2ed0 msgr2=0x5578266d42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 shutdown_connections 2024-04-02T13:36:19.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 1 -- 172.21.15.122:0/1160760963 wait complete. 2024-04-02T13:36:19.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.791+0000 7f41ae82b740 20 client.25520 trim_cache size 0 max 16384 2024-04-02T13:36:19.802 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 msgr2=0x7f41900a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 0x7f41900a6300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009bd0 tx=0x7f41a0005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 msgr2=0x7f41a419cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f419c004af0 tx=0x7f419c032c90 comp rx=0 tx=0).stop 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 shutdown_connections 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41900a3e40 0x7f41900a6300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41a4107b50 0x7f41a419cbb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a40696c0 0x7f41a4198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 --2- 172.21.15.122:0/4258653541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41a4068d00 0x7f41a4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 >> 172.21.15.122:0/4258653541 conn(0x7f41a4077b00 msgr2=0x7f41a40fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 shutdown_connections 2024-04-02T13:36:19.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.792+0000 7f41abc47640 1 -- 172.21.15.122:0/4258653541 wait complete. 2024-04-02T13:36:19.810 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:36:19.834 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:19 smithi122 ceph-mon[32114]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:36:19.835 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4011670689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:19.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 -- 172.21.15.122:0/383999035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 msgr2=0x7f7fc4069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 --2- 172.21.15.122:0/383999035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc4069180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009930 tx=0x7f7fb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:19.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 -- 172.21.15.122:0/383999035 shutdown_connections 2024-04-02T13:36:19.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 --2- 172.21.15.122:0/383999035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 --2- 172.21.15.122:0/383999035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc4069180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 --2- 172.21.15.122:0/383999035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 0x7f7fc4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 -- 172.21.15.122:0/383999035 >> 172.21.15.122:0/383999035 conn(0x7f7fc4077b00 msgr2=0x7f7fc4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 -- 172.21.15.122:0/383999035 shutdown_connections 2024-04-02T13:36:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.915+0000 7f7fca01b640 1 -- 172.21.15.122:0/383999035 wait complete. 2024-04-02T13:36:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 Processor -- start 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 -- start start 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 0x7f7fc419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4111ea0 con 0x7f7fc4068d00 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4111d20 con 0x7f7fc4107b50 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.916+0000 7f7fca01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4112020 con 0x7f7fc40696c0 2024-04-02T13:36:19.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59460/0 (socket says 172.21.15.122:59460) 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2212176396 learned_addr learned my addr 172.21.15.122:0/2212176396 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 0x7f7fc419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2212176396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 msgr2=0x7f7fc419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2212176396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 msgr2=0x7f7fc419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 0x7f7fc419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2212176396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc419f550 con 0x7f7fc4068d00 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d950 tx=0x7f7fb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc0ff9640 1 -- 172.21.15.122:0/2212176396 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4017030 con 0x7f7fc4068d00 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc419f840 con 0x7f7fc4068d00 2024-04-02T13:36:19.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fc0ff9640 1 -- 172.21.15.122:0/2212176396 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb4004020 con 0x7f7fc4068d00 2024-04-02T13:36:19.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.917+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a70d0 con 0x7f7fc4068d00 2024-04-02T13:36:19.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.918+0000 7f7fc0ff9640 1 -- 172.21.15.122:0/2212176396 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40047a0 con 0x7f7fc4068d00 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc0ff9640 1 -- 172.21.15.122:0/2212176396 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f7fb40041d0 con 0x7f7fc4068d00 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 Processor -- start 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 -- start start 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609ceab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609ceae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55609ce067b0 con 0x55609cea27a0 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55609cda5570 con 0x55609ceabca0 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fccc03740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55609ce02760 con 0x55609cea30c0 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc0ff9640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 0x7f7fac0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59462/0 (socket says 172.21.15.122:59462) 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609ceae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc3fff640 1 -- 172.21.15.122:0/2910246225 learned_addr learned my addr 172.21.15.122:0/2910246225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.921+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609ceab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 0x7f7fac0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc0ff9640 1 -- 172.21.15.122:0/2212176396 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7fb40d00c0 con 0x7f7fc4068d00 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2910246225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 msgr2=0x55609ceab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609ceab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2910246225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 msgr2=0x55609cea2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2910246225 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609cb1a3f0 con 0x55609ceabca0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609ceae090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009230 tx=0x7f7fb8005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=897167b142d9e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 0x7f7fac0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009fd0 tx=0x7f7fbc009cd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fa9ffb640 1 -- 172.21.15.122:0/2910246225 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8042070 con 0x55609ceabca0 2024-04-02T13:36:19.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fa9ffb640 1 -- 172.21.15.122:0/2910246225 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb8004240 con 0x55609ceabca0 2024-04-02T13:36:19.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.922+0000 7f7fa9ffb640 1 -- 172.21.15.122:0/2910246225 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80046c0 con 0x55609ceabca0 2024-04-02T13:36:19.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 -- 172.21.15.122:0/2910246225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 msgr2=0x55609ceae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609ceae090 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009230 tx=0x7f7fb8005e50 comp rx=0 tx=0).stop 2024-04-02T13:36:19.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 -- 172.21.15.122:0/2910246225 shutdown_connections 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609ceae090 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609ceab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2910246225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 -- 172.21.15.122:0/2910246225 >> 172.21.15.122:0/2910246225 conn(0x55609ce9ced0 msgr2=0x55609ce9d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 -- 172.21.15.122:0/2910246225 shutdown_connections 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.923+0000 7f7fccc03740 1 -- 172.21.15.122:0/2910246225 wait complete. 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fccc03740 1 -- start start 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fccc03740 1 Processor -- start 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.924+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59476/0 (socket says 172.21.15.122:59476) 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 -- 172.21.15.122:0/421649012 learned_addr learned my addr 172.21.15.122:0/421649012 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55609ceaea30 con 0x55609cea27a0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55609cdc1e80 con 0x55609ceabca0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55609ceaebb0 con 0x55609cea30c0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fccc03740 10 client.0 _ll_register_callbacks cb 0x55609cdfdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 -- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 msgr2=0x55609cea8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 -- 172.21.15.122:0/421649012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 msgr2=0x55609cea91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609cbe47d0 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb8004240 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7fb409b780 tx=0x7f7fb409b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226858b3d78e24fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.925+0000 7f7fa2ffd640 10 client.16026 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/421649012 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb40a4460 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/421649012 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7fb40cd5e0 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/421649012 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40cd840 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 10 client.16026 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55609ceba980 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/421649012 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fb409aed0 con 0x55609cea27a0 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 10 client.16026 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 10 client.16026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 20 client.16026 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cebe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 4 client.16026 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.926+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55609ce13cf0 con 0x55609cebbcd0 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.927+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cebe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.927+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cebe170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd3be4e8d9773c3e server_cookie=f2b58f7210e6980c in_seq=0 out_seq=0 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.927+0000 7f7fa2ffd640 10 client.16026 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/421649012 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55609ce13cf0 con 0x55609cebbcd0 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fa2ffd640 10 client.16026 handle_command_reply: tid=0 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 client.16026 shutdown 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 msgr2=0x55609ceab6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7fb409b780 tx=0x7f7fb409b630 comp rx=0 tx=0).stop 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 shutdown_connections 2024-04-02T13:36:19.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cebe170 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fa2ffd640 0 client.16026 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 --2- 172.21.15.122:0/421649012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609ceab6f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 >> 172.21.15.122:0/421649012 conn(0x55609ce9ced0 msgr2=0x55609ce9ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 shutdown_connections 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 1 -- 172.21.15.122:0/421649012 wait complete. 2024-04-02T13:36:19.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.928+0000 7f7fccc03740 20 client.16026 trim_cache size 0 max 16384 2024-04-02T13:36:19.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.941+0000 7f7fccc03740 1 Processor -- start 2024-04-02T13:36:19.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 -- start start 2024-04-02T13:36:19.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609cea27a0 0x55609ceb7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609ceabca0 0x55609cec3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb000fed0 con 0x55609cea30c0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0011a10 con 0x55609cea27a0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fccc03740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb00105a0 con 0x55609ceabca0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59482/0 (socket says 172.21.15.122:59482) 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2215871340 learned_addr learned my addr 172.21.15.122:0/2215871340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609ceabca0 0x55609cec3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609cea27a0 0x55609ceb7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2215871340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609ceabca0 msgr2=0x55609cec3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609ceabca0 0x55609cec3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2215871340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609cea27a0 msgr2=0x55609ceb7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609cea27a0 0x55609ceb7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.942+0000 7f7fc2ffd640 1 -- 172.21.15.122:0/2215871340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55609cea9530 con 0x55609cea30c0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.943+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005ec0 tx=0x7f7fb8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c79d08f3110fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.943+0000 7f7fa97fa640 1 -- 172.21.15.122:0/2215871340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8059070 con 0x55609cea30c0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.943+0000 7f7fa97fa640 1 -- 172.21.15.122:0/2215871340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb802cab0 con 0x55609cea30c0 2024-04-02T13:36:19.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.943+0000 7f7fa97fa640 1 -- 172.21.15.122:0/2215871340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8042040 con 0x55609cea30c0 2024-04-02T13:36:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- 172.21.15.122:0/2215871340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 msgr2=0x55609cebe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005ec0 tx=0x7f7fb8009650 comp rx=0 tx=0).stop 2024-04-02T13:36:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- 172.21.15.122:0/2215871340 shutdown_connections 2024-04-02T13:36:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609ceabca0 0x55609cec3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea30c0 0x55609cebe0f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2215871340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609cea27a0 0x55609ceb7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- 172.21.15.122:0/2215871340 >> 172.21.15.122:0/2215871340 conn(0x55609ce9ced0 msgr2=0x55609ce9d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- 172.21.15.122:0/2215871340 shutdown_connections 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- 172.21.15.122:0/2215871340 wait complete. 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 -- start start 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.944+0000 7f7fccc03740 1 Processor -- start 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0011340 con 0x55609cea27a0 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0010c70 con 0x55609ceabca0 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb000deb0 con 0x55609cea30c0 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fccc03740 10 client.0 _ll_register_callbacks cb 0x7f7fb40044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:19.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59486/0 (socket says 172.21.15.122:59486) 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2726892113 learned_addr learned my addr 172.21.15.122:0/2726892113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc3fff640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 msgr2=0x55609cea5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2726892113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 msgr2=0x55609cea5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 -- 172.21.15.122:0/2726892113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55609ceb2e30 con 0x55609cea27a0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fc37fe640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4005320 tx=0x7f7fb40cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d9e8ca491fb0bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.945+0000 7f7fa2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb409b850 con 0x55609cea27a0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7fb40c4dd0 con 0x55609cea27a0 2024-04-02T13:36:19.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fb802cab0 con 0x55609cea27a0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7fb40cd240 con 0x55609cea27a0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 10 client.16032 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55609ceb4130 con 0x55609cea27a0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb802cab0 con 0x55609cea27a0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7fb40b8050 con 0x55609cea27a0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 10 client.16032 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 10 client.16032 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 20 client.16032 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cf242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 4 client.16032 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55609ce0e040 con 0x55609cebbcd0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.946+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cf242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fa2ffd640 10 client.16032 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:19.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fc2ffd640 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cf242c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8299949a57721405 server_cookie=8e1bd934543503fd in_seq=0 out_seq=0 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fa2ffd640 1 -- 172.21.15.122:0/2726892113 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55609ce0e040 con 0x55609cebbcd0 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fa2ffd640 10 client.16032 handle_command_reply: tid=0 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 client.16032 shutdown 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 msgr2=0x55609cea4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4005320 tx=0x7f7fb40cf040 comp rx=0 tx=0).stop 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 shutdown_connections 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55609cebbcd0 0x55609cf242c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55609ceabca0 0x55609cea5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55609cea30c0 0x55609cea5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 --2- 172.21.15.122:0/2726892113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55609cea27a0 0x55609cea4cf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 >> 172.21.15.122:0/2726892113 conn(0x55609ce9ced0 msgr2=0x55609cea7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.947+0000 7f7fa2ffd640 0 client.16032 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 shutdown_connections 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fccc03740 1 -- 172.21.15.122:0/2726892113 wait complete. 2024-04-02T13:36:19.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fccc03740 20 client.16032 trim_cache size 0 max 16384 2024-04-02T13:36:19.956 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:19.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 msgr2=0x7f7fac0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 0x7f7fac0a6380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc009fd0 tx=0x7f7fbc009cd0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 msgr2=0x7f7fc419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d950 tx=0x7f7fb400de10 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 shutdown_connections 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fac0a3ec0 0x7f7fac0a6380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fc4107b50 0x7f7fc419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fc40696c0 0x7f7fc419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 --2- 172.21.15.122:0/2212176396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc4068d00 0x7f7fc419a560 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.948+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 >> 172.21.15.122:0/2212176396 conn(0x7f7fc4077b00 msgr2=0x7f7fc410d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.949+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 shutdown_connections 2024-04-02T13:36:19.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:19.949+0000 7f7fca01b640 1 -- 172.21.15.122:0/2212176396 wait complete. 2024-04-02T13:36:19.963 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:36:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:19 smithi187 ceph-mon[36064]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:36:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4011670689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:19 smithi188 ceph-mon[36117]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-04-02T13:36:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4011670689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.110+0000 7f7c6fb52640 1 -- 172.21.15.122:0/139485341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c6810c160 msgr2=0x7f7c6810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.110+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/139485341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c6810c160 0x7f7c6810e590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009ca0 tx=0x7f7c6002c190 comp rx=0 tx=0).stop 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 -- 172.21.15.122:0/139485341 shutdown_connections 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/139485341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c6810c160 0x7f7c6810e590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/139485341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c681036b0 0x7f7c6810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/139485341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c68102d90 0x7f7c68103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 -- 172.21.15.122:0/139485341 >> 172.21.15.122:0/139485341 conn(0x7f7c68077b60 msgr2=0x7f7c68077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 -- 172.21.15.122:0/139485341 shutdown_connections 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 -- 172.21.15.122:0/139485341 wait complete. 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 Processor -- start 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 -- start start 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.111+0000 7f7c6fb52640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c681036b0 0x7f7c681988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6fb52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 0x7f7c6819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6fb52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111ef0 con 0x7f7c68102d90 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6fb52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68111d70 con 0x7f7c6810c160 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6fb52640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68112070 con 0x7f7c681036b0 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59502/0 (socket says 172.21.15.122:59502) 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/964895688 learned_addr learned my addr 172.21.15.122:0/964895688 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c681036b0 0x7f7c681988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 0x7f7c6819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/964895688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c681036b0 msgr2=0x7f7c681988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c681036b0 0x7f7c681988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/964895688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 msgr2=0x7f7c6819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 0x7f7c6819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/964895688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6819d380 con 0x7f7c68102d90 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 0x7f7c6819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400d770 tx=0x7f7c6400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c5effd640 1 -- 172.21.15.122:0/964895688 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64017070 con 0x7f7c68102d90 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.112+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6819d670 con 0x7f7c68102d90 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.113+0000 7f7c5effd640 1 -- 172.21.15.122:0/964895688 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64005500 con 0x7f7c68102d90 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.113+0000 7f7c5effd640 1 -- 172.21.15.122:0/964895688 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004e00 con 0x7f7c68102d90 2024-04-02T13:36:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.113+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a4f00 con 0x7f7c68102d90 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.115+0000 7f7c7281e740 1 Processor -- start 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 -- start start 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5629d6ba67b0 con 0x5629d6c4bca0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5629d6b45570 con 0x5629d6c430c0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c7281e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5629d6ba2760 con 0x5629d6c427a0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59512/0 (socket says 172.21.15.122:59512) 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c5effd640 1 -- 172.21.15.122:0/964895688 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f7c64004fa0 con 0x7f7c68102d90 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/847676174 learned_addr learned my addr 172.21.15.122:0/847676174 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/847676174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 msgr2=0x5629d6c42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/847676174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 msgr2=0x5629d6c4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c4b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/847676174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d68ba3f0 con 0x5629d6c4bca0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.116+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009c10 tx=0x7f7c58011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29901258bd094503 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c53fff640 1 -- 172.21.15.122:0/847676174 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004230 con 0x5629d6c4bca0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c53fff640 1 -- 172.21.15.122:0/847676174 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c580043d0 con 0x5629d6c4bca0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c53fff640 1 -- 172.21.15.122:0/847676174 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58004e10 con 0x5629d6c4bca0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c5effd640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 0x7f7c540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c5effd640 1 -- 172.21.15.122:0/964895688 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c640d0870 con 0x7f7c68102d90 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.117+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 0x7f7c540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- 172.21.15.122:0/847676174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 msgr2=0x5629d6c4e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009c10 tx=0x7f7c58011810 comp rx=0 tx=0).stop 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 0x7f7c540a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009430 tx=0x7f7c60004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- 172.21.15.122:0/847676174 shutdown_connections 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c4e090 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 --2- 172.21.15.122:0/847676174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- 172.21.15.122:0/847676174 >> 172.21.15.122:0/847676174 conn(0x5629d6c3ced0 msgr2=0x5629d6c3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- 172.21.15.122:0/847676174 shutdown_connections 2024-04-02T13:36:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- 172.21.15.122:0/847676174 wait complete. 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.118+0000 7f7c7281e740 1 -- start start 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 Processor -- start 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c430c0 0x5629d6c48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5629d6c4ea90 con 0x5629d6c4bca0 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5629d6b61e80 con 0x5629d6c427a0 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5629d6c4ec10 con 0x5629d6c430c0 2024-04-02T13:36:20.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c6d8ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c6d8ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36840/0 (socket says 172.21.15.122:36840) 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/424907621 learned_addr learned my addr 172.21.15.122:0/424907621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c7281e740 10 client.0 _ll_register_callbacks cb 0x5629d6b9daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c430c0 0x5629d6c48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.119+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/424907621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c430c0 msgr2=0x5629d6c48a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c430c0 0x5629d6c48a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/424907621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 msgr2=0x5629d6c49050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c49050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 -- 172.21.15.122:0/424907621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d69847d0 con 0x5629d6c427a0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009610 tx=0x7f7c640056b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ce3ba4be2291d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64020040 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c640988a0 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c580043d0 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c6409c230 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c7281e740 10 client.25544 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629d6c5a9e0 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.120+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c580043d0 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c640a0120 con 0x5629d6c427a0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 10 client.25544 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 10 client.25544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 20 client.25544 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c5c390 0x5629d6c5e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 4 client.25544 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629d6bb3cf0 con 0x5629d6c5c390 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c5c390 0x5629d6c5e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c517fa640 10 client.25544 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.121+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c5c390 0x5629d6c5e800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78d7a6a748cabc1 server_cookie=f5905620d7cf9690 in_seq=0 out_seq=0 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c517fa640 1 -- 172.21.15.122:0/424907621 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629d6bb3cf0 con 0x5629d6c5c390 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c517fa640 10 client.25544 handle_command_reply: tid=0 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 client.25544 shutdown 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 msgr2=0x5629d6c4b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009610 tx=0x7f7c640056b0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 shutdown_connections 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c5c390 0x5629d6c5e800 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c430c0 0x5629d6c48a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 --2- 172.21.15.122:0/424907621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c427a0 0x5629d6c4b620 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 >> 172.21.15.122:0/424907621 conn(0x5629d6c3ced0 msgr2=0x5629d6c3ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.122+0000 7f7c517fa640 0 client.25544 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.123+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 shutdown_connections 2024-04-02T13:36:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.123+0000 7f7c7281e740 1 -- 172.21.15.122:0/424907621 wait complete. 2024-04-02T13:36:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.123+0000 7f7c7281e740 20 client.25544 trim_cache size 0 max 16384 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.148+0000 7f7c7281e740 1 Processor -- start 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.148+0000 7f7c7281e740 1 -- start start 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.148+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.148+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c4bca0 0x5629d6c48320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c7281e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681aef70 con 0x5629d6c430c0 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c7281e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b0ab0 con 0x5629d6c4bca0 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c7281e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681af640 con 0x5629d6c427a0 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59540/0 (socket says 172.21.15.122:59540) 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/671683165 learned_addr learned my addr 172.21.15.122:0/671683165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c4bca0 0x5629d6c48320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/671683165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 msgr2=0x5629d6c57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/671683165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c4bca0 msgr2=0x5629d6c48320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c4bca0 0x5629d6c48320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 -- 172.21.15.122:0/671683165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629d6c493c0 con 0x5629d6c430c0 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7c58004940 tx=0x7f7c58005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d91867843e14ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.149+0000 7f7c537fe640 1 -- 172.21.15.122:0/671683165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580097a0 con 0x5629d6c430c0 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c537fe640 1 -- 172.21.15.122:0/671683165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c58004530 con 0x5629d6c430c0 2024-04-02T13:36:20.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c537fe640 1 -- 172.21.15.122:0/671683165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580035f0 con 0x5629d6c430c0 2024-04-02T13:36:20.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 -- 172.21.15.122:0/671683165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 msgr2=0x5629d6c489e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7c58004940 tx=0x7f7c58005850 comp rx=0 tx=0).stop 2024-04-02T13:36:20.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 -- 172.21.15.122:0/671683165 shutdown_connections 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c4bca0 0x5629d6c48320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c430c0 0x5629d6c489e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 --2- 172.21.15.122:0/671683165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 -- 172.21.15.122:0/671683165 >> 172.21.15.122:0/671683165 conn(0x5629d6c3ced0 msgr2=0x5629d6c3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 -- 172.21.15.122:0/671683165 shutdown_connections 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.150+0000 7f7c7281e740 1 -- 172.21.15.122:0/671683165 wait complete. 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 -- start start 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 Processor -- start 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681b03e0 con 0x5629d6c4bca0 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681afd10 con 0x5629d6c430c0 2024-04-02T13:36:20.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681acf50 con 0x5629d6c427a0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c7281e740 10 client.0 _ll_register_callbacks cb 0x7f7c6400de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c6e0ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c6e0ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59550/0 (socket says 172.21.15.122:59550) 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c6e0ef640 1 -- 172.21.15.122:0/3799269526 learned_addr learned my addr 172.21.15.122:0/3799269526 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.151+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6d0ed640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 -- 172.21.15.122:0/3799269526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 msgr2=0x5629d6c542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 -- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 msgr2=0x5629d6c45e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c45e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 -- 172.21.15.122:0/3799269526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629d6c42ef0 con 0x5629d6c4bca0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c6e0ef640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009ca0 tx=0x7f7c6002c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87171d1f60979e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c517fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60051030 con 0x5629d6c4bca0 2024-04-02T13:36:20.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7c600047a0 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c58004530 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7c60039040 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c7281e740 10 client.16050 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.152+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5629d6c52ec0 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c60046040 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7c60034880 con 0x5629d6c4bca0 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 10 client.16050 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 20 client.16050 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c53690 0x5629d6cc4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:20.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 4 client.16050 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7c58004530 con 0x5629d6c53690 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c53690 0x5629d6cc4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c6d8ee640 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c53690 0x5629d6cc4990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb5051b8bf3e1a server_cookie=e04d052f3fee44cc in_seq=0 out_seq=0 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.153+0000 7f7c517fa640 10 client.16050 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c517fa640 1 -- 172.21.15.122:0/3799269526 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7c58004530 con 0x5629d6c53690 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c517fa640 10 client.16050 handle_command_reply: tid=0 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 client.16050 shutdown 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 msgr2=0x5629d6c46370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009ca0 tx=0x7f7c6002c190 comp rx=0 tx=0).stop 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 shutdown_connections 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5629d6c53690 0x5629d6cc4990 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5629d6c4bca0 0x5629d6c46370 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5629d6c430c0 0x5629d6c45e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 --2- 172.21.15.122:0/3799269526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5629d6c427a0 0x5629d6c542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 >> 172.21.15.122:0/3799269526 conn(0x5629d6c3ced0 msgr2=0x5629d6c3b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c517fa640 0 client.16050 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 shutdown_connections 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 1 -- 172.21.15.122:0/3799269526 wait complete. 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.154+0000 7f7c7281e740 20 client.16050 trim_cache size 0 max 16384 2024-04-02T13:36:20.163 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 msgr2=0x7f7c540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 0x7f7c540a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009430 tx=0x7f7c60004a40 comp rx=0 tx=0).stop 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 msgr2=0x7f7c68198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400d770 tx=0x7f7c6400dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 shutdown_connections 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7c540a3e40 0x7f7c540a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c6810c160 0x7f7c6819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c681036b0 0x7f7c681988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 --2- 172.21.15.122:0/964895688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c68102d90 0x7f7c68198390 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 >> 172.21.15.122:0/964895688 conn(0x7f7c68077b60 msgr2=0x7f7c680fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 shutdown_connections 2024-04-02T13:36:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:20.155+0000 7f7c6fb52640 1 -- 172.21.15.122:0/964895688 wait complete. 2024-04-02T13:36:21.171 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:36:21.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:21.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:36:21.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 -- 172.21.15.122:0/2036623320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18103690 msgr2=0x7efd1810bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 --2- 172.21.15.122:0/2036623320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18103690 0x7efd1810bb70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7efd08009900 tx=0x7efd0802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 -- 172.21.15.122:0/2036623320 shutdown_connections 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 --2- 172.21.15.122:0/2036623320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 0x7efd1810e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 --2- 172.21.15.122:0/2036623320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18103690 0x7efd1810bb70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 --2- 172.21.15.122:0/2036623320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18102d70 0x7efd18103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 -- 172.21.15.122:0/2036623320 >> 172.21.15.122:0/2036623320 conn(0x7efd18077b60 msgr2=0x7efd18077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 -- 172.21.15.122:0/2036623320 shutdown_connections 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.287+0000 7efd1faa5640 1 -- 172.21.15.122:0/2036623320 wait complete. 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 Processor -- start 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 -- start start 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18103690 0x7efd18100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 0x7efd18101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111f10 con 0x7efd18102d70 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd18111d90 con 0x7efd1810c140 2024-04-02T13:36:21.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1faa5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd18112090 con 0x7efd18103690 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1d841640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1d841640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59566/0 (socket says 172.21.15.122:59566) 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1d841640 1 -- 172.21.15.122:0/735438076 learned_addr learned my addr 172.21.15.122:0/735438076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1e042640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 0x7efd18101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.288+0000 7efd1d040640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18103690 0x7efd18100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 -- 172.21.15.122:0/735438076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18103690 msgr2=0x7efd18100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18103690 0x7efd18100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 -- 172.21.15.122:0/735438076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 msgr2=0x7efd18101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 0x7efd18101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 -- 172.21.15.122:0/735438076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd181a68f0 con 0x7efd18102d70 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1d841640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d950 tx=0x7efd1400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd0effd640 1 -- 172.21.15.122:0/735438076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14017070 con 0x7efd18102d70 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd181a6be0 con 0x7efd18102d70 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd0effd640 1 -- 172.21.15.122:0/735438076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14009c30 con 0x7efd18102d70 2024-04-02T13:36:21.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd0effd640 1 -- 172.21.15.122:0/735438076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1400f6b0 con 0x7efd18102d70 2024-04-02T13:36:21.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.289+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a7150 con 0x7efd18102d70 2024-04-02T13:36:21.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.292+0000 7efd0effd640 1 -- 172.21.15.122:0/735438076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7efd1400f850 con 0x7efd18102d70 2024-04-02T13:36:21.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.292+0000 7efd2276f740 1 Processor -- start 2024-04-02T13:36:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 -- start start 2024-04-02T13:36:21.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce94be0 0x5565bce9d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 0x5565bce9fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5565bcdf8770 con 0x5565bce942c0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5565bcda0960 con 0x5565bce9d7c0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd2276f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565bcdf48c0 con 0x5565bce94be0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd1e042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd1e042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59578/0 (socket says 172.21.15.122:59578) 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd1e042640 1 -- 172.21.15.122:0/3303500485 learned_addr learned my addr 172.21.15.122:0/3303500485 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd1d040640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 0x5565bce9fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd1d841640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce94be0 0x5565bce9d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.293+0000 7efd0effd640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 0x7efd040a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 -- 172.21.15.122:0/3303500485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce94be0 msgr2=0x5565bce9d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce94be0 0x5565bce9d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 -- 172.21.15.122:0/3303500485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 msgr2=0x5565bce9fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 0x5565bce9fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 -- 172.21.15.122:0/3303500485 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bcbd47e0 con 0x5565bce942c0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1d040640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 0x5565bce9fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd0effd640 1 -- 172.21.15.122:0/735438076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7efd140d0980 con 0x7efd18102d70 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 0x7efd040a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efd10010440 tx=0x7efd10010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3938eb86ad50ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd03fff640 1 -- 172.21.15.122:0/3303500485 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1001d400 con 0x5565bce942c0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd03fff640 1 -- 172.21.15.122:0/3303500485 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd100134c0 con 0x5565bce942c0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd1e042640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 0x7efd040a6330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd10021000 tx=0x7efd1001e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.294+0000 7efd03fff640 1 -- 172.21.15.122:0/3303500485 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10011940 con 0x5565bce942c0 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 -- 172.21.15.122:0/3303500485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 msgr2=0x5565bce946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7efd10010440 tx=0x7efd10010900 comp rx=0 tx=0).stop 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 -- 172.21.15.122:0/3303500485 shutdown_connections 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce9d7c0 0x5565bce9fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce94be0 0x5565bce9d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 --2- 172.21.15.122:0/3303500485 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce946a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 -- 172.21.15.122:0/3303500485 >> 172.21.15.122:0/3303500485 conn(0x5565bce8e9f0 msgr2=0x5565bce8edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 -- 172.21.15.122:0/3303500485 shutdown_connections 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.295+0000 7efd2276f740 1 -- 172.21.15.122:0/3303500485 wait complete. 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 -- start start 2024-04-02T13:36:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 Processor -- start 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce942c0 0x5565bce9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bce9ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5565bcea05b0 con 0x5565bce9d7c0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5565bcdb3d20 con 0x5565bce94be0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565bcea0730 con 0x5565bce942c0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.296+0000 7efd2276f740 10 client.0 _ll_register_callbacks cb 0x5565bcd80a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59594/0 (socket says 172.21.15.122:59594) 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 -- 172.21.15.122:0/2159925525 learned_addr learned my addr 172.21.15.122:0/2159925525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1d841640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce942c0 0x5565bce9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1d040640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bce9ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 -- 172.21.15.122:0/2159925525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce942c0 msgr2=0x5565bce9a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce942c0 0x5565bce9a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 -- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 msgr2=0x5565bce9ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bce9ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 -- 172.21.15.122:0/2159925525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bcc34f70 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1d040640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bce9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd1e042640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efd10009c70 tx=0x7efd10009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1745f9d18910c495 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10035060 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7efd10011b50 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd100134c0 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd1003c050 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd2276f740 10 client.16062 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.297+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565bcacc170 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd10011d50 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd1000f040 con 0x5565bce9d7c0 2024-04-02T13:36:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 10 client.16062 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 20 client.16062 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bceadb00 0x5565bceafef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 4 client.16062 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd100134c0 con 0x5565bceadb00 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.298+0000 7efd1d841640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bceadb00 0x5565bceafef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.299+0000 7efd1d841640 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bceadb00 0x5565bceafef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce6e18b102791a66 server_cookie=b2cbd064fa90b0c9 in_seq=0 out_seq=0 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.299+0000 7efd017fa640 10 client.16062 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.299+0000 7efd017fa640 1 -- 172.21.15.122:0/2159925525 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7efd100134c0 con 0x5565bceadb00 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.299+0000 7efd017fa640 10 client.16062 handle_command_reply: tid=0 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 client.16062 shutdown 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 msgr2=0x5565bce9b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7efd10009c70 tx=0x7efd10009ca0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 shutdown_connections 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bceadb00 0x5565bceafef0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce9d7c0 0x5565bce9b1c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd017fa640 0 client.16062 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bce9ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 --2- 172.21.15.122:0/2159925525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce942c0 0x5565bce9a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 >> 172.21.15.122:0/2159925525 conn(0x5565bce8e9f0 msgr2=0x5565bce90740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 shutdown_connections 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 1 -- 172.21.15.122:0/2159925525 wait complete. 2024-04-02T13:36:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.300+0000 7efd2276f740 20 client.16062 trim_cache size 0 max 16384 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.317+0000 7efd2276f740 1 Processor -- start 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.317+0000 7efd2276f740 1 -- start start 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.317+0000 7efd2276f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd2276f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bcea94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd2276f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bceb5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd2276f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd181b1140 con 0x5565bce942c0 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd2276f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd181b2c80 con 0x5565bce94be0 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd2276f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd181b1810 con 0x5565bce9d7c0 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59600/0 (socket says 172.21.15.122:59600) 2024-04-02T13:36:21.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 -- 172.21.15.122:0/2030798069 learned_addr learned my addr 172.21.15.122:0/2030798069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1e042640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bceb5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d040640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bcea94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 -- 172.21.15.122:0/2030798069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 msgr2=0x5565bceb5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bceb5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 -- 172.21.15.122:0/2030798069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 msgr2=0x5565bcea94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bcea94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 -- 172.21.15.122:0/2030798069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bce996f0 con 0x5565bce942c0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1e042640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bceb5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd1d841640 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd14009610 tx=0x7efd140d2e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30856d95cda6246e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd037fe640 1 -- 172.21.15.122:0/2030798069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14020040 con 0x5565bce942c0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd037fe640 1 -- 172.21.15.122:0/2030798069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14009de0 con 0x5565bce942c0 2024-04-02T13:36:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.318+0000 7efd037fe640 1 -- 172.21.15.122:0/2030798069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1409a780 con 0x5565bce942c0 2024-04-02T13:36:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 -- 172.21.15.122:0/2030798069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 msgr2=0x5565bce9ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd14009610 tx=0x7efd140d2e90 comp rx=0 tx=0).stop 2024-04-02T13:36:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 -- 172.21.15.122:0/2030798069 shutdown_connections 2024-04-02T13:36:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bceb5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce94be0 0x5565bcea94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 --2- 172.21.15.122:0/2030798069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce942c0 0x5565bce9ce70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.319+0000 7efd2276f740 1 -- 172.21.15.122:0/2030798069 >> 172.21.15.122:0/2030798069 conn(0x5565bce8e9f0 msgr2=0x5565bce8dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- 172.21.15.122:0/2030798069 shutdown_connections 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- 172.21.15.122:0/2030798069 wait complete. 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- start start 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 Processor -- start 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 0x5565bcea5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bcea2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd181b25b0 con 0x5565bce94be0 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efd181b1ee0 con 0x5565bce942c0 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.320+0000 7efd2276f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd181af120 con 0x5565bce9d7c0 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59606/0 (socket says 172.21.15.122:59606) 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd2276f740 10 client.0 _ll_register_callbacks cb 0x7efd10013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 -- 172.21.15.122:0/2041310052 learned_addr learned my addr 172.21.15.122:0/2041310052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d841640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 0x5565bcea5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1e042640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bcea2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 -- 172.21.15.122:0/2041310052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 msgr2=0x5565bcea2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bcea2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 -- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 msgr2=0x5565bcea5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 0x5565bcea5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 -- 172.21.15.122:0/2041310052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bce94a10 con 0x5565bce94be0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d841640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 0x5565bcea5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd1d040640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd08009230 tx=0x7efd08004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd6d966e75b3e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0804c060 con 0x5565bce94be0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7efd08004340 con 0x5565bce94be0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.321+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd14009de0 con 0x5565bce94be0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efd08053050 con 0x5565bce94be0 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 10 client.16068 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5565bcf14bd0 con 0x5565bce94be0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14009de0 con 0x5565bce94be0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7efd08005470 con 0x5565bce94be0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 10 client.16068 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 20 client.16068 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bcf15380 0x5565bcf177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 4 client.16068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565bce05cb0 con 0x5565bcf15380 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.322+0000 7efd1d841640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bcf15380 0x5565bcf177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.323+0000 7efd017fa640 10 client.16068 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.323+0000 7efd1d841640 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bcf15380 0x5565bcf177a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ee8ae15606913d server_cookie=a0f2a220af1c312d in_seq=0 out_seq=0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd017fa640 1 -- 172.21.15.122:0/2041310052 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565bce05cb0 con 0x5565bcf15380 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd017fa640 10 client.16068 handle_command_reply: tid=0 2024-04-02T13:36:21.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd2276f740 1 client.16068 shutdown 2024-04-02T13:36:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 msgr2=0x5565bcea1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd08009230 tx=0x7efd08004040 comp rx=0 tx=0).stop 2024-04-02T13:36:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.324+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 shutdown_connections 2024-04-02T13:36:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5565bcf15380 0x5565bcf177a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565bce9d7c0 0x5565bcea2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd017fa640 0 client.16068 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565bce94be0 0x5565bcea1fc0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 --2- 172.21.15.122:0/2041310052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565bce942c0 0x5565bcea5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 >> 172.21.15.122:0/2041310052 conn(0x5565bce8e9f0 msgr2=0x5565bce9a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 shutdown_connections 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 1 -- 172.21.15.122:0/2041310052 wait complete. 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd2276f740 20 client.16068 trim_cache size 0 max 16384 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "d6b7bafd-b020-4fa6-9620-83d133908a97", 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:36:21.332 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 msgr2=0x7efd040a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 0x7efd040a6330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd10021000 tx=0x7efd1001e000 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 msgr2=0x7efd181009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efd1400d950 tx=0x7efd1400de10 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 shutdown_connections 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efd040a3e70 0x7efd040a6330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efd1810c140 0x7efd18101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd18103690 0x7efd18100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 --2- 172.21.15.122:0/735438076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd18102d70 0x7efd181009c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.325+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 >> 172.21.15.122:0/735438076 conn(0x7efd18077b60 msgr2=0x7efd180fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.326+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 shutdown_connections 2024-04-02T13:36:21.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.326+0000 7efd1faa5640 1 -- 172.21.15.122:0/735438076 wait complete. 2024-04-02T13:36:21.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:36:21.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:36:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.445+0000 7fa64d80f640 1 -- 172.21.15.122:0/1819372947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa648101360 msgr2=0x7fa64810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.445+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1819372947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa648101360 0x7fa64810d820 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fa640009ca0 tx=0x7fa64002c180 comp rx=0 tx=0).stop 2024-04-02T13:36:21.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 -- 172.21.15.122:0/1819372947 shutdown_connections 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1819372947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa648101360 0x7fa64810d820 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1819372947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 0x7fa648100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1819372947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa6481058a0 0x7fa648105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 -- 172.21.15.122:0/1819372947 >> 172.21.15.122:0/1819372947 conn(0x7fa648077a90 msgr2=0x7fa648077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 -- 172.21.15.122:0/1819372947 shutdown_connections 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 -- 172.21.15.122:0/1819372947 wait complete. 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 Processor -- start 2024-04-02T13:36:21.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.446+0000 7fa64d80f640 1 -- start start 2024-04-02T13:36:21.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 0x7fa648198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa648101360 0x7fa648198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa648110d90 con 0x7fa6481058a0 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa648110c10 con 0x7fa648101360 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa64d80f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa648110f10 con 0x7fa6481009c0 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59636/0 (socket says 172.21.15.122:59636) 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 -- 172.21.15.122:0/1567861698 learned_addr learned my addr 172.21.15.122:0/1567861698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6467fc640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa648101360 0x7fa648198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa646ffd640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 0x7fa648198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 -- 172.21.15.122:0/1567861698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 msgr2=0x7fa648198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 0x7fa648198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 -- 172.21.15.122:0/1567861698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa648101360 msgr2=0x7fa648198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa648101360 0x7fa648198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 -- 172.21.15.122:0/1567861698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64819d340 con 0x7fa6481058a0 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.447+0000 7fa6477fe640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa64002c790 tx=0x7fa640035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.448+0000 7fa64c80d640 1 -- 172.21.15.122:0/1567861698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa640039040 con 0x7fa6481058a0 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.448+0000 7fa64c80d640 1 -- 172.21.15.122:0/1567861698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa640044070 con 0x7fa6481058a0 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.448+0000 7fa64c80d640 1 -- 172.21.15.122:0/1567861698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400329f0 con 0x7fa6481058a0 2024-04-02T13:36:21.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.448+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64819d5d0 con 0x7fa6481058a0 2024-04-02T13:36:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.448+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa64819d9b0 con 0x7fa6481058a0 2024-04-02T13:36:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.450+0000 7fa64c80d640 1 -- 172.21.15.122:0/1567861698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fa640032b90 con 0x7fa6481058a0 2024-04-02T13:36:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa64c80d640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 0x7fa6300a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa64c80d640 1 -- 172.21.15.122:0/1567861698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa6400f3300 con 0x7fa6481058a0 2024-04-02T13:36:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa646ffd640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 0x7fa6300a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa6503f0740 1 Processor -- start 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa6503f0740 1 -- start start 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa646ffd640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 0x7fa6300a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa638005fd0 tx=0x7fa638005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.451+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0273890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b02761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6503f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5635b01ce830 con 0x5635b026a8d0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6503f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5635b0176820 con 0x5635b026b1f0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6503f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5635b01ca7e0 con 0x5635b0273dd0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59652/0 (socket says 172.21.15.122:59652) 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 -- 172.21.15.122:0/2227105266 learned_addr learned my addr 172.21.15.122:0/2227105266 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa6467fc640 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0273890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 -- 172.21.15.122:0/2227105266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 msgr2=0x5635b02761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b02761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 -- 172.21.15.122:0/2227105266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 msgr2=0x5635b0273890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0273890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 -- 172.21.15.122:0/2227105266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635affac7d0 con 0x5635b026a8d0 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa646ffd640 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa63800fe50 tx=0x7fa638012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f5a99910cea31a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa62dffb640 1 -- 172.21.15.122:0/2227105266 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638022070 con 0x5635b026a8d0 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa62dffb640 1 -- 172.21.15.122:0/2227105266 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa63800fe80 con 0x5635b026a8d0 2024-04-02T13:36:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.452+0000 7fa62dffb640 1 -- 172.21.15.122:0/2227105266 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638010990 con 0x5635b026a8d0 2024-04-02T13:36:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 -- 172.21.15.122:0/2227105266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 msgr2=0x5635b026acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa63800fe50 tx=0x7fa638012000 comp rx=0 tx=0).stop 2024-04-02T13:36:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 -- 172.21.15.122:0/2227105266 shutdown_connections 2024-04-02T13:36:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b02761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0273890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2227105266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026a8d0 0x5635b026acb0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.453+0000 7fa6503f0740 1 -- 172.21.15.122:0/2227105266 >> 172.21.15.122:0/2227105266 conn(0x5635b0265000 msgr2=0x5635b02653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- 172.21.15.122:0/2227105266 shutdown_connections 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- 172.21.15.122:0/2227105266 wait complete. 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- start start 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 Processor -- start 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 0x5635b02736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0270a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5635b0276bc0 con 0x5635b0273dd0 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5635b0189d30 con 0x5635b026b1f0 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5635b0276d40 con 0x5635b026a8d0 2024-04-02T13:36:21.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.454+0000 7fa6503f0740 10 client.0 _ll_register_callbacks cb 0x5635b0189720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59656/0 (socket says 172.21.15.122:59656) 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 -- 172.21.15.122:0/3793587076 learned_addr learned my addr 172.21.15.122:0/3793587076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6467fc640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0270a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa646ffd640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 0x5635b02736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 -- 172.21.15.122:0/3793587076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 msgr2=0x5635b02736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 0x5635b02736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 -- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 msgr2=0x5635b0270a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0270a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 -- 172.21.15.122:0/3793587076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635afee23f0 con 0x5635b0273dd0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa646ffd640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 0x5635b02736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6477fe640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa640009dc0 tx=0x7fa6400be910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c46814220887e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400f3160 con 0x5635b0273dd0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa6400bf030 con 0x5635b0273dd0 2024-04-02T13:36:21.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635b01dbd70 con 0x5635b0273dd0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.455+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6400335a0 con 0x5635b0273dd0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635afea20f0 con 0x5635b0273dd0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5635b01dbd70 con 0x5635b0273dd0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa64003c030 con 0x5635b0273dd0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 20 client.16080 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b0283db0 0x5635b02861a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 4 client.16080 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635b01d60c0 con 0x5635b0283db0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.456+0000 7fa646ffd640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b0283db0 0x5635b02861a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.457+0000 7fa626ffd640 10 client.16080 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.457+0000 7fa646ffd640 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b0283db0 0x5635b02861a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620596ec31da1102 server_cookie=845064c362cf3ca1 in_seq=0 out_seq=0 2024-04-02T13:36:21.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa626ffd640 1 -- 172.21.15.122:0/3793587076 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635b01d60c0 con 0x5635b0283db0 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa626ffd640 10 client.16080 handle_command_reply: tid=0 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 client.16080 shutdown 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 msgr2=0x5635b0270f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa640009dc0 tx=0x7fa6400be910 comp rx=0 tx=0).stop 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 shutdown_connections 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b0283db0 0x5635b02861a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0270f90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026b1f0 0x5635b0270a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 --2- 172.21.15.122:0/3793587076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026a8d0 0x5635b02736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 >> 172.21.15.122:0/3793587076 conn(0x5635b0265000 msgr2=0x5635b0266d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa626ffd640 0 client.16080 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 shutdown_connections 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 1 -- 172.21.15.122:0/3793587076 wait complete. 2024-04-02T13:36:21.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.458+0000 7fa6503f0740 20 client.16080 trim_cache size 0 max 16384 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.475+0000 7fa6503f0740 1 Processor -- start 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.475+0000 7fa6503f0740 1 -- start start 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b026c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026b1f0 0x5635b02640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481aef90 con 0x5635b0273dd0 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b0ad0 con 0x5635b026a8d0 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6503f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481af660 con 0x5635b026b1f0 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59672/0 (socket says 172.21.15.122:59672) 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 -- 172.21.15.122:0/2945317373 learned_addr learned my addr 172.21.15.122:0/2945317373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6467fc640 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026b1f0 0x5635b02640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa646ffd640 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b026c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 -- 172.21.15.122:0/2945317373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026b1f0 msgr2=0x5635b02640c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026b1f0 0x5635b02640c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 -- 172.21.15.122:0/2945317373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 msgr2=0x5635b026c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b026c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 -- 172.21.15.122:0/2945317373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635b0260c50 con 0x5635b0273dd0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.476+0000 7fa6477fe640 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa6400bb750 tx=0x7fa6400bd2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1821c9fc4449776 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa62d7fa640 1 -- 172.21.15.122:0/2945317373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400f3160 con 0x5635b0273dd0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa62d7fa640 1 -- 172.21.15.122:0/2945317373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6400f0590 con 0x5635b0273dd0 2024-04-02T13:36:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa62d7fa640 1 -- 172.21.15.122:0/2945317373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6400be730 con 0x5635b0273dd0 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 -- 172.21.15.122:0/2945317373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 msgr2=0x5635b0273180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa6400bb750 tx=0x7fa6400bd2e0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 -- 172.21.15.122:0/2945317373 shutdown_connections 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b0273dd0 0x5635b0273180 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b026b1f0 0x5635b02640c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 --2- 172.21.15.122:0/2945317373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b026c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.477+0000 7fa6503f0740 1 -- 172.21.15.122:0/2945317373 >> 172.21.15.122:0/2945317373 conn(0x5635b0265000 msgr2=0x5635b02639b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- 172.21.15.122:0/2945317373 shutdown_connections 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- 172.21.15.122:0/2945317373 wait complete. 2024-04-02T13:36:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- start start 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 Processor -- start 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b027c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b026da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481b0400 con 0x5635b026b1f0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481afd30 con 0x5635b026a8d0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6481acf70 con 0x5635b0273dd0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6503f0740 10 client.0 _ll_register_callbacks cb 0x7fa6400ba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59680/0 (socket says 172.21.15.122:59680) 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.478+0000 7fa6467fc640 1 -- 172.21.15.122:0/174124890 learned_addr learned my addr 172.21.15.122:0/174124890 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa646ffd640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b027c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6477fe640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b026da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 -- 172.21.15.122:0/174124890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 msgr2=0x5635b026da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b026da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 -- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 msgr2=0x5635b027c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b027c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 -- 172.21.15.122:0/174124890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635b026e260 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6467fc640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5635b026a3a0 tx=0x7fa63c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afafce6fbf777c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c005220 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa63c0053c0 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6400f0590 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa63c027050 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6503f0740 10 client.16086 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.479+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5635b02ee810 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6400f0590 con 0x5635b026b1f0 2024-04-02T13:36:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa63c01b070 con 0x5635b026b1f0 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 10 client.16086 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 20 client.16086 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b02ef040 0x5635b02f1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 4 client.16086 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635b0153560 con 0x5635b02ef040 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.480+0000 7fa646ffd640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b02ef040 0x5635b02f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa646ffd640 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b02ef040 0x5635b02f1460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6b387b1ae591ea5 server_cookie=f359ac473b182eec in_seq=0 out_seq=0 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa626ffd640 10 client.16086 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa626ffd640 1 -- 172.21.15.122:0/174124890 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5635b0153560 con 0x5635b02ef040 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa626ffd640 10 client.16086 handle_command_reply: tid=0 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 client.16086 shutdown 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 msgr2=0x5635b026d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5635b026a3a0 tx=0x7fa63c0128e0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 shutdown_connections 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5635b02ef040 0x5635b02f1460 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5635b0273dd0 0x5635b026da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa626ffd640 0 client.16086 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5635b026b1f0 0x5635b026d4d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 --2- 172.21.15.122:0/174124890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5635b026a8d0 0x5635b027c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 >> 172.21.15.122:0/174124890 conn(0x5635b0265000 msgr2=0x5635b0264240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 shutdown_connections 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.481+0000 7fa6503f0740 1 -- 172.21.15.122:0/174124890 wait complete. 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa6503f0740 20 client.16086 trim_cache size 0 max 16384 2024-04-02T13:36:21.490 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:36:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:36:21.491 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:21.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 msgr2=0x7fa6300a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 0x7fa6300a6380 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa638005fd0 tx=0x7fa638005850 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 msgr2=0x7fa64819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa64002c790 tx=0x7fa640035ed0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 shutdown_connections 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa6300a3ec0 0x7fa6300a6380 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6481058a0 0x7fa64819cc10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa648101360 0x7fa648198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 --2- 172.21.15.122:0/1567861698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6481009c0 0x7fa648198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 >> 172.21.15.122:0/1567861698 conn(0x7fa648077a90 msgr2=0x7fa64806a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 shutdown_connections 2024-04-02T13:36:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.482+0000 7fa64d80f640 1 -- 172.21.15.122:0/1567861698 wait complete. 2024-04-02T13:36:21.497 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:36:21.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 -- 172.21.15.122:0/1237885921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c10c140 msgr2=0x7fa52c10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 --2- 172.21.15.122:0/1237885921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c10c140 0x7fa52c10e5e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa524009ca0 tx=0x7fa52402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 -- 172.21.15.122:0/1237885921 shutdown_connections 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 --2- 172.21.15.122:0/1237885921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c10c140 0x7fa52c10e5e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 --2- 172.21.15.122:0/1237885921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c103690 0x7fa52c10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 --2- 172.21.15.122:0/1237885921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c102d70 0x7fa52c103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 -- 172.21.15.122:0/1237885921 >> 172.21.15.122:0/1237885921 conn(0x7fa52c077b00 msgr2=0x7fa52c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 -- 172.21.15.122:0/1237885921 shutdown_connections 2024-04-02T13:36:21.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.600+0000 7fa5344eb640 1 -- 172.21.15.122:0/1237885921 wait complete. 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 Processor -- start 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 -- start start 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c102d70 0x7fa52c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c10c140 0x7fa52c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c111f40 con 0x7fa52c103690 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c111dc0 con 0x7fa52c102d70 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.601+0000 7fa5344eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1120c0 con 0x7fa52c10c140 2024-04-02T13:36:21.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59690/0 (socket says 172.21.15.122:59690) 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 -- 172.21.15.122:0/3434741048 learned_addr learned my addr 172.21.15.122:0/3434741048 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa532a88640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c10c140 0x7fa52c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa532287640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c102d70 0x7fa52c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 -- 172.21.15.122:0/3434741048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c10c140 msgr2=0x7fa52c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c10c140 0x7fa52c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 -- 172.21.15.122:0/3434741048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c102d70 msgr2=0x7fa52c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c102d70 0x7fa52c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 -- 172.21.15.122:0/3434741048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c19d350 con 0x7fa52c103690 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa531a86640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d950 tx=0x7fa51c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c017030 con 0x7fa52c103690 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c19d640 con 0x7fa52c103690 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa51c004020 con 0x7fa52c103690 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0047a0 con 0x7fa52c103690 2024-04-02T13:36:21.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.602+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c1a4ed0 con 0x7fa52c103690 2024-04-02T13:36:21.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.606+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fa51c004940 con 0x7fa52c103690 2024-04-02T13:36:21.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.606+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52c0688f0 con 0x7fa52c103690 2024-04-02T13:36:21.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.607+0000 7fa5237fe640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 0x7fa5180a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:21.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.607+0000 7fa532287640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 0x7fa5180a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:21.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.608+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa51c0d1870 con 0x7fa52c103690 2024-04-02T13:36:21.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.608+0000 7fa532287640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 0x7fa5180a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5280095f0 tx=0x7fa528005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:21.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.612+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa51c09d8d0 con 0x7fa52c103690 2024-04-02T13:36:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:36:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:36:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:36:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:21 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:36:21.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.921+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52c1a5190 con 0x7fa52c103690 2024-04-02T13:36:21.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.922+0000 7fa5237fe640 1 -- 172.21.15.122:0/3434741048 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7fa51c09d1d0 con 0x7fa52c103690 2024-04-02T13:36:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:36:21.926 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:36:21.926 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.924+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 msgr2=0x7fa5180a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.924+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 0x7fa5180a6300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa5280095f0 tx=0x7fa528005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.924+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 msgr2=0x7fa52c1988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.924+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa51c00d950 tx=0x7fa51c00de10 comp rx=0 tx=0).stop 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 shutdown_connections 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5180a3e40 0x7fa5180a6300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa52c10c140 0x7fa52c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa52c103690 0x7fa52c1988a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 --2- 172.21.15.122:0/3434741048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa52c102d70 0x7fa52c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 >> 172.21.15.122:0/3434741048 conn(0x7fa52c077b00 msgr2=0x7fa52c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 shutdown_connections 2024-04-02T13:36:21.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:21.925+0000 7fa5344eb640 1 -- 172.21.15.122:0/3434741048 wait complete. 2024-04-02T13:36:21.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:21 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:21 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:36:22.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 -- 172.21.15.122:0/3254724370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba200696a0 msgr2=0x7fba2010e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 --2- 172.21.15.122:0/3254724370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba200696a0 0x7fba2010e940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fba18009cd0 tx=0x7fba1802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 -- 172.21.15.122:0/3254724370 shutdown_connections 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 --2- 172.21.15.122:0/3254724370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba200696a0 0x7fba2010e940 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 --2- 172.21.15.122:0/3254724370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 --2- 172.21.15.122:0/3254724370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba20107a80 0x7fba20107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 -- 172.21.15.122:0/3254724370 >> 172.21.15.122:0/3254724370 conn(0x7fba20077b00 msgr2=0x7fba20077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 -- 172.21.15.122:0/3254724370 shutdown_connections 2024-04-02T13:36:22.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.050+0000 7fba273a8640 1 -- 172.21.15.122:0/3254724370 wait complete. 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 Processor -- start 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 -- start start 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 0x7fba201988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba20111e90 con 0x7fba20107a80 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba20111d10 con 0x7fba20068ce0 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba273a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba20112010 con 0x7fba200696a0 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba25945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba25945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59714/0 (socket says 172.21.15.122:59714) 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba25945640 1 -- 172.21.15.122:0/3637457887 learned_addr learned my addr 172.21.15.122:0/3637457887 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.051+0000 7fba25144640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba24943640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 0x7fba201988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 -- 172.21.15.122:0/3637457887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 msgr2=0x7fba201988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 0x7fba201988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 -- 172.21.15.122:0/3637457887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 msgr2=0x7fba20198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 -- 172.21.15.122:0/3637457887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2019d360 con 0x7fba20107a80 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba24943640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 0x7fba201988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25144640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba25945640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fba180348f0 tx=0x7fba18032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba167fc640 1 -- 172.21.15.122:0/3637457887 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18041020 con 0x7fba20107a80 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2019d5f0 con 0x7fba20107a80 2024-04-02T13:36:22.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba167fc640 1 -- 172.21.15.122:0/3637457887 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18032e80 con 0x7fba20107a80 2024-04-02T13:36:22.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba167fc640 1 -- 172.21.15.122:0/3637457887 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18039040 con 0x7fba20107a80 2024-04-02T13:36:22.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.052+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba201a4ee0 con 0x7fba20107a80 2024-04-02T13:36:22.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.055+0000 7fba29f8c740 1 Processor -- start 2024-04-02T13:36:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba167fc640 1 -- 172.21.15.122:0/3637457887 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fba18044020 con 0x7fba20107a80 2024-04-02T13:36:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 -- start start 2024-04-02T13:36:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44ba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fe440a7b0 con 0x563fe44b83d0 2024-04-02T13:36:22.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fe43a9570 con 0x563fe44a6bc0 2024-04-02T13:36:22.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.056+0000 7fba29f8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fe4406760 con 0x563fe44a6210 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba24943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44ba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba167fc640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 0x7fba0c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37010/0 (socket says 172.21.15.122:37010) 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 -- 172.21.15.122:0/198781220 learned_addr learned my addr 172.21.15.122:0/198781220 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba167fc640 1 -- 172.21.15.122:0/3637457887 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fba180f2450 con 0x7fba20107a80 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25945640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 0x7fba0c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 -- 172.21.15.122:0/198781220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 msgr2=0x563fe44a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 -- 172.21.15.122:0/198781220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 msgr2=0x563fe44ba7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25945640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 0x7fba0c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fba180ef800 tx=0x7fba1802c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44ba7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba25144640 1 -- 172.21.15.122:0/198781220 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe411e3f0 con 0x563fe44a6bc0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.057+0000 7fba24943640 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44ba7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba25144640 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009ca0 tx=0x7fba1c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=febc3002eefb414c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba0b7fe640 1 -- 172.21.15.122:0/198781220 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012450 con 0x563fe44a6bc0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba0b7fe640 1 -- 172.21.15.122:0/198781220 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c012ad0 con 0x563fe44a6bc0 2024-04-02T13:36:22.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba0b7fe640 1 -- 172.21.15.122:0/198781220 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c00fc30 con 0x563fe44a6bc0 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba29f8c740 1 -- 172.21.15.122:0/198781220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 msgr2=0x563fe44a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.058+0000 7fba29f8c740 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009ca0 tx=0x7fba1c0117b0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 -- 172.21.15.122:0/198781220 shutdown_connections 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44ba7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44a7040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 --2- 172.21.15.122:0/198781220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 -- 172.21.15.122:0/198781220 >> 172.21.15.122:0/198781220 conn(0x563fe44a0ed0 msgr2=0x563fe44a12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 -- 172.21.15.122:0/198781220 shutdown_connections 2024-04-02T13:36:22.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 -- 172.21.15.122:0/198781220 wait complete. 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 -- start start 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.059+0000 7fba29f8c740 1 Processor -- start 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fe44abeb0 con 0x563fe44b83d0 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fe43c5e80 con 0x563fe44a6bc0 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fe44ac030 con 0x563fe44a6210 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba29f8c740 10 client.0 _ll_register_callbacks cb 0x563fe4401aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59740/0 (socket says 172.21.15.122:59740) 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 -- 172.21.15.122:0/3508806836 learned_addr learned my addr 172.21.15.122:0/3508806836 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba25144640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 -- 172.21.15.122:0/3508806836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 msgr2=0x563fe44b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 -- 172.21.15.122:0/3508806836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 msgr2=0x563fe44b21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44b21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 -- 172.21.15.122:0/3508806836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe41e87d0 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba24943640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fba10012e00 tx=0x7fba10005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4142b2867ea3b7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.060+0000 7fba08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10005030 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fba100051d0 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1c012ad0 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba10029050 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba29f8c740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fe44cfb70 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1c012ad0 con 0x563fe44b83d0 2024-04-02T13:36:22.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.061+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba1001e020 con 0x563fe44b83d0 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 10 client.16104 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 20 client.16104 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe44d1030 0x563fe44d3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 4 client.16104 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fe4417cf0 con 0x563fe44d1030 2024-04-02T13:36:22.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba25144640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe44d1030 0x563fe44d3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba08ff9640 10 client.16104 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.062+0000 7fba25144640 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe44d1030 0x563fe44d3420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24589a44d88e4de1 server_cookie=d7096fd17cf5f41f in_seq=0 out_seq=0 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.063+0000 7fba08ff9640 1 -- 172.21.15.122:0/3508806836 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fe4417cf0 con 0x563fe44d1030 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.063+0000 7fba08ff9640 10 client.16104 handle_command_reply: tid=0 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.063+0000 7fba29f8c740 1 client.16104 shutdown 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 msgr2=0x563fe44b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fba10012e00 tx=0x7fba10005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 shutdown_connections 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe44d1030 0x563fe44d3420 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44b79a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba08ff9640 0 client.16104 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44b21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3508806836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 >> 172.21.15.122:0/3508806836 conn(0x563fe44a0ed0 msgr2=0x563fe44a2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 shutdown_connections 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 1 -- 172.21.15.122:0/3508806836 wait complete. 2024-04-02T13:36:22.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.064+0000 7fba29f8c740 20 client.16104 trim_cache size 0 max 16384 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.090+0000 7fba29f8c740 1 Processor -- start 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.090+0000 7fba29f8c740 1 -- start start 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.090+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.090+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba29f8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba201acdb0 con 0x563fe44b83d0 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba29f8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba201ae8f0 con 0x563fe44a6bc0 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba29f8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba201ad480 con 0x563fe44a6210 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59750/0 (socket says 172.21.15.122:59750) 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 -- 172.21.15.122:0/1649728518 learned_addr learned my addr 172.21.15.122:0/1649728518 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba25144640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 -- 172.21.15.122:0/1649728518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 msgr2=0x563fe44b78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 -- 172.21.15.122:0/1649728518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 msgr2=0x563fe44d8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44d8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 -- 172.21.15.122:0/1649728518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fe44a5890 con 0x563fe44b83d0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba25144640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44d8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba24943640 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fba100097f0 tx=0x7fba100125a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522215e2af0e95d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.091+0000 7fba0affd640 1 -- 172.21.15.122:0/1649728518 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10034070 con 0x563fe44b83d0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba0affd640 1 -- 172.21.15.122:0/1649728518 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba10012c30 con 0x563fe44b83d0 2024-04-02T13:36:22.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba0affd640 1 -- 172.21.15.122:0/1649728518 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba10020d30 con 0x563fe44b83d0 2024-04-02T13:36:22.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 -- 172.21.15.122:0/1649728518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 msgr2=0x563fe44d91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fba100097f0 tx=0x7fba100125a0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 -- 172.21.15.122:0/1649728518 shutdown_connections 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44b83d0 0x563fe44d91c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44d8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 --2- 172.21.15.122:0/1649728518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44b78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 -- 172.21.15.122:0/1649728518 >> 172.21.15.122:0/1649728518 conn(0x563fe44a0ed0 msgr2=0x563fe44ba300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 -- 172.21.15.122:0/1649728518 shutdown_connections 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.092+0000 7fba29f8c740 1 -- 172.21.15.122:0/1649728518 wait complete. 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 -- start start 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 Processor -- start 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44e44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44d1030 0x563fe44dfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba201ae220 con 0x563fe44d1030 2024-04-02T13:36:22.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba201adb50 con 0x563fe44a6bc0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba201aad90 con 0x563fe44a6210 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba29f8c740 10 client.0 _ll_register_callbacks cb 0x7fba10009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba25144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba24943640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44e44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba25144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37038/0 (socket says 172.21.15.122:37038) 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba24943640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44e44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52350/0 (socket says 172.21.15.122:52350) 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba25144640 1 -- 172.21.15.122:0/3863860304 learned_addr learned my addr 172.21.15.122:0/3863860304 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.093+0000 7fba24943640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44d1030 0x563fe44dfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 -- 172.21.15.122:0/3863860304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 msgr2=0x563fe44e44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44e44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 -- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44d1030 msgr2=0x563fe44dfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44d1030 0x563fe44dfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 -- 172.21.15.122:0/3863860304 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fe44e0410 con 0x563fe44a6bc0 2024-04-02T13:36:22.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba25144640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00f780 tx=0x7fba1c009750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a92c37f2876bfe51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba08ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c005450 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba10012c30 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fba1c0055f0 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fba1c03c050 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba29f8c740 10 client.25607 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.094+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fe4536ea0 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba10012c30 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fba1c02b5c0 con 0x563fe44a6bc0 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 10 client.25607 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 10 client.25607 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 20 client.25607 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe4537280 0x563fe4539670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 4 client.25607 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:22.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fe4412040 con 0x563fe4537280 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba24943640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe4537280 0x563fe4539670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba08ff9640 10 client.25607 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.095+0000 7fba24943640 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe4537280 0x563fe4539670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a1200bfff9acd8 server_cookie=62f5109082526905 in_seq=0 out_seq=0 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba08ff9640 1 -- 172.21.15.122:0/3863860304 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fe4412040 con 0x563fe4537280 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba08ff9640 10 client.25607 handle_command_reply: tid=0 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 client.25607 shutdown 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 msgr2=0x563fe44df830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00f780 tx=0x7fba1c009750 comp rx=0 tx=0).stop 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 shutdown_connections 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563fe4537280 0x563fe4539670 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba08ff9640 0 client.25607 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fe44d1030 0x563fe44dfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fe44a6bc0 0x563fe44df830 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 --2- 172.21.15.122:0/3863860304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fe44a6210 0x563fe44e44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 >> 172.21.15.122:0/3863860304 conn(0x563fe44a0ed0 msgr2=0x563fe449ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 shutdown_connections 2024-04-02T13:36:22.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 1 -- 172.21.15.122:0/3863860304 wait complete. 2024-04-02T13:36:22.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.096+0000 7fba29f8c740 20 client.25607 trim_cache size 0 max 16384 2024-04-02T13:36:22.105 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 msgr2=0x7fba0c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 0x7fba0c0a6300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fba180ef800 tx=0x7fba1802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 msgr2=0x7fba2019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fba180348f0 tx=0x7fba18032c90 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 shutdown_connections 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba0c0a3e40 0x7fba0c0a6300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba20107a80 0x7fba2019cc30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba200696a0 0x7fba201988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 --2- 172.21.15.122:0/3637457887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba20068ce0 0x7fba20198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 >> 172.21.15.122:0/3637457887 conn(0x7fba20077b00 msgr2=0x7fba200fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 shutdown_connections 2024-04-02T13:36:22.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.097+0000 7fba273a8640 1 -- 172.21.15.122:0/3637457887 wait complete. 2024-04-02T13:36:22.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:36:22.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 -- 172.21.15.122:0/617948872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4408068d00 msgr2=0x7f4408069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 --2- 172.21.15.122:0/617948872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4408068d00 0x7f4408069180 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f43f80098c0 tx=0x7f43f802bb50 comp rx=0 tx=0).stop 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 -- 172.21.15.122:0/617948872 shutdown_connections 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 --2- 172.21.15.122:0/617948872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44080696c0 0x7f440810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 --2- 172.21.15.122:0/617948872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4408068d00 0x7f4408069180 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 --2- 172.21.15.122:0/617948872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f4408107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 -- 172.21.15.122:0/617948872 >> 172.21.15.122:0/617948872 conn(0x7f4408077b00 msgr2=0x7f4408077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 -- 172.21.15.122:0/617948872 shutdown_connections 2024-04-02T13:36:22.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.215+0000 7f440f622640 1 -- 172.21.15.122:0/617948872 wait complete. 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 Processor -- start 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 -- start start 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4408068d00 0x7f4408198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44080696c0 0x7f4408198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4408111e70 con 0x7f4408107b50 2024-04-02T13:36:22.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4408111cf0 con 0x7f44080696c0 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440f622640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4408111ff0 con 0x7f4408068d00 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440dbbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440dbbf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59776/0 (socket says 172.21.15.122:59776) 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440dbbf640 1 -- 172.21.15.122:0/2086418085 learned_addr learned my addr 172.21.15.122:0/2086418085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440d3be640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4408068d00 0x7f4408198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440cbbd640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44080696c0 0x7f4408198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440dbbf640 1 -- 172.21.15.122:0/2086418085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4408068d00 msgr2=0x7f4408198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.216+0000 7f440dbbf640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4408068d00 0x7f4408198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440dbbf640 1 -- 172.21.15.122:0/2086418085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44080696c0 msgr2=0x7f4408198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440dbbf640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44080696c0 0x7f4408198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440dbbf640 1 -- 172.21.15.122:0/2086418085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440819d400 con 0x7f4408107b50 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440dbbf640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f440000eaa0 tx=0x7f440000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f43fe7fc640 1 -- 172.21.15.122:0/2086418085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440000cdb0 con 0x7f4408107b50 2024-04-02T13:36:22.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f43fe7fc640 1 -- 172.21.15.122:0/2086418085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4400005020 con 0x7f4408107b50 2024-04-02T13:36:22.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440819d6f0 con 0x7f4408107b50 2024-04-02T13:36:22.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f43fe7fc640 1 -- 172.21.15.122:0/2086418085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44000116b0 con 0x7f4408107b50 2024-04-02T13:36:22.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.217+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44081008f0 con 0x7f4408107b50 2024-04-02T13:36:22.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 Processor -- start 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f43fe7fc640 1 -- 172.21.15.122:0/2086418085 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f4400005560 con 0x7f4408107b50 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 -- start start 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc44b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 0x5615bdc50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5615bdba87b0 con 0x5615bdc4dca0 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5615bdb47570 con 0x5615bdc447a0 2024-04-02T13:36:22.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.220+0000 7f4412207740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615bdba4760 con 0x5615bdc450c0 2024-04-02T13:36:22.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440cbbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 0x5615bdc50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f43fe7fc640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 0x7f43f40a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52382/0 (socket says 172.21.15.122:52382) 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 -- 172.21.15.122:0/1161708322 learned_addr learned my addr 172.21.15.122:0/1161708322 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440cbbd640 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc44b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440dbbf640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 0x7f43f40a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f43fe7fc640 1 -- 172.21.15.122:0/2086418085 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f44000d0f60 con 0x7f4408107b50 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 -- 172.21.15.122:0/1161708322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 msgr2=0x5615bdc44b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc44b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 -- 172.21.15.122:0/1161708322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 msgr2=0x5615bdc50090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 0x5615bdc50090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440d3be640 1 -- 172.21.15.122:0/1161708322 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bd8bc3f0 con 0x5615bdc450c0 2024-04-02T13:36:22.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.221+0000 7f440cbbd640 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 0x5615bdc50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:22.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.222+0000 7f440dbbf640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 0x7f43f40a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44000ce200 tx=0x7f440001eb40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.222+0000 7f440d3be640 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4404009c60 tx=0x7f4404011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a1a3c4c5871e639 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.222+0000 7f43f37fe640 1 -- 172.21.15.122:0/1161708322 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404005220 con 0x5615bdc450c0 2024-04-02T13:36:22.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.222+0000 7f43f37fe640 1 -- 172.21.15.122:0/1161708322 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404009e10 con 0x5615bdc450c0 2024-04-02T13:36:22.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.222+0000 7f43f37fe640 1 -- 172.21.15.122:0/1161708322 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040197d0 con 0x5615bdc450c0 2024-04-02T13:36:22.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- 172.21.15.122:0/1161708322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 msgr2=0x5615bdc4d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4404009c60 tx=0x7f4404011810 comp rx=0 tx=0).stop 2024-04-02T13:36:22.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- 172.21.15.122:0/1161708322 shutdown_connections 2024-04-02T13:36:22.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc4dca0 0x5615bdc50090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4d760 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 --2- 172.21.15.122:0/1161708322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc44b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- 172.21.15.122:0/1161708322 >> 172.21.15.122:0/1161708322 conn(0x5615bdc3eed0 msgr2=0x5615bdc3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- 172.21.15.122:0/1161708322 shutdown_connections 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- 172.21.15.122:0/1161708322 wait complete. 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.223+0000 7f4412207740 1 -- start start 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 Processor -- start 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc4dca0 0x5615bdc4b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5615bdc50a90 con 0x5615bdc447a0 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5615bdb63e80 con 0x5615bdc4dca0 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615bdc50c10 con 0x5615bdc450c0 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f440cbbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f4412207740 10 client.0 _ll_register_callbacks cb 0x5615bdb9faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f440cbbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59788/0 (socket says 172.21.15.122:59788) 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f440cbbd640 1 -- 172.21.15.122:0/1400640707 learned_addr learned my addr 172.21.15.122:0/1400640707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.224+0000 7f440d3be640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 -- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 msgr2=0x5615bdc4aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 -- 172.21.15.122:0/1400640707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc4dca0 msgr2=0x5615bdc4b010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc4dca0 0x5615bdc4b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 -- 172.21.15.122:0/1400640707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bd9867d0 con 0x5615bdc447a0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f440cbbd640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f80057c0 tx=0x7f43f8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b17bd89faee845 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f43f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8046040 con 0x5615bdc447a0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f43f8004480 con 0x5615bdc447a0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.225+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4404009e10 con 0x5615bdc447a0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f43f8005020 con 0x5615bdc447a0 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f4412207740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615bdc5ca60 con 0x5615bdc447a0 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404009e10 con 0x5615bdc447a0 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f43f804a070 con 0x5615bdc447a0 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.226+0000 7f4412207740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f4412207740 10 client.16122 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f4412207740 20 client.16122 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdc5df70 0x5615bdc60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f4412207740 4 client.16122 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615bdbb5cf0 con 0x5615bdc5df70 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f440d3be640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdc5df70 0x5615bdc60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f43f0ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:22.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.227+0000 7f440d3be640 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdc5df70 0x5615bdc60380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbeaab62078c11fc server_cookie=db92b38f0a4d860d in_seq=0 out_seq=0 2024-04-02T13:36:22.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.228+0000 7f43f0ff9640 1 -- 172.21.15.122:0/1400640707 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615bdbb5cf0 con 0x5615bdc5df70 2024-04-02T13:36:22.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.228+0000 7f43f0ff9640 10 client.16122 handle_command_reply: tid=0 2024-04-02T13:36:22.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.228+0000 7f4412207740 1 client.16122 shutdown 2024-04-02T13:36:22.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.228+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 msgr2=0x5615bdc4d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.228+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f43f80057c0 tx=0x7f43f8004040 comp rx=0 tx=0).stop 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 shutdown_connections 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc4dca0 0x5615bdc4b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc450c0 0x5615bdc4aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdc5df70 0x5615bdc60380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 --2- 172.21.15.122:0/1400640707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc447a0 0x5615bdc4d670 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 >> 172.21.15.122:0/1400640707 conn(0x5615bdc3eed0 msgr2=0x5615bdc40c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f43f0ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 shutdown_connections 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 1 -- 172.21.15.122:0/1400640707 wait complete. 2024-04-02T13:36:22.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.229+0000 7f4412207740 20 client.16122 trim_cache size 0 max 16384 2024-04-02T13:36:22.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 Processor -- start 2024-04-02T13:36:22.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 -- start start 2024-04-02T13:36:22.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc59950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc4d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44081acee0 con 0x5615bdc450c0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44081aea20 con 0x5615bdc447a0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f4412207740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44081ad5b0 con 0x5615bdc4dca0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f440d3be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f440cbbd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc4d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f440d3be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59790/0 (socket says 172.21.15.122:59790) 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.255+0000 7f440d3be640 1 -- 172.21.15.122:0/865386254 learned_addr learned my addr 172.21.15.122:0/865386254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440cbbd640 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc59950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 -- 172.21.15.122:0/865386254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 msgr2=0x5615bdc4d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc4d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 -- 172.21.15.122:0/865386254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 msgr2=0x5615bdc59950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc59950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 -- 172.21.15.122:0/865386254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615bdc44ef0 con 0x5615bdc450c0 2024-04-02T13:36:22.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f440d3be640 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4404009530 tx=0x7f4404019430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e3c1d6ca0c561 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f43f2ffd640 1 -- 172.21.15.122:0/865386254 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440402bdc0 con 0x5615bdc450c0 2024-04-02T13:36:22.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f43f2ffd640 1 -- 172.21.15.122:0/865386254 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404005500 con 0x5615bdc450c0 2024-04-02T13:36:22.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.256+0000 7f43f2ffd640 1 -- 172.21.15.122:0/865386254 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040199c0 con 0x5615bdc450c0 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 -- 172.21.15.122:0/865386254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 msgr2=0x5615bdc4d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4404009530 tx=0x7f4404019430 comp rx=0 tx=0).stop 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 -- 172.21.15.122:0/865386254 shutdown_connections 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc4d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4d130 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 --2- 172.21.15.122:0/865386254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc59950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 -- 172.21.15.122:0/865386254 >> 172.21.15.122:0/865386254 conn(0x5615bdc3eed0 msgr2=0x5615bdc3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 -- 172.21.15.122:0/865386254 shutdown_connections 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.257+0000 7f4412207740 1 -- 172.21.15.122:0/865386254 wait complete. 2024-04-02T13:36:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.258+0000 7f4412207740 1 -- start start 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.258+0000 7f4412207740 1 Processor -- start 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.258+0000 7f4412207740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc4e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f44081ae350 con 0x5615bdc450c0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44081adc80 con 0x5615bdc447a0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44081aaec0 con 0x5615bdc4dca0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f4412207740 10 client.0 _ll_register_callbacks cb 0x7f43f80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59802/0 (socket says 172.21.15.122:59802) 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 -- 172.21.15.122:0/21489646 learned_addr learned my addr 172.21.15.122:0/21489646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440cbbd640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc4e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440cbbd640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 -- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 msgr2=0x5615bdc54e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc54e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 -- 172.21.15.122:0/21489646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 msgr2=0x5615bdc4e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc4e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 -- 172.21.15.122:0/21489646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615bdc553a0 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440cbbd640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc54e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f440d3be640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f44040099b0 tx=0x7f4404005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d2f91c5a939d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f43f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.259+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404038060 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4404005020 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4404005500 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f440403f050 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f4412207740 10 client.16128 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615bdc566a0 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404005500 con 0x5615bdc450c0 2024-04-02T13:36:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.260+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4404019880 con 0x5615bdc450c0 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 10 client.16128 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 10 client.16128 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 20 client.16128 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdcc4670 0x5615bdcc6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 4 client.16128 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615bdbb0040 con 0x5615bdcc4670 2024-04-02T13:36:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f440cbbd640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdcc4670 0x5615bdcc6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f440cbbd640 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdcc4670 0x5615bdcc6a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d656d7fae5a34be7 server_cookie=f4b6c35ac2cfd675 in_seq=0 out_seq=0 2024-04-02T13:36:22.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.261+0000 7f43f0ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:22.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f43f0ff9640 1 -- 172.21.15.122:0/21489646 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615bdbb0040 con 0x5615bdcc4670 2024-04-02T13:36:22.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f43f0ff9640 10 client.16128 handle_command_reply: tid=0 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 client.16128 shutdown 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 msgr2=0x5615bdc4e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f44040099b0 tx=0x7f4404005e80 comp rx=0 tx=0).stop 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 shutdown_connections 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615bdc4dca0 0x5615bdc54e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5615bdcc4670 0x5615bdcc6a90 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615bdc450c0 0x5615bdc4e9d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 --2- 172.21.15.122:0/21489646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615bdc447a0 0x5615bdc4e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 >> 172.21.15.122:0/21489646 conn(0x5615bdc3eed0 msgr2=0x5615bdc492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f43f0ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 shutdown_connections 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 1 -- 172.21.15.122:0/21489646 wait complete. 2024-04-02T13:36:22.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.262+0000 7f4412207740 20 client.16128 trim_cache size 0 max 16384 2024-04-02T13:36:22.270 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:22.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 msgr2=0x7f43f40a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 0x7f43f40a6380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44000ce200 tx=0x7f440001eb40 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 msgr2=0x7f440819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f440000eaa0 tx=0x7f440000ef60 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 shutdown_connections 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43f40a3ec0 0x7f43f40a6380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4408107b50 0x7f440819ccd0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f44080696c0 0x7f4408198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 --2- 172.21.15.122:0/2086418085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4408068d00 0x7f4408198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 >> 172.21.15.122:0/2086418085 conn(0x7f4408077b00 msgr2=0x7f44080fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 shutdown_connections 2024-04-02T13:36:22.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.263+0000 7f440f622640 1 -- 172.21.15.122:0/2086418085 wait complete. 2024-04-02T13:36:22.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:36:22.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 -- 172.21.15.122:0/2156103836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 msgr2=0x7fdfd810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 --2- 172.21.15.122:0/2156103836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd810e960 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009cd0 tx=0x7fdfd002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 -- 172.21.15.122:0/2156103836 shutdown_connections 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 --2- 172.21.15.122:0/2156103836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd810e960 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 --2- 172.21.15.122:0/2156103836 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8068d00 0x7fdfd8069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 --2- 172.21.15.122:0/2156103836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8107b50 0x7fdfd8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 -- 172.21.15.122:0/2156103836 >> 172.21.15.122:0/2156103836 conn(0x7fdfd8077b00 msgr2=0x7fdfd8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 -- 172.21.15.122:0/2156103836 shutdown_connections 2024-04-02T13:36:22.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.387+0000 7fdfdd047640 1 -- 172.21.15.122:0/2156103836 wait complete. 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 Processor -- start 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 -- start start 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8068d00 0x7fdfd81960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8107b50 0x7fdfd819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810fca0 con 0x7fdfd80696c0 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810fb20 con 0x7fdfd8107b50 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfdd047640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd810fe20 con 0x7fdfd8068d00 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59828/0 (socket says 172.21.15.122:59828) 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfd659c640 1 -- 172.21.15.122:0/853602911 learned_addr learned my addr 172.21.15.122:0/853602911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8068d00 0x7fdfd81960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.388+0000 7fdfd759e640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8107b50 0x7fdfd819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 -- 172.21.15.122:0/853602911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8068d00 msgr2=0x7fdfd81960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8068d00 0x7fdfd81960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 -- 172.21.15.122:0/853602911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8107b50 msgr2=0x7fdfd819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8107b50 0x7fdfd819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 -- 172.21.15.122:0/853602911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd819b0d0 con 0x7fdfd80696c0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfd659c640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009be0 tx=0x7fdfcc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfbffff640 1 -- 172.21.15.122:0/853602911 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00f460 con 0x7fdfd80696c0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd819b3c0 con 0x7fdfd80696c0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd81a2c50 con 0x7fdfd80696c0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfbffff640 1 -- 172.21.15.122:0/853602911 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc00e4a0 con 0x7fdfd80696c0 2024-04-02T13:36:22.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.389+0000 7fdfbffff640 1 -- 172.21.15.122:0/853602911 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc018610 con 0x7fdfd80696c0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 Processor -- start 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 -- start start 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c57090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557198baf7b0 con 0x557198c4c0c0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557198b4e570 con 0x557198c54ca0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.392+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557198bab760 con 0x557198c4b7a0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfbffff640 1 -- 172.21.15.122:0/853602911 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7fdfcc017070 con 0x7fdfd80696c0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59840/0 (socket says 172.21.15.122:59840) 2024-04-02T13:36:22.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd6d9d640 1 -- 172.21.15.122:0/2138267077 learned_addr learned my addr 172.21.15.122:0/2138267077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 -- 172.21.15.122:0/2138267077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 msgr2=0x557198c57090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd659c640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c57090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c57090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 -- 172.21.15.122:0/2138267077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 msgr2=0x557198c54760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 -- 172.21.15.122:0/2138267077 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571988c33f0 con 0x557198c4b7a0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfbffff640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 0x7fdfc00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.393+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c4bb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfd002c630 tx=0x7fdfd0009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd8e1cc7240afb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.394+0000 7fdfbffff640 1 -- 172.21.15.122:0/853602911 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdfcc0d0760 con 0x7fdfd80696c0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.394+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 0x7fdfc00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.394+0000 7fdfbd7fa640 1 -- 172.21.15.122:0/2138267077 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0053070 con 0x557198c4b7a0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.394+0000 7fdfbd7fa640 1 -- 172.21.15.122:0/2138267077 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd0003970 con 0x557198c4b7a0 2024-04-02T13:36:22.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.394+0000 7fdfbd7fa640 1 -- 172.21.15.122:0/2138267077 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0039040 con 0x557198c4b7a0 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 0x7fdfc00a6380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x557198c4b270 tx=0x7fdfc8013ba0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2138267077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 msgr2=0x557198c4bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c4bb80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfd002c630 tx=0x7fdfd0009900 comp rx=0 tx=0).stop 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2138267077 shutdown_connections 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c57090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2138267077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c4bb80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2138267077 >> 172.21.15.122:0/2138267077 conn(0x557198c45ed0 msgr2=0x557198c462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2138267077 shutdown_connections 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2138267077 wait complete. 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- start start 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 Processor -- start 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4b7a0 0x557198c546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c54ca0 0x557198c52050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557198c57a80 con 0x557198c4c0c0 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557198b6ae80 con 0x557198c4b7a0 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.395+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557198c57c00 con 0x557198c54ca0 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfdfc2b740 10 client.0 _ll_register_callbacks cb 0x557198ba6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59852/0 (socket says 172.21.15.122:59852) 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 -- 172.21.15.122:0/213811429 learned_addr learned my addr 172.21.15.122:0/213811429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4b7a0 0x557198c546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 -- 172.21.15.122:0/213811429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c54ca0 msgr2=0x557198c52050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c54ca0 0x557198c52050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 -- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4b7a0 msgr2=0x557198c546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4b7a0 0x557198c546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 -- 172.21.15.122:0/213811429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719898d7d0 con 0x557198c4c0c0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfd659c640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00ef00 tx=0x7fdfcc09bb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eae93d33424084 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0a4d90 con 0x557198c4c0c0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdfcc0d03b0 con 0x557198c4c0c0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.396+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd0003970 con 0x557198c4c0c0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.397+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdfcc0b0bc0 con 0x557198c4c0c0 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.397+0000 7fdfdfc2b740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.397+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557198c63ae0 con 0x557198c4c0c0 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.397+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfcc0b0d60 con 0x557198c4c0c0 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdfcc09b570 con 0x557198c4c0c0 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 20 client.16140 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198c65040 0x557198c67460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 4 client.16140 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdfd0003970 con 0x557198c65040 2024-04-02T13:36:22.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198c65040 0x557198c67460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198c65040 0x557198c67460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7693320f9960793 server_cookie=36d432586f1a929f in_seq=0 out_seq=0 2024-04-02T13:36:22.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.398+0000 7fdfbaff5640 10 client.16140 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:22.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfbaff5640 1 -- 172.21.15.122:0/213811429 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fdfd0003970 con 0x557198c65040 2024-04-02T13:36:22.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfbaff5640 10 client.16140 handle_command_reply: tid=0 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 client.16140 shutdown 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 msgr2=0x557198c51a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00ef00 tx=0x7fdfcc09bb80 comp rx=0 tx=0).stop 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 shutdown_connections 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c54ca0 0x557198c52050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c51a40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.399+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198c65040 0x557198c67460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/213811429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4b7a0 0x557198c546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfbaff5640 0 client.16140 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 >> 172.21.15.122:0/213811429 conn(0x557198c45ed0 msgr2=0x557198c47c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 shutdown_connections 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/213811429 wait complete. 2024-04-02T13:36:22.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.400+0000 7fdfdfc2b740 20 client.16140 trim_cache size 0 max 16384 2024-04-02T13:36:22.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 Processor -- start 2024-04-02T13:36:22.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 -- start start 2024-04-02T13:36:22.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc400fee0 con 0x557198c4c0c0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4011a20 con 0x557198c54ca0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfdfc2b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc40105b0 con 0x557198c4b7a0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfd759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52450/0 (socket says 172.21.15.122:52450) 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfd6d9d640 1 -- 172.21.15.122:0/2207196127 learned_addr learned my addr 172.21.15.122:0/2207196127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.414+0000 7fdfd659c640 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 -- 172.21.15.122:0/2207196127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 msgr2=0x557198c60970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 -- 172.21.15.122:0/2207196127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 msgr2=0x557198c54170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 -- 172.21.15.122:0/2207196127 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557198c523c0 con 0x557198c54ca0 2024-04-02T13:36:22.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfd759e640 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c546b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009bf0 tx=0x7fdfd0009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70856070c6e96f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfbcff9640 1 -- 172.21.15.122:0/2207196127 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd005c020 con 0x557198c54ca0 2024-04-02T13:36:22.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfbcff9640 1 -- 172.21.15.122:0/2207196127 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd0003970 con 0x557198c54ca0 2024-04-02T13:36:22.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.415+0000 7fdfbcff9640 1 -- 172.21.15.122:0/2207196127 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0053040 con 0x557198c54ca0 2024-04-02T13:36:22.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2207196127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 msgr2=0x557198c546b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c546b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0009bf0 tx=0x7fdfd0009360 comp rx=0 tx=0).stop 2024-04-02T13:36:22.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2207196127 shutdown_connections 2024-04-02T13:36:22.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c54ca0 0x557198c546b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c4c0c0 0x557198c54170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/2207196127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c60970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2207196127 >> 172.21.15.122:0/2207196127 conn(0x557198c45ed0 msgr2=0x557198c4cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2207196127 shutdown_connections 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.416+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/2207196127 wait complete. 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 -- start start 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 Processor -- start 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c5d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c5d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37140/0 (socket says 172.21.15.122:37140) 2024-04-02T13:36:22.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd6d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c5d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52462/0 (socket says 172.21.15.122:52462) 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59878/0 (socket says 172.21.15.122:59878) 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfd659c640 1 -- 172.21.15.122:0/3028655666 learned_addr learned my addr 172.21.15.122:0/3028655666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4011350 con 0x557198c54ca0 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4010c80 con 0x557198c4c0c0 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.417+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc400dec0 con 0x557198c4b7a0 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 -- 172.21.15.122:0/3028655666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 msgr2=0x557198c5d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c5d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 -- 172.21.15.122:0/3028655666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 msgr2=0x557198c55d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557198c56560 con 0x557198c4c0c0 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfdfc2b740 10 client.0 _ll_register_callbacks cb 0x7fdfcc0983c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd759e640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd0003970 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfd659c640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc09a3d0 tx=0x7fdfcc09b230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=513bcd5d15614dce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfbaff5640 10 client.25637 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfbaff5640 1 -- 172.21.15.122:0/3028655666 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc07e030 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfbaff5640 1 -- 172.21.15.122:0/3028655666 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdfcc0a0120 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfbaff5640 1 -- 172.21.15.122:0/3028655666 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdfcc0cd8e0 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfdfc2b740 10 client.25637 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.418+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=28}) -- 0x557198ccb530 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfbaff5640 1 -- 172.21.15.122:0/3028655666 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fdfcc0d03b0 con 0x557198c4c0c0 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 10 client.25637 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 10 client.25637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 20 client.25637 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198ccba90 0x557198ccdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 4 client.25637 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:22.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557198bbccf0 con 0x557198ccba90 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198ccba90 0x557198ccdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfbaff5640 10 client.25637 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.419+0000 7fdfd6d9d640 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198ccba90 0x557198ccdf10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5973e6a5aa890057 server_cookie=c31b2304c8c6c56b in_seq=0 out_seq=0 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfbaff5640 1 -- 172.21.15.122:0/3028655666 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557198bbccf0 con 0x557198ccba90 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfbaff5640 10 client.25637 handle_command_reply: tid=0 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 client.25637 shutdown 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 msgr2=0x557198c557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc09a3d0 tx=0x7fdfcc09b230 comp rx=0 tx=0).stop 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 shutdown_connections 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557198c54ca0 0x557198c55d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x557198ccba90 0x557198ccdf10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557198c4c0c0 0x557198c557b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfbaff5640 0 client.25637 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:22.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 --2- 172.21.15.122:0/3028655666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557198c4b7a0 0x557198c5d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 >> 172.21.15.122:0/3028655666 conn(0x557198c45ed0 msgr2=0x557198c502d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 shutdown_connections 2024-04-02T13:36:22.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 1 -- 172.21.15.122:0/3028655666 wait complete. 2024-04-02T13:36:22.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.420+0000 7fdfdfc2b740 20 client.25637 trim_cache size 0 max 16384 2024-04-02T13:36:22.429 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 msgr2=0x7fdfc00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 0x7fdfc00a6380 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x557198c4b270 tx=0x7fdfc8013ba0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 msgr2=0x7fdfd8196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009be0 tx=0x7fdfcc00b260 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 shutdown_connections 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfc00a3ec0 0x7fdfc00a6380 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfd8107b50 0x7fdfd819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfd80696c0 0x7fdfd8196620 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 --2- 172.21.15.122:0/853602911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd8068d00 0x7fdfd81960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 >> 172.21.15.122:0/853602911 conn(0x7fdfd8077b00 msgr2=0x7fdfd80fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 shutdown_connections 2024-04-02T13:36:22.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.421+0000 7fdfdd047640 1 -- 172.21.15.122:0/853602911 wait complete. 2024-04-02T13:36:22.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 -- 172.21.15.122:0/3261771329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 msgr2=0x7f546c110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 --2- 172.21.15.122:0/3261771329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c110740 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5464009ca0 tx=0x7f546402c180 comp rx=0 tx=0).stop 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 -- 172.21.15.122:0/3261771329 shutdown_connections 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 --2- 172.21.15.122:0/3261771329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c110740 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 --2- 172.21.15.122:0/3261771329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 --2- 172.21.15.122:0/3261771329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 0x7f546c0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 -- 172.21.15.122:0/3261771329 >> 172.21.15.122:0/3261771329 conn(0x7f546c0fd5a0 msgr2=0x7f546c0ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 -- 172.21.15.122:0/3261771329 shutdown_connections 2024-04-02T13:36:22.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.541+0000 7f547251c640 1 -- 172.21.15.122:0/3261771329 wait complete. 2024-04-02T13:36:22.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 Processor -- start 2024-04-02T13:36:22.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 -- start start 2024-04-02T13:36:22.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 0x7f546c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f546c114090 con 0x7f546c10e330 2024-04-02T13:36:22.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.542+0000 7f547251c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f546c113f10 con 0x7f546c073ed0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f547251c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f546c114210 con 0x7f546c0747f0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59898/0 (socket says 172.21.15.122:59898) 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f5470ab9640 1 -- 172.21.15.122:0/3000232690 learned_addr learned my addr 172.21.15.122:0/3000232690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546bfff640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 0x7f546c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 -- 172.21.15.122:0/3000232690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 msgr2=0x7f546c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 0x7f546c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 -- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 msgr2=0x7f546c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 -- 172.21.15.122:0/3000232690 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c19f5b0 con 0x7f546c0747f0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f5470ab9640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f546b7fe640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c19ab50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f60 tx=0x7f5460009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f54697fa640 1 -- 172.21.15.122:0/3000232690 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460022070 con 0x7f546c0747f0 2024-04-02T13:36:22.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c19f8a0 con 0x7f546c0747f0 2024-04-02T13:36:22.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f54697fa640 1 -- 172.21.15.122:0/3000232690 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54600035f0 con 0x7f546c0747f0 2024-04-02T13:36:22.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.543+0000 7f54697fa640 1 -- 172.21.15.122:0/3000232690 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546001b780 con 0x7f546c0747f0 2024-04-02T13:36:22.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.544+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c101980 con 0x7f546c0747f0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 Processor -- start 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 -- start start 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54697fa640 1 -- 172.21.15.122:0/3000232690 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f5460025070 con 0x7f546c0747f0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c96711d7b0 con 0x55c9671b97a0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c9670bc570 con 0x55c9671c2ca0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f54751e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c967119760 con 0x55c9671ba0c0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59912/0 (socket says 172.21.15.122:59912) 2024-04-02T13:36:22.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 -- 172.21.15.122:0/3958212966 learned_addr learned my addr 172.21.15.122:0/3958212966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f546b7fe640 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f546bfff640 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 -- 172.21.15.122:0/3958212966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 msgr2=0x55c9671c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 -- 172.21.15.122:0/3958212966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 msgr2=0x55c9671c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.547+0000 7f5470ab9640 1 -- 172.21.15.122:0/3958212966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c966e313f0 con 0x55c9671b97a0 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f54697fa640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 0x7f54540a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f5470ab9640 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5464037040 tx=0x7f5464009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd70874057bfe9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f54697fa640 1 -- 172.21.15.122:0/3000232690 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f54600d89f0 con 0x7f546c0747f0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f546bfff640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 0x7f54540a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f54527fc640 1 -- 172.21.15.122:0/3958212966 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464043070 con 0x55c9671b97a0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f54527fc640 1 -- 172.21.15.122:0/3958212966 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464003e50 con 0x55c9671b97a0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.548+0000 7f54527fc640 1 -- 172.21.15.122:0/3958212966 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464039040 con 0x55c9671b97a0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- 172.21.15.122:0/3958212966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 msgr2=0x55c9671b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5464037040 tx=0x7f5464009900 comp rx=0 tx=0).stop 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f546bfff640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 0x7f54540a6380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55c9671b9270 tx=0x7f545c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- 172.21.15.122:0/3958212966 shutdown_connections 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 --2- 172.21.15.122:0/3958212966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671b9b80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- 172.21.15.122:0/3958212966 >> 172.21.15.122:0/3958212966 conn(0x55c9671b3ed0 msgr2=0x55c9671b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- 172.21.15.122:0/3958212966 shutdown_connections 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- 172.21.15.122:0/3958212966 wait complete. 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 -- start start 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.549+0000 7f54751e5740 1 Processor -- start 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c9671c5a30 con 0x55c9671b97a0 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c9670d8e80 con 0x55c9671c2ca0 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c9671c5bb0 con 0x55c9671ba0c0 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f54751e5740 10 client.0 _ll_register_callbacks cb 0x55c967114aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59922/0 (socket says 172.21.15.122:59922) 2024-04-02T13:36:22.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 -- 172.21.15.122:0/385107303 learned_addr learned my addr 172.21.15.122:0/385107303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546b7fe640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f5470ab9640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 -- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 msgr2=0x55c9671c00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 -- 172.21.15.122:0/385107303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 msgr2=0x55c9671c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 -- 172.21.15.122:0/385107303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c966efb7d0 con 0x55c9671b97a0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f546bfff640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f545c0045e0 tx=0x7f545c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c11672944d3d601 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.550+0000 7f544b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c009e60 con 0x55c9671b97a0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f545c01e680 con 0x55c9671b97a0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5464003e50 con 0x55c9671b97a0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f545c01e880 con 0x55c9671b97a0 2024-04-02T13:36:22.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 10 client.16158 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c9671d1bc0 con 0x55c9671b97a0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5464003e50 con 0x55c9671b97a0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f545c027070 con 0x55c9671b97a0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 20 client.16158 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c9671d30d0 0x55c9671d5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 4 client.16158 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.551+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96712acf0 con 0x55c9671d30d0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.552+0000 7f546b7fe640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c9671d30d0 0x55c9671d5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.552+0000 7f544b7fe640 10 client.16158 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.552+0000 7f546b7fe640 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c9671d30d0 0x55c9671d5520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94598b93345136da server_cookie=91d95436deecde0c in_seq=0 out_seq=0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f544b7fe640 1 -- 172.21.15.122:0/385107303 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c96712acf0 con 0x55c9671d30d0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f544b7fe640 10 client.16158 handle_command_reply: tid=0 2024-04-02T13:36:22.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 client.16158 shutdown 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 msgr2=0x55c9671bfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f545c0045e0 tx=0x7f545c004730 comp rx=0 tx=0).stop 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 shutdown_connections 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c9671d30d0 0x55c9671d5520 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671c2ca0 0x55c9671c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 --2- 172.21.15.122:0/385107303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671b97a0 0x55c9671bfb60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 >> 172.21.15.122:0/385107303 conn(0x55c9671b3ed0 msgr2=0x55c9671b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f544b7fe640 0 client.16158 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 shutdown_connections 2024-04-02T13:36:22.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 1 -- 172.21.15.122:0/385107303 wait complete. 2024-04-02T13:36:22.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.553+0000 7f54751e5740 20 client.16158 trim_cache size 0 max 16384 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.568+0000 7f54751e5740 1 Processor -- start 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.568+0000 7f54751e5740 1 -- start start 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.568+0000 7f54751e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.568+0000 7f54751e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671c2ca0 0x55c9671ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.568+0000 7f54751e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671beb40 0x55c9671dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37176/0 (socket says 172.21.15.122:37176) 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 -- 172.21.15.122:0/2148744448 learned_addr learned my addr 172.21.15.122:0/2148744448 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1acdc0 con 0x55c9671beb40 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546b7fe640 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671c2ca0 0x55c9671ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f5470ab9640 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671beb40 0x55c9671dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1ae900 con 0x55c9671ba0c0 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1ad490 con 0x55c9671c2ca0 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 -- 172.21.15.122:0/2148744448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671c2ca0 msgr2=0x55c9671ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671c2ca0 0x55c9671ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 -- 172.21.15.122:0/2148744448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671beb40 msgr2=0x55c9671dadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671beb40 0x55c9671dadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 -- 172.21.15.122:0/2148744448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9671af7a0 con 0x55c9671ba0c0 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.569+0000 7f546bfff640 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f545c029f70 tx=0x7f545c004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a685cd789d3af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.570+0000 7f5451ffb640 1 -- 172.21.15.122:0/2148744448 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c009e60 con 0x55c9671ba0c0 2024-04-02T13:36:22.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.570+0000 7f5451ffb640 1 -- 172.21.15.122:0/2148744448 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c01e680 con 0x55c9671ba0c0 2024-04-02T13:36:22.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.570+0000 7f5451ffb640 1 -- 172.21.15.122:0/2148744448 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c029440 con 0x55c9671ba0c0 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.570+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 msgr2=0x55c9671c2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.570+0000 7f54751e5740 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f545c029f70 tx=0x7f545c004450 comp rx=0 tx=0).stop 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 shutdown_connections 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671beb40 0x55c9671dadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671c2ca0 0x55c9671ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- 172.21.15.122:0/2148744448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671ba0c0 0x55c9671c2290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 >> 172.21.15.122:0/2148744448 conn(0x55c9671b3ed0 msgr2=0x55c9671be430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 shutdown_connections 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- 172.21.15.122:0/2148744448 wait complete. 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- start start 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 Processor -- start 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671beb40 0x55c9671c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1ae230 con 0x55c9671c2ca0 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1adb60 con 0x55c9671beb40 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f546c1aada0 con 0x55c9671ba0c0 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.571+0000 7f54751e5740 10 client.0 _ll_register_callbacks cb 0x7f545c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59938/0 (socket says 172.21.15.122:59938) 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 -- 172.21.15.122:0/837036728 learned_addr learned my addr 172.21.15.122:0/837036728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f546bfff640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f546b7fe640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671beb40 0x55c9671c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 -- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 msgr2=0x55c9671c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 -- 172.21.15.122:0/837036728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671beb40 msgr2=0x55c9671c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671beb40 0x55c9671c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 -- 172.21.15.122:0/837036728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c967237b90 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f5470ab9640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5464034ee0 tx=0x7f5464034e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a4dbc968210438 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.572+0000 7f544b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464050060 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5464003b70 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f545c01e680 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5464039040 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f54751e5740 10 client.16164 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c967238e40 con 0x55c9671c2ca0 2024-04-02T13:36:22.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.573+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f545c01e680 con 0x55c9671c2ca0 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f54640040c0 con 0x55c9671c2ca0 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 10 client.16164 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 20 client.16164 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c967239180 0x55c96723b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 4 client.16164 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c967125040 con 0x55c967239180 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.574+0000 7f546bfff640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c967239180 0x55c96723b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f544b7fe640 10 client.16164 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f546bfff640 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c967239180 0x55c96723b570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b58a7ca9d7d5ce8c server_cookie=f9d57c657661b83e in_seq=0 out_seq=0 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f544b7fe640 1 -- 172.21.15.122:0/837036728 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c967125040 con 0x55c967239180 2024-04-02T13:36:22.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f544b7fe640 10 client.16164 handle_command_reply: tid=0 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 client.16164 shutdown 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 msgr2=0x55c967237650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5464034ee0 tx=0x7f5464034e70 comp rx=0 tx=0).stop 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 shutdown_connections 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c967239180 0x55c96723b570 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c9671c2ca0 0x55c967237650 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c9671beb40 0x55c9671c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f544b7fe640 0 client.16164 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 --2- 172.21.15.122:0/837036728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9671ba0c0 0x55c9671c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 >> 172.21.15.122:0/837036728 conn(0x55c9671b3ed0 msgr2=0x55c9671bf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 shutdown_connections 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 1 -- 172.21.15.122:0/837036728 wait complete. 2024-04-02T13:36:22.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.575+0000 7f54751e5740 20 client.16164 trim_cache size 0 max 16384 2024-04-02T13:36:22.584 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 msgr2=0x7f54540a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 0x7f54540a6380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x55c9671b9270 tx=0x7f545c005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 msgr2=0x7f546c19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c19ab50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5460009f60 tx=0x7f5460009f90 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 shutdown_connections 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f54540a3ec0 0x7f54540a6380 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f546c10e330 0x7f546c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f546c0747f0 0x7f546c19ab50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 --2- 172.21.15.122:0/3000232690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f546c073ed0 0x7f546c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 >> 172.21.15.122:0/3000232690 conn(0x7f546c0fd5a0 msgr2=0x7f546c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 shutdown_connections 2024-04-02T13:36:22.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.576+0000 7f547251c640 1 -- 172.21.15.122:0/3000232690 wait complete. 2024-04-02T13:36:22.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:36:22.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 -- 172.21.15.122:0/4216773968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 msgr2=0x7f93400690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 --2- 172.21.15.122:0/4216773968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f93400690c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f933c009900 tx=0x7f933c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 -- 172.21.15.122:0/4216773968 shutdown_connections 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 --2- 172.21.15.122:0/4216773968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 0x7f9340110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 --2- 172.21.15.122:0/4216773968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 0x7f934010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 --2- 172.21.15.122:0/4216773968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f93400690c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 -- 172.21.15.122:0/4216773968 >> 172.21.15.122:0/4216773968 conn(0x7f9340077b00 msgr2=0x7f9340077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 -- 172.21.15.122:0/4216773968 shutdown_connections 2024-04-02T13:36:22.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.699+0000 7f93472f2640 1 -- 172.21.15.122:0/4216773968 wait complete. 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 Processor -- start 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 -- start start 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 0x7f934019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f934508e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f934508e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59958/0 (socket says 172.21.15.122:59958) 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f934508e640 1 -- 172.21.15.122:0/4150473101 learned_addr learned my addr 172.21.15.122:0/4150473101 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 0x7f934019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114080 con 0x7f9340068ce0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9340113f00 con 0x7f934010e4a0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114200 con 0x7f9340069600 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.700+0000 7f934488d640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 0x7f934019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934588f640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 0x7f934019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 -- 172.21.15.122:0/4150473101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 msgr2=0x7f934019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 0x7f934019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 -- 172.21.15.122:0/4150473101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 msgr2=0x7f934019ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 0x7f934019ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934019f570 con 0x7f9340068ce0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f934508e640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f933c006010 tx=0x7f933c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f93367fc640 1 -- 172.21.15.122:0/4150473101 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c041070 con 0x7f9340068ce0 2024-04-02T13:36:22.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934019f800 con 0x7f9340068ce0 2024-04-02T13:36:22.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93401a70f0 con 0x7f9340068ce0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f93367fc640 1 -- 172.21.15.122:0/4150473101 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f933c004450 con 0x7f9340068ce0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.701+0000 7f93367fc640 1 -- 172.21.15.122:0/4150473101 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c04b060 con 0x7f9340068ce0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 Processor -- start 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 -- start start 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cc6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc53c2a7b0 con 0x55dc53cc70c0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc53bc9570 con 0x55dc53cc67a0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.704+0000 7f9349fbc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc53c26760 con 0x55dc53ccfca0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934588f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cc6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f93367fc640 1 -- 172.21.15.122:0/4150473101 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f933c044070 con 0x7f9340068ce0 2024-04-02T13:36:22.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52560/0 (socket says 172.21.15.122:52560) 2024-04-02T13:36:22.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 -- 172.21.15.122:0/3455118111 learned_addr learned my addr 172.21.15.122:0/3455118111 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934508e640 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 -- 172.21.15.122:0/3455118111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 msgr2=0x55dc53cc6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cc6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 -- 172.21.15.122:0/3455118111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 msgr2=0x55dc53ccf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 -- 172.21.15.122:0/3455118111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc5393e3f0 con 0x55dc53ccfca0 2024-04-02T13:36:22.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934508e640 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f934488d640 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f933000b740 tx=0x7f933000bc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddf7269ca236f48e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f93367fc640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 0x7f932c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.705+0000 7f932b7fe640 1 -- 172.21.15.122:0/3455118111 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330003dd0 con 0x55dc53ccfca0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.706+0000 7f932b7fe640 1 -- 172.21.15.122:0/3455118111 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9330015070 con 0x55dc53ccfca0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.706+0000 7f93367fc640 1 -- 172.21.15.122:0/4150473101 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f933c0f27e0 con 0x7f9340068ce0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.706+0000 7f934588f640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 0x7f932c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f932b7fe640 1 -- 172.21.15.122:0/3455118111 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330011440 con 0x55dc53ccfca0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f934588f640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 0x7f932c0a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f934006b030 tx=0x7f9338009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- 172.21.15.122:0/3455118111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 msgr2=0x55dc53cd2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f933000b740 tx=0x7f933000bc00 comp rx=0 tx=0).stop 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- 172.21.15.122:0/3455118111 shutdown_connections 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cd2090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3455118111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cc6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- 172.21.15.122:0/3455118111 >> 172.21.15.122:0/3455118111 conn(0x55dc53cc0ed0 msgr2=0x55dc53cc12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- 172.21.15.122:0/3455118111 shutdown_connections 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- 172.21.15.122:0/3455118111 wait complete. 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 -- start start 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.707+0000 7f9349fbc740 1 Processor -- start 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53ccf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 0x55dc53ccce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc53cd2b30 con 0x55dc53cc70c0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dc53be5e80 con 0x55dc53ccfca0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dc53cd2cb0 con 0x55dc53cc67a0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f9349fbc740 10 client.0 _ll_register_callbacks cb 0x55dc53c21aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934488d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53ccf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934488d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59980/0 (socket says 172.21.15.122:59980) 2024-04-02T13:36:22.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934488d640 1 -- 172.21.15.122:0/871764222 learned_addr learned my addr 172.21.15.122:0/871764222 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934588f640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 0x55dc53ccce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 -- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 msgr2=0x55dc53ccce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 0x55dc53ccce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 -- 172.21.15.122:0/871764222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 msgr2=0x55dc53ccc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934508e640 1 -- 172.21.15.122:0/871764222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc53a087d0 con 0x55dc53cc67a0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934488d640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.708+0000 7f934588f640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 0x55dc53ccce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f934508e640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53ccf5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f933c0efb30 tx=0x7f933c0bddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26a161283f5e387c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933c0be2b0 con 0x55dc53cc67a0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f933c0f23f0 con 0x55dc53cc67a0 2024-04-02T13:36:22.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9330015070 con 0x55dc53cc67a0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f933c0c2660 con 0x55dc53cc67a0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 10 client.25561 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc53cd2b30 con 0x55dc53cc67a0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9330015070 con 0x55dc53cc67a0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f933c0f1070 con 0x55dc53cc67a0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 10 client.25561 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.709+0000 7f9349fbc740 20 client.25561 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53cdfdf0 0x55dc53ce22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f9349fbc740 4 client.25561 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc53c37cf0 con 0x55dc53cdfdf0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f934488d640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53cdfdf0 0x55dc53ce22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f934488d640 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53cdfdf0 0x55dc53ce22b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3158bf70b0751c server_cookie=89b23d4d02e6780b in_seq=0 out_seq=0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.710+0000 7f9328ff9640 10 client.25561 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9328ff9640 1 -- 172.21.15.122:0/871764222 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc53c37cf0 con 0x55dc53cdfdf0 2024-04-02T13:36:22.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9328ff9640 10 client.25561 handle_command_reply: tid=0 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 client.25561 shutdown 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 msgr2=0x55dc53ccf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53ccf5b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f933c0efb30 tx=0x7f933c0bddf0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 shutdown_connections 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53cdfdf0 0x55dc53ce22b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53ccfca0 0x55dc53ccce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53ccc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 --2- 172.21.15.122:0/871764222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53ccf5b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 >> 172.21.15.122:0/871764222 conn(0x55dc53cc0ed0 msgr2=0x55dc53cc2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9328ff9640 0 client.25561 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 shutdown_connections 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 1 -- 172.21.15.122:0/871764222 wait complete. 2024-04-02T13:36:22.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.711+0000 7f9349fbc740 20 client.25561 trim_cache size 0 max 16384 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.725+0000 7f9349fbc740 1 Processor -- start 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.725+0000 7f9349fbc740 1 -- start start 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.725+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53cdbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.725+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.725+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f9349fbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f932400fe90 con 0x55dc53ccfca0 2024-04-02T13:36:22.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f9349fbc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93240119d0 con 0x55dc53cc70c0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f9349fbc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9324010560 con 0x55dc53cc67a0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59992/0 (socket says 172.21.15.122:59992) 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 -- 172.21.15.122:0/2810584812 learned_addr learned my addr 172.21.15.122:0/2810584812 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934488d640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934508e640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53cdbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 -- 172.21.15.122:0/2810584812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 msgr2=0x55dc53cdbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53cdbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 -- 172.21.15.122:0/2810584812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 msgr2=0x55dc53ccf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 -- 172.21.15.122:0/2810584812 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc53cc6170 con 0x55dc53ccfca0 2024-04-02T13:36:22.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934488d640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f934588f640 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f933801f570 tx=0x7f9338016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17efe2aeac57af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f932affd640 1 -- 172.21.15.122:0/2810584812 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933802d070 con 0x55dc53ccfca0 2024-04-02T13:36:22.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f932affd640 1 -- 172.21.15.122:0/2810584812 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f933801d9b0 con 0x55dc53ccfca0 2024-04-02T13:36:22.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.726+0000 7f932affd640 1 -- 172.21.15.122:0/2810584812 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338011a50 con 0x55dc53ccfca0 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.727+0000 7f9349fbc740 1 -- 172.21.15.122:0/2810584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 msgr2=0x55dc53ccf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.727+0000 7f9349fbc740 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f933801f570 tx=0x7f9338016000 comp rx=0 tx=0).stop 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 -- 172.21.15.122:0/2810584812 shutdown_connections 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53ccfca0 0x55dc53ccf5b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc70c0 0x55dc53ccf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 --2- 172.21.15.122:0/2810584812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53cc67a0 0x55dc53cdbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 -- 172.21.15.122:0/2810584812 >> 172.21.15.122:0/2810584812 conn(0x55dc53cc0ed0 msgr2=0x55dc53ccb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 -- 172.21.15.122:0/2810584812 shutdown_connections 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 -- 172.21.15.122:0/2810584812 wait complete. 2024-04-02T13:36:22.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 -- start start 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.728+0000 7f9349fbc740 1 Processor -- start 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cd7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9324011300 con 0x55dc53cc70c0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60004/0 (socket says 172.21.15.122:60004) 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 -- 172.21.15.122:0/3501740902 learned_addr learned my addr 172.21.15.122:0/3501740902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9324010c30 con 0x55dc53cc67a0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934588f640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934508e640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cd7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f932400de70 con 0x55dc53ccfca0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f9349fbc740 10 client.0 _ll_register_callbacks cb 0x7f933c0ba620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 -- 172.21.15.122:0/3501740902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 msgr2=0x55dc53cc9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cc9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 -- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 msgr2=0x55dc53cd7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cd7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934488d640 1 -- 172.21.15.122:0/3501740902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc53cca370 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.729+0000 7f934588f640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cc9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f934488d640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9330004620 tx=0x7f9330005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d38d82e693a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9330030060 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9330003bd0 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f933801d9b0 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9330037050 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9349fbc740 10 client.16179 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc53cd0bd0 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f933801d9b0 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.730+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9330011540 con 0x55dc53cc70c0 2024-04-02T13:36:22.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 10 client.16179 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 20 client.16179 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53d464a0 0x55dc53d48890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 4 client.16179 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc53c32040 con 0x55dc53d464a0 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.731+0000 7f934508e640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53d464a0 0x55dc53d48890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f934508e640 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53d464a0 0x55dc53d48890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=556e65b72ff7abc6 server_cookie=a5ded3f8f8e4684f in_seq=0 out_seq=0 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9328ff9640 10 client.16179 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9328ff9640 1 -- 172.21.15.122:0/3501740902 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc53c32040 con 0x55dc53d464a0 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9328ff9640 10 client.16179 handle_command_reply: tid=0 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 client.16179 shutdown 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 msgr2=0x55dc53cd84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9330004620 tx=0x7f9330005bf0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 shutdown_connections 2024-04-02T13:36:22.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dc53d464a0 0x55dc53d48890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc53ccfca0 0x55dc53cc9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc53cc70c0 0x55dc53cd84f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 --2- 172.21.15.122:0/3501740902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dc53cc67a0 0x55dc53cd7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 >> 172.21.15.122:0/3501740902 conn(0x55dc53cc0ed0 msgr2=0x55dc53cc2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9328ff9640 0 client.16179 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 shutdown_connections 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 1 -- 172.21.15.122:0/3501740902 wait complete. 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.732+0000 7f9349fbc740 20 client.16179 trim_cache size 0 max 16384 2024-04-02T13:36:22.741 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 msgr2=0x7f932c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 0x7f932c0a6380 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f934006b030 tx=0x7f9338009500 comp rx=0 tx=0).stop 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 msgr2=0x7f934019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f933c006010 tx=0x7f933c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 shutdown_connections 2024-04-02T13:36:22.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f932c0a3ec0 0x7f932c0a6380 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f934010e4a0 0x7f934019ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9340069600 0x7f934019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 --2- 172.21.15.122:0/4150473101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9340068ce0 0x7f934019a580 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.733+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 >> 172.21.15.122:0/4150473101 conn(0x7f9340077b00 msgr2=0x7f934010f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.734+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 shutdown_connections 2024-04-02T13:36:22.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:22.734+0000 7f93472f2640 1 -- 172.21.15.122:0/4150473101 wait complete. 2024-04-02T13:36:22.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:22 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3434741048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:22 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3434741048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:23.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:22 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3434741048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:23.751 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:36:23.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:23.752 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:36:23.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 -- 172.21.15.122:0/3386187559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 msgr2=0x7f86ac0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 --2- 172.21.15.122:0/3386187559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac0fffa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f869c009900 tx=0x7f869c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 -- 172.21.15.122:0/3386187559 shutdown_connections 2024-04-02T13:36:23.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 --2- 172.21.15.122:0/3386187559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 0x7f86ac110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 --2- 172.21.15.122:0/3386187559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 0x7f86ac1004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 --2- 172.21.15.122:0/3386187559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac0fffa0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 -- 172.21.15.122:0/3386187559 >> 172.21.15.122:0/3386187559 conn(0x7f86ac076f20 msgr2=0x7f86ac0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 -- 172.21.15.122:0/3386187559 shutdown_connections 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 -- 172.21.15.122:0/3386187559 wait complete. 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.859+0000 7f86b1967640 1 Processor -- start 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 -- start start 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 0x7f86ac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 0x7f86ac19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1140a0 con 0x7f86ac1029c0 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac113f20 con 0x7f86ac100b00 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86b1967640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac114220 con 0x7f86ac102da0 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60020/0 (socket says 172.21.15.122:60020) 2024-04-02T13:36:23.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 -- 172.21.15.122:0/3095217427 learned_addr learned my addr 172.21.15.122:0/3095217427 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aaffd640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 0x7f86ac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 0x7f86ac19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 -- 172.21.15.122:0/3095217427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 msgr2=0x7f86ac19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 0x7f86ac19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 -- 172.21.15.122:0/3095217427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 msgr2=0x7f86ac19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 0x7f86ac19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.860+0000 7f86aa7fc640 1 -- 172.21.15.122:0/3095217427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86ac19f5b0 con 0x7f86ac1029c0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d770 tx=0x7f86a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86b0965640 1 -- 172.21.15.122:0/3095217427 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0017070 con 0x7f86ac1029c0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86ac19f8a0 con 0x7f86ac1029c0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86b0965640 1 -- 172.21.15.122:0/3095217427 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a0005500 con 0x7f86ac1029c0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86b0965640 1 -- 172.21.15.122:0/3095217427 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0004e00 con 0x7f86ac1029c0 2024-04-02T13:36:23.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.861+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86ac1a7130 con 0x7f86ac1029c0 2024-04-02T13:36:23.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.863+0000 7f86b462e740 1 Processor -- start 2024-04-02T13:36:23.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.863+0000 7f86b462e740 1 -- start start 2024-04-02T13:36:23.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x5649721686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x564972173bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5649720cc770 con 0x564972168be0 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564972074960 con 0x5649721682c0 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b462e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649720c88c0 con 0x5649721717c0 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60032/0 (socket says 172.21.15.122:60032) 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 -- 172.21.15.122:0/3911680493 learned_addr learned my addr 172.21.15.122:0/3911680493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x5649721686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86b0965640 1 -- 172.21.15.122:0/3095217427 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f86a0004fa0 con 0x7f86ac1029c0 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 -- 172.21.15.122:0/3911680493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 msgr2=0x564972173bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x564972173bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 -- 172.21.15.122:0/3911680493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 msgr2=0x5649721686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x5649721686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86aaffd640 1 -- 172.21.15.122:0/3911680493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564971ea87e0 con 0x564972168be0 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.864+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x5649721686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f86aaffd640 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f869c000c00 tx=0x7f869c0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8c4b1d28ff9a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f8691ffb640 1 -- 172.21.15.122:0/3911680493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c045b40 con 0x564972168be0 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f8691ffb640 1 -- 172.21.15.122:0/3911680493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c045ce0 con 0x564972168be0 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f8691ffb640 1 -- 172.21.15.122:0/3911680493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c004920 con 0x564972168be0 2024-04-02T13:36:23.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f86b0965640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 0x7f86940a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 0x7f86940a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.865+0000 7f86b0965640 1 -- 172.21.15.122:0/3095217427 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f86a00d0850 con 0x7f86ac1029c0 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- 172.21.15.122:0/3911680493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 msgr2=0x564972171280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 0x7f86940a6380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86a4005fd0 tx=0x7f86a4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f869c000c00 tx=0x7f869c0095e0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- 172.21.15.122:0/3911680493 shutdown_connections 2024-04-02T13:36:23.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x564972173bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x564972171280 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 --2- 172.21.15.122:0/3911680493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x5649721686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- 172.21.15.122:0/3911680493 >> 172.21.15.122:0/3911680493 conn(0x5649721629f0 msgr2=0x564972162dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- 172.21.15.122:0/3911680493 shutdown_connections 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- 172.21.15.122:0/3911680493 wait complete. 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.866+0000 7f86b462e740 1 -- start start 2024-04-02T13:36:23.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 Processor -- start 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 0x56497216e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 0x56497216f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564972174550 con 0x564972168be0 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60044/0 (socket says 172.21.15.122:60044) 2024-04-02T13:36:23.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86aa7fc640 1 -- 172.21.15.122:0/413280747 learned_addr learned my addr 172.21.15.122:0/413280747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 0x56497216f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564972087d20 con 0x5649721717c0 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86aaffd640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 0x56497216e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.867+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649721746d0 con 0x5649721682c0 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 -- 172.21.15.122:0/413280747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 msgr2=0x56497216e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 0x56497216e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 -- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 msgr2=0x56497216f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 0x56497216f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564971f08f70 con 0x564972168be0 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86b462e740 10 client.0 _ll_register_callbacks cb 0x564972054a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aaffd640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 0x56497216e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 0x56497216f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:23.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d740 tx=0x7f86a00056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b4c5b3056e1683 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869c045ce0 con 0x564972168be0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f868affd640 10 client.16191 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f868affd640 1 -- 172.21.15.122:0/413280747 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0020040 con 0x564972168be0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f868affd640 1 -- 172.21.15.122:0/413280747 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f86a0098820 con 0x564972168be0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f868affd640 1 -- 172.21.15.122:0/413280747 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86a009c1b0 con 0x564972168be0 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86b462e740 10 client.16191 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:23.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.868+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=28}) -- 0x564971da0170 con 0x564972168be0 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f868affd640 1 -- 172.21.15.122:0/413280747 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86a00a0170 con 0x564972168be0 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 10 client.16191 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 20 client.16191 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x564972183d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 4 client.16191 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:23.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649720d9cb0 con 0x564972181960 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.869+0000 7f86aaffd640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x564972183d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.870+0000 7f86aaffd640 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x564972183d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60d2726fad9a1ea server_cookie=e47d4f2c2bc0233a in_seq=0 out_seq=0 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.870+0000 7f868affd640 10 client.16191 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.870+0000 7f868affd640 1 -- 172.21.15.122:0/413280747 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649720d9cb0 con 0x564972181960 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.870+0000 7f868affd640 10 client.16191 handle_command_reply: tid=0 2024-04-02T13:36:23.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.870+0000 7f86b462e740 1 client.16191 shutdown 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 msgr2=0x56497216ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d740 tx=0x7f86a00056b0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 shutdown_connections 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x564972183d50 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f868affd640 0 client.16191 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:23.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721717c0 0x56497216f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564972168be0 0x56497216ede0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 --2- 172.21.15.122:0/413280747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721682c0 0x56497216e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 >> 172.21.15.122:0/413280747 conn(0x5649721629f0 msgr2=0x564972164740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 shutdown_connections 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 1 -- 172.21.15.122:0/413280747 wait complete. 2024-04-02T13:36:23.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.871+0000 7f86b462e740 20 client.16191 trim_cache size 0 max 16384 2024-04-02T13:36:23.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 Processor -- start 2024-04-02T13:36:23.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 -- start start 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x56497217d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721717c0 0x564972189620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1acd60 con 0x5649721717c0 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1ae8a0 con 0x5649721682c0 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86b462e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1ad430 con 0x564972168be0 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52630/0 (socket says 172.21.15.122:52630) 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 -- 172.21.15.122:0/367369658 learned_addr learned my addr 172.21.15.122:0/367369658 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721717c0 0x564972189620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aaffd640 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x56497217d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 -- 172.21.15.122:0/367369658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 msgr2=0x56497217d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x56497217d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 -- 172.21.15.122:0/367369658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721717c0 msgr2=0x564972189620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721717c0 0x564972189620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.905+0000 7f86aa7fc640 1 -- 172.21.15.122:0/367369658 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649721840c0 con 0x564972168be0 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.906+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86a007afd0 tx=0x7f86a009e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e287f85f13b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.906+0000 7f86917fa640 1 -- 172.21.15.122:0/367369658 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0020070 con 0x564972168be0 2024-04-02T13:36:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.906+0000 7f86917fa640 1 -- 172.21.15.122:0/367369658 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a0015040 con 0x564972168be0 2024-04-02T13:36:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.906+0000 7f86917fa640 1 -- 172.21.15.122:0/367369658 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a009ad00 con 0x564972168be0 2024-04-02T13:36:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- 172.21.15.122:0/367369658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 msgr2=0x56497216f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f86a007afd0 tx=0x7f86a009e000 comp rx=0 tx=0).stop 2024-04-02T13:36:23.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- 172.21.15.122:0/367369658 shutdown_connections 2024-04-02T13:36:23.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721717c0 0x564972189620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564972168be0 0x56497216f140 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 --2- 172.21.15.122:0/367369658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649721682c0 0x56497217d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- 172.21.15.122:0/367369658 >> 172.21.15.122:0/367369658 conn(0x5649721629f0 msgr2=0x564972161ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- 172.21.15.122:0/367369658 shutdown_connections 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- 172.21.15.122:0/367369658 wait complete. 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 -- start start 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.907+0000 7f86b462e740 1 Processor -- start 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 0x564972175c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x5649721e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1ae1d0 con 0x5649721682c0 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1adb00 con 0x564972168be0 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86ac1aad40 con 0x5649721717c0 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86b462e740 10 client.0 _ll_register_callbacks cb 0x7f86a00cdac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60056/0 (socket says 172.21.15.122:60056) 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 -- 172.21.15.122:0/2666564698 learned_addr learned my addr 172.21.15.122:0/2666564698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:23.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86ab7fe640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x5649721e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 0x564972175c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 -- 172.21.15.122:0/2666564698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 msgr2=0x5649721e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x5649721e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 -- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 msgr2=0x564972175c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 0x564972175c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 -- 172.21.15.122:0/2666564698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564972168a10 con 0x5649721682c0 2024-04-02T13:36:23.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 0x564972175c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f86aaffd640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f869c02c210 tx=0x7f869c02c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cef70eba401309e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f868affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.908+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c05d020 con 0x5649721682c0 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f869c004240 con 0x5649721682c0 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a0015040 con 0x5649721682c0 2024-04-02T13:36:23.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f869c068050 con 0x5649721682c0 2024-04-02T13:36:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 10 client.16197 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5649721e8440 con 0x5649721682c0 2024-04-02T13:36:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869c004c20 con 0x5649721682c0 2024-04-02T13:36:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f869c004ee0 con 0x5649721682c0 2024-04-02T13:36:23.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 10 client.16197 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:23.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 10 client.16197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:23.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 20 client.16197 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:23.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x5649721ea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:23.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 4 client.16197 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f86a0015040 con 0x564972181960 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.909+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x5649721ea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.910+0000 7f86aa7fc640 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x5649721ea7e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1561610303049e6 server_cookie=f875be07924a95a9 in_seq=0 out_seq=0 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.910+0000 7f868affd640 10 client.16197 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f868affd640 1 -- 172.21.15.122:0/2666564698 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f86a0015040 con 0x564972181960 2024-04-02T13:36:23.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f868affd640 10 client.16197 handle_command_reply: tid=0 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 client.16197 shutdown 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 msgr2=0x564972175720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f869c02c210 tx=0x7f869c02c470 comp rx=0 tx=0).stop 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 shutdown_connections 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564972181960 0x5649721ea7e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649721717c0 0x5649721e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f868affd640 0 client.16197 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564972168be0 0x564972175c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 --2- 172.21.15.122:0/2666564698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649721682c0 0x564972175720 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 >> 172.21.15.122:0/2666564698 conn(0x5649721629f0 msgr2=0x564972163af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 shutdown_connections 2024-04-02T13:36:23.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.911+0000 7f86b462e740 1 -- 172.21.15.122:0/2666564698 wait complete. 2024-04-02T13:36:23.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b462e740 20 client.16197 trim_cache size 0 max 16384 2024-04-02T13:36:23.932 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:23.933 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:36:23.933 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "58bb0f3a-cbd7-4651-8600-975fd286b287", 2024-04-02T13:36:23.933 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:36:23.933 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 msgr2=0x7f86940a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 0x7f86940a6380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f86a4005fd0 tx=0x7f86a4009500 comp rx=0 tx=0).stop 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 msgr2=0x7f86ac19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f86a000d770 tx=0x7f86a000dc30 comp rx=0 tx=0).stop 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 shutdown_connections 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86940a3ec0 0x7f86940a6380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86ac102da0 0x7f86ac19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86ac1029c0 0x7f86ac19ab00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 --2- 172.21.15.122:0/3095217427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86ac100b00 0x7f86ac19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 >> 172.21.15.122:0/3095217427 conn(0x7f86ac076f20 msgr2=0x7f86ac0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 shutdown_connections 2024-04-02T13:36:23.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:23.912+0000 7f86b1967640 1 -- 172.21.15.122:0/3095217427 wait complete. 2024-04-02T13:36:23.937 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:23 smithi122 ceph-mon[32114]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:36:23.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:36:23.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:36:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:23 smithi187 ceph-mon[36064]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:36:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:23 smithi188 ceph-mon[36117]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 170 B/s wr, 12 op/s 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 -- 172.21.15.122:0/368678800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8107b50 msgr2=0x7f86b8107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 --2- 172.21.15.122:0/368678800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8107b50 0x7f86b8107f30 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009930 tx=0x7f86a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 -- 172.21.15.122:0/368678800 shutdown_connections 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 --2- 172.21.15.122:0/368678800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 0x7f86b810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 --2- 172.21.15.122:0/368678800 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b8068d00 0x7f86b8069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 --2- 172.21.15.122:0/368678800 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8107b50 0x7f86b8107f30 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 -- 172.21.15.122:0/368678800 >> 172.21.15.122:0/368678800 conn(0x7f86b8077b00 msgr2=0x7f86b8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 -- 172.21.15.122:0/368678800 shutdown_connections 2024-04-02T13:36:24.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.099+0000 7f86be1a8640 1 -- 172.21.15.122:0/368678800 wait complete. 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.100+0000 7f86be1a8640 1 Processor -- start 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.100+0000 7f86be1a8640 1 -- start start 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 0x7f86b81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b8107b50 0x7f86b819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60078/0 (socket says 172.21.15.122:60078) 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 -- 172.21.15.122:0/3613037317 learned_addr learned my addr 172.21.15.122:0/3613037317 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8111ef0 con 0x7f86b8068d00 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8111d70 con 0x7f86b8107b50 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8112070 con 0x7f86b80696c0 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 0x7f86b81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 -- 172.21.15.122:0/3613037317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 msgr2=0x7f86b81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 0x7f86b81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 -- 172.21.15.122:0/3613037317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b8107b50 msgr2=0x7f86b819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b8107b50 0x7f86b819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.101+0000 7f86b77fe640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b819d350 con 0x7f86b8068d00 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86b77fe640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009900 tx=0x7f86a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86b4ff9640 1 -- 172.21.15.122:0/3613037317 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8041070 con 0x7f86b8068d00 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86b4ff9640 1 -- 172.21.15.122:0/3613037317 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a802ccc0 con 0x7f86b8068d00 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b819d5e0 con 0x7f86b8068d00 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86b4ff9640 1 -- 172.21.15.122:0/3613037317 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8003cd0 con 0x7f86b8068d00 2024-04-02T13:36:24.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.102+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b81008f0 con 0x7f86b8068d00 2024-04-02T13:36:24.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.105+0000 7f86b4ff9640 1 -- 172.21.15.122:0/3613037317 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 24) ==== 138880+0+0 (secure 0 0 0) 0x7f86a8044070 con 0x7f86b8068d00 2024-04-02T13:36:24.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.105+0000 7f86c0d8c740 1 Processor -- start 2024-04-02T13:36:24.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 -- start start 2024-04-02T13:36:24.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6988d0 0x563e3d698cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6a1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e3d5fc830 con 0x563e3d6a1dd0 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e3d5a4820 con 0x563e3d6988d0 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e3d5f87e0 con 0x563e3d6991f0 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60092/0 (socket says 172.21.15.122:60092) 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 -- 172.21.15.122:0/1023434907 learned_addr learned my addr 172.21.15.122:0/1023434907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:24.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b4ff9640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 0x7f86a00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 -- 172.21.15.122:0/1023434907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 msgr2=0x563e3d6a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 -- 172.21.15.122:0/1023434907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6988d0 msgr2=0x563e3d698cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6988d0 0x563e3d698cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.106+0000 7f86b6ffd640 1 -- 172.21.15.122:0/1023434907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3d3da7d0 con 0x563e3d6a1dd0 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f86b4ff9640 1 -- 172.21.15.122:0/3613037317 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f86a80f2bf0 con 0x7f86b8068d00 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bc80 tx=0x7f86ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67a3a4bcb2d8cee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f86b7fff640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 0x7f86a00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f869dffb640 1 -- 172.21.15.122:0/1023434907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac015070 con 0x563e3d6a1dd0 2024-04-02T13:36:24.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f869dffb640 1 -- 172.21.15.122:0/1023434907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac011920 con 0x563e3d6a1dd0 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.107+0000 7f869dffb640 1 -- 172.21.15.122:0/1023434907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac010930 con 0x563e3d6a1dd0 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1023434907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 msgr2=0x563e3d6a41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86b7fff640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 0x7f86a00a6380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009270 tx=0x7f86b00091e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00bc80 tx=0x7f86ac00f730 comp rx=0 tx=0).stop 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1023434907 shutdown_connections 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d6a41c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1023434907 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6988d0 0x563e3d698cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1023434907 >> 172.21.15.122:0/1023434907 conn(0x563e3d693000 msgr2=0x563e3d6933e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1023434907 shutdown_connections 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.108+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1023434907 wait complete. 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 -- start start 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 Processor -- start 2024-04-02T13:36:24.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e3d6ad3d0 con 0x563e3d6988d0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e3d5b7d30 con 0x563e3d6a1dd0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e3d6ad550 con 0x563e3d6991f0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86c0d8c740 10 client.0 _ll_register_callbacks cb 0x563e3d5b7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60108/0 (socket says 172.21.15.122:60108) 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37316/0 (socket says 172.21.15.122:37316) 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b77fe640 1 -- 172.21.15.122:0/1871214727 learned_addr learned my addr 172.21.15.122:0/1871214727 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.109+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52682/0 (socket says 172.21.15.122:52682) 2024-04-02T13:36:24.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 -- 172.21.15.122:0/1871214727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 msgr2=0x563e3d69ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 -- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 msgr2=0x563e3d69ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 -- 172.21.15.122:0/1871214727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3d3103f0 con 0x563e3d6988d0 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b7fff640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86b77fe640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f86a80be360 tx=0x7f86a8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab7566d0a3c3358 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f8696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a804b040 con 0x563e3d6988d0 2024-04-02T13:36:24.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f86a80bead0 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac011920 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86a80be1a0 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86c0d8c740 10 client.16209 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.110+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3d2d00f0 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac011920 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86a80dd420 con 0x563e3d6988d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 10 client.16209 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 20 client.16209 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d6ba5d0 0x563e3d6bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 4 client.16209 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e3d609d70 con 0x563e3d6ba5d0 2024-04-02T13:36:24.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.111+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d6ba5d0 0x563e3d6bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.112+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d6ba5d0 0x563e3d6bc9c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a307e66d2b3516ed server_cookie=ae86ab63dca4771e in_seq=0 out_seq=0 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.112+0000 7f8696ffd640 10 client.16209 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.112+0000 7f8696ffd640 1 -- 172.21.15.122:0/1871214727 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e3d609d70 con 0x563e3d6ba5d0 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.112+0000 7f8696ffd640 10 client.16209 handle_command_reply: tid=0 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.112+0000 7f86c0d8c740 1 client.16209 shutdown 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 msgr2=0x563e3d6a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f86a80be360 tx=0x7f86a8009560 comp rx=0 tx=0).stop 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 shutdown_connections 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d6ba5d0 0x563e3d6bc9c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1dd0 0x563e3d69ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d69ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f8696ffd640 0 client.16209 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/1871214727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6988d0 0x563e3d6a1680 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 >> 172.21.15.122:0/1871214727 conn(0x563e3d693000 msgr2=0x563e3d694d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 shutdown_connections 2024-04-02T13:36:24.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 1 -- 172.21.15.122:0/1871214727 wait complete. 2024-04-02T13:36:24.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.113+0000 7f86c0d8c740 20 client.16209 trim_cache size 0 max 16384 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 Processor -- start 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 -- start start 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 0x563e3d6a1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81acd00 con 0x563e3d6a1dd0 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ae840 con 0x563e3d6a1140 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ad3d0 con 0x563e3d6991f0 2024-04-02T13:36:24.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.147+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60116/0 (socket says 172.21.15.122:60116) 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 -- 172.21.15.122:0/3010103970 learned_addr learned my addr 172.21.15.122:0/3010103970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b77fe640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b7fff640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 0x563e3d6a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 -- 172.21.15.122:0/3010103970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 msgr2=0x563e3d6b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6b62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 -- 172.21.15.122:0/3010103970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 msgr2=0x563e3d6a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 0x563e3d6a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 -- 172.21.15.122:0/3010103970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e3d692e30 con 0x563e3d6a1dd0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b7fff640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 0x563e3d6a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f86ac010580 tx=0x7f86ac00b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0b7b241b841f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f869d7fa640 1 -- 172.21.15.122:0/3010103970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac015040 con 0x563e3d6a1dd0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f869d7fa640 1 -- 172.21.15.122:0/3010103970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac005500 con 0x563e3d6a1dd0 2024-04-02T13:36:24.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.148+0000 7f869d7fa640 1 -- 172.21.15.122:0/3010103970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0048e0 con 0x563e3d6a1dd0 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 -- 172.21.15.122:0/3010103970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 msgr2=0x563e3d69d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f86ac010580 tx=0x7f86ac00b5a0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 -- 172.21.15.122:0/3010103970 shutdown_connections 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6a1140 0x563e3d6a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1dd0 0x563e3d69d950 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/3010103970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6991f0 0x563e3d6b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.149+0000 7f86c0d8c740 1 -- 172.21.15.122:0/3010103970 >> 172.21.15.122:0/3010103970 conn(0x563e3d693000 msgr2=0x563e3d69a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 -- 172.21.15.122:0/3010103970 shutdown_connections 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 -- 172.21.15.122:0/3010103970 wait complete. 2024-04-02T13:36:24.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 -- start start 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 Processor -- start 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6991f0 0x563e3d6b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.150+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 0x563e3d69b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ae170 con 0x563e3d6a1140 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81adaa0 con 0x563e3d6991f0 2024-04-02T13:36:24.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81aace0 con 0x563e3d6a1dd0 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 10 client.0 _ll_register_callbacks cb 0x7f86a80044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60122/0 (socket says 172.21.15.122:60122) 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 -- 172.21.15.122:0/198922969 learned_addr learned my addr 172.21.15.122:0/198922969 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b77fe640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6991f0 0x563e3d6b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b7fff640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 0x563e3d69b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 -- 172.21.15.122:0/198922969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 msgr2=0x563e3d69b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 0x563e3d69b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 -- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6991f0 msgr2=0x563e3d6b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6991f0 0x563e3d6b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 -- 172.21.15.122:0/198922969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e3d69bff0 con 0x563e3d6a1140 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b7fff640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 0x563e3d69b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:36:24.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86b6ffd640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02c040 tx=0x7f86ac005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10c4ed0b9e6e0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f8696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac015070 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac005500 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.151+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f86ac004240 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f86ac03e050 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 10 client.16215 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563e3d7209f0 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac005500 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f86ac0038f0 con 0x563e3d6a1140 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 10 client.16215 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 20 client.16215 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d721030 0x563e3d723450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 4 client.16215 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:36:24.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563e3d6040c0 con 0x563e3d721030 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.152+0000 7f86b77fe640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d721030 0x563e3d723450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86b77fe640 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d721030 0x563e3d723450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed686e6010aecd68 server_cookie=f6889be52b474fdc in_seq=0 out_seq=0 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f8696ffd640 10 client.16215 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f8696ffd640 1 -- 172.21.15.122:0/198922969 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563e3d6040c0 con 0x563e3d721030 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f8696ffd640 10 client.16215 handle_command_reply: tid=0 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 client.16215 shutdown 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 msgr2=0x563e3d69b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f86ac02c040 tx=0x7f86ac005730 comp rx=0 tx=0).stop 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 shutdown_connections 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563e3d721030 0x563e3d723450 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e3d6a1dd0 0x563e3d69b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e3d6a1140 0x563e3d69b320 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f8696ffd640 0 client.16215 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 --2- 172.21.15.122:0/198922969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e3d6991f0 0x563e3d6b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 >> 172.21.15.122:0/198922969 conn(0x563e3d693000 msgr2=0x563e3d692240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 shutdown_connections 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 1 -- 172.21.15.122:0/198922969 wait complete. 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.153+0000 7f86c0d8c740 20 client.16215 trim_cache size 0 max 16384 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:36:24.163 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: "58bb0f3a-cbd7-4651-8600-975fd286b287": { 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "58bb0f3a-cbd7-4651-8600-975fd286b287", 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:36:24.164 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:24.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 msgr2=0x7f86a00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 0x7f86a00a6380 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009270 tx=0x7f86b00091e0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 msgr2=0x7f86b81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009900 tx=0x7f86a80042f0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 shutdown_connections 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86a00a3ec0 0x7f86a00a6380 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f86b8107b50 0x7f86b819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86b80696c0 0x7f86b81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 --2- 172.21.15.122:0/3613037317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86b8068d00 0x7f86b81983b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 >> 172.21.15.122:0/3613037317 conn(0x7f86b8077b00 msgr2=0x7f86b80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 shutdown_connections 2024-04-02T13:36:24.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:24.154+0000 7f86be1a8640 1 -- 172.21.15.122:0/3613037317 wait complete. 2024-04-02T13:36:24.171 INFO:tasks.cephfs.filesystem:scrub status for tag:58bb0f3a-cbd7-4651-8600-975fd286b287 - {'path': '/', 'tag': '58bb0f3a-cbd7-4651-8600-975fd286b287', 'options': 'recursive,force'} 2024-04-02T13:36:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:24 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:36:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:24 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:36:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:24 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:36:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:24 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:36:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:25.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:24 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:36:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:24 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:36:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:25 smithi187 ceph-mon[36064]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:26.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:25 smithi188 ceph-mon[36117]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:26.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:25 smithi122 ceph-mon[32114]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:26.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:26 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:36:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:26 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:36:27.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:26 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:36:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:27 smithi187 ceph-mon[36064]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:27 smithi188 ceph-mon[36117]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:27 smithi122 ceph-mon[32114]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:29 smithi187 ceph-mon[36064]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:29 smithi188 ceph-mon[36117]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:29 smithi122 ceph-mon[32114]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:31 smithi187 ceph-mon[36064]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:31 smithi188 ceph-mon[36117]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:31 smithi122 ceph-mon[32114]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:33 smithi187 ceph-mon[36064]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:33 smithi188 ceph-mon[36117]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:34.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:33 smithi122 ceph-mon[32114]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:35 smithi187 ceph-mon[36064]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:35 smithi188 ceph-mon[36117]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:35 smithi122 ceph-mon[32114]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-02T13:36:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:37 smithi187 ceph-mon[36064]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:37 smithi188 ceph-mon[36117]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:37 smithi122 ceph-mon[32114]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 12 op/s 2024-04-02T13:36:38.716 INFO:tasks.workunit.client.0.smithi122.stderr:Note: switching to '191d963ac9de7322da8aff20b567b3fa458e2ef6'. 2024-04-02T13:36:38.716 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.716 INFO:tasks.workunit.client.0.smithi122.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-02T13:36:38.716 INFO:tasks.workunit.client.0.smithi122.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr:state without impacting any branches by switching back to a branch. 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.717 INFO:tasks.workunit.client.0.smithi122.stderr: git switch -c 2024-04-02T13:36:38.718 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.718 INFO:tasks.workunit.client.0.smithi122.stderr:Or undo this operation with: 2024-04-02T13:36:38.718 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.718 INFO:tasks.workunit.client.0.smithi122.stderr: git switch - 2024-04-02T13:36:38.718 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.719 INFO:tasks.workunit.client.0.smithi122.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-02T13:36:38.719 INFO:tasks.workunit.client.0.smithi122.stderr: 2024-04-02T13:36:38.719 INFO:tasks.workunit.client.0.smithi122.stderr:HEAD is now at 191d963ac9d Revert "Merge pull request #55436 from tchaikov/mgr-python-3.12" 2024-04-02T13:36:38.721 DEBUG:teuthology.orchestra.run.smithi122:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-02T13:36:38.747 INFO:tasks.workunit.client.0.smithi122.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-02T13:36:38.749 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:36:38.749 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-02T13:36:39.558 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-02T13:36:39.622 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-02T13:36:39.665 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-02T13:36:39.666 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:36:39.666 INFO:tasks.workunit.client.0.smithi122.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-02T13:36:39.705 INFO:tasks.workunit.client.0.smithi122.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-02T13:36:39.708 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:36:39.708 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-02T13:36:39.764 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-02T13:36:39.765 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-02T13:36:39.765 DEBUG:teuthology.orchestra.run.smithi122:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-02T13:36:39.824 INFO:tasks.workunit.client.0.smithi122.stderr:+ dbench 1 2024-04-02T13:36:39.832 INFO:tasks.workunit.client.0.smithi122.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-02T13:36:39.832 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-04-02T13:36:39.832 INFO:tasks.workunit.client.0.smithi122.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-02T13:36:39.838 INFO:tasks.workunit.client.0.smithi122.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-02T13:36:39.938 INFO:tasks.workunit.client.0.smithi122.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-02T13:36:39.938 INFO:tasks.workunit.client.0.smithi122.stdout:releasing clients 2024-04-02T13:36:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:39 smithi187 ceph-mon[36064]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:39 smithi188 ceph-mon[36117]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:40.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:39 smithi122 ceph-mon[32114]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:40.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 3000 33.54 MB/sec warmup 1 sec latency 9.596 ms 2024-04-02T13:36:41.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 5385 22.73 MB/sec warmup 2 sec latency 15.424 ms 2024-04-02T13:36:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:41 smithi187 ceph-mon[36064]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:42.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:41 smithi188 ceph-mon[36117]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:41 smithi122 ceph-mon[32114]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-02T13:36:42.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 8045 20.00 MB/sec warmup 3 sec latency 11.348 ms 2024-04-02T13:36:43.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 11111 19.58 MB/sec warmup 4 sec latency 9.385 ms 2024-04-02T13:36:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:43 smithi187 ceph-mon[36064]: pgmap v49: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 129 KiB/s wr, 29 op/s 2024-04-02T13:36:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:43 smithi187 ceph-mon[36064]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-02T13:36:44.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:43 smithi188 ceph-mon[36117]: pgmap v49: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 129 KiB/s wr, 29 op/s 2024-04-02T13:36:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:43 smithi188 ceph-mon[36117]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-02T13:36:44.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:43 smithi122 ceph-mon[32114]: pgmap v49: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 129 KiB/s wr, 29 op/s 2024-04-02T13:36:44.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:43 smithi122 ceph-mon[32114]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-02T13:36:44.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 14286 19.20 MB/sec warmup 5 sec latency 10.799 ms 2024-04-02T13:36:45.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 16645 17.43 MB/sec warmup 6 sec latency 19.810 ms 2024-04-02T13:36:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:45 smithi187 ceph-mon[36064]: pgmap v50: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 25 op/s 2024-04-02T13:36:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:45 smithi188 ceph-mon[36117]: pgmap v50: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 25 op/s 2024-04-02T13:36:46.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:45 smithi122 ceph-mon[32114]: pgmap v50: 129 pgs: 129 active+clean; 2.6 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 25 op/s 2024-04-02T13:36:46.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 18907 16.94 MB/sec warmup 7 sec latency 21.902 ms 2024-04-02T13:36:47.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 21873 16.97 MB/sec warmup 8 sec latency 6.996 ms 2024-04-02T13:36:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:47 smithi187 ceph-mon[36064]: pgmap v51: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 819 KiB/s wr, 108 op/s 2024-04-02T13:36:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:47 smithi188 ceph-mon[36117]: pgmap v51: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 819 KiB/s wr, 108 op/s 2024-04-02T13:36:48.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:47 smithi122 ceph-mon[32114]: pgmap v51: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 819 KiB/s wr, 108 op/s 2024-04-02T13:36:48.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 25099 17.17 MB/sec warmup 9 sec latency 12.743 ms 2024-04-02T13:36:49.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 27470 16.29 MB/sec warmup 10 sec latency 11.938 ms 2024-04-02T13:36:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:49 smithi187 ceph-mon[36064]: pgmap v52: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:49 smithi188 ceph-mon[36117]: pgmap v52: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:50.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:49 smithi122 ceph-mon[32114]: pgmap v52: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:50.938 INFO:tasks.workunit.client.0.smithi122.stdout: 1 29802 16.27 MB/sec warmup 11 sec latency 17.984 ms 2024-04-02T13:36:51.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 32560 16.09 MB/sec warmup 12 sec latency 12.140 ms 2024-04-02T13:36:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:51 smithi187 ceph-mon[36064]: pgmap v53: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:51 smithi188 ceph-mon[36117]: pgmap v53: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:51 smithi122 ceph-mon[32114]: pgmap v53: 129 pgs: 129 active+clean; 9.2 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 819 KiB/s wr, 104 op/s 2024-04-02T13:36:52.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 35660 16.22 MB/sec warmup 13 sec latency 23.131 ms 2024-04-02T13:36:53.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 38115 15.73 MB/sec warmup 14 sec latency 13.862 ms 2024-04-02T13:36:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:53 smithi187 ceph-mon[36064]: pgmap v54: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-02T13:36:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:53 smithi188 ceph-mon[36117]: pgmap v54: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-02T13:36:54.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:53 smithi122 ceph-mon[32114]: pgmap v54: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-02T13:36:54.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:36:54.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 -- 172.21.15.122:0/786373519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81009b0 msgr2=0x7f38b8100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 --2- 172.21.15.122:0/786373519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81009b0 0x7f38b8100e30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009930 tx=0x7f38a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 -- 172.21.15.122:0/786373519 shutdown_connections 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 --2- 172.21.15.122:0/786373519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b8101370 0x7f38b810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 --2- 172.21.15.122:0/786373519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81009b0 0x7f38b8100e30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 --2- 172.21.15.122:0/786373519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b81069c0 0x7f38b8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 -- 172.21.15.122:0/786373519 >> 172.21.15.122:0/786373519 conn(0x7f38b8076d80 msgr2=0x7f38b8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 -- 172.21.15.122:0/786373519 shutdown_connections 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.278+0000 7f38bd167640 1 -- 172.21.15.122:0/786373519 wait complete. 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 Processor -- start 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 -- start start 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 0x7f38b8197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 0x7f38b8197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8110dd0 con 0x7f38b81069c0 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8110c50 con 0x7f38b81009b0 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38bd167640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38b8110f50 con 0x7f38b8101370 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38b759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 0x7f38b8197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38b759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35618/0 (socket says 172.21.15.122:35618) 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38b759e640 1 -- 172.21.15.122:0/2351736272 learned_addr learned my addr 172.21.15.122:0/2351736272 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.279+0000 7f38affff640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 0x7f38b8197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 -- 172.21.15.122:0/2351736272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 msgr2=0x7f38b8197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 0x7f38b8197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 -- 172.21.15.122:0/2351736272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 msgr2=0x7f38b8197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 0x7f38b8197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 -- 172.21.15.122:0/2351736272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b819c2d0 con 0x7f38b81069c0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38affff640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 0x7f38b8197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 0x7f38b8197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b759e640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009fd0 tx=0x7f38b000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b4d99640 1 -- 172.21.15.122:0/2351736272 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0017070 con 0x7f38b81069c0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b4d99640 1 -- 172.21.15.122:0/2351736272 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b0003e50 con 0x7f38b81069c0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b819c5c0 con 0x7f38b81069c0 2024-04-02T13:36:54.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38b4d99640 1 -- 172.21.15.122:0/2351736272 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00106c0 con 0x7f38b81069c0 2024-04-02T13:36:54.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.280+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b81a3e50 con 0x7f38b81069c0 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.282+0000 7f38b4d99640 1 -- 172.21.15.122:0/2351736272 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f38b0003970 con 0x7f38b81069c0 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 Processor -- start 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 -- start start 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae298d0 0x560bfae29cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38b4d99640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 0x7f38a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560bfad8d830 con 0x560bfae2a1f0 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560bfad35820 con 0x560bfae32dd0 2024-04-02T13:36:54.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bfad897e0 con 0x560bfae298d0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 0x7f38a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38b759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35624/0 (socket says 172.21.15.122:35624) 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38b759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48532/0 (socket says 172.21.15.122:48532) 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38affff640 1 -- 172.21.15.122:0/2936435933 learned_addr learned my addr 172.21.15.122:0/2936435933 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38affff640 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae298d0 0x560bfae29cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.283+0000 7f38b4d99640 1 -- 172.21.15.122:0/2351736272 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f38b00d0880 con 0x7f38b81069c0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 -- 172.21.15.122:0/2936435933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae298d0 msgr2=0x560bfae29cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae298d0 0x560bfae29cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 -- 172.21.15.122:0/2936435933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 msgr2=0x560bfae32890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 -- 172.21.15.122:0/2936435933 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfab6b7d0 con 0x560bfae32dd0 2024-04-02T13:36:54.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38affff640 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 0x7f38a40a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009f40 tx=0x7f38a8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38b759e640 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cdc20 tx=0x7f38b001ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64074889243ac173 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38ad7fa640 1 -- 172.21.15.122:0/2936435933 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0020040 con 0x560bfae32dd0 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38ad7fa640 1 -- 172.21.15.122:0/2936435933 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b009c520 con 0x560bfae32dd0 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.284+0000 7f38ad7fa640 1 -- 172.21.15.122:0/2936435933 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b009d7a0 con 0x560bfae32dd0 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2936435933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 msgr2=0x560bfae351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f38b00cdc20 tx=0x7f38b001ee30 comp rx=0 tx=0).stop 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2936435933 shutdown_connections 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae351c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae2a1f0 0x560bfae32890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2936435933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae298d0 0x560bfae29cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2936435933 >> 172.21.15.122:0/2936435933 conn(0x560bfae24000 msgr2=0x560bfae243e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2936435933 shutdown_connections 2024-04-02T13:36:54.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2936435933 wait complete. 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- start start 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 Processor -- start 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae2a1f0 0x560bfae3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae32dd0 0x560bfae31ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.285+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560bfae35b60 con 0x560bfae298d0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560bfad48d30 con 0x560bfae2a1f0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bfae35ce0 con 0x560bfae32dd0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 10 client.0 _ll_register_callbacks cb 0x560bfad48720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35632/0 (socket says 172.21.15.122:35632) 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 -- 172.21.15.122:0/509204271 learned_addr learned my addr 172.21.15.122:0/509204271 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38affff640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae2a1f0 0x560bfae3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b759e640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae32dd0 0x560bfae31ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 -- 172.21.15.122:0/509204271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae32dd0 msgr2=0x560bfae31ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae32dd0 0x560bfae31ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 -- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae2a1f0 msgr2=0x560bfae3af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae2a1f0 0x560bfae3af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 -- 172.21.15.122:0/509204271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfaaa13f0 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f38a800fce0 tx=0x7f38a8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7f0c663615dd3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f389e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a800eab0 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f38a800ec50 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b009c520 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38a800fe90 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 10 client.16227 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.286+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bfaa610f0 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b009c520 con 0x560bfae298d0 2024-04-02T13:36:54.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38a801c580 con 0x560bfae298d0 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 10 client.16227 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 10 client.16227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 20 client.16227 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfae44fd0 0x560bfae47420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 4 client.16227 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bfad9ad70 con 0x560bfae44fd0 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38affff640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfae44fd0 0x560bfae47420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f38affff640 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfae44fd0 0x560bfae47420 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbabd9232fcdc30d server_cookie=d1d3475644974a39 in_seq=0 out_seq=0 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.287+0000 7f389e7fc640 10 client.16227 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f389e7fc640 1 -- 172.21.15.122:0/509204271 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bfad9ad70 con 0x560bfae44fd0 2024-04-02T13:36:54.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f389e7fc640 10 client.16227 handle_command_reply: tid=0 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 client.16227 shutdown 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 msgr2=0x560bfae3aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f38a800fce0 tx=0x7f38a8004730 comp rx=0 tx=0).stop 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 shutdown_connections 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfae44fd0 0x560bfae47420 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae32dd0 0x560bfae31ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae2a1f0 0x560bfae3af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f389e7fc640 0 client.16227 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/509204271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae3aa40 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 >> 172.21.15.122:0/509204271 conn(0x560bfae24000 msgr2=0x560bfae25d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 shutdown_connections 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 1 -- 172.21.15.122:0/509204271 wait complete. 2024-04-02T13:36:54.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.289+0000 7f38bfe2d740 20 client.16227 trim_cache size 0 max 16384 2024-04-02T13:36:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.307+0000 7f38bfe2d740 1 Processor -- start 2024-04-02T13:36:54.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.307+0000 7f38bfe2d740 1 -- start start 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae2ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae2b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ace50 con 0x560bfae298d0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ae990 con 0x560bfae32dd0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ad520 con 0x560bfae2a1f0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35648/0 (socket says 172.21.15.122:35648) 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 -- 172.21.15.122:0/3690257560 learned_addr learned my addr 172.21.15.122:0/3690257560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38affff640 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae2ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b759e640 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae2b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 -- 172.21.15.122:0/3690257560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 msgr2=0x560bfae2ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae2ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 -- 172.21.15.122:0/3690257560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 msgr2=0x560bfae2b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae2b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 -- 172.21.15.122:0/3690257560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfae1fc50 con 0x560bfae298d0 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f38a80059f0 tx=0x7f38a800fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6eeee48652cb0d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38acff9640 1 -- 172.21.15.122:0/3690257560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a800e780 con 0x560bfae298d0 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38acff9640 1 -- 172.21.15.122:0/3690257560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a8039070 con 0x560bfae298d0 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.308+0000 7f38acff9640 1 -- 172.21.15.122:0/3690257560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a801d6e0 con 0x560bfae298d0 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- 172.21.15.122:0/3690257560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 msgr2=0x560bfae23240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f38a80059f0 tx=0x7f38a800fe90 comp rx=0 tx=0).stop 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- 172.21.15.122:0/3690257560 shutdown_connections 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae32dd0 0x560bfae2b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae2ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/3690257560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae298d0 0x560bfae23240 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- 172.21.15.122:0/3690257560 >> 172.21.15.122:0/3690257560 conn(0x560bfae24000 msgr2=0x560bfae2a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- 172.21.15.122:0/3690257560 shutdown_connections 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- 172.21.15.122:0/3690257560 wait complete. 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 -- start start 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.309+0000 7f38bfe2d740 1 Processor -- start 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae298d0 0x560bfae3f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae3f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81ae2c0 con 0x560bfae32dd0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81adbf0 con 0x560bfae298d0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38b81aae30 con 0x560bfae2a1f0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 10 client.0 _ll_register_callbacks cb 0x7f38a800f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae298d0 0x560bfae3f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35660/0 (socket says 172.21.15.122:35660) 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 -- 172.21.15.122:0/2315568511 learned_addr learned my addr 172.21.15.122:0/2315568511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38affff640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae3f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 -- 172.21.15.122:0/2315568511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 msgr2=0x560bfae3f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae3f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 -- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae298d0 msgr2=0x560bfae3f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae298d0 0x560bfae3f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 -- 172.21.15.122:0/2315568511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfaeaaab0 con 0x560bfae32dd0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38b759e640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005c40 tx=0x7f38b009c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86784379694d0319 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f389e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00c1060 con 0x560bfae32dd0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f38b00c1200 con 0x560bfae32dd0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38a8039070 con 0x560bfae32dd0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f38b0014070 con 0x560bfae32dd0 2024-04-02T13:36:54.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 10 client.16233 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.310+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560bfaeabd70 con 0x560bfae32dd0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38a8039070 con 0x560bfae32dd0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f38b00cdea0 con 0x560bfae32dd0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 10 client.16233 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 20 client.16233 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfaeac0b0 0x560bfaeae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 4 client.16233 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bfad950c0 con 0x560bfaeac0b0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfaeac0b0 0x560bfaeae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f389e7fc640 10 client.16233 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.311+0000 7f38b6d9d640 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfaeac0b0 0x560bfaeae4a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70a0bddecfcc0b4 server_cookie=80fd0378c1ebfa2f in_seq=0 out_seq=0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.312+0000 7f389e7fc640 1 -- 172.21.15.122:0/2315568511 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560bfad950c0 con 0x560bfaeac0b0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.312+0000 7f389e7fc640 10 client.16233 handle_command_reply: tid=0 2024-04-02T13:36:54.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.312+0000 7f38bfe2d740 1 client.16233 shutdown 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.312+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 msgr2=0x560bfaeaa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f38b0005c40 tx=0x7f38b009c120 comp rx=0 tx=0).stop 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 shutdown_connections 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560bfaeac0b0 0x560bfaeae4a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560bfae32dd0 0x560bfaeaa570 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f389e7fc640 0 client.16233 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bfae2a1f0 0x560bfae3f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 --2- 172.21.15.122:0/2315568511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560bfae298d0 0x560bfae3f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 >> 172.21.15.122:0/2315568511 conn(0x560bfae24000 msgr2=0x560bfae2ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 shutdown_connections 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 1 -- 172.21.15.122:0/2315568511 wait complete. 2024-04-02T13:36:54.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bfe2d740 20 client.16233 trim_cache size 0 max 16384 2024-04-02T13:36:54.323 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:54.323 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:36:54.323 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:36:54.323 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 msgr2=0x7f38a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 0x7f38a40a61e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009f40 tx=0x7f38a8005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 msgr2=0x7f38b819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009fd0 tx=0x7f38b000b260 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 shutdown_connections 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38a40a3d20 0x7f38a40a61e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b81069c0 0x7f38b819bba0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f38b8101370 0x7f38b8197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 --2- 172.21.15.122:0/2351736272 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f38b81009b0 0x7f38b8197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 >> 172.21.15.122:0/2351736272 conn(0x7f38b8076d80 msgr2=0x7f38b80fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 shutdown_connections 2024-04-02T13:36:54.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.313+0000 7f38bd167640 1 -- 172.21.15.122:0/2351736272 wait complete. 2024-04-02T13:36:54.327 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:36:54.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- 172.21.15.122:0/2439297560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590102450 msgr2=0x7f359010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 --2- 172.21.15.122:0/2439297560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590102450 0x7f359010e960 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3588009cd0 tx=0x7f358802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- 172.21.15.122:0/2439297560 shutdown_connections 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 --2- 172.21.15.122:0/2439297560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590102450 0x7f359010e960 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 --2- 172.21.15.122:0/2439297560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590101a90 0x7f3590101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 --2- 172.21.15.122:0/2439297560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590107aa0 0x7f3590107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- 172.21.15.122:0/2439297560 >> 172.21.15.122:0/2439297560 conn(0x7f35900fd5a0 msgr2=0x7f35900ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- 172.21.15.122:0/2439297560 shutdown_connections 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- 172.21.15.122:0/2439297560 wait complete. 2024-04-02T13:36:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 Processor -- start 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.430+0000 7f35958ba640 1 -- start start 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590101a90 0x7f3590198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590102450 0x7f35901988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3590111eb0 con 0x7f3590107aa0 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3590111d30 con 0x7f3590101a90 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3590112030 con 0x7f3590102450 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35682/0 (socket says 172.21.15.122:35682) 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 -- 172.21.15.122:0/3192325567 learned_addr learned my addr 172.21.15.122:0/3192325567 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358effd640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590101a90 0x7f3590198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358e7fc640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590102450 0x7f35901988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 -- 172.21.15.122:0/3192325567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590102450 msgr2=0x7f35901988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590102450 0x7f35901988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 -- 172.21.15.122:0/3192325567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590101a90 msgr2=0x7f3590198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590101a90 0x7f3590198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 -- 172.21.15.122:0/3192325567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359019d380 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f358f7fe640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35880348f0 tx=0x7f3588032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588041020 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3588032e80 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3588039040 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359019d610 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.431+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901a4f00 con 0x7f3590107aa0 2024-04-02T13:36:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.433+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590068d40 con 0x7f3590107aa0 2024-04-02T13:36:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.433+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3588044020 con 0x7f3590107aa0 2024-04-02T13:36:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.434+0000 7f35948b8640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 0x7f35780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.434+0000 7f358effd640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 0x7f35780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.434+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f35880f2630 con 0x7f3590107aa0 2024-04-02T13:36:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.434+0000 7f358effd640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 0x7f35780a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3584009b50 tx=0x7f3584005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.439+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f358804b020 con 0x7f3590107aa0 2024-04-02T13:36:54.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.681+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f359010a020 con 0x7f3590107aa0 2024-04-02T13:36:54.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.681+0000 7f35948b8640 1 -- 172.21.15.122:0/3192325567 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11942 (secure 0 0 0) 0x7f35880bea50 con 0x7f3590107aa0 2024-04-02T13:36:54.685 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:36:54.685 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:35:08.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:36:54.685 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 28 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 msgr2=0x7f35780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 0x7f35780a61e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3584009b50 tx=0x7f3584005f90 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 msgr2=0x7f359019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35880348f0 tx=0x7f3588032c90 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 shutdown_connections 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35780a3d20 0x7f35780a61e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3590107aa0 0x7f359019cc50 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3590102450 0x7f35901988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 --2- 172.21.15.122:0/3192325567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3590101a90 0x7f3590198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 >> 172.21.15.122:0/3192325567 conn(0x7f35900fd5a0 msgr2=0x7f35900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 shutdown_connections 2024-04-02T13:36:54.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.683+0000 7f35958ba640 1 -- 172.21.15.122:0/3192325567 wait complete. 2024-04-02T13:36:54.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:36:54.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 -- 172.21.15.122:0/1417739319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101150 msgr2=0x7f116c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 --2- 172.21.15.122:0/1417739319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101150 0x7f116c1015b0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1160009900 tx=0x7f116002bc80 comp rx=0 tx=0).stop 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 -- 172.21.15.122:0/1417739319 shutdown_connections 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 --2- 172.21.15.122:0/1417739319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c101af0 0x7f116c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 --2- 172.21.15.122:0/1417739319 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101150 0x7f116c1015b0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 --2- 172.21.15.122:0/1417739319 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c1059b0 0x7f116c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 -- 172.21.15.122:0/1417739319 >> 172.21.15.122:0/1417739319 conn(0x7f116c076fc0 msgr2=0x7f116c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 -- 172.21.15.122:0/1417739319 shutdown_connections 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 -- 172.21.15.122:0/1417739319 wait complete. 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.832+0000 7f117160d640 1 Processor -- start 2024-04-02T13:36:54.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 -- start start 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c101150 0x7f116c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 0x7f116c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f116c110eb0 con 0x7f116c101af0 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f116c110d30 con 0x7f116c101150 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f116c111030 con 0x7f116c1059b0 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35700/0 (socket says 172.21.15.122:35700) 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 -- 172.21.15.122:0/3028706464 learned_addr learned my addr 172.21.15.122:0/3028706464 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116affd640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c101150 0x7f116c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 0x7f116c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 -- 172.21.15.122:0/3028706464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 msgr2=0x7f116c19bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 0x7f116c19bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 -- 172.21.15.122:0/3028706464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c101150 msgr2=0x7f116c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c101150 0x7f116c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 -- 172.21.15.122:0/3028706464 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116c19c2e0 con 0x7f116c101af0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 0x7f116c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f116a7fc640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1160006010 tx=0x7f1160004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f1153fff640 1 -- 172.21.15.122:0/3028706464 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f116002caa0 con 0x7f116c101af0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116c19c570 con 0x7f116c101af0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f1153fff640 1 -- 172.21.15.122:0/3028706464 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f116002cc40 con 0x7f116c101af0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.833+0000 7f1153fff640 1 -- 172.21.15.122:0/3028706464 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160003e50 con 0x7f116c101af0 2024-04-02T13:36:54.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.834+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f116c100040 con 0x7f116c101af0 2024-04-02T13:36:54.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f1153fff640 1 -- 172.21.15.122:0/3028706464 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f11600035f0 con 0x7f116c101af0 2024-04-02T13:36:54.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 Processor -- start 2024-04-02T13:36:54.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f1153fff640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 0x7f11540a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 -- start start 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 0x7f11540a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f1153fff640 1 -- 172.21.15.122:0/3028706464 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f11600f4780 con 0x7f116c101af0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e26648760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e2664b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e265a37b0 con 0x556e266400c0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e26542570 con 0x556e2663f7a0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f11741f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556e2659f760 con 0x556e26648ca0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e26648760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e2664b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e26648760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35704/0 (socket says 172.21.15.122:35704) 2024-04-02T13:36:54.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116a7fc640 1 -- 172.21.15.122:0/3998026801 learned_addr learned my addr 172.21.15.122:0/3998026801 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.836+0000 7f116affd640 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48608/0 (socket says 172.21.15.122:48608) 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 -- 172.21.15.122:0/3998026801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 msgr2=0x556e2664b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e2664b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 -- 172.21.15.122:0/3998026801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 msgr2=0x556e26648760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e26648760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 -- 172.21.15.122:0/3998026801 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e262b73f0 con 0x556e2663f7a0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 0x7f11540a6380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1164009fd0 tx=0x7f1164014cb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f116affd640 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f115c009ba0 tx=0x7f115c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e2ec6ca493fc8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f11517fa640 1 -- 172.21.15.122:0/3998026801 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004050 con 0x556e2663f7a0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f11517fa640 1 -- 172.21.15.122:0/3998026801 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f115c005500 con 0x556e2663f7a0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.837+0000 7f11517fa640 1 -- 172.21.15.122:0/3998026801 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004d90 con 0x556e2663f7a0 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- 172.21.15.122:0/3998026801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 msgr2=0x556e2663fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f115c009ba0 tx=0x7f115c011810 comp rx=0 tx=0).stop 2024-04-02T13:36:54.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- 172.21.15.122:0/3998026801 shutdown_connections 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e2664b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e26648760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- 172.21.15.122:0/3998026801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e2663fb80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- 172.21.15.122:0/3998026801 >> 172.21.15.122:0/3998026801 conn(0x556e26639ed0 msgr2=0x556e2663a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- 172.21.15.122:0/3998026801 shutdown_connections 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- 172.21.15.122:0/3998026801 wait complete. 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- start start 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 Processor -- start 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e266400c0 0x556e266459d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 0x556e26645f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556e2664fea0 con 0x556e2663f7a0 2024-04-02T13:36:54.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556e2655ee80 con 0x556e26648ca0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556e26650020 con 0x556e266400c0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.838+0000 7f11741f9740 10 client.0 _ll_register_callbacks cb 0x556e2659aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e266400c0 0x556e266459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35706/0 (socket says 172.21.15.122:35706) 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 0x556e26645f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 -- 172.21.15.122:0/2199662863 learned_addr learned my addr 172.21.15.122:0/2199662863 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 -- 172.21.15.122:0/2199662863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e266400c0 msgr2=0x556e266459d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e266400c0 0x556e266459d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 -- 172.21.15.122:0/2199662863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 msgr2=0x556e26645f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 0x556e26645f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 -- 172.21.15.122:0/2199662863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e263817d0 con 0x556e2663f7a0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116b7fe640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 0x556e26645f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f116affd640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f115c009380 tx=0x7f115c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c9f64219664550 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f114eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c004f20 con 0x556e2663f7a0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f115c0050c0 con 0x556e2663f7a0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115c005500 con 0x556e2663f7a0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f115c033050 con 0x556e2663f7a0 2024-04-02T13:36:54.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f11741f9740 10 client.16251 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e2665be50 con 0x556e2663f7a0 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.839+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f115c005500 con 0x556e2663f7a0 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f115c02e440 con 0x556e2663f7a0 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 10 client.16251 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 10 client.16251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:54.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 20 client.16251 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e2665d250 0x556e2665f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 4 client.16251 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e265b0cf0 con 0x556e2665d250 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f116a7fc640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e2665d250 0x556e2665f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f114eff5640 10 client.16251 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.840+0000 7f116a7fc640 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e2665d250 0x556e2665f640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf9a40a471ae427d server_cookie=cb89d0825b8cb03 in_seq=0 out_seq=0 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f114eff5640 1 -- 172.21.15.122:0/2199662863 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e265b0cf0 con 0x556e2665d250 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f114eff5640 10 client.16251 handle_command_reply: tid=0 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 client.16251 shutdown 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 msgr2=0x556e26648640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f115c009380 tx=0x7f115c005d30 comp rx=0 tx=0).stop 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 shutdown_connections 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e2665d250 0x556e2665f640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e26648ca0 0x556e26645f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e266400c0 0x556e266459d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f114eff5640 0 client.16251 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 --2- 172.21.15.122:0/2199662863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e2663f7a0 0x556e26648640 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 >> 172.21.15.122:0/2199662863 conn(0x556e26639ed0 msgr2=0x556e2663bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 shutdown_connections 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 1 -- 172.21.15.122:0/2199662863 wait complete. 2024-04-02T13:36:54.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.841+0000 7f11741f9740 20 client.16251 trim_cache size 0 max 16384 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.854+0000 7f11741f9740 1 Processor -- start 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.854+0000 7f11741f9740 1 -- start start 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e26645990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e26638db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1acc90 con 0x556e266400c0 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1ae7d0 con 0x556e2663f7a0 2024-04-02T13:36:54.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f11741f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1ad360 con 0x556e26648ca0 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e26645990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35714/0 (socket says 172.21.15.122:35714) 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 -- 172.21.15.122:0/1635332457 learned_addr learned my addr 172.21.15.122:0/1635332457 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116b7fe640 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e26638db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 -- 172.21.15.122:0/1635332457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 msgr2=0x556e26638db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e26638db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 -- 172.21.15.122:0/1635332457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 msgr2=0x556e26645990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e26645990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 -- 172.21.15.122:0/1635332457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e266462e0 con 0x556e266400c0 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f116a7fc640 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f11600bce70 tx=0x7f1160005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700b7a2e42dc6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f1150ff9640 1 -- 172.21.15.122:0/1635332457 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600d8070 con 0x556e266400c0 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f1150ff9640 1 -- 172.21.15.122:0/1635332457 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1160004c60 con 0x556e266400c0 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.855+0000 7f1150ff9640 1 -- 172.21.15.122:0/1635332457 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600be7a0 con 0x556e266400c0 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- 172.21.15.122:0/1635332457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 msgr2=0x556e266452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f11600bce70 tx=0x7f1160005700 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- 172.21.15.122:0/1635332457 shutdown_connections 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e26648ca0 0x556e26638db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e266400c0 0x556e266452b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 --2- 172.21.15.122:0/1635332457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e2663f7a0 0x556e26645990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- 172.21.15.122:0/1635332457 >> 172.21.15.122:0/1635332457 conn(0x556e26639ed0 msgr2=0x556e266386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- 172.21.15.122:0/1635332457 shutdown_connections 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- 172.21.15.122:0/1635332457 wait complete. 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 -- start start 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.856+0000 7f11741f9740 1 Processor -- start 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 0x556e266434c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e266400c0 0x556e26641c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1ae100 con 0x556e26648ca0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1ada30 con 0x556e266400c0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f116c1aac70 con 0x556e2663f7a0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 10 client.0 _ll_register_callbacks cb 0x7f116002cdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35716/0 (socket says 172.21.15.122:35716) 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 -- 172.21.15.122:0/3197615643 learned_addr learned my addr 172.21.15.122:0/3197615643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116a7fc640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e266400c0 0x556e26641c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116affd640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 0x556e266434c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 -- 172.21.15.122:0/3197615643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 msgr2=0x556e266434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 0x556e266434c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 -- 172.21.15.122:0/3197615643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e266400c0 msgr2=0x556e26641c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e266400c0 0x556e26641c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 -- 172.21.15.122:0/3197615643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e2665f9b0 con 0x556e26648ca0 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116affd640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 0x556e266434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f116b7fe640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f116401c000 tx=0x7f116401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cca5158da1d7481 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f114eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1164020440 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f11640205e0 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1160004c60 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f11640207e0 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 10 client.16257 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.857+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556e26655060 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1160004c60 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1164011a80 con 0x556e26648ca0 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 10 client.16257 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:54.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 10 client.16257 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 20 client.16257 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e26655810 0x556e266c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 4 client.16257 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e265ab040 con 0x556e26655810 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f116affd640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e26655810 0x556e266c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f114eff5640 10 client.16257 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f116affd640 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e26655810 0x556e266c5650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=279ae1da345882f0 server_cookie=8caf0914bc407946 in_seq=0 out_seq=0 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f114eff5640 1 -- 172.21.15.122:0/3197615643 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e265ab040 con 0x556e26655810 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f114eff5640 10 client.16257 handle_command_reply: tid=0 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 1 client.16257 shutdown 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.858+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 msgr2=0x556e26642180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f116401c000 tx=0x7f116401a000 comp rx=0 tx=0).stop 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 shutdown_connections 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556e26655810 0x556e266c5650 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556e26648ca0 0x556e26642180 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f114eff5640 0 client.16257 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556e266400c0 0x556e26641c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 --2- 172.21.15.122:0/3197615643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556e2663f7a0 0x556e266434c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 >> 172.21.15.122:0/3197615643 conn(0x556e26639ed0 msgr2=0x556e26638f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 shutdown_connections 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 1 -- 172.21.15.122:0/3197615643 wait complete. 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f11741f9740 20 client.16257 trim_cache size 0 max 16384 2024-04-02T13:36:54.872 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:54.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 msgr2=0x7f11540a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 0x7f11540a6380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1164009fd0 tx=0x7f1164014cb0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 msgr2=0x7f116c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1160006010 tx=0x7f1160004270 comp rx=0 tx=0).stop 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 shutdown_connections 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f11540a3ec0 0x7f11540a6380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f116c1059b0 0x7f116c19bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f116c101af0 0x7f116c197830 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 --2- 172.21.15.122:0/3028706464 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f116c101150 0x7f116c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 >> 172.21.15.122:0/3028706464 conn(0x7f116c076fc0 msgr2=0x7f116c1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 shutdown_connections 2024-04-02T13:36:54.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.859+0000 7f117160d640 1 -- 172.21.15.122:0/3028706464 wait complete. 2024-04-02T13:36:54.876 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:36:54.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 40276 15.56 MB/sec warmup 15 sec latency 27.139 ms 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/1407851073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 msgr2=0x7fc5c0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/1407851073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c0100e30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009930 tx=0x7fc5a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/1407851073 shutdown_connections 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/1407851073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 0x7fc5c010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/1407851073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c0100e30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/1407851073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 0x7fc5c0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/1407851073 >> 172.21.15.122:0/1407851073 conn(0x7fc5c0076d80 msgr2=0x7fc5c0077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:54.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/1407851073 shutdown_connections 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.983+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/1407851073 wait complete. 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 Processor -- start 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 -- start start 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 0x7fc5c00fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 0x7fc5c00feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0110df0 con 0x7fc5c01009b0 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0110c70 con 0x7fc5c01069c0 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.984+0000 7fc5c4d2b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c0110f70 con 0x7fc5c0101370 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35730/0 (socket says 172.21.15.122:35730) 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 -- 172.21.15.122:0/2374209830 learned_addr learned my addr 172.21.15.122:0/2374209830 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 0x7fc5c00feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 0x7fc5c00fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 -- 172.21.15.122:0/2374209830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 msgr2=0x7fc5c00fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 0x7fc5c00fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 -- 172.21.15.122:0/2374209830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 msgr2=0x7fc5c00feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 0x7fc5c00feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 -- 172.21.15.122:0/2374209830 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c01a68f0 con 0x7fc5c01009b0 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000e9f0 tx=0x7fc5b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:54.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5b77fe640 1 -- 172.21.15.122:0/2374209830 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b000cd80 con 0x7fc5c01009b0 2024-04-02T13:36:54.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5b77fe640 1 -- 172.21.15.122:0/2374209830 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b0010910 con 0x7fc5c01009b0 2024-04-02T13:36:54.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c01a6be0 con 0x7fc5c01009b0 2024-04-02T13:36:54.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5b77fe640 1 -- 172.21.15.122:0/2374209830 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00188f0 con 0x7fc5c01009b0 2024-04-02T13:36:54.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.985+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c01a7100 con 0x7fc5c01009b0 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.987+0000 7fc5b77fe640 1 -- 172.21.15.122:0/2374209830 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc5b0010ac0 con 0x7fc5c01009b0 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 Processor -- start 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 -- start start 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b851e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b8521090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5602b84797b0 con 0x5602b85157a0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5602b8418570 con 0x5602b85160c0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5602b8475760 con 0x5602b851eca0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5b77fe640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b851e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35744/0 (socket says 172.21.15.122:35744) 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5bed9d640 1 -- 172.21.15.122:0/4079851250 learned_addr learned my addr 172.21.15.122:0/4079851250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b8521090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.988+0000 7fc5b77fe640 1 -- 172.21.15.122:0/2374209830 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fc5b00d0eb0 con 0x7fc5c01009b0 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 -- 172.21.15.122:0/4079851250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 msgr2=0x5602b8521090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b8521090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 -- 172.21.15.122:0/4079851250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 msgr2=0x5602b851e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b851e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 -- 172.21.15.122:0/4079851250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b818d3f0 con 0x5602b85157a0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5be59c640 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b851e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8012d80 tx=0x7fc5b8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459998f7b7ae2c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8010c40 tx=0x7fc5b800c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5b4ff9640 1 -- 172.21.15.122:0/4079851250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8024070 con 0x5602b85157a0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5b4ff9640 1 -- 172.21.15.122:0/4079851250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b8009d80 con 0x5602b85157a0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.989+0000 7fc5b4ff9640 1 -- 172.21.15.122:0/4079851250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8005330 con 0x5602b85157a0 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4079851250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 msgr2=0x5602b8515b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8012d80 tx=0x7fc5b8010740 comp rx=0 tx=0).stop 2024-04-02T13:36:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4079851250 shutdown_connections 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b8521090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b851e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4079851250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b8515b80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4079851250 >> 172.21.15.122:0/4079851250 conn(0x5602b850fed0 msgr2=0x5602b85102b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4079851250 shutdown_connections 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4079851250 wait complete. 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- start start 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 Processor -- start 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 0x5602b851e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 0x5602b851bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5602b8521a30 con 0x5602b851eca0 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.990+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5602b8434e80 con 0x5602b85157a0 2024-04-02T13:36:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5602b8521bb0 con 0x5602b85160c0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5c79ed740 10 client.0 _ll_register_callbacks cb 0x5602b8470aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 0x5602b851bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 0x5602b851e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35760/0 (socket says 172.21.15.122:35760) 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5bed9d640 1 -- 172.21.15.122:0/4144598831 learned_addr learned my addr 172.21.15.122:0/4144598831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 -- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 msgr2=0x5602b851bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 0x5602b851bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 -- 172.21.15.122:0/4144598831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 msgr2=0x5602b851c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 -- 172.21.15.122:0/4144598831 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b82577d0 con 0x5602b85157a0 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 0x5602b851bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5be59c640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 0x5602b851e720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ce270 tx=0x7fc5b001d9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4185215e10c76d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b005be60 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b8009d80 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fc5b00991e0 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5b0025170 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5c79ed740 10 client.25751 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.991+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602b852db10 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b8009d80 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5b009bfc0 con 0x5602b85157a0 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 10 client.25751 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 10 client.25751 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 20 client.25751 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b852efd0 0x5602b85313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 4 client.25751 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602b8486cf0 con 0x5602b852efd0 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b852efd0 0x5602b85313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b852efd0 0x5602b85313c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6fdd0d3df3ae48e9 server_cookie=21193528f77dbc74 in_seq=0 out_seq=0 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.992+0000 7fc5a5ffb640 10 client.25751 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/4144598831 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5602b8486cf0 con 0x5602b852efd0 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5a5ffb640 10 client.25751 handle_command_reply: tid=0 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 client.25751 shutdown 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 msgr2=0x5602b851e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 0x5602b851e720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc5b00ce270 tx=0x7fc5b001d9e0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 shutdown_connections 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b852efd0 0x5602b85313c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b851eca0 0x5602b851c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85160c0 0x5602b851bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/4144598831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85157a0 0x5602b851e720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5a5ffb640 0 client.25751 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 >> 172.21.15.122:0/4144598831 conn(0x5602b850fed0 msgr2=0x5602b8511c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 shutdown_connections 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 1 -- 172.21.15.122:0/4144598831 wait complete. 2024-04-02T13:36:55.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:54.993+0000 7fc5c79ed740 20 client.25751 trim_cache size 0 max 16384 2024-04-02T13:36:55.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 Processor -- start 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 -- start start 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b85164a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b851a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01b10c0 con 0x5602b85157a0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01b2c00 con 0x5602b85160c0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01b1790 con 0x5602b851eca0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35774/0 (socket says 172.21.15.122:35774) 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b85164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 -- 172.21.15.122:0/1621896741 learned_addr learned my addr 172.21.15.122:0/1621896741 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b851a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 -- 172.21.15.122:0/1621896741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 msgr2=0x5602b851a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b851a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 -- 172.21.15.122:0/1621896741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 msgr2=0x5602b85164a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b85164a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 -- 172.21.15.122:0/1621896741 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602b851c560 con 0x5602b85157a0 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.008+0000 7fc5be59c640 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc5b009ae20 tx=0x7fc5b0011150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7eac0c84a9e808 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5a7fff640 1 -- 172.21.15.122:0/1621896741 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b009ae70 con 0x5602b85157a0 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5a7fff640 1 -- 172.21.15.122:0/1621896741 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b0010e10 con 0x5602b85157a0 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5a7fff640 1 -- 172.21.15.122:0/1621896741 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b00991e0 con 0x5602b85157a0 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 -- 172.21.15.122:0/1621896741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 msgr2=0x5602b852a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc5b009ae20 tx=0x7fc5b0011150 comp rx=0 tx=0).stop 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 -- 172.21.15.122:0/1621896741 shutdown_connections 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b851eca0 0x5602b851a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b85160c0 0x5602b85164a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/1621896741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85157a0 0x5602b852a8f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 -- 172.21.15.122:0/1621896741 >> 172.21.15.122:0/1621896741 conn(0x5602b850fed0 msgr2=0x5602b851a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 -- 172.21.15.122:0/1621896741 shutdown_connections 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.009+0000 7fc5c79ed740 1 -- 172.21.15.122:0/1621896741 wait complete. 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 -- start start 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 Processor -- start 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85157a0 0x5602b85197b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 0x5602b8518480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01b2530 con 0x5602b85160c0 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01b1e60 con 0x5602b851eca0 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c01af0a0 con 0x5602b85157a0 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5c79ed740 10 client.0 _ll_register_callbacks cb 0x7fc5b000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35784/0 (socket says 172.21.15.122:35784) 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 -- 172.21.15.122:0/2756431398 learned_addr learned my addr 172.21.15.122:0/2756431398 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85157a0 0x5602b85197b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 0x5602b8518480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 -- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85157a0 msgr2=0x5602b85197b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85157a0 0x5602b85197b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 -- 172.21.15.122:0/2756431398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 msgr2=0x5602b8518480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 0x5602b8518480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 -- 172.21.15.122:0/2756431398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602b8531730 con 0x5602b85160c0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5bed9d640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 0x5602b8518480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5b7fff640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8006010 tx=0x7fc5a8004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b85649a170c3d2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.010+0000 7fc5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8055070 con 0x5602b85160c0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fc5a804d750 con 0x5602b85160c0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5b0010e10 con 0x5602b85160c0 2024-04-02T13:36:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc5a8060050 con 0x5602b85160c0 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 10 client.16275 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5602b8526e40 con 0x5602b85160c0 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5b0010e10 con 0x5602b85160c0 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc5a8004ee0 con 0x5602b85160c0 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 10 client.16275 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 10 client.16275 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 20 client.16275 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b8595500 0x5602b85978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 4 client.16275 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602b8481040 con 0x5602b8595500 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.011+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b8595500 0x5602b85978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5a5ffb640 10 client.16275 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5be59c640 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b8595500 0x5602b85978f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e0df845961598586 server_cookie=e7c2d8c1b3416c4c in_seq=0 out_seq=0 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5a5ffb640 1 -- 172.21.15.122:0/2756431398 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602b8481040 con 0x5602b8595500 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5a5ffb640 10 client.16275 handle_command_reply: tid=0 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 client.16275 shutdown 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 msgr2=0x5602b8517d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8006010 tx=0x7fc5a8004590 comp rx=0 tx=0).stop 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 shutdown_connections 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5602b851eca0 0x5602b8518480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5602b8595500 0x5602b85978f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5602b85160c0 0x5602b8517d40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 --2- 172.21.15.122:0/2756431398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602b85157a0 0x5602b85197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5a5ffb640 0 client.16275 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 >> 172.21.15.122:0/2756431398 conn(0x5602b850fed0 msgr2=0x5602b851ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 shutdown_connections 2024-04-02T13:36:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 1 -- 172.21.15.122:0/2756431398 wait complete. 2024-04-02T13:36:55.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.012+0000 7fc5c79ed740 20 client.16275 trim_cache size 0 max 16384 2024-04-02T13:36:55.023 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 msgr2=0x7fc5ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8010c40 tx=0x7fc5b800c000 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 msgr2=0x7fc5c00fe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000e9f0 tx=0x7fc5b000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 shutdown_connections 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc5ac0a3d20 0x7fc5ac0a61e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5c01069c0 0x7fc5c00feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5c0101370 0x7fc5c00fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 --2- 172.21.15.122:0/2374209830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5c01009b0 0x7fc5c00fe0a0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 >> 172.21.15.122:0/2374209830 conn(0x7fc5c0076d80 msgr2=0x7fc5c006a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 shutdown_connections 2024-04-02T13:36:55.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.013+0000 7fc5c4d2b640 1 -- 172.21.15.122:0/2374209830 wait complete. 2024-04-02T13:36:55.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 -- 172.21.15.122:0/3721475367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f40696a0 msgr2=0x7fd8f410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3721475367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f40696a0 0x7fd8f410e940 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009930 tx=0x7fd8ec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 -- 172.21.15.122:0/3721475367 shutdown_connections 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3721475367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f40696a0 0x7fd8f410e940 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3721475367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.134+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3721475367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f4107a80 0x7fd8f4107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- 172.21.15.122:0/3721475367 >> 172.21.15.122:0/3721475367 conn(0x7fd8f4077b00 msgr2=0x7fd8f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- 172.21.15.122:0/3721475367 shutdown_connections 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- 172.21.15.122:0/3721475367 wait complete. 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 Processor -- start 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- start start 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f40696a0 0x7fd8f41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4107a80 0x7fd8f419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4111ee0 con 0x7fd8f4068ce0 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4111d60 con 0x7fd8f40696a0 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8faca0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f4112060 con 0x7fd8f4107a80 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35800/0 (socket says 172.21.15.122:35800) 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8f8a3c640 1 -- 172.21.15.122:0/3603632076 learned_addr learned my addr 172.21.15.122:0/3603632076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.135+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f40696a0 0x7fd8f41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f923d640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4107a80 0x7fd8f419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 -- 172.21.15.122:0/3603632076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4107a80 msgr2=0x7fd8f419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4107a80 0x7fd8f419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 -- 172.21.15.122:0/3603632076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f40696a0 msgr2=0x7fd8f41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f40696a0 0x7fd8f41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 -- 172.21.15.122:0/3603632076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f419d350 con 0x7fd8f4068ce0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ebc0 tx=0x7fd8e400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f1ffb640 1 -- 172.21.15.122:0/3603632076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010630 con 0x7fd8f4068ce0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f1ffb640 1 -- 172.21.15.122:0/3603632076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e4010cb0 con 0x7fd8f4068ce0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8f1ffb640 1 -- 172.21.15.122:0/3603632076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4018bd0 con 0x7fd8f4068ce0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f419d640 con 0x7fd8f4068ce0 2024-04-02T13:36:55.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.136+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f41a4ed0 con 0x7fd8f4068ce0 2024-04-02T13:36:55.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.138+0000 7fd8f1ffb640 1 -- 172.21.15.122:0/3603632076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd8e40107d0 con 0x7fd8f4068ce0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 Processor -- start 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 -- start start 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8f1ffb640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 0x7fd8e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f53090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8f923d640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 0x7fd8e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561fd2eab7b0 con 0x561fd2f50ca0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561fd2e4a570 con 0x561fd2f477a0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.139+0000 7fd8fd885740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561fd2ea7760 con 0x561fd2f480c0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f923d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f53090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f1ffb640 1 -- 172.21.15.122:0/3603632076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd8e40d1160 con 0x7fd8f4068ce0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42226/0 (socket says 172.21.15.122:42226) 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48704/0 (socket says 172.21.15.122:48704) 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 -- 172.21.15.122:0/742914092 learned_addr learned my addr 172.21.15.122:0/742914092 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 -- 172.21.15.122:0/742914092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 msgr2=0x561fd2f47b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 -- 172.21.15.122:0/742914092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 msgr2=0x561fd2f53090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f53090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 -- 172.21.15.122:0/742914092 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2bbf3f0 con 0x561fd2f480c0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f923d640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 0x7fd8e00a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec036040 tx=0x7fd8ec005e00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009ca0 tx=0x7fd8dc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca514f99173fad3e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8eaffd640 1 -- 172.21.15.122:0/742914092 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc01e070 con 0x561fd2f480c0 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8eaffd640 1 -- 172.21.15.122:0/742914092 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc009d90 con 0x561fd2f480c0 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.140+0000 7fd8eaffd640 1 -- 172.21.15.122:0/742914092 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc018730 con 0x561fd2f480c0 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- 172.21.15.122:0/742914092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 msgr2=0x561fd2f50760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009ca0 tx=0x7fd8dc011810 comp rx=0 tx=0).stop 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- 172.21.15.122:0/742914092 shutdown_connections 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f53090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 --2- 172.21.15.122:0/742914092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f47b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- 172.21.15.122:0/742914092 >> 172.21.15.122:0/742914092 conn(0x561fd2f41ed0 msgr2=0x561fd2f422b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- 172.21.15.122:0/742914092 shutdown_connections 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- 172.21.15.122:0/742914092 wait complete. 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 -- start start 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.141+0000 7fd8fd885740 1 Processor -- start 2024-04-02T13:36:55.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f477a0 0x561fd2f506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 0x561fd2f4da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 0x561fd2f4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561fd2f53a90 con 0x561fd2f480c0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561fd2e66e80 con 0x561fd2f50ca0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561fd2f53c10 con 0x561fd2f477a0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8fd885740 10 client.0 _ll_register_callbacks cb 0x561fd2ea2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 0x561fd2f4da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 0x561fd2f4da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35820/0 (socket says 172.21.15.122:35820) 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f3fff640 1 -- 172.21.15.122:0/200328517 learned_addr learned my addr 172.21.15.122:0/200328517 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 0x561fd2f4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f477a0 0x561fd2f506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 -- 172.21.15.122:0/200328517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f477a0 msgr2=0x561fd2f506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f477a0 0x561fd2f506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 -- 172.21.15.122:0/200328517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 msgr2=0x561fd2f4da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 0x561fd2f4da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.142+0000 7fd8f923d640 1 -- 172.21.15.122:0/200328517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd2c897d0 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8f923d640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 0x561fd2f4e050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec02c170 tx=0x7fd8ec02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2c114ba196e8406 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec005040 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8dc009d90 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd8ec0051e0 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8ec05c050 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 10 client.25769 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fd2f5fac0 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8dc009d90 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd8ec0035f0 con 0x561fd2f50ca0 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 10 client.25769 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 10 client.25769 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 20 client.25769 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2f60f80 0x561fd2f633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 4 client.25769 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.143+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fd2eb8cf0 con 0x561fd2f60f80 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.144+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2f60f80 0x561fd2f633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.144+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2f60f80 0x561fd2f633b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2bb494df0a61d597 server_cookie=4cc1e79669904a56 in_seq=0 out_seq=0 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.144+0000 7fd8d3fff640 10 client.25769 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8d3fff640 1 -- 172.21.15.122:0/200328517 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fd2eb8cf0 con 0x561fd2f60f80 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8d3fff640 10 client.25769 handle_command_reply: tid=0 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 client.25769 shutdown 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 msgr2=0x561fd2f4e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 0x561fd2f4e050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec02c170 tx=0x7fd8ec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 shutdown_connections 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2f60f80 0x561fd2f633b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f50ca0 0x561fd2f4e050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8d3fff640 0 client.25769 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f480c0 0x561fd2f4da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 --2- 172.21.15.122:0/200328517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f477a0 0x561fd2f506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 >> 172.21.15.122:0/200328517 conn(0x561fd2f41ed0 msgr2=0x561fd2f43c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 shutdown_connections 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 1 -- 172.21.15.122:0/200328517 wait complete. 2024-04-02T13:36:55.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.145+0000 7fd8fd885740 20 client.25769 trim_cache size 0 max 16384 2024-04-02T13:36:55.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 Processor -- start 2024-04-02T13:36:55.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 -- start start 2024-04-02T13:36:55.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f5c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41acd60 con 0x561fd2f50ca0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41ae8a0 con 0x561fd2f477a0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.158+0000 7fd8fd885740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41ad430 con 0x561fd2f480c0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35830/0 (socket says 172.21.15.122:35830) 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f5c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 -- 172.21.15.122:0/4030897406 learned_addr learned my addr 172.21.15.122:0/4030897406 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 -- 172.21.15.122:0/4030897406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 msgr2=0x561fd2f50170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 -- 172.21.15.122:0/4030897406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 msgr2=0x561fd2f5c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f5c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 -- 172.21.15.122:0/4030897406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fd2f47170 con 0x561fd2f50ca0 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8f923d640 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec003a90 tx=0x7fd8ec02cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425e83140c666a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8ea7fc640 1 -- 172.21.15.122:0/4030897406 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec009520 con 0x561fd2f50ca0 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8ea7fc640 1 -- 172.21.15.122:0/4030897406 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ec053070 con 0x561fd2f50ca0 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.159+0000 7fd8ea7fc640 1 -- 172.21.15.122:0/4030897406 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004920 con 0x561fd2f50ca0 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- 172.21.15.122:0/4030897406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 msgr2=0x561fd2f506b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec003a90 tx=0x7fd8ec02cc50 comp rx=0 tx=0).stop 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- 172.21.15.122:0/4030897406 shutdown_connections 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f506b0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f50170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 --2- 172.21.15.122:0/4030897406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f5c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- 172.21.15.122:0/4030897406 >> 172.21.15.122:0/4030897406 conn(0x561fd2f41ed0 msgr2=0x561fd2f4da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- 172.21.15.122:0/4030897406 shutdown_connections 2024-04-02T13:36:55.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- 172.21.15.122:0/4030897406 wait complete. 2024-04-02T13:36:55.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 -- start start 2024-04-02T13:36:55.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.160+0000 7fd8fd885740 1 Processor -- start 2024-04-02T13:36:55.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41ae1d0 con 0x561fd2f50ca0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41adb00 con 0x561fd2f477a0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f41aad40 con 0x561fd2f480c0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8fd885740 10 client.0 _ll_register_callbacks cb 0x7fd8ec047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f923d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f8a3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48742/0 (socket says 172.21.15.122:48742) 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f923d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35840/0 (socket says 172.21.15.122:35840) 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f8a3c640 1 -- 172.21.15.122:0/1850430458 learned_addr learned my addr 172.21.15.122:0/1850430458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 -- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 msgr2=0x561fd2f58e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 -- 172.21.15.122:0/1850430458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 msgr2=0x561fd2f4acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 -- 172.21.15.122:0/1850430458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fd2f4b360 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f923d640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8f3fff640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f593a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc01afd0 tx=0x7fd8dc005570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab7ed7337adcf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.161+0000 7fd8d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc029070 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8ec053070 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd8dc009780 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8dc034050 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 10 client.25672 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561fd2f51aa0 con 0x561fd2f480c0 2024-04-02T13:36:55.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ec053070 con 0x561fd2f480c0 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd8dc019530 con 0x561fd2f480c0 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 10 client.25672 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 10 client.25672 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 20 client.25672 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2fc7540 0x561fd2fc9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 4 client.25672 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fd2eb3040 con 0x561fd2fc7540 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.162+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2fc7540 0x561fd2fc9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8d3fff640 10 client.25672 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8f8a3c640 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2fc7540 0x561fd2fc9930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5e8739c285a86ce server_cookie=8ffad8d27b78c394 in_seq=0 out_seq=0 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8d3fff640 1 -- 172.21.15.122:0/1850430458 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fd2eb3040 con 0x561fd2fc7540 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8d3fff640 10 client.25672 handle_command_reply: tid=0 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 client.25672 shutdown 2024-04-02T13:36:55.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 msgr2=0x561fd2f593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f593a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc01afd0 tx=0x7fd8dc005570 comp rx=0 tx=0).stop 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 shutdown_connections 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561fd2f50ca0 0x561fd2f4acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561fd2f480c0 0x561fd2f593a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561fd2fc7540 0x561fd2fc9930 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 --2- 172.21.15.122:0/1850430458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561fd2f477a0 0x561fd2f58e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8d3fff640 0 client.25672 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 >> 172.21.15.122:0/1850430458 conn(0x561fd2f41ed0 msgr2=0x561fd2f4c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 shutdown_connections 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 1 -- 172.21.15.122:0/1850430458 wait complete. 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.163+0000 7fd8fd885740 20 client.25672 trim_cache size 0 max 16384 2024-04-02T13:36:55.173 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:55.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 msgr2=0x7fd8e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 0x7fd8e00a6260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec036040 tx=0x7fd8ec005e00 comp rx=0 tx=0).stop 2024-04-02T13:36:55.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 msgr2=0x7fd8f4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ebc0 tx=0x7fd8e400c6a0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 shutdown_connections 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd8e00a3da0 0x7fd8e00a6260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8f4107a80 0x7fd8f419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd8f40696a0 0x7fd8f41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 --2- 172.21.15.122:0/3603632076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8f4068ce0 0x7fd8f4198360 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 >> 172.21.15.122:0/3603632076 conn(0x7fd8f4077b00 msgr2=0x7fd8f40fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 shutdown_connections 2024-04-02T13:36:55.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.164+0000 7fd8faca0640 1 -- 172.21.15.122:0/3603632076 wait complete. 2024-04-02T13:36:55.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 -- 172.21.15.122:0/2957569086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c106980 msgr2=0x7f822c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 --2- 172.21.15.122:0/2957569086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c106980 0x7f822c106d60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8228009930 tx=0x7f822802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 -- 172.21.15.122:0/2957569086 shutdown_connections 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 --2- 172.21.15.122:0/2957569086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c101330 0x7f822c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 --2- 172.21.15.122:0/2957569086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c100970 0x7f822c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 --2- 172.21.15.122:0/2957569086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c106980 0x7f822c106d60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 -- 172.21.15.122:0/2957569086 >> 172.21.15.122:0/2957569086 conn(0x7f822c077b00 msgr2=0x7f822c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 -- 172.21.15.122:0/2957569086 shutdown_connections 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.287+0000 7f823431c640 1 -- 172.21.15.122:0/2957569086 wait complete. 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 Processor -- start 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 -- start start 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 0x7f822c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c106980 0x7f822c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f822c069b60 con 0x7f822c100970 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0699e0 con 0x7f822c101330 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f823431c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1120a0 con 0x7f822c106980 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82320b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 0x7f822c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82320b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35868/0 (socket says 172.21.15.122:35868) 2024-04-02T13:36:55.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82320b8640 1 -- 172.21.15.122:0/254860467 learned_addr learned my addr 172.21.15.122:0/254860467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82328b9640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c106980 0x7f822c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 -- 172.21.15.122:0/254860467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c106980 msgr2=0x7f822c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c106980 0x7f822c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 -- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 msgr2=0x7f822c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 -- 172.21.15.122:0/254860467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c19d3a0 con 0x7f822c101330 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82320b8640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.288+0000 7f82318b7640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 0x7f822c1988f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9f0 tx=0x7f821c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.289+0000 7f82237fe640 1 -- 172.21.15.122:0/254860467 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00ccc0 con 0x7f822c101330 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.289+0000 7f82237fe640 1 -- 172.21.15.122:0/254860467 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c010910 con 0x7f822c101330 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.289+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c19d690 con 0x7f822c101330 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.289+0000 7f82237fe640 1 -- 172.21.15.122:0/254860467 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c018960 con 0x7f822c101330 2024-04-02T13:36:55.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.289+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c1a4f20 con 0x7f822c101330 2024-04-02T13:36:55.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.291+0000 7f8236fe8740 1 Processor -- start 2024-04-02T13:36:55.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.291+0000 7f82237fe640 1 -- 172.21.15.122:0/254860467 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f821c010ac0 con 0x7f822c101330 2024-04-02T13:36:55.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.291+0000 7f8236fe8740 1 -- start start 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc990187b0 con 0x55bc990bdca0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bc98fb7570 con 0x55bc990b50c0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f8236fe8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bc99014760 con 0x55bc990b47a0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35874/0 (socket says 172.21.15.122:35874) 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82320b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 -- 172.21.15.122:0/487187590 learned_addr learned my addr 172.21.15.122:0/487187590 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82237fe640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 0x7f82180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82328b9640 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82328b9640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 0x7f82180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82237fe640 1 -- 172.21.15.122:0/254860467 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f821c0d0f20 con 0x7f822c101330 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 -- 172.21.15.122:0/487187590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 msgr2=0x55bc990b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 -- 172.21.15.122:0/487187590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 msgr2=0x55bc990bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 -- 172.21.15.122:0/487187590 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc98d2c3f0 con 0x55bc990bdca0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82328b9640 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.292+0000 7f82318b7640 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f821c0ce1f0 tx=0x7f821c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=215bcbc49dcf19d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f82328b9640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 0x7f82180a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8224005fd0 tx=0x7f8224009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8220bfe640 1 -- 172.21.15.122:0/487187590 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0a0e80 con 0x55bc990bdca0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8220bfe640 1 -- 172.21.15.122:0/487187590 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c0a5740 con 0x55bc990bdca0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8220bfe640 1 -- 172.21.15.122:0/487187590 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c09bfd0 con 0x55bc990bdca0 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 -- 172.21.15.122:0/487187590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 msgr2=0x55bc990c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f821c0ce1f0 tx=0x7f821c009700 comp rx=0 tx=0).stop 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 -- 172.21.15.122:0/487187590 shutdown_connections 2024-04-02T13:36:55.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990c0090 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 --2- 172.21.15.122:0/487187590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.293+0000 7f8236fe8740 1 -- 172.21.15.122:0/487187590 >> 172.21.15.122:0/487187590 conn(0x55bc990aeed0 msgr2=0x55bc990af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- 172.21.15.122:0/487187590 shutdown_connections 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- 172.21.15.122:0/487187590 wait complete. 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- start start 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 Processor -- start 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990baa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bc990c09d0 con 0x55bc990b47a0 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bc98fd3e80 con 0x55bc990b50c0 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bc990c0b50 con 0x55bc990bdca0 2024-04-02T13:36:55.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f82320b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f8236fe8740 10 client.0 _ll_register_callbacks cb 0x55bc9900faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f82320b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35888/0 (socket says 172.21.15.122:35888) 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f82320b8640 1 -- 172.21.15.122:0/714891382 learned_addr learned my addr 172.21.15.122:0/714891382 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f82318b7640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.294+0000 7f82328b9640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 -- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 msgr2=0x55bc990bb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990bb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 -- 172.21.15.122:0/714891382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 msgr2=0x55bc990baa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990baa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 -- 172.21.15.122:0/714891382 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc98df67d0 con 0x55bc990b47a0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82318b7640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990baa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82328b9640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f82320b8640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8228009230 tx=0x7f8228009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13a1135f19b652f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822802ccc0 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8228004040 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821c0a5740 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8228004240 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 10 client.16305 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc990cca40 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c0a5740 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f82280038c0 con 0x55bc990b47a0 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 10 client.16305 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 10 client.16305 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 20 client.16305 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.295+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990cdf00 0x55bc990d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.296+0000 7f8236fe8740 4 client.16305 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.296+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc99025cf0 con 0x55bc990cdf00 2024-04-02T13:36:55.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.296+0000 7f82318b7640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990cdf00 0x55bc990d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.296+0000 7f8215ffb640 10 client.16305 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.296+0000 7f82318b7640 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990cdf00 0x55bc990d0380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=632c3f2e4f207b44 server_cookie=82d1f896c2e930bc in_seq=0 out_seq=0 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8215ffb640 1 -- 172.21.15.122:0/714891382 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc99025cf0 con 0x55bc990cdf00 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8215ffb640 10 client.16305 handle_command_reply: tid=0 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 client.16305 shutdown 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 msgr2=0x55bc990bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8228009230 tx=0x7f8228009650 comp rx=0 tx=0).stop 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 shutdown_connections 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990cdf00 0x55bc990d0380 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990baa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8215ffb640 0 client.16305 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 --2- 172.21.15.122:0/714891382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b47a0 0x55bc990bd700 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 >> 172.21.15.122:0/714891382 conn(0x55bc990aeed0 msgr2=0x55bc990b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 shutdown_connections 2024-04-02T13:36:55.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 1 -- 172.21.15.122:0/714891382 wait complete. 2024-04-02T13:36:55.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.297+0000 7f8236fe8740 20 client.16305 trim_cache size 0 max 16384 2024-04-02T13:36:55.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 Processor -- start 2024-04-02T13:36:55.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 -- start start 2024-04-02T13:36:55.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1ace60 con 0x55bc990bdca0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1ae9a0 con 0x55bc990b50c0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f8236fe8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1ad530 con 0x55bc990b47a0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82320b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82318b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82328b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82320b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42296/0 (socket says 172.21.15.122:42296) 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82318b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990d02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48776/0 (socket says 172.21.15.122:48776) 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.312+0000 7f82320b8640 1 -- 172.21.15.122:0/596639789 learned_addr learned my addr 172.21.15.122:0/596639789 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 -- 172.21.15.122:0/596639789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 msgr2=0x55bc990d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 -- 172.21.15.122:0/596639789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 msgr2=0x55bc990b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 -- 172.21.15.122:0/596639789 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc990b4ef0 con 0x55bc990b47a0 2024-04-02T13:36:55.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f82320b8640 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8228003d60 tx=0x7f822802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d3458d45a971c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f8217fff640 1 -- 172.21.15.122:0/596639789 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822805a070 con 0x55bc990b47a0 2024-04-02T13:36:55.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f8217fff640 1 -- 172.21.15.122:0/596639789 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8228003ad0 con 0x55bc990b47a0 2024-04-02T13:36:55.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.313+0000 7f8217fff640 1 -- 172.21.15.122:0/596639789 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822802cd20 con 0x55bc990b47a0 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- 172.21.15.122:0/596639789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 msgr2=0x55bc990c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8228003d60 tx=0x7f822802bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- 172.21.15.122:0/596639789 shutdown_connections 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990bdca0 0x55bc990b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b50c0 0x55bc990d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- 172.21.15.122:0/596639789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990b47a0 0x55bc990c9890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- 172.21.15.122:0/596639789 >> 172.21.15.122:0/596639789 conn(0x55bc990aeed0 msgr2=0x55bc990b9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- 172.21.15.122:0/596639789 shutdown_connections 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- 172.21.15.122:0/596639789 wait complete. 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- start start 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 Processor -- start 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b47a0 0x55bc990c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1ae2d0 con 0x55bc990b50c0 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1adc00 con 0x55bc990b47a0 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f822c1aae40 con 0x55bc990bdca0 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f8236fe8740 10 client.0 _ll_register_callbacks cb 0x7f8228009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.314+0000 7f82318b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35908/0 (socket says 172.21.15.122:35908) 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 -- 172.21.15.122:0/1207926632 learned_addr learned my addr 172.21.15.122:0/1207926632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82320b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b47a0 0x55bc990c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82328b9640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 -- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 msgr2=0x55bc990b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 -- 172.21.15.122:0/1207926632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b47a0 msgr2=0x55bc990c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b47a0 0x55bc990c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 -- 172.21.15.122:0/1207926632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc990b8270 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f82318b7640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f821c098fe0 tx=0x7f821c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d961643dac79ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8215ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0d4450 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f821c09ae80 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc99020040 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f821c0a5740 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8236fe8740 10 client.16311 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc990be8d0 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.315+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bc99020040 con 0x55bc990b50c0 2024-04-02T13:36:55.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f821c0b8030 con 0x55bc990b50c0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 10 client.16311 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 20 client.16311 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990becb0 0x55bc99136520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 4 client.16311 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc98f9d6b0 con 0x55bc990becb0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f82320b8640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990becb0 0x55bc99136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f82320b8640 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990becb0 0x55bc99136520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd7214ebe3b243db server_cookie=ca82b02de9f1d783 in_seq=0 out_seq=0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8215ffb640 10 client.16311 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8215ffb640 1 -- 172.21.15.122:0/1207926632 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc98f9d6b0 con 0x55bc990becb0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8215ffb640 10 client.16311 handle_command_reply: tid=0 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 1 client.16311 shutdown 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 msgr2=0x55bc990c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.316+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f821c098fe0 tx=0x7f821c00cf60 comp rx=0 tx=0).stop 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 shutdown_connections 2024-04-02T13:36:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55bc990becb0 0x55bc99136520 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bc990bdca0 0x55bc990b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bc990b50c0 0x55bc990c62e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8215ffb640 0 client.16311 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 --2- 172.21.15.122:0/1207926632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bc990b47a0 0x55bc990c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 >> 172.21.15.122:0/1207926632 conn(0x55bc990aeed0 msgr2=0x55bc990b9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 shutdown_connections 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 1 -- 172.21.15.122:0/1207926632 wait complete. 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f8236fe8740 20 client.16311 trim_cache size 0 max 16384 2024-04-02T13:36:55.326 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 msgr2=0x7f82180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 0x7f82180a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8224005fd0 tx=0x7f8224009500 comp rx=0 tx=0).stop 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 msgr2=0x7f822c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 0x7f822c1988f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f821c00e9f0 tx=0x7f821c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 shutdown_connections 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82180a3d20 0x7f82180a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c106980 0x7f822c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f822c101330 0x7f822c1988f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 --2- 172.21.15.122:0/254860467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f822c100970 0x7f822c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 >> 172.21.15.122:0/254860467 conn(0x7f822c077b00 msgr2=0x7f822c103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 shutdown_connections 2024-04-02T13:36:55.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.317+0000 7f823431c640 1 -- 172.21.15.122:0/254860467 wait complete. 2024-04-02T13:36:55.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 -- 172.21.15.122:0/3001503792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324106980 msgr2=0x7f5324106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 --2- 172.21.15.122:0/3001503792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324106980 0x7f5324106d60 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f5314009930 tx=0x7f531402bc30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 -- 172.21.15.122:0/3001503792 shutdown_connections 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 --2- 172.21.15.122:0/3001503792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324101330 0x7f532410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 --2- 172.21.15.122:0/3001503792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324100970 0x7f5324100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 --2- 172.21.15.122:0/3001503792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324106980 0x7f5324106d60 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 -- 172.21.15.122:0/3001503792 >> 172.21.15.122:0/3001503792 conn(0x7f5324077b00 msgr2=0x7f5324077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 -- 172.21.15.122:0/3001503792 shutdown_connections 2024-04-02T13:36:55.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 -- 172.21.15.122:0/3001503792 wait complete. 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.440+0000 7f532a39d640 1 Processor -- start 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 -- start start 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324101330 0x7f5324198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324106980 0x7f532419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5324110d90 con 0x7f5324100970 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5324110c10 con 0x7f5324101330 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532a39d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5324110f10 con 0x7f5324106980 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35924/0 (socket says 172.21.15.122:35924) 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 -- 172.21.15.122:0/623075347 learned_addr learned my addr 172.21.15.122:0/623075347 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f532893a640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324106980 0x7f532419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f53237fe640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324101330 0x7f5324198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 -- 172.21.15.122:0/623075347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324106980 msgr2=0x7f532419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324106980 0x7f532419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 -- 172.21.15.122:0/623075347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324101330 msgr2=0x7f5324198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324101330 0x7f5324198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.441+0000 7f5323fff640 1 -- 172.21.15.122:0/623075347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532419d3d0 con 0x7f5324100970 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f5323fff640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5314009230 tx=0x7f531402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f53217fa640 1 -- 172.21.15.122:0/623075347 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314041070 con 0x7f5324100970 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f53217fa640 1 -- 172.21.15.122:0/623075347 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5314004450 con 0x7f5324100970 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532419d660 con 0x7f5324100970 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f53217fa640 1 -- 172.21.15.122:0/623075347 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314003e50 con 0x7f5324100970 2024-04-02T13:36:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.442+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53241a4f50 con 0x7f5324100970 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f53217fa640 1 -- 172.21.15.122:0/623075347 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5314044070 con 0x7f5324100970 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 Processor -- start 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 -- start start 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5605905467b0 con 0x5605905ebca0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5605904e5570 con 0x5605905e30c0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532cf88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560590542760 con 0x5605905e27a0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f53237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532893a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f5323fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f53217fa640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 0x7f530c0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532893a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42326/0 (socket says 172.21.15.122:42326) 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f53237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35932/0 (socket says 172.21.15.122:35932) 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.445+0000 7f532893a640 1 -- 172.21.15.122:0/1866770188 learned_addr learned my addr 172.21.15.122:0/1866770188 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f532893a640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 0x7f530c0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f53217fa640 1 -- 172.21.15.122:0/623075347 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f53140f22b0 con 0x7f5324100970 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 -- 172.21.15.122:0/1866770188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 msgr2=0x5605905e2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 -- 172.21.15.122:0/1866770188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 msgr2=0x5605905ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 -- 172.21.15.122:0/1866770188 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059025a3f0 con 0x5605905e30c0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f5323fff640 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53140ef780 tx=0x7f5314000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7045c3e9276c97c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f532893a640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 0x7f530c0a6400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531c0049d0 tx=0x7f531c004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f530a7fc640 1 -- 172.21.15.122:0/1866770188 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531404b070 con 0x5605905e30c0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f530a7fc640 1 -- 172.21.15.122:0/1866770188 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5314004c80 con 0x5605905e30c0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.446+0000 7f530a7fc640 1 -- 172.21.15.122:0/1866770188 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140ef8f0 con 0x5605905e30c0 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- 172.21.15.122:0/1866770188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 msgr2=0x5605905eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb760 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f53140ef780 tx=0x7f5314000bc0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- 172.21.15.122:0/1866770188 shutdown_connections 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb760 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 --2- 172.21.15.122:0/1866770188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- 172.21.15.122:0/1866770188 >> 172.21.15.122:0/1866770188 conn(0x5605905dced0 msgr2=0x5605905dd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- 172.21.15.122:0/1866770188 shutdown_connections 2024-04-02T13:36:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- 172.21.15.122:0/1866770188 wait complete. 2024-04-02T13:36:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 -- start start 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.447+0000 7f532cf88740 1 Processor -- start 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 0x5605905eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 0x5605905e8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5605904e5570 con 0x5605905ebca0 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560590501e80 con 0x5605905e27a0 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5605905ee850 con 0x5605905e30c0 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532cf88740 10 client.0 _ll_register_callbacks cb 0x56059053daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f5323fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 0x5605905eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35936/0 (socket says 172.21.15.122:35936) 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 -- 172.21.15.122:0/1839560496 learned_addr learned my addr 172.21.15.122:0/1839560496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f53237fe640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 0x5605905e8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 -- 172.21.15.122:0/1839560496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 msgr2=0x5605905e8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 0x5605905e8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 -- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 msgr2=0x5605905eb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 0x5605905eb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 -- 172.21.15.122:0/1839560496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605903247d0 con 0x5605905ebca0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f5323fff640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 0x5605905eb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f53237fe640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 0x5605905e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f532893a640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f531c012040 tx=0x7f531c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce3e38db7554696 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.448+0000 7f53037fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c00fe60 con 0x5605905ebca0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f531c004d20 con 0x5605905ebca0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5314004c80 con 0x5605905ebca0 2024-04-02T13:36:55.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f531c032050 con 0x5605905ebca0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 10 client.16323 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605905fa7c0 con 0x5605905ebca0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5314004c80 con 0x5605905ebca0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f531c01f040 con 0x5605905ebca0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 10 client.16323 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 20 client.16323 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605905fbd20 0x5605905fe1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 4 client.16323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560590553cf0 con 0x5605905fbd20 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.449+0000 7f5323fff640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605905fbd20 0x5605905fe1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.450+0000 7f53037fe640 10 client.16323 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.450+0000 7f5323fff640 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605905fbd20 0x5605905fe1a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d171e0fc510fa5d1 server_cookie=7d6dec9701d9718d in_seq=0 out_seq=0 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f53037fe640 1 -- 172.21.15.122:0/1839560496 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560590553cf0 con 0x5605905fbd20 2024-04-02T13:36:55.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f53037fe640 10 client.16323 handle_command_reply: tid=0 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 client.16323 shutdown 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 msgr2=0x5605905e91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f531c012040 tx=0x7f531c004a40 comp rx=0 tx=0).stop 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 shutdown_connections 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605905fbd20 0x5605905fe1a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e91b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e30c0 0x5605905e8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 --2- 172.21.15.122:0/1839560496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e27a0 0x5605905eb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 >> 172.21.15.122:0/1839560496 conn(0x5605905dced0 msgr2=0x5605905dec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f53037fe640 0 client.16323 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 shutdown_connections 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 1 -- 172.21.15.122:0/1839560496 wait complete. 2024-04-02T13:36:55.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.451+0000 7f532cf88740 20 client.16323 trim_cache size 0 max 16384 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 Processor -- start 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 -- start start 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905ebca0 0x5605905eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53241af080 con 0x5605905e27a0 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53241b0bc0 con 0x5605905e30c0 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532cf88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53241af750 con 0x5605905ebca0 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35948/0 (socket says 172.21.15.122:35948) 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 -- 172.21.15.122:0/2121924046 learned_addr learned my addr 172.21.15.122:0/2121924046 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f53237fe640 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f532893a640 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905ebca0 0x5605905eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 -- 172.21.15.122:0/2121924046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905ebca0 msgr2=0x5605905eb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905ebca0 0x5605905eb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 -- 172.21.15.122:0/2121924046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 msgr2=0x5605905eb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 -- 172.21.15.122:0/2121924046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605905e2ef0 con 0x5605905e27a0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.466+0000 7f5323fff640 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f53140be010 tx=0x7f5314005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87830b0d3ac772b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f5309ffb640 1 -- 172.21.15.122:0/2121924046 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531404b040 con 0x5605905e27a0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f5309ffb640 1 -- 172.21.15.122:0/2121924046 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5314004c80 con 0x5605905e27a0 2024-04-02T13:36:55.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f5309ffb640 1 -- 172.21.15.122:0/2121924046 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140efa50 con 0x5605905e27a0 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 -- 172.21.15.122:0/2121924046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 msgr2=0x5605905f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f53140be010 tx=0x7f5314005be0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 -- 172.21.15.122:0/2121924046 shutdown_connections 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905ebca0 0x5605905eb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905eb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 --2- 172.21.15.122:0/2121924046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905e27a0 0x5605905f7590 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.467+0000 7f532cf88740 1 -- 172.21.15.122:0/2121924046 >> 172.21.15.122:0/2121924046 conn(0x5605905dced0 msgr2=0x5605905e3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- 172.21.15.122:0/2121924046 shutdown_connections 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- 172.21.15.122:0/2121924046 wait complete. 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- start start 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 Processor -- start 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905e5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53241b04f0 con 0x5605905ebca0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53241afe20 con 0x5605905e30c0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53241ad060 con 0x5605905e27a0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532cf88740 10 client.0 _ll_register_callbacks cb 0x7f531c0135a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532893a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f53237fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f5323fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532893a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35960/0 (socket says 172.21.15.122:35960) 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.468+0000 7f532893a640 1 -- 172.21.15.122:0/4021687118 learned_addr learned my addr 172.21.15.122:0/4021687118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 -- 172.21.15.122:0/4021687118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 msgr2=0x5605905e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 -- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 msgr2=0x5605905e5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905e5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 -- 172.21.15.122:0/4021687118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605905f2aa0 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532893a640 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f531c005c40 tx=0x7f531c009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a9caf033eed2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f53037fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c036630 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f531c0367d0 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5314004c80 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f531c046050 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532cf88740 10 client.16329 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5605905f3d70 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.469+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5314004c80 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f531c021450 con 0x5605905ebca0 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 10 client.16329 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:55.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 20 client.16329 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605906622e0 0x5605906646d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 4 client.16329 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56059054e040 con 0x5605906622e0 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f5323fff640 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605906622e0 0x5605906646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f53037fe640 10 client.16329 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f5323fff640 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605906622e0 0x5605906646d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81010890d1e404e server_cookie=fda59e1211c6ff2a in_seq=0 out_seq=0 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f53037fe640 1 -- 172.21.15.122:0/4021687118 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56059054e040 con 0x5605906622e0 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f53037fe640 10 client.16329 handle_command_reply: tid=0 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.470+0000 7f532cf88740 1 client.16329 shutdown 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 msgr2=0x5605905e57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f531c005c40 tx=0x7f531c009ac0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 shutdown_connections 2024-04-02T13:36:55.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5605906622e0 0x5605906646d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5605905ebca0 0x5605905e57d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5605905e30c0 0x5605905e5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 --2- 172.21.15.122:0/4021687118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5605905e27a0 0x5605905e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f53037fe640 0 client.16329 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 >> 172.21.15.122:0/4021687118 conn(0x5605905dced0 msgr2=0x5605905e73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 shutdown_connections 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 1 -- 172.21.15.122:0/4021687118 wait complete. 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532cf88740 20 client.16329 trim_cache size 0 max 16384 2024-04-02T13:36:55.480 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 msgr2=0x7f530c0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 0x7f530c0a6400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f531c0049d0 tx=0x7f531c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 msgr2=0x7f53241983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f5314009230 tx=0x7f531402cb30 comp rx=0 tx=0).stop 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 shutdown_connections 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f530c0a3f40 0x7f530c0a6400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.471+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5324106980 0x7f532419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.472+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5324101330 0x7f5324198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.472+0000 7f532a39d640 1 --2- 172.21.15.122:0/623075347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5324100970 0x7f53241983e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.472+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 >> 172.21.15.122:0/623075347 conn(0x7f5324077b00 msgr2=0x7f5324069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.472+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 shutdown_connections 2024-04-02T13:36:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:55.472+0000 7f532a39d640 1 -- 172.21.15.122:0/623075347 wait complete. 2024-04-02T13:36:55.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 43217 15.65 MB/sec warmup 16 sec latency 7.136 ms 2024-04-02T13:36:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:55 smithi187 ceph-mon[36064]: pgmap v55: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:36:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:55 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3192325567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:56.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:55 smithi188 ceph-mon[36117]: pgmap v55: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:36:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:55 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3192325567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:56.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:55 smithi122 ceph-mon[32114]: pgmap v55: 129 pgs: 129 active+clean; 16 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:36:56.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:55 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3192325567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:36:56.486 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:36:56.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:36:56.487 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:36:56.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- 172.21.15.122:0/587814685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4102460 msgr2=0x7f9ef410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/587814685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4102460 0x7f9ef410e970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009ca0 tx=0x7f9eec02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- 172.21.15.122:0/587814685 shutdown_connections 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/587814685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4102460 0x7f9ef410e970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/587814685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/587814685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- 172.21.15.122:0/587814685 >> 172.21.15.122:0/587814685 conn(0x7f9ef40fd590 msgr2=0x7f9ef40ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- 172.21.15.122:0/587814685 shutdown_connections 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- 172.21.15.122:0/587814685 wait complete. 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 Processor -- start 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.597+0000 7f9efb6c5640 1 -- start start 2024-04-02T13:36:56.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4102460 0x7f9ef4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4111e90 con 0x7f9ef4107ab0 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4111d10 con 0x7f9ef4101aa0 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4112010 con 0x7f9ef4102460 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35980/0 (socket says 172.21.15.122:35980) 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 -- 172.21.15.122:0/1872439227 learned_addr learned my addr 172.21.15.122:0/1872439227 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4102460 0x7f9ef4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 -- 172.21.15.122:0/1872439227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4102460 msgr2=0x7f9ef4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4102460 0x7f9ef4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 -- 172.21.15.122:0/1872439227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 msgr2=0x7f9ef4073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 -- 172.21.15.122:0/1872439227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef41a6850 con 0x7f9ef4107ab0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9461640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9ef9c62640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9eec02c700 tx=0x7f9eec032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9eea7fc640 1 -- 172.21.15.122:0/1872439227 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec042020 con 0x7f9ef4107ab0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef41a6ae0 con 0x7f9ef4107ab0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9eea7fc640 1 -- 172.21.15.122:0/1872439227 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eec034640 con 0x7f9ef4107ab0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9eea7fc640 1 -- 172.21.15.122:0/1872439227 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec039040 con 0x7f9ef4107ab0 2024-04-02T13:36:56.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.598+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef41a7020 con 0x7f9ef4107ab0 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.601+0000 7f9eea7fc640 1 -- 172.21.15.122:0/1872439227 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9eec045020 con 0x7f9ef4107ab0 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 Processor -- start 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 -- start start 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9eea7fc640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 0x7f9ee00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a14e7c0 0x55b41a150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b41a0a9770 con 0x55b41a14e7c0 2024-04-02T13:36:56.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b41a051960 con 0x55b41a1452c0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9efe387740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b41a0a58c0 con 0x55b41a145be0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48888/0 (socket says 172.21.15.122:48888) 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2350181000 learned_addr learned my addr 172.21.15.122:0/2350181000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef9c62640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a14e7c0 0x55b41a150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 0x7f9ee00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9eea7fc640 1 -- 172.21.15.122:0/1872439227 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9eec0f2960 con 0x7f9ef4107ab0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2350181000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 msgr2=0x55b41a14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2350181000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a14e7c0 msgr2=0x55b41a150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a14e7c0 0x55b41a150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.602+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2350181000 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b419e857e0 con 0x55b41a1452c0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.603+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40058c0 tx=0x7f9ee4011f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c584e6081a6515 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.603+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 0x7f9ee00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4011fd0 tx=0x7f9ee4018000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.603+0000 7f9edf7fe640 1 -- 172.21.15.122:0/2350181000 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4011050 con 0x55b41a1452c0 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.603+0000 7f9edf7fe640 1 -- 172.21.15.122:0/2350181000 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4009a20 con 0x55b41a1452c0 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.603+0000 7f9edf7fe640 1 -- 172.21.15.122:0/2350181000 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40035f0 con 0x55b41a1452c0 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- 172.21.15.122:0/2350181000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 msgr2=0x55b41a1456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40058c0 tx=0x7f9ee4011f60 comp rx=0 tx=0).stop 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- 172.21.15.122:0/2350181000 shutdown_connections 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a14e7c0 0x55b41a150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- 172.21.15.122:0/2350181000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a1452c0 0x55b41a1456a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- 172.21.15.122:0/2350181000 >> 172.21.15.122:0/2350181000 conn(0x55b41a13f9f0 msgr2=0x55b41a13fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- 172.21.15.122:0/2350181000 shutdown_connections 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- 172.21.15.122:0/2350181000 wait complete. 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- start start 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 Processor -- start 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.604+0000 7f9efe387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b41a151550 con 0x55b41a1452c0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b41a064d20 con 0x55b41a14e7c0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b41a1516d0 con 0x55b41a145be0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 10 client.0 _ll_register_callbacks cb 0x55b41a031a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35988/0 (socket says 172.21.15.122:35988) 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 -- 172.21.15.122:0/3027923698 learned_addr learned my addr 172.21.15.122:0/3027923698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9c62640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 -- 172.21.15.122:0/3027923698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 msgr2=0x55b41a14bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 -- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 msgr2=0x55b41a14c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a14c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 -- 172.21.15.122:0/3027923698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b419ee5f70 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9ef9461640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0012d20 tx=0x7f9ef000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e65ee05924a36f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0012110 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9ef0004050 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee4009a20 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ef00122b0 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 10 client.16341 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.605+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b419d7d170 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee4009a20 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ef00104c0 con 0x55b41a1452c0 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 10 client.16341 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:56.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 20 client.16341 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a15ebd0 0x55b41a161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 4 client.16341 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b41a0b6cb0 con 0x55b41a15ebd0 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a15ebd0 0x55b41a161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9edcff9640 10 client.16341 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.606+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a15ebd0 0x55b41a161020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525eaef8c2e95a11 server_cookie=4a6b52356e1876e6 in_seq=0 out_seq=0 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9edcff9640 1 -- 172.21.15.122:0/3027923698 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b41a0b6cb0 con 0x55b41a15ebd0 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9edcff9640 10 client.16341 handle_command_reply: tid=0 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 client.16341 shutdown 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 msgr2=0x55b41a14b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0012d20 tx=0x7f9ef000f790 comp rx=0 tx=0).stop 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 shutdown_connections 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a15ebd0 0x55b41a161020 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a14c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9edcff9640 0 client.16341 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a145be0 0x55b41a14bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 --2- 172.21.15.122:0/3027923698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a14b860 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.607+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 >> 172.21.15.122:0/3027923698 conn(0x55b41a13f9f0 msgr2=0x55b41a141740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.608+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 shutdown_connections 2024-04-02T13:36:56.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.608+0000 7f9efe387740 1 -- 172.21.15.122:0/3027923698 wait complete. 2024-04-02T13:36:56.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.608+0000 7f9efe387740 20 client.16341 trim_cache size 0 max 16384 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 Processor -- start 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 -- start start 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 0x55b41a14df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a1668f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b1150 con 0x55b41a145be0 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b2c90 con 0x55b41a14e7c0 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9efe387740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b1820 con 0x55b41a1452c0 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36000/0 (socket says 172.21.15.122:36000) 2024-04-02T13:36:56.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 0x55b41a14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2308945770 learned_addr learned my addr 172.21.15.122:0/2308945770 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef9c62640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a1668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2308945770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 msgr2=0x55b41a14df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 0x55b41a14df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2308945770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 msgr2=0x55b41a1668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a1668f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 -- 172.21.15.122:0/2308945770 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41a1459e0 con 0x55b41a145be0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef9461640 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 0x55b41a14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.628+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40058c0 tx=0x7f9ee4009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a716b2c103326 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9edeffd640 1 -- 172.21.15.122:0/2308945770 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40099c0 con 0x55b41a145be0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9edeffd640 1 -- 172.21.15.122:0/2308945770 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee40318b0 con 0x55b41a145be0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9edeffd640 1 -- 172.21.15.122:0/2308945770 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4005310 con 0x55b41a145be0 2024-04-02T13:36:56.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 -- 172.21.15.122:0/2308945770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 msgr2=0x55b41a15a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40058c0 tx=0x7f9ee4009870 comp rx=0 tx=0).stop 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 -- 172.21.15.122:0/2308945770 shutdown_connections 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a14e7c0 0x55b41a1668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a145be0 0x55b41a15a440 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 --2- 172.21.15.122:0/2308945770 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a1452c0 0x55b41a14df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 -- 172.21.15.122:0/2308945770 >> 172.21.15.122:0/2308945770 conn(0x55b41a13f9f0 msgr2=0x55b41a1416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 -- 172.21.15.122:0/2308945770 shutdown_connections 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.629+0000 7f9efe387740 1 -- 172.21.15.122:0/2308945770 wait complete. 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 -- start start 2024-04-02T13:36:56.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 Processor -- start 2024-04-02T13:36:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 0x55b41a152e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a14e7c0 0x55b41a1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b25c0 con 0x55b41a1452c0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b1ef0 con 0x55b41a145be0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41af130 con 0x55b41a14e7c0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9efe387740 10 client.0 _ll_register_callbacks cb 0x7f9ef0009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9ef8c60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 0x55b41a152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9ef9461640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36010/0 (socket says 172.21.15.122:36010) 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9ef9461640 1 -- 172.21.15.122:0/1918206644 learned_addr learned my addr 172.21.15.122:0/1918206644 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.630+0000 7f9ef9c62640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a14e7c0 0x55b41a1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 -- 172.21.15.122:0/1918206644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a14e7c0 msgr2=0x55b41a1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a14e7c0 0x55b41a1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 -- 172.21.15.122:0/1918206644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 msgr2=0x55b41a156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 -- 172.21.15.122:0/1918206644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41a153910 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef9461640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9ef8c60640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 0x55b41a152e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55b41a131140 tx=0x7f9ee400ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab7b13660c838a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4035070 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9ee40115e0 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee40318b0 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ee4009940 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9efe387740 10 client.25829 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b41a1c5f00 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.631+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee40318b0 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f9ee4003860 con 0x55b41a145be0 2024-04-02T13:36:56.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 10 client.25829 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 10 client.25829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 20 client.25829 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a1c6420 0x55b41a1c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 4 client.25829 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b41a0b1000 con 0x55b41a1c6420 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9ef9461640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a1c6420 0x55b41a1c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9edcff9640 10 client.25829 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.632+0000 7f9ef9461640 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a1c6420 0x55b41a1c8810 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1939bab3d61ff3 server_cookie=be1dc29daf33ebe0 in_seq=0 out_seq=0 2024-04-02T13:36:56.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.634+0000 7f9edcff9640 1 -- 172.21.15.122:0/1918206644 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b41a0b1000 con 0x55b41a1c6420 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.634+0000 7f9edcff9640 10 client.25829 handle_command_reply: tid=0 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.634+0000 7f9efe387740 1 client.25829 shutdown 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.634+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 msgr2=0x55b41a152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.634+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 0x55b41a152e90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55b41a131140 tx=0x7f9ee400ef30 comp rx=0 tx=0).stop 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 shutdown_connections 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b41a1c6420 0x55b41a1c8810 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b41a14e7c0 0x55b41a1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b41a145be0 0x55b41a152e90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 --2- 172.21.15.122:0/1918206644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b41a1452c0 0x55b41a156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 >> 172.21.15.122:0/1918206644 conn(0x55b41a13f9f0 msgr2=0x55b41a14b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9edcff9640 0 client.25829 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 shutdown_connections 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 1 -- 172.21.15.122:0/1918206644 wait complete. 2024-04-02T13:36:56.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efe387740 20 client.25829 trim_cache size 0 max 16384 2024-04-02T13:36:56.644 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:56.644 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:36:56.644 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "3f982e1c-f669-4eca-a4be-6e5b6c821ef1", 2024-04-02T13:36:56.644 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:36:56.644 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 msgr2=0x7f9ee00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 0x7f9ee00a6260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4011fd0 tx=0x7f9ee4018000 comp rx=0 tx=0).stop 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 msgr2=0x7f9ef4074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9eec02c700 tx=0x7f9eec032f20 comp rx=0 tx=0).stop 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 shutdown_connections 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ee00a3da0 0x7f9ee00a6260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ef4107ab0 0x7f9ef4074880 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ef4102460 0x7f9ef4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 --2- 172.21.15.122:0/1872439227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ef4101aa0 0x7f9ef4073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.635+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 >> 172.21.15.122:0/1872439227 conn(0x7f9ef40fd590 msgr2=0x7f9ef40fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.636+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 shutdown_connections 2024-04-02T13:36:56.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.636+0000 7f9efb6c5640 1 -- 172.21.15.122:0/1872439227 wait complete. 2024-04-02T13:36:56.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:36:56.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:36:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- 172.21.15.122:0/3735594565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b00696a0 msgr2=0x7f07b010e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3735594565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b00696a0 0x7f07b010e940 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009cd0 tx=0x7f07a802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- 172.21.15.122:0/3735594565 shutdown_connections 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3735594565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b00696a0 0x7f07b010e940 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3735594565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3735594565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b0107a80 0x7f07b0107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- 172.21.15.122:0/3735594565 >> 172.21.15.122:0/3735594565 conn(0x7f07b0077b00 msgr2=0x7f07b0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- 172.21.15.122:0/3735594565 shutdown_connections 2024-04-02T13:36:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- 172.21.15.122:0/3735594565 wait complete. 2024-04-02T13:36:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 Processor -- start 2024-04-02T13:36:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- start start 2024-04-02T13:36:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b00696a0 0x7f07b01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07b0111e90 con 0x7f07b0107a80 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07b0111d10 con 0x7f07b0068ce0 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.759+0000 7f07b81e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b0112010 con 0x7f07b00696a0 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b677f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:48944/0 (socket says 172.21.15.122:48944) 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b677f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36026/0 (socket says 172.21.15.122:36026) 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 -- 172.21.15.122:0/3457500277 learned_addr learned my addr 172.21.15.122:0/3457500277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b577d640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b00696a0 0x7f07b01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 -- 172.21.15.122:0/3457500277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b00696a0 msgr2=0x7f07b01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b00696a0 0x7f07b01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 -- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 msgr2=0x7f07b019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 -- 172.21.15.122:0/3457500277 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b019d360 con 0x7f07b0068ce0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b677f640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00e9f0 tx=0x7f07ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07a6ffd640 1 -- 172.21.15.122:0/3457500277 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00cd80 con 0x7f07b0068ce0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b019d650 con 0x7f07b0068ce0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07a6ffd640 1 -- 172.21.15.122:0/3457500277 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac005500 con 0x7f07b0068ce0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b01a4ee0 con 0x7f07b0068ce0 2024-04-02T13:36:56.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.760+0000 7f07a6ffd640 1 -- 172.21.15.122:0/3457500277 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac010690 con 0x7f07b0068ce0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 Processor -- start 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 -- start start 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07a6ffd640 1 -- 172.21.15.122:0/3457500277 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f07ac0046c0 con 0x7f07b0068ce0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8ef7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 0x5645a8f00890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645a8e5b830 con 0x5645a8f00dd0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5645a8e03820 con 0x5645a8ef78d0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07badc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645a8e577e0 con 0x5645a8ef81f0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07b5f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 0x5645a8f00890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36030/0 (socket says 172.21.15.122:36030) 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07b677f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8ef7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.763+0000 7f07b577d640 1 -- 172.21.15.122:0/2412182220 learned_addr learned my addr 172.21.15.122:0/2412182220 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 -- 172.21.15.122:0/2412182220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 msgr2=0x5645a8ef7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8ef7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 -- 172.21.15.122:0/2412182220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 msgr2=0x5645a8f031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 -- 172.21.15.122:0/2412182220 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a8c397d0 con 0x5645a8ef81f0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b577d640 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07a6ffd640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 0x7f079c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 0x5645a8f00890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0ce2a0 tx=0x7f07ac0d4f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fefe6445e2def4d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b677f640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 0x7f079c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07a6ffd640 1 -- 172.21.15.122:0/3457500277 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f07ac0d0ee0 con 0x7f07b0068ce0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f079bfff640 1 -- 172.21.15.122:0/2412182220 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac098d90 con 0x5645a8ef81f0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f07b677f640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 0x7f079c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005cc0 tx=0x7f07a8005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f079bfff640 1 -- 172.21.15.122:0/2412182220 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac09d1f0 con 0x5645a8ef81f0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.764+0000 7f079bfff640 1 -- 172.21.15.122:0/2412182220 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac09c450 con 0x5645a8ef81f0 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- 172.21.15.122:0/2412182220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 msgr2=0x5645a8f00890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 0x5645a8f00890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0ce2a0 tx=0x7f07ac0d4f90 comp rx=0 tx=0).stop 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- 172.21.15.122:0/2412182220 shutdown_connections 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8f00dd0 0x5645a8f031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef81f0 0x5645a8f00890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 --2- 172.21.15.122:0/2412182220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8ef7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- 172.21.15.122:0/2412182220 >> 172.21.15.122:0/2412182220 conn(0x5645a8ef2000 msgr2=0x5645a8ef23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- 172.21.15.122:0/2412182220 shutdown_connections 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- 172.21.15.122:0/2412182220 wait complete. 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.765+0000 7f07badc8740 1 -- start start 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 Processor -- start 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 0x5645a8f00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 0x5645a8efe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645a8f03bc0 con 0x5645a8ef81f0 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5645a8e16d30 con 0x5645a8f00dd0 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645a8f03d40 con 0x5645a8ef78d0 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07badc8740 10 client.0 _ll_register_callbacks cb 0x5645a8e16720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36034/0 (socket says 172.21.15.122:36034) 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b5f7e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 0x5645a8f00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 -- 172.21.15.122:0/4204249033 learned_addr learned my addr 172.21.15.122:0/4204249033 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b677f640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 0x5645a8efe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 -- 172.21.15.122:0/4204249033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 msgr2=0x5645a8f00720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 0x5645a8f00720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.766+0000 7f07b577d640 1 -- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 msgr2=0x5645a8efe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07b577d640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 0x5645a8efe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07b577d640 1 -- 172.21.15.122:0/4204249033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a8b6f3f0 con 0x5645a8ef81f0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 0x5645a8f00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07b677f640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 0x5645a8efe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07b577d640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f07a0012740 tx=0x7f07a0012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71e039baa895ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a000f920 con 0x5645a8ef81f0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f07a000fac0 con 0x5645a8ef81f0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07ac09d1f0 con 0x5645a8ef81f0 2024-04-02T13:36:56.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f07a000fd00 con 0x5645a8ef81f0 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 10 client.16359 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645a8b2f0f0 con 0x5645a8ef81f0 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac09d1f0 con 0x5645a8ef81f0 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f07a001c540 con 0x5645a8ef81f0 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 10 client.16359 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 10 client.16359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 20 client.16359 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f10e80 0x5645a8f132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 4 client.16359 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.767+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645a8e68d70 con 0x5645a8f10e80 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.768+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f10e80 0x5645a8f132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.768+0000 7f07997fa640 10 client.16359 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.768+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f10e80 0x5645a8f132a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329dfcc1fa030d4c server_cookie=9773f48e9dac0396 in_seq=0 out_seq=0 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07997fa640 1 -- 172.21.15.122:0/4204249033 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645a8e68d70 con 0x5645a8f10e80 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07997fa640 10 client.16359 handle_command_reply: tid=0 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 client.16359 shutdown 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 msgr2=0x5645a8efdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f07a0012740 tx=0x7f07a0012c00 comp rx=0 tx=0).stop 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 shutdown_connections 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f10e80 0x5645a8f132a0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8f00dd0 0x5645a8efe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8efdab0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07997fa640 0 client.16359 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 --2- 172.21.15.122:0/4204249033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8ef78d0 0x5645a8f00720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 >> 172.21.15.122:0/4204249033 conn(0x5645a8ef2000 msgr2=0x5645a8ef3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 shutdown_connections 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 1 -- 172.21.15.122:0/4204249033 wait complete. 2024-04-02T13:36:56.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.769+0000 7f07badc8740 20 client.16359 trim_cache size 0 max 16384 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.789+0000 7f07badc8740 1 Processor -- start 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.789+0000 7f07badc8740 1 -- start start 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f0ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f001e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01af000 con 0x5645a8ef81f0 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01b0b40 con 0x5645a8ef78d0 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07badc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01af6d0 con 0x5645a8f00dd0 2024-04-02T13:36:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b5f7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f0ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36048/0 (socket says 172.21.15.122:36048) 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 -- 172.21.15.122:0/2364074590 learned_addr learned my addr 172.21.15.122:0/2364074590 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b677f640 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 -- 172.21.15.122:0/2364074590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 msgr2=0x5645a8f001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 -- 172.21.15.122:0/2364074590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 msgr2=0x5645a8f0ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f0ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 -- 172.21.15.122:0/2364074590 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645a8ef8020 con 0x5645a8ef81f0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f0ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f07b577d640 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000c00 tx=0x7f07a000fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eae103890a1a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f079b7fe640 1 -- 172.21.15.122:0/2364074590 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a000fea0 con 0x5645a8ef81f0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f079b7fe640 1 -- 172.21.15.122:0/2364074590 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a0012e50 con 0x5645a8ef81f0 2024-04-02T13:36:56.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.790+0000 7f079b7fe640 1 -- 172.21.15.122:0/2364074590 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00048e0 con 0x5645a8ef81f0 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- 172.21.15.122:0/2364074590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 msgr2=0x5645a8ef09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f07a0000c00 tx=0x7f07a000fbb0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- 172.21.15.122:0/2364074590 shutdown_connections 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8ef09b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 --2- 172.21.15.122:0/2364074590 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f0ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- 172.21.15.122:0/2364074590 >> 172.21.15.122:0/2364074590 conn(0x5645a8ef2000 msgr2=0x5645a8ef90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- 172.21.15.122:0/2364074590 shutdown_connections 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- 172.21.15.122:0/2364074590 wait complete. 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 -- start start 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.791+0000 7f07badc8740 1 Processor -- start 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f093c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f04c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01b0470 con 0x5645a8ef81f0 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01afda0 con 0x5645a8ef78d0 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b01acfe0 con 0x5645a8f00dd0 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07badc8740 10 client.0 _ll_register_callbacks cb 0x7f07a0009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36060/0 (socket says 172.21.15.122:36060) 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 -- 172.21.15.122:0/1629575959 learned_addr learned my addr 172.21.15.122:0/1629575959 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:36:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f093c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b677f640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f04c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 -- 172.21.15.122:0/1629575959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 msgr2=0x5645a8f04c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f04c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 -- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 msgr2=0x5645a8f093c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f093c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b577d640 1 -- 172.21.15.122:0/1629575959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645a8f054c0 con 0x5645a8ef81f0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.792+0000 7f07b677f640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07b577d640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f07a000a010 tx=0x7f07a0010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8bc57f1077b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a002f040 con 0x5645a8ef81f0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a0012e50 con 0x5645a8ef81f0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f07a0018070 con 0x5645a8ef81f0 2024-04-02T13:36:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f07a0040050 con 0x5645a8ef81f0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 10 client.16365 fetch_fsmap learned FSMap version 28 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645a8f774d0 con 0x5645a8ef81f0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07a0012e50 con 0x5645a8ef81f0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f07a0032070 con 0x5645a8ef81f0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 10 client.16365 fetch_fsmap finished waiting for FSMap version 28 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 20 client.16365 populate_metadata read hostname 'smithi122' 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f77ad0 0x5645a8f79ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 4 client.16365 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645a8e630c0 con 0x5645a8f77ad0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.793+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f77ad0 0x5645a8f79ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07997fa640 10 client.16365 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07b5f7e640 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f77ad0 0x5645a8f79ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb0d46025cad57e0 server_cookie=7558fd93f57dc965 in_seq=0 out_seq=0 2024-04-02T13:36:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07997fa640 1 -- 172.21.15.122:0/1629575959 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5645a8e630c0 con 0x5645a8f77ad0 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07997fa640 10 client.16365 handle_command_reply: tid=0 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07badc8740 1 client.16365 shutdown 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 msgr2=0x5645a8f04730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.794+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f07a000a010 tx=0x7f07a0010660 comp rx=0 tx=0).stop 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 shutdown_connections 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5645a8f77ad0 0x5645a8f79ef0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645a8f00dd0 0x5645a8f04c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07997fa640 0 client.16365 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645a8ef81f0 0x5645a8f04730 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 --2- 172.21.15.122:0/1629575959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645a8ef78d0 0x5645a8f093c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 >> 172.21.15.122:0/1629575959 conn(0x5645a8ef2000 msgr2=0x5645a8efd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 shutdown_connections 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 1 -- 172.21.15.122:0/1629575959 wait complete. 2024-04-02T13:36:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07badc8740 20 client.16365 trim_cache size 0 max 16384 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "3f982e1c-f669-4eca-a4be-6e5b6c821ef1": { 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "3f982e1c-f669-4eca-a4be-6e5b6c821ef1", 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:36:56.805 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:36:56.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 msgr2=0x7f079c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 0x7f079c0a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07a8005cc0 tx=0x7f07a8005c50 comp rx=0 tx=0).stop 2024-04-02T13:36:56.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 msgr2=0x7f07b0198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00e9f0 tx=0x7f07ac00edf0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 shutdown_connections 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f079c0a3d20 0x7f079c0a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07b0107a80 0x7f07b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b00696a0 0x7f07b01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 --2- 172.21.15.122:0/3457500277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07b0068ce0 0x7f07b0198370 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 >> 172.21.15.122:0/3457500277 conn(0x7f07b0077b00 msgr2=0x7f07b00fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 shutdown_connections 2024-04-02T13:36:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:36:56.795+0000 7f07b81e2640 1 -- 172.21.15.122:0/3457500277 wait complete. 2024-04-02T13:36:56.809 INFO:tasks.cephfs.filesystem:scrub status for tag:3f982e1c-f669-4eca-a4be-6e5b6c821ef1 - {'path': '/', 'tag': '3f982e1c-f669-4eca-a4be-6e5b6c821ef1', 'options': 'recursive,force'} 2024-04-02T13:36:56.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 46400 15.84 MB/sec warmup 17 sec latency 12.559 ms 2024-04-02T13:36:57.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 49065 15.67 MB/sec warmup 18 sec latency 17.690 ms 2024-04-02T13:36:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:57 smithi187 ceph-mon[36064]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-04-02T13:36:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:57 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:36:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:57 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:36:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:57 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:57 smithi188 ceph-mon[36117]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-04-02T13:36:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:57 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:36:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:57 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:36:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:57 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:57 smithi122 ceph-mon[32114]: pgmap v56: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 2.0 MiB/s wr, 246 op/s 2024-04-02T13:36:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:57 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:36:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:57 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:36:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:57 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:36:58.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 51183 15.46 MB/sec warmup 19 sec latency 16.336 ms 2024-04-02T13:36:59.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 53913 15.40 MB/sec warmup 20 sec latency 14.326 ms 2024-04-02T13:36:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:36:59 smithi187 ceph-mon[36064]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:00.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:36:59 smithi188 ceph-mon[36117]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:00.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:36:59 smithi122 ceph-mon[32114]: pgmap v57: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:00.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 56490 15.32 MB/sec warmup 21 sec latency 48.506 ms 2024-04-02T13:37:01.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 57466 14.87 MB/sec warmup 22 sec latency 47.317 ms 2024-04-02T13:37:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:01 smithi187 ceph-mon[36064]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:01 smithi188 ceph-mon[36117]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:01 smithi122 ceph-mon[32114]: pgmap v58: 129 pgs: 129 active+clean; 21 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-02T13:37:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:02.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 60595 15.03 MB/sec warmup 23 sec latency 7.049 ms 2024-04-02T13:37:03.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 63088 14.88 MB/sec warmup 24 sec latency 17.599 ms 2024-04-02T13:37:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:03 smithi187 ceph-mon[36064]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-04-02T13:37:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:03 smithi188 ceph-mon[36117]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-04-02T13:37:04.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:03 smithi122 ceph-mon[32114]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 2.0 MiB/s wr, 264 op/s 2024-04-02T13:37:04.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 65322 14.81 MB/sec warmup 25 sec latency 16.771 ms 2024-04-02T13:37:05.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 67634 14.77 MB/sec warmup 26 sec latency 349.225 ms 2024-04-02T13:37:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:05 smithi187 ceph-mon[36064]: pgmap v60: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-04-02T13:37:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:05 smithi188 ceph-mon[36117]: pgmap v60: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-04-02T13:37:06.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:05 smithi122 ceph-mon[32114]: pgmap v60: 129 pgs: 129 active+clean; 28 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.3 MiB/s wr, 181 op/s 2024-04-02T13:37:06.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 68087 14.24 MB/sec warmup 27 sec latency 370.230 ms 2024-04-02T13:37:07.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 70318 14.15 MB/sec warmup 28 sec latency 72.136 ms 2024-04-02T13:37:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:07 smithi187 ceph-mon[36064]: pgmap v61: 129 pgs: 129 active+clean; 90 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-04-02T13:37:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:07 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:07 smithi188 ceph-mon[36117]: pgmap v61: 129 pgs: 129 active+clean; 90 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-04-02T13:37:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:07 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:07 smithi122 ceph-mon[32114]: pgmap v61: 129 pgs: 129 active+clean; 90 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 6.6 MiB/s wr, 326 op/s 2024-04-02T13:37:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:07 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:08.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71035 13.86 MB/sec warmup 29 sec latency 768.857 ms 2024-04-02T13:37:09.939 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71035 13.40 MB/sec warmup 30 sec latency 1768.929 ms 2024-04-02T13:37:09.995 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:09 smithi187 ceph-mon[36064]: pgmap v62: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:09 smithi188 ceph-mon[36117]: pgmap v62: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:09 smithi122 ceph-mon[32114]: pgmap v62: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:10.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71035 12.97 MB/sec warmup 31 sec latency 2769.018 ms 2024-04-02T13:37:11.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71035 12.56 MB/sec warmup 32 sec latency 3769.109 ms 2024-04-02T13:37:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:11 smithi187 ceph-mon[36064]: pgmap v63: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:11 smithi188 ceph-mon[36117]: pgmap v63: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:11 smithi122 ceph-mon[32114]: pgmap v63: 129 pgs: 129 active+clean; 103 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 7.1 MiB/s wr, 268 op/s 2024-04-02T13:37:12.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71051 12.18 MB/sec warmup 33 sec latency 4468.429 ms 2024-04-02T13:37:12.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:12 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:12 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:13.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:12 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:13.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71051 11.82 MB/sec warmup 34 sec latency 1284.025 ms 2024-04-02T13:37:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:13 smithi187 ceph-mon[36064]: pgmap v64: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 383 op/s 2024-04-02T13:37:14.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:13 smithi188 ceph-mon[36117]: pgmap v64: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 383 op/s 2024-04-02T13:37:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:13 smithi122 ceph-mon[32114]: pgmap v64: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.8 MiB/s wr, 383 op/s 2024-04-02T13:37:14.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71051 11.48 MB/sec warmup 35 sec latency 2284.085 ms 2024-04-02T13:37:15.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71051 11.17 MB/sec warmup 36 sec latency 3284.173 ms 2024-04-02T13:37:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:15 smithi187 ceph-mon[36064]: pgmap v65: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-04-02T13:37:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:15 smithi188 ceph-mon[36117]: pgmap v65: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-04-02T13:37:16.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:15 smithi122 ceph-mon[32114]: pgmap v65: 129 pgs: 129 active+clean; 49 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-04-02T13:37:16.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71051 10.86 MB/sec warmup 37 sec latency 4284.236 ms 2024-04-02T13:37:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:16 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:16 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:37:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:16 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:17.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71337 10.62 MB/sec warmup 38 sec latency 4980.839 ms 2024-04-02T13:37:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:17 smithi187 ceph-mon[36064]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.1 MiB/s wr, 289 op/s 2024-04-02T13:37:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:17 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:17 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:17 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:17 smithi188 ceph-mon[36117]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.1 MiB/s wr, 289 op/s 2024-04-02T13:37:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:17 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:17 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:17 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:17 smithi122 ceph-mon[32114]: pgmap v66: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 7.1 MiB/s wr, 289 op/s 2024-04-02T13:37:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:17 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:37:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:17 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:37:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:17 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:37:18.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71337 10.34 MB/sec warmup 39 sec latency 1012.122 ms 2024-04-02T13:37:19.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71651 10.09 MB/sec warmup 40 sec latency 1645.610 ms 2024-04-02T13:37:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:19 smithi187 ceph-mon[36064]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-04-02T13:37:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:19 smithi188 ceph-mon[36117]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-04-02T13:37:20.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:19 smithi122 ceph-mon[32114]: pgmap v67: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-04-02T13:37:20.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71752 9.85 MB/sec warmup 41 sec latency 928.474 ms 2024-04-02T13:37:21.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71752 9.61 MB/sec warmup 42 sec latency 1928.553 ms 2024-04-02T13:37:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:21 smithi187 ceph-mon[36064]: pgmap v68: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 713 KiB/s wr, 123 op/s 2024-04-02T13:37:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:21 smithi188 ceph-mon[36117]: pgmap v68: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 713 KiB/s wr, 123 op/s 2024-04-02T13:37:22.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:21 smithi122 ceph-mon[32114]: pgmap v68: 129 pgs: 129 active+clean; 49 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 713 KiB/s wr, 123 op/s 2024-04-02T13:37:22.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71752 9.39 MB/sec warmup 43 sec latency 2928.669 ms 2024-04-02T13:37:23.940 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71752 9.18 MB/sec warmup 44 sec latency 3928.777 ms 2024-04-02T13:37:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:23 smithi187 ceph-mon[36064]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 785 KiB/s wr, 144 op/s 2024-04-02T13:37:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:23 smithi188 ceph-mon[36117]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 785 KiB/s wr, 144 op/s 2024-04-02T13:37:24.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:23 smithi122 ceph-mon[32114]: pgmap v69: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 785 KiB/s wr, 144 op/s 2024-04-02T13:37:24.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71753 8.97 MB/sec warmup 45 sec latency 4562.978 ms 2024-04-02T13:37:25.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71753 8.78 MB/sec warmup 46 sec latency 1365.938 ms 2024-04-02T13:37:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:25 smithi187 ceph-mon[36064]: pgmap v70: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 107 KiB/s wr, 28 op/s 2024-04-02T13:37:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:25 smithi188 ceph-mon[36117]: pgmap v70: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 107 KiB/s wr, 28 op/s 2024-04-02T13:37:26.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:25 smithi122 ceph-mon[32114]: pgmap v70: 129 pgs: 129 active+clean; 50 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 107 KiB/s wr, 28 op/s 2024-04-02T13:37:26.810 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:37:26.810 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:37:26.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 -- 172.21.15.122:0/3949822872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 msgr2=0x7fa25010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 --2- 172.21.15.122:0/3949822872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa25010e960 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa248009cd0 tx=0x7fa24802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 -- 172.21.15.122:0/3949822872 shutdown_connections 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 --2- 172.21.15.122:0/3949822872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa25010e960 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 --2- 172.21.15.122:0/3949822872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 0x7fa250101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 --2- 172.21.15.122:0/3949822872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa250107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 -- 172.21.15.122:0/3949822872 >> 172.21.15.122:0/3949822872 conn(0x7fa2500fd560 msgr2=0x7fa2500ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 -- 172.21.15.122:0/3949822872 shutdown_connections 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 -- 172.21.15.122:0/3949822872 wait complete. 2024-04-02T13:37:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.926+0000 7fa255904640 1 Processor -- start 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 -- start start 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 0x7fa250198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa25019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa250111eb0 con 0x7fa250102450 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa250111d30 con 0x7fa250107aa0 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa255904640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa250112030 con 0x7fa250101a90 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53034/0 (socket says 172.21.15.122:53034) 2024-04-02T13:37:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 -- 172.21.15.122:0/3425139652 learned_addr learned my addr 172.21.15.122:0/3425139652 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa25019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24effd640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 0x7fa250198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 -- 172.21.15.122:0/3425139652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 msgr2=0x7fa250198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 0x7fa250198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 -- 172.21.15.122:0/3425139652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 msgr2=0x7fa25019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa25019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 -- 172.21.15.122:0/3425139652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25019d360 con 0x7fa250102450 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa25019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.927+0000 7fa24e7fc640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c920 tx=0x7fa24400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.928+0000 7fa254902640 1 -- 172.21.15.122:0/3425139652 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244007c70 con 0x7fa250102450 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.928+0000 7fa254902640 1 -- 172.21.15.122:0/3425139652 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa244007e10 con 0x7fa250102450 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.928+0000 7fa254902640 1 -- 172.21.15.122:0/3425139652 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244018890 con 0x7fa250102450 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.928+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25019d650 con 0x7fa250102450 2024-04-02T13:37:26.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.928+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2501a4ee0 con 0x7fa250102450 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa254902640 1 -- 172.21.15.122:0/3425139652 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa244018a70 con 0x7fa250102450 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 Processor -- start 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 -- start start 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa254902640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 0x7fa2380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414831f0 0x56444148b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56444148bdd0 0x56444148e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5644413e6830 con 0x5644414828d0 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56444138e820 con 0x5644414831f0 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa2585c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5644413e27e0 con 0x56444148bdd0 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa254902640 1 -- 172.21.15.122:0/3425139652 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa2440d0dc0 con 0x7fa250102450 2024-04-02T13:37:26.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.930+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 0x7fa2380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53044/0 (socket says 172.21.15.122:53044) 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 -- 172.21.15.122:0/2617568691 learned_addr learned my addr 172.21.15.122:0/2617568691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24e7fc640 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414831f0 0x56444148b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56444148bdd0 0x56444148e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 -- 172.21.15.122:0/2617568691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56444148bdd0 msgr2=0x56444148e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56444148bdd0 0x56444148e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 -- 172.21.15.122:0/2617568691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414831f0 msgr2=0x56444148b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414831f0 0x56444148b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 -- 172.21.15.122:0/2617568691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644411c47d0 con 0x5644414828d0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 0x7fa2380a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa248006010 tx=0x7fa24802cca0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa24effd640 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa24000ba30 tx=0x7fa24000bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af1fe3cdf0d7465 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa235ffb640 1 -- 172.21.15.122:0/2617568691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240017420 con 0x5644414828d0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa235ffb640 1 -- 172.21.15.122:0/2617568691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240017aa0 con 0x5644414828d0 2024-04-02T13:37:26.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.931+0000 7fa235ffb640 1 -- 172.21.15.122:0/2617568691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24000f980 con 0x5644414828d0 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- 172.21.15.122:0/2617568691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 msgr2=0x564441482cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa24000ba30 tx=0x7fa24000bef0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- 172.21.15.122:0/2617568691 shutdown_connections 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56444148bdd0 0x56444148e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414831f0 0x56444148b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2617568691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x564441482cb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- 172.21.15.122:0/2617568691 >> 172.21.15.122:0/2617568691 conn(0x56444147d000 msgr2=0x56444147d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- 172.21.15.122:0/2617568691 shutdown_connections 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- 172.21.15.122:0/2617568691 wait complete. 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.932+0000 7fa2585c8740 1 -- start start 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 Processor -- start 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56444148ebc0 con 0x56444148bdd0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5644413a1d30 con 0x5644414828d0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56444148ed40 con 0x5644414831f0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa2585c8740 10 client.0 _ll_register_callbacks cb 0x5644413a1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53056/0 (socket says 172.21.15.122:53056) 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 -- 172.21.15.122:0/1185865947 learned_addr learned my addr 172.21.15.122:0/1185865947 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24effd640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 -- 172.21.15.122:0/1185865947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 msgr2=0x564441488920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 -- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 msgr2=0x56444148b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 -- 172.21.15.122:0/1185865947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644410fa3f0 con 0x56444148bdd0 2024-04-02T13:37:26.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa24802c700 tx=0x7fa24802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f58a8f5ee38f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.933+0000 7fa22effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:26.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248039040 con 0x56444148bdd0 2024-04-02T13:37:26.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa248003b70 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa240017aa0 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa248003d70 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 10 client.16377 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5644410ba0f0 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa240017aa0 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa248005310 con 0x56444148bdd0 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 10 client.16377 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 10 client.16377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 20 client.16377 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:26.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56444149c120 0x56444149e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 4 client.16377 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644413f3d70 con 0x56444149c120 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.934+0000 7fa24effd640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56444149c120 0x56444149e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.935+0000 7fa24effd640 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56444149c120 0x56444149e5e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9eb841ea2b38e65 server_cookie=5210b472452c1b60 in_seq=0 out_seq=0 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.935+0000 7fa22effd640 10 client.16377 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa22effd640 1 -- 172.21.15.122:0/1185865947 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644413f3d70 con 0x56444149c120 2024-04-02T13:37:26.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa22effd640 10 client.16377 handle_command_reply: tid=0 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 client.16377 shutdown 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 msgr2=0x564441488e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa24802c700 tx=0x7fa24802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 shutdown_connections 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56444149c120 0x56444149e5e0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441488e60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa22effd640 0 client.16377 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1185865947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 >> 172.21.15.122:0/1185865947 conn(0x56444147d000 msgr2=0x56444147ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 shutdown_connections 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 1 -- 172.21.15.122:0/1185865947 wait complete. 2024-04-02T13:37:26.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.936+0000 7fa2585c8740 20 client.16377 trim_cache size 0 max 16384 2024-04-02T13:37:26.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71753 8.59 MB/sec warmup 47 sec latency 2365.975 ms 2024-04-02T13:37:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.953+0000 7fa2585c8740 1 Processor -- start 2024-04-02T13:37:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.953+0000 7fa2585c8740 1 -- start start 2024-04-02T13:37:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 0x56444147c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501aefe0 con 0x5644414828d0 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501b0be0 con 0x56444148bdd0 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2585c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501af6b0 con 0x5644414831f0 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53064/0 (socket says 172.21.15.122:53064) 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 -- 172.21.15.122:0/1447008716 learned_addr learned my addr 172.21.15.122:0/1447008716 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 0x56444147c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 -- 172.21.15.122:0/1447008716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 msgr2=0x564441488200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 -- 172.21.15.122:0/1447008716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 msgr2=0x56444147c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 0x56444147c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 -- 172.21.15.122:0/1447008716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564441483020 con 0x5644414828d0 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 0x56444147c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa24effd640 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa24000b950 tx=0x7fa24000f560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879c014807c28d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2357fa640 1 -- 172.21.15.122:0/1447008716 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240031070 con 0x5644414828d0 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2357fa640 1 -- 172.21.15.122:0/1447008716 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24000b500 con 0x5644414828d0 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.954+0000 7fa2357fa640 1 -- 172.21.15.122:0/1447008716 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240017680 con 0x5644414828d0 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- 172.21.15.122:0/1447008716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 msgr2=0x5644414840c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa24000b950 tx=0x7fa24000f560 comp rx=0 tx=0).stop 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- 172.21.15.122:0/1447008716 shutdown_connections 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56444148bdd0 0x56444147c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x564441488200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 --2- 172.21.15.122:0/1447008716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644414828d0 0x5644414840c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- 172.21.15.122:0/1447008716 >> 172.21.15.122:0/1447008716 conn(0x56444147d000 msgr2=0x56444147b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- 172.21.15.122:0/1447008716 shutdown_connections 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- 172.21.15.122:0/1447008716 wait complete. 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 -- start start 2024-04-02T13:37:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.955+0000 7fa2585c8740 1 Processor -- start 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x56444148f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501b0450 con 0x56444148bdd0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501afd80 con 0x5644414828d0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2501acfc0 con 0x5644414831f0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 10 client.0 _ll_register_callbacks cb 0x7fa248032d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53080/0 (socket says 172.21.15.122:53080) 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 -- 172.21.15.122:0/2827368411 learned_addr learned my addr 172.21.15.122:0/2827368411 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24e7fc640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x56444148f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24effd640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 -- 172.21.15.122:0/2827368411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 msgr2=0x56444148f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x56444148f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 -- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 msgr2=0x56444148ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 -- 172.21.15.122:0/2827368411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56444148fd30 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa24f7fe640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa248032d00 tx=0x7fa24802c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2298e63d3e046b6b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa22effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248003b80 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa248004ed0 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.956+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5644413ee0c0 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa248068050 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 10 client.16383 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564441501ce0 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5644413ee0c0 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa248004520 con 0x56444148bdd0 2024-04-02T13:37:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 10 client.16383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 20 client.16383 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564441502520 0x564441504940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 4 client.16383 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56444136b560 con 0x564441502520 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa24effd640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564441502520 0x564441504940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa24effd640 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564441502520 0x564441504940 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a78bec1d0847d06f server_cookie=85802a8f2cd5ced1 in_seq=0 out_seq=0 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.957+0000 7fa22effd640 10 client.16383 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa22effd640 1 -- 172.21.15.122:0/2827368411 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56444136b560 con 0x564441502520 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa22effd640 10 client.16383 handle_command_reply: tid=0 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 client.16383 shutdown 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 msgr2=0x564441500750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa248032d00 tx=0x7fa24802c770 comp rx=0 tx=0).stop 2024-04-02T13:37:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 shutdown_connections 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564441502520 0x564441504940 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56444148bdd0 0x564441500750 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa22effd640 0 client.16383 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644414831f0 0x56444148f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 --2- 172.21.15.122:0/2827368411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5644414828d0 0x56444148ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 >> 172.21.15.122:0/2827368411 conn(0x56444147d000 msgr2=0x56444147c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 shutdown_connections 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 1 -- 172.21.15.122:0/2827368411 wait complete. 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa2585c8740 20 client.16383 trim_cache size 0 max 16384 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:37:26.968 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:37:26.969 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:37:26.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 msgr2=0x7fa2380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 0x7fa2380a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa248006010 tx=0x7fa24802cca0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.958+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 msgr2=0x7fa2501988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:26.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa24400c920 tx=0x7fa24400cde0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 shutdown_connections 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2380a3d20 0x7fa2380a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa250107aa0 0x7fa25019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa250102450 0x7fa2501988b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 --2- 172.21.15.122:0/3425139652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa250101a90 0x7fa250198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 >> 172.21.15.122:0/3425139652 conn(0x7fa2500fd560 msgr2=0x7fa2500fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 shutdown_connections 2024-04-02T13:37:26.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:26.959+0000 7fa255904640 1 -- 172.21.15.122:0/3425139652 wait complete. 2024-04-02T13:37:26.972 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:37:26.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:37:27.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 -- 172.21.15.122:0/2065701636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81981013c0 msgr2=0x7f819810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 --2- 172.21.15.122:0/2065701636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81981013c0 0x7f819810d8d0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8188009930 tx=0x7f818802bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:27.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 -- 172.21.15.122:0/2065701636 shutdown_connections 2024-04-02T13:37:27.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 --2- 172.21.15.122:0/2065701636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81981013c0 0x7f819810d8d0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 --2- 172.21.15.122:0/2065701636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198100a00 0x7f8198100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 --2- 172.21.15.122:0/2065701636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8198106a10 0x7f8198106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.071+0000 7f819d74d640 1 -- 172.21.15.122:0/2065701636 >> 172.21.15.122:0/2065701636 conn(0x7f8198077b00 msgr2=0x7f8198077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- 172.21.15.122:0/2065701636 shutdown_connections 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- 172.21.15.122:0/2065701636 wait complete. 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 Processor -- start 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- start start 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81981013c0 0x7f81981988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198106a10 0x7f819819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8198110df0 con 0x7f8198100a00 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8198110c70 con 0x7f8198106a10 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.072+0000 7f819d74d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8198110f70 con 0x7f81981013c0 2024-04-02T13:37:27.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53094/0 (socket says 172.21.15.122:53094) 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 -- 172.21.15.122:0/1444255838 learned_addr learned my addr 172.21.15.122:0/1444255838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f81967fc640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81981013c0 0x7f81981988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f81977fe640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198106a10 0x7f819819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 -- 172.21.15.122:0/1444255838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81981013c0 msgr2=0x7f81981988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81981013c0 0x7f81981988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 -- 172.21.15.122:0/1444255838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198106a10 msgr2=0x7f819819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198106a10 0x7f819819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 -- 172.21.15.122:0/1444255838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819d3a0 con 0x7f8198100a00 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f8196ffd640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f819000d950 tx=0x7f819000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8190017070 con 0x7f8198100a00 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819d690 con 0x7f8198100a00 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8190009c30 con 0x7f8198100a00 2024-04-02T13:37:27.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819000f6b0 con 0x7f8198100a00 2024-04-02T13:37:27.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.073+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a4f20 con 0x7f8198100a00 2024-04-02T13:37:27.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.074+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81980688f0 con 0x7f8198100a00 2024-04-02T13:37:27.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.075+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f81900109d0 con 0x7f8198100a00 2024-04-02T13:37:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.076+0000 7f817ffff640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 0x7f81800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.076+0000 7f81967fc640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 0x7f81800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.076+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f81900d0800 con 0x7f8198100a00 2024-04-02T13:37:27.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.077+0000 7f81967fc640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 0x7f81800a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8198100550 tx=0x7f818c009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.080+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8190009de0 con 0x7f8198100a00 2024-04-02T13:37:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.295+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8198199700 con 0x7f8198100a00 2024-04-02T13:37:27.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.296+0000 7f817ffff640 1 -- 172.21.15.122:0/1444255838 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11949 (secure 0 0 0) 0x7f8198199700 con 0x7f8198100a00 2024-04-02T13:37:27.300 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:37:27.300 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:37:11.399875+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:37:27.301 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 30 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 msgr2=0x7f81800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 0x7f81800a61e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8198100550 tx=0x7f818c009500 comp rx=0 tx=0).stop 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 msgr2=0x7f81981983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f819000d950 tx=0x7f819000de10 comp rx=0 tx=0).stop 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 shutdown_connections 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f81800a3d20 0x7f81800a61e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8198106a10 0x7f819819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81981013c0 0x7f81981988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 --2- 172.21.15.122:0/1444255838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8198100a00 0x7f81981983b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 >> 172.21.15.122:0/1444255838 conn(0x7f8198077b00 msgr2=0x7f8198068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 shutdown_connections 2024-04-02T13:37:27.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.298+0000 7f819d74d640 1 -- 172.21.15.122:0/1444255838 wait complete. 2024-04-02T13:37:27.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.412+0000 7ff2a41eb640 1 -- 172.21.15.122:0/2535798002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c068ce0 msgr2=0x7ff29c069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.412+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/2535798002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c068ce0 0x7ff29c069160 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009930 tx=0x7ff28c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- 172.21.15.122:0/2535798002 shutdown_connections 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/2535798002 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 0x7ff29c10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/2535798002 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c068ce0 0x7ff29c069160 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/2535798002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c107a80 0x7ff29c107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- 172.21.15.122:0/2535798002 >> 172.21.15.122:0/2535798002 conn(0x7ff29c077b00 msgr2=0x7ff29c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- 172.21.15.122:0/2535798002 shutdown_connections 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- 172.21.15.122:0/2535798002 wait complete. 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 Processor -- start 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- start start 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c068ce0 0x7ff29c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 0x7ff29c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c111e90 con 0x7ff29c107a80 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c111d10 con 0x7ff29c0696a0 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a41eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c112010 con 0x7ff29c068ce0 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.413+0000 7ff2a2788640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53104/0 (socket says 172.21.15.122:53104) 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a1f87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c068ce0 0x7ff29c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 -- 172.21.15.122:0/1694578451 learned_addr learned my addr 172.21.15.122:0/1694578451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a1786640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 0x7ff29c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 -- 172.21.15.122:0/1694578451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c068ce0 msgr2=0x7ff29c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c068ce0 0x7ff29c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 -- 172.21.15.122:0/1694578451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 msgr2=0x7ff29c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 0x7ff29c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 -- 172.21.15.122:0/1694578451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29c19d3e0 con 0x7ff29c107a80 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a2788640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff29400eb10 tx=0x7ff29400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff292ffd640 1 -- 172.21.15.122:0/1694578451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294004b20 con 0x7ff29c107a80 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff292ffd640 1 -- 172.21.15.122:0/1694578451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff294004cc0 con 0x7ff29c107a80 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29c19d6d0 con 0x7ff29c107a80 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff292ffd640 1 -- 172.21.15.122:0/1694578451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff294010760 con 0x7ff29c107a80 2024-04-02T13:37:27.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.414+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff29c1a4f60 con 0x7ff29c107a80 2024-04-02T13:37:27.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.416+0000 7ff292ffd640 1 -- 172.21.15.122:0/1694578451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff294005020 con 0x7ff29c107a80 2024-04-02T13:37:27.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.416+0000 7ff2a6dcb740 1 Processor -- start 2024-04-02T13:37:27.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 -- start start 2024-04-02T13:37:27.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 0x55daef56a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef57a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55daef4ce7b0 con 0x55daef56adf0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55daef46d570 con 0x55daef56a4d0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55daef4ca760 con 0x55daef5780b0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff292ffd640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 0x7ff2880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53120/0 (socket says 172.21.15.122:53120) 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 -- 172.21.15.122:0/3451813129 learned_addr learned my addr 172.21.15.122:0/3451813129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a2788640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 0x7ff2880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1786640 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 0x55daef56a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 -- 172.21.15.122:0/3451813129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 msgr2=0x55daef57a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef57a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff292ffd640 1 -- 172.21.15.122:0/1694578451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff2940d1110 con 0x7ff29c107a80 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 -- 172.21.15.122:0/3451813129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 msgr2=0x55daef56a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 0x55daef56a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 -- 172.21.15.122:0/3451813129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daef1e23f0 con 0x55daef56adf0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1786640 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 0x55daef56a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.417+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b710 tx=0x7ff298004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ff4b22293f6d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff287fff640 1 -- 172.21.15.122:0/3451813129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298003bd0 con 0x55daef56adf0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a2788640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 0x7ff2880a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff29409ca20 tx=0x7ff29400efd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff287fff640 1 -- 172.21.15.122:0/3451813129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff298015070 con 0x55daef56adf0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff287fff640 1 -- 172.21.15.122:0/3451813129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298004e60 con 0x55daef56adf0 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3451813129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 msgr2=0x55daef577b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff29800b710 tx=0x7ff298004610 comp rx=0 tx=0).stop 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3451813129 shutdown_connections 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef57a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef577b70 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3451813129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56a4d0 0x55daef56a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.418+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3451813129 >> 172.21.15.122:0/3451813129 conn(0x55daef564ed0 msgr2=0x55daef5652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3451813129 shutdown_connections 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3451813129 wait complete. 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- start start 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 Processor -- start 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56a4d0 0x55daef574fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55daef57ae70 con 0x55daef56adf0 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55daef489e80 con 0x55daef5780b0 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55daef57aff0 con 0x55daef56a4d0 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a6dcb740 10 client.0 _ll_register_callbacks cb 0x55daef4c5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a1f87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a1f87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53126/0 (socket says 172.21.15.122:53126) 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.419+0000 7ff2a1f87640 1 -- 172.21.15.122:0/2493252064 learned_addr learned my addr 172.21.15.122:0/2493252064 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56a4d0 0x55daef574fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 -- 172.21.15.122:0/2493252064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56a4d0 msgr2=0x55daef574fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56a4d0 0x55daef574fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 -- 172.21.15.122:0/2493252064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 msgr2=0x55daef575a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 -- 172.21.15.122:0/2493252064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daef2ac7d0 con 0x55daef56adf0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff298004960 tx=0x7ff29800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd301dc2cf4ceb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298011440 con 0x55daef56adf0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff2980115e0 con 0x55daef56adf0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff298015070 con 0x55daef56adf0 2024-04-02T13:37:27.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff298030050 con 0x55daef56adf0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a6dcb740 10 client.16401 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55daef586fb0 con 0x55daef56adf0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff298015070 con 0x55daef56adf0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.420+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff298025980 con 0x55daef56adf0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 10 client.16401 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 10 client.16401 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 20 client.16401 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef588410 0x55daef58a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 4 client.16401 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55daef4dbcf0 con 0x55daef588410 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef588410 0x55daef58a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef588410 0x55daef58a800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4f7ef202f08ad602 server_cookie=4effc633b85e455 in_seq=0 out_seq=0 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2857fa640 10 client.16401 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2857fa640 1 -- 172.21.15.122:0/2493252064 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55daef4dbcf0 con 0x55daef588410 2024-04-02T13:37:27.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.421+0000 7ff2857fa640 10 client.16401 handle_command_reply: tid=0 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 client.16401 shutdown 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 msgr2=0x55daef575500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff298004960 tx=0x7ff29800bf10 comp rx=0 tx=0).stop 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 shutdown_connections 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef588410 0x55daef58a800 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2857fa640 0 client.16401 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56adf0 0x55daef575500 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2493252064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56a4d0 0x55daef574fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 >> 172.21.15.122:0/2493252064 conn(0x55daef564ed0 msgr2=0x55daef566c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 shutdown_connections 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2493252064 wait complete. 2024-04-02T13:37:27.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.422+0000 7ff2a6dcb740 20 client.16401 trim_cache size 0 max 16384 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 Processor -- start 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 -- start start 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56adf0 0x55daef574120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1ace20 con 0x55daef56a4d0 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1aea20 con 0x55daef5780b0 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1ad4f0 con 0x55daef56adf0 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53134/0 (socket says 172.21.15.122:53134) 2024-04-02T13:37:27.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 -- 172.21.15.122:0/3861329732 learned_addr learned my addr 172.21.15.122:0/3861329732 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 -- 172.21.15.122:0/3861329732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56adf0 msgr2=0x55daef574120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56adf0 0x55daef574120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 -- 172.21.15.122:0/3861329732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 msgr2=0x55daef575860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.435+0000 7ff2a1786640 1 -- 172.21.15.122:0/3861329732 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55daef569890 con 0x55daef56a4d0 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a1786640 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009900 tx=0x7ff28c049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecb5ca537354fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2877fe640 1 -- 172.21.15.122:0/3861329732 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c02c880 con 0x55daef56a4d0 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2877fe640 1 -- 172.21.15.122:0/3861329732 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c0493e0 con 0x55daef56a4d0 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2877fe640 1 -- 172.21.15.122:0/3861329732 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004b50 con 0x55daef56a4d0 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3861329732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 msgr2=0x55daef583d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff28c009900 tx=0x7ff28c049bd0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3861329732 shutdown_connections 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef5780b0 0x55daef575860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef56adf0 0x55daef574120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/3861329732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef583d70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3861329732 >> 172.21.15.122:0/3861329732 conn(0x55daef564ed0 msgr2=0x55daef566320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3861329732 shutdown_connections 2024-04-02T13:37:27.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.436+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/3861329732 wait complete. 2024-04-02T13:37:27.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 -- start start 2024-04-02T13:37:27.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 Processor -- start 2024-04-02T13:37:27.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56adf0 0x55daef5807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef571290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1ae290 con 0x55daef56a4d0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1adbc0 con 0x55daef56adf0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff29c1aae00 con 0x55daef5780b0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a6dcb740 10 client.0 _ll_register_callbacks cb 0x7ff298004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a1786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a1786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53138/0 (socket says 172.21.15.122:53138) 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a1786640 1 -- 172.21.15.122:0/2126464633 learned_addr learned my addr 172.21.15.122:0/2126464633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56adf0 0x55daef5807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.437+0000 7ff2a1786640 1 -- 172.21.15.122:0/2126464633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 msgr2=0x55daef571290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef571290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1786640 1 -- 172.21.15.122:0/2126464633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56adf0 msgr2=0x55daef5807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56adf0 0x55daef5807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1786640 1 -- 172.21.15.122:0/2126464633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55daef571900 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1786640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff28c004770 tx=0x7ff28c049960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f782a05566adca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004040 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff28c0041e0 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff28c0493e0 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff28c0043e0 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 10 client.16407 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55daef5ee140 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c0493e0 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff28c005480 con 0x55daef56a4d0 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 10 client.16407 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:37:27.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 20 client.16407 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef5ee860 0x55daef5f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 4 client.16407 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55daef4d6040 con 0x55daef5ee860 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.438+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef5ee860 0x55daef5f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2857fa640 10 client.16407 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a1f87640 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef5ee860 0x55daef5f0c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fab0ee73012d9986 server_cookie=4ef6340bcfce8710 in_seq=0 out_seq=0 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2857fa640 1 -- 172.21.15.122:0/2126464633 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55daef4d6040 con 0x55daef5ee860 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2857fa640 10 client.16407 handle_command_reply: tid=0 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 client.16407 shutdown 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 msgr2=0x55daef580280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff28c004770 tx=0x7ff28c049960 comp rx=0 tx=0).stop 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 shutdown_connections 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55daef5ee860 0x55daef5f0c80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55daef5780b0 0x55daef571290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2857fa640 0 client.16407 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55daef56adf0 0x55daef5807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 --2- 172.21.15.122:0/2126464633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55daef56a4d0 0x55daef580280 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 >> 172.21.15.122:0/2126464633 conn(0x55daef564ed0 msgr2=0x55daef563f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 shutdown_connections 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 1 -- 172.21.15.122:0/2126464633 wait complete. 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.439+0000 7ff2a6dcb740 20 client.16407 trim_cache size 0 max 16384 2024-04-02T13:37:27.450 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:27.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 msgr2=0x7ff2880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 0x7ff2880a61e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff29409ca20 tx=0x7ff29400efd0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 msgr2=0x7ff29c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff29400eb10 tx=0x7ff29400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 shutdown_connections 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2880a3d20 0x7ff2880a61e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff29c107a80 0x7ff29c19ccb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff29c0696a0 0x7ff29c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 --2- 172.21.15.122:0/1694578451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff29c068ce0 0x7ff29c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 >> 172.21.15.122:0/1694578451 conn(0x7ff29c077b00 msgr2=0x7ff29c0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 shutdown_connections 2024-04-02T13:37:27.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.440+0000 7ff2a41eb640 1 -- 172.21.15.122:0/1694578451 wait complete. 2024-04-02T13:37:27.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 -- 172.21.15.122:0/1147512349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c100c20 msgr2=0x7f775c106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 --2- 172.21.15.122:0/1147512349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c100c20 0x7f775c106750 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7750009900 tx=0x7f775002bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 -- 172.21.15.122:0/1147512349 shutdown_connections 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 --2- 172.21.15.122:0/1147512349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c10ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 --2- 172.21.15.122:0/1147512349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c100c20 0x7f775c106750 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 --2- 172.21.15.122:0/1147512349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 0x7f775c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 -- 172.21.15.122:0/1147512349 >> 172.21.15.122:0/1147512349 conn(0x7f775c076fc0 msgr2=0x7f775c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 -- 172.21.15.122:0/1147512349 shutdown_connections 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.553+0000 7f7761cba640 1 -- 172.21.15.122:0/1147512349 wait complete. 2024-04-02T13:37:27.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 Processor -- start 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 -- start start 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 0x7f775c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f775c111ee0 con 0x7f775c19cc80 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f775c111d60 con 0x7f775c106e80 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f7761cba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f775c112060 con 0x7f775c100270 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f775bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f775bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53150/0 (socket says 172.21.15.122:53150) 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f775bfff640 1 -- 172.21.15.122:0/3554713542 learned_addr learned my addr 172.21.15.122:0/3554713542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f775affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.554+0000 7f775b7fe640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 0x7f775c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 -- 172.21.15.122:0/3554713542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 msgr2=0x7f775c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 0x7f775c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 -- 172.21.15.122:0/3554713542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 msgr2=0x7f775c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 -- 172.21.15.122:0/3554713542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775c19d770 con 0x7f775c19cc80 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775affd640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f775bfff640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f775400eb10 tx=0x7f775400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f7758ff9640 1 -- 172.21.15.122:0/3554713542 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754004b20 con 0x7f775c19cc80 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f7758ff9640 1 -- 172.21.15.122:0/3554713542 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7754004cc0 con 0x7f775c19cc80 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775c19da60 con 0x7f775c19cc80 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f7758ff9640 1 -- 172.21.15.122:0/3554713542 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754010760 con 0x7f775c19cc80 2024-04-02T13:37:27.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.555+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f775c1a51d0 con 0x7f775c19cc80 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.557+0000 7f7758ff9640 1 -- 172.21.15.122:0/3554713542 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7754005020 con 0x7f775c19cc80 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.557+0000 7f7764976740 1 Processor -- start 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.557+0000 7f7764976740 1 -- start start 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a50c1167b0 con 0x55a50c1b30c0 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a50c0b5570 con 0x55a50c1bbca0 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7764976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a50c112760 con 0x55a50c1b27a0 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53164/0 (socket says 172.21.15.122:53164) 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 -- 172.21.15.122:0/2806086917 learned_addr learned my addr 172.21.15.122:0/2806086917 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7758ff9640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 0x7f77440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775affd640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775affd640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 -- 172.21.15.122:0/2806086917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 msgr2=0x55a50c1b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 -- 172.21.15.122:0/2806086917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 msgr2=0x55a50c1be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775bfff640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 0x7f77440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 -- 172.21.15.122:0/2806086917 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50be2a3f0 con 0x55a50c1b30c0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775affd640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7758ff9640 1 -- 172.21.15.122:0/3554713542 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f77540d1090 con 0x7f775c19cc80 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f774c00ac10 tx=0x7f774c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbc348edb4fc3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.558+0000 7f7741ffb640 1 -- 172.21.15.122:0/2806086917 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c01d070 con 0x55a50c1b30c0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7741ffb640 1 -- 172.21.15.122:0/2806086917 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c018b20 con 0x55a50c1b30c0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f775bfff640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 0x7f77440a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77540cea50 tx=0x7f775406e6e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7741ffb640 1 -- 172.21.15.122:0/2806086917 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c010b30 con 0x55a50c1b30c0 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- 172.21.15.122:0/2806086917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 msgr2=0x55a50c1bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f774c00ac10 tx=0x7f774c00f790 comp rx=0 tx=0).stop 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- 172.21.15.122:0/2806086917 shutdown_connections 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1bb760 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 --2- 172.21.15.122:0/2806086917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- 172.21.15.122:0/2806086917 >> 172.21.15.122:0/2806086917 conn(0x55a50c1aced0 msgr2=0x55a50c1ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- 172.21.15.122:0/2806086917 shutdown_connections 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- 172.21.15.122:0/2806086917 wait complete. 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.559+0000 7f7764976740 1 -- start start 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 Processor -- start 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a50c1bea90 con 0x55a50c1b30c0 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a50c0d1e80 con 0x55a50c1bbca0 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a50c1bec10 con 0x55a50c1b27a0 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f7764976740 10 client.0 _ll_register_callbacks cb 0x55a50c10daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53166/0 (socket says 172.21.15.122:53166) 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 -- 172.21.15.122:0/4160464611 learned_addr learned my addr 172.21.15.122:0/4160464611 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775affd640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 -- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 msgr2=0x55a50c1b8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 -- 172.21.15.122:0/4160464611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 msgr2=0x55a50c1b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775affd640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775b7fe640 1 -- 172.21.15.122:0/4160464611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50bef47d0 con 0x55a50c1b30c0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.560+0000 7f775affd640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f775b7fe640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f774c010470 tx=0x7f774c0184c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c367014632fcfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c01d050 con 0x55a50c1b30c0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f774c010870 con 0x55a50c1b30c0 2024-04-02T13:37:27.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f774c018b20 con 0x55a50c1b30c0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f774c031050 con 0x55a50c1b30c0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 10 client.16419 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a50c1caaf0 con 0x55a50c1b30c0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774c018b20 con 0x55a50c1b30c0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f774c0046c0 con 0x55a50c1b30c0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 10 client.16419 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 10 client.16419 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 20 client.16419 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c1cbf50 0x55a50c1ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 4 client.16419 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.561+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50c123cf0 con 0x55a50c1cbf50 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.562+0000 7f775affd640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c1cbf50 0x55a50c1ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.562+0000 7f773affd640 10 client.16419 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.562+0000 7f775affd640 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c1cbf50 0x55a50c1ce340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f83fc9ca66b6c4d0 server_cookie=632f28cb4d8359a9 in_seq=0 out_seq=0 2024-04-02T13:37:27.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f773affd640 1 -- 172.21.15.122:0/4160464611 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a50c123cf0 con 0x55a50c1cbf50 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f773affd640 10 client.16419 handle_command_reply: tid=0 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 client.16419 shutdown 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 msgr2=0x55a50c1b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f774c010470 tx=0x7f774c0184c0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 shutdown_connections 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1bbca0 0x55a50c1b9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c1cbf50 0x55a50c1ce340 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1b9050 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 --2- 172.21.15.122:0/4160464611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b27a0 0x55a50c1b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f773affd640 0 client.16419 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 >> 172.21.15.122:0/4160464611 conn(0x55a50c1aced0 msgr2=0x55a50c1aec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 shutdown_connections 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 1 -- 172.21.15.122:0/4160464611 wait complete. 2024-04-02T13:37:27.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.563+0000 7f7764976740 20 client.16419 trim_cache size 0 max 16384 2024-04-02T13:37:27.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 Processor -- start 2024-04-02T13:37:27.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 -- start start 2024-04-02T13:37:27.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 0x55a50c1c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1aefd0 con 0x55a50c1bbca0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b0bd0 con 0x55a50c1b27a0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.576+0000 7f7764976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1af6a0 con 0x55a50c1b30c0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53176/0 (socket says 172.21.15.122:53176) 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 -- 172.21.15.122:0/3339903048 learned_addr learned my addr 172.21.15.122:0/3339903048 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775b7fe640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 0x55a50c1c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 -- 172.21.15.122:0/3339903048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 msgr2=0x55a50c1c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 0x55a50c1c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 -- 172.21.15.122:0/3339903048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 msgr2=0x55a50c1bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1bb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 -- 172.21.15.122:0/3339903048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50c1b7060 con 0x55a50c1bbca0 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775b7fe640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 0x55a50c1c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f775affd640 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7750009900 tx=0x7f775003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c98ee732642ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f77417fa640 1 -- 172.21.15.122:0/3339903048 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775004d020 con 0x55a50c1bbca0 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f77417fa640 1 -- 172.21.15.122:0/3339903048 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775002cd60 con 0x55a50c1bbca0 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.577+0000 7f77417fa640 1 -- 172.21.15.122:0/3339903048 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750004f20 con 0x55a50c1bbca0 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- 172.21.15.122:0/3339903048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 msgr2=0x55a50c1b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7750009900 tx=0x7f775003f040 comp rx=0 tx=0).stop 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- 172.21.15.122:0/3339903048 shutdown_connections 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1bbca0 0x55a50c1b7af0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1b30c0 0x55a50c1c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- 172.21.15.122:0/3339903048 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- 172.21.15.122:0/3339903048 >> 172.21.15.122:0/3339903048 conn(0x55a50c1aced0 msgr2=0x55a50c1b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- 172.21.15.122:0/3339903048 shutdown_connections 2024-04-02T13:37:27.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- 172.21.15.122:0/3339903048 wait complete. 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- start start 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 Processor -- start 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1bbca0 0x55a50c1b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1b0440 con 0x55a50c1b30c0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.578+0000 7f7764976740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1afd70 con 0x55a50c1b27a0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f7764976740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f775c1acfb0 con 0x55a50c1bbca0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f7764976740 10 client.0 _ll_register_callbacks cb 0x55a50c1bbbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53186/0 (socket says 172.21.15.122:53186) 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 -- 172.21.15.122:0/2470855169 learned_addr learned my addr 172.21.15.122:0/2470855169 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775affd640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775affd640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1bbca0 0x55a50c1b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 -- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1bbca0 msgr2=0x55a50c1b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1bbca0 0x55a50c1b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 -- 172.21.15.122:0/2470855169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 msgr2=0x55a50c1c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 -- 172.21.15.122:0/2470855169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50c1b57d0 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f775b7fe640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f774c0058c0 tx=0x7f774c00a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabf93ee64168ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f773affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c01d040 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f774c018980 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775002cd60 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f774c040050 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f7764976740 10 client.16425 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.579+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a50c231be0 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f775002cd60 con 0x55a50c1b30c0 2024-04-02T13:37:27.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f774c005500 con 0x55a50c1b30c0 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 10 client.16425 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 10 client.16425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 20 client.16425 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c232320 0x55a50c234740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 4 client.16425 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a50c11e040 con 0x55a50c232320 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f775affd640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c232320 0x55a50c234740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f773affd640 10 client.16425 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.580+0000 7f775affd640 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c232320 0x55a50c234740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92aff18aef785530 server_cookie=483bda2581ef9008 in_seq=0 out_seq=0 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f773affd640 1 -- 172.21.15.122:0/2470855169 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a50c11e040 con 0x55a50c232320 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f773affd640 10 client.16425 handle_command_reply: tid=0 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 client.16425 shutdown 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 msgr2=0x55a50c1c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f774c0058c0 tx=0x7f774c00a6d0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 shutdown_connections 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a50c1bbca0 0x55a50c1b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a50c232320 0x55a50c234740 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a50c1b30c0 0x55a50c1c43a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 --2- 172.21.15.122:0/2470855169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a50c1b27a0 0x55a50c1c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f773affd640 0 client.16425 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 >> 172.21.15.122:0/2470855169 conn(0x55a50c1aced0 msgr2=0x55a50c1b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 shutdown_connections 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 1 -- 172.21.15.122:0/2470855169 wait complete. 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7764976740 20 client.16425 trim_cache size 0 max 16384 2024-04-02T13:37:27.591 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:27.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 msgr2=0x7f77440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 0x7f77440a61e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f77540cea50 tx=0x7f775406e6e0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 msgr2=0x7f775c19d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f775400eb10 tx=0x7f775400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 shutdown_connections 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77440a3d20 0x7f77440a61e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f775c19cc80 0x7f775c19d130 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f775c106e80 0x7f775c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.581+0000 7f7761cba640 1 --2- 172.21.15.122:0/3554713542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f775c100270 0x7f775c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.582+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 >> 172.21.15.122:0/3554713542 conn(0x7f775c076fc0 msgr2=0x7f775c1050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.582+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 shutdown_connections 2024-04-02T13:37:27.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.582+0000 7f7761cba640 1 -- 172.21.15.122:0/3554713542 wait complete. 2024-04-02T13:37:27.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:37:27.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 -- 172.21.15.122:0/1289332891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 msgr2=0x7f4f3806a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 --2- 172.21.15.122:0/1289332891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3806a1b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009cd0 tx=0x7f4f3402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 -- 172.21.15.122:0/1289332891 shutdown_connections 2024-04-02T13:37:27.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 --2- 172.21.15.122:0/1289332891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f381045e0 0x7f4f38104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 --2- 172.21.15.122:0/1289332891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f3806a6f0 0x7f4f3806ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 --2- 172.21.15.122:0/1289332891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3806a1b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.696+0000 7f4f3f855640 1 -- 172.21.15.122:0/1289332891 >> 172.21.15.122:0/1289332891 conn(0x7f4f38077ad0 msgr2=0x7f4f38077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- 172.21.15.122:0/1289332891 shutdown_connections 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- 172.21.15.122:0/1289332891 wait complete. 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 Processor -- start 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- start start 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 0x7f4f381a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f381045e0 0x7f4f381a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f38118470 con 0x7f4f38106a70 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381182f0 con 0x7f4f381045e0 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3f855640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381185f0 con 0x7f4f3806a6f0 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3ddf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3ddf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53200/0 (socket says 172.21.15.122:53200) 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.697+0000 7f4f3ddf2640 1 -- 172.21.15.122:0/2548266008 learned_addr learned my addr 172.21.15.122:0/2548266008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f381045e0 0x7f4f381a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 0x7f4f381a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 -- 172.21.15.122:0/2548266008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 msgr2=0x7f4f381a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 0x7f4f381a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 -- 172.21.15.122:0/2548266008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f381045e0 msgr2=0x7f4f381a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f381045e0 0x7f4f381a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 -- 172.21.15.122:0/2548266008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3819f070 con 0x7f4f38106a70 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 0x7f4f381a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3ddf2640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d750 tx=0x7f4f3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f2e7fc640 1 -- 172.21.15.122:0/2548266008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30017070 con 0x7f4f38106a70 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f2e7fc640 1 -- 172.21.15.122:0/2548266008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f30005500 con 0x7f4f38106a70 2024-04-02T13:37:27.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f2e7fc640 1 -- 172.21.15.122:0/2548266008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30004d90 con 0x7f4f38106a70 2024-04-02T13:37:27.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3819f360 con 0x7f4f38106a70 2024-04-02T13:37:27.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.698+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f381ab620 con 0x7f4f38106a70 2024-04-02T13:37:27.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f2e7fc640 1 -- 172.21.15.122:0/2548266008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4f30004f30 con 0x7f4f38106a70 2024-04-02T13:37:27.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 Processor -- start 2024-04-02T13:37:27.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 -- start start 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f93b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dd60ef77b0 con 0x55dd60f940c0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dd60e96570 con 0x55dd60f9cca0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f42437740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd60ef3760 con 0x55dd60f937a0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53206/0 (socket says 172.21.15.122:53206) 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.701+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/146902525 learned_addr learned my addr 172.21.15.122:0/146902525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f2e7fc640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 0x7f4f240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/146902525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 msgr2=0x55dd60f93b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f93b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/146902525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 msgr2=0x55dd60f9f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3ddf2640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 0x7f4f240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/146902525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd60c0b3f0 con 0x55dd60f940c0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f2e7fc640 1 -- 172.21.15.122:0/2548266008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4f300d0610 con 0x7f4f38106a70 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4f340323d0 tx=0x7f4f34032400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5354130a9c0d3c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f237fe640 1 -- 172.21.15.122:0/146902525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34039050 con 0x55dd60f940c0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f3ddf2640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 0x7f4f240a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cdfd0 tx=0x7f4f300916b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f237fe640 1 -- 172.21.15.122:0/146902525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f34045070 con 0x55dd60f940c0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.702+0000 7f4f237fe640 1 -- 172.21.15.122:0/146902525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f340042a0 con 0x55dd60f940c0 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- 172.21.15.122:0/146902525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 msgr2=0x55dd60f9c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4f340323d0 tx=0x7f4f34032400 comp rx=0 tx=0).stop 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- 172.21.15.122:0/146902525 shutdown_connections 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9c760 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 --2- 172.21.15.122:0/146902525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f93b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- 172.21.15.122:0/146902525 >> 172.21.15.122:0/146902525 conn(0x55dd60f8ded0 msgr2=0x55dd60f8e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- 172.21.15.122:0/146902525 shutdown_connections 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- 172.21.15.122:0/146902525 wait complete. 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 -- start start 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.703+0000 7f4f42437740 1 Processor -- start 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 0x55dd60f9c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f9cca0 0x55dd60f9a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dd60f9fa30 con 0x55dd60f940c0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dd60eb2e80 con 0x55dd60f937a0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd60f9fbb0 con 0x55dd60f9cca0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 10 client.0 _ll_register_callbacks cb 0x55dd60eeeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53214/0 (socket says 172.21.15.122:53214) 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/2625681625 learned_addr learned my addr 172.21.15.122:0/2625681625 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 0x55dd60f9c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/2625681625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f9cca0 msgr2=0x55dd60f9a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f9cca0 0x55dd60f9a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 msgr2=0x55dd60f9c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 0x55dd60f9c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/2625681625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd60cd57d0 con 0x55dd60f940c0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 0x55dd60f9c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009df0 tx=0x7f4f3402c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d66e9a9e976f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34050070 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4f34040450 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.704+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f34045070 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f3405b050 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 10 client.16437 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd60faba30 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f34045070 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4f34004080 con 0x55dd60f940c0 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 10 client.16437 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 10 client.16437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 20 client.16437 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd60facd80 0x55dd60faf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 4 client.16437 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:27.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd60f04cf0 con 0x55dd60facd80 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.705+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd60facd80 0x55dd60faf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.706+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd60facd80 0x55dd60faf170 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6651e4657b8da70a server_cookie=4349f00149312ee8 in_seq=0 out_seq=0 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.706+0000 7f4f20ff9640 10 client.16437 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.706+0000 7f4f20ff9640 1 -- 172.21.15.122:0/2625681625 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd60f04cf0 con 0x55dd60facd80 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.706+0000 7f4f20ff9640 10 client.16437 handle_command_reply: tid=0 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.706+0000 7f4f42437740 1 client.16437 shutdown 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 msgr2=0x55dd60f99ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4f34009df0 tx=0x7f4f3402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 shutdown_connections 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f9cca0 0x55dd60f9a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f99ad0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd60facd80 0x55dd60faf170 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 --2- 172.21.15.122:0/2625681625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f937a0 0x55dd60f9c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f20ff9640 0 client.16437 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 >> 172.21.15.122:0/2625681625 conn(0x55dd60f8ded0 msgr2=0x55dd60f8fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 shutdown_connections 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 1 -- 172.21.15.122:0/2625681625 wait complete. 2024-04-02T13:37:27.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.707+0000 7f4f42437740 20 client.16437 trim_cache size 0 max 16384 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 Processor -- start 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 -- start start 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60fa88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f940c0 0x55dd60f9a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.720+0000 7f4f42437740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b5570 con 0x55dd60f9cca0 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f42437740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b7170 con 0x55dd60f940c0 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f42437740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b5c40 con 0x55dd60f937a0 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53228/0 (socket says 172.21.15.122:53228) 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 -- 172.21.15.122:0/1944811827 learned_addr learned my addr 172.21.15.122:0/1944811827 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f940c0 0x55dd60f9a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60fa88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 -- 172.21.15.122:0/1944811827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 msgr2=0x55dd60fa88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60fa88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 -- 172.21.15.122:0/1944811827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f940c0 msgr2=0x55dd60f9a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f940c0 0x55dd60f9a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 -- 172.21.15.122:0/1944811827 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd60f93ef0 con 0x55dd60f9cca0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c3e0 tx=0x7f4f2800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5265ed63fbc7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f22ffd640 1 -- 172.21.15.122:0/1944811827 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f280040f0 con 0x55dd60f9cca0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f22ffd640 1 -- 172.21.15.122:0/1944811827 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f28004290 con 0x55dd60f9cca0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.721+0000 7f4f22ffd640 1 -- 172.21.15.122:0/1944811827 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f28003a10 con 0x55dd60f9cca0 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- 172.21.15.122:0/1944811827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 msgr2=0x55dd60f98ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4f2800c3e0 tx=0x7f4f2800c7e0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- 172.21.15.122:0/1944811827 shutdown_connections 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f9cca0 0x55dd60f98ab0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f940c0 0x55dd60f9a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 --2- 172.21.15.122:0/1944811827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60fa88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- 172.21.15.122:0/1944811827 >> 172.21.15.122:0/1944811827 conn(0x55dd60f8ded0 msgr2=0x55dd60f983a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- 172.21.15.122:0/1944811827 shutdown_connections 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- 172.21.15.122:0/1944811827 wait complete. 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 -- start start 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.722+0000 7f4f42437740 1 Processor -- start 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f9d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b69e0 con 0x55dd60f940c0 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b6310 con 0x55dd60f9cca0 2024-04-02T13:37:27.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f381b3550 con 0x55dd60f937a0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 10 client.0 _ll_register_callbacks cb 0x7f4f2800c640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53234/0 (socket says 172.21.15.122:53234) 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/4042058703 learned_addr learned my addr 172.21.15.122:0/4042058703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f9d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/4042058703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 msgr2=0x55dd60f9d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f9d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 msgr2=0x55dd60f9dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 -- 172.21.15.122:0/4042058703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd60fa3f30 con 0x55dd60f940c0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f3d5f1640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f34004480 tx=0x7f4f3402c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aac0132fa5737d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3404d070 con 0x55dd60f940c0 2024-04-02T13:37:27.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4f34003c80 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f28004290 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4f34039040 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 10 client.16443 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.723+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dd60fa5230 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f28004290 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4f34040920 con 0x55dd60f940c0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 10 client.16443 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 10 client.16443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 20 client.16443 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd61013180 0x55dd61015570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 4 client.16443 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd60eff040 con 0x55dd61013180 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd61013180 0x55dd61015570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f20ff9640 10 client.16443 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.724+0000 7f4f3cdf0640 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd61013180 0x55dd61015570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d49749e6de8ffc54 server_cookie=268a70cc7bbdeb72 in_seq=0 out_seq=0 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f20ff9640 1 -- 172.21.15.122:0/4042058703 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd60eff040 con 0x55dd61013180 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f20ff9640 10 client.16443 handle_command_reply: tid=0 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 client.16443 shutdown 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 msgr2=0x55dd60f9d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4f34004480 tx=0x7f4f3402c7f0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 shutdown_connections 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dd61013180 0x55dd61015570 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dd60f9cca0 0x55dd60f9dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f20ff9640 0 client.16443 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dd60f940c0 0x55dd60f9d680 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 --2- 172.21.15.122:0/4042058703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd60f937a0 0x55dd60f9d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 >> 172.21.15.122:0/4042058703 conn(0x55dd60f8ded0 msgr2=0x55dd60f8c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 shutdown_connections 2024-04-02T13:37:27.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 1 -- 172.21.15.122:0/4042058703 wait complete. 2024-04-02T13:37:27.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f42437740 20 client.16443 trim_cache size 0 max 16384 2024-04-02T13:37:27.735 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 msgr2=0x7f4f240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 0x7f4f240a61e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4f300cdfd0 tx=0x7f4f300916b0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 msgr2=0x7f4f3819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000d750 tx=0x7f4f3000dc10 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 shutdown_connections 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f240a3d20 0x7f4f240a61e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f38106a70 0x7f4f3819e7b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f381045e0 0x7f4f381a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 --2- 172.21.15.122:0/2548266008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f3806a6f0 0x7f4f381a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.725+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 >> 172.21.15.122:0/2548266008 conn(0x7f4f38077ad0 msgr2=0x7f4f380ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.726+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 shutdown_connections 2024-04-02T13:37:27.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.726+0000 7f4f3f855640 1 -- 172.21.15.122:0/2548266008 wait complete. 2024-04-02T13:37:27.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.838+0000 7f30d683c640 1 -- 172.21.15.122:0/855191099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d00696a0 msgr2=0x7f30d010e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.838+0000 7f30d683c640 1 --2- 172.21.15.122:0/855191099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d00696a0 0x7f30d010e940 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009cd0 tx=0x7f30c802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- 172.21.15.122:0/855191099 shutdown_connections 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- 172.21.15.122:0/855191099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d00696a0 0x7f30d010e940 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- 172.21.15.122:0/855191099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d0068ce0 0x7f30d0069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- 172.21.15.122:0/855191099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 0x7f30d0107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- 172.21.15.122:0/855191099 >> 172.21.15.122:0/855191099 conn(0x7f30d0077b00 msgr2=0x7f30d0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- 172.21.15.122:0/855191099 shutdown_connections 2024-04-02T13:37:27.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- 172.21.15.122:0/855191099 wait complete. 2024-04-02T13:37:27.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 Processor -- start 2024-04-02T13:37:27.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- start start 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d00696a0 0x7f30d0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 0x7f30d019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0111e60 con 0x7f30d0068ce0 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0111ce0 con 0x7f30d00696a0 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.839+0000 7f30d683c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30d0111fe0 con 0x7f30d0107a80 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53252/0 (socket says 172.21.15.122:53252) 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 -- 172.21.15.122:0/738761651 learned_addr learned my addr 172.21.15.122:0/738761651 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 0x7f30d019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 -- 172.21.15.122:0/738761651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 msgr2=0x7f30d019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 0x7f30d019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d00696a0 0x7f30d0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 -- 172.21.15.122:0/738761651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d00696a0 msgr2=0x7f30d0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d00696a0 0x7f30d0198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 -- 172.21.15.122:0/738761651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d019d330 con 0x7f30d0068ce0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cffff640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d950 tx=0x7f30c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cd7fa640 1 -- 172.21.15.122:0/738761651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0017070 con 0x7f30d0068ce0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d019d620 con 0x7f30d0068ce0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cd7fa640 1 -- 172.21.15.122:0/738761651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c0009c30 con 0x7f30d0068ce0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30cd7fa640 1 -- 172.21.15.122:0/738761651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c000f6b0 con 0x7f30d0068ce0 2024-04-02T13:37:27.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.840+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d01a4eb0 con 0x7f30d0068ce0 2024-04-02T13:37:27.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.842+0000 7f30cd7fa640 1 -- 172.21.15.122:0/738761651 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f30c00109d0 con 0x7f30d0068ce0 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 Processor -- start 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 -- start start 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c627a0 0x5648e9c62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c6e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648e9bc67b0 con 0x5648e9c630c0 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648e9b65570 con 0x5648e9c627a0 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d9425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648e9bc2760 con 0x5648e9c6bca0 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30cd7fa640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 0x7f30b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53264/0 (socket says 172.21.15.122:53264) 2024-04-02T13:37:27.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30cffff640 1 -- 172.21.15.122:0/76968152 learned_addr learned my addr 172.21.15.122:0/76968152 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 0x7f30b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c6e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.843+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c627a0 0x5648e9c62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cd7fa640 1 -- 172.21.15.122:0/738761651 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f30c00d0820 con 0x7f30d0068ce0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 -- 172.21.15.122:0/76968152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 msgr2=0x5648e9c6e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c6e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 -- 172.21.15.122:0/76968152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c627a0 msgr2=0x5648e9c62b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c627a0 0x5648e9c62b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 -- 172.21.15.122:0/76968152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e98da3f0 con 0x5648e9c630c0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30cffff640 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbdcbfa6e4198e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 0x7f30b80a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009a60 tx=0x7f30c80324f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30b67fc640 1 -- 172.21.15.122:0/76968152 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0020070 con 0x5648e9c630c0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30b67fc640 1 -- 172.21.15.122:0/76968152 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c0009de0 con 0x5648e9c630c0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.844+0000 7f30b67fc640 1 -- 172.21.15.122:0/76968152 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0098b90 con 0x5648e9c630c0 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- 172.21.15.122:0/76968152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 msgr2=0x5648e9c6b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c000bf60 comp rx=0 tx=0).stop 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- 172.21.15.122:0/76968152 shutdown_connections 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c6e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c630c0 0x5648e9c6b760 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- 172.21.15.122:0/76968152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c627a0 0x5648e9c62b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- 172.21.15.122:0/76968152 >> 172.21.15.122:0/76968152 conn(0x5648e9c5ced0 msgr2=0x5648e9c5d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- 172.21.15.122:0/76968152 shutdown_connections 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- 172.21.15.122:0/76968152 wait complete. 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- start start 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 Processor -- start 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c630c0 0x5648e9c68a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c6bca0 0x5648e9c691a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648e9c6ea30 con 0x5648e9c627a0 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648e9b81e80 con 0x5648e9c6bca0 2024-04-02T13:37:27.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.845+0000 7f30d9425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648e9c6ebb0 con 0x5648e9c630c0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30d9425740 10 client.0 _ll_register_callbacks cb 0x5648e9bbdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53268/0 (socket says 172.21.15.122:53268) 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 -- 172.21.15.122:0/1577676934 learned_addr learned my addr 172.21.15.122:0/1577676934 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c630c0 0x5648e9c68a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 -- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c630c0 msgr2=0x5648e9c68a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c6bca0 0x5648e9c691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c630c0 0x5648e9c68a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 -- 172.21.15.122:0/1577676934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c6bca0 msgr2=0x5648e9c691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c6bca0 0x5648e9c691a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 -- 172.21.15.122:0/1577676934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e99a47d0 con 0x5648e9c627a0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30cffff640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f30c0098da0 tx=0x7f30c00986e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16e464697da1d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0020070 con 0x5648e9c627a0 2024-04-02T13:37:27.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f30c0074f10 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c0009de0 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30c00d0620 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30d9425740 10 client.16455 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648e9c7aa80 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.846+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c0009de0 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f30c00bb1a0 con 0x5648e9c627a0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 10 client.16455 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 10 client.16455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 20 client.16455 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9c7bdd0 0x5648e9c7e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 4 client.16455 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648e9bd3cf0 con 0x5648e9c7bdd0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9c7bdd0 0x5648e9c7e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30af7fe640 10 client.16455 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.847+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9c7bdd0 0x5648e9c7e1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1f9a3c290f997a7 server_cookie=64a33bae1ebaf9c5 in_seq=0 out_seq=0 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30af7fe640 1 -- 172.21.15.122:0/1577676934 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648e9bd3cf0 con 0x5648e9c7bdd0 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30af7fe640 10 client.16455 handle_command_reply: tid=0 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 client.16455 shutdown 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 msgr2=0x5648e9c6b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f30c0098da0 tx=0x7f30c00986e0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 shutdown_connections 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9c7bdd0 0x5648e9c7e1c0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.848+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c6bca0 0x5648e9c691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30af7fe640 0 client.16455 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c630c0 0x5648e9c68a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 1 --2- 172.21.15.122:0/1577676934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c6b6d0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 >> 172.21.15.122:0/1577676934 conn(0x5648e9c5ced0 msgr2=0x5648e9c5ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 shutdown_connections 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 1 -- 172.21.15.122:0/1577676934 wait complete. 2024-04-02T13:37:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.849+0000 7f30d9425740 20 client.16455 trim_cache size 0 max 16384 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 Processor -- start 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 -- start start 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c627a0 0x5648e9c67bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c6b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01af020 con 0x5648e9c6bca0 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b0c20 con 0x5648e9c630c0 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d9425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01af6f0 con 0x5648e9c627a0 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53270/0 (socket says 172.21.15.122:53270) 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 -- 172.21.15.122:0/3748443268 learned_addr learned my addr 172.21.15.122:0/3748443268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c6b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 -- 172.21.15.122:0/3748443268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c627a0 msgr2=0x5648e9c67bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c627a0 0x5648e9c67bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 -- 172.21.15.122:0/3748443268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 msgr2=0x5648e9c6b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c6b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.862+0000 7f30d4dd9640 1 -- 172.21.15.122:0/3748443268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648e9c69510 con 0x5648e9c6bca0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d4dd9640 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005f10 tx=0x7f30c8039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a35fc0cf39641c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30b5ffb640 1 -- 172.21.15.122:0/3748443268 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8039070 con 0x5648e9c6bca0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30b5ffb640 1 -- 172.21.15.122:0/3748443268 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8053070 con 0x5648e9c6bca0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30b5ffb640 1 -- 172.21.15.122:0/3748443268 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c804e460 con 0x5648e9c6bca0 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 -- 172.21.15.122:0/3748443268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 msgr2=0x5648e9c6b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005f10 tx=0x7f30c8039000 comp rx=0 tx=0).stop 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 -- 172.21.15.122:0/3748443268 shutdown_connections 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c6bca0 0x5648e9c6b6d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c6b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 --2- 172.21.15.122:0/3748443268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c627a0 0x5648e9c67bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 -- 172.21.15.122:0/3748443268 >> 172.21.15.122:0/3748443268 conn(0x5648e9c5ced0 msgr2=0x5648e9c5d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 -- 172.21.15.122:0/3748443268 shutdown_connections 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.863+0000 7f30d9425740 1 -- 172.21.15.122:0/3748443268 wait complete. 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 -- start start 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 Processor -- start 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c7d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c7d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01b0490 con 0x5648e9c627a0 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01afdc0 con 0x5648e9c630c0 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30d01ad000 con 0x5648e9c6bca0 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30d9425740 10 client.0 _ll_register_callbacks cb 0x7f30c8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53284/0 (socket says 172.21.15.122:53284) 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 -- 172.21.15.122:0/1624905752 learned_addr learned my addr 172.21.15.122:0/1624905752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 -- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 msgr2=0x5648e9c7d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c7d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c7d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 -- 172.21.15.122:0/1624905752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 msgr2=0x5648e9c7d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c7d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.864+0000 7f30cffff640 1 -- 172.21.15.122:0/1624905752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648e9c62ef0 con 0x5648e9c627a0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30cffff640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30c009b9f0 tx=0x7f30c00bb2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed27d534efa185b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0020070 con 0x5648e9c627a0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f30c00106f0 con 0x5648e9c627a0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30c8053070 con 0x5648e9c627a0 2024-04-02T13:37:27.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f30c0098670 con 0x5648e9c627a0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 10 client.16461 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648e9ce2030 con 0x5648e9c627a0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8053070 con 0x5648e9c627a0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f30c00b5b50 con 0x5648e9c627a0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 10 client.16461 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 10 client.16461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 20 client.16461 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9ce2560 0x5648e9ce4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 4 client.16461 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648e9bce040 con 0x5648e9ce2560 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.865+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9ce2560 0x5648e9ce4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30cf7fe640 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9ce2560 0x5648e9ce4950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=455db36b7fe72d53 server_cookie=ef5c99b1936e8cd in_seq=0 out_seq=0 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30af7fe640 10 client.16461 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:37:27.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30af7fe640 1 -- 172.21.15.122:0/1624905752 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648e9bce040 con 0x5648e9ce2560 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30af7fe640 10 client.16461 handle_command_reply: tid=0 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 client.16461 shutdown 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 msgr2=0x5648e9c7cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f30c009b9f0 tx=0x7f30c00bb2b0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 shutdown_connections 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5648e9ce2560 0x5648e9ce4950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648e9c6bca0 0x5648e9c7d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30af7fe640 0 client.16461 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648e9c630c0 0x5648e9c7d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 --2- 172.21.15.122:0/1624905752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648e9c627a0 0x5648e9c7cb90 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 >> 172.21.15.122:0/1624905752 conn(0x5648e9c5ced0 msgr2=0x5648e9c76e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.866+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 shutdown_connections 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d9425740 1 -- 172.21.15.122:0/1624905752 wait complete. 2024-04-02T13:37:27.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d9425740 20 client.16461 trim_cache size 0 max 16384 2024-04-02T13:37:27.877 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:27.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 msgr2=0x7f30b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 0x7f30b80a61e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009a60 tx=0x7f30c80324f0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 msgr2=0x7f30d0198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d950 tx=0x7f30c000de10 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 shutdown_connections 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f30b80a3d20 0x7f30b80a61e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30d0107a80 0x7f30d019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d00696a0 0x7f30d0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 --2- 172.21.15.122:0/738761651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d0068ce0 0x7f30d0198340 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 >> 172.21.15.122:0/738761651 conn(0x7f30d0077b00 msgr2=0x7f30d00fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 shutdown_connections 2024-04-02T13:37:27.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.867+0000 7f30d683c640 1 -- 172.21.15.122:0/738761651 wait complete. 2024-04-02T13:37:27.879 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:27 smithi122 ceph-mon[32114]: pgmap v71: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 115 KiB/s wr, 34 op/s 2024-04-02T13:37:27.880 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:27 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1444255838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:37:27.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:37:27.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71753 8.41 MB/sec warmup 48 sec latency 3366.044 ms 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 -- 172.21.15.122:0/3689477079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f4106a30 msgr2=0x7f94f4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 --2- 172.21.15.122:0/3689477079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f4106a30 0x7f94f4106e10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f94e4009900 tx=0x7f94e402bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 -- 172.21.15.122:0/3689477079 shutdown_connections 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 --2- 172.21.15.122:0/3689477079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f41016a0 0x7f94f410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 --2- 172.21.15.122:0/3689477079 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4100ce0 0x7f94f4101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 --2- 172.21.15.122:0/3689477079 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f4106a30 0x7f94f4106e10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 -- 172.21.15.122:0/3689477079 >> 172.21.15.122:0/3689477079 conn(0x7f94f40779e0 msgr2=0x7f94f4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 -- 172.21.15.122:0/3689477079 shutdown_connections 2024-04-02T13:37:27.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.981+0000 7f94f948f640 1 -- 172.21.15.122:0/3689477079 wait complete. 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 Processor -- start 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 -- start start 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f4100ce0 0x7f94f4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4106a30 0x7f94f41a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4110d50 con 0x7f94f41016a0 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4110bd0 con 0x7f94f4106a30 2024-04-02T13:37:27.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f948f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94f4110ed0 con 0x7f94f4100ce0 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53310/0 (socket says 172.21.15.122:53310) 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 -- 172.21.15.122:0/1449347939 learned_addr learned my addr 172.21.15.122:0/1449347939 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4106a30 0x7f94f41a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f4100ce0 0x7f94f4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 -- 172.21.15.122:0/1449347939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f4100ce0 msgr2=0x7f94f4199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f4100ce0 0x7f94f4199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 -- 172.21.15.122:0/1449347939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4106a30 msgr2=0x7f94f41a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4106a30 0x7f94f41a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 -- 172.21.15.122:0/1449347939 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94f41a46f0 con 0x7f94f41016a0 2024-04-02T13:37:27.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.982+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d950 tx=0x7f94dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.983+0000 7f94f0ff9640 1 -- 172.21.15.122:0/1449347939 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc017030 con 0x7f94f41016a0 2024-04-02T13:37:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.983+0000 7f94f0ff9640 1 -- 172.21.15.122:0/1449347939 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc004020 con 0x7f94f41016a0 2024-04-02T13:37:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.983+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94f41a48c0 con 0x7f94f41016a0 2024-04-02T13:37:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.983+0000 7f94f0ff9640 1 -- 172.21.15.122:0/1449347939 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0047a0 con 0x7f94f41016a0 2024-04-02T13:37:27.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.983+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94f41a4e10 con 0x7f94f41016a0 2024-04-02T13:37:27.991 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:27 smithi187 ceph-mon[36064]: pgmap v71: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 115 KiB/s wr, 34 op/s 2024-04-02T13:37:27.991 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:27 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1444255838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:37:27.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94f0ff9640 1 -- 172.21.15.122:0/1449347939 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f94dc0041d0 con 0x7f94f41016a0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 Processor -- start 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 -- start start 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94f0ff9640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 0x7f94e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 0x7f94e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94f0ff9640 1 -- 172.21.15.122:0/1449347939 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94dc0d0580 con 0x7f94f41016a0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bc2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5580a2b267b0 con 0x5580a2bc30c0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5580a2ac5570 con 0x5580a2bcbca0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.985+0000 7f94fc071740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5580a2b22760 con 0x5580a2bc27a0 2024-04-02T13:37:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53326/0 (socket says 172.21.15.122:53326) 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 -- 172.21.15.122:0/1800390200 learned_addr learned my addr 172.21.15.122:0/1800390200 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bc2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 0x7f94e00a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005fd0 tx=0x7f94ec0096a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 -- 172.21.15.122:0/1800390200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 msgr2=0x5580a2bc2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bc2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 -- 172.21.15.122:0/1800390200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 msgr2=0x5580a2bce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 -- 172.21.15.122:0/1800390200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a283a3f0 con 0x5580a2bc30c0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0985f0 tx=0x7f94dc09bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedbd7548ac04312 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94e97fa640 1 -- 172.21.15.122:0/1800390200 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0a0540 con 0x5580a2bc30c0 2024-04-02T13:37:27.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94e97fa640 1 -- 172.21.15.122:0/1800390200 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc0a4e00 con 0x5580a2bc30c0 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.986+0000 7f94e97fa640 1 -- 172.21.15.122:0/1800390200 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0d0170 con 0x5580a2bc30c0 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- 172.21.15.122:0/1800390200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 msgr2=0x5580a2bcb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0985f0 tx=0x7f94dc09bdd0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- 172.21.15.122:0/1800390200 shutdown_connections 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc30c0 0x5580a2bcb760 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 --2- 172.21.15.122:0/1800390200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bc2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- 172.21.15.122:0/1800390200 >> 172.21.15.122:0/1800390200 conn(0x5580a2bbced0 msgr2=0x5580a2bbd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- 172.21.15.122:0/1800390200 shutdown_connections 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- 172.21.15.122:0/1800390200 wait complete. 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 -- start start 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.987+0000 7f94fc071740 1 Processor -- start 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc27a0 0x5580a2bcb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bc8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5580a2bcea90 con 0x5580a2bcbca0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5580a2ae1e80 con 0x5580a2bc27a0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5580a2bcec10 con 0x5580a2bc30c0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 10 client.0 _ll_register_callbacks cb 0x5580a2b1daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53342/0 (socket says 172.21.15.122:53342) 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 -- 172.21.15.122:0/3563108103 learned_addr learned my addr 172.21.15.122:0/3563108103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94ebfff640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bc8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc27a0 0x5580a2bcb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 -- 172.21.15.122:0/3563108103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 msgr2=0x5580a2bc8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bc8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 -- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc27a0 msgr2=0x5580a2bcb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc27a0 0x5580a2bcb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 -- 172.21.15.122:0/3563108103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a29047d0 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94f37fe640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f94ec011950 tx=0x7f94ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1c1e3cf99bbb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec01dcc0 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94ec003970 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc0a4e00 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94ec01de60 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 10 client.16473 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.988+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580a2bdaaf0 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc0a4e00 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f94ec01e420 con 0x5580a2bcbca0 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 10 client.16473 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 20 client.16473 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2bdc0a0 0x5580a2bde500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 4 client.16473 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580a2b33cf0 con 0x5580a2bdc0a0 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.989+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2bdc0a0 0x5580a2bde500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.990+0000 7f94da7fc640 10 client.16473 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.990+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2bdc0a0 0x5580a2bde500 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca4f2bcefe7b053 server_cookie=9d7b5a03a4babff in_seq=0 out_seq=0 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.990+0000 7f94da7fc640 1 -- 172.21.15.122:0/3563108103 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580a2b33cf0 con 0x5580a2bdc0a0 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.990+0000 7f94da7fc640 10 client.16473 handle_command_reply: tid=0 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.990+0000 7f94fc071740 1 client.16473 shutdown 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 msgr2=0x5580a2bc91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f94ec011950 tx=0x7f94ec004ab0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 shutdown_connections 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2bdc0a0 0x5580a2bde500 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2bc91f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bc8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94da7fc640 0 client.16473 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:28.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 --2- 172.21.15.122:0/3563108103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc27a0 0x5580a2bcb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 >> 172.21.15.122:0/3563108103 conn(0x5580a2bbced0 msgr2=0x5580a2bbec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:28.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 shutdown_connections 2024-04-02T13:37:28.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 1 -- 172.21.15.122:0/3563108103 wait complete. 2024-04-02T13:37:28.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:27.991+0000 7f94fc071740 20 client.16473 trim_cache size 0 max 16384 2024-04-02T13:37:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 Processor -- start 2024-04-02T13:37:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 -- start start 2024-04-02T13:37:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bd7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 0x5580a2bc7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41af090 con 0x5580a2bcbca0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b0c90 con 0x5580a2bc30c0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94fc071740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41af760 con 0x5580a2bc27a0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53352/0 (socket says 172.21.15.122:53352) 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 -- 172.21.15.122:0/1871295987 learned_addr learned my addr 172.21.15.122:0/1871295987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 0x5580a2bc7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bd7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 -- 172.21.15.122:0/1871295987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 msgr2=0x5580a2bd7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bd7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 -- 172.21.15.122:0/1871295987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 msgr2=0x5580a2bc7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 0x5580a2bc7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.004+0000 7f94f37fe640 1 -- 172.21.15.122:0/1871295987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580a2bc1b20 con 0x5580a2bcbca0 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bd7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94ebfff640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 0x5580a2bc7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94f37fe640 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f94ec004a80 tx=0x7f94ec01d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222b72f69c64051a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94e8ff9640 1 -- 172.21.15.122:0/1871295987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec038020 con 0x5580a2bcbca0 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94e8ff9640 1 -- 172.21.15.122:0/1871295987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec003e50 con 0x5580a2bcbca0 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94e8ff9640 1 -- 172.21.15.122:0/1871295987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec004040 con 0x5580a2bcbca0 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94fc071740 1 -- 172.21.15.122:0/1871295987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 msgr2=0x5580a2be3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.005+0000 7f94fc071740 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f94ec004a80 tx=0x7f94ec01d800 comp rx=0 tx=0).stop 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- 172.21.15.122:0/1871295987 shutdown_connections 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bcbca0 0x5580a2be3e10 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bc30c0 0x5580a2bc7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- 172.21.15.122:0/1871295987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc27a0 0x5580a2bd7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- 172.21.15.122:0/1871295987 >> 172.21.15.122:0/1871295987 conn(0x5580a2bbced0 msgr2=0x5580a2bc7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- 172.21.15.122:0/1871295987 shutdown_connections 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- 172.21.15.122:0/1871295987 wait complete. 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- start start 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 Processor -- start 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41b0500 con 0x5580a2bc27a0 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41afe30 con 0x5580a2bcbca0 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94f41ad070 con 0x5580a2bc30c0 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94fc071740 10 client.0 _ll_register_callbacks cb 0x7f94ec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53356/0 (socket says 172.21.15.122:53356) 2024-04-02T13:37:28.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 -- 172.21.15.122:0/3486127219 learned_addr learned my addr 172.21.15.122:0/3486127219 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f37fe640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 -- 172.21.15.122:0/3486127219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 msgr2=0x5580a2bcc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 -- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 msgr2=0x5580a2bccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.006+0000 7f94f2ffd640 1 -- 172.21.15.122:0/3486127219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580a2bcd200 con 0x5580a2bc27a0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94f2ffd640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94e402cfd0 tx=0x7f94e402cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6a300b3748da2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e4004040 con 0x5580a2bc27a0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94e40041e0 con 0x5580a2bc27a0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94ec003e50 con 0x5580a2bc27a0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f94e40043e0 con 0x5580a2bc27a0 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 10 client.16479 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:28.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5580a2bd3c80 con 0x5580a2bc27a0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94ec003e50 con 0x5580a2bc27a0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f94e4004a90 con 0x5580a2bc27a0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 10 client.16479 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 10 client.16479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 20 client.16479 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2c47100 0x5580a2c49520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 4 client.16479 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580a2b2e040 con 0x5580a2c47100 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.007+0000 7f94ebfff640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2c47100 0x5580a2c49520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94ebfff640 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2c47100 0x5580a2c49520 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698bc3fa87b18a6b server_cookie=130c32f43f9a2d5e in_seq=0 out_seq=0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94da7fc640 10 client.16479 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94da7fc640 1 -- 172.21.15.122:0/3486127219 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580a2b2e040 con 0x5580a2c47100 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94da7fc640 10 client.16479 handle_command_reply: tid=0 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 client.16479 shutdown 2024-04-02T13:37:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 msgr2=0x5580a2bd40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f94e402cfd0 tx=0x7f94e402cb20 comp rx=0 tx=0).stop 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 shutdown_connections 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5580a2c47100 0x5580a2c49520 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5580a2bcbca0 0x5580a2bccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5580a2bc30c0 0x5580a2bcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94da7fc640 0 client.16479 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 --2- 172.21.15.122:0/3486127219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5580a2bc27a0 0x5580a2bd40d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 >> 172.21.15.122:0/3486127219 conn(0x5580a2bbced0 msgr2=0x5580a2bc7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 shutdown_connections 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 1 -- 172.21.15.122:0/3486127219 wait complete. 2024-04-02T13:37:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.008+0000 7f94fc071740 20 client.16479 trim_cache size 0 max 16384 2024-04-02T13:37:28.020 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 msgr2=0x7f94e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 0x7f94e00a61e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f94ec005fd0 tx=0x7f94ec0096a0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 msgr2=0x7f94f4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d950 tx=0x7f94dc00de10 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 shutdown_connections 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94e00a3d20 0x7f94e00a61e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94f4106a30 0x7f94f41a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94f41016a0 0x7f94f4199670 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 --2- 172.21.15.122:0/1449347939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94f4100ce0 0x7f94f4199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 >> 172.21.15.122:0/1449347939 conn(0x7f94f40779e0 msgr2=0x7f94f40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 shutdown_connections 2024-04-02T13:37:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:28.009+0000 7f94f948f640 1 -- 172.21.15.122:0/1449347939 wait complete. 2024-04-02T13:37:28.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:27 smithi188 ceph-mon[36117]: pgmap v71: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 115 KiB/s wr, 34 op/s 2024-04-02T13:37:28.022 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:27 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1444255838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:37:28.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 71753 8.24 MB/sec warmup 49 sec latency 4366.127 ms 2024-04-02T13:37:29.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:37:29.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:37:29.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:37:29.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 -- 172.21.15.122:0/311652560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc101020 msgr2=0x7f7bcc106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/311652560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc101020 0x7f7bcc106750 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009900 tx=0x7f7bbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 -- 172.21.15.122:0/311652560 shutdown_connections 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/311652560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 0x7f7bcc10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/311652560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc101020 0x7f7bcc106750 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/311652560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 -- 172.21.15.122:0/311652560 >> 172.21.15.122:0/311652560 conn(0x7f7bcc075c60 msgr2=0x7f7bcc076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 -- 172.21.15.122:0/311652560 shutdown_connections 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.137+0000 7f7bd44d6640 1 -- 172.21.15.122:0/311652560 wait complete. 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 Processor -- start 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 -- start start 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc1015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 0x7f7bcc101b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc111f20 con 0x7f7bcc102060 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc111da0 con 0x7f7bcc100670 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd44d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1120a0 con 0x7f7bcc106e30 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34866/0 (socket says 172.21.15.122:34866) 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 -- 172.21.15.122:0/405525960 learned_addr learned my addr 172.21.15.122:0/405525960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2272640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc1015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 0x7f7bcc101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 -- 172.21.15.122:0/405525960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 msgr2=0x7f7bcc101b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 0x7f7bcc101b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 -- 172.21.15.122:0/405525960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 msgr2=0x7f7bcc1015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc1015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2a73640 1 -- 172.21.15.122:0/405525960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc1a6c90 con 0x7f7bcc102060 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.138+0000 7f7bd2272640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400cce0 tx=0x7f7bc400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bc37fe640 1 -- 172.21.15.122:0/405525960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400f450 con 0x7f7bcc102060 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc1a6f80 con 0x7f7bcc102060 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bc37fe640 1 -- 172.21.15.122:0/405525960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc400fad0 con 0x7f7bcc102060 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bc37fe640 1 -- 172.21.15.122:0/405525960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4018c10 con 0x7f7bcc102060 2024-04-02T13:37:29.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.139+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc1a7450 con 0x7f7bcc102060 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.141+0000 7f7bc37fe640 1 -- 172.21.15.122:0/405525960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7bc400f5f0 con 0x7f7bcc102060 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.141+0000 7f7bd7196740 1 Processor -- start 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 -- start start 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 0x556d1e2746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e27d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bc37fe640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 0x7f7bb80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d1e1d8770 con 0x556d1e27d7c0 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d1e180960 con 0x556d1e2742c0 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd7196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d1e1d48c0 con 0x556d1e274be0 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd1a71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 0x556d1e2746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd1a71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 0x556d1e2746a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43002/0 (socket says 172.21.15.122:43002) 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34882/0 (socket says 172.21.15.122:34882) 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd1a71640 1 -- 172.21.15.122:0/4290299603 learned_addr learned my addr 172.21.15.122:0/4290299603 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bc37fe640 1 -- 172.21.15.122:0/405525960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7bc40d1210 con 0x7f7bcc102060 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 0x7f7bb80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2272640 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e27d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 -- 172.21.15.122:0/4290299603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 msgr2=0x556d1e27d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e27d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 -- 172.21.15.122:0/4290299603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 msgr2=0x556d1e2746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 0x556d1e2746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.142+0000 7f7bd2a73640 1 -- 172.21.15.122:0/4290299603 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1dfb47e0 con 0x556d1e27d7c0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40ce5d0 tx=0x7f7bc401c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b746b1402739430 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 0x7f7bb80a6210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005940 tx=0x7f7bbc000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bc0bfe640 1 -- 172.21.15.122:0/4290299603 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc409d520 con 0x556d1e27d7c0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bc0bfe640 1 -- 172.21.15.122:0/4290299603 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc40a6950 con 0x556d1e27d7c0 2024-04-02T13:37:29.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bc0bfe640 1 -- 172.21.15.122:0/4290299603 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40aa880 con 0x556d1e27d7c0 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 -- 172.21.15.122:0/4290299603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 msgr2=0x556d1e27fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40ce5d0 tx=0x7f7bc401c590 comp rx=0 tx=0).stop 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 -- 172.21.15.122:0/4290299603 shutdown_connections 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27fbb0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e27d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 --2- 172.21.15.122:0/4290299603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e2742c0 0x556d1e2746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.143+0000 7f7bd7196740 1 -- 172.21.15.122:0/4290299603 >> 172.21.15.122:0/4290299603 conn(0x556d1e26e9f0 msgr2=0x556d1e26edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- 172.21.15.122:0/4290299603 shutdown_connections 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- 172.21.15.122:0/4290299603 wait complete. 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- start start 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 Processor -- start 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 0x556d1e27a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e27ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d1e2805b0 con 0x556d1e27d7c0 2024-04-02T13:37:29.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d1e193d20 con 0x556d1e274be0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d1e280730 con 0x556d1e2742c0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd7196740 10 client.0 _ll_register_callbacks cb 0x556d1e160a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34892/0 (socket says 172.21.15.122:34892) 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 -- 172.21.15.122:0/1655225158 learned_addr learned my addr 172.21.15.122:0/1655225158 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e27ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2272640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 0x556d1e27a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 -- 172.21.15.122:0/1655225158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 msgr2=0x556d1e27a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 0x556d1e27a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 -- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 msgr2=0x556d1e27ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.144+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e27ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd2a73640 1 -- 172.21.15.122:0/1655225158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1e014f70 con 0x556d1e27d7c0 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd2272640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 0x556d1e27a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e27ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:29.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7bc409cb20 tx=0x7f7bc400fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4aec25c67fceee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400f8e0 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7bc409d080 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc40a6950 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bc409ce20 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 10 client.16491 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d1deac170 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc40a6950 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7bc40a5600 con 0x556d1e27d7c0 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 10 client.16491 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 20 client.16491 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:29.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e28d9c0 0x556d1e28fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 4 client.16491 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.145+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d1e1e5cb0 con 0x556d1e28d9c0 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.146+0000 7f7bd2272640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e28d9c0 0x556d1e28fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.146+0000 7f7bd2272640 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e28d9c0 0x556d1e28fdb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84ca5e41f0bea12 server_cookie=b31b4908bc457c83 in_seq=0 out_seq=0 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.146+0000 7f7bb5ffb640 10 client.16491 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/1655225158 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d1e1e5cb0 con 0x556d1e28d9c0 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bb5ffb640 10 client.16491 handle_command_reply: tid=0 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 client.16491 shutdown 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 msgr2=0x556d1e27b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7bc409cb20 tx=0x7f7bc400fe90 comp rx=0 tx=0).stop 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 shutdown_connections 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e28d9c0 0x556d1e28fdb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e27d7c0 0x556d1e27b240 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bb5ffb640 0 client.16491 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e27ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 --2- 172.21.15.122:0/1655225158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e2742c0 0x556d1e27a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 >> 172.21.15.122:0/1655225158 conn(0x556d1e26e9f0 msgr2=0x556d1e270740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 shutdown_connections 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 1 -- 172.21.15.122:0/1655225158 wait complete. 2024-04-02T13:37:29.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.147+0000 7f7bd7196740 20 client.16491 trim_cache size 0 max 16384 2024-04-02T13:37:29.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.164+0000 7f7bd7196740 1 Processor -- start 2024-04-02T13:37:29.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 -- start start 2024-04-02T13:37:29.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e2894a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 0x556d1e2953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1b1060 con 0x556d1e2742c0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1b2c60 con 0x556d1e274be0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd7196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1b1730 con 0x556d1e27d7c0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34908/0 (socket says 172.21.15.122:34908) 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 -- 172.21.15.122:0/3326664381 learned_addr learned my addr 172.21.15.122:0/3326664381 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 -- 172.21.15.122:0/3326664381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 msgr2=0x556d1e2953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 0x556d1e2953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 0x556d1e2953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 -- 172.21.15.122:0/3326664381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 msgr2=0x556d1e2894a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e2894a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 -- 172.21.15.122:0/3326664381 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d1e279770 con 0x556d1e2742c0 2024-04-02T13:37:29.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 0x556d1e2953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bd2272640 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x556d1e290350 tx=0x7f7bc800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bea7a6e8e705aea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bb7fff640 1 -- 172.21.15.122:0/3326664381 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc801bdc0 con 0x556d1e2742c0 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bb7fff640 1 -- 172.21.15.122:0/3326664381 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc8020070 con 0x556d1e2742c0 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.165+0000 7f7bb7fff640 1 -- 172.21.15.122:0/3326664381 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80038e0 con 0x556d1e2742c0 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- 172.21.15.122:0/3326664381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 msgr2=0x556d1e27cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x556d1e290350 tx=0x7f7bc800b5a0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- 172.21.15.122:0/3326664381 shutdown_connections 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e27d7c0 0x556d1e2953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e274be0 0x556d1e2894a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 --2- 172.21.15.122:0/3326664381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e27cef0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- 172.21.15.122:0/3326664381 >> 172.21.15.122:0/3326664381 conn(0x556d1e26e9f0 msgr2=0x556d1e2706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- 172.21.15.122:0/3326664381 shutdown_connections 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- 172.21.15.122:0/3326664381 wait complete. 2024-04-02T13:37:29.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 -- start start 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.166+0000 7f7bd7196740 1 Processor -- start 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e281db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e27d7c0 0x556d1e2822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1b24d0 con 0x556d1e2742c0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1b1e00 con 0x556d1e27d7c0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc1af040 con 0x556d1e274be0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 10 client.0 _ll_register_callbacks cb 0x7f7bbc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34918/0 (socket says 172.21.15.122:34918) 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 -- 172.21.15.122:0/2423622626 learned_addr learned my addr 172.21.15.122:0/2423622626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e281db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2a73640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e27d7c0 0x556d1e2822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 -- 172.21.15.122:0/2423622626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 msgr2=0x556d1e281db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e281db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 -- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e27d7c0 msgr2=0x556d1e2822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e27d7c0 0x556d1e2822f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 -- 172.21.15.122:0/2423622626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d1e282830 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e281db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd2272640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b930 tx=0x7f7bc801bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103463ca7e35ca4f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8011470 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7bc8011610 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.167+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bc8020070 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bc8036050 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 10 client.16497 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d1e2f48a0 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bc8020070 con 0x556d1e2742c0 2024-04-02T13:37:29.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7bc80259c0 con 0x556d1e2742c0 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 10 client.16497 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 10 client.16497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 20 client.16497 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e2f5050 0x556d1e2f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 4 client.16497 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d1e1e0000 con 0x556d1e2f5050 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.168+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e2f5050 0x556d1e2f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.169+0000 7f7bb5ffb640 10 client.16497 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.169+0000 7f7bd1a71640 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e2f5050 0x556d1e2f7470 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d1395f54da415d server_cookie=6d621ae0d220ef6 in_seq=0 out_seq=0 2024-04-02T13:37:29.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bb5ffb640 1 -- 172.21.15.122:0/2423622626 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d1e1e0000 con 0x556d1e2f5050 2024-04-02T13:37:29.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bb5ffb640 10 client.16497 handle_command_reply: tid=0 2024-04-02T13:37:29.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 client.16497 shutdown 2024-04-02T13:37:29.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 msgr2=0x556d1e285a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b930 tx=0x7f7bc801bd00 comp rx=0 tx=0).stop 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 shutdown_connections 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d1e2f5050 0x556d1e2f7470 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d1e27d7c0 0x556d1e2822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d1e274be0 0x556d1e281db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bb5ffb640 0 client.16497 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 --2- 172.21.15.122:0/2423622626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d1e2742c0 0x556d1e285a30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 >> 172.21.15.122:0/2423622626 conn(0x556d1e26e9f0 msgr2=0x556d1e26fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 shutdown_connections 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 1 -- 172.21.15.122:0/2423622626 wait complete. 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.170+0000 7f7bd7196740 20 client.16497 trim_cache size 0 max 16384 2024-04-02T13:37:29.179 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:37:29.180 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:37:29.180 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "c1bc57ae-56a1-47f4-8a79-dcc8f2c381a3", 2024-04-02T13:37:29.180 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:37:29.180 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 msgr2=0x7f7bb80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 0x7f7bb80a6210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc005940 tx=0x7f7bbc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 msgr2=0x7f7bcc1a6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400cce0 tx=0x7f7bc400a620 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 shutdown_connections 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7bb80a3d50 0x7f7bb80a6210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7bcc102060 0x7f7bcc1a6590 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc106e30 0x7f7bcc101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 --2- 172.21.15.122:0/405525960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7bcc100670 0x7f7bcc1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 >> 172.21.15.122:0/405525960 conn(0x7f7bcc075c60 msgr2=0x7f7bcc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 shutdown_connections 2024-04-02T13:37:29.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.171+0000 7f7bd44d6640 1 -- 172.21.15.122:0/405525960 wait complete. 2024-04-02T13:37:29.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:37:29.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 -- 172.21.15.122:0/2898677700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674106980 msgr2=0x7fe674106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 --2- 172.21.15.122:0/2898677700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674106980 0x7fe674106d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009930 tx=0x7fe65c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 -- 172.21.15.122:0/2898677700 shutdown_connections 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 --2- 172.21.15.122:0/2898677700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674101330 0x7fe67410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 --2- 172.21.15.122:0/2898677700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674100970 0x7fe674100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 --2- 172.21.15.122:0/2898677700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674106980 0x7fe674106d60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 -- 172.21.15.122:0/2898677700 >> 172.21.15.122:0/2898677700 conn(0x7fe674077b00 msgr2=0x7fe674077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 -- 172.21.15.122:0/2898677700 shutdown_connections 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.292+0000 7fe6795af640 1 -- 172.21.15.122:0/2898677700 wait complete. 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 Processor -- start 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 -- start start 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674101330 0x7fe674199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674106980 0x7fe6741a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe674110d60 con 0x7fe674100970 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe674110be0 con 0x7fe674101330 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6795af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe674110ee0 con 0x7fe674106980 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34930/0 (socket says 172.21.15.122:34930) 2024-04-02T13:37:29.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe672ffd640 1 -- 172.21.15.122:0/2440398972 learned_addr learned my addr 172.21.15.122:0/2440398972 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe6737fe640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674106980 0x7fe6741a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.293+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674101330 0x7fe674199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 -- 172.21.15.122:0/2440398972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674106980 msgr2=0x7fe6741a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674106980 0x7fe6741a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 -- 172.21.15.122:0/2440398972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674101330 msgr2=0x7fe674199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674101330 0x7fe674199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 -- 172.21.15.122:0/2440398972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6741a4780 con 0x7fe674100970 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009230 tx=0x7fe65c02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe670ff9640 1 -- 172.21.15.122:0/2440398972 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c041070 con 0x7fe674100970 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe670ff9640 1 -- 172.21.15.122:0/2440398972 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c004450 con 0x7fe674100970 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6741a4950 con 0x7fe674100970 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe670ff9640 1 -- 172.21.15.122:0/2440398972 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c034440 con 0x7fe674100970 2024-04-02T13:37:29.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.294+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6741a4e10 con 0x7fe674100970 2024-04-02T13:37:29.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.296+0000 7fe670ff9640 1 -- 172.21.15.122:0/2440398972 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe65c044070 con 0x7fe674100970 2024-04-02T13:37:29.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.296+0000 7fe670ff9640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 0x7fe6600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 Processor -- start 2024-04-02T13:37:29.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 -- start start 2024-04-02T13:37:29.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe670ff9640 1 -- 172.21.15.122:0/2440398972 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe65c0f2420 con 0x7fe674100970 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 0x7fe6600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1981f0 0x55d25a1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d25a0fb830 con 0x55d25a1a0dd0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d25a0a3820 con 0x55d25a1978d0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe67c196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d25a0f77e0 con 0x55d25a1981f0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43046/0 (socket says 172.21.15.122:43046) 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34946/0 (socket says 172.21.15.122:34946) 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 -- 172.21.15.122:0/2835279538 learned_addr learned my addr 172.21.15.122:0/2835279538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 0x7fe6600a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe674069810 tx=0x7fe664005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1981f0 0x55d25a1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 -- 172.21.15.122:0/2835279538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1981f0 msgr2=0x55d25a1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1981f0 0x55d25a1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 -- 172.21.15.122:0/2835279538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 msgr2=0x55d25a1a31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe672ffd640 1 -- 172.21.15.122:0/2835279538 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d259ed97d0 con 0x55d25a1978d0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.297+0000 7fe6737fe640 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.298+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0c6880 tx=0x7fe65c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cdbc0fdcb84e96 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.298+0000 7fe6697fa640 1 -- 172.21.15.122:0/2835279538 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c04b040 con 0x55d25a1978d0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.298+0000 7fe6697fa640 1 -- 172.21.15.122:0/2835279538 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c004c80 con 0x55d25a1978d0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.298+0000 7fe6697fa640 1 -- 172.21.15.122:0/2835279538 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c0ba860 con 0x55d25a1978d0 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- 172.21.15.122:0/2835279538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 msgr2=0x55d25a197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0c6880 tx=0x7fe65c009260 comp rx=0 tx=0).stop 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- 172.21.15.122:0/2835279538 shutdown_connections 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1981f0 0x55d25a1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- 172.21.15.122:0/2835279538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1978d0 0x55d25a197cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- 172.21.15.122:0/2835279538 >> 172.21.15.122:0/2835279538 conn(0x55d25a192000 msgr2=0x55d25a1923e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- 172.21.15.122:0/2835279538 shutdown_connections 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- 172.21.15.122:0/2835279538 wait complete. 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- start start 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 Processor -- start 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a19da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1a0dd0 0x55d25a19df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d25a1a3980 con 0x55d25a1978d0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d25a0b6d30 con 0x55d25a1981f0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d25a1a3b00 con 0x55d25a1a0dd0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe67c196740 10 client.0 _ll_register_callbacks cb 0x55d25a0b6720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe672ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34954/0 (socket says 172.21.15.122:34954) 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.299+0000 7fe672ffd640 1 -- 172.21.15.122:0/2769916810 learned_addr learned my addr 172.21.15.122:0/2769916810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe6737fe640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1a0dd0 0x55d25a19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 -- 172.21.15.122:0/2769916810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1a0dd0 msgr2=0x55d25a19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1a0dd0 0x55d25a19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 -- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 msgr2=0x55d25a19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 -- 172.21.15.122:0/2769916810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d259e0f3f0 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe672ffd640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0efde0 tx=0x7fe65c0baaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb17582fa705cba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c04b050 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe65c0bc4f0 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe65c004c80 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe65c0f2270 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe67c196740 10 client.16509 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d259dcf0f0 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe65c004c80 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.300+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe65c0a72f0 con 0x55d25a1978d0 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 10 client.16509 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:29.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 20 client.16509 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a1b0940 0x55d25a1b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 4 client.16509 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d25a108d70 con 0x55d25a1b0940 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a1b0940 0x55d25a1b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe65a7fc640 10 client.16509 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.301+0000 7fe66bfff640 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a1b0940 0x55d25a1b2d30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7d49aa98eda18f server_cookie=330bb38b3e43007c in_seq=0 out_seq=0 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe65a7fc640 1 -- 172.21.15.122:0/2769916810 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d25a108d70 con 0x55d25a1b0940 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe65a7fc640 10 client.16509 handle_command_reply: tid=0 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 client.16509 shutdown 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 msgr2=0x55d25a1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe65c0efde0 tx=0x7fe65c0baaf0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 shutdown_connections 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a1b0940 0x55d25a1b2d30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1a0dd0 0x55d25a19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe65a7fc640 0 client.16509 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 --2- 172.21.15.122:0/2769916810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1978d0 0x55d25a1a0690 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 >> 172.21.15.122:0/2769916810 conn(0x55d25a192000 msgr2=0x55d25a193d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 shutdown_connections 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 1 -- 172.21.15.122:0/2769916810 wait complete. 2024-04-02T13:37:29.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.302+0000 7fe67c196740 20 client.16509 trim_cache size 0 max 16384 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.320+0000 7fe67c196740 1 Processor -- start 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.320+0000 7fe67c196740 1 -- start start 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741af040 con 0x55d25a1981f0 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741b0c40 con 0x55d25a1a0dd0 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe67c196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741af710 con 0x55d25a1978d0 2024-04-02T13:37:29.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34960/0 (socket says 172.21.15.122:34960) 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 -- 172.21.15.122:0/728874864 learned_addr learned my addr 172.21.15.122:0/728874864 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe6737fe640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe672ffd640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 -- 172.21.15.122:0/728874864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 msgr2=0x55d25a1909b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1909b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 -- 172.21.15.122:0/728874864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 msgr2=0x55d25a1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 -- 172.21.15.122:0/728874864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d25a1ad320 con 0x55d25a1981f0 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe672ffd640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe6737fe640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe66bfff640 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe664013010 tx=0x7fe664014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320dcc71079ba24e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe668ff9640 1 -- 172.21.15.122:0/728874864 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664030070 con 0x55d25a1981f0 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe668ff9640 1 -- 172.21.15.122:0/728874864 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe66401e3e0 con 0x55d25a1981f0 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.321+0000 7fe668ff9640 1 -- 172.21.15.122:0/728874864 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664003d40 con 0x55d25a1981f0 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 -- 172.21.15.122:0/728874864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 msgr2=0x55d25a1a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe664013010 tx=0x7fe664014000 comp rx=0 tx=0).stop 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 -- 172.21.15.122:0/728874864 shutdown_connections 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1981f0 0x55d25a1a0150 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 --2- 172.21.15.122:0/728874864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1909b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 -- 172.21.15.122:0/728874864 >> 172.21.15.122:0/728874864 conn(0x55d25a192000 msgr2=0x55d25a198fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 -- 172.21.15.122:0/728874864 shutdown_connections 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.322+0000 7fe67c196740 1 -- 172.21.15.122:0/728874864 wait complete. 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 -- start start 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 Processor -- start 2024-04-02T13:37:29.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1a8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a1a91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741b04b0 con 0x55d25a1a0dd0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741afde0 con 0x55d25a1981f0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6741ad020 con 0x55d25a1978d0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe67c196740 10 client.0 _ll_register_callbacks cb 0x7fe65c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34968/0 (socket says 172.21.15.122:34968) 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a1a91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 -- 172.21.15.122:0/3499779525 learned_addr learned my addr 172.21.15.122:0/3499779525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe672ffd640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1a8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 -- 172.21.15.122:0/3499779525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 msgr2=0x55d25a1a8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1a8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 -- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 msgr2=0x55d25a1a91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a1a91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 -- 172.21.15.122:0/3499779525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d25a1a6330 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe6737fe640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe66c013850 tx=0x7fe66c013d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c955d3f11f08d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.323+0000 7fe65a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c003b70 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe66c003d10 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66401e3e0 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe66c029050 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 10 client.16515 fetch_fsmap learned FSMap version 30 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d25a217750 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe66401e3e0 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe66c024030 con 0x55d25a1a0dd0 2024-04-02T13:37:29.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 10 client.16515 fetch_fsmap finished waiting for FSMap version 30 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 10 client.16515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 20 client.16515 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a218010 0x55d25a21a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 4 client.16515 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d25a191860 con 0x55d25a218010 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.324+0000 7fe672ffd640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a218010 0x55d25a21a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe65a7fc640 10 client.16515 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe672ffd640 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a218010 0x55d25a21a430 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eadb492419e9190 server_cookie=918f1e664fef9dfb in_seq=0 out_seq=0 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe65a7fc640 1 -- 172.21.15.122:0/3499779525 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d25a191860 con 0x55d25a218010 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe65a7fc640 10 client.16515 handle_command_reply: tid=0 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 client.16515 shutdown 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 msgr2=0x55d25a1a5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fe66c013850 tx=0x7fe66c013d10 comp rx=0 tx=0).stop 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 shutdown_connections 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d25a218010 0x55d25a21a430 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d25a1a0dd0 0x55d25a1a5c40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe65a7fc640 0 client.16515 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d25a1981f0 0x55d25a1a91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 --2- 172.21.15.122:0/3499779525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d25a1978d0 0x55d25a1a8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 >> 172.21.15.122:0/3499779525 conn(0x55d25a192000 msgr2=0x55d25a198710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 shutdown_connections 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 1 -- 172.21.15.122:0/3499779525 wait complete. 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.325+0000 7fe67c196740 20 client.16515 trim_cache size 0 max 16384 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:37:29.336 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: "c1bc57ae-56a1-47f4-8a79-dcc8f2c381a3": { 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "c1bc57ae-56a1-47f4-8a79-dcc8f2c381a3", 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:37:29.337 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 msgr2=0x7fe6600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 0x7fe6600a61e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe674069810 tx=0x7fe664005f90 comp rx=0 tx=0).stop 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 msgr2=0x7fe6741991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009230 tx=0x7fe65c02cba0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 shutdown_connections 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6600a3d20 0x7fe6600a61e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe674106980 0x7fe6741a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe674101330 0x7fe674199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 --2- 172.21.15.122:0/2440398972 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe674100970 0x7fe6741991c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:29.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 >> 172.21.15.122:0/2440398972 conn(0x7fe674077b00 msgr2=0x7fe674103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:29.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 shutdown_connections 2024-04-02T13:37:29.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:29.326+0000 7fe6795af640 1 -- 172.21.15.122:0/2440398972 wait complete. 2024-04-02T13:37:29.340 INFO:tasks.cephfs.filesystem:scrub status for tag:c1bc57ae-56a1-47f4-8a79-dcc8f2c381a3 - {'path': '/', 'tag': 'c1bc57ae-56a1-47f4-8a79-dcc8f2c381a3', 'options': 'recursive,force'} 2024-04-02T13:37:29.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73758 8.29 MB/sec warmup 50 sec latency 4627.003 ms 2024-04-02T13:37:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: pgmap v72: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 86 KiB/s wr, 28 op/s 2024-04-02T13:37:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:37:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: pgmap v72: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 86 KiB/s wr, 28 op/s 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: pgmap v72: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 86 KiB/s wr, 28 op/s 2024-04-02T13:37:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73873 8.13 MB/sec warmup 51 sec latency 976.898 ms 2024-04-02T13:37:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:30 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:37:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:30 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:30.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:30 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:30 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:37:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:30 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:31.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:30 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:30 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:37:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:30 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:37:31.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:30 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:31.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73873 7.98 MB/sec warmup 52 sec latency 1976.985 ms 2024-04-02T13:37:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:31 smithi187 ceph-mon[36064]: pgmap v73: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-04-02T13:37:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:31 smithi188 ceph-mon[36117]: pgmap v73: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-04-02T13:37:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:31 smithi122 ceph-mon[32114]: pgmap v73: 129 pgs: 129 active+clean; 50 MiB data, 569 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 82 KiB/s wr, 27 op/s 2024-04-02T13:37:32.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73873 7.82 MB/sec warmup 53 sec latency 2977.036 ms 2024-04-02T13:37:33.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73924 7.70 MB/sec warmup 54 sec latency 3434.359 ms 2024-04-02T13:37:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:33 smithi187 ceph-mon[36064]: pgmap v74: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 205 KiB/s wr, 54 op/s 2024-04-02T13:37:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:33 smithi188 ceph-mon[36117]: pgmap v74: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 205 KiB/s wr, 54 op/s 2024-04-02T13:37:34.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:33 smithi122 ceph-mon[32114]: pgmap v74: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 205 KiB/s wr, 54 op/s 2024-04-02T13:37:34.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73924 7.56 MB/sec warmup 55 sec latency 1500.637 ms 2024-04-02T13:37:35.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73924 7.43 MB/sec warmup 56 sec latency 2500.744 ms 2024-04-02T13:37:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:35 smithi187 ceph-mon[36064]: pgmap v75: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 133 KiB/s wr, 33 op/s 2024-04-02T13:37:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:35 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:35 smithi188 ceph-mon[36117]: pgmap v75: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 133 KiB/s wr, 33 op/s 2024-04-02T13:37:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:35 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:35 smithi122 ceph-mon[32114]: pgmap v75: 129 pgs: 129 active+clean; 51 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 133 KiB/s wr, 33 op/s 2024-04-02T13:37:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:35 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:37:36.941 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73924 7.30 MB/sec warmup 57 sec latency 3500.867 ms 2024-04-02T13:37:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:37.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:37.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:37.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:37.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:37.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 73924 7.17 MB/sec warmup 58 sec latency 4500.944 ms 2024-04-02T13:37:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:37 smithi187 ceph-mon[36064]: pgmap v76: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 138 KiB/s wr, 39 op/s 2024-04-02T13:37:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:37:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:37 smithi188 ceph-mon[36117]: pgmap v76: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 138 KiB/s wr, 39 op/s 2024-04-02T13:37:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:37:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:37 smithi122 ceph-mon[32114]: pgmap v76: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 138 KiB/s wr, 39 op/s 2024-04-02T13:37:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:37:38.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74017 7.05 MB/sec warmup 59 sec latency 4965.630 ms 2024-04-02T13:37:39.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74017 6.93 MB/sec warmup 60 sec latency 1456.591 ms 2024-04-02T13:37:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:39 smithi188 ceph-mon[36117]: pgmap v77: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 132 KiB/s wr, 33 op/s 2024-04-02T13:37:40.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:39 smithi122 ceph-mon[32114]: pgmap v77: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 132 KiB/s wr, 33 op/s 2024-04-02T13:37:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:39 smithi187 ceph-mon[36064]: pgmap v77: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 132 KiB/s wr, 33 op/s 2024-04-02T13:37:40.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74017 6.82 MB/sec warmup 61 sec latency 2456.714 ms 2024-04-02T13:37:41.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74017 6.71 MB/sec warmup 62 sec latency 3456.777 ms 2024-04-02T13:37:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:41 smithi188 ceph-mon[36117]: pgmap v78: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 129 KiB/s wr, 33 op/s 2024-04-02T13:37:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:41 smithi122 ceph-mon[32114]: pgmap v78: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 129 KiB/s wr, 33 op/s 2024-04-02T13:37:42.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:41 smithi187 ceph-mon[36064]: pgmap v78: 129 pgs: 129 active+clean; 51 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 407 KiB/s rd, 129 KiB/s wr, 33 op/s 2024-04-02T13:37:42.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74017 6.60 MB/sec warmup 63 sec latency 4456.824 ms 2024-04-02T13:37:43.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:42 smithi188 ceph-mon[36117]: pgmap v79: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 684 KiB/s wr, 61 op/s 2024-04-02T13:37:43.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:42 smithi122 ceph-mon[32114]: pgmap v79: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 684 KiB/s wr, 61 op/s 2024-04-02T13:37:43.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:42 smithi187 ceph-mon[36064]: pgmap v79: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 684 KiB/s wr, 61 op/s 2024-04-02T13:37:43.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74202 6.51 MB/sec warmup 64 sec latency 4911.166 ms 2024-04-02T13:37:44.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74202 6.41 MB/sec warmup 65 sec latency 1499.780 ms 2024-04-02T13:37:45.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:44 smithi187 ceph-mon[36064]: pgmap v80: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 561 KiB/s wr, 34 op/s 2024-04-02T13:37:45.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:44 smithi188 ceph-mon[36117]: pgmap v80: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 561 KiB/s wr, 34 op/s 2024-04-02T13:37:45.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:44 smithi122 ceph-mon[32114]: pgmap v80: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 561 KiB/s wr, 34 op/s 2024-04-02T13:37:45.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74202 6.32 MB/sec warmup 66 sec latency 2499.891 ms 2024-04-02T13:37:46.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74202 6.22 MB/sec warmup 67 sec latency 3499.954 ms 2024-04-02T13:37:47.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:46 smithi187 ceph-mon[36064]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 570 KiB/s wr, 40 op/s 2024-04-02T13:37:47.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:46 smithi188 ceph-mon[36117]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 570 KiB/s wr, 40 op/s 2024-04-02T13:37:47.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:46 smithi122 ceph-mon[32114]: pgmap v81: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 570 KiB/s wr, 40 op/s 2024-04-02T13:37:47.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74202 6.13 MB/sec warmup 68 sec latency 4500.038 ms 2024-04-02T13:37:48.942 INFO:tasks.workunit.client.0.smithi122.stdout: 1 74951 6.12 MB/sec warmup 69 sec latency 4960.394 ms 2024-04-02T13:37:49.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:48 smithi187 ceph-mon[36064]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 565 KiB/s wr, 36 op/s 2024-04-02T13:37:49.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:48 smithi188 ceph-mon[36117]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 565 KiB/s wr, 36 op/s 2024-04-02T13:37:49.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:48 smithi122 ceph-mon[32114]: pgmap v82: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 565 KiB/s wr, 36 op/s 2024-04-02T13:37:49.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75265 6.04 MB/sec warmup 70 sec latency 720.218 ms 2024-04-02T13:37:50.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75265 5.96 MB/sec warmup 71 sec latency 1720.260 ms 2024-04-02T13:37:51.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:50 smithi187 ceph-mon[36064]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 564 KiB/s wr, 35 op/s 2024-04-02T13:37:51.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:50 smithi188 ceph-mon[36117]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 564 KiB/s wr, 35 op/s 2024-04-02T13:37:51.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:50 smithi122 ceph-mon[32114]: pgmap v83: 129 pgs: 129 active+clean; 53 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 564 KiB/s wr, 35 op/s 2024-04-02T13:37:51.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75265 5.87 MB/sec warmup 72 sec latency 2720.388 ms 2024-04-02T13:37:52.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75265 5.79 MB/sec warmup 73 sec latency 3720.441 ms 2024-04-02T13:37:53.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:52 smithi187 ceph-mon[36064]: pgmap v84: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 634 KiB/s wr, 56 op/s 2024-04-02T13:37:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:52 smithi188 ceph-mon[36117]: pgmap v84: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 634 KiB/s wr, 56 op/s 2024-04-02T13:37:53.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:52 smithi122 ceph-mon[32114]: pgmap v84: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 634 KiB/s wr, 56 op/s 2024-04-02T13:37:53.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 75655 5.74 MB/sec warmup 74 sec latency 4181.653 ms 2024-04-02T13:37:54.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78310 5.89 MB/sec warmup 75 sec latency 9.615 ms 2024-04-02T13:37:55.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:54 smithi187 ceph-mon[36064]: pgmap v85: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 80 KiB/s wr, 28 op/s 2024-04-02T13:37:55.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:54 smithi188 ceph-mon[36117]: pgmap v85: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 80 KiB/s wr, 28 op/s 2024-04-02T13:37:55.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:54 smithi122 ceph-mon[32114]: pgmap v85: 129 pgs: 129 active+clean; 53 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 80 KiB/s wr, 28 op/s 2024-04-02T13:37:55.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78753 5.81 MB/sec warmup 76 sec latency 503.376 ms 2024-04-02T13:37:56.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78753 5.74 MB/sec warmup 77 sec latency 1503.414 ms 2024-04-02T13:37:57.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:56 smithi187 ceph-mon[36064]: pgmap v86: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 360 KiB/s wr, 74 op/s 2024-04-02T13:37:57.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:56 smithi188 ceph-mon[36117]: pgmap v86: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 360 KiB/s wr, 74 op/s 2024-04-02T13:37:57.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:56 smithi122 ceph-mon[32114]: pgmap v86: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 511 KiB/s rd, 360 KiB/s wr, 74 op/s 2024-04-02T13:37:57.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78753 5.66 MB/sec warmup 78 sec latency 2503.503 ms 2024-04-02T13:37:58.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78809 5.59 MB/sec warmup 79 sec latency 2965.760 ms 2024-04-02T13:37:59.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:58 smithi187 ceph-mon[36064]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 422 KiB/s wr, 73 op/s 2024-04-02T13:37:59.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:58 smithi188 ceph-mon[36117]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 422 KiB/s wr, 73 op/s 2024-04-02T13:37:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:58 smithi122 ceph-mon[32114]: pgmap v87: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 422 KiB/s wr, 73 op/s 2024-04-02T13:37:59.341 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:37:59.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- 172.21.15.122:0/3076045524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 msgr2=0x7f1038074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 --2- 172.21.15.122:0/3076045524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f1038074280 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1034009900 tx=0x7f103402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- 172.21.15.122:0/3076045524 shutdown_connections 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 --2- 172.21.15.122:0/3076045524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1038112ab0 0x7f1038114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 --2- 172.21.15.122:0/3076045524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10380747c0 0x7f1038112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 --2- 172.21.15.122:0/3076045524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f1038074280 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- 172.21.15.122:0/3076045524 >> 172.21.15.122:0/3076045524 conn(0x7f10380fd530 msgr2=0x7f10380ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- 172.21.15.122:0/3076045524 shutdown_connections 2024-04-02T13:37:59.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- 172.21.15.122:0/3076045524 wait complete. 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 Processor -- start 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.462+0000 7f103f8eb640 1 -- start start 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10380747c0 0x7f10381a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 0x7f103819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1038118480 con 0x7f1038073ea0 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1038118300 con 0x7f1038112ab0 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1038118600 con 0x7f10380747c0 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43426/0 (socket says 172.21.15.122:43426) 2024-04-02T13:37:59.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 -- 172.21.15.122:0/2012087531 learned_addr learned my addr 172.21.15.122:0/2012087531 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103de88640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 0x7f103819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103ce86640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10380747c0 0x7f10381a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 -- 172.21.15.122:0/2012087531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10380747c0 msgr2=0x7f10381a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10380747c0 0x7f10381a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 -- 172.21.15.122:0/2012087531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 msgr2=0x7f103819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 0x7f103819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 -- 172.21.15.122:0/2012087531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103819f010 con 0x7f1038073ea0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103de88640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 0x7f103819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103d687640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1034005940 tx=0x7f103402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f102e7fc640 1 -- 172.21.15.122:0/2012087531 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034041070 con 0x7f1038073ea0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103819f2a0 con 0x7f1038073ea0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f102e7fc640 1 -- 172.21.15.122:0/2012087531 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1034004450 con 0x7f1038073ea0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.463+0000 7f102e7fc640 1 -- 172.21.15.122:0/2012087531 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034034440 con 0x7f1038073ea0 2024-04-02T13:37:59.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.464+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10381ab520 con 0x7f1038073ea0 2024-04-02T13:37:59.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.466+0000 7f102e7fc640 1 -- 172.21.15.122:0/2012087531 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1034044070 con 0x7f1038073ea0 2024-04-02T13:37:59.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.466+0000 7f10425aa740 1 Processor -- start 2024-04-02T13:37:59.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.466+0000 7f102e7fc640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 0x7f10240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 -- start start 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f102e7fc640 1 -- 172.21.15.122:0/2012087531 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f10340f27b0 con 0x7f1038073ea0 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 0x7f10240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb387dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55acb37e1830 con 0x55acb3886dd0 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55acb3789820 con 0x55acb387e1f0 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f10425aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55acb37dd7e0 con 0x55acb387d8d0 2024-04-02T13:37:59.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103ce86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103ce86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47022/0 (socket says 172.21.15.122:47022) 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103ce86640 1 -- 172.21.15.122:0/3480034893 learned_addr learned my addr 172.21.15.122:0/3480034893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43430/0 (socket says 172.21.15.122:43430) 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103d687640 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb387dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 0x7f10240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f103806b150 tx=0x7f103000e720 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 -- 172.21.15.122:0/3480034893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 msgr2=0x55acb387dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb387dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 -- 172.21.15.122:0/3480034893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 msgr2=0x55acb3886890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.467+0000 7f103de88640 1 -- 172.21.15.122:0/3480034893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb35bf7d0 con 0x55acb3886dd0 2024-04-02T13:37:59.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.468+0000 7f103de88640 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1030012500 tx=0x7f1030019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61437d27a6ba24e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.468+0000 7f10237fe640 1 -- 172.21.15.122:0/3480034893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030003b70 con 0x55acb3886dd0 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.468+0000 7f10237fe640 1 -- 172.21.15.122:0/3480034893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1030003d10 con 0x55acb3886dd0 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.468+0000 7f10237fe640 1 -- 172.21.15.122:0/3480034893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103001dc00 con 0x55acb3886dd0 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- 172.21.15.122:0/3480034893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 msgr2=0x55acb38891c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1030012500 tx=0x7f1030019000 comp rx=0 tx=0).stop 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- 172.21.15.122:0/3480034893 shutdown_connections 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb38891c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 --2- 172.21.15.122:0/3480034893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb387dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- 172.21.15.122:0/3480034893 >> 172.21.15.122:0/3480034893 conn(0x55acb3878000 msgr2=0x55acb38783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- 172.21.15.122:0/3480034893 shutdown_connections 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- 172.21.15.122:0/3480034893 wait complete. 2024-04-02T13:37:59.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 -- start start 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.469+0000 7f10425aa740 1 Processor -- start 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb38839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb3883f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55acb3889bc0 con 0x55acb387d8d0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55acb379cd30 con 0x55acb387e1f0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55acb3889d40 con 0x55acb3886dd0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 10 client.0 _ll_register_callbacks cb 0x55acb379c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43442/0 (socket says 172.21.15.122:43442) 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 -- 172.21.15.122:0/1341947450 learned_addr learned my addr 172.21.15.122:0/1341947450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103ce86640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb38839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103de88640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb3883f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 -- 172.21.15.122:0/1341947450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 msgr2=0x55acb3883f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb3883f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 -- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 msgr2=0x55acb38839f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb38839f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 -- 172.21.15.122:0/1341947450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb34f53f0 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103ce86640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb38839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103de88640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb3883f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f103d687640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10340be460 tx=0x7f10340bdef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5badbe37e89108d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f1020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103404b050 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f10340f23d0 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.470+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1030003d10 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10340f25d0 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 10 client.16527 fetch_fsmap learned FSMap version 32 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55acb34b50f0 con 0x55acb387d8d0 2024-04-02T13:37:59.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1030003d10 con 0x55acb387d8d0 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f10340bd920 con 0x55acb387d8d0 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 10 client.16527 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 20 client.16527 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb3896d30 0x55acb3899190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 4 client.16527 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb37eed70 con 0x55acb3896d30 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.471+0000 7f103ce86640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb3896d30 0x55acb3899190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.472+0000 7f1020ff9640 10 client.16527 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.472+0000 7f103ce86640 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb3896d30 0x55acb3899190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8293d0d9651a86 server_cookie=768235d363308b71 in_seq=0 out_seq=0 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.472+0000 7f1020ff9640 1 -- 172.21.15.122:0/1341947450 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acb37eed70 con 0x55acb3896d30 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.472+0000 7f1020ff9640 10 client.16527 handle_command_reply: tid=0 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.472+0000 7f10425aa740 1 client.16527 shutdown 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 msgr2=0x55acb3886660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f10340be460 tx=0x7f10340bdef0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 shutdown_connections 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb3896d30 0x55acb3899190 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb3883f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb38839f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f1020ff9640 0 client.16527 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 --2- 172.21.15.122:0/1341947450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387d8d0 0x55acb3886660 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 >> 172.21.15.122:0/1341947450 conn(0x55acb3878000 msgr2=0x55acb3879d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 shutdown_connections 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 1 -- 172.21.15.122:0/1341947450 wait complete. 2024-04-02T13:37:59.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.473+0000 7f10425aa740 20 client.16527 trim_cache size 0 max 16384 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 Processor -- start 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 -- start start 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb3892ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b5590 con 0x55acb3886dd0 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b71f0 con 0x55acb387e1f0 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f10425aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b5c60 con 0x55acb387d8d0 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43454/0 (socket says 172.21.15.122:43454) 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 -- 172.21.15.122:0/3372511237 learned_addr learned my addr 172.21.15.122:0/3372511237 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103d687640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb3892ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103ce86640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 -- 172.21.15.122:0/3372511237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 msgr2=0x55acb3892ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb3892ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 -- 172.21.15.122:0/3372511237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 msgr2=0x55acb3886120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103de88640 1 -- 172.21.15.122:0/3372511237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb3899810 con 0x55acb3886dd0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.491+0000 7f103d687640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb3892ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f103de88640 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f103000a3f0 tx=0x7f1030004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0e416d939dc4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f1022ffd640 1 -- 172.21.15.122:0/3372511237 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030004d20 con 0x55acb3886dd0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f1022ffd640 1 -- 172.21.15.122:0/3372511237 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103002a070 con 0x55acb3886dd0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f1022ffd640 1 -- 172.21.15.122:0/3372511237 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10300056e0 con 0x55acb3886dd0 2024-04-02T13:37:59.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 -- 172.21.15.122:0/3372511237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 msgr2=0x55acb3886660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f103000a3f0 tx=0x7f1030004ab0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 -- 172.21.15.122:0/3372511237 shutdown_connections 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb3886dd0 0x55acb3886660 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387e1f0 0x55acb3886120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 --2- 172.21.15.122:0/3372511237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb387d8d0 0x55acb3892ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 -- 172.21.15.122:0/3372511237 >> 172.21.15.122:0/3372511237 conn(0x55acb3878000 msgr2=0x55acb3877240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 -- 172.21.15.122:0/3372511237 shutdown_connections 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.492+0000 7f10425aa740 1 -- 172.21.15.122:0/3372511237 wait complete. 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 -- start start 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 Processor -- start 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387d8d0 0x55acb388abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb388b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b6a60 con 0x55acb387e1f0 2024-04-02T13:37:59.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b6390 con 0x55acb387d8d0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10381b3570 con 0x55acb3886dd0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f10425aa740 10 client.0 _ll_register_callbacks cb 0x7f10340be010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f103ce86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f103ce86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43470/0 (socket says 172.21.15.122:43470) 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.493+0000 7f103ce86640 1 -- 172.21.15.122:0/3281562291 learned_addr learned my addr 172.21.15.122:0/3281562291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103d687640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387d8d0 0x55acb388abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103de88640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb388b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 -- 172.21.15.122:0/3281562291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 msgr2=0x55acb388b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb388b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 -- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387d8d0 msgr2=0x55acb388abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387d8d0 0x55acb388abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 -- 172.21.15.122:0/3281562291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb3877e30 con 0x55acb387e1f0 2024-04-02T13:37:59.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f103ce86640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55acb387d3a0 tx=0x7f102801a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975e72ad95fe5576 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f1020ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102801a940 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f102801adc0 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103002a070 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f102802c050 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f10425aa740 10 client.16533 fetch_fsmap learned FSMap version 32 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55acb38fd6c0 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.494+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103002a070 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f102800f040 con 0x55acb387e1f0 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 10 client.16533 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 10 client.16533 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 20 client.16533 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb38fdf00 0x55acb3900320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 4 client.16533 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:37:59.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acb37e90c0 con 0x55acb38fdf00 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f103d687640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb38fdf00 0x55acb3900320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f1020ff9640 10 client.16533 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f103d687640 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb38fdf00 0x55acb3900320 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3528ebc34e79ead7 server_cookie=61120f784d275eb in_seq=0 out_seq=0 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f1020ff9640 1 -- 172.21.15.122:0/3281562291 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55acb37e90c0 con 0x55acb38fdf00 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.495+0000 7f1020ff9640 10 client.16533 handle_command_reply: tid=0 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 client.16533 shutdown 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 msgr2=0x55acb388b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x55acb387d3a0 tx=0x7f102801a600 comp rx=0 tx=0).stop 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 shutdown_connections 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55acb38fdf00 0x55acb3900320 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55acb3886dd0 0x55acb388b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55acb387e1f0 0x55acb388b130 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f1020ff9640 0 client.16533 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 --2- 172.21.15.122:0/3281562291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55acb387d8d0 0x55acb388abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 >> 172.21.15.122:0/3281562291 conn(0x55acb3878000 msgr2=0x55acb387e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 shutdown_connections 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 1 -- 172.21.15.122:0/3281562291 wait complete. 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f10425aa740 20 client.16533 trim_cache size 0 max 16384 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:37:59.505 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:37:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 msgr2=0x7f10240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 0x7f10240a61e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f103806b150 tx=0x7f103000e720 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 msgr2=0x7f10381a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1034005940 tx=0x7f103402cd10 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 shutdown_connections 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f10240a3d20 0x7f10240a61e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1038112ab0 0x7f103819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10380747c0 0x7f10381a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 --2- 172.21.15.122:0/2012087531 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1038073ea0 0x7f10381a46d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.496+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 >> 172.21.15.122:0/2012087531 conn(0x7f10380fd530 msgr2=0x7f10380fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.497+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 shutdown_connections 2024-04-02T13:37:59.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.497+0000 7f103f8eb640 1 -- 172.21.15.122:0/2012087531 wait complete. 2024-04-02T13:37:59.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:37:59.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:37:59.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- 172.21.15.122:0/2135700032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f40106a10 msgr2=0x7f2f40106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 --2- 172.21.15.122:0/2135700032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f40106a10 0x7f2f40106df0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009930 tx=0x7f2f3402bc30 comp rx=0 tx=0).stop 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- 172.21.15.122:0/2135700032 shutdown_connections 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 --2- 172.21.15.122:0/2135700032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f401013c0 0x7f2f4010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 --2- 172.21.15.122:0/2135700032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 --2- 172.21.15.122:0/2135700032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f40106a10 0x7f2f40106df0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- 172.21.15.122:0/2135700032 >> 172.21.15.122:0/2135700032 conn(0x7f2f40077b00 msgr2=0x7f2f40077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- 172.21.15.122:0/2135700032 shutdown_connections 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- 172.21.15.122:0/2135700032 wait complete. 2024-04-02T13:37:59.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 Processor -- start 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.613+0000 7f2f45d50640 1 -- start start 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 0x7f2f4019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40110e20 con 0x7f2f401013c0 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40110ca0 con 0x7f2f40106a10 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40110fa0 con 0x7f2f40100a00 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43476/0 (socket says 172.21.15.122:43476) 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 -- 172.21.15.122:0/3541278908 learned_addr learned my addr 172.21.15.122:0/3541278908 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3f7fe640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3ffff640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 0x7f2f4019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 -- 172.21.15.122:0/3541278908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 msgr2=0x7f2f40197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 -- 172.21.15.122:0/3541278908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 msgr2=0x7f2f4019bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 0x7f2f4019bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 -- 172.21.15.122:0/3541278908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4019c2f0 con 0x7f2f401013c0 2024-04-02T13:37:59.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3ffff640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 0x7f2f4019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3f7fe640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3effd640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d770 tx=0x7f2f3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30017070 con 0x7f2f401013c0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4019c5e0 con 0x7f2f401013c0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f30005500 con 0x7f2f401013c0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30004e00 con 0x7f2f401013c0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.614+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f401a3e70 con 0x7f2f401013c0 2024-04-02T13:37:59.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.615+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f400688f0 con 0x7f2f401013c0 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.616+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2f30004fa0 con 0x7f2f401013c0 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.617+0000 7f2f3cff9640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 0x7f2f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.617+0000 7f2f3f7fe640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 0x7f2f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.617+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2f300d0a80 con 0x7f2f401013c0 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.617+0000 7f2f3f7fe640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 0x7f2f280a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006010 tx=0x7f2f3402c7d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.621+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f2f3009cd90 con 0x7f2f401013c0 2024-04-02T13:37:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.838+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f40108fa0 con 0x7f2f401013c0 2024-04-02T13:37:59.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.838+0000 7f2f3cff9640 1 -- 172.21.15.122:0/3541278908 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7f2f3009c690 con 0x7f2f401013c0 2024-04-02T13:37:59.841 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:37:59.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:37:34.559237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:37:59.841 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 32 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 msgr2=0x7f2f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 0x7f2f280a61e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2f34006010 tx=0x7f2f3402c7d0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 msgr2=0x7f2f40197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000d770 tx=0x7f2f3000dc30 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 shutdown_connections 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2f280a3d20 0x7f2f280a61e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2f40106a10 0x7f2f4019bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f401013c0 0x7f2f40197840 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 --2- 172.21.15.122:0/3541278908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f40100a00 0x7f2f40197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 >> 172.21.15.122:0/3541278908 conn(0x7f2f40077b00 msgr2=0x7f2f40103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 shutdown_connections 2024-04-02T13:37:59.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.841+0000 7f2f45d50640 1 -- 172.21.15.122:0/3541278908 wait complete. 2024-04-02T13:37:59.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:37:59.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78809 5.52 MB/sec warmup 80 sec latency 1494.714 ms 2024-04-02T13:37:59.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 -- 172.21.15.122:0/1126439300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 msgr2=0x7f565c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 --2- 172.21.15.122:0/1126439300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c10d850 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5654009cd0 tx=0x7f565402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 -- 172.21.15.122:0/1126439300 shutdown_connections 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 --2- 172.21.15.122:0/1126439300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c10d850 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 --2- 172.21.15.122:0/1126439300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 0x7f565c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 --2- 172.21.15.122:0/1126439300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 0x7f565c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 -- 172.21.15.122:0/1126439300 >> 172.21.15.122:0/1126439300 conn(0x7f565c077ad0 msgr2=0x7f565c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 -- 172.21.15.122:0/1126439300 shutdown_connections 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.956+0000 7f5661a35640 1 -- 172.21.15.122:0/1126439300 wait complete. 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 Processor -- start 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 -- start start 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 0x7f565c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 0x7f565c1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110da0 con 0x7f565c101350 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110c20 con 0x7f565c106990 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5661a35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f565c110f20 con 0x7f565c100990 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43498/0 (socket says 172.21.15.122:43498) 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f5653fff640 1 -- 172.21.15.122:0/878749868 learned_addr learned my addr 172.21.15.122:0/878749868 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f565b7fe640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 0x7f565c1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.957+0000 7f565affd640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 0x7f565c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 -- 172.21.15.122:0/878749868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 msgr2=0x7f565c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 0x7f565c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 -- 172.21.15.122:0/878749868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 msgr2=0x7f565c1a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 0x7f565c1a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 -- 172.21.15.122:0/878749868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565c1a47a0 con 0x7f565c101350 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5653fff640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f564400e9f0 tx=0x7f564400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5660a33640 1 -- 172.21.15.122:0/878749868 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564400cd80 con 0x7f565c101350 2024-04-02T13:37:59.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5660a33640 1 -- 172.21.15.122:0/878749868 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5644005500 con 0x7f565c101350 2024-04-02T13:37:59.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565c1a4970 con 0x7f565c101350 2024-04-02T13:37:59.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5660a33640 1 -- 172.21.15.122:0/878749868 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644010690 con 0x7f565c101350 2024-04-02T13:37:59.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.958+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f565c1a4e70 con 0x7f565c101350 2024-04-02T13:37:59.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 Processor -- start 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f5660a33640 1 -- 172.21.15.122:0/878749868 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f56440046c0 con 0x7f565c101350 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 -- start start 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77457a0 0x5584f7745b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f7751090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5584f76a97b0 con 0x5584f77460c0 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5584f7648570 con 0x5584f77457a0 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f566461d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584f76a5760 con 0x5584f774eca0 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43512/0 (socket says 172.21.15.122:43512) 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565affd640 1 -- 172.21.15.122:0/580412312 learned_addr learned my addr 172.21.15.122:0/580412312 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565b7fe640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77457a0 0x5584f7745b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f5653fff640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f7751090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565affd640 1 -- 172.21.15.122:0/580412312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 msgr2=0x5584f7751090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.961+0000 7f565affd640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f7751090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f565affd640 1 -- 172.21.15.122:0/580412312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77457a0 msgr2=0x5584f7745b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f565affd640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77457a0 0x5584f7745b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f565affd640 1 -- 172.21.15.122:0/580412312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f73bd3f0 con 0x5584f77460c0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5653fff640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f7751090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5660a33640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 0x7f56480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f565b7fe640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 0x7f56480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f565affd640 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f564c00bc50 tx=0x7f564c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f832197de18aa925 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5660a33640 1 -- 172.21.15.122:0/878749868 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f56440d0fa0 con 0x7f565c101350 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5651ffb640 1 -- 172.21.15.122:0/580412312 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c01c480 con 0x5584f77460c0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5651ffb640 1 -- 172.21.15.122:0/580412312 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c01cb00 con 0x5584f77460c0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.962+0000 7f5651ffb640 1 -- 172.21.15.122:0/580412312 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c011ce0 con 0x5584f77460c0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- 172.21.15.122:0/580412312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 msgr2=0x5584f774e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f565b7fe640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 0x7f56480a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5654005cc0 tx=0x7f5654005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f564c00bc50 tx=0x7f564c0107a0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- 172.21.15.122:0/580412312 shutdown_connections 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f7751090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f774e760 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- 172.21.15.122:0/580412312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77457a0 0x5584f7745b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- 172.21.15.122:0/580412312 >> 172.21.15.122:0/580412312 conn(0x5584f773fed0 msgr2=0x5584f77402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- 172.21.15.122:0/580412312 shutdown_connections 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- 172.21.15.122:0/580412312 wait complete. 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- start start 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 Processor -- start 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f774c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f774c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5584f7751a90 con 0x5584f77457a0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5584f7664e80 con 0x5584f77460c0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5584f7751c10 con 0x5584f774eca0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f566461d740 10 client.0 _ll_register_callbacks cb 0x5584f76a0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.963+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43526/0 (socket says 172.21.15.122:43526) 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 -- 172.21.15.122:0/1081287030 learned_addr learned my addr 172.21.15.122:0/1081287030 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f5653fff640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f774c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565b7fe640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f774c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 -- 172.21.15.122:0/1081287030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 msgr2=0x5584f774c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f774c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 -- 172.21.15.122:0/1081287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 msgr2=0x5584f774c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f774c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 -- 172.21.15.122:0/1081287030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f74877d0 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565b7fe640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f774c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f565affd640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f564c011010 tx=0x7f564c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24b29dd202988ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f563affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c011df0 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f564c01c480 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c01cb00 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f564c032050 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f566461d740 10 client.16551 fetch_fsmap learned FSMap version 32 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5584f775dad0 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.964+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c01cb00 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f564c02d030 con 0x5584f77457a0 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 10 client.16551 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 10 client.16551 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 20 client.16551 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:59.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f775ef30 0x5584f7761320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 4 client.16551 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584f76b6cf0 con 0x5584f775ef30 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f5653fff640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f775ef30 0x5584f7761320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f563affd640 10 client.16551 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.965+0000 7f5653fff640 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f775ef30 0x5584f7761320 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb37a6749846ca0d server_cookie=5e14b889cc6d5cfe in_seq=0 out_seq=0 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f563affd640 1 -- 172.21.15.122:0/1081287030 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584f76b6cf0 con 0x5584f775ef30 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f563affd640 10 client.16551 handle_command_reply: tid=0 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 client.16551 shutdown 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 msgr2=0x5584f774bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f564c011010 tx=0x7f564c0114b0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 shutdown_connections 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f775ef30 0x5584f7761320 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f774c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f774c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 --2- 172.21.15.122:0/1081287030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774bb00 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f563affd640 0 client.16551 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:59.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 >> 172.21.15.122:0/1081287030 conn(0x5584f773fed0 msgr2=0x5584f7741c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 shutdown_connections 2024-04-02T13:37:59.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 1 -- 172.21.15.122:0/1081287030 wait complete. 2024-04-02T13:37:59.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.966+0000 7f566461d740 20 client.16551 trim_cache size 0 max 16384 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 Processor -- start 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 -- start start 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f775a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f773ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1af020 con 0x5584f77457a0 2024-04-02T13:37:59.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b0c80 con 0x5584f77460c0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f566461d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1af6f0 con 0x5584f774eca0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f565affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43538/0 (socket says 172.21.15.122:43538) 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f565affd640 1 -- 172.21.15.122:0/1876053291 learned_addr learned my addr 172.21.15.122:0/1876053291 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.979+0000 7f565b7fe640 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f773ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 -- 172.21.15.122:0/1876053291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 msgr2=0x5584f773ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f773ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 -- 172.21.15.122:0/1876053291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 msgr2=0x5584f775a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f775a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 -- 172.21.15.122:0/1876053291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584f773f470 con 0x5584f77457a0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f565affd640 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b470 tx=0x7f564c011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3a0553e5813b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f56517fa640 1 -- 172.21.15.122:0/1876053291 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c03a070 con 0x5584f77457a0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f56517fa640 1 -- 172.21.15.122:0/1876053291 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c011bf0 con 0x5584f77457a0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f56517fa640 1 -- 172.21.15.122:0/1876053291 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c025870 con 0x5584f77457a0 2024-04-02T13:37:59.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 -- 172.21.15.122:0/1876053291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 msgr2=0x5584f774e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f564c00b470 tx=0x7f564c011870 comp rx=0 tx=0).stop 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 -- 172.21.15.122:0/1876053291 shutdown_connections 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f774eca0 0x5584f773ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f77460c0 0x5584f775a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 --2- 172.21.15.122:0/1876053291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77457a0 0x5584f774e230 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 -- 172.21.15.122:0/1876053291 >> 172.21.15.122:0/1876053291 conn(0x5584f773fed0 msgr2=0x5584f77402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 -- 172.21.15.122:0/1876053291 shutdown_connections 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.980+0000 7f566461d740 1 -- 172.21.15.122:0/1876053291 wait complete. 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 -- start start 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 Processor -- start 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f77457a0 0x5584f7756fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f774eca0 0x5584f7752960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1b04f0 con 0x5584f77460c0 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1afe20 con 0x5584f774eca0 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f565c1ad000 con 0x5584f77457a0 2024-04-02T13:37:59.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f566461d740 10 client.0 _ll_register_callbacks cb 0x7f5644005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43540/0 (socket says 172.21.15.122:43540) 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 -- 172.21.15.122:0/143940670 learned_addr learned my addr 172.21.15.122:0/143940670 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f565affd640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f77457a0 0x5584f7756fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f565b7fe640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f774eca0 0x5584f7752960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 -- 172.21.15.122:0/143940670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f77457a0 msgr2=0x5584f7756fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f77457a0 0x5584f7756fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 -- 172.21.15.122:0/143940670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f774eca0 msgr2=0x5584f7752960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.981+0000 7f5653fff640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f774eca0 0x5584f7752960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f5653fff640 1 -- 172.21.15.122:0/143940670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584f7752f20 con 0x5584f77460c0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f5653fff640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5644099010 tx=0x7f564409c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d898be5ec7b1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56440b6060 con 0x5584f77460c0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f56440ce5e0 con 0x5584f77460c0 2024-04-02T13:37:59.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f564c011bf0 con 0x5584f77460c0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f564409cbb0 con 0x5584f77460c0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 10 client.16557 fetch_fsmap learned FSMap version 32 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5584f77c4fb0 con 0x5584f77460c0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c011bf0 con 0x5584f77460c0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f56440d30b0 con 0x5584f77460c0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 10 client.16557 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 10 client.16557 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 20 client.16557 populate_metadata read hostname 'smithi122' 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f77c57b0 0x5584f77c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 4 client.16557 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584f76b1040 con 0x5584f77c57b0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.982+0000 7f565affd640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f77c57b0 0x5584f77c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f563affd640 10 client.16557 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f565affd640 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f77c57b0 0x5584f77c7ba0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93293603a08fca01 server_cookie=5eb909bd917a72af in_seq=0 out_seq=0 2024-04-02T13:37:59.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f563affd640 1 -- 172.21.15.122:0/143940670 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584f76b1040 con 0x5584f77c57b0 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f563affd640 10 client.16557 handle_command_reply: tid=0 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 client.16557 shutdown 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 msgr2=0x5584f7752420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5644099010 tx=0x7f564409c7e0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 shutdown_connections 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5584f77c57b0 0x5584f77c7ba0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5584f774eca0 0x5584f7752960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5584f77460c0 0x5584f7752420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 --2- 172.21.15.122:0/143940670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5584f77457a0 0x5584f7756fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f563affd640 0 client.16557 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 >> 172.21.15.122:0/143940670 conn(0x5584f773fed0 msgr2=0x5584f7741180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 shutdown_connections 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 1 -- 172.21.15.122:0/143940670 wait complete. 2024-04-02T13:37:59.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.983+0000 7f566461d740 20 client.16557 trim_cache size 0 max 16384 2024-04-02T13:37:59.992 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 msgr2=0x7f56480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 0x7f56480a61e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5654005cc0 tx=0x7f5654005c50 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 msgr2=0x7f565c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f564400e9f0 tx=0x7f564400edf0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 shutdown_connections 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56480a3d20 0x7f56480a61e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f565c106990 0x7f565c1a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f565c101350 0x7f565c199720 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 --2- 172.21.15.122:0/878749868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f565c100990 0x7f565c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 >> 172.21.15.122:0/878749868 conn(0x7f565c077ad0 msgr2=0x7f565c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 shutdown_connections 2024-04-02T13:37:59.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:37:59.984+0000 7f5661a35640 1 -- 172.21.15.122:0/878749868 wait complete. 2024-04-02T13:37:59.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:38:00.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- 172.21.15.122:0/2342014647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 msgr2=0x7fcc8410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 --2- 172.21.15.122:0/2342014647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc8410d840 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009cd0 tx=0x7fcc7c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- 172.21.15.122:0/2342014647 shutdown_connections 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 --2- 172.21.15.122:0/2342014647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc8410d840 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 --2- 172.21.15.122:0/2342014647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 --2- 172.21.15.122:0/2342014647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc84106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- 172.21.15.122:0/2342014647 >> 172.21.15.122:0/2342014647 conn(0x7fcc84077b00 msgr2=0x7fcc84077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- 172.21.15.122:0/2342014647 shutdown_connections 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- 172.21.15.122:0/2342014647 wait complete. 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 Processor -- start 2024-04-02T13:38:00.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.098+0000 7fcc89b60640 1 -- start start 2024-04-02T13:38:00.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc841a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84110d90 con 0x7fcc84101330 2024-04-02T13:38:00.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84110c10 con 0x7fcc84106980 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc84110f10 con 0x7fcc84100970 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43556/0 (socket says 172.21.15.122:43556) 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 -- 172.21.15.122:0/1380155008 learned_addr learned my addr 172.21.15.122:0/1380155008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc837fe640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc83fff640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc841a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 -- 172.21.15.122:0/1380155008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 msgr2=0x7fcc84199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 -- 172.21.15.122:0/1380155008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 msgr2=0x7fcc841a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc841a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 -- 172.21.15.122:0/1380155008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc841a4750 con 0x7fcc84101330 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc837fe640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc83fff640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc841a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc7affd640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d770 tx=0x7fcc6c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc817fa640 1 -- 172.21.15.122:0/1380155008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c017070 con 0x7fcc84101330 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc817fa640 1 -- 172.21.15.122:0/1380155008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6c005500 con 0x7fcc84101330 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc841a4920 con 0x7fcc84101330 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc817fa640 1 -- 172.21.15.122:0/1380155008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c004e00 con 0x7fcc84101330 2024-04-02T13:38:00.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.099+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc841a4e20 con 0x7fcc84101330 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 Processor -- start 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc817fa640 1 -- 172.21.15.122:0/1380155008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcc6c004fa0 con 0x7fcc84101330 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 -- start start 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdba2387b0 con 0x55cdba2d47a0 2024-04-02T13:38:00.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.102+0000 7fcc8c829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cdba1d7570 con 0x55cdba2ddca0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc8c829740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdba234760 con 0x55cdba2d50c0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43558/0 (socket says 172.21.15.122:43558) 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 -- 172.21.15.122:0/888438 learned_addr learned my addr 172.21.15.122:0/888438 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc837fe640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc7affd640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 -- 172.21.15.122:0/888438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 msgr2=0x55cdba2dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 -- 172.21.15.122:0/888438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 msgr2=0x55cdba2e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 -- 172.21.15.122:0/888438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdb9f4c3f0 con 0x55cdba2d47a0 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc7affd640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc817fa640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 0x7fcc640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc837fe640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 0x7fcc640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc83fff640 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c700 tx=0x7fcc7c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297228dc65e9530d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc817fa640 1 -- 172.21.15.122:0/1380155008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcc6c0d0720 con 0x7fcc84101330 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc79ffb640 1 -- 172.21.15.122:0/888438 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c043020 con 0x55cdba2d47a0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc79ffb640 1 -- 172.21.15.122:0/888438 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c02cca0 con 0x55cdba2d47a0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.103+0000 7fcc79ffb640 1 -- 172.21.15.122:0/888438 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c039040 con 0x55cdba2d47a0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- 172.21.15.122:0/888438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 msgr2=0x55cdba2d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc837fe640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 0x7fcc640a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55cdba2d4270 tx=0x7fcc74005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c700 tx=0x7fcc7c005fa0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- 172.21.15.122:0/888438 shutdown_connections 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 --2- 172.21.15.122:0/888438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2d4b80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- 172.21.15.122:0/888438 >> 172.21.15.122:0/888438 conn(0x55cdba2ceed0 msgr2=0x55cdba2cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- 172.21.15.122:0/888438 shutdown_connections 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- 172.21.15.122:0/888438 wait complete. 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.104+0000 7fcc8c829740 1 -- start start 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 Processor -- start 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdba2e0a30 con 0x55cdba2d47a0 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cdba1f3e80 con 0x55cdba2ddca0 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdba2e0bb0 con 0x55cdba2d50c0 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc8c829740 10 client.0 _ll_register_callbacks cb 0x55cdba22faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43574/0 (socket says 172.21.15.122:43574) 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 -- 172.21.15.122:0/2083122853 learned_addr learned my addr 172.21.15.122:0/2083122853 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc83fff640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc7affd640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 -- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 msgr2=0x55cdba2daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 -- 172.21.15.122:0/2083122853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 msgr2=0x55cdba2db1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2db1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 -- 172.21.15.122:0/2083122853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdba0167d0 con 0x55cdba2d47a0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc83fff640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2db1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc837fe640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011c30 tx=0x7fcc740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abad8b066e5140ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.105+0000 7fcc72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740108d0 con 0x55cdba2d47a0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcc74010a70 con 0x55cdba2d47a0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c02cca0 con 0x55cdba2d47a0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc74010cb0 con 0x55cdba2d47a0 2024-04-02T13:38:00.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 10 client.16569 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdba2eca50 con 0x55cdba2d47a0 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c02cca0 con 0x55cdba2d47a0 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcc7401e410 con 0x55cdba2d47a0 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 10 client.16569 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 10 client.16569 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 20 client.16569 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba2edf60 0x55cdba2f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 4 client.16569 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.106+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdba245cf0 con 0x55cdba2edf60 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.107+0000 7fcc7affd640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba2edf60 0x55cdba2f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.107+0000 7fcc72ffd640 10 client.16569 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.107+0000 7fcc7affd640 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba2edf60 0x55cdba2f03b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6917f0c8a123e780 server_cookie=829e6433cf11a2e9 in_seq=0 out_seq=0 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc72ffd640 1 -- 172.21.15.122:0/2083122853 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdba245cf0 con 0x55cdba2edf60 2024-04-02T13:38:00.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc72ffd640 10 client.16569 handle_command_reply: tid=0 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 client.16569 shutdown 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 msgr2=0x55cdba2dd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcc74011c30 tx=0x7fcc740046c0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 shutdown_connections 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2ddca0 0x55cdba2db1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d50c0 0x55cdba2daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba2edf60 0x55cdba2f03b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2083122853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d47a0 0x55cdba2dd6f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc72ffd640 0 client.16569 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 >> 172.21.15.122:0/2083122853 conn(0x55cdba2ceed0 msgr2=0x55cdba2d0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 shutdown_connections 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 1 -- 172.21.15.122:0/2083122853 wait complete. 2024-04-02T13:38:00.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.108+0000 7fcc8c829740 20 client.16569 trim_cache size 0 max 16384 2024-04-02T13:38:00.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.121+0000 7fcc8c829740 1 Processor -- start 2024-04-02T13:38:00.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.121+0000 7fcc8c829740 1 -- start start 2024-04-02T13:38:00.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.121+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 0x55cdba2e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.121+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d50c0 0x55cdba2d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc8c829740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841af000 con 0x55cdba2ddca0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc8c829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b0c60 con 0x55cdba2d50c0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc8c829740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841af6d0 con 0x55cdba2d47a0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43588/0 (socket says 172.21.15.122:43588) 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 -- 172.21.15.122:0/2627799685 learned_addr learned my addr 172.21.15.122:0/2627799685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc837fe640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 0x55cdba2e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc7affd640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d50c0 0x55cdba2d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 -- 172.21.15.122:0/2627799685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 msgr2=0x55cdba2e98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 0x55cdba2e98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 -- 172.21.15.122:0/2627799685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d50c0 msgr2=0x55cdba2d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d50c0 0x55cdba2d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 -- 172.21.15.122:0/2627799685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdba2d8fd0 con 0x55cdba2ddca0 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc837fe640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 0x55cdba2e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc83fff640 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c700 tx=0x7fcc7c02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9040bd28d385085d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc797fa640 1 -- 172.21.15.122:0/2627799685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c04c070 con 0x55cdba2ddca0 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc797fa640 1 -- 172.21.15.122:0/2627799685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c003970 con 0x55cdba2ddca0 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.122+0000 7fcc797fa640 1 -- 172.21.15.122:0/2627799685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c039050 con 0x55cdba2ddca0 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- 172.21.15.122:0/2627799685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 msgr2=0x55cdba2f5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c02c700 tx=0x7fcc7c02c770 comp rx=0 tx=0).stop 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- 172.21.15.122:0/2627799685 shutdown_connections 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2ddca0 0x55cdba2f5c80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d50c0 0x55cdba2d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 --2- 172.21.15.122:0/2627799685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2d47a0 0x55cdba2e98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- 172.21.15.122:0/2627799685 >> 172.21.15.122:0/2627799685 conn(0x55cdba2ceed0 msgr2=0x55cdba2d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- 172.21.15.122:0/2627799685 shutdown_connections 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- 172.21.15.122:0/2627799685 wait complete. 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 -- start start 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.123+0000 7fcc8c829740 1 Processor -- start 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 0x55cdba2e6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2ddca0 0x55cdba2deca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841b04d0 con 0x55cdba2d50c0 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841afe00 con 0x55cdba2d47a0 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcc841acfe0 con 0x55cdba2ddca0 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 10 client.0 _ll_register_callbacks cb 0x7fcc74009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43598/0 (socket says 172.21.15.122:43598) 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 -- 172.21.15.122:0/819513817 learned_addr learned my addr 172.21.15.122:0/819513817 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc837fe640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 0x55cdba2e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 -- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2ddca0 msgr2=0x55cdba2deca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2ddca0 0x55cdba2deca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 -- 172.21.15.122:0/819513817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 msgr2=0x55cdba2e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 0x55cdba2e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 -- 172.21.15.122:0/819513817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdba2df4d0 con 0x55cdba2d50c0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc837fe640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 0x55cdba2e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc7affd640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0cdae0 tx=0x7fcc6c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc272f34707f5775 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c020040 con 0x55cdba2d50c0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcc6c09a5d0 con 0x55cdba2d50c0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc7c003970 con 0x55cdba2d50c0 2024-04-02T13:38:00.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcc6c0a0680 con 0x55cdba2d50c0 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 10 client.16575 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.124+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdba354550 con 0x55cdba2d50c0 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c003970 con 0x55cdba2d50c0 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcc6c0cdd00 con 0x55cdba2d50c0 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 10 client.16575 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 10 client.16575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 20 client.16575 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba354c80 0x55cdba3570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 4 client.16575 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cdba240040 con 0x55cdba354c80 2024-04-02T13:38:00.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.125+0000 7fcc837fe640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba354c80 0x55cdba3570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc837fe640 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba354c80 0x55cdba3570e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5bfec18bf4ed871 server_cookie=899b209a28889335 in_seq=0 out_seq=0 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc72ffd640 10 client.16575 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc72ffd640 1 -- 172.21.15.122:0/819513817 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cdba240040 con 0x55cdba354c80 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc72ffd640 10 client.16575 handle_command_reply: tid=0 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 client.16575 shutdown 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 msgr2=0x55cdba2de760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c0cdae0 tx=0x7fcc6c0048a0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 shutdown_connections 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdba2ddca0 0x55cdba2deca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cdba354c80 0x55cdba3570e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdba2d50c0 0x55cdba2de760 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 --2- 172.21.15.122:0/819513817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdba2d47a0 0x55cdba2e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc72ffd640 0 client.16575 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 >> 172.21.15.122:0/819513817 conn(0x55cdba2ceed0 msgr2=0x55cdba2d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 shutdown_connections 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 1 -- 172.21.15.122:0/819513817 wait complete. 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc8c829740 20 client.16575 trim_cache size 0 max 16384 2024-04-02T13:38:00.136 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 msgr2=0x7fcc640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 0x7fcc640a61e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x55cdba2d4270 tx=0x7fcc74005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 msgr2=0x7fcc841996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.126+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c00d770 tx=0x7fcc6c00dc30 comp rx=0 tx=0).stop 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 shutdown_connections 2024-04-02T13:38:00.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcc640a3d20 0x7fcc640a61e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc84106980 0x7fcc841a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc84101330 0x7fcc841996d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 --2- 172.21.15.122:0/1380155008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc84100970 0x7fcc84199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 >> 172.21.15.122:0/1380155008 conn(0x7fcc84077b00 msgr2=0x7fcc841030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 shutdown_connections 2024-04-02T13:38:00.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.127+0000 7fcc89b60640 1 -- 172.21.15.122:0/1380155008 wait complete. 2024-04-02T13:38:00.139 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:37:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3541278908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:00.139 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:38:00.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:37:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3541278908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/1665979521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 msgr2=0x7ff8b0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/1665979521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b0100df0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/1665979521 shutdown_connections 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/1665979521 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/1665979521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b0100df0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/1665979521 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 0x7ff8b0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.240+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/1665979521 >> 172.21.15.122:0/1665979521 conn(0x7ff8b0077b00 msgr2=0x7ff8b0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/1665979521 shutdown_connections 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/1665979521 wait complete. 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 Processor -- start 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- start start 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b01996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 0x7ff8b01a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0110d60 con 0x7ff8b0100970 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0110be0 con 0x7ff8b0106980 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b7ec8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b0110ee0 con 0x7ff8b0101330 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b5c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b5c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43622/0 (socket says 172.21.15.122:43622) 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.241+0000 7ff8b5c64640 1 -- 172.21.15.122:0/910261591 learned_addr learned my addr 172.21.15.122:0/910261591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5463640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b01996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b6465640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 0x7ff8b01a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 -- 172.21.15.122:0/910261591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 msgr2=0x7ff8b01996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b01996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 -- 172.21.15.122:0/910261591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 msgr2=0x7ff8b01a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 0x7ff8b01a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 -- 172.21.15.122:0/910261591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b01a4760 con 0x7ff8b0100970 2024-04-02T13:38:00.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5463640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00cce0 tx=0x7ff8ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8a6ffd640 1 -- 172.21.15.122:0/910261591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac00f650 con 0x7ff8b0100970 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8a6ffd640 1 -- 172.21.15.122:0/910261591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ac00fcd0 con 0x7ff8b0100970 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8b01a4930 con 0x7ff8b0100970 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8a6ffd640 1 -- 172.21.15.122:0/910261591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac018c70 con 0x7ff8b0100970 2024-04-02T13:38:00.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.242+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b01a4e50 con 0x7ff8b0100970 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.244+0000 7ff8a6ffd640 1 -- 172.21.15.122:0/910261591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff8ac00f7f0 con 0x7ff8b0100970 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.244+0000 7ff8bab88740 1 Processor -- start 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.244+0000 7ff8bab88740 1 -- start start 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736e0c0 0x55cbc7376760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc7379090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbc72d17b0 con 0x55cbc736d7a0 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbc7270570 con 0x55cbc736e0c0 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8bab88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbc72cd760 con 0x55cbc7376ca0 2024-04-02T13:38:00.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8a6ffd640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43630/0 (socket says 172.21.15.122:43630) 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 -- 172.21.15.122:0/3800213055 learned_addr learned my addr 172.21.15.122:0/3800213055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736e0c0 0x55cbc7376760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc7379090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8a6ffd640 1 -- 172.21.15.122:0/910261591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8ac0d10a0 con 0x7ff8b0100970 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 -- 172.21.15.122:0/3800213055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 msgr2=0x55cbc7379090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc7379090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 -- 172.21.15.122:0/3800213055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736e0c0 msgr2=0x55cbc7376760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736e0c0 0x55cbc7376760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 -- 172.21.15.122:0/3800213055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc6fe53f0 con 0x55cbc736d7a0 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc7379090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013740 tx=0x7ff8a8013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef91ffdaf624ce2e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.245+0000 7ff89bfff640 1 -- 172.21.15.122:0/3800213055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801d7c0 con 0x55cbc736d7a0 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff89bfff640 1 -- 172.21.15.122:0/3800213055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8022070 con 0x55cbc736d7a0 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff89bfff640 1 -- 172.21.15.122:0/3800213055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801c930 con 0x55cbc736d7a0 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8b6465640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8021000 tx=0x7ff8a801f000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 -- 172.21.15.122:0/3800213055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 msgr2=0x55cbc736db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013740 tx=0x7ff8a8013c00 comp rx=0 tx=0).stop 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 -- 172.21.15.122:0/3800213055 shutdown_connections 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc7379090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736e0c0 0x55cbc7376760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3800213055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc736db80 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 -- 172.21.15.122:0/3800213055 >> 172.21.15.122:0/3800213055 conn(0x55cbc7367ed0 msgr2=0x55cbc73682b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 -- 172.21.15.122:0/3800213055 shutdown_connections 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.246+0000 7ff8bab88740 1 -- 172.21.15.122:0/3800213055 wait complete. 2024-04-02T13:38:00.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 -- start start 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 Processor -- start 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7376700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc73741b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbc7379a90 con 0x55cbc736e0c0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbc728ce80 con 0x55cbc736d7a0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbc7379c10 con 0x55cbc7376ca0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8bab88740 10 client.0 _ll_register_callbacks cb 0x55cbc72c8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43634/0 (socket says 172.21.15.122:43634) 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 -- 172.21.15.122:0/3791989389 learned_addr learned my addr 172.21.15.122:0/3791989389 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7376700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc73741b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 -- 172.21.15.122:0/3791989389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 msgr2=0x55cbc73741b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc73741b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 -- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 msgr2=0x55cbc7376700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.247+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7376700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8b5463640 1 -- 172.21.15.122:0/3791989389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc70af7d0 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc73741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8cecd3fbefc2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0005030 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8a00051d0 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8022070 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff8a00053d0 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 10 client.16587 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cbc7385a70 con 0x55cbc736e0c0 2024-04-02T13:38:00.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8022070 con 0x55cbc736e0c0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff8a0003830 con 0x55cbc736e0c0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 10 client.16587 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 10 client.16587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 20 client.16587 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc7386fd0 0x55cbc73893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 4 client.16587 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.248+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc72decf0 con 0x55cbc7386fd0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.249+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc7386fd0 0x55cbc73893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.249+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc7386fd0 0x55cbc73893e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdd17d8131caade5 server_cookie=7f2952c0e74c7978 in_seq=0 out_seq=0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.249+0000 7ff8997fa640 10 client.16587 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8997fa640 1 -- 172.21.15.122:0/3791989389 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbc72decf0 con 0x55cbc7386fd0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8997fa640 10 client.16587 handle_command_reply: tid=0 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 client.16587 shutdown 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 msgr2=0x55cbc7373a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009930 tx=0x7ff8a0004040 comp rx=0 tx=0).stop 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 shutdown_connections 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc7376ca0 0x55cbc73741b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736e0c0 0x55cbc7373a90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc7386fd0 0x55cbc73893e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3791989389 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7376700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8997fa640 0 client.16587 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 >> 172.21.15.122:0/3791989389 conn(0x55cbc7367ed0 msgr2=0x55cbc7369c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 shutdown_connections 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 1 -- 172.21.15.122:0/3791989389 wait complete. 2024-04-02T13:38:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.250+0000 7ff8bab88740 20 client.16587 trim_cache size 0 max 16384 2024-04-02T13:38:00.274 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:37:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3541278908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 Processor -- start 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 -- start start 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7372bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc73699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01aeea0 con 0x55cbc7376ca0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b0b00 con 0x55cbc736d7a0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.263+0000 7ff8bab88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01af570 con 0x55cbc736e0c0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43646/0 (socket says 172.21.15.122:43646) 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 -- 172.21.15.122:0/1650719626 learned_addr learned my addr 172.21.15.122:0/1650719626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7372bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b5463640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc73699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 -- 172.21.15.122:0/1650719626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 msgr2=0x55cbc73699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc73699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 -- 172.21.15.122:0/1650719626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 msgr2=0x55cbc7372bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7372bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 -- 172.21.15.122:0/1650719626 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc7373df0 con 0x55cbc7376ca0 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7372bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff8b6465640 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013660 tx=0x7ff8a800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfacd8aadad8f43b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff89b7fe640 1 -- 172.21.15.122:0/1650719626 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a800be60 con 0x55cbc7376ca0 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff89b7fe640 1 -- 172.21.15.122:0/1650719626 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8034070 con 0x55cbc7376ca0 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.264+0000 7ff89b7fe640 1 -- 172.21.15.122:0/1650719626 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a801c800 con 0x55cbc7376ca0 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- 172.21.15.122:0/1650719626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 msgr2=0x55cbc738ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8013660 tx=0x7ff8a800b640 comp rx=0 tx=0).stop 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- 172.21.15.122:0/1650719626 shutdown_connections 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc7376ca0 0x55cbc738ecb0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc73699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- 172.21.15.122:0/1650719626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc736d7a0 0x55cbc7372bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- 172.21.15.122:0/1650719626 >> 172.21.15.122:0/1650719626 conn(0x55cbc7367ed0 msgr2=0x55cbc73692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- 172.21.15.122:0/1650719626 shutdown_connections 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- 172.21.15.122:0/1650719626 wait complete. 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- start start 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 Processor -- start 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc7378100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc7376ca0 0x55cbc7378640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01b0370 con 0x55cbc736d7a0 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01afca0 con 0x55cbc7376ca0 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8b01ace80 con 0x55cbc736e0c0 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.265+0000 7ff8bab88740 10 client.0 _ll_register_callbacks cb 0x7ff8a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43654/0 (socket says 172.21.15.122:43654) 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 -- 172.21.15.122:0/3271293875 learned_addr learned my addr 172.21.15.122:0/3271293875 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b6465640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc7376ca0 0x55cbc7378640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc7378100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 -- 172.21.15.122:0/3271293875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 msgr2=0x55cbc7378100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc7378100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 -- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc7376ca0 msgr2=0x55cbc7378640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc7376ca0 0x55cbc7378640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 -- 172.21.15.122:0/3271293875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc737d290 con 0x55cbc736d7a0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8b5c64640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac09c9b0 tx=0x7ff8ac008420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c305bea051e6a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8997fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac098f50 con 0x55cbc736d7a0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8ac099320 con 0x55cbc736d7a0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8a8034070 con 0x55cbc736d7a0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff8ac09d3b0 con 0x55cbc736d7a0 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8bab88740 10 client.16593 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cbc73ed950 con 0x55cbc736d7a0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.266+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a8034070 con 0x55cbc736d7a0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff8ac0bd0b0 con 0x55cbc736d7a0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 10 client.16593 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 20 client.16593 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc73ee030 0x55cbc73f0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 4 client.16593 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbc72d9040 con 0x55cbc73ee030 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc73ee030 0x55cbc73f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8997fa640 10 client.16593 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8b5463640 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc73ee030 0x55cbc73f0450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cba72dc56adc103 server_cookie=277878950e87c5db in_seq=0 out_seq=0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8997fa640 1 -- 172.21.15.122:0/3271293875 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbc72d9040 con 0x55cbc73ee030 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8997fa640 10 client.16593 handle_command_reply: tid=0 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 1 client.16593 shutdown 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 msgr2=0x55cbc7377bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.267+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac09c9b0 tx=0x7ff8ac008420 comp rx=0 tx=0).stop 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 shutdown_connections 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cbc73ee030 0x55cbc73f0450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbc7376ca0 0x55cbc7378640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbc736e0c0 0x55cbc7378100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 --2- 172.21.15.122:0/3271293875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbc736d7a0 0x55cbc7377bc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8997fa640 0 client.16593 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 >> 172.21.15.122:0/3271293875 conn(0x55cbc7367ed0 msgr2=0x55cbc7369b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 shutdown_connections 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 1 -- 172.21.15.122:0/3271293875 wait complete. 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8bab88740 20 client.16593 trim_cache size 0 max 16384 2024-04-02T13:38:00.280 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 msgr2=0x7ff89c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8021000 tx=0x7ff8a801f000 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 msgr2=0x7ff8b01991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00cce0 tx=0x7ff8ac00a620 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 shutdown_connections 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff89c0a3d20 0x7ff89c0a61e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8b0106980 0x7ff8b01a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8b0101330 0x7ff8b01996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 --2- 172.21.15.122:0/910261591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b0100970 0x7ff8b01991a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 >> 172.21.15.122:0/910261591 conn(0x7ff8b0077b00 msgr2=0x7ff8b01030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 shutdown_connections 2024-04-02T13:38:00.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.268+0000 7ff8b7ec8640 1 -- 172.21.15.122:0/910261591 wait complete. 2024-04-02T13:38:00.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/4025809011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 msgr2=0x7fbf88103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/4025809011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88103120 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009900 tx=0x7fbf8402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/4025809011 shutdown_connections 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/4025809011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 0x7fbf8810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/4025809011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 0x7fbf8810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/4025809011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88103120 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.384+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/4025809011 >> 172.21.15.122:0/4025809011 conn(0x7fbf88076d80 msgr2=0x7fbf88077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/4025809011 shutdown_connections 2024-04-02T13:38:00.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/4025809011 wait complete. 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 Processor -- start 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- start start 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 0x7fbf881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 0x7fbf8819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88111ea0 con 0x7fbf88102d40 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88111d20 con 0x7fbf88103660 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8f3e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf88112020 con 0x7fbf8810c110 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43678/0 (socket says 172.21.15.122:43678) 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8d185640 1 -- 172.21.15.122:0/1161068228 learned_addr learned my addr 172.21.15.122:0/1161068228 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.385+0000 7fbf8c984640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 0x7fbf881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d986640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 0x7fbf8819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 -- 172.21.15.122:0/1161068228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 msgr2=0x7fbf8819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 0x7fbf8819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 -- 172.21.15.122:0/1161068228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 msgr2=0x7fbf881988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 0x7fbf881988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 -- 172.21.15.122:0/1161068228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8819d370 con 0x7fbf88102d40 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8d185640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005940 tx=0x7fbf8402cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf7e7fc640 1 -- 172.21.15.122:0/1161068228 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84041070 con 0x7fbf88102d40 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf7e7fc640 1 -- 172.21.15.122:0/1161068228 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf84004450 con 0x7fbf88102d40 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8819d600 con 0x7fbf88102d40 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf7e7fc640 1 -- 172.21.15.122:0/1161068228 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf84003df0 con 0x7fbf88102d40 2024-04-02T13:38:00.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.386+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf881a4ef0 con 0x7fbf88102d40 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.388+0000 7fbf7e7fc640 1 -- 172.21.15.122:0/1161068228 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbf84044070 con 0x7fbf88102d40 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 Processor -- start 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 -- start start 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf7e7fc640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af38090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c09ae907b0 con 0x55c09af2c7a0 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c09ae2f570 con 0x55c09af2d0c0 2024-04-02T13:38:00.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c09ae8c760 con 0x55c09af35ca0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d986640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43694/0 (socket says 172.21.15.122:43694) 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 -- 172.21.15.122:0/324166600 learned_addr learned my addr 172.21.15.122:0/324166600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf7e7fc640 1 -- 172.21.15.122:0/1161068228 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf840f2660 con 0x7fbf88102d40 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8c984640 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 -- 172.21.15.122:0/324166600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 msgr2=0x55c09af38090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af38090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 -- 172.21.15.122:0/324166600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 msgr2=0x55c09af35760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af35760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d185640 1 -- 172.21.15.122:0/324166600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09aba43f0 con 0x55c09af2c7a0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8c984640 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.389+0000 7fbf8d986640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 0x7fbf740a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005c40 tx=0x7fbf80009a80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf8d185640 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbf840ba7e0 tx=0x7fbf8404cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b59a329268d248 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf737fe640 1 -- 172.21.15.122:0/324166600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840f1070 con 0x55c09af2c7a0 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf737fe640 1 -- 172.21.15.122:0/324166600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8402ce30 con 0x55c09af2c7a0 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf737fe640 1 -- 172.21.15.122:0/324166600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8404b070 con 0x55c09af2c7a0 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 -- 172.21.15.122:0/324166600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 msgr2=0x55c09af2cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbf840ba7e0 tx=0x7fbf8404cf00 comp rx=0 tx=0).stop 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 -- 172.21.15.122:0/324166600 shutdown_connections 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af38090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af35760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/324166600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af2cb80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 -- 172.21.15.122:0/324166600 >> 172.21.15.122:0/324166600 conn(0x55c09af26ed0 msgr2=0x55c09af272b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 -- 172.21.15.122:0/324166600 shutdown_connections 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.390+0000 7fbf91fd1740 1 -- 172.21.15.122:0/324166600 wait complete. 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 -- start start 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 Processor -- start 2024-04-02T13:38:00.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af32a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af33050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c09af38850 con 0x55c09af2c7a0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c09ae4be80 con 0x55c09af2d0c0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c09af389d0 con 0x55c09af35ca0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf91fd1740 10 client.0 _ll_register_callbacks cb 0x55c09ae87aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43700/0 (socket says 172.21.15.122:43700) 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf8d185640 1 -- 172.21.15.122:0/3086837146 learned_addr learned my addr 172.21.15.122:0/3086837146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf8d986640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af33050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.391+0000 7fbf8c984640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af32a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 -- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 msgr2=0x55c09af33050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af33050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 -- 172.21.15.122:0/3086837146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 msgr2=0x55c09af32a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af32a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 -- 172.21.15.122:0/3086837146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09ac6e7d0 con 0x55c09af2c7a0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d986640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af33050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8c984640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf8d185640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf840efbe0 tx=0x7fbf840efed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e65d725e68f6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840dc030 con 0x55c09af2c7a0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf840bc420 con 0x55c09af2c7a0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf8402ce30 con 0x55c09af2c7a0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf840e8050 con 0x55c09af2c7a0 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 10 client.16605 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c09af38850 con 0x55c09af2c7a0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf8402ce30 con 0x55c09af2c7a0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbf840f2270 con 0x55c09af2c7a0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 10 client.16605 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 10 client.16605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 20 client.16605 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09af45a10 0x55c09af47e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 4 client.16605 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.392+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c09ae9dcf0 con 0x55c09af45a10 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.393+0000 7fbf8c984640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09af45a10 0x55c09af47e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.393+0000 7fbf70ff9640 10 client.16605 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.393+0000 7fbf8c984640 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09af45a10 0x55c09af47e20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=57c131faba6d442b server_cookie=b42ad9d601e153b in_seq=0 out_seq=0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf70ff9640 1 -- 172.21.15.122:0/3086837146 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c09ae9dcf0 con 0x55c09af45a10 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf70ff9640 10 client.16605 handle_command_reply: tid=0 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 client.16605 shutdown 2024-04-02T13:38:00.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 msgr2=0x55c09af356b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbf840efbe0 tx=0x7fbf840efed0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 shutdown_connections 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09af45a10 0x55c09af47e20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af33050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af32a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf70ff9640 0 client.16605 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/3086837146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af356b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 >> 172.21.15.122:0/3086837146 conn(0x55c09af26ed0 msgr2=0x55c09af28c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 shutdown_connections 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 1 -- 172.21.15.122:0/3086837146 wait complete. 2024-04-02T13:38:00.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.394+0000 7fbf91fd1740 20 client.16605 trim_cache size 0 max 16384 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.407+0000 7fbf91fd1740 1 Processor -- start 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.407+0000 7fbf91fd1740 1 -- start start 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af31a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af35170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881aeec0 con 0x55c09af2c7a0 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881b0b20 con 0x55c09af2d0c0 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881af590 con 0x55c09af35ca0 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43708/0 (socket says 172.21.15.122:43708) 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 -- 172.21.15.122:0/2147721715 learned_addr learned my addr 172.21.15.122:0/2147721715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 -- 172.21.15.122:0/2147721715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 msgr2=0x55c09af35170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af35170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 -- 172.21.15.122:0/2147721715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 msgr2=0x55c09af31a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af31a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 -- 172.21.15.122:0/2147721715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c09af333c0 con 0x55c09af2c7a0 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf8d185640 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbf840be310 tx=0x7fbf840c6bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71eab75709ac413e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf72ffd640 1 -- 172.21.15.122:0/2147721715 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840e2070 con 0x55c09af2c7a0 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf72ffd640 1 -- 172.21.15.122:0/2147721715 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf840bea80 con 0x55c09af2c7a0 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.408+0000 7fbf72ffd640 1 -- 172.21.15.122:0/2147721715 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf840dc070 con 0x55c09af2c7a0 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- 172.21.15.122:0/2147721715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 msgr2=0x55c09af41650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fbf840be310 tx=0x7fbf840c6bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- 172.21.15.122:0/2147721715 shutdown_connections 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af35170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2d0c0 0x55c09af31a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/2147721715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2c7a0 0x55c09af41650 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- 172.21.15.122:0/2147721715 >> 172.21.15.122:0/2147721715 conn(0x55c09af26ed0 msgr2=0x55c09af31310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- 172.21.15.122:0/2147721715 shutdown_connections 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- 172.21.15.122:0/2147721715 wait complete. 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 -- start start 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.409+0000 7fbf91fd1740 1 Processor -- start 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 0x55c09af3dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881b0390 con 0x55c09af2d0c0 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881afcc0 con 0x55c09af2c7a0 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf881acea0 con 0x55c09af35ca0 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 10 client.0 _ll_register_callbacks cb 0x7fbf84005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43718/0 (socket says 172.21.15.122:43718) 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 -- 172.21.15.122:0/340606104 learned_addr learned my addr 172.21.15.122:0/340606104 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8d185640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 0x55c09af3dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 -- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 msgr2=0x55c09af30200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 -- 172.21.15.122:0/340606104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 msgr2=0x55c09af3dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 0x55c09af3dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 -- 172.21.15.122:0/340606104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c09af30790 con 0x55c09af2d0c0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8d185640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 0x55c09af3dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf8c984640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b880 tx=0x7fbf78004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea5759e4706ec10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf78021020 con 0x55c09af2d0c0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf7801d420 con 0x55c09af2d0c0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf840bea80 con 0x55c09af2d0c0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbf7802c050 con 0x55c09af2d0c0 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 10 client.16611 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.410+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c09afabf30 con 0x55c09af2d0c0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf840bea80 con 0x55c09af2d0c0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbf78010430 con 0x55c09af2d0c0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 10 client.16611 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 10 client.16611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 20 client.16611 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09afac5c0 0x55c09afae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 4 client.16611 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c09ae98040 con 0x55c09afac5c0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf8d185640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09afac5c0 0x55c09afae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf70ff9640 10 client.16611 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.411+0000 7fbf8d185640 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09afac5c0 0x55c09afae9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a338dc434744de5 server_cookie=1b35e822735f1f09 in_seq=0 out_seq=0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf70ff9640 1 -- 172.21.15.122:0/340606104 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c09ae98040 con 0x55c09afac5c0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf70ff9640 10 client.16611 handle_command_reply: tid=0 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 client.16611 shutdown 2024-04-02T13:38:00.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 msgr2=0x55c09af2fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbf7800b880 tx=0x7fbf78004600 comp rx=0 tx=0).stop 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 shutdown_connections 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55c09afac5c0 0x55c09afae9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c09af35ca0 0x55c09af30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c09af2d0c0 0x55c09af2fcc0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 --2- 172.21.15.122:0/340606104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c09af2c7a0 0x55c09af3dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf70ff9640 0 client.16611 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 >> 172.21.15.122:0/340606104 conn(0x55c09af26ed0 msgr2=0x55c09af256a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 shutdown_connections 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 1 -- 172.21.15.122:0/340606104 wait complete. 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf91fd1740 20 client.16611 trim_cache size 0 max 16384 2024-04-02T13:38:00.420 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:00.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 msgr2=0x7fbf740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 0x7fbf740a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbf80005c40 tx=0x7fbf80009a80 comp rx=0 tx=0).stop 2024-04-02T13:38:00.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 msgr2=0x7fbf88198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005940 tx=0x7fbf8402cc40 comp rx=0 tx=0).stop 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 shutdown_connections 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf740a3d20 0x7fbf740a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf8810c110 0x7fbf8819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf88103660 0x7fbf881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 --2- 172.21.15.122:0/1161068228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf88102d40 0x7fbf88198380 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.412+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 >> 172.21.15.122:0/1161068228 conn(0x7fbf88076d80 msgr2=0x7fbf880fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.413+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 shutdown_connections 2024-04-02T13:38:00.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.413+0000 7fbf8f3e9640 1 -- 172.21.15.122:0/1161068228 wait complete. 2024-04-02T13:38:00.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:38:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 -- 172.21.15.122:0/1661339651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41009b0 msgr2=0x7fe9a4100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 --2- 172.21.15.122:0/1661339651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41009b0 0x7fe9a4100e30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe990009930 tx=0x7fe99002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 -- 172.21.15.122:0/1661339651 shutdown_connections 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 --2- 172.21.15.122:0/1661339651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a4101370 0x7fe9a410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 --2- 172.21.15.122:0/1661339651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41009b0 0x7fe9a4100e30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 --2- 172.21.15.122:0/1661339651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41069c0 0x7fe9a4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 -- 172.21.15.122:0/1661339651 >> 172.21.15.122:0/1661339651 conn(0x7fe9a4076d80 msgr2=0x7fe9a4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 -- 172.21.15.122:0/1661339651 shutdown_connections 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.525+0000 7fe9aad04640 1 -- 172.21.15.122:0/1661339651 wait complete. 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 Processor -- start 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 -- start start 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 0x7fe9a41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a4101370 0x7fe9a4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4110dd0 con 0x7fe9a41069c0 2024-04-02T13:38:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4110c50 con 0x7fe9a4101370 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a4110f50 con 0x7fe9a41009b0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43730/0 (socket says 172.21.15.122:43730) 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 -- 172.21.15.122:0/2056388492 learned_addr learned my addr 172.21.15.122:0/2056388492 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 0x7fe9a41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 -- 172.21.15.122:0/2056388492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 msgr2=0x7fe9a41991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 0x7fe9a41991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 -- 172.21.15.122:0/2056388492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a4101370 msgr2=0x7fe9a4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a4101370 0x7fe9a4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 -- 172.21.15.122:0/2056388492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a41a47b0 con 0x7fe9a41069c0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 0x7fe9a41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00cce0 tx=0x7fe99c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a1ffb640 1 -- 172.21.15.122:0/2056388492 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00f450 con 0x7fe9a41069c0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a41a4980 con 0x7fe9a41069c0 2024-04-02T13:38:00.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.526+0000 7fe9a1ffb640 1 -- 172.21.15.122:0/2056388492 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99c00fad0 con 0x7fe9a41069c0 2024-04-02T13:38:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.527+0000 7fe9a1ffb640 1 -- 172.21.15.122:0/2056388492 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c018c10 con 0x7fe9a41069c0 2024-04-02T13:38:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.527+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a41a4e50 con 0x7fe9a41069c0 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9a1ffb640 1 -- 172.21.15.122:0/2056388492 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe99c00f5f0 con 0x7fe9a41069c0 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9a1ffb640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9ad9ca740 1 Processor -- start 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9ad9ca740 1 -- start start 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9a1ffb640 1 -- 172.21.15.122:0/2056388492 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe99c0d1320 con 0x7fe9a41069c0 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.529+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa299fe7b0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aa2999d570 con 0x55aa29a9b0c0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aa299fa760 con 0x55aa29aa3ca0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43746/0 (socket says 172.21.15.122:43746) 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/4103413129 learned_addr learned my addr 172.21.15.122:0/4103413129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 0x7fe9940a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe998015470 tx=0x7fe998015870 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/4103413129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 msgr2=0x55aa29aa6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/4103413129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 msgr2=0x55aa29aa3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/4103413129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa297123f0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe998015d70 tx=0x7fe99801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6983c0baebf4e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe98effd640 1 -- 172.21.15.122:0/4103413129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998024070 con 0x55aa29a9a7a0 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe98effd640 1 -- 172.21.15.122:0/4103413129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99801f930 con 0x55aa29a9a7a0 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.530+0000 7fe98effd640 1 -- 172.21.15.122:0/4103413129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998013890 con 0x55aa29a9a7a0 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/4103413129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 msgr2=0x55aa29a9ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe998015d70 tx=0x7fe99801c000 comp rx=0 tx=0).stop 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/4103413129 shutdown_connections 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/4103413129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29a9ab80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/4103413129 >> 172.21.15.122:0/4103413129 conn(0x55aa29a94ed0 msgr2=0x55aa29a952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/4103413129 shutdown_connections 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/4103413129 wait complete. 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 -- start start 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.531+0000 7fe9ad9ca740 1 Processor -- start 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa29aa6a30 con 0x55aa29a9a7a0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aa299b9e80 con 0x55aa29a9b0c0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aa29aa6bb0 con 0x55aa29aa3ca0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9ad9ca740 10 client.0 _ll_register_callbacks cb 0x55aa299f5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43752/0 (socket says 172.21.15.122:43752) 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2894140709 learned_addr learned my addr 172.21.15.122:0/2894140709 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a92a1640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2894140709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 msgr2=0x55aa29aa11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 msgr2=0x55aa29aa0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2894140709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa297dc7d0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe9980095d0 tx=0x7fe998000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89373ed0ef1f73b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.532+0000 7fe983fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99802f060 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe99801f450 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99801f930 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe998036050 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 10 client.16623 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa29ab2ac0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99801f930 con 0x55aa29a9a7a0 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe99801e3b0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 10 client.16623 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 20 client.16623 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29ab3f80 0x55aa29ab6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 4 client.16623 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa29a0bcf0 con 0x55aa29ab3f80 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.533+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29ab3f80 0x55aa29ab6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.534+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29ab3f80 0x55aa29ab6370 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f99f5b9e33f66981 server_cookie=fc891f98aad24267 in_seq=0 out_seq=0 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.534+0000 7fe983fff640 10 client.16623 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.534+0000 7fe983fff640 1 -- 172.21.15.122:0/2894140709 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa29a0bcf0 con 0x55aa29ab3f80 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.534+0000 7fe983fff640 10 client.16623 handle_command_reply: tid=0 2024-04-02T13:38:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 client.16623 shutdown 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 msgr2=0x55aa29aa3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe9980095d0 tx=0x7fe998000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 shutdown_connections 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29ab3f80 0x55aa29ab6370 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aa0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2894140709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aa3720 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe983fff640 0 client.16623 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 >> 172.21.15.122:0/2894140709 conn(0x55aa29a94ed0 msgr2=0x55aa29a96c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 shutdown_connections 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2894140709 wait complete. 2024-04-02T13:38:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.535+0000 7fe9ad9ca740 20 client.16623 trim_cache size 0 max 16384 2024-04-02T13:38:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 Processor -- start 2024-04-02T13:38:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 -- start start 2024-04-02T13:38:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29a9b0c0 0x55aa29a9fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41aefa0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41b0c00 con 0x55aa29aa3ca0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41af670 con 0x55aa29a9b0c0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43766/0 (socket says 172.21.15.122:43766) 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2986067986 learned_addr learned my addr 172.21.15.122:0/2986067986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2986067986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29a9b0c0 msgr2=0x55aa29a9fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29a9b0c0 0x55aa29a9fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2986067986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29aa3ca0 msgr2=0x55aa29aa31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.548+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/2986067986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa29aa1560 con 0x55aa29a9a7a0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe99801e8f0 tx=0x7fe998013400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b079db7f61f594 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe98e7fc640 1 -- 172.21.15.122:0/2986067986 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99802f040 con 0x55aa29a9a7a0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe98e7fc640 1 -- 172.21.15.122:0/2986067986 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe998026050 con 0x55aa29a9a7a0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe98e7fc640 1 -- 172.21.15.122:0/2986067986 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998024030 con 0x55aa29a9a7a0 2024-04-02T13:38:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2986067986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 msgr2=0x55aa29aaf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe99801e8f0 tx=0x7fe998013400 comp rx=0 tx=0).stop 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2986067986 shutdown_connections 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29aa3ca0 0x55aa29aa31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29a9b0c0 0x55aa29a9fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/2986067986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aaf8f0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2986067986 >> 172.21.15.122:0/2986067986 conn(0x55aa29a94ed0 msgr2=0x55aa29a9f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2986067986 shutdown_connections 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.549+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/2986067986 wait complete. 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 -- start start 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 Processor -- start 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29a9d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41b0470 con 0x55aa29a9a7a0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41afda0 con 0x55aa29a9b0c0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a41acf80 con 0x55aa29aa3ca0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9ad9ca740 10 client.0 _ll_register_callbacks cb 0x7fe998022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43778/0 (socket says 172.21.15.122:43778) 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/695192300 learned_addr learned my addr 172.21.15.122:0/695192300 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/695192300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 msgr2=0x55aa29a9d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29a9d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 msgr2=0x55aa29aac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.550+0000 7fe9a8aa0640 1 -- 172.21.15.122:0/695192300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa29a9da50 con 0x55aa29a9a7a0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9a8aa0640 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe9980125b0 tx=0x7fe998009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41455524589d51f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:00.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998024070 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe99801f450 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe998026050 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe99804b050 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 10 client.16629 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa29b19ee0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe998026050 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe9980052a0 con 0x55aa29a9a7a0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 10 client.16629 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 20 client.16629 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29b1a650 0x55aa29b1cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 4 client.16629 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa29a06040 con 0x55aa29b1a650 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.551+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29b1a650 0x55aa29b1cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:00.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe983fff640 10 client.16629 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9a3fff640 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29b1a650 0x55aa29b1cab0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688095a781c1ac5b server_cookie=3a990420bd58d3f2 in_seq=0 out_seq=0 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe983fff640 1 -- 172.21.15.122:0/695192300 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa29a06040 con 0x55aa29b1a650 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe983fff640 10 client.16629 handle_command_reply: tid=0 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 client.16629 shutdown 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 msgr2=0x55aa29aabe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe9980125b0 tx=0x7fe998009590 comp rx=0 tx=0).stop 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 shutdown_connections 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55aa29b1a650 0x55aa29b1cab0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa29aa3ca0 0x55aa29a9d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe983fff640 0 client.16629 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa29a9b0c0 0x55aa29aac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 --2- 172.21.15.122:0/695192300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa29a9a7a0 0x55aa29aabe00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 >> 172.21.15.122:0/695192300 conn(0x55aa29a94ed0 msgr2=0x55aa29a961d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 shutdown_connections 2024-04-02T13:38:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 1 -- 172.21.15.122:0/695192300 wait complete. 2024-04-02T13:38:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.552+0000 7fe9ad9ca740 20 client.16629 trim_cache size 0 max 16384 2024-04-02T13:38:00.561 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 msgr2=0x7fe9940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 0x7fe9940a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe998015470 tx=0x7fe998015870 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 msgr2=0x7fe9a41a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00cce0 tx=0x7fe99c00a620 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 shutdown_connections 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe9940a3d20 0x7fe9940a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9a41069c0 0x7fe9a41a4270 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9a4101370 0x7fe9a4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 --2- 172.21.15.122:0/2056388492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9a41009b0 0x7fe9a41991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 >> 172.21.15.122:0/2056388492 conn(0x7fe9a4076d80 msgr2=0x7fe9a40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 shutdown_connections 2024-04-02T13:38:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:00.553+0000 7fe9aad04640 1 -- 172.21.15.122:0/2056388492 wait complete. 2024-04-02T13:38:00.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 78877 5.46 MB/sec warmup 81 sec latency 2159.482 ms 2024-04-02T13:38:01.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:00 smithi187 ceph-mon[36064]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 421 KiB/s wr, 72 op/s 2024-04-02T13:38:01.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:00 smithi188 ceph-mon[36117]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 421 KiB/s wr, 72 op/s 2024-04-02T13:38:01.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:00 smithi122 ceph-mon[32114]: pgmap v88: 129 pgs: 129 active+clean; 54 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 421 KiB/s wr, 72 op/s 2024-04-02T13:38:01.566 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:38:01.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:01.567 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:38:01.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 -- 172.21.15.122:0/3647888930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c101330 msgr2=0x7f613c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 --2- 172.21.15.122:0/3647888930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c101330 0x7f613c10d840 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f6134009cd0 tx=0x7f613402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 -- 172.21.15.122:0/3647888930 shutdown_connections 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 --2- 172.21.15.122:0/3647888930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c101330 0x7f613c10d840 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 --2- 172.21.15.122:0/3647888930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c100970 0x7f613c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 --2- 172.21.15.122:0/3647888930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c106980 0x7f613c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 -- 172.21.15.122:0/3647888930 >> 172.21.15.122:0/3647888930 conn(0x7f613c077b00 msgr2=0x7f613c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 -- 172.21.15.122:0/3647888930 shutdown_connections 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 -- 172.21.15.122:0/3647888930 wait complete. 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.691+0000 7f61430eb640 1 Processor -- start 2024-04-02T13:38:01.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 -- start start 2024-04-02T13:38:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 0x7f613c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c101330 0x7f613c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f613c110d60 con 0x7f613c106980 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613c110be0 con 0x7f613c100970 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f61430eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f613c110ee0 con 0x7f613c101330 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43806/0 (socket says 172.21.15.122:43806) 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 -- 172.21.15.122:0/4065537208 learned_addr learned my addr 172.21.15.122:0/4065537208 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6140e87640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 0x7f613c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f613bfff640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c101330 0x7f613c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 -- 172.21.15.122:0/4065537208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c101330 msgr2=0x7f613c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c101330 0x7f613c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 -- 172.21.15.122:0/4065537208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 msgr2=0x7f613c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 0x7f613c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 -- 172.21.15.122:0/4065537208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c19d2a0 con 0x7f613c106980 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6140e87640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 0x7f613c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.692+0000 7f6141688640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6134037040 tx=0x7f6134035ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.693+0000 7f6139ffb640 1 -- 172.21.15.122:0/4065537208 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134039040 con 0x7f613c106980 2024-04-02T13:38:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.693+0000 7f6139ffb640 1 -- 172.21.15.122:0/4065537208 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6134044070 con 0x7f613c106980 2024-04-02T13:38:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.693+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c19d530 con 0x7f613c106980 2024-04-02T13:38:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.693+0000 7f6139ffb640 1 -- 172.21.15.122:0/4065537208 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134004330 con 0x7f613c106980 2024-04-02T13:38:01.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.693+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1a4e20 con 0x7f613c106980 2024-04-02T13:38:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.695+0000 7f6139ffb640 1 -- 172.21.15.122:0/4065537208 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6134003970 con 0x7f613c106980 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 Processor -- start 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 -- start start 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6139ffb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 0x7f612c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fa76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fb2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5625a4f0b770 con 0x5625a4fb07c0 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5625a4eb3960 con 0x5625a4fa72c0 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6145dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5625a4f078c0 con 0x5625a4fa7be0 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f613bfff640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 0x7f612c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6139ffb640 1 -- 172.21.15.122:0/4065537208 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f61340f3070 con 0x7f613c106980 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f613bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f613bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:43228/0 (socket says 172.21.15.122:43228) 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6141688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fb2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f613bfff640 1 -- 172.21.15.122:0/4183219504 learned_addr learned my addr 172.21.15.122:0/4183219504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.696+0000 7f6140e87640 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fa76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 -- 172.21.15.122:0/4183219504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 msgr2=0x5625a4fb0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 -- 172.21.15.122:0/4183219504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 msgr2=0x5625a4fa76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fa76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 -- 172.21.15.122:0/4183219504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a4ce77e0 con 0x5625a4fb07c0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f613bfff640 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f613bfff640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 0x7f612c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f612800afd0 tx=0x7f612800ab40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6141688640 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fb2bb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f61340f0540 tx=0x7f61340baa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ec0badf43626f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6126ffd640 1 -- 172.21.15.122:0/4183219504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340bf490 con 0x5625a4fb07c0 2024-04-02T13:38:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6126ffd640 1 -- 172.21.15.122:0/4183219504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340bed90 con 0x5625a4fb07c0 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.697+0000 7f6126ffd640 1 -- 172.21.15.122:0/4183219504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340333a0 con 0x5625a4fb07c0 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- 172.21.15.122:0/4183219504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 msgr2=0x5625a4fb2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fb2bb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f61340f0540 tx=0x7f61340baa30 comp rx=0 tx=0).stop 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- 172.21.15.122:0/4183219504 shutdown_connections 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fb2bb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fb0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 --2- 172.21.15.122:0/4183219504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fa76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- 172.21.15.122:0/4183219504 >> 172.21.15.122:0/4183219504 conn(0x5625a4fa19f0 msgr2=0x5625a4fa1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- 172.21.15.122:0/4183219504 shutdown_connections 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- 172.21.15.122:0/4183219504 wait complete. 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 -- start start 2024-04-02T13:38:01.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.698+0000 7f6145dab740 1 Processor -- start 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fadc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a4fae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5625a4fb3370 con 0x5625a4fa72c0 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5625a4ec6d20 con 0x5625a4fb07c0 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5625a4fb34f0 con 0x5625a4fa7be0 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6145dab740 10 client.0 _ll_register_callbacks cb 0x5625a4e93a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43820/0 (socket says 172.21.15.122:43820) 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 -- 172.21.15.122:0/2383805216 learned_addr learned my addr 172.21.15.122:0/2383805216 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f613bfff640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fadc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6141688640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a4fae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 -- 172.21.15.122:0/2383805216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 msgr2=0x5625a4fadc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fadc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 -- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 msgr2=0x5625a4fae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a4fae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.699+0000 7f6140e87640 1 -- 172.21.15.122:0/2383805216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a4d47f70 con 0x5625a4fa72c0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6140e87640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6130011a30 tx=0x7f6130011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8080ee851f5550cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613000f940 con 0x5625a4fa72c0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f613000fae0 con 0x5625a4fa72c0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61340bed90 con 0x5625a4fa72c0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f613000fce0 con 0x5625a4fa72c0 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 10 client.16641 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625a4bdf170 con 0x5625a4fa72c0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340bed90 con 0x5625a4fa72c0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6130018070 con 0x5625a4fa72c0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 10 client.16641 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 20 client.16641 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a4fc08d0 0x5625a4fc2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 4 client.16641 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.700+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625a4f18cb0 con 0x5625a4fc08d0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.701+0000 7f613bfff640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a4fc08d0 0x5625a4fc2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.701+0000 7f611bfff640 10 client.16641 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.701+0000 7f613bfff640 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a4fc08d0 0x5625a4fc2d70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30dffa48bdd57d6 server_cookie=db1031c99c2c3396 in_seq=0 out_seq=0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f611bfff640 1 -- 172.21.15.122:0/2383805216 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5625a4f18cb0 con 0x5625a4fc08d0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f611bfff640 10 client.16641 handle_command_reply: tid=0 2024-04-02T13:38:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 client.16641 shutdown 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 msgr2=0x5625a4fad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6130011a30 tx=0x7f6130011ef0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 shutdown_connections 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a4fc08d0 0x5625a4fc2d70 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a4fae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4fadc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f611bfff640 0 client.16641 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 --2- 172.21.15.122:0/2383805216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa72c0 0x5625a4fad700 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 >> 172.21.15.122:0/2383805216 conn(0x5625a4fa19f0 msgr2=0x5625a4fa3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 shutdown_connections 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 1 -- 172.21.15.122:0/2383805216 wait complete. 2024-04-02T13:38:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.702+0000 7f6145dab740 20 client.16641 trim_cache size 0 max 16384 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 Processor -- start 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 -- start start 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fbc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4facc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1acc10 con 0x5625a4fb07c0 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1ae870 con 0x5625a4fa72c0 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6145dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1ad2e0 con 0x5625a4fa7be0 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6141688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6141688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43830/0 (socket says 172.21.15.122:43830) 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6141688640 1 -- 172.21.15.122:0/4191460619 learned_addr learned my addr 172.21.15.122:0/4191460619 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.720+0000 7f6140e87640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fbc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f613bfff640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4facc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 -- 172.21.15.122:0/4191460619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 msgr2=0x5625a4facc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4facc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 -- 172.21.15.122:0/4191460619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 msgr2=0x5625a4fbc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fbc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 -- 172.21.15.122:0/4191460619 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625a4fa3cf0 con 0x5625a4fb07c0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6140e87640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fbc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f6141688640 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f61340bb760 tx=0x7f6134003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f3d4fbac78963d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f61267fc640 1 -- 172.21.15.122:0/4191460619 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340bf490 con 0x5625a4fb07c0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f61267fc640 1 -- 172.21.15.122:0/4191460619 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340c3140 con 0x5625a4fb07c0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.721+0000 7f61267fc640 1 -- 172.21.15.122:0/4191460619 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340333a0 con 0x5625a4fb07c0 2024-04-02T13:38:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- 172.21.15.122:0/4191460619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 msgr2=0x5625a4fc8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f61340bb760 tx=0x7f6134003d80 comp rx=0 tx=0).stop 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- 172.21.15.122:0/4191460619 shutdown_connections 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fb07c0 0x5625a4fc8640 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa7be0 0x5625a4facc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- 172.21.15.122:0/4191460619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fa72c0 0x5625a4fbc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- 172.21.15.122:0/4191460619 >> 172.21.15.122:0/4191460619 conn(0x5625a4fa19f0 msgr2=0x5625a4fa0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- 172.21.15.122:0/4191460619 shutdown_connections 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- 172.21.15.122:0/4191460619 wait complete. 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- start start 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 Processor -- start 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa72c0 0x5625a4fb4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a501fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1ae0e0 con 0x5625a4fa7be0 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1ada10 con 0x5625a4fb07c0 2024-04-02T13:38:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.722+0000 7f6145dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1aabf0 con 0x5625a4fa72c0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6145dab740 10 client.0 _ll_register_callbacks cb 0x7f6130011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43832/0 (socket says 172.21.15.122:43832) 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 -- 172.21.15.122:0/1816966918 learned_addr learned my addr 172.21.15.122:0/1816966918 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6140e87640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa72c0 0x5625a4fb4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6141688640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a501fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 -- 172.21.15.122:0/1816966918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa72c0 msgr2=0x5625a4fb4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa72c0 0x5625a4fb4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 -- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 msgr2=0x5625a501fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a501fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 -- 172.21.15.122:0/1816966918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625a4fa7a10 con 0x5625a4fa7be0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6141688640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a501fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f613bfff640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f61280106d0 tx=0x7f6128004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592b74b203483384 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f611bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61280297f0 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61340c3140 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f6128005030 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f612803b050 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6145dab740 10 client.16647 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.723+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5625a5027670 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61340c3140 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f612801d040 con 0x5625a4fa7be0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 10 client.16647 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 20 client.16647 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a5027e80 0x5625a502a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 4 client.16647 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625a4f13000 con 0x5625a5027e80 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6140e87640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a5027e80 0x5625a502a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f611bfff640 10 client.16647 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.724+0000 7f6140e87640 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a5027e80 0x5625a502a2a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9130ad7ce0db7d9b server_cookie=72b0cac89ee0f558 in_seq=0 out_seq=0 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.725+0000 7f611bfff640 1 -- 172.21.15.122:0/1816966918 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625a4f13000 con 0x5625a5027e80 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f611bfff640 10 client.16647 handle_command_reply: tid=0 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 client.16647 shutdown 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 msgr2=0x5625a4fb4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f61280106d0 tx=0x7f6128004730 comp rx=0 tx=0).stop 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 shutdown_connections 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5625a5027e80 0x5625a502a2a0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5625a4fb07c0 0x5625a501fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5625a4fa7be0 0x5625a4fb4ea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 --2- 172.21.15.122:0/1816966918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5625a4fa72c0 0x5625a4fb4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 >> 172.21.15.122:0/1816966918 conn(0x5625a4fa19f0 msgr2=0x5625a4fadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f611bfff640 0 client.16647 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 shutdown_connections 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 1 -- 172.21.15.122:0/1816966918 wait complete. 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.726+0000 7f6145dab740 20 client.16647 trim_cache size 0 max 16384 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "f406e05a-8e9d-4b96-96a3-a8e1eb496d89", 2024-04-02T13:38:01.734 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:38:01.735 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:38:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 msgr2=0x7f612c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 0x7f612c0a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f612800afd0 tx=0x7f612800ab40 comp rx=0 tx=0).stop 2024-04-02T13:38:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 msgr2=0x7f613c19cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6134037040 tx=0x7f6134035ae0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 shutdown_connections 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f612c0a3d20 0x7f612c0a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f613c106980 0x7f613c19cb70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f613c101330 0x7f613c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 --2- 172.21.15.122:0/4065537208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f613c100970 0x7f613c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 >> 172.21.15.122:0/4065537208 conn(0x7f613c077b00 msgr2=0x7f613c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 shutdown_connections 2024-04-02T13:38:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.727+0000 7f61430eb640 1 -- 172.21.15.122:0/4065537208 wait complete. 2024-04-02T13:38:01.741 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:38:01.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 -- 172.21.15.122:0/764395361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0106980 msgr2=0x7feef0106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 --2- 172.21.15.122:0/764395361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0106980 0x7feef0106d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7feee4009930 tx=0x7feee402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 -- 172.21.15.122:0/764395361 shutdown_connections 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 --2- 172.21.15.122:0/764395361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0101330 0x7feef010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 --2- 172.21.15.122:0/764395361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0100970 0x7feef0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 --2- 172.21.15.122:0/764395361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0106980 0x7feef0106d60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 -- 172.21.15.122:0/764395361 >> 172.21.15.122:0/764395361 conn(0x7feef0077b00 msgr2=0x7feef0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 -- 172.21.15.122:0/764395361 shutdown_connections 2024-04-02T13:38:01.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 -- 172.21.15.122:0/764395361 wait complete. 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.867+0000 7feef6aa7640 1 Processor -- start 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 -- start start 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 0x7feef0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0106980 0x7feef019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feef0110d60 con 0x7feef0101330 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feef0110be0 con 0x7feef0100970 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef6aa7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feef0110ee0 con 0x7feef0106980 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43850/0 (socket says 172.21.15.122:43850) 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 -- 172.21.15.122:0/1023792463 learned_addr learned my addr 172.21.15.122:0/1023792463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef4843640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 0x7feef0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef5044640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0106980 0x7feef019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 -- 172.21.15.122:0/1023792463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0106980 msgr2=0x7feef019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0106980 0x7feef019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 -- 172.21.15.122:0/1023792463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 msgr2=0x7feef0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 0x7feef0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 -- 172.21.15.122:0/1023792463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef019d380 con 0x7feef0101330 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feef4843640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 0x7feef0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.868+0000 7feeeffff640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feedc00e9f0 tx=0x7feedc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.869+0000 7feeedffb640 1 -- 172.21.15.122:0/1023792463 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc00cd80 con 0x7feef0101330 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.869+0000 7feeedffb640 1 -- 172.21.15.122:0/1023792463 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feedc010910 con 0x7feef0101330 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.869+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef019d670 con 0x7feef0101330 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.869+0000 7feeedffb640 1 -- 172.21.15.122:0/1023792463 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc018990 con 0x7feef0101330 2024-04-02T13:38:01.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.869+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01a4f00 con 0x7feef0101330 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.871+0000 7feeedffb640 1 -- 172.21.15.122:0/1023792463 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7feedc010ac0 con 0x7feef0101330 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.871+0000 7feef9768740 1 Processor -- start 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 -- start start 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeedffb640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 0x7feee00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1a2f830 con 0x55f0f1ad4dd0 2024-04-02T13:38:01.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0f19d7820 con 0x55f0f1acc1f0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef9768740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1a2b7e0 con 0x55f0f1acb8d0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef5044640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 0x7feee00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeedffb640 1 -- 172.21.15.122:0/1023792463 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7feedc0d0f60 con 0x7feef0101330 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef4843640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef5044640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef4843640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:43286/0 (socket says 172.21.15.122:43286) 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef4843640 1 -- 172.21.15.122:0/4214461342 learned_addr learned my addr 172.21.15.122:0/4214461342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef5044640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 0x7feee00a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feee8005fd0 tx=0x7feee8012a70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 -- 172.21.15.122:0/4214461342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 msgr2=0x55f0f1acbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 -- 172.21.15.122:0/4214461342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 msgr2=0x55f0f1ad71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 -- 172.21.15.122:0/4214461342 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f180d7d0 con 0x55f0f1acc1f0 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef4843640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feef5044640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.872+0000 7feeeffff640 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad4890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ce320 tx=0x7feedc09d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69552f7409e9c7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feedaffd640 1 -- 172.21.15.122:0/4214461342 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc0a57e0 con 0x55f0f1acc1f0 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feedaffd640 1 -- 172.21.15.122:0/4214461342 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feedc0bd070 con 0x55f0f1acc1f0 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feedaffd640 1 -- 172.21.15.122:0/4214461342 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc018d90 con 0x55f0f1acc1f0 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 -- 172.21.15.122:0/4214461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 msgr2=0x55f0f1ad4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad4890 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ce320 tx=0x7feedc09d330 comp rx=0 tx=0).stop 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 -- 172.21.15.122:0/4214461342 shutdown_connections 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad4890 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 --2- 172.21.15.122:0/4214461342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acb8d0 0x55f0f1acbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 -- 172.21.15.122:0/4214461342 >> 172.21.15.122:0/4214461342 conn(0x55f0f1ac6000 msgr2=0x55f0f1ac63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 -- 172.21.15.122:0/4214461342 shutdown_connections 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.873+0000 7feef9768740 1 -- 172.21.15.122:0/4214461342 wait complete. 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 -- start start 2024-04-02T13:38:01.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 Processor -- start 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1ad7b00 con 0x55f0f1acc1f0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.874+0000 7feef9768740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0f19ead30 con 0x55f0f1acb8d0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef9768740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0f1ad7c80 con 0x55f0f1ad4dd0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feeeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef9768740 10 client.0 _ll_register_callbacks cb 0x55f0f19ea720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:01.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47360/0 (socket says 172.21.15.122:47360) 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 -- 172.21.15.122:0/2352751747 learned_addr learned my addr 172.21.15.122:0/2352751747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef5044640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 -- 172.21.15.122:0/2352751747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 msgr2=0x55f0f1ad1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 -- 172.21.15.122:0/2352751747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 msgr2=0x55f0f1ad1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 -- 172.21.15.122:0/2352751747 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f17433f0 con 0x55f0f1acb8d0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feeeffff640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef5044640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef4843640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feee4009230 tx=0x7feee402cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4dc33e1f63772a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4004040 con 0x55f0f1acb8d0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7feee40041e0 con 0x55f0f1acb8d0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.875+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feedc0bd070 con 0x55f0f1acb8d0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feee40043e0 con 0x55f0f1acb8d0 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 10 client.26036 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:01.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0f17030f0 con 0x55f0f1acb8d0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feedc0bd070 con 0x55f0f1acb8d0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7feee4003990 con 0x55f0f1acb8d0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 10 client.26036 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 10 client.26036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 20 client.26036 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1ae4d90 0x55f0f1ae71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 4 client.26036 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0f1a3cd70 con 0x55f0f1ae4d90 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.876+0000 7feeeffff640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1ae4d90 0x55f0f1ae71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feecffff640 10 client.26036 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feeeffff640 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1ae4d90 0x55f0f1ae71a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fecf43f637b4a043 server_cookie=2de275cca5be64fa in_seq=0 out_seq=0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feecffff640 1 -- 172.21.15.122:0/2352751747 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0f1a3cd70 con 0x55f0f1ae4d90 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feecffff640 10 client.26036 handle_command_reply: tid=0 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 client.26036 shutdown 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 msgr2=0x55f0f1ad4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feee4009230 tx=0x7feee402cab0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 shutdown_connections 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1ae4d90 0x55f0f1ae71a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feecffff640 0 client.26036 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ad1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 --2- 172.21.15.122:0/2352751747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ad4690 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.877+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 >> 172.21.15.122:0/2352751747 conn(0x55f0f1ac6000 msgr2=0x55f0f1ac7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.878+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 shutdown_connections 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.878+0000 7feef9768740 1 -- 172.21.15.122:0/2352751747 wait complete. 2024-04-02T13:38:01.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.878+0000 7feef9768740 20 client.26036 trim_cache size 0 max 16384 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 Processor -- start 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 -- start start 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ac5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feef01aef90 con 0x55f0f1acb8d0 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b0bf0 con 0x55f0f1ad4dd0 2024-04-02T13:38:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef9768740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feef01af660 con 0x55f0f1acc1f0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef4843640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef4843640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43868/0 (socket says 172.21.15.122:43868) 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef4843640 1 -- 172.21.15.122:0/2702367201 learned_addr learned my addr 172.21.15.122:0/2702367201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feeeffff640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ac5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.898+0000 7feef5044640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 -- 172.21.15.122:0/2702367201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 msgr2=0x55f0f1ac5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ac5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 -- 172.21.15.122:0/2702367201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1ad4dd0 msgr2=0x55f0f1ad4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 -- 172.21.15.122:0/2702367201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0f1ac5e30 con 0x55f0f1acb8d0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feeeffff640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ac5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feef4843640 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feee4000c00 tx=0x7feee402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf45f808b829710e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feeda7fc640 1 -- 172.21.15.122:0/2702367201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee405a070 con 0x55f0f1acb8d0 2024-04-02T13:38:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feeda7fc640 1 -- 172.21.15.122:0/2702367201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee4009520 con 0x55f0f1acb8d0 2024-04-02T13:38:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.899+0000 7feeda7fc640 1 -- 172.21.15.122:0/2702367201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4034780 con 0x55f0f1acb8d0 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- 172.21.15.122:0/2702367201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 msgr2=0x55f0f1ae09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7feee4000c00 tx=0x7feee402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- 172.21.15.122:0/2702367201 shutdown_connections 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ad4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ac5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 --2- 172.21.15.122:0/2702367201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acb8d0 0x55f0f1ae09f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- 172.21.15.122:0/2702367201 >> 172.21.15.122:0/2702367201 conn(0x55f0f1ac6000 msgr2=0x55f0f1ad0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- 172.21.15.122:0/2702367201 shutdown_connections 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- 172.21.15.122:0/2702367201 wait complete. 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 -- start start 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.900+0000 7feef9768740 1 Processor -- start 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1add0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ace5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feef01b0460 con 0x55f0f1acc1f0 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feef01afd90 con 0x55f0f1acb8d0 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feef01acf70 con 0x55f0f1ad4dd0 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 10 client.0 _ll_register_callbacks cb 0x7feee402c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43880/0 (socket says 172.21.15.122:43880) 2024-04-02T13:38:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 -- 172.21.15.122:0/192595942 learned_addr learned my addr 172.21.15.122:0/192595942 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef4843640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1add0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef5044640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ace5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 -- 172.21.15.122:0/192595942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 msgr2=0x55f0f1ace5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ace5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 -- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 msgr2=0x55f0f1add0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1add0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 -- 172.21.15.122:0/192595942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0f1ad71c0 con 0x55f0f1acc1f0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feeeffff640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ce920 tx=0x7feedc098f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b056c3215ef7b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feedc018b30 con 0x55f0f1acc1f0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7feedc018cd0 con 0x55f0f1acc1f0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee4009520 con 0x55f0f1acc1f0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.901+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feedc0ac050 con 0x55f0f1acc1f0 2024-04-02T13:38:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 10 client.16665 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0f1b4f220 con 0x55f0f1acc1f0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feee4009520 con 0x55f0f1acc1f0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7feedc0b7020 con 0x55f0f1acc1f0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 10 client.16665 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 20 client.16665 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1b4f560 0x55f0f1b51980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 4 client.16665 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0f1a370c0 con 0x55f0f1b4f560 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef4843640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1b4f560 0x55f0f1b51980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feef4843640 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1b4f560 0x55f0f1b51980 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b98b15c5ce65aab server_cookie=74ae1d42d34f7e4c in_seq=0 out_seq=0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.902+0000 7feecffff640 10 client.16665 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feecffff640 1 -- 172.21.15.122:0/192595942 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f0f1a370c0 con 0x55f0f1b4f560 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feecffff640 10 client.16665 handle_command_reply: tid=0 2024-04-02T13:38:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 client.16665 shutdown 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 msgr2=0x55f0f1ace0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7feedc0ce920 tx=0x7feedc098f40 comp rx=0 tx=0).stop 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 shutdown_connections 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0f1b4f560 0x55f0f1b51980 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0f1ad4dd0 0x55f0f1ace5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0f1acc1f0 0x55f0f1ace0a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 --2- 172.21.15.122:0/192595942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0f1acb8d0 0x55f0f1add0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feecffff640 0 client.16665 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 >> 172.21.15.122:0/192595942 conn(0x55f0f1ac6000 msgr2=0x55f0f1ac5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 shutdown_connections 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 1 -- 172.21.15.122:0/192595942 wait complete. 2024-04-02T13:38:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.903+0000 7feef9768740 20 client.16665 trim_cache size 0 max 16384 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "f406e05a-8e9d-4b96-96a3-a8e1eb496d89": { 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "f406e05a-8e9d-4b96-96a3-a8e1eb496d89", 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:38:01.912 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 msgr2=0x7feee00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 0x7feee00a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feee8005fd0 tx=0x7feee8012a70 comp rx=0 tx=0).stop 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 msgr2=0x7feef01988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7feedc00e9f0 tx=0x7feedc00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 shutdown_connections 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feee00a3d20 0x7feee00a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feef0106980 0x7feef019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feef0101330 0x7feef01988d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 --2- 172.21.15.122:0/1023792463 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feef0100970 0x7feef0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 >> 172.21.15.122:0/1023792463 conn(0x7feef0077b00 msgr2=0x7feef0068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 shutdown_connections 2024-04-02T13:38:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:01.904+0000 7feef6aa7640 1 -- 172.21.15.122:0/1023792463 wait complete. 2024-04-02T13:38:01.917 INFO:tasks.cephfs.filesystem:scrub status for tag:f406e05a-8e9d-4b96-96a3-a8e1eb496d89 - {'path': '/', 'tag': 'f406e05a-8e9d-4b96-96a3-a8e1eb496d89', 'options': 'recursive,force'} 2024-04-02T13:38:01.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 79094 5.40 MB/sec warmup 82 sec latency 1088.354 ms 2024-04-02T13:38:02.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:02.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:02.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:02.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 80422 5.40 MB/sec warmup 83 sec latency 58.969 ms 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:02 smithi187 ceph-mon[36064]: pgmap v89: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 485 KiB/s wr, 100 op/s 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:02 smithi188 ceph-mon[36117]: pgmap v89: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 485 KiB/s wr, 100 op/s 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:03.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:02 smithi122 ceph-mon[32114]: pgmap v89: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 485 KiB/s wr, 100 op/s 2024-04-02T13:38:03.943 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82092 5.49 MB/sec warmup 84 sec latency 14.381 ms 2024-04-02T13:38:04.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82302 5.42 MB/sec warmup 85 sec latency 850.682 ms 2024-04-02T13:38:05.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:04 smithi187 ceph-mon[36064]: pgmap v90: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 415 KiB/s wr, 79 op/s 2024-04-02T13:38:05.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:04 smithi188 ceph-mon[36117]: pgmap v90: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 415 KiB/s wr, 79 op/s 2024-04-02T13:38:05.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:04 smithi122 ceph-mon[32114]: pgmap v90: 129 pgs: 129 active+clean; 53 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 415 KiB/s wr, 79 op/s 2024-04-02T13:38:05.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82302 5.36 MB/sec warmup 86 sec latency 1850.791 ms 2024-04-02T13:38:06.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82302 5.30 MB/sec warmup 87 sec latency 2850.917 ms 2024-04-02T13:38:07.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:06 smithi187 ceph-mon[36064]: pgmap v91: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 652 KiB/s wr, 126 op/s 2024-04-02T13:38:07.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:06 smithi188 ceph-mon[36117]: pgmap v91: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 652 KiB/s wr, 126 op/s 2024-04-02T13:38:07.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:06 smithi122 ceph-mon[32114]: pgmap v91: 129 pgs: 129 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 652 KiB/s wr, 126 op/s 2024-04-02T13:38:07.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82302 5.24 MB/sec warmup 88 sec latency 3850.953 ms 2024-04-02T13:38:08.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82426 5.18 MB/sec warmup 89 sec latency 4311.996 ms 2024-04-02T13:38:09.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:08 smithi187 ceph-mon[36064]: pgmap v92: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 425 KiB/s wr, 93 op/s 2024-04-02T13:38:09.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:08 smithi188 ceph-mon[36117]: pgmap v92: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 425 KiB/s wr, 93 op/s 2024-04-02T13:38:09.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:08 smithi122 ceph-mon[32114]: pgmap v92: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 425 KiB/s wr, 93 op/s 2024-04-02T13:38:09.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82426 5.12 MB/sec warmup 90 sec latency 1432.324 ms 2024-04-02T13:38:10.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82426 5.07 MB/sec warmup 91 sec latency 2432.364 ms 2024-04-02T13:38:11.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:10 smithi187 ceph-mon[36064]: pgmap v93: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 354 KiB/s wr, 87 op/s 2024-04-02T13:38:11.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:10 smithi188 ceph-mon[36117]: pgmap v93: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 354 KiB/s wr, 87 op/s 2024-04-02T13:38:11.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:10 smithi122 ceph-mon[32114]: pgmap v93: 129 pgs: 129 active+clean; 56 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 354 KiB/s wr, 87 op/s 2024-04-02T13:38:11.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82426 5.01 MB/sec warmup 92 sec latency 3432.404 ms 2024-04-02T13:38:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:11 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:11 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:11 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:11 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:11 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:11 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:11 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:11 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:11 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:11 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:12.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:11 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:12.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:11 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:12.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:11 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:12.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:11 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:12.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:11 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:12.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 82426 4.96 MB/sec warmup 93 sec latency 4432.491 ms 2024-04-02T13:38:13.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:12 smithi187 ceph-mon[36064]: pgmap v94: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 396 KiB/s wr, 99 op/s 2024-04-02T13:38:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:12 smithi188 ceph-mon[36117]: pgmap v94: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 396 KiB/s wr, 99 op/s 2024-04-02T13:38:13.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:12 smithi122 ceph-mon[32114]: pgmap v94: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 396 KiB/s wr, 99 op/s 2024-04-02T13:38:13.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 84031 4.98 MB/sec warmup 94 sec latency 4886.799 ms 2024-04-02T13:38:14.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 85643 5.05 MB/sec warmup 95 sec latency 16.408 ms 2024-04-02T13:38:15.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:14 smithi187 ceph-mon[36064]: pgmap v95: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 332 KiB/s wr, 71 op/s 2024-04-02T13:38:15.266 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:14 smithi188 ceph-mon[36117]: pgmap v95: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 332 KiB/s wr, 71 op/s 2024-04-02T13:38:15.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:14 smithi122 ceph-mon[32114]: pgmap v95: 129 pgs: 129 active+clean; 56 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 332 KiB/s wr, 71 op/s 2024-04-02T13:38:15.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 85843 5.00 MB/sec warmup 96 sec latency 848.085 ms 2024-04-02T13:38:16.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 85843 4.95 MB/sec warmup 97 sec latency 1848.213 ms 2024-04-02T13:38:17.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:16 smithi188 ceph-mon[36117]: pgmap v96: 129 pgs: 129 active+clean; 58 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 520 KiB/s wr, 109 op/s 2024-04-02T13:38:17.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:16 smithi122 ceph-mon[32114]: pgmap v96: 129 pgs: 129 active+clean; 58 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 520 KiB/s wr, 109 op/s 2024-04-02T13:38:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:16 smithi187 ceph-mon[36064]: pgmap v96: 129 pgs: 129 active+clean; 58 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 520 KiB/s wr, 109 op/s 2024-04-02T13:38:17.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 85843 4.90 MB/sec warmup 98 sec latency 2848.273 ms 2024-04-02T13:38:18.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:17 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:18.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:17 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:18.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:17 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:38:18.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 87132 4.90 MB/sec warmup 99 sec latency 3304.785 ms 2024-04-02T13:38:19.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:18 smithi188 ceph-mon[36117]: pgmap v97: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 310 KiB/s wr, 66 op/s 2024-04-02T13:38:19.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:18 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:19.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:18 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:19.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:18 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:38:19.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:18 smithi122 ceph-mon[32114]: pgmap v97: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 310 KiB/s wr, 66 op/s 2024-04-02T13:38:19.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:18 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:19.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:18 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:19.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:18 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:38:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:18 smithi187 ceph-mon[36064]: pgmap v97: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 310 KiB/s wr, 66 op/s 2024-04-02T13:38:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:18 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:38:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:18 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:38:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:18 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:38:19.944 INFO:tasks.workunit.client.0.smithi122.stdout: 1 89125 4.99 MB/sec warmup 100 sec latency 10.210 ms 2024-04-02T13:38:20.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 89448 4.94 MB/sec warmup 101 sec latency 688.481 ms 2024-04-02T13:38:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:20 smithi188 ceph-mon[36117]: pgmap v98: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 257 KiB/s wr, 54 op/s 2024-04-02T13:38:21.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:20 smithi122 ceph-mon[32114]: pgmap v98: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 257 KiB/s wr, 54 op/s 2024-04-02T13:38:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:20 smithi187 ceph-mon[36064]: pgmap v98: 129 pgs: 129 active+clean; 58 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 257 KiB/s wr, 54 op/s 2024-04-02T13:38:21.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 89448 4.90 MB/sec warmup 102 sec latency 1688.600 ms 2024-04-02T13:38:22.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 89448 4.85 MB/sec warmup 103 sec latency 2688.692 ms 2024-04-02T13:38:23.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:22 smithi188 ceph-mon[36117]: pgmap v99: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 507 KiB/s wr, 100 op/s 2024-04-02T13:38:23.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:22 smithi122 ceph-mon[32114]: pgmap v99: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 507 KiB/s wr, 100 op/s 2024-04-02T13:38:23.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:22 smithi187 ceph-mon[36064]: pgmap v99: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 929 KiB/s rd, 507 KiB/s wr, 100 op/s 2024-04-02T13:38:23.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 91022 4.86 MB/sec warmup 104 sec latency 3141.140 ms 2024-04-02T13:38:24.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 92694 4.93 MB/sec warmup 105 sec latency 19.960 ms 2024-04-02T13:38:25.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:25 smithi188 ceph-mon[36117]: pgmap v100: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 465 KiB/s wr, 88 op/s 2024-04-02T13:38:25.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:24 smithi122 ceph-mon[32114]: pgmap v100: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 465 KiB/s wr, 88 op/s 2024-04-02T13:38:25.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:25 smithi187 ceph-mon[36064]: pgmap v100: 129 pgs: 129 active+clean; 60 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 465 KiB/s wr, 88 op/s 2024-04-02T13:38:25.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 92992 4.89 MB/sec warmup 106 sec latency 739.123 ms 2024-04-02T13:38:26.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 92992 4.85 MB/sec warmup 107 sec latency 1739.249 ms 2024-04-02T13:38:27.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:27 smithi188 ceph-mon[36117]: pgmap v101: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 KiB/s wr, 132 op/s 2024-04-02T13:38:27.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:27 smithi122 ceph-mon[32114]: pgmap v101: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 KiB/s wr, 132 op/s 2024-04-02T13:38:27.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:27 smithi187 ceph-mon[36064]: pgmap v101: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 795 KiB/s wr, 132 op/s 2024-04-02T13:38:27.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 92992 4.80 MB/sec warmup 108 sec latency 2739.327 ms 2024-04-02T13:38:28.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 94567 4.82 MB/sec warmup 109 sec latency 3194.628 ms 2024-04-02T13:38:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:29 smithi188 ceph-mon[36117]: pgmap v102: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 672 KiB/s wr, 101 op/s 2024-04-02T13:38:29.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:29 smithi122 ceph-mon[32114]: pgmap v102: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 672 KiB/s wr, 101 op/s 2024-04-02T13:38:29.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:29 smithi187 ceph-mon[36064]: pgmap v102: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 672 KiB/s wr, 101 op/s 2024-04-02T13:38:29.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 96241 4.88 MB/sec warmup 110 sec latency 14.099 ms 2024-04-02T13:38:30.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 96483 4.84 MB/sec warmup 111 sec latency 780.200 ms 2024-04-02T13:38:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:31 smithi188 ceph-mon[36117]: pgmap v103: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 645 KiB/s wr, 96 op/s 2024-04-02T13:38:31.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:31 smithi122 ceph-mon[32114]: pgmap v103: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 645 KiB/s wr, 96 op/s 2024-04-02T13:38:31.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:31 smithi187 ceph-mon[36064]: pgmap v103: 129 pgs: 129 active+clean; 63 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 645 KiB/s wr, 96 op/s 2024-04-02T13:38:31.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:38:31.946 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:38:31.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 96483 4.80 MB/sec warmup 112 sec latency 1780.315 ms 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/2436206778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf38106e80 msgr2=0x7fcf3810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/2436206778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf38106e80 0x7fcf3810ea00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009900 tx=0x7fcf3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/2436206778 shutdown_connections 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/2436206778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf38106e80 0x7fcf3810ea00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/2436206778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100c20 0x7fcf38106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/2436206778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38100270 0x7fcf38100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/2436206778 >> 172.21.15.122:0/2436206778 conn(0x7fcf38076fc0 msgr2=0x7fcf380fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/2436206778 shutdown_connections 2024-04-02T13:38:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.068+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/2436206778 wait complete. 2024-04-02T13:38:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 Processor -- start 2024-04-02T13:38:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 -- start start 2024-04-02T13:38:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100270 0x7fcf381983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 0x7fcf38198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38111ee0 con 0x7fcf3819cca0 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38111d60 con 0x7fcf38106e80 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.069+0000 7fcf3d7a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38112060 con 0x7fcf38100270 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56286/0 (socket says 172.21.15.122:56286) 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 -- 172.21.15.122:0/904349414 learned_addr learned my addr 172.21.15.122:0/904349414 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100270 0x7fcf381983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf367fc640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 0x7fcf38198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 -- 172.21.15.122:0/904349414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100270 msgr2=0x7fcf381983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100270 0x7fcf381983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 -- 172.21.15.122:0/904349414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 msgr2=0x7fcf38198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 0x7fcf38198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 -- 172.21.15.122:0/904349414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3819d790 con 0x7fcf3819cca0 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf367fc640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 0x7fcf38198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf377fe640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009230 tx=0x7fcf3002cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf1ffff640 1 -- 172.21.15.122:0/904349414 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30041070 con 0x7fcf3819cca0 2024-04-02T13:38:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3819da20 con 0x7fcf3819cca0 2024-04-02T13:38:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf1ffff640 1 -- 172.21.15.122:0/904349414 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf30004450 con 0x7fcf3819cca0 2024-04-02T13:38:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.070+0000 7fcf1ffff640 1 -- 172.21.15.122:0/904349414 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30034440 con 0x7fcf3819cca0 2024-04-02T13:38:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.071+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381a51f0 con 0x7fcf3819cca0 2024-04-02T13:38:32.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.073+0000 7fcf1ffff640 1 -- 172.21.15.122:0/904349414 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcf30044070 con 0x7fcf3819cca0 2024-04-02T13:38:32.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 Processor -- start 2024-04-02T13:38:32.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 -- start start 2024-04-02T13:38:32.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f4b84a830 con 0x560f4b8e6f20 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560f4b7f2820 con 0x560f4b8e6600 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf40382740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560f4b8467e0 con 0x560f4b8f41e0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf1ffff640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 0x7fcf200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56292/0 (socket says 172.21.15.122:56292) 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf36ffd640 1 -- 172.21.15.122:0/2675346665 learned_addr learned my addr 172.21.15.122:0/2675346665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf377fe640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 0x7fcf200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.074+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf1ffff640 1 -- 172.21.15.122:0/904349414 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcf300f2650 con 0x7fcf3819cca0 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 -- 172.21.15.122:0/2675346665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 msgr2=0x560f4b8f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 -- 172.21.15.122:0/2675346665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 msgr2=0x560f4b8e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 -- 172.21.15.122:0/2675346665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4b6287d0 con 0x560f4b8e6f20 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcf280039f0 tx=0x7fcf28003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2587705627a324a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf377fe640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 0x7fcf200a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf300be300 tx=0x7fcf3008fe70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf1d7fa640 1 -- 172.21.15.122:0/2675346665 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28004b20 con 0x560f4b8e6f20 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf1d7fa640 1 -- 172.21.15.122:0/2675346665 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28004cc0 con 0x560f4b8e6f20 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.075+0000 7fcf1d7fa640 1 -- 172.21.15.122:0/2675346665 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28012740 con 0x560f4b8e6f20 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- 172.21.15.122:0/2675346665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 msgr2=0x560f4b8f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcf280039f0 tx=0x7fcf28003eb0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- 172.21.15.122:0/2675346665 shutdown_connections 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f3ca0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 --2- 172.21.15.122:0/2675346665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- 172.21.15.122:0/2675346665 >> 172.21.15.122:0/2675346665 conn(0x560f4b8e1000 msgr2=0x560f4b8e13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- 172.21.15.122:0/2675346665 shutdown_connections 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- 172.21.15.122:0/2675346665 wait complete. 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 -- start start 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 Processor -- start 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.076+0000 7fcf40382740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6600 0x560f4b8f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8f41e0 0x560f4b8f1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560f4b8f6fa0 con 0x560f4b8e6f20 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560f4b805d30 con 0x560f4b8f41e0 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560f4b8f7120 con 0x560f4b8e6600 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 10 client.0 _ll_register_callbacks cb 0x560f4b805720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56308/0 (socket says 172.21.15.122:56308) 2024-04-02T13:38:32.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 -- 172.21.15.122:0/3309595 learned_addr learned my addr 172.21.15.122:0/3309595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf367fc640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8f41e0 0x560f4b8f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf367fc640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6600 0x560f4b8f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 -- 172.21.15.122:0/3309595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6600 msgr2=0x560f4b8f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6600 0x560f4b8f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 -- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8f41e0 msgr2=0x560f4b8f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8f41e0 0x560f4b8f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 -- 172.21.15.122:0/3309595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4b55e3f0 con 0x560f4b8e6f20 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005a90 tx=0x7fcf28005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bb7d06b92e1790 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28005280 con 0x560f4b8e6f20 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcf28005420 con 0x560f4b8e6f20 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.077+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf28004cc0 con 0x560f4b8e6f20 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf28005620 con 0x560f4b8e6f20 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 10 client.16677 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f4b51e0f0 con 0x560f4b8e6f20 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28004cc0 con 0x560f4b8e6f20 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcf2800f040 con 0x560f4b8e6f20 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 10 client.16677 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 20 client.16677 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b904080 0x560f4b906470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 4 client.16677 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.078+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f4b857d70 con 0x560f4b904080 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.079+0000 7fcf367fc640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b904080 0x560f4b906470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.079+0000 7fcf367fc640 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b904080 0x560f4b906470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8260f078cb157d9c server_cookie=f81a065488c6e28d in_seq=0 out_seq=0 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.079+0000 7fcf1aff5640 10 client.16677 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf1aff5640 1 -- 172.21.15.122:0/3309595 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f4b857d70 con 0x560f4b904080 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf1aff5640 10 client.16677 handle_command_reply: tid=0 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 client.16677 shutdown 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 msgr2=0x560f4b8f0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005a90 tx=0x7fcf28005bd0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 shutdown_connections 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b904080 0x560f4b906470 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8f41e0 0x560f4b8f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6f20 0x560f4b8f0df0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 --2- 172.21.15.122:0/3309595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6600 0x560f4b8f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf1aff5640 0 client.16677 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 >> 172.21.15.122:0/3309595 conn(0x560f4b8e1000 msgr2=0x560f4b8e2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 shutdown_connections 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 1 -- 172.21.15.122:0/3309595 wait complete. 2024-04-02T13:38:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.080+0000 7fcf40382740 20 client.16677 trim_cache size 0 max 16384 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.097+0000 7fcf40382740 1 Processor -- start 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 -- start start 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 0x560f4b8eff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381acde0 con 0x560f4b8e6600 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381aea40 con 0x560f4b8e6f20 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf40382740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381ad4b0 con 0x560f4b8f41e0 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56310/0 (socket says 172.21.15.122:56310) 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 -- 172.21.15.122:0/2330233277 learned_addr learned my addr 172.21.15.122:0/2330233277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 0x560f4b8eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 -- 172.21.15.122:0/2330233277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 msgr2=0x560f4b8f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 -- 172.21.15.122:0/2330233277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 msgr2=0x560f4b8eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 0x560f4b8eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 -- 172.21.15.122:0/2330233277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f4b906ad0 con 0x560f4b8e6600 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 0x560f4b8eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.098+0000 7fcf367fc640 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcf38074280 tx=0x7fcf2c00eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4116430d34f8398 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf1cff9640 1 -- 172.21.15.122:0/2330233277 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c028030 con 0x560f4b8e6600 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf1cff9640 1 -- 172.21.15.122:0/2330233277 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c013df0 con 0x560f4b8e6600 2024-04-02T13:38:32.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf1cff9640 1 -- 172.21.15.122:0/2330233277 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c012030 con 0x560f4b8e6600 2024-04-02T13:38:32.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- 172.21.15.122:0/2330233277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 msgr2=0x560f4b8ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcf38074280 tx=0x7fcf2c00eb30 comp rx=0 tx=0).stop 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- 172.21.15.122:0/2330233277 shutdown_connections 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8f41e0 0x560f4b8f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6f20 0x560f4b8eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 --2- 172.21.15.122:0/2330233277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8e6600 0x560f4b8ffed0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- 172.21.15.122:0/2330233277 >> 172.21.15.122:0/2330233277 conn(0x560f4b8e1000 msgr2=0x560f4b8ec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- 172.21.15.122:0/2330233277 shutdown_connections 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- 172.21.15.122:0/2330233277 wait complete. 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.099+0000 7fcf40382740 1 -- start start 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 Processor -- start 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8fc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 0x560f4b8fc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381ae2b0 con 0x560f4b8f41e0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381adbe0 con 0x560f4b8e6600 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf381aadc0 con 0x560f4b8e6f20 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf40382740 10 client.0 _ll_register_callbacks cb 0x7fcf28009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56316/0 (socket says 172.21.15.122:56316) 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 -- 172.21.15.122:0/1697279056 learned_addr learned my addr 172.21.15.122:0/1697279056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 0x560f4b8fc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8fc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 -- 172.21.15.122:0/1697279056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 msgr2=0x560f4b8fc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 0x560f4b8fc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 -- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 msgr2=0x560f4b8fc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8fc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 -- 172.21.15.122:0/1697279056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f4b8edc50 con 0x560f4b8f41e0 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 0x560f4b8fc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf367fc640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c013d10 tx=0x7fcf2c004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e91bd1dc9fd217 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.100+0000 7fcf1aff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004320 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcf2c0223c0 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c013df0 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c0225c0 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 10 client.16683 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560f4b96a050 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c013df0 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fcf2c02e430 con 0x560f4b8f41e0 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 10 client.16683 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 10 client.16683 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 20 client.16683 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b96a710 0x560f4b96cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 4 client.16683 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:38:32.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f4b8520c0 con 0x560f4b96a710 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.101+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b96a710 0x560f4b96cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf1aff5640 10 client.16683 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf36ffd640 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b96a710 0x560f4b96cb30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6625ac374e059deb server_cookie=1021634b3f7666e0 in_seq=0 out_seq=0 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf1aff5640 1 -- 172.21.15.122:0/1697279056 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f4b8520c0 con 0x560f4b96a710 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf1aff5640 10 client.16683 handle_command_reply: tid=0 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 client.16683 shutdown 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 msgr2=0x560f4b8ed450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c013d10 tx=0x7fcf2c004d40 comp rx=0 tx=0).stop 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 shutdown_connections 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560f4b96a710 0x560f4b96cb30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560f4b8f41e0 0x560f4b8ed450 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f4b8e6f20 0x560f4b8fc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf1aff5640 0 client.16683 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 --2- 172.21.15.122:0/1697279056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560f4b8e6600 0x560f4b8fc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 >> 172.21.15.122:0/1697279056 conn(0x560f4b8e1000 msgr2=0x560f4b8df9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 shutdown_connections 2024-04-02T13:38:32.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 1 -- 172.21.15.122:0/1697279056 wait complete. 2024-04-02T13:38:32.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.102+0000 7fcf40382740 20 client.16683 trim_cache size 0 max 16384 2024-04-02T13:38:32.112 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:38:32.112 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:38:32.112 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:38:32.112 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 msgr2=0x7fcf200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 0x7fcf200a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcf300be300 tx=0x7fcf3008fe70 comp rx=0 tx=0).stop 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 msgr2=0x7fcf3819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009230 tx=0x7fcf3002cbf0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 shutdown_connections 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf200a3d20 0x7fcf200a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf3819cca0 0x7fcf3819d150 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf38106e80 0x7fcf38198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 --2- 172.21.15.122:0/904349414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf38100270 0x7fcf381983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 >> 172.21.15.122:0/904349414 conn(0x7fcf38076fc0 msgr2=0x7fcf38104f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 shutdown_connections 2024-04-02T13:38:32.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.103+0000 7fcf3d7a3640 1 -- 172.21.15.122:0/904349414 wait complete. 2024-04-02T13:38:32.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:38:32.118 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 -- 172.21.15.122:0/3036453107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 msgr2=0x7f8c10100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 --2- 172.21.15.122:0/3036453107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10100e80 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009930 tx=0x7f8c0402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 -- 172.21.15.122:0/3036453107 shutdown_connections 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 --2- 172.21.15.122:0/3036453107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 0x7f8c1010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 --2- 172.21.15.122:0/3036453107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10100e80 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 --2- 172.21.15.122:0/3036453107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 0x7f8c10106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.219+0000 7f8c154ba640 1 -- 172.21.15.122:0/3036453107 >> 172.21.15.122:0/3036453107 conn(0x7f8c10077b00 msgr2=0x7f8c10077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- 172.21.15.122:0/3036453107 shutdown_connections 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- 172.21.15.122:0/3036453107 wait complete. 2024-04-02T13:38:32.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 Processor -- start 2024-04-02T13:38:32.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- start start 2024-04-02T13:38:32.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 0x7f8c101997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 0x7f8c101a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10110e20 con 0x7f8c10100a00 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10110ca0 con 0x7f8c10106a10 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c154ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10110fa0 con 0x7f8c101013c0 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.220+0000 7f8c0effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56340/0 (socket says 172.21.15.122:56340) 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 -- 172.21.15.122:0/477650657 learned_addr learned my addr 172.21.15.122:0/477650657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0f7fe640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 0x7f8c101a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0e7fc640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 0x7f8c101997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 -- 172.21.15.122:0/477650657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 msgr2=0x7f8c101997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 0x7f8c101997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 -- 172.21.15.122:0/477650657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 msgr2=0x7f8c101a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 0x7f8c101a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 -- 172.21.15.122:0/477650657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c101a4820 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c0effd640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000ebc0 tx=0x7f8c0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00010630 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c101a49f0 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00010cb0 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00018bd0 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.221+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101a4f10 con 0x7f8c10100a00 2024-04-02T13:38:32.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.222+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100688f0 con 0x7f8c10100a00 2024-04-02T13:38:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.223+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8c000107d0 con 0x7f8c10100a00 2024-04-02T13:38:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.224+0000 7f8bf7fff640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 0x7f8bf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.224+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8c000d1270 con 0x7f8c10100a00 2024-04-02T13:38:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.224+0000 7f8c0e7fc640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 0x7f8bf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.224+0000 7f8c0e7fc640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 0x7f8bf80a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006010 tx=0x7f8c04000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.228+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8c0009d580 con 0x7f8c10100a00 2024-04-02T13:38:32.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.450+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c10069250 con 0x7f8c10100a00 2024-04-02T13:38:32.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.451+0000 7f8bf7fff640 1 -- 172.21.15.122:0/477650657 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7f8c0009ce80 con 0x7f8c10100a00 2024-04-02T13:38:32.454 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:38:32.454 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:37:34.559237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:38:32.455 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 32 2024-04-02T13:38:32.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 msgr2=0x7f8bf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 0x7f8bf80a61e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8c04006010 tx=0x7f8c04000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 msgr2=0x7f8c10199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000ebc0 tx=0x7f8c0000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 shutdown_connections 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8bf80a3d20 0x7f8bf80a61e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8c10106a10 0x7f8c101a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c101013c0 0x7f8c101997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 --2- 172.21.15.122:0/477650657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8c10100a00 0x7f8c10199260 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 >> 172.21.15.122:0/477650657 conn(0x7f8c10077b00 msgr2=0x7f8c101032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 shutdown_connections 2024-04-02T13:38:32.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.453+0000 7f8c154ba640 1 -- 172.21.15.122:0/477650657 wait complete. 2024-04-02T13:38:32.467 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 -- 172.21.15.122:0/206943875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc10c160 msgr2=0x7fb8cc10e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 --2- 172.21.15.122:0/206943875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc10c160 0x7fb8cc10e5b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ca0 tx=0x7fb8c402c180 comp rx=0 tx=0).stop 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 -- 172.21.15.122:0/206943875 shutdown_connections 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 --2- 172.21.15.122:0/206943875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc10c160 0x7fb8cc10e5b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 --2- 172.21.15.122:0/206943875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc10bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 --2- 172.21.15.122:0/206943875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 0x7fb8cc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.567+0000 7fb8d189a640 1 -- 172.21.15.122:0/206943875 >> 172.21.15.122:0/206943875 conn(0x7fb8cc077b60 msgr2=0x7fb8cc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 -- 172.21.15.122:0/206943875 shutdown_connections 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 -- 172.21.15.122:0/206943875 wait complete. 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 Processor -- start 2024-04-02T13:38:32.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 -- start start 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 0x7fb8cc198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.568+0000 7fb8d189a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc10c160 0x7fb8cc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc111f40 con 0x7fb8cc1036b0 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc111dc0 con 0x7fb8cc10c160 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1120c0 con 0x7fb8cc102d90 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56364/0 (socket says 172.21.15.122:56364) 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 -- 172.21.15.122:0/1267140681 learned_addr learned my addr 172.21.15.122:0/1267140681 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc10c160 0x7fb8cc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8caffd640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 0x7fb8cc198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 -- 172.21.15.122:0/1267140681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 msgr2=0x7fb8cc198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 0x7fb8cc198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 -- 172.21.15.122:0/1267140681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc10c160 msgr2=0x7fb8cc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc10c160 0x7fb8cc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 -- 172.21.15.122:0/1267140681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc19d3e0 con 0x7fb8cc1036b0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00d950 tx=0x7fb8bc00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d0898640 1 -- 172.21.15.122:0/1267140681 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc017030 con 0x7fb8cc1036b0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d0898640 1 -- 172.21.15.122:0/1267140681 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8bc004020 con 0x7fb8cc1036b0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d0898640 1 -- 172.21.15.122:0/1267140681 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc005650 con 0x7fb8cc1036b0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8cc19d6d0 con 0x7fb8cc1036b0 2024-04-02T13:38:32.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.569+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8cc1a4f60 con 0x7fb8cc1036b0 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.571+0000 7fb8d0898640 1 -- 172.21.15.122:0/1267140681 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb8bc0041d0 con 0x7fb8cc1036b0 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 Processor -- start 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 -- start start 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b4c64f7b0 con 0x562b4c6eb7a0 2024-04-02T13:38:32.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562b4c5ee570 con 0x562b4c6f4ca0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d4564740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b4c64b760 con 0x562b4c6ec0c0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d0898640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 0x7fb8b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56372/0 (socket says 172.21.15.122:56372) 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/1091490562 learned_addr learned my addr 172.21.15.122:0/1091490562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 0x7fb8b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8d0898640 1 -- 172.21.15.122:0/1267140681 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb8bc0d0470 con 0x7fb8cc1036b0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8caffd640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/1091490562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 msgr2=0x562b4c6f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/1091490562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 msgr2=0x562b4c6f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/1091490562 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4c3633f0 con 0x562b4c6eb7a0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.572+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4037040 tx=0x7fb8c4003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fff1467e762e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8b1ffb640 1 -- 172.21.15.122:0/1091490562 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4050070 con 0x562b4c6eb7a0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 0x7fb8b40a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb8c404d000 tx=0x7fb8c4048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8b1ffb640 1 -- 172.21.15.122:0/1091490562 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4004790 con 0x562b4c6eb7a0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8b1ffb640 1 -- 172.21.15.122:0/1091490562 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4039040 con 0x562b4c6eb7a0 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8d4564740 1 -- 172.21.15.122:0/1091490562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 msgr2=0x562b4c6ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.573+0000 7fb8d4564740 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4037040 tx=0x7fb8c4003c90 comp rx=0 tx=0).stop 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- 172.21.15.122:0/1091490562 shutdown_connections 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- 172.21.15.122:0/1091490562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6eb7a0 0x562b4c6ebb80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- 172.21.15.122:0/1091490562 >> 172.21.15.122:0/1091490562 conn(0x562b4c6e5ed0 msgr2=0x562b4c6e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- 172.21.15.122:0/1091490562 shutdown_connections 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- 172.21.15.122:0/1091490562 wait complete. 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- start start 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 Processor -- start 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6eb7a0 0x562b4c6f1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562b4c6f7a90 con 0x562b4c6f4ca0 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562b4c60ae80 con 0x562b4c6eb7a0 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b4c6f7c10 con 0x562b4c6ec0c0 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8d4564740 10 client.0 _ll_register_callbacks cb 0x562b4c646aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56386/0 (socket says 172.21.15.122:56386) 2024-04-02T13:38:32.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.574+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/417413660 learned_addr learned my addr 172.21.15.122:0/417413660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8caffd640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6eb7a0 0x562b4c6f1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/417413660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 msgr2=0x562b4c6f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/417413660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6eb7a0 msgr2=0x562b4c6f1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6eb7a0 0x562b4c6f1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/417413660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4c42d7d0 con 0x562b4c6f4ca0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402c6c0 tx=0x7fb8c40344e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c672e0c6296c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4039070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb8c4034720 con 0x562b4c6f4ca0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c4004790 con 0x562b4c6f4ca0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8c4062050 con 0x562b4c6f4ca0 2024-04-02T13:38:32.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8d4564740 10 client.16701 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b4c6f7a90 con 0x562b4c6f4ca0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.575+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4004790 con 0x562b4c6f4ca0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb8c403f070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 10 client.16701 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 10 client.16701 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 20 client.16701 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c704d00 0x562b4c7070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 4 client.16701 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b4c65ccf0 con 0x562b4c704d00 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8caffd640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c704d00 0x562b4c7070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8aaffd640 10 client.16701 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.576+0000 7fb8caffd640 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c704d00 0x562b4c7070f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19c8940f17bed56f server_cookie=1d38f7849920fb13 in_seq=0 out_seq=0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8aaffd640 1 -- 172.21.15.122:0/417413660 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b4c65ccf0 con 0x562b4c704d00 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8aaffd640 10 client.16701 handle_command_reply: tid=0 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 client.16701 shutdown 2024-04-02T13:38:32.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 msgr2=0x562b4c6f25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402c6c0 tx=0x7fb8c40344e0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 shutdown_connections 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c704d00 0x562b4c7070f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f25a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8aaffd640 0 client.16701 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 --2- 172.21.15.122:0/417413660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6eb7a0 0x562b4c6f1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 >> 172.21.15.122:0/417413660 conn(0x562b4c6e5ed0 msgr2=0x562b4c6e7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 shutdown_connections 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 1 -- 172.21.15.122:0/417413660 wait complete. 2024-04-02T13:38:32.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.577+0000 7fb8d4564740 20 client.16701 trim_cache size 0 max 16384 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 Processor -- start 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 -- start start 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c700950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1aef70 con 0x562b4c6f4ca0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b0bd0 con 0x562b4c6ec0c0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8d4564740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1af640 con 0x562b4c6eb7a0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.590+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56388/0 (socket says 172.21.15.122:56388) 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/2737787061 learned_addr learned my addr 172.21.15.122:0/2737787061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8caffd640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c700950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/2737787061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 msgr2=0x562b4c700950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c700950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/2737787061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 msgr2=0x562b4c6f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/2737787061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b4c6ebef0 con 0x562b4c6f4ca0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4004af0 tx=0x7fb8c4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e77ee368d931cef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8b17fa640 1 -- 172.21.15.122:0/2737787061 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4039070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8b17fa640 1 -- 172.21.15.122:0/2737787061 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4059070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.591+0000 7fb8b17fa640 1 -- 172.21.15.122:0/2737787061 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4050040 con 0x562b4c6f4ca0 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- 172.21.15.122:0/2737787061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 msgr2=0x562b4c70c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4004af0 tx=0x7fb8c4005f90 comp rx=0 tx=0).stop 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- 172.21.15.122:0/2737787061 shutdown_connections 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c70c8b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- 172.21.15.122:0/2737787061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c700950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- 172.21.15.122:0/2737787061 >> 172.21.15.122:0/2737787061 conn(0x562b4c6e5ed0 msgr2=0x562b4c6e4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- 172.21.15.122:0/2737787061 shutdown_connections 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- 172.21.15.122:0/2737787061 wait complete. 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- start start 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 Processor -- start 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c6fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1b0440 con 0x562b4c6f4ca0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1afd70 con 0x562b4c6ec0c0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.592+0000 7fb8d4564740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb8cc1acf50 con 0x562b4c6eb7a0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8d4564740 10 client.0 _ll_register_callbacks cb 0x7fb8c404e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56402/0 (socket says 172.21.15.122:56402) 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/560566725 learned_addr learned my addr 172.21.15.122:0/560566725 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8ca7fc640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8caffd640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c6fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/560566725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 msgr2=0x562b4c6fd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c6fd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/560566725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 msgr2=0x562b4c6f8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 -- 172.21.15.122:0/560566725 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b4c6f9420 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8cb7fe640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009600 tx=0x7fb8c4032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d5e891de07d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c406b060 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb8c40047a0 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8c4059070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb8c4050030 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8d4564740 10 client.16707 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.593+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b4c76b160 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8c4059070 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb8c4054ce0 con 0x562b4c6f4ca0 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 10 client.16707 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 10 client.16707 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 20 client.16707 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c76b6c0 0x562b4c76dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 4 client.16707 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b4c657040 con 0x562b4c76b6c0 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8caffd640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c76b6c0 0x562b4c76dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8aaffd640 10 client.16707 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8caffd640 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c76b6c0 0x562b4c76dae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21b973172580af8a server_cookie=e21aaaae81105c01 in_seq=0 out_seq=0 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8aaffd640 1 -- 172.21.15.122:0/560566725 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b4c657040 con 0x562b4c76b6c0 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8aaffd640 10 client.16707 handle_command_reply: tid=0 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 1 client.16707 shutdown 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 msgr2=0x562b4c6f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.594+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009600 tx=0x7fb8c4032bb0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 shutdown_connections 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x562b4c76b6c0 0x562b4c76dae0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562b4c6f4ca0 0x562b4c6f8bf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562b4c6ec0c0 0x562b4c6f8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8aaffd640 0 client.16707 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:38:32.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 --2- 172.21.15.122:0/560566725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b4c6eb7a0 0x562b4c6fd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 >> 172.21.15.122:0/560566725 conn(0x562b4c6e5ed0 msgr2=0x562b4c6f0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 shutdown_connections 2024-04-02T13:38:32.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 1 -- 172.21.15.122:0/560566725 wait complete. 2024-04-02T13:38:32.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d4564740 20 client.16707 trim_cache size 0 max 16384 2024-04-02T13:38:32.605 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 msgr2=0x7fb8b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 0x7fb8b40a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb8c404d000 tx=0x7fb8c4048000 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 msgr2=0x7fb8cc198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00d950 tx=0x7fb8bc00dd50 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 shutdown_connections 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb8b40a3d20 0x7fb8b40a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8cc10c160 0x7fb8cc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8cc1036b0 0x7fb8cc198980 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 --2- 172.21.15.122:0/1267140681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8cc102d90 0x7fb8cc198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 >> 172.21.15.122:0/1267140681 conn(0x7fb8cc077b60 msgr2=0x7fb8cc0fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 shutdown_connections 2024-04-02T13:38:32.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.595+0000 7fb8d189a640 1 -- 172.21.15.122:0/1267140681 wait complete. 2024-04-02T13:38:32.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:38:32.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- 172.21.15.122:0/373248983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 msgr2=0x7fa788101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 --2- 172.21.15.122:0/373248983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788101f20 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009930 tx=0x7fa77c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:32.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- 172.21.15.122:0/373248983 shutdown_connections 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 --2- 172.21.15.122:0/373248983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 0x7fa78810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 --2- 172.21.15.122:0/373248983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788101f20 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 --2- 172.21.15.122:0/373248983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- 172.21.15.122:0/373248983 >> 172.21.15.122:0/373248983 conn(0x7fa7880fd530 msgr2=0x7fa7880ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- 172.21.15.122:0/373248983 shutdown_connections 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- 172.21.15.122:0/373248983 wait complete. 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 Processor -- start 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.711+0000 7fa78e99d640 1 -- start start 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 0x7fa788074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa788111ec0 con 0x7fa788101aa0 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa788111d40 con 0x7fa788102460 2024-04-02T13:38:32.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa788112040 con 0x7fa788107ab0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56428/0 (socket says 172.21.15.122:56428) 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 -- 172.21.15.122:0/2750189693 learned_addr learned my addr 172.21.15.122:0/2750189693 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 0x7fa788074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 -- 172.21.15.122:0/2750189693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 msgr2=0x7fa788074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 -- 172.21.15.122:0/2750189693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 msgr2=0x7fa788074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 0x7fa788074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 -- 172.21.15.122:0/2750189693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7881a69b0 con 0x7fa788101aa0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa787fff640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa77800b4a0 tx=0x7fa77800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa7857fa640 1 -- 172.21.15.122:0/2750189693 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778014070 con 0x7fa788101aa0 2024-04-02T13:38:32.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa7857fa640 1 -- 172.21.15.122:0/2750189693 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778010af0 con 0x7fa788101aa0 2024-04-02T13:38:32.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7881a6ca0 con 0x7fa788101aa0 2024-04-02T13:38:32.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa7857fa640 1 -- 172.21.15.122:0/2750189693 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778017d10 con 0x7fa788101aa0 2024-04-02T13:38:32.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.712+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7881a71c0 con 0x7fa788101aa0 2024-04-02T13:38:32.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa7857fa640 1 -- 172.21.15.122:0/2750189693 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa778017070 con 0x7fa788101aa0 2024-04-02T13:38:32.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 Processor -- start 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 -- start start 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88707a0 0x561cc8870b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc887c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561cc87d47b0 con 0x561cc88710c0 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561cc8773570 con 0x561cc8879ca0 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa791585740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561cc87d0760 con 0x561cc88707a0 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56440/0 (socket says 172.21.15.122:56440) 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa787fff640 1 -- 172.21.15.122:0/2644152489 learned_addr learned my addr 172.21.15.122:0/2644152489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa7857fa640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 0x7fa7700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 0x7fa7700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.715+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc887c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88707a0 0x561cc8870b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa7857fa640 1 -- 172.21.15.122:0/2750189693 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa7780cfe00 con 0x7fa788101aa0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 -- 172.21.15.122:0/2644152489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88707a0 msgr2=0x561cc8870b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88707a0 0x561cc8870b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 -- 172.21.15.122:0/2644152489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 msgr2=0x561cc887c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc887c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 -- 172.21.15.122:0/2644152489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc84e83f0 con 0x561cc88710c0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa787fff640 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cd270 tx=0x7fa7780047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b652234abdf7b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 0x7fa7700a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa780009fd0 tx=0x7fa780009530 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa76e7fc640 1 -- 172.21.15.122:0/2644152489 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77801e030 con 0x561cc88710c0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa76e7fc640 1 -- 172.21.15.122:0/2644152489 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77809c220 con 0x561cc88710c0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.716+0000 7fa76e7fc640 1 -- 172.21.15.122:0/2644152489 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778010ca0 con 0x561cc88710c0 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- 172.21.15.122:0/2644152489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 msgr2=0x561cc8879760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cd270 tx=0x7fa7780047d0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- 172.21.15.122:0/2644152489 shutdown_connections 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc887c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879760 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 --2- 172.21.15.122:0/2644152489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88707a0 0x561cc8870b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- 172.21.15.122:0/2644152489 >> 172.21.15.122:0/2644152489 conn(0x561cc886aed0 msgr2=0x561cc886b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- 172.21.15.122:0/2644152489 shutdown_connections 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- 172.21.15.122:0/2644152489 wait complete. 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 -- start start 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 Processor -- start 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.717+0000 7fa791585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc8876ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc88771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561cc887c9d0 con 0x561cc88707a0 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561cc878fe80 con 0x561cc8879ca0 2024-04-02T13:38:32.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561cc887cb50 con 0x561cc88710c0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 10 client.0 _ll_register_callbacks cb 0x561cc87cbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56452/0 (socket says 172.21.15.122:56452) 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 -- 172.21.15.122:0/2302071361 learned_addr learned my addr 172.21.15.122:0/2302071361 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc88771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc8876ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 -- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 msgr2=0x561cc8876ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc8876ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 -- 172.21.15.122:0/2302071361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 msgr2=0x561cc88771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc88771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 -- 172.21.15.122:0/2302071361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc85b27d0 con 0x561cc88707a0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc8876ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa787fff640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa778097f80 tx=0x7fa77809bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b2cb46d7b78544 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7677fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77801e070 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa778010ca0 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77809c220 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa7780cd380 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 10 client.16719 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.718+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cc8888a60 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77809c220 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa7780b7080 con 0x561cc88707a0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 10 client.16719 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 10 client.16719 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 20 client.16719 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc8889db0 0x561cc888c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 4 client.16719 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cc87e1cf0 con 0x561cc8889db0 2024-04-02T13:38:32.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc8889db0 0x561cc888c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa7877fe640 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc8889db0 0x561cc888c1a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=39ef7a9f4af9cf55 server_cookie=a6f8c58d91033bdd in_seq=0 out_seq=0 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.719+0000 7fa7677fe640 10 client.16719 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa7677fe640 1 -- 172.21.15.122:0/2302071361 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cc87e1cf0 con 0x561cc8889db0 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa7677fe640 10 client.16719 handle_command_reply: tid=0 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 client.16719 shutdown 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 msgr2=0x561cc8879740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fa778097f80 tx=0x7fa77809bc50 comp rx=0 tx=0).stop 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 shutdown_connections 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc88771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc8876ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc8889db0 0x561cc888c1a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 --2- 172.21.15.122:0/2302071361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc8879740 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa7677fe640 0 client.16719 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:32.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 >> 172.21.15.122:0/2302071361 conn(0x561cc886aed0 msgr2=0x561cc886cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 shutdown_connections 2024-04-02T13:38:32.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 1 -- 172.21.15.122:0/2302071361 wait complete. 2024-04-02T13:38:32.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.720+0000 7fa791585740 20 client.16719 trim_cache size 0 max 16384 2024-04-02T13:38:32.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.733+0000 7fa791585740 1 Processor -- start 2024-04-02T13:38:32.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.733+0000 7fa791585740 1 -- start start 2024-04-02T13:38:32.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc88707a0 0x561cc8879200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc8879ca0 0x561cc888f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881b1290 con 0x561cc88710c0 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881b2ef0 con 0x561cc88707a0 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa791585740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881b1960 con 0x561cc8879ca0 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56464/0 (socket says 172.21.15.122:56464) 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 -- 172.21.15.122:0/236832692 learned_addr learned my addr 172.21.15.122:0/236832692 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc8879ca0 0x561cc888f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 -- 172.21.15.122:0/236832692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc8879ca0 msgr2=0x561cc888f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc8879ca0 0x561cc888f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 -- 172.21.15.122:0/236832692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc88707a0 msgr2=0x561cc8879200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc88707a0 0x561cc8879200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 -- 172.21.15.122:0/236832692 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cc8877530 con 0x561cc88710c0 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa7877fe640 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa77c005ab0 tx=0x7fa77c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf6f77416059a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa76dffb640 1 -- 172.21.15.122:0/236832692 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c02cd20 con 0x561cc88710c0 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa76dffb640 1 -- 172.21.15.122:0/236832692 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c04b070 con 0x561cc88710c0 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.734+0000 7fa76dffb640 1 -- 172.21.15.122:0/236832692 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c0046c0 con 0x561cc88710c0 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- 172.21.15.122:0/236832692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 msgr2=0x561cc8879740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fa77c005ab0 tx=0x7fa77c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- 172.21.15.122:0/236832692 shutdown_connections 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc8879ca0 0x561cc888f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88710c0 0x561cc8879740 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 --2- 172.21.15.122:0/236832692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc88707a0 0x561cc8879200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- 172.21.15.122:0/236832692 >> 172.21.15.122:0/236832692 conn(0x561cc886aed0 msgr2=0x561cc8871de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- 172.21.15.122:0/236832692 shutdown_connections 2024-04-02T13:38:32.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- 172.21.15.122:0/236832692 wait complete. 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 -- start start 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.735+0000 7fa791585740 1 Processor -- start 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc888c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc888cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881b2760 con 0x561cc88707a0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881b2090 con 0x561cc8879ca0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881af270 con 0x561cc88710c0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 10 client.0 _ll_register_callbacks cb 0x7fa778017630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56470/0 (socket says 172.21.15.122:56470) 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 -- 172.21.15.122:0/1695162923 learned_addr learned my addr 172.21.15.122:0/1695162923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa7877fe640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc888c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 -- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 msgr2=0x561cc888c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc888c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa78cf3a640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc888cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 -- 172.21.15.122:0/1695162923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 msgr2=0x561cc888cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc888cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 -- 172.21.15.122:0/1695162923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cc888d3a0 con 0x561cc88707a0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa7877fe640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc888c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa787fff640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cd7c0 tx=0x7fa77809bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d034e3cd420b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa7677fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77801e070 con 0x561cc88707a0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa77809aff0 con 0x561cc88707a0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.736+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa77c04b070 con 0x561cc88707a0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa77809b1f0 con 0x561cc88707a0 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 10 client.16725 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561cc88efe00 con 0x561cc88707a0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa77c04b070 con 0x561cc88707a0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa7780bace0 con 0x561cc88707a0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 10 client.16725 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 10 client.16725 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 20 client.16725 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc88f0300 0x561cc88f2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 4 client.16725 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cc87dc040 con 0x561cc88f0300 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.737+0000 7fa7877fe640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc88f0300 0x561cc88f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa7677fe640 10 client.16725 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa7877fe640 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc88f0300 0x561cc88f2720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9e507e5af426a7b server_cookie=b6c9e7a1a7e7a384 in_seq=0 out_seq=0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa7677fe640 1 -- 172.21.15.122:0/1695162923 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cc87dc040 con 0x561cc88f0300 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa7677fe640 10 client.16725 handle_command_reply: tid=0 2024-04-02T13:38:32.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 client.16725 shutdown 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 msgr2=0x561cc888f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cd7c0 tx=0x7fa77809bbb0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 shutdown_connections 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561cc8879ca0 0x561cc888cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561cc88710c0 0x561cc888c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x561cc88f0300 0x561cc88f2720 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 --2- 172.21.15.122:0/1695162923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561cc88707a0 0x561cc888f3f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa7677fe640 0 client.16725 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 >> 172.21.15.122:0/1695162923 conn(0x561cc886aed0 msgr2=0x561cc88696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 shutdown_connections 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.738+0000 7fa791585740 1 -- 172.21.15.122:0/1695162923 wait complete. 2024-04-02T13:38:32.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa791585740 20 client.16725 trim_cache size 0 max 16384 2024-04-02T13:38:32.749 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:32.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 msgr2=0x7fa7700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 0x7fa7700a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa780009fd0 tx=0x7fa780009530 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 msgr2=0x7fa788073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa77800b4a0 tx=0x7fa77800b960 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 shutdown_connections 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7700a3d20 0x7fa7700a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa788107ab0 0x7fa788074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa788102460 0x7fa788074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 --2- 172.21.15.122:0/2750189693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa788101aa0 0x7fa788073e00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 >> 172.21.15.122:0/2750189693 conn(0x7fa7880fd530 msgr2=0x7fa7880fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 shutdown_connections 2024-04-02T13:38:32.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.739+0000 7fa78e99d640 1 -- 172.21.15.122:0/2750189693 wait complete. 2024-04-02T13:38:32.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:38:32.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 -- 172.21.15.122:0/3708337569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 msgr2=0x7fea2c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 --2- 172.21.15.122:0/3708337569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c10d840 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009cd0 tx=0x7fea1c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 -- 172.21.15.122:0/3708337569 shutdown_connections 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 --2- 172.21.15.122:0/3708337569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c10d840 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 --2- 172.21.15.122:0/3708337569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 0x7fea2c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 --2- 172.21.15.122:0/3708337569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 0x7fea2c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 -- 172.21.15.122:0/3708337569 >> 172.21.15.122:0/3708337569 conn(0x7fea2c077b00 msgr2=0x7fea2c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 -- 172.21.15.122:0/3708337569 shutdown_connections 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.853+0000 7fea315f6640 1 -- 172.21.15.122:0/3708337569 wait complete. 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 Processor -- start 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 -- start start 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 0x7fea2c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 0x7fea2c1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c110d90 con 0x7fea2c101330 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c110c10 con 0x7fea2c106980 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea315f6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c110f10 con 0x7fea2c100970 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56476/0 (socket says 172.21.15.122:56476) 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3228985748 learned_addr learned my addr 172.21.15.122:0/3228985748 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 0x7fea2c1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.854+0000 7fea2affd640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 0x7fea2c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3228985748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 msgr2=0x7fea2c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 0x7fea2c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3228985748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 msgr2=0x7fea2c1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 0x7fea2c1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3228985748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c1a4750 con 0x7fea2c101330 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d770 tx=0x7fea2000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea13fff640 1 -- 172.21.15.122:0/3228985748 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20017070 con 0x7fea2c101330 2024-04-02T13:38:32.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea13fff640 1 -- 172.21.15.122:0/3228985748 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea20005500 con 0x7fea2c101330 2024-04-02T13:38:32.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1a4920 con 0x7fea2c101330 2024-04-02T13:38:32.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea13fff640 1 -- 172.21.15.122:0/3228985748 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20004e00 con 0x7fea2c101330 2024-04-02T13:38:32.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.855+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a4df0 con 0x7fea2c101330 2024-04-02T13:38:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 Processor -- start 2024-04-02T13:38:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea13fff640 1 -- 172.21.15.122:0/3228985748 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fea20004fa0 con 0x7fea2c101330 2024-04-02T13:38:32.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 -- start start 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054267a0 0x55a805426b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054270c0 0x55a80542f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a80538a7b0 con 0x55a80542fca0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a805329570 con 0x55a8054267a0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea341da740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a805386760 con 0x55a8054270c0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56480/0 (socket says 172.21.15.122:56480) 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3262440276 learned_addr learned my addr 172.21.15.122:0/3262440276 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054267a0 0x55a805426b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea13fff640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2affd640 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054270c0 0x55a80542f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3262440276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054270c0 msgr2=0x55a80542f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054270c0 0x55a80542f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3262440276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054267a0 msgr2=0x55a805426b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.858+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054267a0 0x55a805426b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea2a7fc640 1 -- 172.21.15.122:0/3262440276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a80509e3f0 con 0x55a80542fca0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea13fff640 1 -- 172.21.15.122:0/3228985748 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fea200d0590 con 0x7fea2c101330 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fea200cd830 tx=0x7fea200668a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40fd4e9868ac411 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea117fa640 1 -- 172.21.15.122:0/3262440276 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20020070 con 0x55a80542fca0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea117fa640 1 -- 172.21.15.122:0/3262440276 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea200046c0 con 0x55a80542fca0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.859+0000 7fea117fa640 1 -- 172.21.15.122:0/3262440276 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2009c1f0 con 0x55a80542fca0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- 172.21.15.122:0/3262440276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 msgr2=0x55a805432090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 0x7fea140a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005cc0 tx=0x7fea1c02e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fea200cd830 tx=0x7fea200668a0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- 172.21.15.122:0/3262440276 shutdown_connections 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805432090 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054270c0 0x55a80542f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- 172.21.15.122:0/3262440276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054267a0 0x55a805426b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- 172.21.15.122:0/3262440276 >> 172.21.15.122:0/3262440276 conn(0x55a805420ed0 msgr2=0x55a8054212b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- 172.21.15.122:0/3262440276 shutdown_connections 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- 172.21.15.122:0/3262440276 wait complete. 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- start start 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 Processor -- start 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a80542f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a80542ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a8054329d0 con 0x55a80542fca0 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a805345e80 con 0x55a8054270c0 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a805432b50 con 0x55a8054267a0 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea341da740 10 client.0 _ll_register_callbacks cb 0x55a805381aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56492/0 (socket says 172.21.15.122:56492) 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.860+0000 7fea2b7fe640 1 -- 172.21.15.122:0/1861034762 learned_addr learned my addr 172.21.15.122:0/1861034762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2affd640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a80542f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a80542ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 -- 172.21.15.122:0/1861034762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 msgr2=0x55a80542f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a80542f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 -- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 msgr2=0x55a80542ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a80542ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 -- 172.21.15.122:0/1861034762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8051687d0 con 0x55a80542fca0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fea1c032530 tx=0x7fea1c005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc6687398ce4f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c039040 con 0x55a80542fca0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fea1c040460 con 0x55a80542fca0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a805397cf0 con 0x55a80542fca0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea1c05a050 con 0x55a80542fca0 2024-04-02T13:38:32.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea341da740 10 client.16737 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a80543ea50 con 0x55a80542fca0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.861+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a805397cf0 con 0x55a80542fca0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fea1c034780 con 0x55a80542fca0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 10 client.16737 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 10 client.16737 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 20 client.16737 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a80543ff60 0x55a8054423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 4 client.16737 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a805392040 con 0x55a80543ff60 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea2affd640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a80543ff60 0x55a8054423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea0eff5640 10 client.16737 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.862+0000 7fea2affd640 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a80543ff60 0x55a8054423a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=321c690405f41ddb server_cookie=b72ffd1a82668241 in_seq=0 out_seq=0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea0eff5640 1 -- 172.21.15.122:0/1861034762 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a805392040 con 0x55a80543ff60 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea0eff5640 10 client.16737 handle_command_reply: tid=0 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 client.16737 shutdown 2024-04-02T13:38:32.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 msgr2=0x55a80542d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fea1c032530 tx=0x7fea1c005c50 comp rx=0 tx=0).stop 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 shutdown_connections 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a80542d1b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a80543ff60 0x55a8054423a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a80542ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 --2- 172.21.15.122:0/1861034762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a80542f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea0eff5640 0 client.16737 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 >> 172.21.15.122:0/1861034762 conn(0x55a805420ed0 msgr2=0x55a805422c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 shutdown_connections 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.863+0000 7fea341da740 1 -- 172.21.15.122:0/1861034762 wait complete. 2024-04-02T13:38:32.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.864+0000 7fea341da740 20 client.16737 trim_cache size 0 max 16384 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.876+0000 7fea341da740 1 Processor -- start 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.876+0000 7fea341da740 1 -- start start 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805442380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a80542fca0 0x55a805447c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aeee0 con 0x55a8054270c0 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b0b40 con 0x55a80542fca0 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea341da740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1af5b0 con 0x55a8054267a0 2024-04-02T13:38:32.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56508/0 (socket says 172.21.15.122:56508) 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 -- 172.21.15.122:0/861385246 learned_addr learned my addr 172.21.15.122:0/861385246 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 -- 172.21.15.122:0/861385246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 msgr2=0x55a805442380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805442380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 -- 172.21.15.122:0/861385246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a80542fca0 msgr2=0x55a805447c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a80542fca0 0x55a805447c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 -- 172.21.15.122:0/861385246 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a805432660 con 0x55a8054270c0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fea2006d080 tx=0x7fea200048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3b8dd618cd8a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea10ff9640 1 -- 172.21.15.122:0/861385246 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20020040 con 0x55a8054270c0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea10ff9640 1 -- 172.21.15.122:0/861385246 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea200a04f0 con 0x55a8054270c0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.877+0000 7fea10ff9640 1 -- 172.21.15.122:0/861385246 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2006f1e0 con 0x55a8054270c0 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- 172.21.15.122:0/861385246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 msgr2=0x55a80543b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fea2006d080 tx=0x7fea200048a0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- 172.21.15.122:0/861385246 shutdown_connections 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a80542fca0 0x55a805447c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a8054270c0 0x55a80543b890 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- 172.21.15.122:0/861385246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805442380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- 172.21.15.122:0/861385246 >> 172.21.15.122:0/861385246 conn(0x55a805420ed0 msgr2=0x55a805427d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- 172.21.15.122:0/861385246 shutdown_connections 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- 172.21.15.122:0/861385246 wait complete. 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- start start 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 Processor -- start 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805430680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a805430bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b03b0 con 0x55a80542fca0 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1afce0 con 0x55a8054270c0 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1acec0 con 0x55a8054267a0 2024-04-02T13:38:32.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.878+0000 7fea341da740 10 client.0 _ll_register_callbacks cb 0x7fea1c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56514/0 (socket says 172.21.15.122:56514) 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 -- 172.21.15.122:0/2203789409 learned_addr learned my addr 172.21.15.122:0/2203789409 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2a7fc640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a805430bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2affd640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805430680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 -- 172.21.15.122:0/2203789409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 msgr2=0x55a805430680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805430680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 -- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 msgr2=0x55a805430bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a805430bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 -- 172.21.15.122:0/2203789409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8054372e0 con 0x55a80542fca0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2affd640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805430680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea2b7fe640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fea1c032bc0 tx=0x7fea1c02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc01e569d0f45cae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea0eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c062070 con 0x55a80542fca0 2024-04-02T13:38:32.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fea1c034640 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea200a04f0 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea1c06a050 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea341da740 10 client.16743 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a8054385e0 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.879+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea200a04f0 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fea1c065030 con 0x55a80542fca0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 10 client.16743 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 10 client.16743 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 20 client.16743 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a8054a6920 0x55a8054a8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 4 client.16743 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a80530f6b0 con 0x55a8054a6920 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea2affd640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a8054a6920 0x55a8054a8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea0eff5640 10 client.16743 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea2affd640 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a8054a6920 0x55a8054a8d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a27760c20d08f89 server_cookie=f0f668a5becafb4a in_seq=0 out_seq=0 2024-04-02T13:38:32.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea0eff5640 1 -- 172.21.15.122:0/2203789409 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a80530f6b0 con 0x55a8054a6920 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea0eff5640 10 client.16743 handle_command_reply: tid=0 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.880+0000 7fea341da740 1 client.16743 shutdown 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 msgr2=0x55a805436da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fea1c032bc0 tx=0x7fea1c02c770 comp rx=0 tx=0).stop 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 shutdown_connections 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a80542fca0 0x55a805436da0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a8054a6920 0x55a8054a8d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a8054270c0 0x55a805430bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 --2- 172.21.15.122:0/2203789409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a8054267a0 0x55a805430680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea0eff5640 0 client.16743 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 >> 172.21.15.122:0/2203789409 conn(0x55a805420ed0 msgr2=0x55a80541f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 shutdown_connections 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 1 -- 172.21.15.122:0/2203789409 wait complete. 2024-04-02T13:38:32.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea341da740 20 client.16743 trim_cache size 0 max 16384 2024-04-02T13:38:32.891 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:32.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 msgr2=0x7fea140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 0x7fea140a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fea1c005cc0 tx=0x7fea1c02e040 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 msgr2=0x7fea2c1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fea2000d770 tx=0x7fea2000dc30 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 shutdown_connections 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fea140a3d20 0x7fea140a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fea2c106980 0x7fea2c1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fea2c101330 0x7fea2c1996d0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 --2- 172.21.15.122:0/3228985748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea2c100970 0x7fea2c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 >> 172.21.15.122:0/3228985748 conn(0x7fea2c077b00 msgr2=0x7fea2c1030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 shutdown_connections 2024-04-02T13:38:32.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.881+0000 7fea315f6640 1 -- 172.21.15.122:0/3228985748 wait complete. 2024-04-02T13:38:32.895 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:38:32.945 INFO:tasks.workunit.client.0.smithi122.stdout: 1 96483 4.76 MB/sec warmup 113 sec latency 2780.384 ms 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.995+0000 7fa72268f640 1 -- 172.21.15.122:0/2936128370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 msgr2=0x7fa71c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.995+0000 7fa72268f640 1 --2- 172.21.15.122:0/2936128370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c106d70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009930 tx=0x7fa70c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- 172.21.15.122:0/2936128370 shutdown_connections 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- 172.21.15.122:0/2936128370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 0x7fa71c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- 172.21.15.122:0/2936128370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 0x7fa71c100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- 172.21.15.122:0/2936128370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c106d70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- 172.21.15.122:0/2936128370 >> 172.21.15.122:0/2936128370 conn(0x7fa71c076d80 msgr2=0x7fa71c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- 172.21.15.122:0/2936128370 shutdown_connections 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- 172.21.15.122:0/2936128370 wait complete. 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 Processor -- start 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- start start 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 0x7fa71c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 0x7fa71c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c110da0 con 0x7fa71c106990 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c110c20 con 0x7fa71c101340 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.996+0000 7fa72268f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c110f20 con 0x7fa71c100980 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56538/0 (socket says 172.21.15.122:56538) 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 -- 172.21.15.122:0/236776250 learned_addr learned my addr 172.21.15.122:0/236776250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 0x7fa71c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa71bfff640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 0x7fa71c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 -- 172.21.15.122:0/236776250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 msgr2=0x7fa71c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 0x7fa71c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 -- 172.21.15.122:0/236776250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 msgr2=0x7fa71c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 0x7fa71c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 -- 172.21.15.122:0/236776250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c1a6920 con 0x7fa71c106990 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa720c2c640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa71400cad0 tx=0x7fa71400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa7197fa640 1 -- 172.21.15.122:0/236776250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71400ad60 con 0x7fa71c106990 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c1a6c10 con 0x7fa71c106990 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa7197fa640 1 -- 172.21.15.122:0/236776250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa714005020 con 0x7fa71c106990 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa7197fa640 1 -- 172.21.15.122:0/236776250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71400f6b0 con 0x7fa71c106990 2024-04-02T13:38:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:32.997+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c1a7160 con 0x7fa71c106990 2024-04-02T13:38:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa7197fa640 1 -- 172.21.15.122:0/236776250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa714005560 con 0x7fa71c106990 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 Processor -- start 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 -- start start 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3ad7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5650e3a3b7b0 con 0x5650e3ae0ca0 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5650e39da570 con 0x5650e3ad77a0 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa725277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650e3a37760 con 0x5650e3ad80c0 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa7197fa640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 0x7fa7040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56552/0 (socket says 172.21.15.122:56552) 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1036405430 learned_addr learned my addr 172.21.15.122:0/1036405430 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71bfff640 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3ad7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa720c2c640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 0x7fa7040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa7197fa640 1 -- 172.21.15.122:0/236776250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa7140d0b80 con 0x7fa71c106990 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1036405430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 msgr2=0x5650e3ae0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.000+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1036405430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 msgr2=0x5650e3ad7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3ad7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1036405430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e374f3f0 con 0x5650e3ae0ca0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa7100101f0 tx=0x7fa7100095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb1d22589f0b0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa720c2c640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 0x7fa7040a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7140cdff0 tx=0x7fa714091c20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa7027fc640 1 -- 172.21.15.122:0/1036405430 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710017610 con 0x5650e3ae0ca0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa7027fc640 1 -- 172.21.15.122:0/1036405430 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71001c070 con 0x5650e3ae0ca0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa7027fc640 1 -- 172.21.15.122:0/1036405430 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710015a10 con 0x5650e3ae0ca0 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa725277740 1 -- 172.21.15.122:0/1036405430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 msgr2=0x5650e3ae3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.001+0000 7fa725277740 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa7100101f0 tx=0x7fa7100095a0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- 172.21.15.122:0/1036405430 shutdown_connections 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae3090 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- 172.21.15.122:0/1036405430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3ad7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- 172.21.15.122:0/1036405430 >> 172.21.15.122:0/1036405430 conn(0x5650e3ad1ed0 msgr2=0x5650e3ad22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- 172.21.15.122:0/1036405430 shutdown_connections 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- 172.21.15.122:0/1036405430 wait complete. 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- start start 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 Processor -- start 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3ae0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad80c0 0x5650e3addae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5650e3ae3a90 con 0x5650e3ae0ca0 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5650e39f6e80 con 0x5650e3ad80c0 2024-04-02T13:38:33.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5650e3ae3c10 con 0x5650e3ad77a0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.002+0000 7fa725277740 10 client.0 _ll_register_callbacks cb 0x5650e3a32aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56562/0 (socket says 172.21.15.122:56562) 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 -- 172.21.15.122:0/2751694171 learned_addr learned my addr 172.21.15.122:0/2751694171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3ae0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad80c0 0x5650e3addae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 -- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 msgr2=0x5650e3ae0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3ae0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 -- 172.21.15.122:0/2751694171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad80c0 msgr2=0x5650e3addae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad80c0 0x5650e3addae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 -- 172.21.15.122:0/2751694171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e38197d0 con 0x5650e3ae0ca0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7100156f0 tx=0x7fa7100155f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae55ac0a3be62a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa6fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:33.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710015bf0 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa710015d90 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa71001c070 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa710017410 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa725277740 10 client.16755 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650e3aefb40 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.003+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71001c070 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa710021660 con 0x5650e3ae0ca0 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 10 client.16755 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 10 client.16755 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 20 client.16755 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3af3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 4 client.16755 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650e3a48cf0 con 0x5650e3af0e90 2024-04-02T13:38:33.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3af3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa6fb7fe640 10 client.16755 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.004+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3af3280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=16e1415be18f51b7 server_cookie=6e3dc39278f0392c in_seq=0 out_seq=0 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2751694171 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650e3a48cf0 con 0x5650e3af0e90 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa6fb7fe640 10 client.16755 handle_command_reply: tid=0 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 client.16755 shutdown 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 msgr2=0x5650e3ade220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7100156f0 tx=0x7fa7100155f0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 shutdown_connections 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3af3280 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ade220 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa6fb7fe640 0 client.16755 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad80c0 0x5650e3addae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 --2- 172.21.15.122:0/2751694171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3ae0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 >> 172.21.15.122:0/2751694171 conn(0x5650e3ad1ed0 msgr2=0x5650e3ad3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 shutdown_connections 2024-04-02T13:38:33.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 1 -- 172.21.15.122:0/2751694171 wait complete. 2024-04-02T13:38:33.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.005+0000 7fa725277740 20 client.16755 trim_cache size 0 max 16384 2024-04-02T13:38:33.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 Processor -- start 2024-04-02T13:38:33.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 -- start start 2024-04-02T13:38:33.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3aec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1b1280 con 0x5650e3ae0ca0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1b2ee0 con 0x5650e3ad77a0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa725277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1b1950 con 0x5650e3ad80c0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56576/0 (socket says 172.21.15.122:56576) 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1274286712 learned_addr learned my addr 172.21.15.122:0/1274286712 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3aec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71bfff640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1274286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 msgr2=0x5650e3ae0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1274286712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 msgr2=0x5650e3aec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3aec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 -- 172.21.15.122:0/1274286712 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650e3ad9670 con 0x5650e3ae0ca0 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71bfff640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa710009f00 tx=0x7fa7100047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe4a6e0d97e1477 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa701ffb640 1 -- 172.21.15.122:0/1274286712 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710009180 con 0x5650e3ae0ca0 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa701ffb640 1 -- 172.21.15.122:0/1274286712 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71001c040 con 0x5650e3ae0ca0 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.021+0000 7fa701ffb640 1 -- 172.21.15.122:0/1274286712 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710021400 con 0x5650e3ae0ca0 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- 172.21.15.122:0/1274286712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 msgr2=0x5650e3ae0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa710009f00 tx=0x7fa7100047d0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- 172.21.15.122:0/1274286712 shutdown_connections 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ae0ca0 0x5650e3ae0750 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad80c0 0x5650e3ae0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 --2- 172.21.15.122:0/1274286712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ad77a0 0x5650e3aec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- 172.21.15.122:0/1274286712 >> 172.21.15.122:0/1274286712 conn(0x5650e3ad1ed0 msgr2=0x5650e3ad22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- 172.21.15.122:0/1274286712 shutdown_connections 2024-04-02T13:38:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- 172.21.15.122:0/1274286712 wait complete. 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.022+0000 7fa725277740 1 -- start start 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 Processor -- start 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3adb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ae0ca0 0x5650e3ada440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1b2750 con 0x5650e3ad80c0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1b2080 con 0x5650e3ae0ca0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1af260 con 0x5650e3ad77a0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa725277740 10 client.0 _ll_register_callbacks cb 0x7fa70c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56590/0 (socket says 172.21.15.122:56590) 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 -- 172.21.15.122:0/2315843940 learned_addr learned my addr 172.21.15.122:0/2315843940 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3adb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ae0ca0 0x5650e3ada440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 -- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 msgr2=0x5650e3adb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3adb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 -- 172.21.15.122:0/2315843940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ae0ca0 msgr2=0x5650e3ada440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ae0ca0 0x5650e3ada440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 -- 172.21.15.122:0/2315843940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650e3ae7e60 con 0x5650e3ad80c0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3adb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa71bfff640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005ab0 tx=0x7fa70c02ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd15539f31faffa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.023+0000 7fa6fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c04d030 con 0x5650e3ad80c0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa70c004250 con 0x5650e3ad80c0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa71001c040 con 0x5650e3ad80c0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa70c05a050 con 0x5650e3ad80c0 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 10 client.16761 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5650e3ae90d0 con 0x5650e3ad80c0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa71001c040 con 0x5650e3ad80c0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa70c0049f0 con 0x5650e3ad80c0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 10 client.16761 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 10 client.16761 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 20 client.16761 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3b59270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 4 client.16761 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650e3a43040 con 0x5650e3af0e90 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.024+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3b59270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa6fb7fe640 10 client.16761 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa71b7fe640 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3b59270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28af844d937093ef server_cookie=b49269239e22552c in_seq=0 out_seq=0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa6fb7fe640 1 -- 172.21.15.122:0/2315843940 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650e3a43040 con 0x5650e3af0e90 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa6fb7fe640 10 client.16761 handle_command_reply: tid=0 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 client.16761 shutdown 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 msgr2=0x5650e3ad9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fa70c005ab0 tx=0x7fa70c02ccd0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 shutdown_connections 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5650e3af0e90 0x5650e3b59270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5650e3ae0ca0 0x5650e3ada440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa6fb7fe640 0 client.16761 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5650e3ad80c0 0x5650e3ad9d50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 --2- 172.21.15.122:0/2315843940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650e3ad77a0 0x5650e3adb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 >> 172.21.15.122:0/2315843940 conn(0x5650e3ad1ed0 msgr2=0x5650e3adc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 shutdown_connections 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 1 -- 172.21.15.122:0/2315843940 wait complete. 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.025+0000 7fa725277740 20 client.16761 trim_cache size 0 max 16384 2024-04-02T13:38:33.035 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 msgr2=0x7fa7040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 0x7fa7040a61e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa7140cdff0 tx=0x7fa714091c20 comp rx=0 tx=0).stop 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 msgr2=0x7fa71c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa71400cad0 tx=0x7fa71400ced0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 shutdown_connections 2024-04-02T13:38:33.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7040a3d20 0x7fa7040a61e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa71c106990 0x7fa71c197b30 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa71c101340 0x7fa71c1975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 --2- 172.21.15.122:0/236776250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa71c100980 0x7fa71c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 >> 172.21.15.122:0/236776250 conn(0x7fa71c076d80 msgr2=0x7fa71c0fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 shutdown_connections 2024-04-02T13:38:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.026+0000 7fa72268f640 1 -- 172.21.15.122:0/236776250 wait complete. 2024-04-02T13:38:33.040 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:38:33.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- 172.21.15.122:0/3642956535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41069c0 msgr2=0x7f4ce4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 --2- 172.21.15.122:0/3642956535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41069c0 0x7f4ce4106da0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009930 tx=0x7f4ce002bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:33.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- 172.21.15.122:0/3642956535 shutdown_connections 2024-04-02T13:38:33.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 --2- 172.21.15.122:0/3642956535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 --2- 172.21.15.122:0/3642956535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41009b0 0x7f4ce4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 --2- 172.21.15.122:0/3642956535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41069c0 0x7f4ce4106da0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- 172.21.15.122:0/3642956535 >> 172.21.15.122:0/3642956535 conn(0x7f4ce4076d80 msgr2=0x7f4ce4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- 172.21.15.122:0/3642956535 shutdown_connections 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- 172.21.15.122:0/3642956535 wait complete. 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 Processor -- start 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.140+0000 7f4cec1af640 1 -- start start 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41069c0 0x7f4ce419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4110dd0 con 0x7f4ce41009b0 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4110c50 con 0x7f4ce4101370 2024-04-02T13:38:33.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4110f50 con 0x7f4ce41069c0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56616/0 (socket says 172.21.15.122:56616) 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/2275298219 learned_addr learned my addr 172.21.15.122:0/2275298219 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41069c0 0x7f4ce419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce974a640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/2275298219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41069c0 msgr2=0x7f4ce419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41069c0 0x7f4ce419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/2275298219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 msgr2=0x7f4ce4197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce4197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/2275298219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce419c300 con 0x7f4ce41009b0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce974a640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009230 tx=0x7f4ce002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cdaffd640 1 -- 172.21.15.122:0/2275298219 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0041070 con 0x7f4ce41009b0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cdaffd640 1 -- 172.21.15.122:0/2275298219 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0004450 con 0x7f4ce41009b0 2024-04-02T13:38:33.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cdaffd640 1 -- 172.21.15.122:0/2275298219 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0034440 con 0x7f4ce41009b0 2024-04-02T13:38:33.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.141+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce419c590 con 0x7f4ce41009b0 2024-04-02T13:38:33.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.142+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41a3e80 con 0x7f4ce41009b0 2024-04-02T13:38:33.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.143+0000 7f4cdaffd640 1 -- 172.21.15.122:0/2275298219 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4ce0044070 con 0x7f4ce41009b0 2024-04-02T13:38:33.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 Processor -- start 2024-04-02T13:38:33.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 -- start start 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cdaffd640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a33eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a347760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55602a2a27b0 con 0x55602a347ca0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55602a241570 con 0x55602a33f0c0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ceed99740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55602a29e760 con 0x55602a33e7a0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cdaffd640 1 -- 172.21.15.122:0/2275298219 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4ce00f25f0 con 0x7f4ce41009b0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cea74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ce974a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a347760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cea74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56624/0 (socket says 172.21.15.122:56624) 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4cea74c640 1 -- 172.21.15.122:0/1070218629 learned_addr learned my addr 172.21.15.122:0/1070218629 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.144+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a33eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 -- 172.21.15.122:0/1070218629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 msgr2=0x55602a33eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a33eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 -- 172.21.15.122:0/1070218629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 msgr2=0x55602a347760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a347760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 -- 172.21.15.122:0/1070218629 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556029fb63f0 con 0x55602a347ca0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc013a70 tx=0x7f4cdc009cb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4cea74c640 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010c70 tx=0x7f4cdc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d725d90a8f1b5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4ccffff640 1 -- 172.21.15.122:0/1070218629 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc021070 con 0x55602a347ca0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4ccffff640 1 -- 172.21.15.122:0/1070218629 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc00ebf0 con 0x55602a347ca0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.145+0000 7f4ccffff640 1 -- 172.21.15.122:0/1070218629 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005220 con 0x55602a347ca0 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- 172.21.15.122:0/1070218629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 msgr2=0x55602a34a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010c70 tx=0x7f4cdc01a000 comp rx=0 tx=0).stop 2024-04-02T13:38:33.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- 172.21.15.122:0/1070218629 shutdown_connections 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a34a090 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a347760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 --2- 172.21.15.122:0/1070218629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a33eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- 172.21.15.122:0/1070218629 >> 172.21.15.122:0/1070218629 conn(0x55602a338ed0 msgr2=0x55602a3392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- 172.21.15.122:0/1070218629 shutdown_connections 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- 172.21.15.122:0/1070218629 wait complete. 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 -- start start 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 Processor -- start 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.146+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a3476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a344a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55602a34aa30 con 0x55602a347ca0 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55602a25de80 con 0x55602a33f0c0 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55602a34abb0 con 0x55602a33e7a0 2024-04-02T13:38:33.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 10 client.0 _ll_register_callbacks cb 0x55602a299aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56628/0 (socket says 172.21.15.122:56628) 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 -- 172.21.15.122:0/2639400024 learned_addr learned my addr 172.21.15.122:0/2639400024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a3476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ce974a640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a344a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 -- 172.21.15.122:0/2639400024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 msgr2=0x55602a3476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a3476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 -- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 msgr2=0x55602a344a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a344a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 -- 172.21.15.122:0/2639400024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602a0807d0 con 0x55602a347ca0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4cea74c640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010740 tx=0x7f4cdc009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27129be1d128f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:33.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc032060 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4cdc02d410 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc00ebf0 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc039050 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 10 client.16773 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.147+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55602a356b60 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc00ebf0 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4cdc00f040 con 0x55602a347ca0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 10 client.16773 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 20 client.16773 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a358020 0x55602a35a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 4 client.16773 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55602a2afcf0 con 0x55602a358020 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.148+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a358020 0x55602a35a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ccd7fa640 10 client.16773 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:33.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a358020 0x55602a35a410 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddeee540e661fbed server_cookie=66fae57c27f67bb8 in_seq=0 out_seq=0 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/2639400024 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55602a2afcf0 con 0x55602a358020 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ccd7fa640 10 client.16773 handle_command_reply: tid=0 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 client.16773 shutdown 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 msgr2=0x55602a345060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc010740 tx=0x7f4cdc009f30 comp rx=0 tx=0).stop 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 shutdown_connections 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a358020 0x55602a35a410 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a347ca0 0x55602a345060 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a344a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 --2- 172.21.15.122:0/2639400024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a33e7a0 0x55602a3476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ccd7fa640 0 client.16773 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.149+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 >> 172.21.15.122:0/2639400024 conn(0x55602a338ed0 msgr2=0x55602a33ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.150+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 shutdown_connections 2024-04-02T13:38:33.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.150+0000 7f4ceed99740 1 -- 172.21.15.122:0/2639400024 wait complete. 2024-04-02T13:38:33.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.150+0000 7f4ceed99740 20 client.16773 trim_cache size 0 max 16384 2024-04-02T13:38:33.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 Processor -- start 2024-04-02T13:38:33.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 -- start start 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a33aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a347180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41addf0 con 0x55602a33e7a0 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41afa50 con 0x55602a33f0c0 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ceed99740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41ae4c0 con 0x55602a347ca0 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56644/0 (socket says 172.21.15.122:56644) 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/4252114725 learned_addr learned my addr 172.21.15.122:0/4252114725 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce974a640 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a33aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/4252114725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 msgr2=0x55602a347180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a347180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/4252114725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 msgr2=0x55602a33aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a33aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/4252114725 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55602a35a780 con 0x55602a33e7a0 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.163+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00efb20 tx=0x7f4ce0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b888b9bf5282fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ccf7fe640 1 -- 172.21.15.122:0/4252114725 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce004b040 con 0x55602a33e7a0 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ccf7fe640 1 -- 172.21.15.122:0/4252114725 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce00c6f80 con 0x55602a33e7a0 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ccf7fe640 1 -- 172.21.15.122:0/4252114725 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00c6af0 con 0x55602a33e7a0 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- 172.21.15.122:0/4252114725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 msgr2=0x55602a3539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00efb20 tx=0x7f4ce0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- 172.21.15.122:0/4252114725 shutdown_connections 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a347180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a33aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 --2- 172.21.15.122:0/4252114725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a3539a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- 172.21.15.122:0/4252114725 >> 172.21.15.122:0/4252114725 conn(0x55602a338ed0 msgr2=0x55602a33a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- 172.21.15.122:0/4252114725 shutdown_connections 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- 172.21.15.122:0/4252114725 wait complete. 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 -- start start 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.164+0000 7f4ceed99740 1 Processor -- start 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a3485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a348b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41af2c0 con 0x55602a33e7a0 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41aebf0 con 0x55602a33f0c0 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41abdd0 con 0x55602a347ca0 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 10 client.0 _ll_register_callbacks cb 0x7f4cd400b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56652/0 (socket says 172.21.15.122:56652) 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/3134875744 learned_addr learned my addr 172.21.15.122:0/3134875744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:33.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4cea74c640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a348b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce974a640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a3485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/3134875744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 msgr2=0x55602a348b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a348b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 msgr2=0x55602a3485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a3485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 -- 172.21.15.122:0/3134875744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55602a34efe0 con 0x55602a33e7a0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ce9f4b640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0006010 tx=0x7f4ce0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473ae725bb505100 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce004b070 con 0x55602a33e7a0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f4ce00036f0 con 0x55602a33e7a0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.165+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce00c6f80 con 0x55602a33e7a0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4ce00c6af0 con 0x55602a33e7a0 2024-04-02T13:38:33.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 10 client.16779 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55602a3502a0 con 0x55602a33e7a0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce00c6f80 con 0x55602a33e7a0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f4ce00bd760 con 0x55602a33e7a0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 10 client.16779 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 10 client.16779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 20 client.16779 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a3c2970 0x55602a3c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 4 client.16779 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55602a340650 con 0x55602a3c2970 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.166+0000 7f4ce974a640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a3c2970 0x55602a3c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ccd7fa640 10 client.16779 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ce974a640 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a3c2970 0x55602a3c4d60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa513d806df85b8b server_cookie=681af81a64c2fa31 in_seq=0 out_seq=0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ccd7fa640 1 -- 172.21.15.122:0/3134875744 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55602a340650 con 0x55602a3c2970 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ccd7fa640 10 client.16779 handle_command_reply: tid=0 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 client.16779 shutdown 2024-04-02T13:38:33.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 msgr2=0x55602a348080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0006010 tx=0x7f4ce0009260 comp rx=0 tx=0).stop 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 shutdown_connections 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55602a3c2970 0x55602a3c4d60 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55602a347ca0 0x55602a348b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55602a33f0c0 0x55602a3485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 --2- 172.21.15.122:0/3134875744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55602a33e7a0 0x55602a348080 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ccd7fa640 0 client.16779 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 >> 172.21.15.122:0/3134875744 conn(0x55602a338ed0 msgr2=0x55602a337f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 shutdown_connections 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 1 -- 172.21.15.122:0/3134875744 wait complete. 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.167+0000 7f4ceed99740 20 client.16779 trim_cache size 0 max 16384 2024-04-02T13:38:33.177 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:38:33.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 msgr2=0x7f4cd00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc013a70 tx=0x7f4cdc009cb0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 msgr2=0x7f4ce4197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009230 tx=0x7f4ce002cc70 comp rx=0 tx=0).stop 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 shutdown_connections 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4cd00a3d20 0x7f4cd00a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ce41069c0 0x7f4ce419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ce4101370 0x7f4ce4197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 --2- 172.21.15.122:0/2275298219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ce41009b0 0x7f4ce4197310 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 >> 172.21.15.122:0/2275298219 conn(0x7f4ce4076d80 msgr2=0x7f4ce40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 shutdown_connections 2024-04-02T13:38:33.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:33.168+0000 7f4cec1af640 1 -- 172.21.15.122:0/2275298219 wait complete. 2024-04-02T13:38:33.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:33 smithi188 ceph-mon[36117]: pgmap v104: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-02T13:38:33.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/477650657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:33.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:33 smithi122 ceph-mon[32114]: pgmap v104: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-02T13:38:33.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/477650657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:33 smithi187 ceph-mon[36064]: pgmap v104: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-04-02T13:38:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/477650657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:38:33.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 96607 4.72 MB/sec warmup 114 sec latency 3235.335 ms 2024-04-02T13:38:34.182 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:38:34.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:38:34.182 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 -- 172.21.15.122:0/975714070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100a20 msgr2=0x7fe160100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 --2- 172.21.15.122:0/975714070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100a20 0x7fe160100ea0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe154009930 tx=0x7fe15402bc30 comp rx=0 tx=0).stop 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 -- 172.21.15.122:0/975714070 shutdown_connections 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 --2- 172.21.15.122:0/975714070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1601013e0 0x7fe16010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 --2- 172.21.15.122:0/975714070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100a20 0x7fe160100ea0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 --2- 172.21.15.122:0/975714070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe160106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 -- 172.21.15.122:0/975714070 >> 172.21.15.122:0/975714070 conn(0x7fe160077b60 msgr2=0x7fe160077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 -- 172.21.15.122:0/975714070 shutdown_connections 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.294+0000 7fe1663a8640 1 -- 172.21.15.122:0/975714070 wait complete. 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 Processor -- start 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 -- start start 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 0x7fe160197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe16019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110e40 con 0x7fe1601013e0 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110cc0 con 0x7fe160106a30 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe1663a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110fc0 con 0x7fe160100a20 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.295+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56662/0 (socket says 172.21.15.122:56662) 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3552158495 learned_addr learned my addr 172.21.15.122:0/3552158495 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 0x7fe160197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe164945640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe16019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3552158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 msgr2=0x7fe160197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 0x7fe160197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3552158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 msgr2=0x7fe16019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe16019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3552158495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16019c350 con 0x7fe1601013e0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 0x7fe160197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe164945640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe16019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe154009230 tx=0x7fe15402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15d7fa640 1 -- 172.21.15.122:0/3552158495 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154041070 con 0x7fe1601013e0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16019c5e0 con 0x7fe1601013e0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15d7fa640 1 -- 172.21.15.122:0/3552158495 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154004450 con 0x7fe1601013e0 2024-04-02T13:38:34.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe15d7fa640 1 -- 172.21.15.122:0/3552158495 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154003e50 con 0x7fe1601013e0 2024-04-02T13:38:34.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.296+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a3ed0 con 0x7fe1601013e0 2024-04-02T13:38:34.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.298+0000 7fe15d7fa640 1 -- 172.21.15.122:0/3552158495 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe154044070 con 0x7fe1601013e0 2024-04-02T13:38:34.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 Processor -- start 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 -- start start 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15d7fa640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 0x7fe1480a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041d96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041e27c0 0x5604041e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56040413d770 con 0x5604041d9be0 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5604040e5960 con 0x5604041e27c0 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe168f8f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5604041398c0 con 0x5604041d92c0 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe164945640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 0x7fe1480a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56664/0 (socket says 172.21.15.122:56664) 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3655135153 learned_addr learned my addr 172.21.15.122:0/3655135153 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15d7fa640 1 -- 172.21.15.122:0/3552158495 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe1540f2480 con 0x7fe1601013e0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe164945640 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041e27c0 0x5604041e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.299+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041d96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3655135153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 msgr2=0x5604041d96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041d96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3655135153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041e27c0 msgr2=0x5604041e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041e27c0 0x5604041e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 -- 172.21.15.122:0/3655135153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560403f197e0 con 0x5604041d9be0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe154006010 tx=0x7fe154049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f00f8fdb5b3bdf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe164945640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 0x7fe1480a6380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe158005fd0 tx=0x7fe158012bb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe1467fc640 1 -- 172.21.15.122:0/3655135153 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15404b040 con 0x5604041d9be0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe1467fc640 1 -- 172.21.15.122:0/3655135153 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154004c80 con 0x5604041d9be0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe1467fc640 1 -- 172.21.15.122:0/3655135153 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540be1a0 con 0x5604041d9be0 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe168f8f740 1 -- 172.21.15.122:0/3655135153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 msgr2=0x5604041e2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.300+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fe154006010 tx=0x7fe154049000 comp rx=0 tx=0).stop 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- 172.21.15.122:0/3655135153 shutdown_connections 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041e27c0 0x5604041e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d9be0 0x5604041e2280 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3655135153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041d96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- 172.21.15.122:0/3655135153 >> 172.21.15.122:0/3655135153 conn(0x5604041d39f0 msgr2=0x5604041d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- 172.21.15.122:0/3655135153 shutdown_connections 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- 172.21.15.122:0/3655135153 wait complete. 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- start start 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 Processor -- start 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041df720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041dfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5604041e5370 con 0x5604041e27c0 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5604040f8d20 con 0x5604041d9be0 2024-04-02T13:38:34.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5604041e54f0 con 0x5604041d92c0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe168f8f740 10 client.0 _ll_register_callbacks cb 0x5604040c5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.301+0000 7fe164945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56672/0 (socket says 172.21.15.122:56672) 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 -- 172.21.15.122:0/3516071826 learned_addr learned my addr 172.21.15.122:0/3516071826 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041dfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 -- 172.21.15.122:0/3516071826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 msgr2=0x5604041df720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041df720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 -- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 msgr2=0x5604041dfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041dfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 -- 172.21.15.122:0/3516071826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560403f79f70 con 0x5604041e27c0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe164945640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe15800f620 tx=0x7fe158004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f306b6492f3bd785 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158023070 con 0x5604041e27c0 2024-04-02T13:38:34.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe15801f6d0 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe154004c80 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe158031050 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe168f8f740 10 client.16791 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.302+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560403e11170 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154004c80 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe15801e070 con 0x5604041e27c0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 10 client.16791 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 20 client.16791 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5604041f2890 0x5604041f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 4 client.16791 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56040414acb0 con 0x5604041f2890 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5604041f2890 0x5604041f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe15ffff640 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5604041f2890 0x5604041f4cd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf16dbe3ef3a9e3 server_cookie=a8f469c968cb6b3e in_seq=0 out_seq=0 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.303+0000 7fe13f7fe640 10 client.16791 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.304+0000 7fe13f7fe640 1 -- 172.21.15.122:0/3516071826 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56040414acb0 con 0x5604041f2890 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.304+0000 7fe13f7fe640 10 client.16791 handle_command_reply: tid=0 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.304+0000 7fe168f8f740 1 client.16791 shutdown 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 msgr2=0x5604041e01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe15800f620 tx=0x7fe158004ab0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 shutdown_connections 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5604041f2890 0x5604041f4cd0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x5604041e01a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041dfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 --2- 172.21.15.122:0/3516071826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d92c0 0x5604041df720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe13f7fe640 0 client.16791 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 >> 172.21.15.122:0/3516071826 conn(0x5604041d39f0 msgr2=0x5604041d5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 shutdown_connections 2024-04-02T13:38:34.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 1 -- 172.21.15.122:0/3516071826 wait complete. 2024-04-02T13:38:34.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.305+0000 7fe168f8f740 20 client.16791 trim_cache size 0 max 16384 2024-04-02T13:38:34.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 Processor -- start 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 -- start start 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 0x5604041d2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601adde0 con 0x5604041d92c0 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601afa40 con 0x5604041d9be0 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.322+0000 7fe168f8f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601ae4b0 con 0x5604041e27c0 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56678/0 (socket says 172.21.15.122:56678) 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 -- 172.21.15.122:0/1313248474 learned_addr learned my addr 172.21.15.122:0/1313248474 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 -- 172.21.15.122:0/1313248474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 msgr2=0x5604041d2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe164945640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 0x5604041d2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 0x5604041d2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 -- 172.21.15.122:0/1313248474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 msgr2=0x5604041ee230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041ee230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 -- 172.21.15.122:0/1313248474 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604041d9a10 con 0x5604041d92c0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe164945640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 0x5604041d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe150014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2439433645f2b810 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe145ffb640 1 -- 172.21.15.122:0/1313248474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1500142d0 con 0x5604041d92c0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe145ffb640 1 -- 172.21.15.122:0/1313248474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1500035f0 con 0x5604041d92c0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.323+0000 7fe145ffb640 1 -- 172.21.15.122:0/1313248474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15001e280 con 0x5604041d92c0 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- 172.21.15.122:0/1313248474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 msgr2=0x5604041e1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe150000c00 tx=0x7fe150014840 comp rx=0 tx=0).stop 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- 172.21.15.122:0/1313248474 shutdown_connections 2024-04-02T13:38:34.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041e27c0 0x5604041d2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d9be0 0x5604041ee230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1313248474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041d92c0 0x5604041e1e50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- 172.21.15.122:0/1313248474 >> 172.21.15.122:0/1313248474 conn(0x5604041d39f0 msgr2=0x5604041d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- 172.21.15.122:0/1313248474 shutdown_connections 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- 172.21.15.122:0/1313248474 wait complete. 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 -- start start 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 Processor -- start 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 0x5604041e87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d9be0 0x5604041e8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.324+0000 7fe168f8f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601af2b0 con 0x5604041e27c0 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601aebe0 con 0x5604041d92c0 2024-04-02T13:38:34.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601abdc0 con 0x5604041d9be0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 10 client.0 _ll_register_callbacks cb 0x7fe1580098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56690/0 (socket says 172.21.15.122:56690) 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 -- 172.21.15.122:0/1734511965 learned_addr learned my addr 172.21.15.122:0/1734511965 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe15f7fe640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d9be0 0x5604041e8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 0x5604041e87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 -- 172.21.15.122:0/1734511965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d9be0 msgr2=0x5604041e8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d9be0 0x5604041e8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 -- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 msgr2=0x5604041e87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 0x5604041e87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 -- 172.21.15.122:0/1734511965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56040425a3b0 con 0x5604041e27c0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 0x5604041e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe164945640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe15800a010 tx=0x7fe15801c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decfb5d8321bcf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe13f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158023050 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe1580124d0 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1500035f0 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe15800ed40 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 10 client.16797 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56040425b590 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.325+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1500035f0 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe158034070 con 0x5604041e27c0 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 10 client.16797 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 10 client.16797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 20 client.16797 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56040425be20 0x56040425e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 4 client.16797 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:38:34.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560404145000 con 0x56040425be20 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56040425be20 0x56040425e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe15ffff640 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56040425be20 0x56040425e240 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0f3f076da93dea2 server_cookie=5fd4021fb0606625 in_seq=0 out_seq=0 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.326+0000 7fe13f7fe640 10 client.16797 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.327+0000 7fe13f7fe640 1 -- 172.21.15.122:0/1734511965 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560404145000 con 0x56040425be20 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.327+0000 7fe13f7fe640 10 client.16797 handle_command_reply: tid=0 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.327+0000 7fe168f8f740 1 client.16797 shutdown 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 msgr2=0x560404259e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe15800a010 tx=0x7fe15801c5d0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 shutdown_connections 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56040425be20 0x56040425e240 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5604041e27c0 0x560404259e70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5604041d9be0 0x5604041e8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe13f7fe640 0 client.16797 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 --2- 172.21.15.122:0/1734511965 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5604041d92c0 0x5604041e87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 >> 172.21.15.122:0/1734511965 conn(0x5604041d39f0 msgr2=0x5604041dff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 shutdown_connections 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 1 -- 172.21.15.122:0/1734511965 wait complete. 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe168f8f740 20 client.16797 trim_cache size 0 max 16384 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "c941859e-558f-4770-aec1-cda469361e16", 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:38:34.338 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:38:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 msgr2=0x7fe1480a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 0x7fe1480a6380 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe158005fd0 tx=0x7fe158012bb0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 msgr2=0x7fe1601978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe154009230 tx=0x7fe15402cb30 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 shutdown_connections 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3ec0 0x7fe1480a6380 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106a30 0x7fe16019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.328+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe1601013e0 0x7fe1601978a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.329+0000 7fe1663a8640 1 --2- 172.21.15.122:0/3552158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100a20 0x7fe160197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.329+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 >> 172.21.15.122:0/3552158495 conn(0x7fe160077b60 msgr2=0x7fe160103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.329+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 shutdown_connections 2024-04-02T13:38:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.329+0000 7fe1663a8640 1 -- 172.21.15.122:0/3552158495 wait complete. 2024-04-02T13:38:34.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:38:34.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.462+0000 7f496821b640 1 -- 172.21.15.122:0/2001124125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960101330 msgr2=0x7f496010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.462+0000 7f496821b640 1 --2- 172.21.15.122:0/2001124125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960101330 0x7f496010d840 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4958009cd0 tx=0x7f495802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- 172.21.15.122:0/2001124125 shutdown_connections 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- 172.21.15.122:0/2001124125 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960101330 0x7f496010d840 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- 172.21.15.122:0/2001124125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960100970 0x7f4960100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- 172.21.15.122:0/2001124125 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960106980 0x7f4960106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- 172.21.15.122:0/2001124125 >> 172.21.15.122:0/2001124125 conn(0x7f4960077b00 msgr2=0x7f4960077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- 172.21.15.122:0/2001124125 shutdown_connections 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- 172.21.15.122:0/2001124125 wait complete. 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 Processor -- start 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- start start 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 0x7f4960197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960101330 0x7f4960197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4960110d90 con 0x7f4960106980 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4960110c10 con 0x7f4960100970 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.463+0000 7f496821b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4960110f10 con 0x7f4960101330 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f49667b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 0x7f4960197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f49657b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960101330 0x7f4960197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f49667b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56718/0 (socket says 172.21.15.122:56718) 2024-04-02T13:38:34.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f49667b8640 1 -- 172.21.15.122:0/1528296043 learned_addr learned my addr 172.21.15.122:0/1528296043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 -- 172.21.15.122:0/1528296043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960101330 msgr2=0x7f4960197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960101330 0x7f4960197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 -- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 msgr2=0x7f496019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 -- 172.21.15.122:0/1528296043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496019c220 con 0x7f4960100970 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f49667b8640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4965fb7640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 0x7f4960197230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f495c00eb00 tx=0x7f495c009e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4956ffd640 1 -- 172.21.15.122:0/1528296043 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c01c070 con 0x7f4960100970 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4956ffd640 1 -- 172.21.15.122:0/1528296043 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c00bb70 con 0x7f4960100970 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496019c510 con 0x7f4960100970 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f4956ffd640 1 -- 172.21.15.122:0/1528296043 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c016610 con 0x7f4960100970 2024-04-02T13:38:34.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.464+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4960068c60 con 0x7f4960100970 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f496ae04740 1 Processor -- start 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f4956ffd640 1 -- 172.21.15.122:0/1528296043 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f495c01f070 con 0x7f4960100970 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f496ae04740 1 -- start start 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f496ae04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f6639e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f496ae04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f670ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.467+0000 7f496ae04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f496ae04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55981f5c7830 con 0x55981f663600 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f496ae04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55981f56f820 con 0x55981f6711e0 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f496ae04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55981f5c37e0 con 0x55981f663f20 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f670ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:55620/0 (socket says 172.21.15.122:55620) 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 -- 172.21.15.122:0/2786921847 learned_addr learned my addr 172.21.15.122:0/2786921847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f670ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47194/0 (socket says 172.21.15.122:47194) 2024-04-02T13:38:34.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49667b8640 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f6639e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f4956ffd640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 0x7f494c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 -- 172.21.15.122:0/2786921847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 msgr2=0x55981f670ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f670ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 -- 172.21.15.122:0/2786921847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 msgr2=0x55981f6639e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f6639e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 -- 172.21.15.122:0/2786921847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981f3a57d0 con 0x55981f6711e0 2024-04-02T13:38:34.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49667b8640 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f6639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49667b8640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 0x7f494c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f4956ffd640 1 -- 172.21.15.122:0/1528296043 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f495c0d2080 con 0x7f4960100970 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f49657b6640 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ca0 tx=0x7f4950011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4710df58ad2c2893 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f494bfff640 1 -- 172.21.15.122:0/2786921847 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495001e070 con 0x55981f6711e0 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.468+0000 7f494bfff640 1 -- 172.21.15.122:0/2786921847 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4950019b10 con 0x55981f6711e0 2024-04-02T13:38:34.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f494bfff640 1 -- 172.21.15.122:0/2786921847 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495000ea80 con 0x55981f6711e0 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 -- 172.21.15.122:0/2786921847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 msgr2=0x55981f6735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4950009ca0 tx=0x7f4950011810 comp rx=0 tx=0).stop 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f49667b8640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 0x7f494c0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4958009a60 tx=0x7f495802e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 -- 172.21.15.122:0/2786921847 shutdown_connections 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f6735d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f670ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 --2- 172.21.15.122:0/2786921847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f6639e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 -- 172.21.15.122:0/2786921847 >> 172.21.15.122:0/2786921847 conn(0x55981f65e000 msgr2=0x55981f65e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 -- 172.21.15.122:0/2786921847 shutdown_connections 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.469+0000 7f496ae04740 1 -- 172.21.15.122:0/2786921847 wait complete. 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 -- start start 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 Processor -- start 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f66de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f66e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55981f673fa0 con 0x55981f663600 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55981f582d30 con 0x55981f6711e0 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55981f674120 con 0x55981f663f20 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f496ae04740 10 client.0 _ll_register_callbacks cb 0x55981f582720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56730/0 (socket says 172.21.15.122:56730) 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f4965fb7640 1 -- 172.21.15.122:0/3963341793 learned_addr learned my addr 172.21.15.122:0/3963341793 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.470+0000 7f49667b8640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f66e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49657b6640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f66de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 -- 172.21.15.122:0/3963341793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 msgr2=0x55981f66de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f66de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 -- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 msgr2=0x55981f66e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f66e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 -- 172.21.15.122:0/3963341793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55981f2db3f0 con 0x55981f663600 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f4965fb7640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b690 tx=0x7f495c0d35b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e3a2ae460f18ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49497fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c09d6a0 con 0x55981f663600 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f495c00e0d0 con 0x55981f663600 2024-04-02T13:38:34.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4950019b10 con 0x55981f663600 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f495c09dd40 con 0x55981f663600 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f496ae04740 10 client.16809 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55981f29b0f0 con 0x55981f663600 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.471+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4950019b10 con 0x55981f663600 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f495c0cfb90 con 0x55981f663600 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 10 client.16809 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:34.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 20 client.16809 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f681200 0x55981f683680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 4 client.16809 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55981f5d4d70 con 0x55981f681200 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f49657b6640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f681200 0x55981f683680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f49497fa640 10 client.16809 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.472+0000 7f49657b6640 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f681200 0x55981f683680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c2434c749a0c1b server_cookie=1abf3cf097774dc1 in_seq=0 out_seq=0 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f49497fa640 1 -- 172.21.15.122:0/3963341793 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55981f5d4d70 con 0x55981f681200 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f49497fa640 10 client.16809 handle_command_reply: tid=0 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 client.16809 shutdown 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 msgr2=0x55981f670a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b690 tx=0x7f495c0d35b0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 shutdown_connections 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f681200 0x55981f683680 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f66e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f66de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 --2- 172.21.15.122:0/3963341793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663600 0x55981f670a70 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f49497fa640 0 client.16809 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 >> 172.21.15.122:0/3963341793 conn(0x55981f65e000 msgr2=0x55981f65fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 shutdown_connections 2024-04-02T13:38:34.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 1 -- 172.21.15.122:0/3963341793 wait complete. 2024-04-02T13:38:34.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.473+0000 7f496ae04740 20 client.16809 trim_cache size 0 max 16384 2024-04-02T13:38:34.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 Processor -- start 2024-04-02T13:38:34.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 -- start start 2024-04-02T13:38:34.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663600 0x55981f67cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f688f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49601acc60 con 0x55981f663f20 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49601ae8c0 con 0x55981f6711e0 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f496ae04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49601ad330 con 0x55981f663600 2024-04-02T13:38:34.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f49657b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f49657b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56734/0 (socket says 172.21.15.122:56734) 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f49657b6640 1 -- 172.21.15.122:0/3106086857 learned_addr learned my addr 172.21.15.122:0/3106086857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f4965fb7640 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663600 0x55981f67cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.492+0000 7f49667b8640 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f688f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 -- 172.21.15.122:0/3106086857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663600 msgr2=0x55981f67cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663600 0x55981f67cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 -- 172.21.15.122:0/3106086857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 msgr2=0x55981f688f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f688f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 -- 172.21.15.122:0/3106086857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981f65de30 con 0x55981f663f20 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f49657b6640 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4950009230 tx=0x7f4950009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562a5ad914bcacb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f494b7fe640 1 -- 172.21.15.122:0/3106086857 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495001e040 con 0x55981f663f20 2024-04-02T13:38:34.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f494b7fe640 1 -- 172.21.15.122:0/3106086857 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495002e070 con 0x55981f663f20 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.493+0000 7f494b7fe640 1 -- 172.21.15.122:0/3106086857 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495002a440 con 0x55981f663f20 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- 172.21.15.122:0/3106086857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 msgr2=0x55981f669350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4950009230 tx=0x7f4950009780 comp rx=0 tx=0).stop 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- 172.21.15.122:0/3106086857 shutdown_connections 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f6711e0 0x55981f688f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f663f20 0x55981f669350 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- 172.21.15.122:0/3106086857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663600 0x55981f67cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- 172.21.15.122:0/3106086857 >> 172.21.15.122:0/3106086857 conn(0x55981f65e000 msgr2=0x55981f668c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- 172.21.15.122:0/3106086857 shutdown_connections 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- 172.21.15.122:0/3106086857 wait complete. 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- start start 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 Processor -- start 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 0x55981f6797d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f674b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49601ae130 con 0x55981f6711e0 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49601ada60 con 0x55981f663600 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.494+0000 7f496ae04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49601aac40 con 0x55981f663f20 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f496ae04740 10 client.0 _ll_register_callbacks cb 0x7f495c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49667b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 0x55981f6797d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49657b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f674b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49667b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56744/0 (socket says 172.21.15.122:56744) 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49667b8640 1 -- 172.21.15.122:0/2704880086 learned_addr learned my addr 172.21.15.122:0/2704880086 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 -- 172.21.15.122:0/2704880086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 msgr2=0x55981f674b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f674b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 -- 172.21.15.122:0/2704880086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 msgr2=0x55981f6750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:38:34.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 -- 172.21.15.122:0/2704880086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55981f675920 con 0x55981f663600 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49667b8640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f4965fb7640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 0x55981f6797d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f495c09a1a0 tx=0x7f495c0be2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af8a9064a60f5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49497fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c09d6a0 con 0x55981f663600 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f495c09d190 con 0x55981f663600 2024-04-02T13:38:34.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495002e070 con 0x55981f663600 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.495+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f495c0a2040 con 0x55981f663600 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 10 client.26165 fetch_fsmap learned FSMap version 32 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55981f6e7840 con 0x55981f663600 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495002e070 con 0x55981f663600 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f495c07cb60 con 0x55981f663600 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 10 client.26165 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:38:34.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 10 client.26165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 20 client.26165 populate_metadata read hostname 'smithi122' 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f6e8070 0x55981f6ea490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 4 client.26165 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55981f5cf0c0 con 0x55981f6e8070 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f49657b6640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f6e8070 0x55981f6ea490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f49657b6640 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f6e8070 0x55981f6ea490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a755a3a931850ca server_cookie=5de539ef70e9b4a4 in_seq=0 out_seq=0 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.496+0000 7f49497fa640 10 client.26165 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f49497fa640 1 -- 172.21.15.122:0/2704880086 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55981f5cf0c0 con 0x55981f6e8070 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f49497fa640 10 client.26165 handle_command_reply: tid=0 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 client.26165 shutdown 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 msgr2=0x55981f6797d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 0x55981f6797d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f495c09a1a0 tx=0x7f495c0be2e0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 shutdown_connections 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55981f6e8070 0x55981f6ea490 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55981f6711e0 0x55981f6750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f49497fa640 0 client.26165 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55981f663f20 0x55981f674b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 --2- 172.21.15.122:0/2704880086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55981f663600 0x55981f6797d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 >> 172.21.15.122:0/2704880086 conn(0x55981f65e000 msgr2=0x55981f65c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 shutdown_connections 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 1 -- 172.21.15.122:0/2704880086 wait complete. 2024-04-02T13:38:34.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.497+0000 7f496ae04740 20 client.26165 trim_cache size 0 max 16384 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout: "c941859e-558f-4770-aec1-cda469361e16": { 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:38:34.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "c941859e-558f-4770-aec1-cda469361e16", 2024-04-02T13:38:34.521 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:38:34.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:38:34.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:38:34.521 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:38:34.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 msgr2=0x7f494c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 0x7f494c0a61e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4958009a60 tx=0x7f495802e040 comp rx=0 tx=0).stop 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 msgr2=0x7f4960197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 0x7f4960197230 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f495c00eb00 tx=0x7f495c009e60 comp rx=0 tx=0).stop 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 shutdown_connections 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f494c0a3d20 0x7f494c0a61e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4960106980 0x7f496019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4960101330 0x7f4960197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 --2- 172.21.15.122:0/1528296043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4960100970 0x7f4960197230 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:38:34.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 >> 172.21.15.122:0/1528296043 conn(0x7f4960077b00 msgr2=0x7f49601030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:38:34.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 shutdown_connections 2024-04-02T13:38:34.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:38:34.498+0000 7f496821b640 1 -- 172.21.15.122:0/1528296043 wait complete. 2024-04-02T13:38:34.525 INFO:tasks.cephfs.filesystem:scrub status for tag:c941859e-558f-4770-aec1-cda469361e16 - {'path': '/', 'tag': 'c941859e-558f-4770-aec1-cda469361e16', 'options': 'recursive,force'} 2024-04-02T13:38:34.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 98218 4.73 MB/sec warmup 115 sec latency 852.217 ms 2024-04-02T13:38:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: pgmap v105: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 791 KiB/s wr, 97 op/s 2024-04-02T13:38:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:38:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:35 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: pgmap v105: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 791 KiB/s wr, 97 op/s 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:38:35.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:35 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: pgmap v105: 129 pgs: 129 active+clean; 65 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 791 KiB/s wr, 97 op/s 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:35 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:38:35.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 99897 4.80 MB/sec warmup 116 sec latency 10.571 ms 2024-04-02T13:38:36.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100028 4.76 MB/sec warmup 117 sec latency 898.243 ms 2024-04-02T13:38:37.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:37 smithi122 ceph-mon[32114]: pgmap v106: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-02T13:38:37.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:37 smithi187 ceph-mon[36064]: pgmap v106: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-02T13:38:37.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:37 smithi188 ceph-mon[36117]: pgmap v106: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-04-02T13:38:37.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100028 4.72 MB/sec warmup 118 sec latency 1898.343 ms 2024-04-02T13:38:38.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100084 4.68 MB/sec warmup 119 sec latency 2361.916 ms 2024-04-02T13:38:39.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:39 smithi122 ceph-mon[32114]: pgmap v107: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 933 KiB/s wr, 105 op/s 2024-04-02T13:38:39.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:39 smithi187 ceph-mon[36064]: pgmap v107: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 933 KiB/s wr, 105 op/s 2024-04-02T13:38:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:39 smithi188 ceph-mon[36117]: pgmap v107: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 933 KiB/s wr, 105 op/s 2024-04-02T13:38:40.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100084 0.00 MB/sec execute 1 sec latency 2494.750 ms 2024-04-02T13:38:41.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:41 smithi122 ceph-mon[32114]: pgmap v108: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 867 KiB/s wr, 99 op/s 2024-04-02T13:38:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:41 smithi187 ceph-mon[36064]: pgmap v108: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 867 KiB/s wr, 99 op/s 2024-04-02T13:38:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:41 smithi188 ceph-mon[36117]: pgmap v108: 129 pgs: 129 active+clean; 67 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 867 KiB/s wr, 99 op/s 2024-04-02T13:38:41.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100084 0.00 MB/sec execute 2 sec latency 3494.877 ms 2024-04-02T13:38:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:38:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:42.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:38:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:42.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:38:42.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100084 0.00 MB/sec execute 3 sec latency 4494.987 ms 2024-04-02T13:38:43.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:43 smithi122 ceph-mon[32114]: pgmap v109: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 914 KiB/s wr, 115 op/s 2024-04-02T13:38:43.386 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:43 smithi187 ceph-mon[36064]: pgmap v109: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 914 KiB/s wr, 115 op/s 2024-04-02T13:38:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:43 smithi188 ceph-mon[36117]: pgmap v109: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 914 KiB/s wr, 115 op/s 2024-04-02T13:38:43.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100152 0.04 MB/sec execute 4 sec latency 4951.439 ms 2024-04-02T13:38:44.946 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100152 0.03 MB/sec execute 5 sec latency 1489.677 ms 2024-04-02T13:38:45.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:45 smithi122 ceph-mon[32114]: pgmap v110: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 518 KiB/s wr, 68 op/s 2024-04-02T13:38:45.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:45 smithi187 ceph-mon[36064]: pgmap v110: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 518 KiB/s wr, 68 op/s 2024-04-02T13:38:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:45 smithi188 ceph-mon[36117]: pgmap v110: 129 pgs: 129 active+clean; 67 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 518 KiB/s wr, 68 op/s 2024-04-02T13:38:45.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100152 0.03 MB/sec execute 6 sec latency 2489.822 ms 2024-04-02T13:38:46.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100152 0.02 MB/sec execute 7 sec latency 3489.907 ms 2024-04-02T13:38:47.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:47 smithi122 ceph-mon[32114]: pgmap v111: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 750 KiB/s wr, 80 op/s 2024-04-02T13:38:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:47 smithi187 ceph-mon[36064]: pgmap v111: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 750 KiB/s wr, 80 op/s 2024-04-02T13:38:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:47 smithi188 ceph-mon[36117]: pgmap v111: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 750 KiB/s wr, 80 op/s 2024-04-02T13:38:47.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 100152 0.02 MB/sec execute 8 sec latency 4490.010 ms 2024-04-02T13:38:48.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 101730 0.74 MB/sec execute 9 sec latency 4944.766 ms 2024-04-02T13:38:49.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:49 smithi122 ceph-mon[32114]: pgmap v112: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 398 KiB/s wr, 35 op/s 2024-04-02T13:38:49.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:49 smithi187 ceph-mon[36064]: pgmap v112: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 398 KiB/s wr, 35 op/s 2024-04-02T13:38:49.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:49 smithi188 ceph-mon[36117]: pgmap v112: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 398 KiB/s wr, 35 op/s 2024-04-02T13:38:49.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 103370 1.89 MB/sec execute 10 sec latency 14.536 ms 2024-04-02T13:38:50.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 103576 1.73 MB/sec execute 11 sec latency 835.957 ms 2024-04-02T13:38:51.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:51 smithi122 ceph-mon[32114]: pgmap v113: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 305 KiB/s wr, 29 op/s 2024-04-02T13:38:51.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:51 smithi187 ceph-mon[36064]: pgmap v113: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 305 KiB/s wr, 29 op/s 2024-04-02T13:38:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:51 smithi188 ceph-mon[36117]: pgmap v113: 129 pgs: 129 active+clean; 70 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 305 KiB/s wr, 29 op/s 2024-04-02T13:38:51.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 103576 1.58 MB/sec execute 12 sec latency 1836.057 ms 2024-04-02T13:38:52.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 103576 1.46 MB/sec execute 13 sec latency 2836.130 ms 2024-04-02T13:38:53.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:53 smithi122 ceph-mon[32114]: pgmap v114: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 770 KiB/s wr, 84 op/s 2024-04-02T13:38:53.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:53 smithi187 ceph-mon[36064]: pgmap v114: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 770 KiB/s wr, 84 op/s 2024-04-02T13:38:53.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:53 smithi188 ceph-mon[36117]: pgmap v114: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 726 KiB/s rd, 770 KiB/s wr, 84 op/s 2024-04-02T13:38:53.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 104951 1.75 MB/sec execute 14 sec latency 3292.503 ms 2024-04-02T13:38:54.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 106905 2.53 MB/sec execute 15 sec latency 11.117 ms 2024-04-02T13:38:55.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:55 smithi122 ceph-mon[32114]: pgmap v115: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 724 KiB/s wr, 69 op/s 2024-04-02T13:38:55.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:55 smithi187 ceph-mon[36064]: pgmap v115: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 724 KiB/s wr, 69 op/s 2024-04-02T13:38:55.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:55 smithi188 ceph-mon[36117]: pgmap v115: 129 pgs: 129 active+clean; 70 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 724 KiB/s wr, 69 op/s 2024-04-02T13:38:55.947 INFO:tasks.workunit.client.0.smithi122.stdout: 1 107241 2.40 MB/sec execute 16 sec latency 705.867 ms 2024-04-02T13:38:56.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 107241 2.26 MB/sec execute 17 sec latency 1705.942 ms 2024-04-02T13:38:57.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:57 smithi122 ceph-mon[32114]: pgmap v116: 129 pgs: 129 active+clean; 72 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-02T13:38:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:57 smithi187 ceph-mon[36064]: pgmap v116: 129 pgs: 129 active+clean; 72 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-02T13:38:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:57 smithi188 ceph-mon[36117]: pgmap v116: 129 pgs: 129 active+clean; 72 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.2 MiB/s wr, 122 op/s 2024-04-02T13:38:57.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 107241 2.13 MB/sec execute 18 sec latency 2706.069 ms 2024-04-02T13:38:58.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 108880 2.38 MB/sec execute 19 sec latency 3157.575 ms 2024-04-02T13:38:59.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:38:59 smithi122 ceph-mon[32114]: pgmap v117: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:38:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:38:59 smithi187 ceph-mon[36064]: pgmap v117: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:38:59.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:38:59 smithi188 ceph-mon[36117]: pgmap v117: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:38:59.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 110543 2.86 MB/sec execute 20 sec latency 12.394 ms 2024-04-02T13:39:00.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 110723 2.73 MB/sec execute 21 sec latency 649.301 ms 2024-04-02T13:39:01.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:01 smithi122 ceph-mon[32114]: pgmap v118: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-04-02T13:39:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:01 smithi187 ceph-mon[36064]: pgmap v118: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-04-02T13:39:01.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:01 smithi188 ceph-mon[36117]: pgmap v118: 129 pgs: 129 active+clean; 72 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 114 op/s 2024-04-02T13:39:01.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 110723 2.61 MB/sec execute 22 sec latency 1217.810 ms 2024-04-02T13:39:02.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 110723 2.49 MB/sec execute 23 sec latency 2217.874 ms 2024-04-02T13:39:03.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:03 smithi122 ceph-mon[32114]: pgmap v119: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-02T13:39:03.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:03 smithi187 ceph-mon[36064]: pgmap v119: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-02T13:39:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:03 smithi188 ceph-mon[36117]: pgmap v119: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-04-02T13:39:03.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 111028 2.44 MB/sec execute 24 sec latency 2673.157 ms 2024-04-02T13:39:04.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:04.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:04.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.635+0000 7f51ca28f640 1 -- 172.21.15.122:0/4014338182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4100970 msgr2=0x7f51c4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.635+0000 7f51ca28f640 1 --2- 172.21.15.122:0/4014338182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4100970 0x7f51c4100df0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009930 tx=0x7f51b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- 172.21.15.122:0/4014338182 shutdown_connections 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- 172.21.15.122:0/4014338182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4101330 0x7f51c410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- 172.21.15.122:0/4014338182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4100970 0x7f51c4100df0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- 172.21.15.122:0/4014338182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 0x7f51c4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- 172.21.15.122:0/4014338182 >> 172.21.15.122:0/4014338182 conn(0x7f51c4077b00 msgr2=0x7f51c4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- 172.21.15.122:0/4014338182 shutdown_connections 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- 172.21.15.122:0/4014338182 wait complete. 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 Processor -- start 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- start start 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 0x7f51c41991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 0x7f51c41a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.636+0000 7f51ca28f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4110d90 con 0x7f51c4101330 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51ca28f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4110c10 con 0x7f51c4100970 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51ca28f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51c4110f10 con 0x7f51c4106980 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 0x7f51c41991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37918/0 (socket says 172.21.15.122:37918) 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c37fe640 1 -- 172.21.15.122:0/307343502 learned_addr learned my addr 172.21.15.122:0/307343502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c882c640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 0x7f51c41a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 -- 172.21.15.122:0/307343502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 msgr2=0x7f51c41a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 0x7f51c41a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 -- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 msgr2=0x7f51c4199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 -- 172.21.15.122:0/307343502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c41a47b0 con 0x7f51c4100970 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c37fe640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c3fff640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 0x7f51c41991f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51b400e9f0 tx=0x7f51b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c17fa640 1 -- 172.21.15.122:0/307343502 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b400ccc0 con 0x7f51c4100970 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c41a4980 con 0x7f51c4100970 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c17fa640 1 -- 172.21.15.122:0/307343502 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b4005020 con 0x7f51c4100970 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51c17fa640 1 -- 172.21.15.122:0/307343502 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40049e0 con 0x7f51c4100970 2024-04-02T13:39:04.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.637+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c41a4ea0 con 0x7f51c4100970 2024-04-02T13:39:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51c17fa640 1 -- 172.21.15.122:0/307343502 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f51b4005560 con 0x7f51c4100970 2024-04-02T13:39:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 Processor -- start 2024-04-02T13:39:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51c17fa640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 -- start start 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51c882c640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51c17fa640 1 -- 172.21.15.122:0/307343502 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f51b40d0a70 con 0x7f51c4100970 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x555688760890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 0x5556887631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5556886bb830 con 0x5556887578d0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555688663820 con 0x5556887581f0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.640+0000 7f51cce77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5556886b77e0 con 0x555688760dd0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37930/0 (socket says 172.21.15.122:37930) 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 -- 172.21.15.122:0/607878450 learned_addr learned my addr 172.21.15.122:0/607878450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c37fe640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x555688760890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c882c640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005fd0 tx=0x7f51bc009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c882c640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 0x5556887631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 -- 172.21.15.122:0/607878450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 msgr2=0x5556887631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 0x5556887631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 -- 172.21.15.122:0/607878450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 msgr2=0x555688760890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x555688760890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 -- 172.21.15.122:0/607878450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556884997d0 con 0x5556887578d0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c882c640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 0x5556887631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51c3fff640 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f51b40a4dc0 tx=0x7f51b4005290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb768c0939f0d943 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51aa7fc640 1 -- 172.21.15.122:0/607878450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b40b3030 con 0x5556887578d0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51aa7fc640 1 -- 172.21.15.122:0/607878450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b4087070 con 0x5556887578d0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.641+0000 7f51aa7fc640 1 -- 172.21.15.122:0/607878450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b401c040 con 0x5556887578d0 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- 172.21.15.122:0/607878450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 msgr2=0x555688757cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f51b40a4dc0 tx=0x7f51b4005290 comp rx=0 tx=0).stop 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- 172.21.15.122:0/607878450 shutdown_connections 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555688760dd0 0x5556887631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x555688760890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 --2- 172.21.15.122:0/607878450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5556887578d0 0x555688757cb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- 172.21.15.122:0/607878450 >> 172.21.15.122:0/607878450 conn(0x555688752000 msgr2=0x5556887523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- 172.21.15.122:0/607878450 shutdown_connections 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- 172.21.15.122:0/607878450 wait complete. 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 -- start start 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.642+0000 7f51cce77740 1 Processor -- start 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x555688760670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555688763bc0 con 0x555688760dd0 2024-04-02T13:39:04.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555688676d30 con 0x5556887581f0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555688763d40 con 0x5556887578d0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 10 client.0 _ll_register_callbacks cb 0x555688676720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37938/0 (socket says 172.21.15.122:37938) 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 -- 172.21.15.122:0/3246166722 learned_addr learned my addr 172.21.15.122:0/3246166722 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c3fff640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x555688760670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 -- 172.21.15.122:0/3246166722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 msgr2=0x555688760670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x555688760670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 -- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 msgr2=0x55568875da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 -- 172.21.15.122:0/3246166722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556883cf3f0 con 0x555688760dd0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51c882c640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00ed00 tx=0x7f51bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ab752514832c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc01de80 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f51bc003970 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b4087070 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f51bc003b70 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.643+0000 7f51cce77740 10 client.16827 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55568838f0f0 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b4087070 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f51bc01e5c0 con 0x555688760dd0 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 10 client.16827 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 20 client.16827 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555688770ee0 0x555688773340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 4 client.16827 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556886c8d70 con 0x555688770ee0 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51c3fff640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555688770ee0 0x555688773340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51a37fe640 10 client.16827 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.644+0000 7f51c3fff640 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555688770ee0 0x555688773340 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125cf3755f73d4f4 server_cookie=dcd187a80cffe334 in_seq=0 out_seq=0 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51a37fe640 1 -- 172.21.15.122:0/3246166722 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556886c8d70 con 0x555688770ee0 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51a37fe640 10 client.16827 handle_command_reply: tid=0 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 client.16827 shutdown 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 msgr2=0x55568875dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00ed00 tx=0x7f51bc004ab0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 shutdown_connections 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555688770ee0 0x555688773340 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875dfb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51a37fe640 0 client.16827 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:04.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 --2- 172.21.15.122:0/3246166722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x555688760670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.645+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 >> 172.21.15.122:0/3246166722 conn(0x555688752000 msgr2=0x555688753d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.646+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 shutdown_connections 2024-04-02T13:39:04.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.646+0000 7f51cce77740 1 -- 172.21.15.122:0/3246166722 wait complete. 2024-04-02T13:39:04.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.646+0000 7f51cce77740 20 client.16827 trim_cache size 0 max 16384 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 Processor -- start 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 -- start start 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887578d0 0x55568876ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887581f0 0x55568875d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41aefa0 con 0x555688760dd0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41b0c00 con 0x5556887578d0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51cce77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41af670 con 0x5556887581f0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37946/0 (socket says 172.21.15.122:37946) 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 -- 172.21.15.122:0/1307364198 learned_addr learned my addr 172.21.15.122:0/1307364198 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c3fff640 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887578d0 0x55568876ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c37fe640 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887581f0 0x55568875d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 -- 172.21.15.122:0/1307364198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887581f0 msgr2=0x55568875d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887581f0 0x55568875d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 -- 172.21.15.122:0/1307364198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887578d0 msgr2=0x55568876ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887578d0 0x55568876ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 -- 172.21.15.122:0/1307364198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555688751e30 con 0x555688760dd0 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.664+0000 7f51c882c640 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f51bc026420 tx=0x7f51bc01dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d2eff83fc6e5f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51a9ffb640 1 -- 172.21.15.122:0/1307364198 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc038020 con 0x555688760dd0 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51a9ffb640 1 -- 172.21.15.122:0/1307364198 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc005bd0 con 0x555688760dd0 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51a9ffb640 1 -- 172.21.15.122:0/1307364198 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc034800 con 0x555688760dd0 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 -- 172.21.15.122:0/1307364198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 msgr2=0x55568875f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f51bc026420 tx=0x7f51bc01dc90 comp rx=0 tx=0).stop 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 -- 172.21.15.122:0/1307364198 shutdown_connections 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875f8a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887581f0 0x55568875d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 --2- 172.21.15.122:0/1307364198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887578d0 0x55568876ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 -- 172.21.15.122:0/1307364198 >> 172.21.15.122:0/1307364198 conn(0x555688752000 msgr2=0x55568875f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 -- 172.21.15.122:0/1307364198 shutdown_connections 2024-04-02T13:39:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.665+0000 7f51cce77740 1 -- 172.21.15.122:0/1307364198 wait complete. 2024-04-02T13:39:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 -- start start 2024-04-02T13:39:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 Processor -- start 2024-04-02T13:39:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x55568875a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41b0470 con 0x555688760dd0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41afda0 con 0x5556887581f0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51c41acf80 con 0x5556887578d0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51cce77740 10 client.0 _ll_register_callbacks cb 0x7f51bc00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37954/0 (socket says 172.21.15.122:37954) 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 -- 172.21.15.122:0/1025566694 learned_addr learned my addr 172.21.15.122:0/1025566694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c3fff640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x55568875a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c37fe640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 -- 172.21.15.122:0/1025566694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 msgr2=0x55568875a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x55568875a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 -- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 msgr2=0x55568875ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.666+0000 7f51c882c640 1 -- 172.21.15.122:0/1025566694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556887d6480 con 0x555688760dd0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51c882c640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000f80 tx=0x7f51bc01ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d089c764c8f023 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc038030 con 0x555688760dd0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f51bc003970 con 0x555688760dd0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc005bd0 con 0x555688760dd0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f51bc04c050 con 0x555688760dd0 2024-04-02T13:39:04.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 10 client.16833 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5556887d7630 con 0x555688760dd0 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc005bd0 con 0x555688760dd0 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f51bc034aa0 con 0x555688760dd0 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 10 client.16833 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 20 client.16833 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5556887d7d50 0x5556887da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 4 client.16833 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.667+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556886c30c0 con 0x5556887d7d50 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.668+0000 7f51c3fff640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5556887d7d50 0x5556887da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.668+0000 7f51c3fff640 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5556887d7d50 0x5556887da170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ed711c0586b85b server_cookie=ee3e81725a52c91d in_seq=0 out_seq=0 2024-04-02T13:39:04.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.668+0000 7f51a37fe640 10 client.16833 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.669+0000 7f51a37fe640 1 -- 172.21.15.122:0/1025566694 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5556886c30c0 con 0x5556887d7d50 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.669+0000 7f51a37fe640 10 client.16833 handle_command_reply: tid=0 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.669+0000 7f51cce77740 1 client.16833 shutdown 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.669+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 msgr2=0x55568875b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.669+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000f80 tx=0x7f51bc01ee50 comp rx=0 tx=0).stop 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 shutdown_connections 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5556887d7d50 0x5556887da170 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555688760dd0 0x55568875b090 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5556887581f0 0x55568875ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51a37fe640 0 client.16833 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 --2- 172.21.15.122:0/1025566694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556887578d0 0x55568875a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 >> 172.21.15.122:0/1025566694 conn(0x555688752000 msgr2=0x555688758fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 shutdown_connections 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 1 -- 172.21.15.122:0/1025566694 wait complete. 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51cce77740 20 client.16833 trim_cache size 0 max 16384 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:39:04.682 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 msgr2=0x7f51ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51bc005fd0 tx=0x7f51bc009500 comp rx=0 tx=0).stop 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 msgr2=0x7f51c41991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 0x7f51c41991f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f51b400e9f0 tx=0x7f51b400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 shutdown_connections 2024-04-02T13:39:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f51ac0a3d20 0x7f51ac0a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c4106980 0x7f51c41a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51c4101330 0x7f51c4199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 --2- 172.21.15.122:0/307343502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51c4100970 0x7f51c41991f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 >> 172.21.15.122:0/307343502 conn(0x7f51c4077b00 msgr2=0x7f51c41031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 shutdown_connections 2024-04-02T13:39:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.670+0000 7f51ca28f640 1 -- 172.21.15.122:0/307343502 wait complete. 2024-04-02T13:39:04.685 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:39:04.685 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:39:04.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 -- 172.21.15.122:0/2837916106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 msgr2=0x7f796010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 --2- 172.21.15.122:0/2837916106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f796010d8d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7958009cd0 tx=0x7f795802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 -- 172.21.15.122:0/2837916106 shutdown_connections 2024-04-02T13:39:04.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 --2- 172.21.15.122:0/2837916106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f796010d8d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 --2- 172.21.15.122:0/2837916106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f7960100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 --2- 172.21.15.122:0/2837916106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 0x7f7960106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 -- 172.21.15.122:0/2837916106 >> 172.21.15.122:0/2837916106 conn(0x7f7960077b00 msgr2=0x7f7960077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 -- 172.21.15.122:0/2837916106 shutdown_connections 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.790+0000 7f79661d1640 1 -- 172.21.15.122:0/2837916106 wait complete. 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 Processor -- start 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 -- start start 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f79601996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 0x7f79601a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7960110e20 con 0x7f79601013c0 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7960110ca0 con 0x7f7960100a00 2024-04-02T13:39:04.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f79661d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7960110fa0 con 0x7f7960106a10 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f79601996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47194/0 (socket says 172.21.15.122:47194) 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 -- 172.21.15.122:0/2583034310 learned_addr learned my addr 172.21.15.122:0/2583034310 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795ffff640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 0x7f79601a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 -- 172.21.15.122:0/2583034310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 msgr2=0x7f79601a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 0x7f79601a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 -- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 msgr2=0x7f79601996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f79601996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.791+0000 7f795f7fe640 1 -- 172.21.15.122:0/2583034310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79601a4760 con 0x7f7960100a00 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f795f7fe640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f795000e9f0 tx=0x7f795000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795000cd80 con 0x7f7960100a00 2024-04-02T13:39:04.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601a4930 con 0x7f7960100a00 2024-04-02T13:39:04.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7950005500 con 0x7f7960100a00 2024-04-02T13:39:04.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950010690 con 0x7f7960100a00 2024-04-02T13:39:04.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.792+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79601a4e80 con 0x7f7960100a00 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.793+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79600688f0 con 0x7f7960100a00 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.794+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f79500046c0 con 0x7f7960100a00 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.795+0000 7f795cff9640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 0x7f79480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.795+0000 7f795effd640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 0x7f79480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.795+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f79500d0ef0 con 0x7f7960100a00 2024-04-02T13:39:04.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.795+0000 7f795effd640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 0x7f79480a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79600697d0 tx=0x7f7954005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:04.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:04.799+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f795009d200 con 0x7f7960100a00 2024-04-02T13:39:04.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 112876 2.73 MB/sec execute 25 sec latency 56.915 ms 2024-04-02T13:39:05.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.039+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7960108f70 con 0x7f7960100a00 2024-04-02T13:39:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.040+0000 7f795cff9640 1 -- 172.21.15.122:0/2583034310 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11952 (secure 0 0 0) 0x7f795009cb00 con 0x7f7960100a00 2024-04-02T13:39:05.043 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:39:05.043 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:37:34.559237+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:39:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 32 2024-04-02T13:39:05.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 msgr2=0x7f79480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 0x7f79480a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f79600697d0 tx=0x7f7954005f90 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 msgr2=0x7f79601991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f795000e9f0 tx=0x7f795000edf0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 shutdown_connections 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f79480a3d20 0x7f79480a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7960106a10 0x7f79601a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79601013c0 0x7f79601996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 --2- 172.21.15.122:0/2583034310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7960100a00 0x7f79601991a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 >> 172.21.15.122:0/2583034310 conn(0x7f7960077b00 msgr2=0x7f7960103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 shutdown_connections 2024-04-02T13:39:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.042+0000 7f79661d1640 1 -- 172.21.15.122:0/2583034310 wait complete. 2024-04-02T13:39:05.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:39:05.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 -- 172.21.15.122:0/3599676938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 msgr2=0x7ff81c106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 --2- 172.21.15.122:0/3599676938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c106e10 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff810009930 tx=0x7ff81002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:05.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 -- 172.21.15.122:0/3599676938 shutdown_connections 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 --2- 172.21.15.122:0/3599676938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 0x7ff81c10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 --2- 172.21.15.122:0/3599676938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 --2- 172.21.15.122:0/3599676938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c106e10 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 -- 172.21.15.122:0/3599676938 >> 172.21.15.122:0/3599676938 conn(0x7ff81c077b60 msgr2=0x7ff81c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 -- 172.21.15.122:0/3599676938 shutdown_connections 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.167+0000 7ff822fc8640 1 -- 172.21.15.122:0/3599676938 wait complete. 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 Processor -- start 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 -- start start 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 0x7ff81c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c110e40 con 0x7ff81c106a30 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c110cc0 con 0x7ff81c100a20 2024-04-02T13:39:05.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff822fc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c110fc0 con 0x7ff81c1013e0 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff820d64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37988/0 (socket says 172.21.15.122:37988) 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 -- 172.21.15.122:0/1005573778 learned_addr learned my addr 172.21.15.122:0/1005573778 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff820d64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47200/0 (socket says 172.21.15.122:47200) 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff81bfff640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 0x7ff81c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 -- 172.21.15.122:0/1005573778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 msgr2=0x7ff81c199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 0x7ff81c199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 -- 172.21.15.122:0/1005573778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 msgr2=0x7ff81c1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.168+0000 7ff821565640 1 -- 172.21.15.122:0/1005573778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff81c1a47a0 con 0x7ff81c106a30 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff821565640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff81400eaa0 tx=0x7ff81400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff819ffb640 1 -- 172.21.15.122:0/1005573778 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81400cdb0 con 0x7ff81c106a30 2024-04-02T13:39:05.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff819ffb640 1 -- 172.21.15.122:0/1005573778 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814005020 con 0x7ff81c106a30 2024-04-02T13:39:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff81c1a4970 con 0x7ff81c106a30 2024-04-02T13:39:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff819ffb640 1 -- 172.21.15.122:0/1005573778 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140116b0 con 0x7ff81c106a30 2024-04-02T13:39:05.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.169+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff81c1a4ec0 con 0x7ff81c106a30 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.171+0000 7ff825baa740 1 Processor -- start 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.171+0000 7ff819ffb640 1 -- 172.21.15.122:0/1005573778 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff814005560 con 0x7ff81c106a30 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 -- start start 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 0x559f60911760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f60914090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f6086c7b0 con 0x559f609087a0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f6080b570 con 0x559f609090c0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff825baa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f60868760 con 0x559f60911ca0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37994/0 (socket says 172.21.15.122:37994) 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 -- 172.21.15.122:0/4036161922 learned_addr learned my addr 172.21.15.122:0/4036161922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff820d64640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 0x559f60911760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff819ffb640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 0x7ff80c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f60914090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 -- 172.21.15.122:0/4036161922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 msgr2=0x559f60914090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f60914090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 -- 172.21.15.122:0/4036161922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 msgr2=0x559f60911760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 0x559f60911760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff81bfff640 1 -- 172.21.15.122:0/4036161922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f605803f0 con 0x559f609087a0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff820d64640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 0x559f60911760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff821565640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 0x7ff80c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.172+0000 7ff819ffb640 1 -- 172.21.15.122:0/1005573778 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8140d0d60 con 0x7ff81c106a30 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff81bfff640 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff808009fa0 tx=0x7ff8080097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81c87d651c4d18c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff806ffd640 1 -- 172.21.15.122:0/4036161922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808010490 con 0x559f609087a0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff806ffd640 1 -- 172.21.15.122:0/4036161922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff808010b10 con 0x559f609087a0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff806ffd640 1 -- 172.21.15.122:0/4036161922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff80800f9a0 con 0x559f609087a0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff821565640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 0x7ff80c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8140ce720 tx=0x7ff8140054d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff825baa740 1 -- 172.21.15.122:0/4036161922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 msgr2=0x559f60908b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.173+0000 7ff825baa740 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7ff808009fa0 tx=0x7ff8080097b0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- 172.21.15.122:0/4036161922 shutdown_connections 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f60914090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609090c0 0x559f60911760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- 172.21.15.122:0/4036161922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609087a0 0x559f60908b80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- 172.21.15.122:0/4036161922 >> 172.21.15.122:0/4036161922 conn(0x559f60902ed0 msgr2=0x559f609032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- 172.21.15.122:0/4036161922 shutdown_connections 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- 172.21.15.122:0/4036161922 wait complete. 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- start start 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 Processor -- start 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f60911760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f6090f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559f60914a90 con 0x559f609090c0 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559f60827e80 con 0x559f609087a0 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559f60914c10 con 0x559f60911ca0 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.174+0000 7ff825baa740 10 client.0 _ll_register_callbacks cb 0x559f60863aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38004/0 (socket says 172.21.15.122:38004) 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 -- 172.21.15.122:0/365189767 learned_addr learned my addr 172.21.15.122:0/365189767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff81bfff640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f60911760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff81bfff640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f6090f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 -- 172.21.15.122:0/365189767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 msgr2=0x559f6090f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f6090f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 -- 172.21.15.122:0/365189767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 msgr2=0x559f60911760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f60911760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 -- 172.21.15.122:0/365189767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f6064a7d0 con 0x559f609090c0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff820d64640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff810009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6e5f3b06a4c3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004240 con 0x559f609090c0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8100043e0 con 0x559f609090c0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff808010b10 con 0x559f609090c0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.175+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff8100046c0 con 0x559f609090c0 2024-04-02T13:39:05.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 10 client.16851 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f60920b70 con 0x559f609090c0 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff808010b10 con 0x559f609090c0 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff8100035f0 con 0x559f609090c0 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 10 client.16851 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 10 client.16851 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 20 client.16851 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60922030 0x559f60924460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 4 client.16851 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f60879cf0 con 0x559f60922030 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff81bfff640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60922030 0x559f60924460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff7fbfff640 10 client.16851 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.176+0000 7ff81bfff640 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60922030 0x559f60924460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44e121199d04eb1f server_cookie=2cc535e8db988193 in_seq=0 out_seq=0 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff7fbfff640 1 -- 172.21.15.122:0/365189767 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f60879cf0 con 0x559f60922030 2024-04-02T13:39:05.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff7fbfff640 10 client.16851 handle_command_reply: tid=0 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 client.16851 shutdown 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 msgr2=0x559f6090eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ff810036040 tx=0x7ff810009600 comp rx=0 tx=0).stop 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 shutdown_connections 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60922030 0x559f60924460 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f60911ca0 0x559f6090f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff7fbfff640 0 client.16851 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f6090eaf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 --2- 172.21.15.122:0/365189767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f60911760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 >> 172.21.15.122:0/365189767 conn(0x559f60902ed0 msgr2=0x559f60904c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 shutdown_connections 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.177+0000 7ff825baa740 1 -- 172.21.15.122:0/365189767 wait complete. 2024-04-02T13:39:05.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.178+0000 7ff825baa740 20 client.16851 trim_cache size 0 max 16384 2024-04-02T13:39:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 Processor -- start 2024-04-02T13:39:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 -- start start 2024-04-02T13:39:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 0x559f6091d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f60911ca0 0x559f60911760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1aeff0 con 0x559f609090c0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1b0c50 con 0x559f60911ca0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.191+0000 7ff825baa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1af6c0 con 0x559f609087a0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 0x559f6091d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38008/0 (socket says 172.21.15.122:38008) 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 -- 172.21.15.122:0/96966336 learned_addr learned my addr 172.21.15.122:0/96966336 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff81bfff640 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f60911ca0 0x559f60911760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 -- 172.21.15.122:0/96966336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 msgr2=0x559f6091d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 0x559f6091d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 -- 172.21.15.122:0/96966336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f60911ca0 msgr2=0x559f60911760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f60911ca0 0x559f60911760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 -- 172.21.15.122:0/96966336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6090f5a0 con 0x559f609090c0 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff81bfff640 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 0x559f6091d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff820d64640 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff810009930 tx=0x7ff810005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62781e88ee5a5a34 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff8067fc640 1 -- 172.21.15.122:0/96966336 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81005a070 con 0x559f609090c0 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff8067fc640 1 -- 172.21.15.122:0/96966336 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810004d40 con 0x559f609090c0 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.192+0000 7ff8067fc640 1 -- 172.21.15.122:0/96966336 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff810004300 con 0x559f609090c0 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- 172.21.15.122:0/96966336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 msgr2=0x559f60911220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff810009930 tx=0x7ff810005a20 comp rx=0 tx=0).stop 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- 172.21.15.122:0/96966336 shutdown_connections 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f60911ca0 0x559f60911760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f609090c0 0x559f60911220 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 --2- 172.21.15.122:0/96966336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609087a0 0x559f6091d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- 172.21.15.122:0/96966336 >> 172.21.15.122:0/96966336 conn(0x559f60902ed0 msgr2=0x559f6090d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- 172.21.15.122:0/96966336 shutdown_connections 2024-04-02T13:39:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- 172.21.15.122:0/96966336 wait complete. 2024-04-02T13:39:05.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 -- start start 2024-04-02T13:39:05.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.193+0000 7ff825baa740 1 Processor -- start 2024-04-02T13:39:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f6090b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609090c0 0x559f6090bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1b04c0 con 0x559f60911ca0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1afdf0 con 0x559f609087a0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff81c1acfd0 con 0x559f609090c0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff825baa740 10 client.0 _ll_register_callbacks cb 0x7ff810009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38020/0 (socket says 172.21.15.122:38020) 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 -- 172.21.15.122:0/1717442591 learned_addr learned my addr 172.21.15.122:0/1717442591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f6090b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 -- 172.21.15.122:0/1717442591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609090c0 msgr2=0x559f6090bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609090c0 0x559f6090bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 -- 172.21.15.122:0/1717442591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 msgr2=0x559f6090b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f6090b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 -- 172.21.15.122:0/1717442591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f609180e0 con 0x559f60911ca0 2024-04-02T13:39:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff81bfff640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff80800a010 tx=0x7ff81002ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c1e5fe6ad4002 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.194+0000 7ff7fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff808028040 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff8080052f0 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff810004d40 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff80803c050 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 10 client.16857 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559f609888c0 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff810004d40 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff80800f910 con 0x559f60911ca0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 10 client.16857 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 10 client.16857 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 20 client.16857 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60988e20 0x559f6098b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 4 client.16857 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559f60874040 con 0x559f60988e20 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.195+0000 7ff820d64640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60988e20 0x559f6098b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff7fbfff640 10 client.16857 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff820d64640 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60988e20 0x559f6098b240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ec1bab6868b9ca08 server_cookie=1f7e12309dc0b71f in_seq=0 out_seq=0 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff7fbfff640 1 -- 172.21.15.122:0/1717442591 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559f60874040 con 0x559f60988e20 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff7fbfff640 10 client.16857 handle_command_reply: tid=0 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 client.16857 shutdown 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 msgr2=0x559f6090c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff80800a010 tx=0x7ff81002ccd0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 shutdown_connections 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559f60988e20 0x559f6098b240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559f60911ca0 0x559f6090c2a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff7fbfff640 0 client.16857 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559f609090c0 0x559f6090bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 --2- 172.21.15.122:0/1717442591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559f609087a0 0x559f6090b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 >> 172.21.15.122:0/1717442591 conn(0x559f60902ed0 msgr2=0x559f60904300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 shutdown_connections 2024-04-02T13:39:05.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.196+0000 7ff825baa740 1 -- 172.21.15.122:0/1717442591 wait complete. 2024-04-02T13:39:05.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff825baa740 20 client.16857 trim_cache size 0 max 16384 2024-04-02T13:39:05.206 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 msgr2=0x7ff80c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 0x7ff80c0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8140ce720 tx=0x7ff8140054d0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 msgr2=0x7ff81c1a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7ff81400eaa0 tx=0x7ff81400ef60 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 shutdown_connections 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff80c0a3d20 0x7ff80c0a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff81c106a30 0x7ff81c1a4260 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff81c1013e0 0x7ff81c199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 --2- 172.21.15.122:0/1005573778 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff81c100a20 0x7ff81c1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 >> 172.21.15.122:0/1005573778 conn(0x7ff81c077b60 msgr2=0x7ff81c1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 shutdown_connections 2024-04-02T13:39:05.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.197+0000 7ff822fc8640 1 -- 172.21.15.122:0/1005573778 wait complete. 2024-04-02T13:39:05.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:39:05.319 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:05 smithi188 ceph-mon[36117]: pgmap v120: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-04-02T13:39:05.319 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2583034310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 -- 172.21.15.122:0/2745725841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 msgr2=0x7fed30106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 --2- 172.21.15.122:0/2745725841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed30106d60 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fed20009930 tx=0x7fed2002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 -- 172.21.15.122:0/2745725841 shutdown_connections 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 --2- 172.21.15.122:0/2745725841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed3010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 --2- 172.21.15.122:0/2745725841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 --2- 172.21.15.122:0/2745725841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed30106d60 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 -- 172.21.15.122:0/2745725841 >> 172.21.15.122:0/2745725841 conn(0x7fed30077b00 msgr2=0x7fed30077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 -- 172.21.15.122:0/2745725841 shutdown_connections 2024-04-02T13:39:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.316+0000 7fed35ed9640 1 -- 172.21.15.122:0/2745725841 wait complete. 2024-04-02T13:39:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 Processor -- start 2024-04-02T13:39:05.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:05 smithi122 ceph-mon[32114]: pgmap v120: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-04-02T13:39:05.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2583034310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 -- start start 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed301a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed30110d90 con 0x7fed30106980 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed30110c10 con 0x7fed30101330 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed35ed9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed30110f10 con 0x7fed30100970 2024-04-02T13:39:05.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47232/0 (socket says 172.21.15.122:47232) 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2effd640 1 -- 172.21.15.122:0/1510091171 learned_addr learned my addr 172.21.15.122:0/1510091171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:41524/0 (socket says 172.21.15.122:41524) 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.317+0000 7fed2ffff640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed301a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 -- 172.21.15.122:0/1510091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 msgr2=0x7fed30199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 -- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 msgr2=0x7fed301a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed301a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 -- 172.21.15.122:0/1510091171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed301a47d0 con 0x7fed30101330 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2ffff640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed301a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2effd640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed2400e9f0 tx=0x7fed2400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2cff9640 1 -- 172.21.15.122:0/1510091171 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2400cd80 con 0x7fed30101330 2024-04-02T13:39:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed301a49a0 con 0x7fed30101330 2024-04-02T13:39:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2cff9640 1 -- 172.21.15.122:0/1510091171 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24005500 con 0x7fed30101330 2024-04-02T13:39:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed2cff9640 1 -- 172.21.15.122:0/1510091171 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010690 con 0x7fed30101330 2024-04-02T13:39:05.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.318+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed301a4ec0 con 0x7fed30101330 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 Processor -- start 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed2cff9640 1 -- 172.21.15.122:0/1510091171 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fed240046c0 con 0x7fed30101330 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 -- start start 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83a7a0 0x56502a83ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a843760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56502a79e7b0 con 0x56502a843ca0 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56502a73d570 con 0x56502a83a7a0 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.321+0000 7fed38ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56502a79a760 con 0x56502a83b0c0 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38050/0 (socket says 172.21.15.122:38050) 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 -- 172.21.15.122:0/605894440 learned_addr learned my addr 172.21.15.122:0/605894440 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83a7a0 0x56502a83ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a843760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 -- 172.21.15.122:0/605894440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 msgr2=0x56502a843760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a843760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 -- 172.21.15.122:0/605894440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83a7a0 msgr2=0x56502a83ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83a7a0 0x56502a83ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 -- 172.21.15.122:0/605894440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502a4b23f0 con 0x56502a843ca0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.322+0000 7fed2effd640 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fed240ce2b0 tx=0x7fed240d91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7c533a4e8a62b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed15ffb640 1 -- 172.21.15.122:0/605894440 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240e4070 con 0x56502a843ca0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed15ffb640 1 -- 172.21.15.122:0/605894440 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed240d9b40 con 0x56502a843ca0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed15ffb640 1 -- 172.21.15.122:0/605894440 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240df400 con 0x56502a843ca0 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed2cff9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 0x7fed180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed2ffff640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 0x7fed180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.323+0000 7fed2cff9640 1 -- 172.21.15.122:0/1510091171 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed240d0f80 con 0x7fed30101330 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- 172.21.15.122:0/605894440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 msgr2=0x56502a846090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fed240ce2b0 tx=0x7fed240d91a0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed2ffff640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 0x7fed180a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed28009fd0 tx=0x7fed28009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- 172.21.15.122:0/605894440 shutdown_connections 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a843ca0 0x56502a846090 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a843760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 --2- 172.21.15.122:0/605894440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83a7a0 0x56502a83ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- 172.21.15.122:0/605894440 >> 172.21.15.122:0/605894440 conn(0x56502a834ed0 msgr2=0x56502a8352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- 172.21.15.122:0/605894440 shutdown_connections 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- 172.21.15.122:0/605894440 wait complete. 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.324+0000 7fed38ba0740 1 -- start start 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 Processor -- start 2024-04-02T13:39:05.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83a7a0 0x56502a843710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56502a84aea0 con 0x56502a83b0c0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56502a759e80 con 0x56502a843ca0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56502a84b020 con 0x56502a83a7a0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed38ba0740 10 client.0 _ll_register_callbacks cb 0x56502a795aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38054/0 (socket says 172.21.15.122:38054) 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 -- 172.21.15.122:0/3069669143 learned_addr learned my addr 172.21.15.122:0/3069669143 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83a7a0 0x56502a843710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2ffff640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 -- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83a7a0 msgr2=0x56502a843710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83a7a0 0x56502a843710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 -- 172.21.15.122:0/3069669143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 msgr2=0x56502a8411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.325+0000 7fed2effd640 1 -- 172.21.15.122:0/3069669143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502a57c7d0 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed2effd640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fed240ce3f0 tx=0x7fed24009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574256f92bb030ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24098e30 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed240d9b40 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed240bd0e0 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed2409c4f0 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 10 client.16869 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56502a84aea0 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed240d9b40 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fed24099200 con 0x56502a83b0c0 2024-04-02T13:39:05.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 10 client.16869 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 10 client.16869 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 20 client.16869 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8580b0 0x56502a85a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 4 client.16869 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.326+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56502a7abcf0 con 0x56502a8580b0 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.327+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8580b0 0x56502a85a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.327+0000 7fed0effd640 10 client.16869 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:05.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.327+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8580b0 0x56502a85a4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58ce3e5a3587b75a server_cookie=fa909cea6e1d2e70 in_seq=0 out_seq=0 2024-04-02T13:39:05.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed0effd640 1 -- 172.21.15.122:0/3069669143 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56502a7abcf0 con 0x56502a8580b0 2024-04-02T13:39:05.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed0effd640 10 client.16869 handle_command_reply: tid=0 2024-04-02T13:39:05.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 client.16869 shutdown 2024-04-02T13:39:05.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 msgr2=0x56502a840aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fed240ce3f0 tx=0x7fed24009310 comp rx=0 tx=0).stop 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 shutdown_connections 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8580b0 0x56502a85a4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83b0c0 0x56502a840aa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 --2- 172.21.15.122:0/3069669143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83a7a0 0x56502a843710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed0effd640 0 client.16869 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 >> 172.21.15.122:0/3069669143 conn(0x56502a834ed0 msgr2=0x56502a836c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 shutdown_connections 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 1 -- 172.21.15.122:0/3069669143 wait complete. 2024-04-02T13:39:05.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.328+0000 7fed38ba0740 20 client.16869 trim_cache size 0 max 16384 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 Processor -- start 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 -- start start 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83b0c0 0x56502a85a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a843ca0 0x56502a83fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed301acde0 con 0x56502a83a7a0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed301aea40 con 0x56502a83b0c0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed38ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed301ad4b0 con 0x56502a843ca0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38064/0 (socket says 172.21.15.122:38064) 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed2f7fe640 1 -- 172.21.15.122:0/1683249113 learned_addr learned my addr 172.21.15.122:0/1683249113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.342+0000 7fed2effd640 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83b0c0 0x56502a85a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2ffff640 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a843ca0 0x56502a83fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 -- 172.21.15.122:0/1683249113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a843ca0 msgr2=0x56502a83fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a843ca0 0x56502a83fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 -- 172.21.15.122:0/1683249113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83b0c0 msgr2=0x56502a85a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83b0c0 0x56502a85a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 -- 172.21.15.122:0/1683249113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502a846490 con 0x56502a83a7a0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed20004e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c5bd0370e43421 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed157fa640 1 -- 172.21.15.122:0/1683249113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2004c070 con 0x56502a83a7a0 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed157fa640 1 -- 172.21.15.122:0/1683249113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2002c880 con 0x56502a83a7a0 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.343+0000 7fed157fa640 1 -- 172.21.15.122:0/1683249113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed200487c0 con 0x56502a83a7a0 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- 172.21.15.122:0/1683249113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 msgr2=0x56502a853d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fed20009900 tx=0x7fed20004e70 comp rx=0 tx=0).stop 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- 172.21.15.122:0/1683249113 shutdown_connections 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a843ca0 0x56502a83fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a83b0c0 0x56502a85a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 --2- 172.21.15.122:0/1683249113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a853d60 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- 172.21.15.122:0/1683249113 >> 172.21.15.122:0/1683249113 conn(0x56502a834ed0 msgr2=0x56502a83f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- 172.21.15.122:0/1683249113 shutdown_connections 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- 172.21.15.122:0/1683249113 wait complete. 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 -- start start 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.344+0000 7fed38ba0740 1 Processor -- start 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a84ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8b6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed301ae2b0 con 0x56502a83a7a0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed301adbe0 con 0x56502a843ca0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed301aadc0 con 0x56502a83b0c0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 10 client.0 _ll_register_callbacks cb 0x7fed240e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38068/0 (socket says 172.21.15.122:38068) 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 -- 172.21.15.122:0/206272093 learned_addr learned my addr 172.21.15.122:0/206272093 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2effd640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a84ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2ffff640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8b6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 -- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 msgr2=0x56502a84ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a84ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 -- 172.21.15.122:0/206272093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 msgr2=0x56502a8b6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8b6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 -- 172.21.15.122:0/206272093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56502a8b6940 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed2f7fe640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fed20004650 tx=0x7fed20004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a744a3ce79acbdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed0effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20054030 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed20048440 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.345+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2002c880 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed20061050 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 10 client.16875 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56502a8bdcf0 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2002c880 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fed2005b4f0 con 0x56502a83a7a0 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 10 client.16875 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 10 client.16875 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 20 client.16875 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8be410 0x56502a8c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 4 client.16875 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56502a7a6040 con 0x56502a8be410 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.346+0000 7fed2effd640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8be410 0x56502a8c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed2effd640 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8be410 0x56502a8c0830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd55cae6c1a3fd3d server_cookie=ed0d67ce6202727c in_seq=0 out_seq=0 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed0effd640 10 client.16875 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed0effd640 1 -- 172.21.15.122:0/206272093 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56502a7a6040 con 0x56502a8be410 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed0effd640 10 client.16875 handle_command_reply: tid=0 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 client.16875 shutdown 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 msgr2=0x56502a84b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fed20004650 tx=0x7fed20004be0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 shutdown_connections 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56502a843ca0 0x56502a8b6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56502a83b0c0 0x56502a84ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56502a8be410 0x56502a8c0830 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 --2- 172.21.15.122:0/206272093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56502a83a7a0 0x56502a84b4c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 >> 172.21.15.122:0/206272093 conn(0x56502a834ed0 msgr2=0x56502a8336a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed0effd640 0 client.16875 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 shutdown_connections 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 1 -- 172.21.15.122:0/206272093 wait complete. 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.347+0000 7fed38ba0740 20 client.16875 trim_cache size 0 max 16384 2024-04-02T13:39:05.357 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 msgr2=0x7fed180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 0x7fed180a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed28009fd0 tx=0x7fed28009500 comp rx=0 tx=0).stop 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 msgr2=0x7fed30199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed2400e9f0 tx=0x7fed2400edf0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 shutdown_connections 2024-04-02T13:39:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed180a3d20 0x7fed180a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed30106980 0x7fed301a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed30101330 0x7fed30199750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 --2- 172.21.15.122:0/1510091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed30100970 0x7fed30199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 >> 172.21.15.122:0/1510091171 conn(0x7fed30077b00 msgr2=0x7fed301031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 shutdown_connections 2024-04-02T13:39:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.348+0000 7fed35ed9640 1 -- 172.21.15.122:0/1510091171 wait complete. 2024-04-02T13:39:05.362 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 -- 172.21.15.122:0/2291257419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8106980 msgr2=0x7f62b8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 --2- 172.21.15.122:0/2291257419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8106980 0x7f62b8106d60 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009930 tx=0x7f62a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 -- 172.21.15.122:0/2291257419 shutdown_connections 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 --2- 172.21.15.122:0/2291257419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8101330 0x7f62b810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 --2- 172.21.15.122:0/2291257419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8100970 0x7f62b8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 --2- 172.21.15.122:0/2291257419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8106980 0x7f62b8106d60 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 -- 172.21.15.122:0/2291257419 >> 172.21.15.122:0/2291257419 conn(0x7f62b8077b00 msgr2=0x7f62b8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 -- 172.21.15.122:0/2291257419 shutdown_connections 2024-04-02T13:39:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.480+0000 7f62be01b640 1 -- 172.21.15.122:0/2291257419 wait complete. 2024-04-02T13:39:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 Processor -- start 2024-04-02T13:39:05.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 -- start start 2024-04-02T13:39:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:05 smithi187 ceph-mon[36064]: pgmap v120: 129 pgs: 129 active+clean; 75 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-04-02T13:39:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2583034310' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 0x7f62b81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 0x7f62b8197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 0x7f62b819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8110d60 con 0x7f62b8101330 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8110be0 con 0x7f62b8100970 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62be01b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8110ee0 con 0x7f62b8106980 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 0x7f62b8197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 0x7f62b81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.481+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 0x7f62b81972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47294/0 (socket says 172.21.15.122:47294) 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 0x7f62b819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b77fe640 1 -- 172.21.15.122:0/824046300 learned_addr learned my addr 172.21.15.122:0/824046300 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 -- 172.21.15.122:0/824046300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 msgr2=0x7f62b81972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 0x7f62b81972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 -- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 msgr2=0x7f62b8197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 0x7f62b8197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 -- 172.21.15.122:0/824046300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b819c2b0 con 0x7f62b8106980 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 0x7f62b8197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 0x7f62b819bb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009fd0 tx=0x7f62b000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b4ff9640 1 -- 172.21.15.122:0/824046300 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000ce20 con 0x7f62b8106980 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b819c5a0 con 0x7f62b8106980 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b8068c60 con 0x7f62b8106980 2024-04-02T13:39:05.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b4ff9640 1 -- 172.21.15.122:0/824046300 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b0010430 con 0x7f62b8106980 2024-04-02T13:39:05.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.482+0000 7f62b4ff9640 1 -- 172.21.15.122:0/824046300 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000f620 con 0x7f62b8106980 2024-04-02T13:39:05.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b4ff9640 1 -- 172.21.15.122:0/824046300 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f62b00105e0 con 0x7f62b8106980 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 Processor -- start 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 -- start start 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cfb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593e1c537b0 con 0x5593e1cf00c0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593e1bf2570 con 0x5593e1cf8ca0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62c0c02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593e1c4f760 con 0x5593e1cef7a0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b4ff9640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 0x7f62a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38096/0 (socket says 172.21.15.122:38096) 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b77fe640 1 -- 172.21.15.122:0/2756644436 learned_addr learned my addr 172.21.15.122:0/2756644436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 0x7f62a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cfb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b4ff9640 1 -- 172.21.15.122:0/824046300 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f62b00d0d70 con 0x7f62b8106980 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2756644436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 msgr2=0x5593e1cfb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cfb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2756644436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 msgr2=0x5593e1cf8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.485+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2756644436 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e19673f0 con 0x5593e1cef7a0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f62b77fe640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f62b7fff640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 0x7f62a00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62b0098de0 tx=0x7f62b00bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cefb80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62b8105060 tx=0x7f62ac005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3ed7a7e687bc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f629dffb640 1 -- 172.21.15.122:0/2756644436 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac01ad60 con 0x5593e1cef7a0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f629dffb640 1 -- 172.21.15.122:0/2756644436 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac005500 con 0x5593e1cef7a0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f629dffb640 1 -- 172.21.15.122:0/2756644436 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac01e460 con 0x5593e1cef7a0 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f62c0c02740 1 -- 172.21.15.122:0/2756644436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 msgr2=0x5593e1cefb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.486+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cefb80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f62b8105060 tx=0x7f62ac005850 comp rx=0 tx=0).stop 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- 172.21.15.122:0/2756644436 shutdown_connections 2024-04-02T13:39:05.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cfb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1cf8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2756644436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cefb80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- 172.21.15.122:0/2756644436 >> 172.21.15.122:0/2756644436 conn(0x5593e1ce9ed0 msgr2=0x5593e1cea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- 172.21.15.122:0/2756644436 shutdown_connections 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- 172.21.15.122:0/2756644436 wait complete. 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- start start 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 Processor -- start 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593e1cfba30 con 0x5593e1cef7a0 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593e1c0ee80 con 0x5593e1cf8ca0 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593e1cfbbb0 con 0x5593e1cf00c0 2024-04-02T13:39:05.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62c0c02740 10 client.0 _ll_register_callbacks cb 0x5593e1c4aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.487+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38106/0 (socket says 172.21.15.122:38106) 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 -- 172.21.15.122:0/1422602996 learned_addr learned my addr 172.21.15.122:0/1422602996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 -- 172.21.15.122:0/1422602996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 msgr2=0x5593e1cf5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 -- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 msgr2=0x5593e1cf6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 -- 172.21.15.122:0/1422602996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e1a317d0 con 0x5593e1cef7a0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b77fe640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005940 tx=0x7f62ac004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4544cfe43c849d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f6296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac01ab60 con 0x5593e1cef7a0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f62ac01ad00 con 0x5593e1cef7a0 2024-04-02T13:39:05.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62ac005500 con 0x5593e1cef7a0 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62ac038050 con 0x5593e1cef7a0 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62c0c02740 10 client.16887 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593e1d07af0 con 0x5593e1cef7a0 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.488+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62ac005500 con 0x5593e1cef7a0 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f62ac01e9c0 con 0x5593e1cef7a0 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 10 client.16887 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 10 client.16887 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 20 client.16887 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d08fb0 0x5593e1d0b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 4 client.16887 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:05.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593e1c60cf0 con 0x5593e1d08fb0 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62b77fe640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d08fb0 0x5593e1d0b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f62b77fe640 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d08fb0 0x5593e1d0b3a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79ca6c5ac9f7a87a server_cookie=27dd078da0c24397 in_seq=0 out_seq=0 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.489+0000 7f6296ffd640 10 client.16887 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f6296ffd640 1 -- 172.21.15.122:0/1422602996 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593e1c60cf0 con 0x5593e1d08fb0 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f6296ffd640 10 client.16887 handle_command_reply: tid=0 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 client.16887 shutdown 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 msgr2=0x5593e1cf86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f62ac005940 tx=0x7f62ac004c10 comp rx=0 tx=0).stop 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 shutdown_connections 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d08fb0 0x5593e1d0b3a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 --2- 172.21.15.122:0/1422602996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf86c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f6296ffd640 0 client.16887 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:05.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 >> 172.21.15.122:0/1422602996 conn(0x5593e1ce9ed0 msgr2=0x5593e1cebc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 shutdown_connections 2024-04-02T13:39:05.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 1 -- 172.21.15.122:0/1422602996 wait complete. 2024-04-02T13:39:05.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.490+0000 7f62c0c02740 20 client.16887 trim_cache size 0 max 16384 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 Processor -- start 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 -- start start 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cf04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62a400fee0 con 0x5593e1cf00c0 2024-04-02T13:39:05.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4011b40 con 0x5593e1cf8ca0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62c0c02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62a40105b0 con 0x5593e1cef7a0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38116/0 (socket says 172.21.15.122:38116) 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 -- 172.21.15.122:0/3515406941 learned_addr learned my addr 172.21.15.122:0/3515406941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cf04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 -- 172.21.15.122:0/3515406941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 msgr2=0x5593e1cf04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cf04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 -- 172.21.15.122:0/3515406941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 msgr2=0x5593e1cf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.504+0000 7f62b77fe640 1 -- 172.21.15.122:0/3515406941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593e1cf63d0 con 0x5593e1cf00c0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.505+0000 7f62b77fe640 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009900 tx=0x7f62a80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5481563a9cacffd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.505+0000 7f629d7fa640 1 -- 172.21.15.122:0/3515406941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8004040 con 0x5593e1cf00c0 2024-04-02T13:39:05.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.505+0000 7f629d7fa640 1 -- 172.21.15.122:0/3515406941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004ba0 con 0x5593e1cf00c0 2024-04-02T13:39:05.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.505+0000 7f629d7fa640 1 -- 172.21.15.122:0/3515406941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8005400 con 0x5593e1cf00c0 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.505+0000 7f62c0c02740 1 -- 172.21.15.122:0/3515406941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 msgr2=0x5593e1d04980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009900 tx=0x7f62a80058d0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- 172.21.15.122:0/3515406941 shutdown_connections 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cf00c0 0x5593e1d04980 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- 172.21.15.122:0/3515406941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cef7a0 0x5593e1cf04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- 172.21.15.122:0/3515406941 >> 172.21.15.122:0/3515406941 conn(0x5593e1ce9ed0 msgr2=0x5593e1cf4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- 172.21.15.122:0/3515406941 shutdown_connections 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- 172.21.15.122:0/3515406941 wait complete. 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- start start 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 Processor -- start 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62a40113b0 con 0x5593e1cef7a0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62a4010ce0 con 0x5593e1cf8ca0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62a400dec0 con 0x5593e1cf00c0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62c0c02740 10 client.0 _ll_register_callbacks cb 0x7f62ac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38128/0 (socket says 172.21.15.122:38128) 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2220809911 learned_addr learned my addr 172.21.15.122:0/2220809911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.506+0000 7f62b77fe640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2220809911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 msgr2=0x5593e1cf97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 msgr2=0x5593e1cf9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 -- 172.21.15.122:0/2220809911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593e1d00150 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62b6ffd640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009250 tx=0x7f62ac004c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8ba0cb71a73dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac01ab30 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f62ac01acd0 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a8004ba0 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f62ac004020 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 10 client.16893 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593e1d01450 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62a8004ba0 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f62ac004230 con 0x5593e1cef7a0 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 10 client.16893 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 10 client.16893 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 20 client.16893 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d6f470 0x5593e1d71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 4 client.16893 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.507+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593e1c5b040 con 0x5593e1d6f470 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f62b77fe640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d6f470 0x5593e1d71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f6296ffd640 10 client.16893 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f62b77fe640 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d6f470 0x5593e1d71890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9287a673c592389 server_cookie=406b9f48b19f6d22 in_seq=0 out_seq=0 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f6296ffd640 1 -- 172.21.15.122:0/2220809911 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593e1c5b040 con 0x5593e1d6f470 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f6296ffd640 10 client.16893 handle_command_reply: tid=0 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f62c0c02740 1 client.16893 shutdown 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 msgr2=0x5593e1cf9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.508+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f62ac009250 tx=0x7f62ac004c10 comp rx=0 tx=0).stop 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 shutdown_connections 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5593e1d6f470 0x5593e1d71890 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593e1cf8ca0 0x5593e1cf9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593e1cf00c0 0x5593e1cf97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f6296ffd640 0 client.16893 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:05.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 --2- 172.21.15.122:0/2220809911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593e1cef7a0 0x5593e1cf9260 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 >> 172.21.15.122:0/2220809911 conn(0x5593e1ce9ed0 msgr2=0x5593e1ce86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 shutdown_connections 2024-04-02T13:39:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 1 -- 172.21.15.122:0/2220809911 wait complete. 2024-04-02T13:39:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62c0c02740 20 client.16893 trim_cache size 0 max 16384 2024-04-02T13:39:05.519 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 msgr2=0x7f62a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 0x7f62a00a61e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f62b0098de0 tx=0x7f62b00bd040 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 msgr2=0x7f62b819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 0x7f62b819bb80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009fd0 tx=0x7f62b000ef10 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 shutdown_connections 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f62a00a3d20 0x7f62a00a61e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b8106980 0x7f62b819bb80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62b8101330 0x7f62b8197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 --2- 172.21.15.122:0/824046300 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8100970 0x7f62b81972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 >> 172.21.15.122:0/824046300 conn(0x7f62b8077b00 msgr2=0x7f62b8103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 shutdown_connections 2024-04-02T13:39:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.509+0000 7f62be01b640 1 -- 172.21.15.122:0/824046300 wait complete. 2024-04-02T13:39:05.523 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:39:05.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- 172.21.15.122:0/896252093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c100670 msgr2=0x7fab2c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 --2- 172.21.15.122:0/896252093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c100670 0x7fab2c100a50 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009900 tx=0x7fab1c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:05.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- 172.21.15.122:0/896252093 shutdown_connections 2024-04-02T13:39:05.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 --2- 172.21.15.122:0/896252093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 --2- 172.21.15.122:0/896252093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c101020 0x7fab2c106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 --2- 172.21.15.122:0/896252093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c100670 0x7fab2c100a50 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- 172.21.15.122:0/896252093 >> 172.21.15.122:0/896252093 conn(0x7fab2c075c60 msgr2=0x7fab2c076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- 172.21.15.122:0/896252093 shutdown_connections 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- 172.21.15.122:0/896252093 wait complete. 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 Processor -- start 2024-04-02T13:39:05.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.631+0000 7fab31ed3640 1 -- start start 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 0x7fab2c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c111f20 con 0x7fab2c19cd20 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c111da0 con 0x7fab2c106e30 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab31ed3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1120a0 con 0x7fab2c100670 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 0x7fab2c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:41626/0 (socket says 172.21.15.122:41626) 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2b7fe640 1 -- 172.21.15.122:0/2550307227 learned_addr learned my addr 172.21.15.122:0/2550307227 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 -- 172.21.15.122:0/2550307227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 msgr2=0x7fab2c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 -- 172.21.15.122:0/2550307227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 msgr2=0x7fab2c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 -- 172.21.15.122:0/2550307227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c19d810 con 0x7fab2c19cd20 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2affd640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.632+0000 7fab2bfff640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 0x7fab2c19d1d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fab2400da10 tx=0x7fab2400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.633+0000 7fab28ff9640 1 -- 172.21.15.122:0/2550307227 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2400bd70 con 0x7fab2c19cd20 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.633+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c19db00 con 0x7fab2c19cd20 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.633+0000 7fab28ff9640 1 -- 172.21.15.122:0/2550307227 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24003e50 con 0x7fab2c19cd20 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.633+0000 7fab28ff9640 1 -- 172.21.15.122:0/2550307227 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240106a0 con 0x7fab2c19cd20 2024-04-02T13:39:05.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.633+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c1a5270 con 0x7fab2c19cd20 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab28ff9640 1 -- 172.21.15.122:0/2550307227 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fab24003970 con 0x7fab2c19cd20 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 Processor -- start 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 -- start start 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837c7a0 0x56135837cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 0x561358388090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613582e07b0 con 0x56135837d0c0 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56135827f570 con 0x561358385ca0 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.635+0000 7fab34b8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5613582dc760 con 0x56135837c7a0 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab28ff9640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 0x7fab140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38156/0 (socket says 172.21.15.122:38156) 2024-04-02T13:39:05.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2b7fe640 1 -- 172.21.15.122:0/3910798336 learned_addr learned my addr 172.21.15.122:0/3910798336 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 0x561358388090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837c7a0 0x56135837cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2bfff640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 0x7fab140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 -- 172.21.15.122:0/3910798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837c7a0 msgr2=0x56135837cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837c7a0 0x56135837cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 -- 172.21.15.122:0/3910798336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 msgr2=0x561358385760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 -- 172.21.15.122:0/3910798336 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561357ff43f0 con 0x561358385ca0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab28ff9640 1 -- 172.21.15.122:0/2550307227 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fab240d0ee0 con 0x7fab2c19cd20 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2affd640 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 0x561358388090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ffff0 tx=0x7fab20005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42395ac8fe0a63a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab2bfff640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 0x7fab140a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab24099460 tx=0x7fab240c7ba0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab11ffb640 1 -- 172.21.15.122:0/3910798336 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20005020 con 0x561358385ca0 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab11ffb640 1 -- 172.21.15.122:0/3910798336 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2000f850 con 0x561358385ca0 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.636+0000 7fab11ffb640 1 -- 172.21.15.122:0/3910798336 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20004b30 con 0x561358385ca0 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 -- 172.21.15.122:0/3910798336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 msgr2=0x561358388090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 0x561358388090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0ffff0 tx=0x7fab20005850 comp rx=0 tx=0).stop 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 -- 172.21.15.122:0/3910798336 shutdown_connections 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561358385ca0 0x561358388090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358385760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3910798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837c7a0 0x56135837cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 -- 172.21.15.122:0/3910798336 >> 172.21.15.122:0/3910798336 conn(0x561358376ed0 msgr2=0x5613583772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 -- 172.21.15.122:0/3910798336 shutdown_connections 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.637+0000 7fab34b8d740 1 -- 172.21.15.122:0/3910798336 wait complete. 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 -- start start 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 Processor -- start 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x561358382bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x5613583830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561358388a90 con 0x561358385ca0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56135829be80 con 0x56135837c7a0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561358388c10 con 0x56135837d0c0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab34b8d740 10 client.0 _ll_register_callbacks cb 0x5613582d7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38172/0 (socket says 172.21.15.122:38172) 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 -- 172.21.15.122:0/910194974 learned_addr learned my addr 172.21.15.122:0/910194974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x5613583830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x561358382bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 -- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 msgr2=0x5613583830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x5613583830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 -- 172.21.15.122:0/910194974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 msgr2=0x561358382bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x561358382bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.638+0000 7fab2affd640 1 -- 172.21.15.122:0/910194974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613580be7d0 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab2affd640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fab20009250 tx=0x7fab20005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80869374845d8c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2000fce0 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fab20004030 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2000f850 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab20032050 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 10 client.16905 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561358394c00 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2000f850 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fab20004df0 con 0x561358385ca0 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 10 client.16905 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 10 client.16905 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 20 client.16905 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x561358395f50 0x561358398340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 4 client.16905 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:05.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.639+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613582edcf0 con 0x561358395f50 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.640+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x561358395f50 0x561358398340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.640+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x561358395f50 0x561358398340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96bcd16531bdde01 server_cookie=b26f187deda852a4 in_seq=0 out_seq=0 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.640+0000 7fab0affd640 10 client.16905 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab0affd640 1 -- 172.21.15.122:0/910194974 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613582edcf0 con 0x561358395f50 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab0affd640 10 client.16905 handle_command_reply: tid=0 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 client.16905 shutdown 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 msgr2=0x561358383630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fab20009250 tx=0x7fab20005f90 comp rx=0 tx=0).stop 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 shutdown_connections 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x561358395f50 0x561358398340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x561358383630 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x5613583830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab0affd640 0 client.16905 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 --2- 172.21.15.122:0/910194974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x561358382bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 >> 172.21.15.122:0/910194974 conn(0x561358376ed0 msgr2=0x561358378c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 shutdown_connections 2024-04-02T13:39:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 1 -- 172.21.15.122:0/910194974 wait complete. 2024-04-02T13:39:05.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.641+0000 7fab34b8d740 20 client.16905 trim_cache size 0 max 16384 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 Processor -- start 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 -- start start 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x5613583852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561358385ca0 0x561358382110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1aef60 con 0x56135837d0c0 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b0bc0 con 0x56135837c7a0 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab34b8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1af630 con 0x561358385ca0 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38180/0 (socket says 172.21.15.122:38180) 2024-04-02T13:39:05.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab2b7fe640 1 -- 172.21.15.122:0/2171310801 learned_addr learned my addr 172.21.15.122:0/2171310801 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab2affd640 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x5613583852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.655+0000 7fab2affd640 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561358385ca0 0x561358382110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 -- 172.21.15.122:0/2171310801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561358385ca0 msgr2=0x561358382110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561358385ca0 0x561358382110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 -- 172.21.15.122:0/2171310801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 msgr2=0x5613583852e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x5613583852e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 -- 172.21.15.122:0/2171310801 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56135837cef0 con 0x56135837d0c0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fab1c036040 tx=0x7fab1c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534e6453b20116af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab117fa640 1 -- 172.21.15.122:0/2171310801 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c04c070 con 0x56135837d0c0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab117fa640 1 -- 172.21.15.122:0/2171310801 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c02c880 con 0x56135837d0c0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.656+0000 7fab117fa640 1 -- 172.21.15.122:0/2171310801 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c004f00 con 0x56135837d0c0 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- 172.21.15.122:0/2171310801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 msgr2=0x561358391950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fab1c036040 tx=0x7fab1c0058d0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- 172.21.15.122:0/2171310801 shutdown_connections 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561358385ca0 0x561358382110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56135837d0c0 0x561358391950 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- 172.21.15.122:0/2171310801 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x5613583852e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- 172.21.15.122:0/2171310801 >> 172.21.15.122:0/2171310801 conn(0x561358376ed0 msgr2=0x561358375f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- 172.21.15.122:0/2171310801 shutdown_connections 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- 172.21.15.122:0/2171310801 wait complete. 2024-04-02T13:39:05.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- start start 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 Processor -- start 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x56135838e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x561358380310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1b0430 con 0x561358385ca0 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1afd60 con 0x56135837c7a0 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c1acf40 con 0x56135837d0c0 2024-04-02T13:39:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.657+0000 7fab34b8d740 10 client.0 _ll_register_callbacks cb 0x7fab1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x561358380310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38182/0 (socket says 172.21.15.122:38182) 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 -- 172.21.15.122:0/3954652865 learned_addr learned my addr 172.21.15.122:0/3954652865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x56135838e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 -- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 msgr2=0x561358380310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x561358380310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 -- 172.21.15.122:0/3954652865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 msgr2=0x56135838e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x56135838e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 -- 172.21.15.122:0/3954652865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56135837f110 con 0x561358385ca0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab2affd640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fab20011960 tx=0x7fab200114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df8369a0de4f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab0affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab20004570 con 0x561358385ca0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fab20004c00 con 0x561358385ca0 2024-04-02T13:39:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab1c02c880 con 0x561358385ca0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab200055f0 con 0x561358385ca0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab34b8d740 10 client.16911 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5613583fb960 con 0x561358385ca0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.658+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab1c02c880 con 0x561358385ca0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fab20036bb0 con 0x561358385ca0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 10 client.16911 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 10 client.16911 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 20 client.16911 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613583fc020 0x5613583fe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 4 client.16911 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613582e8040 con 0x5613583fc020 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613583fc020 0x5613583fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab0affd640 10 client.16911 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab2b7fe640 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613583fc020 0x5613583fe410 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dcedcdc80d0e8891 server_cookie=b0647c57e8afbb90 in_seq=0 out_seq=0 2024-04-02T13:39:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab0affd640 1 -- 172.21.15.122:0/3954652865 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613582e8040 con 0x5613583fc020 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab0affd640 10 client.16911 handle_command_reply: tid=0 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.659+0000 7fab34b8d740 1 client.16911 shutdown 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 msgr2=0x5613583808d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fab20011960 tx=0x7fab200114d0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 shutdown_connections 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613583fc020 0x5613583fe410 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561358385ca0 0x5613583808d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56135837d0c0 0x561358380310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 --2- 172.21.15.122:0/3954652865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56135837c7a0 0x56135838e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab0affd640 0 client.16911 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 >> 172.21.15.122:0/3954652865 conn(0x561358376ed0 msgr2=0x5613583756a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 shutdown_connections 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 1 -- 172.21.15.122:0/3954652865 wait complete. 2024-04-02T13:39:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab34b8d740 20 client.16911 trim_cache size 0 max 16384 2024-04-02T13:39:05.670 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 msgr2=0x7fab140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 0x7fab140a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fab24099460 tx=0x7fab240c7ba0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 msgr2=0x7fab2c19d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 0x7fab2c19d1d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fab2400da10 tx=0x7fab2400ded0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 shutdown_connections 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fab140a3d20 0x7fab140a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab2c19cd20 0x7fab2c19d1d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fab2c106e30 0x7fab2c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 --2- 172.21.15.122:0/2550307227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab2c100670 0x7fab2c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.660+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 >> 172.21.15.122:0/2550307227 conn(0x7fab2c075c60 msgr2=0x7fab2c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.661+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 shutdown_connections 2024-04-02T13:39:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.661+0000 7fab31ed3640 1 -- 172.21.15.122:0/2550307227 wait complete. 2024-04-02T13:39:05.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:39:05.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.782+0000 7f03237fe640 1 -- 172.21.15.122:0/3679760979 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314032990 con 0x7f0324101aa0 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.782+0000 7f032b093640 1 -- 172.21.15.122:0/3679760979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 msgr2=0x7f0324101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.782+0000 7f032b093640 1 --2- 172.21.15.122:0/3679760979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324101f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0314009ca0 tx=0x7f031402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 -- 172.21.15.122:0/3679760979 shutdown_connections 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 --2- 172.21.15.122:0/3679760979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f032410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 --2- 172.21.15.122:0/3679760979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324101f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 --2- 172.21.15.122:0/3679760979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 0x7f0324107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 -- 172.21.15.122:0/3679760979 >> 172.21.15.122:0/3679760979 conn(0x7f03240fd530 msgr2=0x7f03240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 -- 172.21.15.122:0/3679760979 shutdown_connections 2024-04-02T13:39:05.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 -- 172.21.15.122:0/3679760979 wait complete. 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 Processor -- start 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.783+0000 7f032b093640 1 -- start start 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f0324071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 0x7f03241a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f032406b9d0 con 0x7f0324102460 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f032406b850 con 0x7f0324101aa0 2024-04-02T13:39:05.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f032b093640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f032406bb50 con 0x7f0324107ab0 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f0323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f0324071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f0323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f0324071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38200/0 (socket says 172.21.15.122:38200) 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f0323fff640 1 -- 172.21.15.122:0/1580180849 learned_addr learned my addr 172.21.15.122:0/1580180849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f0329630640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 0x7f03241a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.784+0000 7f0328e2f640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 -- 172.21.15.122:0/1580180849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 msgr2=0x7f03241a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 0x7f03241a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 -- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 msgr2=0x7f0324071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f0324071c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 -- 172.21.15.122:0/1580180849 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03241a7eb0 con 0x7f0324101aa0 2024-04-02T13:39:05.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0328e2f640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324071740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e9f0 tx=0x7f030c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f0321ffb640 1 -- 172.21.15.122:0/1580180849 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00ccc0 con 0x7f0324101aa0 2024-04-02T13:39:05.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.785+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03241a81a0 con 0x7f0324101aa0 2024-04-02T13:39:05.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.786+0000 7f0321ffb640 1 -- 172.21.15.122:0/1580180849 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c0035f0 con 0x7f0324101aa0 2024-04-02T13:39:05.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.786+0000 7f0321ffb640 1 -- 172.21.15.122:0/1580180849 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c00ce60 con 0x7f0324101aa0 2024-04-02T13:39:05.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.786+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032406cf30 con 0x7f0324101aa0 2024-04-02T13:39:05.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.789+0000 7f0321ffb640 1 -- 172.21.15.122:0/1580180849 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f030c0037a0 con 0x7f0324101aa0 2024-04-02T13:39:05.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.789+0000 7f032d922740 1 Processor -- start 2024-04-02T13:39:05.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.789+0000 7f032d922740 1 -- start start 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7f99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620e7ef47b0 con 0x5620e7f907a0 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5620e7e93570 con 0x5620e7f910c0 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f032d922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5620e7ef0760 con 0x5620e7f99ca0 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f0321ffb640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 0x7f03100a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f0323fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f0323fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:41682/0 (socket says 172.21.15.122:41682) 2024-04-02T13:39:05.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.790+0000 7f0323fff640 1 -- 172.21.15.122:0/1916881761 learned_addr learned my addr 172.21.15.122:0/1916881761 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0321ffb640 1 -- 172.21.15.122:0/1580180849 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f030c0d0d40 con 0x7f0324101aa0 2024-04-02T13:39:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0328e2f640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7f99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0329630640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 0x7f03100a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0329630640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0323fff640 1 -- 172.21.15.122:0/1916881761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 msgr2=0x5620e7f99760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0323fff640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7f99760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0323fff640 1 -- 172.21.15.122:0/1916881761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 msgr2=0x5620e7f90b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.791+0000 7f0323fff640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f90b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.792+0000 7f0323fff640 1 -- 172.21.15.122:0/1916881761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e7c083f0 con 0x5620e7f99ca0 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.792+0000 7f0329630640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.792+0000 7f0329630640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 0x7f03100a62e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f031c011940 tx=0x7f031c009900 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.792+0000 7f0323fff640 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0314006010 tx=0x7f0314005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80780f611295645c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.793+0000 7f031affd640 1 -- 172.21.15.122:0/1916881761 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031404f070 con 0x5620e7f99ca0 2024-04-02T13:39:05.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.793+0000 7f031affd640 1 -- 172.21.15.122:0/1916881761 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314040980 con 0x5620e7f99ca0 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f031affd640 1 -- 172.21.15.122:0/1916881761 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314003bd0 con 0x5620e7f99ca0 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- 172.21.15.122:0/1916881761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 msgr2=0x5620e7f9c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0314006010 tx=0x7f0314005f80 comp rx=0 tx=0).stop 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- 172.21.15.122:0/1916881761 shutdown_connections 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f9c090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7f99760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 --2- 172.21.15.122:0/1916881761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- 172.21.15.122:0/1916881761 >> 172.21.15.122:0/1916881761 conn(0x5620e7f8aed0 msgr2=0x5620e7f8b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- 172.21.15.122:0/1916881761 shutdown_connections 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- 172.21.15.122:0/1916881761 wait complete. 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.794+0000 7f032d922740 1 -- start start 2024-04-02T13:39:05.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 Processor -- start 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 0x5620e7f99670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f97010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5620e7f9ca80 con 0x5620e7f910c0 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5620e7eafe80 con 0x5620e7f907a0 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5620e7f9cc00 con 0x5620e7f99ca0 2024-04-02T13:39:05.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f032d922740 10 client.0 _ll_register_callbacks cb 0x5620e7eebaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0323fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38224/0 (socket says 172.21.15.122:38224) 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0323fff640 1 -- 172.21.15.122:0/207351884 learned_addr learned my addr 172.21.15.122:0/207351884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0328e2f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 0x5620e7f99670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0329630640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f97010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0323fff640 1 -- 172.21.15.122:0/207351884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 msgr2=0x5620e7f97010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.795+0000 7f0323fff640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f97010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0323fff640 1 -- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 msgr2=0x5620e7f99670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0323fff640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 0x5620e7f99670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0323fff640 1 -- 172.21.15.122:0/207351884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e7cd27d0 con 0x5620e7f910c0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0328e2f640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 0x5620e7f99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0323fff640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0314037040 tx=0x7f0314032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec6d95b86b80b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031404f040 con 0x5620e7f910c0 2024-04-02T13:39:05.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0314040980 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f0314040db0 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0314032590 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f032d922740 10 client.16923 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5620e7fa8aa0 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314040980 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.796+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f031405d030 con 0x5620e7f910c0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 10 client.16923 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 10 client.16923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 20 client.16923 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e7fac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 4 client.16923 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620e7f01cf0 con 0x5620e7faa0b0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f0328e2f640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e7fac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f0303fff640 10 client.16923 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:05.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.797+0000 7f0328e2f640 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e7fac550 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72848356e98539dc server_cookie=32c51ee70bcbdb26 in_seq=0 out_seq=0 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f0303fff640 1 -- 172.21.15.122:0/207351884 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620e7f01cf0 con 0x5620e7faa0b0 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f0303fff640 10 client.16923 handle_command_reply: tid=0 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 client.16923 shutdown 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 msgr2=0x5620e7f96a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0314037040 tx=0x7f0314032b80 comp rx=0 tx=0).stop 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 shutdown_connections 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e7fac550 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f99ca0 0x5620e7f97010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f0303fff640 0 client.16923 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f910c0 0x5620e7f96a00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 --2- 172.21.15.122:0/207351884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f907a0 0x5620e7f99670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 >> 172.21.15.122:0/207351884 conn(0x5620e7f8aed0 msgr2=0x5620e7f8cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.798+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 shutdown_connections 2024-04-02T13:39:05.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.799+0000 7f032d922740 1 -- 172.21.15.122:0/207351884 wait complete. 2024-04-02T13:39:05.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.799+0000 7f032d922740 20 client.16923 trim_cache size 0 max 16384 2024-04-02T13:39:05.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.816+0000 7f032d922740 1 Processor -- start 2024-04-02T13:39:05.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.816+0000 7f032d922740 1 -- start start 2024-04-02T13:39:05.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.817+0000 7f032d922740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f9bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.817+0000 7f032d922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f910c0 0x5620e7f959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.817+0000 7f032d922740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.817+0000 7f032d922740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b5610 con 0x5620e7f907a0 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f032d922740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b7270 con 0x5620e7f99ca0 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f032d922740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b5ce0 con 0x5620e7f910c0 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0328e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f9bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47424/0 (socket says 172.21.15.122:47424) 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0328e2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f9bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38234/0 (socket says 172.21.15.122:38234) 2024-04-02T13:39:05.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 -- 172.21.15.122:0/777307131 learned_addr learned my addr 172.21.15.122:0/777307131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 -- 172.21.15.122:0/777307131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f910c0 msgr2=0x5620e7f959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f910c0 0x5620e7f959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 -- 172.21.15.122:0/777307131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 msgr2=0x5620e7f9bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f9bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.818+0000 7f0329630640 1 -- 172.21.15.122:0/777307131 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e7f97380 con 0x5620e7f99ca0 2024-04-02T13:39:05.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.819+0000 7f0329630640 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f031c0095c0 tx=0x7f031c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155bc7582389feff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.819+0000 7f031a7fc640 1 -- 172.21.15.122:0/777307131 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c02e070 con 0x5620e7f99ca0 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.819+0000 7f031a7fc640 1 -- 172.21.15.122:0/777307131 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f031c020a70 con 0x5620e7f99ca0 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.819+0000 7f031a7fc640 1 -- 172.21.15.122:0/777307131 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c01fc10 con 0x5620e7f99ca0 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- 172.21.15.122:0/777307131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 msgr2=0x5620e7f99130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f031c0095c0 tx=0x7f031c005f90 comp rx=0 tx=0).stop 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- 172.21.15.122:0/777307131 shutdown_connections 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f99ca0 0x5620e7f99130 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f910c0 0x5620e7f959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 --2- 172.21.15.122:0/777307131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f907a0 0x5620e7f9bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- 172.21.15.122:0/777307131 >> 172.21.15.122:0/777307131 conn(0x5620e7f8aed0 msgr2=0x5620e7f952d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- 172.21.15.122:0/777307131 shutdown_connections 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- 172.21.15.122:0/777307131 wait complete. 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 -- start start 2024-04-02T13:39:05.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.820+0000 7f032d922740 1 Processor -- start 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.821+0000 7f032d922740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.821+0000 7f032d922740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7fad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f0328e2f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f0328e2f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:41698/0 (socket says 172.21.15.122:41698) 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f0328e2f640 1 -- 172.21.15.122:0/740683868 learned_addr learned my addr 172.21.15.122:0/740683868 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f99ca0 0x5620e7fad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f0323fff640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7fad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b6ae0 con 0x5620e7f99ca0 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b6410 con 0x5620e7f910c0 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03241b35f0 con 0x5620e7f907a0 2024-04-02T13:39:05.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.822+0000 7f0329630640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f99ca0 0x5620e7fad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f032d922740 10 client.0 _ll_register_callbacks cb 0x7f031404d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 -- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 msgr2=0x5620e7fad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7fad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 -- 172.21.15.122:0/740683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f99ca0 msgr2=0x5620e7fad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f99ca0 0x5620e7fad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e800e6c0 con 0x5620e7f907a0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0323fff640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7fad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0328e2f640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f030c00c6a0 tx=0x7f030c09c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=998ae90277660b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0303fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030c0a5560 con 0x5620e7f907a0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f030c098f60 con 0x5620e7f907a0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.823+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f031c020a70 con 0x5620e7f907a0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.824+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f030c080050 con 0x5620e7f907a0 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.824+0000 7f032d922740 10 client.26134 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:05.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.824+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5620e800fa00 con 0x5620e7f907a0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.824+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f030c004000 con 0x5620e7f907a0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.824+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f030c09bdd0 con 0x5620e7f907a0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 10 client.26134 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 20 client.26134 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e80122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 4 client.26134 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f031c020a70 con 0x5620e7faa0b0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f0323fff640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e80122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f0323fff640 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e80122f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ff92adcff12f87 server_cookie=780fc433c5f7166e in_seq=0 out_seq=0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.825+0000 7f0303fff640 10 client.26134 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f0303fff640 1 -- 172.21.15.122:0/740683868 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f031c020a70 con 0x5620e7faa0b0 2024-04-02T13:39:05.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f0303fff640 10 client.26134 handle_command_reply: tid=0 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 client.26134 shutdown 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 msgr2=0x5620e7facb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f030c00c6a0 tx=0x7f030c09c6c0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 shutdown_connections 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5620e7faa0b0 0x5620e80122f0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5620e7f99ca0 0x5620e7fad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f0303fff640 0 client.26134 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5620e7f910c0 0x5620e7fad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 --2- 172.21.15.122:0/740683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5620e7f907a0 0x5620e7facb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 >> 172.21.15.122:0/740683868 conn(0x5620e7f8aed0 msgr2=0x5620e7fa4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 shutdown_connections 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 1 -- 172.21.15.122:0/740683868 wait complete. 2024-04-02T13:39:05.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.826+0000 7f032d922740 20 client.26134 trim_cache size 0 max 16384 2024-04-02T13:39:05.833 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:05.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 msgr2=0x7f03100a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 0x7f03100a62e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f031c011940 tx=0x7f031c009900 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 msgr2=0x7f0324071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324071740 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f030c00e9f0 tx=0x7f030c00edf0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 shutdown_connections 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03100a3e20 0x7f03100a62e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0324107ab0 0x7f03241a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0324102460 0x7f0324071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 --2- 172.21.15.122:0/1580180849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0324101aa0 0x7f0324071740 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 >> 172.21.15.122:0/1580180849 conn(0x7f03240fd530 msgr2=0x7f03240fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 shutdown_connections 2024-04-02T13:39:05.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:05.827+0000 7f032b093640 1 -- 172.21.15.122:0/1580180849 wait complete. 2024-04-02T13:39:05.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114210 2.94 MB/sec execute 26 sec latency 26.210 ms 2024-04-02T13:39:06.841 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:39:06.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:39:06.842 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:39:06.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114210 2.83 MB/sec execute 27 sec latency 1025.566 ms 2024-04-02T13:39:06.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 -- 172.21.15.122:0/87050906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160106980 msgr2=0x7fe160106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 --2- 172.21.15.122:0/87050906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160106980 0x7fe160106d60 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009930 tx=0x7fe15c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 -- 172.21.15.122:0/87050906 shutdown_connections 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 --2- 172.21.15.122:0/87050906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160101330 0x7fe16010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 --2- 172.21.15.122:0/87050906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160100970 0x7fe160100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 --2- 172.21.15.122:0/87050906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160106980 0x7fe160106d60 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 -- 172.21.15.122:0/87050906 >> 172.21.15.122:0/87050906 conn(0x7fe160077b00 msgr2=0x7fe160077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 -- 172.21.15.122:0/87050906 shutdown_connections 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.960+0000 7fe166c06640 1 -- 172.21.15.122:0/87050906 wait complete. 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 Processor -- start 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 -- start start 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160101330 0x7fe1600fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106980 0x7fe1600fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110d90 con 0x7fe160100970 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110c10 con 0x7fe160106980 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe166c06640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe160110f10 con 0x7fe160101330 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38258/0 (socket says 172.21.15.122:38258) 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe1649a2640 1 -- 172.21.15.122:0/734155124 learned_addr learned my addr 172.21.15.122:0/734155124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe1651a3640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106980 0x7fe1600fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.961+0000 7fe157fff640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160101330 0x7fe1600fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 -- 172.21.15.122:0/734155124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160101330 msgr2=0x7fe1600fdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160101330 0x7fe1600fdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 -- 172.21.15.122:0/734155124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106980 msgr2=0x7fe1600fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106980 0x7fe1600fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 -- 172.21.15.122:0/734155124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1601a6890 con 0x7fe160100970 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe1649a2640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009230 tx=0x7fe15c02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe155ffb640 1 -- 172.21.15.122:0/734155124 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c041070 con 0x7fe160100970 2024-04-02T13:39:06.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe155ffb640 1 -- 172.21.15.122:0/734155124 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c004450 con 0x7fe160100970 2024-04-02T13:39:06.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1601a6b20 con 0x7fe160100970 2024-04-02T13:39:06.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe155ffb640 1 -- 172.21.15.122:0/734155124 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c034440 con 0x7fe160100970 2024-04-02T13:39:06.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.962+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1601a7010 con 0x7fe160100970 2024-04-02T13:39:06.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.964+0000 7fe155ffb640 1 -- 172.21.15.122:0/734155124 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe15c044070 con 0x7fe160100970 2024-04-02T13:39:06.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 Processor -- start 2024-04-02T13:39:06.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 -- start start 2024-04-02T13:39:06.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb456be0 0x5593bb45f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb461bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593bb3ba770 con 0x5593bb4562c0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593bb362960 con 0x5593bb456be0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1697eb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593bb3b68c0 con 0x5593bb45f7c0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe155ffb640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 0x7fe1480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38264/0 (socket says 172.21.15.122:38264) 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 -- 172.21.15.122:0/138696844 learned_addr learned my addr 172.21.15.122:0/138696844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 0x7fe1480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe155ffb640 1 -- 172.21.15.122:0/734155124 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe15c0f2490 con 0x7fe160100970 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe157fff640 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb461bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1649a2640 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb456be0 0x5593bb45f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 -- 172.21.15.122:0/138696844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 msgr2=0x5593bb461bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb461bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 -- 172.21.15.122:0/138696844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb456be0 msgr2=0x5593bb45f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb456be0 0x5593bb45f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.965+0000 7fe1651a3640 1 -- 172.21.15.122:0/138696844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bb1967e0 con 0x5593bb4562c0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe1651a3640 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe158012d00 tx=0x7fe158010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309e22aaf9743b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe1651a3640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 0x7fe1480a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe158010c40 tx=0x7fe15800c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe146ffd640 1 -- 172.21.15.122:0/138696844 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158024070 con 0x5593bb4562c0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe146ffd640 1 -- 172.21.15.122:0/138696844 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158009d20 con 0x5593bb4562c0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe146ffd640 1 -- 172.21.15.122:0/138696844 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158005260 con 0x5593bb4562c0 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe1697eb740 1 -- 172.21.15.122:0/138696844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 msgr2=0x5593bb4566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.966+0000 7fe1697eb740 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe158012d00 tx=0x7fe158010740 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- 172.21.15.122:0/138696844 shutdown_connections 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb461bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb456be0 0x5593bb45f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- 172.21.15.122:0/138696844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb4566a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- 172.21.15.122:0/138696844 >> 172.21.15.122:0/138696844 conn(0x5593bb4509f0 msgr2=0x5593bb450dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- 172.21.15.122:0/138696844 shutdown_connections 2024-04-02T13:39:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- 172.21.15.122:0/138696844 wait complete. 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- start start 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 Processor -- start 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb45cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb45d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593bb462550 con 0x5593bb4562c0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593bb375d20 con 0x5593bb45f7c0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593bb4626d0 con 0x5593bb456be0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.967+0000 7fe1697eb740 10 client.0 _ll_register_callbacks cb 0x5593bb342a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38276/0 (socket says 172.21.15.122:38276) 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 -- 172.21.15.122:0/1546247609 learned_addr learned my addr 172.21.15.122:0/1546247609 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe157fff640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb45cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1651a3640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb45d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 -- 172.21.15.122:0/1546247609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 msgr2=0x5593bb45cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb45cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 -- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 msgr2=0x5593bb45d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb45d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 -- 172.21.15.122:0/1546247609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593bb1f6f70 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1649a2640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0bac30 tx=0x7fe15c0be030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b8b2350a9769e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe13ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c04b040 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe15c0f2150 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593bb3c7cb0 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe15c0be260 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1697eb740 10 client.16941 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.968+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593bb08e170 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5593bb3c7cb0 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe15c0be4c0 con 0x5593bb4562c0 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 10 client.16941 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 20 client.16941 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb46fb30 0x5593bb471f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 4 client.16941 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593bb3c2000 con 0x5593bb46fb30 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe157fff640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb46fb30 0x5593bb471f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe13ffff640 10 client.16941 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.969+0000 7fe157fff640 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb46fb30 0x5593bb471f90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c39179eef975e52e server_cookie=6adf8c0449463b06 in_seq=0 out_seq=0 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe13ffff640 1 -- 172.21.15.122:0/1546247609 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593bb3c2000 con 0x5593bb46fb30 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe13ffff640 10 client.16941 handle_command_reply: tid=0 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 client.16941 shutdown 2024-04-02T13:39:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 msgr2=0x5593bb45c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0bac30 tx=0x7fe15c0be030 comp rx=0 tx=0).stop 2024-04-02T13:39:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 shutdown_connections 2024-04-02T13:39:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb46fb30 0x5593bb471f90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb45d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe13ffff640 0 client.16941 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb45cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 --2- 172.21.15.122:0/1546247609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45c7e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 >> 172.21.15.122:0/1546247609 conn(0x5593bb4509f0 msgr2=0x5593bb452740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 shutdown_connections 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 1 -- 172.21.15.122:0/1546247609 wait complete. 2024-04-02T13:39:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.970+0000 7fe1697eb740 20 client.16941 trim_cache size 0 max 16384 2024-04-02T13:39:06.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 Processor -- start 2024-04-02T13:39:06.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 -- start start 2024-04-02T13:39:06.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb46b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb4778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601b1090 con 0x5593bb4562c0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601b2cf0 con 0x5593bb45f7c0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1697eb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601b1760 con 0x5593bb456be0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38292/0 (socket says 172.21.15.122:38292) 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 -- 172.21.15.122:0/2176943152 learned_addr learned my addr 172.21.15.122:0/2176943152 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1651a3640 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb4778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 -- 172.21.15.122:0/2176943152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 msgr2=0x5593bb46b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb46b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 -- 172.21.15.122:0/2176943152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 msgr2=0x5593bb4778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb4778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1649a2640 1 -- 172.21.15.122:0/2176943152 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593bb472610 con 0x5593bb4562c0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.988+0000 7fe1651a3640 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb4778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1649a2640 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c0be160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12fe8c0fbf771da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1467fc640 1 -- 172.21.15.122:0/2176943152 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0bdda0 con 0x5593bb4562c0 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1467fc640 1 -- 172.21.15.122:0/2176943152 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c0c2560 con 0x5593bb4562c0 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1467fc640 1 -- 172.21.15.122:0/2176943152 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c0036f0 con 0x5593bb4562c0 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 -- 172.21.15.122:0/2176943152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 msgr2=0x5593bb45ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009900 tx=0x7fe15c0be160 comp rx=0 tx=0).stop 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 -- 172.21.15.122:0/2176943152 shutdown_connections 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb45f7c0 0x5593bb4778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb456be0 0x5593bb46b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 --2- 172.21.15.122:0/2176943152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb4562c0 0x5593bb45ef10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 -- 172.21.15.122:0/2176943152 >> 172.21.15.122:0/2176943152 conn(0x5593bb4509f0 msgr2=0x5593bb45d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 -- 172.21.15.122:0/2176943152 shutdown_connections 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.989+0000 7fe1697eb740 1 -- 172.21.15.122:0/2176943152 wait complete. 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 -- start start 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 Processor -- start 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 0x5593bb459890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb4d5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601b2560 con 0x5593bb456be0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601b1e90 con 0x5593bb4562c0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe1601af070 con 0x5593bb45f7c0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1697eb740 10 client.0 _ll_register_callbacks cb 0x7fe15c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe157fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe157fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38304/0 (socket says 172.21.15.122:38304) 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe157fff640 1 -- 172.21.15.122:0/501650591 learned_addr learned my addr 172.21.15.122:0/501650591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.990+0000 7fe1649a2640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 0x5593bb459890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1651a3640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb4d5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 -- 172.21.15.122:0/501650591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 msgr2=0x5593bb4d5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb4d5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 -- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 msgr2=0x5593bb459890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 0x5593bb459890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 -- 172.21.15.122:0/501650591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593bb461bb0 con 0x5593bb456be0 2024-04-02T13:39:06.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1649a2640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 0x5593bb459890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe157fff640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x5593bb451b80 tx=0x7fe150010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20b5a5fa689b183 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150020640 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe15c0c2560 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe1500207e0 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe150030050 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1697eb740 10 client.16947 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593bb4d66f0 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe15c0c2560 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fe15000f590 con 0x5593bb456be0 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1697eb740 10 client.16947 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.991+0000 7fe1697eb740 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1697eb740 20 client.16947 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb4d6f60 0x5593bb4d9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1697eb740 4 client.16947 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5593bb33fbe0 con 0x5593bb4d6f60 2024-04-02T13:39:06.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1649a2640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb4d6f60 0x5593bb4d9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe13ffff640 10 client.16947 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.992+0000 7fe1649a2640 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb4d6f60 0x5593bb4d9380 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859b73612afc51a1 server_cookie=22f89472cfd227c4 in_seq=0 out_seq=0 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe13ffff640 1 -- 172.21.15.122:0/501650591 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5593bb33fbe0 con 0x5593bb4d6f60 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe13ffff640 10 client.16947 handle_command_reply: tid=0 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 client.16947 shutdown 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 msgr2=0x5593bb459dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x5593bb451b80 tx=0x7fe150010af0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 shutdown_connections 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593bb4d6f60 0x5593bb4d9380 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593bb45f7c0 0x5593bb4d5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593bb456be0 0x5593bb459dd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe13ffff640 0 client.16947 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 --2- 172.21.15.122:0/501650591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593bb4562c0 0x5593bb459890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.993+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 >> 172.21.15.122:0/501650591 conn(0x5593bb4509f0 msgr2=0x5593bb44f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 shutdown_connections 2024-04-02T13:39:07.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe1697eb740 1 -- 172.21.15.122:0/501650591 wait complete. 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe1697eb740 20 client.16947 trim_cache size 0 max 16384 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "bd1463a1-3293-47b0-aa0a-d78e20ca0cb5", 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:39:07.002 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 msgr2=0x7fe1480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 0x7fe1480a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe158010c40 tx=0x7fe15800c000 comp rx=0 tx=0).stop 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 msgr2=0x7fe1600fd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009230 tx=0x7fe15c02cc70 comp rx=0 tx=0).stop 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 shutdown_connections 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1480a3d20 0x7fe1480a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe160106980 0x7fe1600fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe160101330 0x7fe1600fdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 --2- 172.21.15.122:0/734155124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe160100970 0x7fe1600fd9a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 >> 172.21.15.122:0/734155124 conn(0x7fe160077b00 msgr2=0x7fe1600ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 shutdown_connections 2024-04-02T13:39:07.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:06.994+0000 7fe166c06640 1 -- 172.21.15.122:0/734155124 wait complete. 2024-04-02T13:39:07.009 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:07.009 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- 172.21.15.122:0/3787126728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528100970 msgr2=0x7fb528100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- 172.21.15.122:0/3787126728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528100970 0x7fb528100df0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb518009930 tx=0x7fb51802bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- 172.21.15.122:0/3787126728 shutdown_connections 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- 172.21.15.122:0/3787126728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528101330 0x7fb52810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- 172.21.15.122:0/3787126728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528100970 0x7fb528100df0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- 172.21.15.122:0/3787126728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 0x7fb528106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- 172.21.15.122:0/3787126728 >> 172.21.15.122:0/3787126728 conn(0x7fb528077b00 msgr2=0x7fb528077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- 172.21.15.122:0/3787126728 shutdown_connections 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- 172.21.15.122:0/3787126728 wait complete. 2024-04-02T13:39:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 Processor -- start 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- start start 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528100970 0x7fb5281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 0x7fb52819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb528111e90 con 0x7fb528101330 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb528111d10 con 0x7fb528100970 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.119+0000 7fb53061b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112010 con 0x7fb528106980 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38314/0 (socket says 172.21.15.122:38314) 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 -- 172.21.15.122:0/3133220280 learned_addr learned my addr 172.21.15.122:0/3133220280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528100970 0x7fb5281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 0x7fb52819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 -- 172.21.15.122:0/3133220280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 msgr2=0x7fb52819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 0x7fb52819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 -- 172.21.15.122:0/3133220280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528100970 msgr2=0x7fb5281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528100970 0x7fb5281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 -- 172.21.15.122:0/3133220280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52819d3a0 con 0x7fb528101330 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb52dbb6640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb518009230 tx=0x7fb51802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb51f7fe640 1 -- 172.21.15.122:0/3133220280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518041070 con 0x7fb528101330 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb51f7fe640 1 -- 172.21.15.122:0/3133220280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb518004450 con 0x7fb528101330 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52819d630 con 0x7fb528101330 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb51f7fe640 1 -- 172.21.15.122:0/3133220280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518034440 con 0x7fb528101330 2024-04-02T13:39:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.120+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a4f20 con 0x7fb528101330 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.122+0000 7fb51f7fe640 1 -- 172.21.15.122:0/3133220280 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb518044070 con 0x7fb528101330 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 Processor -- start 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 -- start start 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb51f7fe640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561ed9320830 con 0x561ed93bc600 2024-04-02T13:39:07.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561ed92c8820 con 0x561ed93ca1e0 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb533204740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561ed931c7e0 con 0x561ed93bcf20 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52dbb6640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38320/0 (socket says 172.21.15.122:38320) 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2839522442 learned_addr learned my addr 172.21.15.122:0/2839522442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb51f7fe640 1 -- 172.21.15.122:0/3133220280 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb5180f24d0 con 0x7fb528101330 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.123+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2839522442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 msgr2=0x561ed93c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2839522442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 msgr2=0x561ed93cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2839522442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed90fe7d0 con 0x561ed93bc600 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52dbb6640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 0x7fb5140a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5180efa50 tx=0x7fb51804bbd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x561ed93b8190 tx=0x7fb520009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61614620ed029306 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb51cbfe640 1 -- 172.21.15.122:0/2839522442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000ed80 con 0x561ed93bc600 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb51cbfe640 1 -- 172.21.15.122:0/2839522442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5200128f0 con 0x561ed93bc600 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.124+0000 7fb51cbfe640 1 -- 172.21.15.122:0/2839522442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5200107d0 con 0x561ed93bc600 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- 172.21.15.122:0/2839522442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 msgr2=0x561ed93bc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x561ed93b8190 tx=0x7fb520009750 comp rx=0 tx=0).stop 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- 172.21.15.122:0/2839522442 shutdown_connections 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- 172.21.15.122:0/2839522442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93bc9e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- 172.21.15.122:0/2839522442 >> 172.21.15.122:0/2839522442 conn(0x561ed93b7000 msgr2=0x561ed93b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- 172.21.15.122:0/2839522442 shutdown_connections 2024-04-02T13:39:07.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- 172.21.15.122:0/2839522442 wait complete. 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- start start 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 Processor -- start 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561ed93ccf40 con 0x561ed93bc600 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561ed92dbd30 con 0x561ed93ca1e0 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561ed93cd0c0 con 0x561ed93bcf20 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb533204740 10 client.0 _ll_register_callbacks cb 0x561ed92db720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.125+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38332/0 (socket says 172.21.15.122:38332) 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2911513250 learned_addr learned my addr 172.21.15.122:0/2911513250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2911513250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 msgr2=0x561ed93c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 msgr2=0x561ed93c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2911513250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed90343f0 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb520006010 tx=0x7fb520009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dd17a9490863e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520026030 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb520012620 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5200128f0 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb520014820 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb533204740 10 client.16959 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ed8ff40f0 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.126+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5200128f0 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb52000f040 con 0x561ed93bc600 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 10 client.16959 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:07.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 20 client.16959 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed93dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 4 client.16959 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ed932dd70 con 0x561ed93da040 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed93dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb511ffb640 10 client.16959 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.127+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed93dc430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6c3573d6a1f0ca6 server_cookie=71708dd8632086b0 in_seq=0 out_seq=0 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb511ffb640 1 -- 172.21.15.122:0/2911513250 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ed932dd70 con 0x561ed93da040 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb511ffb640 10 client.16959 handle_command_reply: tid=0 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 client.16959 shutdown 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 msgr2=0x561ed93c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fb520006010 tx=0x7fb520009880 comp rx=0 tx=0).stop 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 shutdown_connections 2024-04-02T13:39:07.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed93dc430 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb511ffb640 0 client.16959 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 --2- 172.21.15.122:0/2911513250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93c9a90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 >> 172.21.15.122:0/2911513250 conn(0x561ed93b7000 msgr2=0x561ed93b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 shutdown_connections 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 1 -- 172.21.15.122:0/2911513250 wait complete. 2024-04-02T13:39:07.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.128+0000 7fb533204740 20 client.16959 trim_cache size 0 max 16384 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 Processor -- start 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 -- start start 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281acdf0 con 0x561ed93bc600 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281aea50 con 0x561ed93ca1e0 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.145+0000 7fb533204740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281ad4c0 con 0x561ed93bcf20 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38334/0 (socket says 172.21.15.122:38334) 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2495495698 learned_addr learned my addr 172.21.15.122:0/2495495698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:07.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2495495698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 msgr2=0x561ed93b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2495495698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 msgr2=0x561ed93c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 -- 172.21.15.122:0/2495495698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed93c7960 con 0x561ed93bc600 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb520014b70 tx=0x7fb5200144c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34529e4f671b465 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb513fff640 1 -- 172.21.15.122:0/2495495698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520026030 con 0x561ed93bc600 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb513fff640 1 -- 172.21.15.122:0/2495495698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb520005870 con 0x561ed93bc600 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb513fff640 1 -- 172.21.15.122:0/2495495698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520005410 con 0x561ed93bc600 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb533204740 1 -- 172.21.15.122:0/2495495698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 msgr2=0x561ed93d5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.146+0000 7fb533204740 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fb520014b70 tx=0x7fb5200144c0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- 172.21.15.122:0/2495495698 shutdown_connections 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93ca1e0 0x561ed93c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- 172.21.15.122:0/2495495698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93bc600 0x561ed93d5e70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- 172.21.15.122:0/2495495698 >> 172.21.15.122:0/2495495698 conn(0x561ed93b7000 msgr2=0x561ed93c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- 172.21.15.122:0/2495495698 shutdown_connections 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- 172.21.15.122:0/2495495698 wait complete. 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- start start 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 Processor -- start 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93bc600 0x561ed93cd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281ae2c0 con 0x561ed93ca1e0 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281adbf0 con 0x561ed93bc600 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5281aadd0 con 0x561ed93bcf20 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb533204740 10 client.0 _ll_register_callbacks cb 0x7fb520014aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38338/0 (socket says 172.21.15.122:38338) 2024-04-02T13:39:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 -- 172.21.15.122:0/789320757 learned_addr learned my addr 172.21.15.122:0/789320757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93bc600 0x561ed93cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 -- 172.21.15.122:0/789320757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 msgr2=0x561ed93cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 -- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93bc600 msgr2=0x561ed93cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93bc600 0x561ed93cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.147+0000 7fb52ebb8640 1 -- 172.21.15.122:0/789320757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed943ea60 con 0x561ed93ca1e0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb52ebb8640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ef10 tx=0x7fb5200121f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371598baf96692db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520026040 con 0x561ed93ca1e0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb520005870 con 0x561ed93ca1e0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb520012620 con 0x561ed93ca1e0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb520005670 con 0x561ed93ca1e0 2024-04-02T13:39:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 10 client.16965 fetch_fsmap learned FSMap version 32 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ed943fca0 con 0x561ed93ca1e0 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb520005870 con 0x561ed93ca1e0 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb520010600 con 0x561ed93ca1e0 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 20 client.16965 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed94422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 4 client.16965 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.148+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ed93280c0 con 0x561ed93da040 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed94422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb511ffb640 10 client.16965 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb52e3b7640 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed94422a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60eede7cc0e7b129 server_cookie=a4fb951cee09a7ce in_seq=0 out_seq=0 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb511ffb640 1 -- 172.21.15.122:0/789320757 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561ed93280c0 con 0x561ed93da040 2024-04-02T13:39:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb511ffb640 10 client.16965 handle_command_reply: tid=0 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 client.16965 shutdown 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 msgr2=0x561ed943e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb52000ef10 tx=0x7fb5200121f0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 shutdown_connections 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561ed93da040 0x561ed94422a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561ed93ca1e0 0x561ed943e520 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561ed93bcf20 0x561ed93cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb511ffb640 0 client.16965 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 --2- 172.21.15.122:0/789320757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561ed93bc600 0x561ed93cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 >> 172.21.15.122:0/789320757 conn(0x561ed93b7000 msgr2=0x561ed93c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 shutdown_connections 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 1 -- 172.21.15.122:0/789320757 wait complete. 2024-04-02T13:39:07.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.149+0000 7fb533204740 20 client.16965 trim_cache size 0 max 16384 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "bd1463a1-3293-47b0-aa0a-d78e20ca0cb5": { 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "bd1463a1-3293-47b0-aa0a-d78e20ca0cb5", 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:39:07.160 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 msgr2=0x7fb5140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 0x7fb5140a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5180efa50 tx=0x7fb51804bbd0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 msgr2=0x7fb5281988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fb518009230 tx=0x7fb51802cba0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 shutdown_connections 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb5140a3d20 0x7fb5140a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb528106980 0x7fb52819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb528101330 0x7fb5281988f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 --2- 172.21.15.122:0/3133220280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb528100970 0x7fb5281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 >> 172.21.15.122:0/3133220280 conn(0x7fb528077b00 msgr2=0x7fb5281031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 shutdown_connections 2024-04-02T13:39:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:07.150+0000 7fb53061b640 1 -- 172.21.15.122:0/3133220280 wait complete. 2024-04-02T13:39:07.162 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:07 smithi122 ceph-mon[32114]: pgmap v121: 129 pgs: 129 active+clean; 75 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T13:39:07.163 INFO:tasks.cephfs.filesystem:scrub status for tag:bd1463a1-3293-47b0-aa0a-d78e20ca0cb5 - {'path': '/', 'tag': 'bd1463a1-3293-47b0-aa0a-d78e20ca0cb5', 'options': 'recursive,force'} 2024-04-02T13:39:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:07 smithi187 ceph-mon[36064]: pgmap v121: 129 pgs: 129 active+clean; 75 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T13:39:07.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:07 smithi188 ceph-mon[36117]: pgmap v121: 129 pgs: 129 active+clean; 75 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-04-02T13:39:07.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114266 2.73 MB/sec execute 28 sec latency 1103.434 ms 2024-04-02T13:39:08.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:08.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:08 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:08 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:08 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:08.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114334 2.64 MB/sec execute 29 sec latency 1326.886 ms 2024-04-02T13:39:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:09 smithi187 ceph-mon[36064]: pgmap v122: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-02T13:39:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:09 smithi188 ceph-mon[36117]: pgmap v122: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-02T13:39:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:09 smithi122 ceph-mon[32114]: pgmap v122: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-04-02T13:39:09.948 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114334 2.55 MB/sec execute 30 sec latency 1495.248 ms 2024-04-02T13:39:10.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114334 2.47 MB/sec execute 31 sec latency 2495.339 ms 2024-04-02T13:39:11.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:11 smithi187 ceph-mon[36064]: pgmap v123: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T13:39:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:11 smithi188 ceph-mon[36117]: pgmap v123: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T13:39:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:11 smithi122 ceph-mon[32114]: pgmap v123: 129 pgs: 129 active+clean; 76 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-04-02T13:39:11.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114334 2.39 MB/sec execute 32 sec latency 3495.415 ms 2024-04-02T13:39:12.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 114334 2.32 MB/sec execute 33 sec latency 4495.480 ms 2024-04-02T13:39:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:13 smithi187 ceph-mon[36064]: pgmap v124: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-04-02T13:39:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:13 smithi188 ceph-mon[36117]: pgmap v124: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-04-02T13:39:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:13 smithi122 ceph-mon[32114]: pgmap v124: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-04-02T13:39:13.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 115946 2.45 MB/sec execute 34 sec latency 4948.479 ms 2024-04-02T13:39:14.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 117574 2.72 MB/sec execute 35 sec latency 11.774 ms 2024-04-02T13:39:15.372 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:15 smithi122 ceph-mon[32114]: pgmap v125: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 613 KiB/s wr, 84 op/s 2024-04-02T13:39:15.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:15 smithi187 ceph-mon[36064]: pgmap v125: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 613 KiB/s wr, 84 op/s 2024-04-02T13:39:15.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:15 smithi188 ceph-mon[36117]: pgmap v125: 129 pgs: 129 active+clean; 75 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 613 KiB/s wr, 84 op/s 2024-04-02T13:39:15.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 117812 2.66 MB/sec execute 36 sec latency 830.274 ms 2024-04-02T13:39:16.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 117812 2.58 MB/sec execute 37 sec latency 1830.321 ms 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: pgmap v126: 129 pgs: 129 active+clean; 76 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 816 KiB/s wr, 124 op/s 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: pgmap v126: 129 pgs: 129 active+clean; 76 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 816 KiB/s wr, 124 op/s 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: pgmap v126: 129 pgs: 129 active+clean; 76 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 653 KiB/s rd, 816 KiB/s wr, 124 op/s 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:17.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 117812 2.52 MB/sec execute 38 sec latency 2830.427 ms 2024-04-02T13:39:18.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 119290 2.59 MB/sec execute 39 sec latency 3281.779 ms 2024-04-02T13:39:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:19 smithi187 ceph-mon[36064]: pgmap v127: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 507 KiB/s wr, 75 op/s 2024-04-02T13:39:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:19 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:19 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:19 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:19 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:39:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:19 smithi188 ceph-mon[36117]: pgmap v127: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 507 KiB/s wr, 75 op/s 2024-04-02T13:39:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:19 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:19 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:19 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:19 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:39:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:19 smithi122 ceph-mon[32114]: pgmap v127: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 637 KiB/s rd, 507 KiB/s wr, 75 op/s 2024-04-02T13:39:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:19 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:39:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:19 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:39:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:19 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:39:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:19 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:39:19.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 121070 2.86 MB/sec execute 40 sec latency 10.363 ms 2024-04-02T13:39:20.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 121356 2.80 MB/sec execute 41 sec latency 646.007 ms 2024-04-02T13:39:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:21 smithi187 ceph-mon[36064]: pgmap v128: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 329 KiB/s wr, 61 op/s 2024-04-02T13:39:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:21 smithi188 ceph-mon[36117]: pgmap v128: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 329 KiB/s wr, 61 op/s 2024-04-02T13:39:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:21 smithi122 ceph-mon[32114]: pgmap v128: 129 pgs: 129 active+clean; 77 MiB data, 774 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 329 KiB/s wr, 61 op/s 2024-04-02T13:39:21.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 121356 2.73 MB/sec execute 42 sec latency 1088.817 ms 2024-04-02T13:39:22.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 121356 2.67 MB/sec execute 43 sec latency 2088.911 ms 2024-04-02T13:39:23.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:23 smithi187 ceph-mon[36064]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 827 KiB/s wr, 110 op/s 2024-04-02T13:39:23.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:23 smithi188 ceph-mon[36117]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 827 KiB/s wr, 110 op/s 2024-04-02T13:39:23.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:23 smithi122 ceph-mon[32114]: pgmap v129: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 827 KiB/s wr, 110 op/s 2024-04-02T13:39:23.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 122967 2.76 MB/sec execute 44 sec latency 2538.519 ms 2024-04-02T13:39:24.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124627 2.97 MB/sec execute 45 sec latency 10.276 ms 2024-04-02T13:39:25.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:25 smithi187 ceph-mon[36064]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 735 KiB/s wr, 94 op/s 2024-04-02T13:39:25.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:25 smithi188 ceph-mon[36117]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 735 KiB/s wr, 94 op/s 2024-04-02T13:39:25.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:25 smithi122 ceph-mon[32114]: pgmap v130: 129 pgs: 129 active+clean; 82 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 735 KiB/s wr, 94 op/s 2024-04-02T13:39:25.949 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124849 2.91 MB/sec execute 46 sec latency 805.038 ms 2024-04-02T13:39:26.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124849 2.84 MB/sec execute 47 sec latency 1805.126 ms 2024-04-02T13:39:27.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:27 smithi187 ceph-mon[36064]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 135 op/s 2024-04-02T13:39:27.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:27 smithi188 ceph-mon[36117]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 135 op/s 2024-04-02T13:39:27.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:27 smithi122 ceph-mon[32114]: pgmap v131: 129 pgs: 129 active+clean; 81 MiB data, 799 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 926 KiB/s wr, 135 op/s 2024-04-02T13:39:27.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124849 2.79 MB/sec execute 48 sec latency 2805.225 ms 2024-04-02T13:39:28.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124973 2.73 MB/sec execute 49 sec latency 3259.834 ms 2024-04-02T13:39:29.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:29 smithi187 ceph-mon[36064]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 837 KiB/s wr, 102 op/s 2024-04-02T13:39:29.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:29 smithi188 ceph-mon[36117]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 837 KiB/s wr, 102 op/s 2024-04-02T13:39:29.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:29 smithi122 ceph-mon[32114]: pgmap v132: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 837 KiB/s wr, 102 op/s 2024-04-02T13:39:29.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124973 2.68 MB/sec execute 50 sec latency 1436.338 ms 2024-04-02T13:39:30.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124973 2.63 MB/sec execute 51 sec latency 2436.437 ms 2024-04-02T13:39:31.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:31 smithi187 ceph-mon[36064]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-04-02T13:39:31.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:31 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:31.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:31 smithi188 ceph-mon[36117]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-04-02T13:39:31.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:31 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:31.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:31 smithi122 ceph-mon[32114]: pgmap v133: 129 pgs: 129 active+clean; 82 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 803 KiB/s wr, 97 op/s 2024-04-02T13:39:31.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:31 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:31.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124973 2.58 MB/sec execute 52 sec latency 3436.557 ms 2024-04-02T13:39:32.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 124973 2.53 MB/sec execute 53 sec latency 4436.704 ms 2024-04-02T13:39:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:33 smithi187 ceph-mon[36064]: pgmap v134: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-02T13:39:33.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:33 smithi188 ceph-mon[36117]: pgmap v134: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-02T13:39:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:33 smithi122 ceph-mon[32114]: pgmap v134: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-04-02T13:39:33.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 126571 2.60 MB/sec execute 54 sec latency 4888.191 ms 2024-04-02T13:39:34.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128254 2.78 MB/sec execute 55 sec latency 10.124 ms 2024-04-02T13:39:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:35 smithi187 ceph-mon[36064]: pgmap v135: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 544 KiB/s wr, 71 op/s 2024-04-02T13:39:35.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:35 smithi188 ceph-mon[36117]: pgmap v135: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 544 KiB/s wr, 71 op/s 2024-04-02T13:39:35.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:35 smithi122 ceph-mon[32114]: pgmap v135: 129 pgs: 129 active+clean; 81 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 544 KiB/s wr, 71 op/s 2024-04-02T13:39:35.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128393 2.73 MB/sec execute 56 sec latency 898.201 ms 2024-04-02T13:39:36.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128393 2.68 MB/sec execute 57 sec latency 1898.272 ms 2024-04-02T13:39:37.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:37.165 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- 172.21.15.122:0/1388508445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4106980 msgr2=0x7f94e4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 --2- 172.21.15.122:0/1388508445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4106980 0x7f94e4106d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009900 tx=0x7f94e002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- 172.21.15.122:0/1388508445 shutdown_connections 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 --2- 172.21.15.122:0/1388508445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 --2- 172.21.15.122:0/1388508445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4100970 0x7f94e4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 --2- 172.21.15.122:0/1388508445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4106980 0x7f94e4106d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- 172.21.15.122:0/1388508445 >> 172.21.15.122:0/1388508445 conn(0x7f94e4077b00 msgr2=0x7f94e4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- 172.21.15.122:0/1388508445 shutdown_connections 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- 172.21.15.122:0/1388508445 wait complete. 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 Processor -- start 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.273+0000 7f94eb97b640 1 -- start start 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4100970 0x7f94e4197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4106980 0x7f94e419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110d90 con 0x7f94e4101330 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110c10 con 0x7f94e4100970 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4110f10 con 0x7f94e4106980 2024-04-02T13:39:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37534/0 (socket says 172.21.15.122:37534) 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 -- 172.21.15.122:0/3099976254 learned_addr learned my addr 172.21.15.122:0/3099976254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e9717640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4100970 0x7f94e4197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e9f18640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4106980 0x7f94e419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 -- 172.21.15.122:0/3099976254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4106980 msgr2=0x7f94e419bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4106980 0x7f94e419bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 -- 172.21.15.122:0/3099976254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4100970 msgr2=0x7f94e4197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4100970 0x7f94e4197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 -- 172.21.15.122:0/3099976254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e419c280 con 0x7f94e4101330 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94e8f16640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9f0 tx=0x7f94d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94da7fc640 1 -- 172.21.15.122:0/3099976254 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400cd80 con 0x7f94e4101330 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e419c570 con 0x7f94e4101330 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e41a3e00 con 0x7f94e4101330 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94da7fc640 1 -- 172.21.15.122:0/3099976254 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d4005500 con 0x7f94e4101330 2024-04-02T13:39:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.274+0000 7f94da7fc640 1 -- 172.21.15.122:0/3099976254 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4010690 con 0x7f94e4101330 2024-04-02T13:39:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94da7fc640 1 -- 172.21.15.122:0/3099976254 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f94d40046c0 con 0x7f94e4101330 2024-04-02T13:39:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 Processor -- start 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 -- start start 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94da7fc640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 0x7f94d00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3869cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d3872890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d38751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5615d37cd830 con 0x5615d3872dd0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5615d3775820 con 0x5615d386a1f0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94ee566740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615d37c97e0 con 0x5615d38698d0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94e9f18640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 0x7f94d00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.277+0000 7f94e9f18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d38751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94da7fc640 1 -- 172.21.15.122:0/3099976254 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94d40d0dc0 con 0x7f94e4101330 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d3872890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e9f18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d38751c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37536/0 (socket says 172.21.15.122:37536) 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e9f18640 1 -- 172.21.15.122:0/2013699667 learned_addr learned my addr 172.21.15.122:0/2013699667 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e9717640 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3869cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 -- 172.21.15.122:0/2013699667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 msgr2=0x5615d3869cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3869cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 -- 172.21.15.122:0/2013699667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 msgr2=0x5615d38751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d38751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 -- 172.21.15.122:0/2013699667 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d35ab7d0 con 0x5615d386a1f0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e9f18640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 0x7f94d00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f94dc012c20 tx=0x7f94dc009dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94e8f16640 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d3872890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f94d40ce180 tx=0x7f94d4065710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f177fa5831cf7e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94cf7fe640 1 -- 172.21.15.122:0/2013699667 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d406e780 con 0x5615d386a1f0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94cf7fe640 1 -- 172.21.15.122:0/2013699667 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d409d0d0 con 0x5615d386a1f0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.278+0000 7f94cf7fe640 1 -- 172.21.15.122:0/2013699667 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40056b0 con 0x5615d386a1f0 2024-04-02T13:39:37.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- 172.21.15.122:0/2013699667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 msgr2=0x5615d3872890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d3872890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f94d40ce180 tx=0x7f94d4065710 comp rx=0 tx=0).stop 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- 172.21.15.122:0/2013699667 shutdown_connections 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d38751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d3872890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 --2- 172.21.15.122:0/2013699667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3869cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- 172.21.15.122:0/2013699667 >> 172.21.15.122:0/2013699667 conn(0x5615d3864000 msgr2=0x5615d38643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- 172.21.15.122:0/2013699667 shutdown_connections 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- 172.21.15.122:0/2013699667 wait complete. 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 -- start start 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.279+0000 7f94ee566740 1 Processor -- start 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3872630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d386f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5615d3875bc0 con 0x5615d3872dd0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5615d3788d30 con 0x5615d386a1f0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5615d3875d40 con 0x5615d38698d0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 10 client.0 _ll_register_callbacks cb 0x5615d3788720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37546/0 (socket says 172.21.15.122:37546) 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e8f16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d386f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 -- 172.21.15.122:0/855404730 learned_addr learned my addr 172.21.15.122:0/855404730 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9717640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3872630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 -- 172.21.15.122:0/855404730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 msgr2=0x5615d3872630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3872630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 -- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 msgr2=0x5615d386f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d386f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 -- 172.21.15.122:0/855404730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d34e13f0 con 0x5615d3872dd0 2024-04-02T13:39:37.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9717640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3872630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e8f16640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d386f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94e9f18640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0098d0 tx=0x7f94dc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5a15f30b47790a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0122c0 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94dc020410 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.280+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94d409d0d0 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94dc020610 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 10 client.16977 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5615d34a10f0 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d409d0d0 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f94dc01f050 con 0x5615d3872dd0 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 10 client.16977 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:37.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 20 client.16977 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d3882e10 0x5615d3885230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 4 client.16977 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615d37dad70 con 0x5615d3882e10 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.281+0000 7f94e9717640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d3882e10 0x5615d3885230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ccff9640 10 client.16977 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94e9717640 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d3882e10 0x5615d3885230 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d4300645eaf369 server_cookie=98b40346c1b8afb4 in_seq=0 out_seq=0 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ccff9640 1 -- 172.21.15.122:0/855404730 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615d37dad70 con 0x5615d3882e10 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ccff9640 10 client.16977 handle_command_reply: tid=0 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 client.16977 shutdown 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 msgr2=0x5615d386ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0098d0 tx=0x7f94dc009500 comp rx=0 tx=0).stop 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 shutdown_connections 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d3882e10 0x5615d3885230 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d3872dd0 0x5615d386ff00 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ccff9640 0 client.16977 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d386f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 --2- 172.21.15.122:0/855404730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d38698d0 0x5615d3872630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.282+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 >> 172.21.15.122:0/855404730 conn(0x5615d3864000 msgr2=0x5615d3865d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.283+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 shutdown_connections 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.283+0000 7f94ee566740 1 -- 172.21.15.122:0/855404730 wait complete. 2024-04-02T13:39:37.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.283+0000 7f94ee566740 20 client.16977 trim_cache size 0 max 16384 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 Processor -- start 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 -- start start 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d387b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d3872dd0 0x5615d38630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800fef0 con 0x5615d38698d0 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8011ac0 con 0x5615d386a1f0 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94ee566740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80105c0 con 0x5615d3872dd0 2024-04-02T13:39:37.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37562/0 (socket says 172.21.15.122:37562) 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 -- 172.21.15.122:0/1167651580 learned_addr learned my addr 172.21.15.122:0/1167651580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e8f16640 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d387b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 -- 172.21.15.122:0/1167651580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d3872dd0 msgr2=0x5615d38630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d3872dd0 0x5615d38630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 -- 172.21.15.122:0/1167651580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 msgr2=0x5615d387b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d387b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 -- 172.21.15.122:0/1167651580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d3863e30 con 0x5615d38698d0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.300+0000 7f94e9717640 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f94e002c170 tx=0x7f94e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaaff0ff5f6bde2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ceffd640 1 -- 172.21.15.122:0/1167651580 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0050070 con 0x5615d38698d0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ceffd640 1 -- 172.21.15.122:0/1167651580 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0004400 con 0x5615d38698d0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ceffd640 1 -- 172.21.15.122:0/1167651580 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0004920 con 0x5615d38698d0 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 -- 172.21.15.122:0/1167651580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 msgr2=0x5615d386eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f94e002c170 tx=0x7f94e0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 -- 172.21.15.122:0/1167651580 shutdown_connections 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d3872dd0 0x5615d38630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d386a1f0 0x5615d387b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 --2- 172.21.15.122:0/1167651580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386eb20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 -- 172.21.15.122:0/1167651580 >> 172.21.15.122:0/1167651580 conn(0x5615d3864000 msgr2=0x5615d38629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 -- 172.21.15.122:0/1167651580 shutdown_connections 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.301+0000 7f94ee566740 1 -- 172.21.15.122:0/1167651580 wait complete. 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 -- start start 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 Processor -- start 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d386a1f0 0x5615d386c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 0x5615d38ea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8011390 con 0x5615d38698d0 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8010cc0 con 0x5615d3872dd0 2024-04-02T13:39:37.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800ded0 con 0x5615d386a1f0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94ee566740 10 client.0 _ll_register_callbacks cb 0x7f94dc012570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37566/0 (socket says 172.21.15.122:37566) 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 -- 172.21.15.122:0/4162366181 learned_addr learned my addr 172.21.15.122:0/4162366181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9f18640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 0x5615d38ea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e8f16640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d386a1f0 0x5615d386c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 -- 172.21.15.122:0/4162366181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d386a1f0 msgr2=0x5615d386c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d386a1f0 0x5615d386c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 -- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 msgr2=0x5615d38ea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 0x5615d38ea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9717640 1 -- 172.21.15.122:0/4162366181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d38ea910 con 0x5615d38698d0 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.302+0000 7f94e9f18640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 0x5615d38ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:37.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94e9717640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009230 tx=0x7f94e004a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e336c0c4b296acf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0050030 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94e0004200 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e0004400 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f94e005f050 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 10 client.16983 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5615d38ebac0 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0004400 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f94e0047070 con 0x5615d38698d0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 10 client.16983 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 20 client.16983 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d38ec1b0 0x5615d38ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 4 client.16983 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5615d37d50c0 con 0x5615d38ec1b0 2024-04-02T13:39:37.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.303+0000 7f94e8f16640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d38ec1b0 0x5615d38ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94e8f16640 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d38ec1b0 0x5615d38ee5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33eed686bad1d098 server_cookie=20707a2fef0a44cd in_seq=0 out_seq=0 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ccff9640 10 client.16983 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ccff9640 1 -- 172.21.15.122:0/4162366181 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5615d37d50c0 con 0x5615d38ec1b0 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ccff9640 10 client.16983 handle_command_reply: tid=0 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 client.16983 shutdown 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 msgr2=0x5615d386bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009230 tx=0x7f94e004a7f0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 shutdown_connections 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5615d38ec1b0 0x5615d38ee5d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5615d3872dd0 0x5615d38ea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ccff9640 0 client.16983 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5615d386a1f0 0x5615d386c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 --2- 172.21.15.122:0/4162366181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5615d38698d0 0x5615d386bc10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 >> 172.21.15.122:0/4162366181 conn(0x5615d3864000 msgr2=0x5615d38629b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 shutdown_connections 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 1 -- 172.21.15.122:0/4162366181 wait complete. 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.304+0000 7f94ee566740 20 client.16983 trim_cache size 0 max 16384 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:39:37.315 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 msgr2=0x7f94d00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 0x7f94d00a6260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f94dc012c20 tx=0x7f94dc009dd0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 msgr2=0x7f94e41977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f94d400e9f0 tx=0x7f94d400edf0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 shutdown_connections 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94d00a3da0 0x7f94d00a6260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94e4106980 0x7f94e419bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94e4101330 0x7f94e41977d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 --2- 172.21.15.122:0/3099976254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94e4100970 0x7f94e4197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 >> 172.21.15.122:0/3099976254 conn(0x7f94e4077b00 msgr2=0x7f94e4103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 shutdown_connections 2024-04-02T13:39:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.305+0000 7f94eb97b640 1 -- 172.21.15.122:0/3099976254 wait complete. 2024-04-02T13:39:37.318 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:39:37.318 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.415+0000 7fdb2b374640 1 -- 172.21.15.122:0/541501787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241009d0 msgr2=0x7fdb24100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.415+0000 7fdb2b374640 1 --2- 172.21.15.122:0/541501787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241009d0 0x7fdb24100e50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009930 tx=0x7fdb1402bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- 172.21.15.122:0/541501787 shutdown_connections 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- 172.21.15.122:0/541501787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 0x7fdb2410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- 172.21.15.122:0/541501787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241009d0 0x7fdb24100e50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- 172.21.15.122:0/541501787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241069e0 0x7fdb24106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- 172.21.15.122:0/541501787 >> 172.21.15.122:0/541501787 conn(0x7fdb24076d80 msgr2=0x7fdb24077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- 172.21.15.122:0/541501787 shutdown_connections 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- 172.21.15.122:0/541501787 wait complete. 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 Processor -- start 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- start start 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 0x7fdb24197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 0x7fdb24197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24110df0 con 0x7fdb241069e0 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24110c70 con 0x7fdb24101390 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.416+0000 7fdb2b374640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24110f70 con 0x7fdb241009d0 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37570/0 (socket says 172.21.15.122:37570) 2024-04-02T13:39:37.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 -- 172.21.15.122:0/1555015701 learned_addr learned my addr 172.21.15.122:0/1555015701 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb2890f640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 0x7fdb24197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29110640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 0x7fdb24197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 -- 172.21.15.122:0/1555015701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 msgr2=0x7fdb24197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 0x7fdb24197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 -- 172.21.15.122:0/1555015701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 msgr2=0x7fdb24197850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 0x7fdb24197850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 -- 172.21.15.122:0/1555015701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2419c300 con 0x7fdb241069e0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29110640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 0x7fdb24197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb29911640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9b0 tx=0x7fdb2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2000cd80 con 0x7fdb241069e0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb200035f0 con 0x7fdb241069e0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2419c5f0 con 0x7fdb241069e0 2024-04-02T13:39:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20010670 con 0x7fdb241069e0 2024-04-02T13:39:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.417+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241a3e80 con 0x7fdb241069e0 2024-04-02T13:39:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.418+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb24102110 con 0x7fdb241069e0 2024-04-02T13:39:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.420+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdb200037a0 con 0x7fdb241069e0 2024-04-02T13:39:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.420+0000 7fdb1a7fc640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 0x7fdb100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.420+0000 7fdb29110640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 0x7fdb100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.420+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdb200d0fd0 con 0x7fdb241069e0 2024-04-02T13:39:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.421+0000 7fdb29110640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 0x7fdb100a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0092f0 tx=0x7fdb1c009280 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.424+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdb2009d370 con 0x7fdb241069e0 2024-04-02T13:39:37.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:37 smithi187 ceph-mon[36064]: pgmap v136: 129 pgs: 129 active+clean; 83 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 975 KiB/s wr, 115 op/s 2024-04-02T13:39:37.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:37 smithi188 ceph-mon[36117]: pgmap v136: 129 pgs: 129 active+clean; 83 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 975 KiB/s wr, 115 op/s 2024-04-02T13:39:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:37 smithi122 ceph-mon[32114]: pgmap v136: 129 pgs: 129 active+clean; 83 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 975 KiB/s wr, 115 op/s 2024-04-02T13:39:37.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.641+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb24068d40 con 0x7fdb241069e0 2024-04-02T13:39:37.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.641+0000 7fdb1a7fc640 1 -- 172.21.15.122:0/1555015701 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11946 (secure 0 0 0) 0x7fdb24068d40 con 0x7fdb241069e0 2024-04-02T13:39:37.645 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:39:37.645 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:39:36.403715+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:39:37.645 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 34 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 msgr2=0x7fdb100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 0x7fdb100a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0092f0 tx=0x7fdb1c009280 comp rx=0 tx=0).stop 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 msgr2=0x7fdb2419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000e9b0 tx=0x7fdb2000ee70 comp rx=0 tx=0).stop 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 shutdown_connections 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdb100a3d20 0x7fdb100a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb241069e0 0x7fdb2419bbd0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdb24101390 0x7fdb24197850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 --2- 172.21.15.122:0/1555015701 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb241009d0 0x7fdb24197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 >> 172.21.15.122:0/1555015701 conn(0x7fdb24076d80 msgr2=0x7fdb240fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 shutdown_connections 2024-04-02T13:39:37.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.643+0000 7fdb2b374640 1 -- 172.21.15.122:0/1555015701 wait complete. 2024-04-02T13:39:37.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:39:37.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- 172.21.15.122:0/4031490152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 msgr2=0x7f942810d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 --2- 172.21.15.122:0/4031490152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942810d860 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9410009930 tx=0x7f941002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- 172.21.15.122:0/4031490152 shutdown_connections 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 --2- 172.21.15.122:0/4031490152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942810d860 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 --2- 172.21.15.122:0/4031490152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9428100990 0x7f9428100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 --2- 172.21.15.122:0/4031490152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94281069a0 0x7f9428106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- 172.21.15.122:0/4031490152 >> 172.21.15.122:0/4031490152 conn(0x7f9428077ad0 msgr2=0x7f9428077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- 172.21.15.122:0/4031490152 shutdown_connections 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- 172.21.15.122:0/4031490152 wait complete. 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 Processor -- start 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.757+0000 7f942f047640 1 -- start start 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 0x7f942819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 0x7f942819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94281140b0 con 0x7f9428101350 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9428113f30 con 0x7f9428100990 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9428114230 con 0x7f94281069a0 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37600/0 (socket says 172.21.15.122:37600) 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 -- 172.21.15.122:0/441613241 learned_addr learned my addr 172.21.15.122:0/441613241 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942cde3640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 0x7f942819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942d5e4640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 0x7f942819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 -- 172.21.15.122:0/441613241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 msgr2=0x7f942819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 0x7f942819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 -- 172.21.15.122:0/441613241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 msgr2=0x7f942819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 0x7f942819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 -- 172.21.15.122:0/441613241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942819f5c0 con 0x7f9428101350 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942cde3640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 0x7f942819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942d5e4640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 0x7f942819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9427fff640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f941800b4a0 tx=0x7f941800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9425ffb640 1 -- 172.21.15.122:0/441613241 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418014070 con 0x7f9428101350 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9425ffb640 1 -- 172.21.15.122:0/441613241 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418010af0 con 0x7f9428101350 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942819f8b0 con 0x7f9428101350 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.758+0000 7f9425ffb640 1 -- 172.21.15.122:0/441613241 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418017d10 con 0x7f9428101350 2024-04-02T13:39:37.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.759+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94281a7140 con 0x7f9428101350 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9425ffb640 1 -- 172.21.15.122:0/441613241 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9418017070 con 0x7f9428101350 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 Processor -- start 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 -- start start 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9425ffb640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 0x7f94140a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a0789b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a0795090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0a06ed7b0 con 0x55f0a078a0c0 2024-04-02T13:39:37.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0a068c570 con 0x55f0a07897a0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.761+0000 7f9431c2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0a06e9760 con 0x55f0a0792ca0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f942d5e4640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 0x7f94140a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37616/0 (socket says 172.21.15.122:37616) 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 -- 172.21.15.122:0/3457491749 learned_addr learned my addr 172.21.15.122:0/3457491749 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9425ffb640 1 -- 172.21.15.122:0/441613241 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f94180cfeb0 con 0x7f9428101350 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f942d5e4640 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a0795090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 -- 172.21.15.122:0/3457491749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 msgr2=0x55f0a0795090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a0795090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 -- 172.21.15.122:0/3457491749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 msgr2=0x55f0a0789b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a0789b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 -- 172.21.15.122:0/3457491749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a04013f0 con 0x55f0a078a0c0 2024-04-02T13:39:37.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f9427fff640 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f94180cd380 tx=0x7f9418017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ca5291aabcfb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f942d5e4640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 0x7f94140a61d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9410006010 tx=0x7f941002cb70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f941effd640 1 -- 172.21.15.122:0/3457491749 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941801e070 con 0x55f0a078a0c0 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f941effd640 1 -- 172.21.15.122:0/3457491749 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f941809c270 con 0x55f0a078a0c0 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.762+0000 7f941effd640 1 -- 172.21.15.122:0/3457491749 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94180ba070 con 0x55f0a078a0c0 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- 172.21.15.122:0/3457491749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 msgr2=0x55f0a0792760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f94180cd380 tx=0x7f9418017730 comp rx=0 tx=0).stop 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- 172.21.15.122:0/3457491749 shutdown_connections 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a0795090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a078a0c0 0x55f0a0792760 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3457491749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a0789b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- 172.21.15.122:0/3457491749 >> 172.21.15.122:0/3457491749 conn(0x55f0a0783ed0 msgr2=0x55f0a07842b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- 172.21.15.122:0/3457491749 shutdown_connections 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- 172.21.15.122:0/3457491749 wait complete. 2024-04-02T13:39:37.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 -- start start 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.763+0000 7f9431c2c740 1 Processor -- start 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 0x55f0a0790060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a07905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0a0795a30 con 0x55f0a07897a0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0a06a8e80 con 0x55f0a078a0c0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0a0795bb0 con 0x55f0a0792ca0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 10 client.0 _ll_register_callbacks cb 0x55f0a06e4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37618/0 (socket says 172.21.15.122:37618) 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 -- 172.21.15.122:0/3603811311 learned_addr learned my addr 172.21.15.122:0/3603811311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9427fff640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 0x55f0a0790060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942d5e4640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a07905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 -- 172.21.15.122:0/3603811311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 msgr2=0x55f0a07905a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a07905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 -- 172.21.15.122:0/3603811311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 msgr2=0x55f0a0790060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 0x55f0a0790060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 -- 172.21.15.122:0/3603811311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a04cb7d0 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9427fff640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 0x55f0a0790060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f942cde3640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9420009a00 tx=0x7f942000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e180cc6617d94fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9407fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420003970 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9420003b10 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f941809c270 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9420023050 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.764+0000 7f9431c2c740 10 client.17001 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f0a07a1b80 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f941809c270 con 0x55f0a07897a0 2024-04-02T13:39:37.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f9420018020 con 0x55f0a07897a0 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 10 client.17001 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 10 client.17001 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 20 client.17001 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a07a3040 0x55f0a07a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 4 client.17001 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0a06facf0 con 0x55f0a07a3040 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9427fff640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a07a3040 0x55f0a07a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9407fff640 10 client.17001 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.765+0000 7f9427fff640 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a07a3040 0x55f0a07a5430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f18075058b076df2 server_cookie=f44ec5f2ee141c6 in_seq=0 out_seq=0 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9407fff640 1 -- 172.21.15.122:0/3603811311 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0a06facf0 con 0x55f0a07a3040 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9407fff640 10 client.17001 handle_command_reply: tid=0 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 client.17001 shutdown 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 msgr2=0x55f0a078fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9420009a00 tx=0x7f942000f790 comp rx=0 tx=0).stop 2024-04-02T13:39:37.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 shutdown_connections 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a07a3040 0x55f0a07a5430 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a0792ca0 0x55f0a07905a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9407fff640 0 client.17001 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a078a0c0 0x55f0a0790060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3603811311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a078fb20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 >> 172.21.15.122:0/3603811311 conn(0x55f0a0783ed0 msgr2=0x55f0a0785c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 shutdown_connections 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 1 -- 172.21.15.122:0/3603811311 wait complete. 2024-04-02T13:39:37.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.766+0000 7f9431c2c740 20 client.17001 trim_cache size 0 max 16384 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 Processor -- start 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 -- start start 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a0792ca0 0x55f0a078ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b1280 con 0x55f0a07897a0 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b2e50 con 0x55f0a0792ca0 2024-04-02T13:39:37.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.779+0000 7f9431c2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b1950 con 0x55f0a078a0c0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37626/0 (socket says 172.21.15.122:37626) 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 -- 172.21.15.122:0/1478703420 learned_addr learned my addr 172.21.15.122:0/1478703420 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942d5e4640 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a0792ca0 0x55f0a078ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 -- 172.21.15.122:0/1478703420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 msgr2=0x55f0a079e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 -- 172.21.15.122:0/1478703420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a0792ca0 msgr2=0x55f0a078ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a0792ca0 0x55f0a078ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 -- 172.21.15.122:0/1478703420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a07a57a0 con 0x55f0a07897a0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f942cde3640 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9420009580 tx=0x7f9420009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16b7574862df87a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f941e7fc640 1 -- 172.21.15.122:0/1478703420 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942002b020 con 0x55f0a07897a0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f941e7fc640 1 -- 172.21.15.122:0/1478703420 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f942001acd0 con 0x55f0a07897a0 2024-04-02T13:39:37.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.780+0000 7f941e7fc640 1 -- 172.21.15.122:0/1478703420 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9420020040 con 0x55f0a07897a0 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- 172.21.15.122:0/1478703420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 msgr2=0x55f0a0792250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9420009580 tx=0x7f9420009630 comp rx=0 tx=0).stop 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- 172.21.15.122:0/1478703420 shutdown_connections 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a0792ca0 0x55f0a078ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 --2- 172.21.15.122:0/1478703420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a07897a0 0x55f0a0792250 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- 172.21.15.122:0/1478703420 >> 172.21.15.122:0/1478703420 conn(0x55f0a0783ed0 msgr2=0x55f0a07826a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- 172.21.15.122:0/1478703420 shutdown_connections 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- 172.21.15.122:0/1478703420 wait complete. 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 -- start start 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.781+0000 7f9431c2c740 1 Processor -- start 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a079add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b2720 con 0x55f0a0792ca0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94281b2050 con 0x55f0a07897a0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94281af260 con 0x55f0a078a0c0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 10 client.0 _ll_register_callbacks cb 0x7f9420009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942cde3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a079add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37628/0 (socket says 172.21.15.122:37628) 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 -- 172.21.15.122:0/3936727466 learned_addr learned my addr 172.21.15.122:0/3936727466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9427fff640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 -- 172.21.15.122:0/3936727466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 msgr2=0x55f0a079b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 -- 172.21.15.122:0/3936727466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 msgr2=0x55f0a079add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a079add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 -- 172.21.15.122:0/3936727466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a0796b90 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9427fff640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942cde3640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a079add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f942d5e4640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9410009230 tx=0x7f941002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364f7fcde162168e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9407fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9410051020 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9410004ad0 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.782+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f942001acd0 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f941005c050 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 10 client.17007 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f0a0809180 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f942001acd0 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f941004d480 con 0x55f0a0792ca0 2024-04-02T13:39:37.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 10 client.17007 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 10 client.17007 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 20 client.17007 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a0809950 0x55f0a080bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 4 client.17007 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0a06f5040 con 0x55f0a0809950 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f942cde3640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a0809950 0x55f0a080bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f942cde3640 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a0809950 0x55f0a080bd70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96b0f6961d586f79 server_cookie=540aa160bba8a5d8 in_seq=0 out_seq=0 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.783+0000 7f9407fff640 10 client.17007 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9407fff640 1 -- 172.21.15.122:0/3936727466 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0a06f5040 con 0x55f0a0809950 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9407fff640 10 client.17007 handle_command_reply: tid=0 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 client.17007 shutdown 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 msgr2=0x55f0a0796390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9410009230 tx=0x7f941002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 shutdown_connections 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55f0a0809950 0x55f0a080bd70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a0792ca0 0x55f0a0796390 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9407fff640 0 client.17007 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a078a0c0 0x55f0a079b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 --2- 172.21.15.122:0/3936727466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a07897a0 0x55f0a079add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 >> 172.21.15.122:0/3936727466 conn(0x55f0a0783ed0 msgr2=0x55f0a0782f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 shutdown_connections 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 1 -- 172.21.15.122:0/3936727466 wait complete. 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f9431c2c740 20 client.17007 trim_cache size 0 max 16384 2024-04-02T13:39:37.794 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 msgr2=0x7f94140a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 0x7f94140a61d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9410006010 tx=0x7f941002cb70 comp rx=0 tx=0).stop 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 msgr2=0x7f942819ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f941800b4a0 tx=0x7f941800b960 comp rx=0 tx=0).stop 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 shutdown_connections 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94140a3d10 0x7f94140a61d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94281069a0 0x7f942819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.784+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9428101350 0x7f942819ab10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.785+0000 7f942f047640 1 --2- 172.21.15.122:0/441613241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9428100990 0x7f942819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.785+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 >> 172.21.15.122:0/441613241 conn(0x7f9428077ad0 msgr2=0x7f94281030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.785+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 shutdown_connections 2024-04-02T13:39:37.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.785+0000 7f942f047640 1 -- 172.21.15.122:0/441613241 wait complete. 2024-04-02T13:39:37.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- 172.21.15.122:0/1198208689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098100270 msgr2=0x7f5098100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 --2- 172.21.15.122:0/1198208689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098100270 0x7f5098100650 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5080009900 tx=0x7f508002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- 172.21.15.122:0/1198208689 shutdown_connections 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 --2- 172.21.15.122:0/1198208689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098106e80 0x7f509810ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 --2- 172.21.15.122:0/1198208689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5098100c20 0x7f5098106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 --2- 172.21.15.122:0/1198208689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098100270 0x7f5098100650 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- 172.21.15.122:0/1198208689 >> 172.21.15.122:0/1198208689 conn(0x7f5098076fc0 msgr2=0x7f50980fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- 172.21.15.122:0/1198208689 shutdown_connections 2024-04-02T13:39:37.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- 172.21.15.122:0/1198208689 wait complete. 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 Processor -- start 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.900+0000 7f509cc7b640 1 -- start start 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098100270 0x7f5098198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f509819cc20 0x7f509819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5098111ee0 con 0x7f5098106e80 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5098111d60 con 0x7f5098100270 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5098112060 con 0x7f509819cc20 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37642/0 (socket says 172.21.15.122:37642) 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 -- 172.21.15.122:0/3326494690 learned_addr learned my addr 172.21.15.122:0/3326494690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509659c640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098100270 0x7f5098198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 -- 172.21.15.122:0/3326494690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f509819cc20 msgr2=0x7f509819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f509819cc20 0x7f509819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 -- 172.21.15.122:0/3326494690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098100270 msgr2=0x7f5098198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098100270 0x7f5098198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 -- 172.21.15.122:0/3326494690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f509819d710 con 0x7f5098106e80 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f5095d9b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f508800c8c0 tx=0x7f508800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f508f7fe640 1 -- 172.21.15.122:0/3326494690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088007bb0 con 0x7f5098106e80 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f508f7fe640 1 -- 172.21.15.122:0/3326494690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5088007d50 con 0x7f5098106e80 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f509819da00 con 0x7f5098106e80 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f508f7fe640 1 -- 172.21.15.122:0/3326494690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088018830 con 0x7f5098106e80 2024-04-02T13:39:37.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.901+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50981a51a0 con 0x7f5098106e80 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f508f7fe640 1 -- 172.21.15.122:0/3326494690 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5088018a10 con 0x7f5098106e80 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 Processor -- start 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 -- start start 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f508f7fe640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 0x7f50840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090fea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090feaf60 0x563090ff7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563090f4e7b0 con 0x563090ff80b0 2024-04-02T13:39:37.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563090eed570 con 0x563090fea5b0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f509f869740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563090f4a760 con 0x563090feaf60 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f5096d9d640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 0x7f50840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f508f7fe640 1 -- 172.21.15.122:0/3326494690 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f50880d0c90 con 0x7f5098106e80 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.904+0000 7f5096d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090fea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f5096d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37652/0 (socket says 172.21.15.122:37652) 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f5096d9d640 1 -- 172.21.15.122:0/1825280941 learned_addr learned my addr 172.21.15.122:0/1825280941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f5095d9b640 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090feaf60 0x563090ff7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 -- 172.21.15.122:0/1825280941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090feaf60 msgr2=0x563090ff7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090feaf60 0x563090ff7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 -- 172.21.15.122:0/1825280941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 msgr2=0x563090ffa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 -- 172.21.15.122:0/1825280941 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563090c623f0 con 0x563090fea5b0 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f5096d9d640 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:37.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f5096d9d640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 0x7f50840a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5090013b10 tx=0x7f50900098e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f509659c640 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090fea990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5080036040 tx=0x7f508002c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921ec08c796def8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f508cff9640 1 -- 172.21.15.122:0/1825280941 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508002ca80 con 0x563090fea5b0 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f508cff9640 1 -- 172.21.15.122:0/1825280941 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508002cc20 con 0x563090fea5b0 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.905+0000 7f508cff9640 1 -- 172.21.15.122:0/1825280941 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080005160 con 0x563090fea5b0 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- 172.21.15.122:0/1825280941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 msgr2=0x563090fea990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090fea990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5080036040 tx=0x7f508002c5d0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- 172.21.15.122:0/1825280941 shutdown_connections 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090ff80b0 0x563090ffa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090feaf60 0x563090ff7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 --2- 172.21.15.122:0/1825280941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090fea990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- 172.21.15.122:0/1825280941 >> 172.21.15.122:0/1825280941 conn(0x563090fe4ed0 msgr2=0x563090fe52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- 172.21.15.122:0/1825280941 shutdown_connections 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- 172.21.15.122:0/1825280941 wait complete. 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.906+0000 7f509f869740 1 -- start start 2024-04-02T13:39:37.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 Processor -- start 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563090ffae10 con 0x563090feaf60 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563090f09e80 con 0x563090fea5b0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563090ffaf90 con 0x563090ff80b0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 10 client.0 _ll_register_callbacks cb 0x563090f45aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37660/0 (socket says 172.21.15.122:37660) 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 -- 172.21.15.122:0/2644671607 learned_addr learned my addr 172.21.15.122:0/2644671607 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509659c640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5096d9d640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 -- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 msgr2=0x563090ff55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 -- 172.21.15.122:0/2644671607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 msgr2=0x563090ff7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 -- 172.21.15.122:0/2644671607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563090d2c7d0 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5096d9d640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f5095d9b640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5088098d00 tx=0x7f508809c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd17d7ebdc6a1e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f507dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088093040 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f508800fac0 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.907+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508002cc20 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5088018d00 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 10 client.17019 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563091006f40 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508002cc20 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f5088094260 con 0x563090feaf60 2024-04-02T13:39:37.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 10 client.17019 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 10 client.17019 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 20 client.17019 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5630910083a0 0x56309100a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 4 client.17019 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563090f5bcf0 con 0x5630910083a0 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.908+0000 7f509659c640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5630910083a0 0x56309100a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.909+0000 7f507dffb640 10 client.17019 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.909+0000 7f509659c640 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5630910083a0 0x56309100a790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b5f9dcd9765ce58 server_cookie=9dc9a9d5ce360741 in_seq=0 out_seq=0 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.909+0000 7f507dffb640 1 -- 172.21.15.122:0/2644671607 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563090f5bcf0 con 0x5630910083a0 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.909+0000 7f507dffb640 10 client.17019 handle_command_reply: tid=0 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.909+0000 7f509f869740 1 client.17019 shutdown 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 msgr2=0x563090ff4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5088098d00 tx=0x7f508809c8a0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 shutdown_connections 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5630910083a0 0x56309100a790 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff4ed0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f507dffb640 0 client.17019 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 --2- 172.21.15.122:0/2644671607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 >> 172.21.15.122:0/2644671607 conn(0x563090fe4ed0 msgr2=0x563090fe6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 shutdown_connections 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 1 -- 172.21.15.122:0/2644671607 wait complete. 2024-04-02T13:39:37.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.910+0000 7f509f869740 20 client.17019 trim_cache size 0 max 16384 2024-04-02T13:39:37.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 Processor -- start 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 -- start start 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090fea5b0 0x563091003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 0x563090ff7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50981acce0 con 0x563090feaf60 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ae8b0 con 0x563090ff80b0 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509f869740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ad3b0 con 0x563090fea5b0 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37668/0 (socket says 172.21.15.122:37668) 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 -- 172.21.15.122:0/276631934 learned_addr learned my addr 172.21.15.122:0/276631934 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5096d9d640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 0x563090ff7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f509659c640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090fea5b0 0x563091003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 -- 172.21.15.122:0/276631934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090fea5b0 msgr2=0x563091003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090fea5b0 0x563091003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 -- 172.21.15.122:0/276631934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 msgr2=0x563090ff7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 0x563090ff7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 -- 172.21.15.122:0/276631934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563090fe9890 con 0x563090feaf60 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5096d9d640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 0x563090ff7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f5095d9b640 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5088007590 tx=0x7f508809adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499cef6b95e289ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.923+0000 7f507ffff640 1 -- 172.21.15.122:0/276631934 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508809c670 con 0x563090feaf60 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f507ffff640 1 -- 172.21.15.122:0/276631934 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508800fac0 con 0x563090feaf60 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f507ffff640 1 -- 172.21.15.122:0/276631934 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50880870c0 con 0x563090feaf60 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- 172.21.15.122:0/276631934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 msgr2=0x563090ff3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5088007590 tx=0x7f508809adb0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- 172.21.15.122:0/276631934 shutdown_connections 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090ff80b0 0x563090ff7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff3eb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 --2- 172.21.15.122:0/276631934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090fea5b0 0x563091003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- 172.21.15.122:0/276631934 >> 172.21.15.122:0/276631934 conn(0x563090fe4ed0 msgr2=0x563090ff37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- 172.21.15.122:0/276631934 shutdown_connections 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- 172.21.15.122:0/276631934 wait complete. 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 -- start start 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.924+0000 7f509f869740 1 Processor -- start 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f50981ae180 con 0x563090feaf60 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50981adab0 con 0x563090fea5b0 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50981aacc0 con 0x563090ff80b0 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 10 client.0 _ll_register_callbacks cb 0x7f508800c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:37.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37680/0 (socket says 172.21.15.122:37680) 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 -- 172.21.15.122:0/232560721 learned_addr learned my addr 172.21.15.122:0/232560721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509659c640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5096d9d640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 -- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 msgr2=0x563090ff99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 -- 172.21.15.122:0/232560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 msgr2=0x563090ff8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 -- 172.21.15.122:0/232560721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563090ffe4f0 con 0x563090feaf60 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f5095d9b640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f508809c830 tx=0x7f5088000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d2d921520ef556d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f507dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508809c670 con 0x563090feaf60 2024-04-02T13:39:37.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f50880870b0 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f508800fac0 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f50880a5080 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 10 client.17025 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.925+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56309106ec70 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f508800fac0 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f508809b760 con 0x563090feaf60 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 10 client.17025 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 10 client.17025 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 20 client.17025 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56309106f0f0 0x5630910714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 4 client.17025 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563090f56040 con 0x56309106f0f0 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509659c640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56309106f0f0 0x5630910714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:37.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f507dffb640 10 client.17025 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.926+0000 7f509659c640 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56309106f0f0 0x5630910714e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6afd5dc19438d46c server_cookie=afc3461fbb572515 in_seq=0 out_seq=0 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f507dffb640 1 -- 172.21.15.122:0/232560721 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563090f56040 con 0x56309106f0f0 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f507dffb640 10 client.17025 handle_command_reply: tid=0 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 client.17025 shutdown 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 msgr2=0x563090ff9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f508809c830 tx=0x7f5088000bc0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 shutdown_connections 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563090ff80b0 0x563090ff99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56309106f0f0 0x5630910714e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563090feaf60 0x563090ff9470 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 --2- 172.21.15.122:0/232560721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563090fea5b0 0x563090ff8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f507dffb640 0 client.17025 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 >> 172.21.15.122:0/232560721 conn(0x563090fe4ed0 msgr2=0x563090ff4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 shutdown_connections 2024-04-02T13:39:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 1 -- 172.21.15.122:0/232560721 wait complete. 2024-04-02T13:39:37.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509f869740 20 client.17025 trim_cache size 0 max 16384 2024-04-02T13:39:37.938 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 msgr2=0x7f50840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 0x7f50840a61e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5090013b10 tx=0x7f50900098e0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 msgr2=0x7f50981988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f508800c8c0 tx=0x7f508800cd80 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 shutdown_connections 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50840a3d20 0x7f50840a61e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f509819cc20 0x7f509819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5098106e80 0x7f50981988a0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 --2- 172.21.15.122:0/3326494690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5098100270 0x7f5098198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:37.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 >> 172.21.15.122:0/3326494690 conn(0x7f5098076fc0 msgr2=0x7f5098104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 shutdown_connections 2024-04-02T13:39:37.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:37.927+0000 7f509cc7b640 1 -- 172.21.15.122:0/3326494690 wait complete. 2024-04-02T13:39:37.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:39:37.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128393 2.63 MB/sec execute 58 sec latency 2898.302 ms 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.040+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/3635486569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 msgr2=0x7fdd34106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.040+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/3635486569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd34106d60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009930 tx=0x7fdd3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/3635486569 shutdown_connections 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/3635486569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34101330 0x7fdd3410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/3635486569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34100970 0x7fdd34100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/3635486569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd34106d60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/3635486569 >> 172.21.15.122:0/3635486569 conn(0x7fdd34077b00 msgr2=0x7fdd34077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/3635486569 shutdown_connections 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/3635486569 wait complete. 2024-04-02T13:39:38.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 Processor -- start 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- start start 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34100970 0x7fdd341972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34101330 0x7fdd34197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34110d90 con 0x7fdd34106980 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34110c10 con 0x7fdd34100970 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3c4f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd34110f10 con 0x7fdd34101330 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3aa8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3aa8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37706/0 (socket says 172.21.15.122:37706) 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3aa8f640 1 -- 172.21.15.122:0/558599844 learned_addr learned my addr 172.21.15.122:0/558599844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34100970 0x7fdd341972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.041+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34101330 0x7fdd34197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 -- 172.21.15.122:0/558599844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34101330 msgr2=0x7fdd34197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34101330 0x7fdd34197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 -- 172.21.15.122:0/558599844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34100970 msgr2=0x7fdd341972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34100970 0x7fdd341972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 -- 172.21.15.122:0/558599844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3419c2c0 con 0x7fdd34106980 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3aa8f640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00eb10 tx=0x7fdd2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd2b7fe640 1 -- 172.21.15.122:0/558599844 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c004b20 con 0x7fdd34106980 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd2b7fe640 1 -- 172.21.15.122:0/558599844 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c004cc0 con 0x7fdd34106980 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3419c5b0 con 0x7fdd34106980 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd2b7fe640 1 -- 172.21.15.122:0/558599844 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c010760 con 0x7fdd34106980 2024-04-02T13:39:38.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.042+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd341a3e40 con 0x7fdd34106980 2024-04-02T13:39:38.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.044+0000 7fdd2b7fe640 1 -- 172.21.15.122:0/558599844 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdd2c005020 con 0x7fdd34106980 2024-04-02T13:39:38.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 Processor -- start 2024-04-02T13:39:38.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 -- start start 2024-04-02T13:39:38.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411460 0x55cde1411840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde141c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd2b7fe640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 0x7fdd200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cde13757b0 con 0x55cde141ceb0 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cde1314570 con 0x55cde1411d80 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cde1371760 con 0x55cde1411460 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3aa8f640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 0x7fdd200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37718/0 (socket says 172.21.15.122:37718) 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde141c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd39a8d640 1 -- 172.21.15.122:0/2493532186 learned_addr learned my addr 172.21.15.122:0/2493532186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd2b7fe640 1 -- 172.21.15.122:0/558599844 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdd2c0d1090 con 0x7fdd34106980 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411460 0x55cde1411840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 -- 172.21.15.122:0/2493532186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411460 msgr2=0x55cde1411840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411460 0x55cde1411840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 -- 172.21.15.122:0/2493532186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 msgr2=0x55cde141f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.045+0000 7fdd3a28e640 1 -- 172.21.15.122:0/2493532186 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde10893f0 con 0x55cde1411d80 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.046+0000 7fdd3aa8f640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 0x7fdd200a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c09c9a0 tx=0x7fdd2c00efd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.046+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde141c970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009230 tx=0x7fdd30005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4981b10e1d18aa0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.046+0000 7fdd28bfe640 1 -- 172.21.15.122:0/2493532186 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30042070 con 0x55cde1411d80 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.046+0000 7fdd28bfe640 1 -- 172.21.15.122:0/2493532186 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd3002c690 con 0x55cde1411d80 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.046+0000 7fdd28bfe640 1 -- 172.21.15.122:0/2493532186 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd30004d40 con 0x55cde1411d80 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2493532186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 msgr2=0x55cde141c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde141c970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009230 tx=0x7fdd30005bf0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2493532186 shutdown_connections 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde141c970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2493532186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411460 0x55cde1411840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2493532186 >> 172.21.15.122:0/2493532186 conn(0x55cde140bed0 msgr2=0x55cde140c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2493532186 shutdown_connections 2024-04-02T13:39:38.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2493532186 wait complete. 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- start start 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 Processor -- start 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde1419cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cde141fc40 con 0x55cde141ceb0 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cde1330e80 con 0x55cde1411460 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cde141fdc0 con 0x55cde1411d80 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3f1ba740 10 client.0 _ll_register_callbacks cb 0x55cde136caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37728/0 (socket says 172.21.15.122:37728) 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd39a8d640 1 -- 172.21.15.122:0/489070098 learned_addr learned my addr 172.21.15.122:0/489070098 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.047+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde1419cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 -- 172.21.15.122:0/489070098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 msgr2=0x55cde1419cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde1419cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 -- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 msgr2=0x55cde141c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 -- 172.21.15.122:0/489070098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde11537d0 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005fd0 tx=0x7fdd24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a7ea476d6fad7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24010c60 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdd24010e00 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cde1382cf0 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd24004030 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 10 client.17037 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cde142bd70 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cde1382cf0 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fdd24021420 con 0x55cde141ceb0 2024-04-02T13:39:38.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 10 client.17037 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 10 client.17037 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 20 client.17037 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde142d230 0x55cde142f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 4 client.17037 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.048+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cde137d040 con 0x55cde142d230 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.049+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde142d230 0x55cde142f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.049+0000 7fdd1dffb640 10 client.17037 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.049+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde142d230 0x55cde142f6b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=549242f0ca8a0a03 server_cookie=a92ebc7551f5138c in_seq=0 out_seq=0 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd1dffb640 1 -- 172.21.15.122:0/489070098 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cde137d040 con 0x55cde142d230 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd1dffb640 10 client.17037 handle_command_reply: tid=0 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 client.17037 shutdown 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 msgr2=0x55cde141a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fdd24005fd0 tx=0x7fdd24004730 comp rx=0 tx=0).stop 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 shutdown_connections 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141a400 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde1419cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde142d230 0x55cde142f6b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/489070098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd1dffb640 0 client.17037 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 >> 172.21.15.122:0/489070098 conn(0x55cde140bed0 msgr2=0x55cde140dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 shutdown_connections 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/489070098 wait complete. 2024-04-02T13:39:38.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.050+0000 7fdd3f1ba740 20 client.17037 trim_cache size 0 max 16384 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.063+0000 7fdd3f1ba740 1 Processor -- start 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.063+0000 7fdd3f1ba740 1 -- start start 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde1425430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde141ceb0 0x55cde140adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341acd80 con 0x55cde1411460 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ae950 con 0x55cde1411d80 2024-04-02T13:39:38.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ad450 con 0x55cde141ceb0 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37732/0 (socket says 172.21.15.122:37732) 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 -- 172.21.15.122:0/3143993010 learned_addr learned my addr 172.21.15.122:0/3143993010 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde1425430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 -- 172.21.15.122:0/3143993010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde141ceb0 msgr2=0x55cde140adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde141ceb0 0x55cde140adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 -- 172.21.15.122:0/3143993010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 msgr2=0x55cde1425430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde1425430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 -- 172.21.15.122:0/3143993010 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cde14297b0 con 0x55cde1411460 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdd24000c00 tx=0x7fdd24012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eaadb116bbe9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd1ffff640 1 -- 172.21.15.122:0/3143993010 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24034070 con 0x55cde1411460 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd1ffff640 1 -- 172.21.15.122:0/3143993010 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2400b870 con 0x55cde1411460 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.064+0000 7fdd1ffff640 1 -- 172.21.15.122:0/3143993010 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd240035f0 con 0x55cde1411460 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/3143993010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 msgr2=0x55cde1428b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fdd24000c00 tx=0x7fdd24012580 comp rx=0 tx=0).stop 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/3143993010 shutdown_connections 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde141ceb0 0x55cde140adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411d80 0x55cde1425430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/3143993010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde1411460 0x55cde1428b40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/3143993010 >> 172.21.15.122:0/3143993010 conn(0x55cde140bed0 msgr2=0x55cde140a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/3143993010 shutdown_connections 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/3143993010 wait complete. 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- start start 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 Processor -- start 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde141bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341ae220 con 0x55cde141ceb0 2024-04-02T13:39:38.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341adb50 con 0x55cde1411460 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdd341aad60 con 0x55cde1411d80 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.065+0000 7fdd3f1ba740 10 client.0 _ll_register_callbacks cb 0x7fdd2400be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37744/0 (socket says 172.21.15.122:37744) 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 -- 172.21.15.122:0/2743541441 learned_addr learned my addr 172.21.15.122:0/2743541441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde141bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 -- 172.21.15.122:0/2743541441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 msgr2=0x55cde141bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde141bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 -- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 msgr2=0x55cde141eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 -- 172.21.15.122:0/2743541441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cde141d290 con 0x55cde141ceb0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd39a8d640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdd24012bc0 tx=0x7fdd24014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b769e3edb4417620 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24042060 con 0x55cde141ceb0 2024-04-02T13:39:38.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fdd24005240 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd2400b870 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd24034040 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd3f1ba740 10 client.17043 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cde1495990 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.066+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2400b870 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fdd2403a050 con 0x55cde141ceb0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 10 client.17043 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 10 client.17043 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 20 client.17043 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde1496080 0x55cde14984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 4 client.17043 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cde12fa6b0 con 0x55cde1496080 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde1496080 0x55cde14984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd1dffb640 10 client.17043 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3a28e640 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde1496080 0x55cde14984a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24ae1478aecc3d41 server_cookie=6993e130b0fc9e0f in_seq=0 out_seq=0 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd1dffb640 1 -- 172.21.15.122:0/2743541441 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cde12fa6b0 con 0x55cde1496080 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd1dffb640 10 client.17043 handle_command_reply: tid=0 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 client.17043 shutdown 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 msgr2=0x55cde141c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdd24012bc0 tx=0x7fdd24014040 comp rx=0 tx=0).stop 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 shutdown_connections 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cde141ceb0 0x55cde141c610 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cde1411d80 0x55cde141bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cde1496080 0x55cde14984a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 --2- 172.21.15.122:0/2743541441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cde1411460 0x55cde141eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 >> 172.21.15.122:0/2743541441 conn(0x55cde140bed0 msgr2=0x55cde1418cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.067+0000 7fdd1dffb640 0 client.17043 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:39:38.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 shutdown_connections 2024-04-02T13:39:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3f1ba740 1 -- 172.21.15.122:0/2743541441 wait complete. 2024-04-02T13:39:38.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3f1ba740 20 client.17043 trim_cache size 0 max 16384 2024-04-02T13:39:38.078 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 msgr2=0x7fdd200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 0x7fdd200a61e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c09c9a0 tx=0x7fdd2c00efd0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 msgr2=0x7fdd3419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00eb10 tx=0x7fdd2c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 shutdown_connections 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdd200a3d20 0x7fdd200a61e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd34106980 0x7fdd3419bb90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd34101330 0x7fdd34197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 --2- 172.21.15.122:0/558599844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd34100970 0x7fdd341972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 >> 172.21.15.122:0/558599844 conn(0x7fdd34077b00 msgr2=0x7fdd34103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 shutdown_connections 2024-04-02T13:39:38.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.068+0000 7fdd3c4f2640 1 -- 172.21.15.122:0/558599844 wait complete. 2024-04-02T13:39:38.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:39:38.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 -- 172.21.15.122:0/2223006436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718101330 msgr2=0x7f571810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2223006436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718101330 0x7f571810d840 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5710009cd0 tx=0x7f571002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 -- 172.21.15.122:0/2223006436 shutdown_connections 2024-04-02T13:39:38.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2223006436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718101330 0x7f571810d840 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2223006436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 0x7f5718100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2223006436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718106980 0x7f5718106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.222+0000 7f571ff3d640 1 -- 172.21.15.122:0/2223006436 >> 172.21.15.122:0/2223006436 conn(0x7f5718077b00 msgr2=0x7f5718077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- 172.21.15.122:0/2223006436 shutdown_connections 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- 172.21.15.122:0/2223006436 wait complete. 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 Processor -- start 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- start start 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 0x7f5718198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 0x7f57181988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110d90 con 0x7f5718106980 2024-04-02T13:39:38.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110c10 con 0x7f5718100970 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571ff3d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5718110f10 con 0x7f5718101330 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571e4da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571e4da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37760/0 (socket says 172.21.15.122:37760) 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571e4da640 1 -- 172.21.15.122:0/2155545237 learned_addr learned my addr 172.21.15.122:0/2155545237 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571dcd9640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 0x7f5718198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.223+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 0x7f57181988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 -- 172.21.15.122:0/2155545237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 msgr2=0x7f57181988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 0x7f57181988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 -- 172.21.15.122:0/2155545237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 msgr2=0x7f5718198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 0x7f5718198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 -- 172.21.15.122:0/2155545237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571819d370 con 0x7f5718106980 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 0x7f57181988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571e4da640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f57100348f0 tx=0x7f5710032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f570effd640 1 -- 172.21.15.122:0/2155545237 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710041020 con 0x7f5718106980 2024-04-02T13:39:38.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f570effd640 1 -- 172.21.15.122:0/2155545237 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5710034cd0 con 0x7f5718106980 2024-04-02T13:39:38.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f570effd640 1 -- 172.21.15.122:0/2155545237 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710039040 con 0x7f5718106980 2024-04-02T13:39:38.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571819d600 con 0x7f5718106980 2024-04-02T13:39:38.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.224+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181a4ef0 con 0x7f5718106980 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.226+0000 7f570effd640 1 -- 172.21.15.122:0/2155545237 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5710044020 con 0x7f5718106980 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.226+0000 7f5722bfe740 1 Processor -- start 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 -- start start 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871643760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871646090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d87159e7b0 con 0x55d87163a7a0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d87153d570 con 0x55d87163b0c0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f5722bfe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d87159a760 con 0x55d871643ca0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f570effd640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 0x7f57040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37772/0 (socket says 172.21.15.122:37772) 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571dcd9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871643760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 -- 172.21.15.122:0/4015708845 learned_addr learned my addr 172.21.15.122:0/4015708845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571e4da640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 0x7f57040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871646090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 -- 172.21.15.122:0/4015708845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 msgr2=0x55d871646090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871646090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 -- 172.21.15.122:0/4015708845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 msgr2=0x55d871643760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871643760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 -- 172.21.15.122:0/4015708845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8712b23f0 con 0x55d87163a7a0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571dcd9640 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871643760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f570effd640 1 -- 172.21.15.122:0/2155545237 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f57100f2920 con 0x7f5718106980 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f570800be80 tx=0x7f570800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a5dcf7b807e03e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.227+0000 7f571e4da640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 0x7f57040a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f57100bb010 tx=0x7f5710090080 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5703fff640 1 -- 172.21.15.122:0/4015708845 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708019030 con 0x55d87163a7a0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5703fff640 1 -- 172.21.15.122:0/4015708845 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5708011a20 con 0x55d87163a7a0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5703fff640 1 -- 172.21.15.122:0/4015708845 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570800f9e0 con 0x55d87163a7a0 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 -- 172.21.15.122:0/4015708845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 msgr2=0x55d87163ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f570800be80 tx=0x7f570800b730 comp rx=0 tx=0).stop 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 -- 172.21.15.122:0/4015708845 shutdown_connections 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871646090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871643760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4015708845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163ab80 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 -- 172.21.15.122:0/4015708845 >> 172.21.15.122:0/4015708845 conn(0x55d871634ed0 msgr2=0x55d8716352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 -- 172.21.15.122:0/4015708845 shutdown_connections 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.228+0000 7f5722bfe740 1 -- 172.21.15.122:0/4015708845 wait complete. 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 -- start start 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 Processor -- start 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163a7a0 0x55d871643710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871640aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d871646a90 con 0x55d871643ca0 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d871559e80 con 0x55d87163b0c0 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d871646c10 con 0x55d87163a7a0 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f5722bfe740 10 client.0 _ll_register_callbacks cb 0x55d871595aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37782/0 (socket says 172.21.15.122:37782) 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 -- 172.21.15.122:0/2863862539 learned_addr learned my addr 172.21.15.122:0/2863862539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163a7a0 0x55d871643710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571dcd9640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871640aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 -- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163a7a0 msgr2=0x55d871643710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163a7a0 0x55d871643710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 -- 172.21.15.122:0/2863862539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 msgr2=0x55d871640aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.229+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871640aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f571d4d8640 1 -- 172.21.15.122:0/2863862539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87137c7d0 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f571d4d8640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f57080134f0 tx=0x7f570800f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b3f3912fabacc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708019040 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5708013020 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5708011a20 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5708013220 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 10 client.17055 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d871652b50 con 0x55d871643ca0 2024-04-02T13:39:38.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5708011a20 con 0x55d871643ca0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f570800f7d0 con 0x55d871643ca0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 10 client.17055 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 10 client.17055 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 20 client.17055 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d871653fb0 0x55d8716563a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 4 client.17055 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.230+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8715abcf0 con 0x55d871653fb0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.231+0000 7f571dcd9640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d871653fb0 0x55d8716563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.231+0000 7f57017fa640 10 client.17055 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.231+0000 7f571dcd9640 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d871653fb0 0x55d8716563a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3a4db9a2a0b0428 server_cookie=b3c7b518a968a82e in_seq=0 out_seq=0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f57017fa640 1 -- 172.21.15.122:0/2863862539 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8715abcf0 con 0x55d871653fb0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f57017fa640 10 client.17055 handle_command_reply: tid=0 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 client.17055 shutdown 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 msgr2=0x55d8716411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f57080134f0 tx=0x7f570800f680 comp rx=0 tx=0).stop 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 shutdown_connections 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d871653fb0 0x55d8716563a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d871643ca0 0x55d8716411e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d871640aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f57017fa640 0 client.17055 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 --2- 172.21.15.122:0/2863862539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163a7a0 0x55d871643710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 >> 172.21.15.122:0/2863862539 conn(0x55d871634ed0 msgr2=0x55d871636c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 shutdown_connections 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 1 -- 172.21.15.122:0/2863862539 wait complete. 2024-04-02T13:39:38.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.232+0000 7f5722bfe740 20 client.17055 trim_cache size 0 max 16384 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 Processor -- start 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 -- start start 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d8716431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871643710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57181aefb0 con 0x55d87163a7a0 2024-04-02T13:39:38.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57181b0b80 con 0x55d87163b0c0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f5722bfe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57181af680 con 0x55d871643ca0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37784/0 (socket says 172.21.15.122:37784) 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 -- 172.21.15.122:0/4181854380 learned_addr learned my addr 172.21.15.122:0/4181854380 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571dcd9640 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d8716431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 -- 172.21.15.122:0/4181854380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 msgr2=0x55d871643710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871643710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 -- 172.21.15.122:0/4181854380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 msgr2=0x55d8716431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d8716431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.245+0000 7f571d4d8640 1 -- 172.21.15.122:0/4181854380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8716505f0 con 0x55d87163a7a0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f571dcd9640 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d8716431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f571d4d8640 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5708013430 tx=0x7f570800f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cb850b09ece056 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f57037fe640 1 -- 172.21.15.122:0/4181854380 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708019070 con 0x55d87163a7a0 2024-04-02T13:39:38.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f57037fe640 1 -- 172.21.15.122:0/4181854380 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570800fe60 con 0x55d87163a7a0 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f57037fe640 1 -- 172.21.15.122:0/4181854380 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57080117a0 con 0x55d87163a7a0 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 -- 172.21.15.122:0/4181854380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 msgr2=0x55d87163fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5708013430 tx=0x7f570800f4e0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 -- 172.21.15.122:0/4181854380 shutdown_connections 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d871643ca0 0x55d871643710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d87163b0c0 0x55d8716431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 --2- 172.21.15.122:0/4181854380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d87163fa80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 -- 172.21.15.122:0/4181854380 >> 172.21.15.122:0/4181854380 conn(0x55d871634ed0 msgr2=0x55d87163f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 -- 172.21.15.122:0/4181854380 shutdown_connections 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.246+0000 7f5722bfe740 1 -- 172.21.15.122:0/4181854380 wait complete. 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 -- start start 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 Processor -- start 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163b0c0 0x55d871659910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d871643ca0 0x55d87163d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57181b0450 con 0x55d87163a7a0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57181afd80 con 0x55d871643ca0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57181acf90 con 0x55d87163b0c0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f5722bfe740 10 client.0 _ll_register_callbacks cb 0x7f5714004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37790/0 (socket says 172.21.15.122:37790) 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 -- 172.21.15.122:0/226231075 learned_addr learned my addr 172.21.15.122:0/226231075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d871643ca0 0x55d87163d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 -- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163b0c0 msgr2=0x55d871659910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163b0c0 0x55d871659910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 -- 172.21.15.122:0/226231075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d871643ca0 msgr2=0x55d87163d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d871643ca0 0x55d87163d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 -- 172.21.15.122:0/226231075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d87163d950 con 0x55d87163a7a0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f571d4d8640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f570800c010 tx=0x7f57080036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a479cdffe206a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.247+0000 7f57017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570803a040 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5708011740 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f570800fe60 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5708019040 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 10 client.17061 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d8716b9bd0 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f570800fe60 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f5708028040 con 0x55d87163a7a0 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 10 client.17061 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 10 client.17061 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 20 client.17061 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d8716ba230 0x55d8716bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 4 client.17061 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8715a6040 con 0x55d8716ba230 2024-04-02T13:39:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.248+0000 7f571dcd9640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d8716ba230 0x55d8716bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f571dcd9640 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d8716ba230 0x55d8716bc620 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7554220277badcd1 server_cookie=12eed48dafcf352 in_seq=0 out_seq=0 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f57017fa640 10 client.17061 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f57017fa640 1 -- 172.21.15.122:0/226231075 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8715a6040 con 0x55d8716ba230 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f57017fa640 10 client.17061 handle_command_reply: tid=0 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 client.17061 shutdown 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 msgr2=0x55d8716593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f570800c010 tx=0x7f57080036b0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 shutdown_connections 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d8716ba230 0x55d8716bc620 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d871643ca0 0x55d87163d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f57017fa640 0 client.17061 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d87163b0c0 0x55d871659910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 --2- 172.21.15.122:0/226231075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d87163a7a0 0x55d8716593d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 >> 172.21.15.122:0/226231075 conn(0x55d871634ed0 msgr2=0x55d87164ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 shutdown_connections 2024-04-02T13:39:38.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 1 -- 172.21.15.122:0/226231075 wait complete. 2024-04-02T13:39:38.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.249+0000 7f5722bfe740 20 client.17061 trim_cache size 0 max 16384 2024-04-02T13:39:38.259 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 msgr2=0x7f57040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 0x7f57040a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f57100bb010 tx=0x7f5710090080 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 msgr2=0x7f571819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f57100348f0 tx=0x7f5710032ec0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 shutdown_connections 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f57040a3d20 0x7f57040a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5718106980 0x7f571819cc40 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5718101330 0x7f57181988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 --2- 172.21.15.122:0/2155545237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5718100970 0x7f5718198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 >> 172.21.15.122:0/2155545237 conn(0x7f5718077b00 msgr2=0x7f5718068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 shutdown_connections 2024-04-02T13:39:38.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.250+0000 7f571ff3d640 1 -- 172.21.15.122:0/2155545237 wait complete. 2024-04-02T13:39:38.262 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:38 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:38.262 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:38 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1555015701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:38.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:39:38.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 -- 172.21.15.122:0/1216365492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c1069a0 msgr2=0x7ff10c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 --2- 172.21.15.122:0/1216365492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c1069a0 0x7ff10c106d80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009930 tx=0x7ff0fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 -- 172.21.15.122:0/1216365492 shutdown_connections 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 --2- 172.21.15.122:0/1216365492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c101350 0x7ff10c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 --2- 172.21.15.122:0/1216365492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c100990 0x7ff10c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 --2- 172.21.15.122:0/1216365492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c1069a0 0x7ff10c106d80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 -- 172.21.15.122:0/1216365492 >> 172.21.15.122:0/1216365492 conn(0x7ff10c077ad0 msgr2=0x7ff10c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 -- 172.21.15.122:0/1216365492 shutdown_connections 2024-04-02T13:39:38.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.364+0000 7ff1109d4640 1 -- 172.21.15.122:0/1216365492 wait complete. 2024-04-02T13:39:38.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 Processor -- start 2024-04-02T13:39:38.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 -- start start 2024-04-02T13:39:38.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c101350 0x7ff10c1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c1069a0 0x7ff10c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110db0 con 0x7ff10c100990 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110c30 con 0x7ff10c1069a0 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff1109d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c110f30 con 0x7ff10c101350 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37806/0 (socket says 172.21.15.122:37806) 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff10a59c640 1 -- 172.21.15.122:0/4004682186 learned_addr learned my addr 172.21.15.122:0/4004682186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c1069a0 0x7ff10c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.365+0000 7ff103fff640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c101350 0x7ff10c1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 -- 172.21.15.122:0/4004682186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c101350 msgr2=0x7ff10c1977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c101350 0x7ff10c1977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 -- 172.21.15.122:0/4004682186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c1069a0 msgr2=0x7ff10c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c1069a0 0x7ff10c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 -- 172.21.15.122:0/4004682186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c19c2a0 con 0x7ff10c100990 2024-04-02T13:39:38.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff10a59c640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009230 tx=0x7ff0fc02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff1037fe640 1 -- 172.21.15.122:0/4004682186 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc041070 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff1037fe640 1 -- 172.21.15.122:0/4004682186 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0fc004450 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c19c530 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff1037fe640 1 -- 172.21.15.122:0/4004682186 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc034440 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.366+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1a3e20 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.368+0000 7ff1037fe640 1 -- 172.21.15.122:0/4004682186 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff0fc044070 con 0x7ff10c100990 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.368+0000 7ff1135be740 1 Processor -- start 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 -- start start 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffde490 0x555f1ffde870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 0x555f1ffe9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1037fe640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 0x7ff0f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555f1ff427b0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f1fee1570 con 0x555f1ffde490 2024-04-02T13:39:38.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1135be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555f1ff3e760 con 0x555f1ffdedb0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff103fff640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 0x7ff0f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37808/0 (socket says 172.21.15.122:37808) 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 -- 172.21.15.122:0/350579625 learned_addr learned my addr 172.21.15.122:0/350579625 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffde490 0x555f1ffde870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff1037fe640 1 -- 172.21.15.122:0/4004682186 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff0fc0f2420 con 0x7ff10c100990 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10a59c640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 0x555f1ffe9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 -- 172.21.15.122:0/350579625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 msgr2=0x555f1ffe9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 0x555f1ffe9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 -- 172.21.15.122:0/350579625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffde490 msgr2=0x555f1ffde870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffde490 0x555f1ffde870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10ad9d640 1 -- 172.21.15.122:0/350579625 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1fc563f0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff10a59c640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 0x555f1ffe9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:38.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.369+0000 7ff103fff640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 0x7ff0f80a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff10c1050c0 tx=0x7ff0f400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff104005f40 tx=0x7ff104005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24e3068ad62cf81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff100ff9640 1 -- 172.21.15.122:0/350579625 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104011130 con 0x555f1ffe9eb0 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff100ff9640 1 -- 172.21.15.122:0/350579625 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1040112d0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff100ff9640 1 -- 172.21.15.122:0/350579625 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400fb80 con 0x555f1ffe9eb0 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 -- 172.21.15.122:0/350579625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 msgr2=0x555f1ffec2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff104005f40 tx=0x7ff104005bd0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 -- 172.21.15.122:0/350579625 shutdown_connections 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffec2a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffdedb0 0x555f1ffe9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 --2- 172.21.15.122:0/350579625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffde490 0x555f1ffde870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.370+0000 7ff1135be740 1 -- 172.21.15.122:0/350579625 >> 172.21.15.122:0/350579625 conn(0x555f1ffd8ed0 msgr2=0x555f1ffd92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- 172.21.15.122:0/350579625 shutdown_connections 2024-04-02T13:39:38.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- 172.21.15.122:0/350579625 wait complete. 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- start start 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 Processor -- start 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffe9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffe6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555f1ffecca0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f1fefde80 con 0x555f1ffdedb0 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555f1ffece20 con 0x555f1ffde490 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff1135be740 10 client.0 _ll_register_callbacks cb 0x555f1ff39aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37824/0 (socket says 172.21.15.122:37824) 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 -- 172.21.15.122:0/1509017494 learned_addr learned my addr 172.21.15.122:0/1509017494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff103fff640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffe6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10a59c640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffe9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 -- 172.21.15.122:0/1509017494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 msgr2=0x555f1ffe9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffe9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 -- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 msgr2=0x555f1ffe6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffe6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.371+0000 7ff10ad9d640 1 -- 172.21.15.122:0/1509017494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1fd207d0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff104011920 tx=0x7ff104004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696f96e8147c0b75 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104011130 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1040112d0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff104002030 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff10400f960 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 10 client.17073 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f1fff8cb0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1040112d0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ff10401d020 con 0x555f1ffe9eb0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 10 client.17073 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 20 client.17073 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f1fffa0b0 0x555f1fffc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 4 client.17073 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.372+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f1ff4fcf0 con 0x555f1fffa0b0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.373+0000 7ff10a59c640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f1fffa0b0 0x555f1fffc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.373+0000 7ff10a59c640 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f1fffa0b0 0x555f1fffc4a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c1324ad49861b6 server_cookie=37079299c835ea43 in_seq=0 out_seq=0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.373+0000 7ff0f1ffb640 10 client.17073 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.373+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/1509017494 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f1ff4fcf0 con 0x555f1fffa0b0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.373+0000 7ff0f1ffb640 10 client.17073 handle_command_reply: tid=0 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 client.17073 shutdown 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 msgr2=0x555f1ffe7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7ff104011920 tx=0x7ff104004f60 comp rx=0 tx=0).stop 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 shutdown_connections 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f1fffa0b0 0x555f1fffc4a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe7210 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffe6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff0f1ffb640 0 client.17073 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 --2- 172.21.15.122:0/1509017494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffe9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 >> 172.21.15.122:0/1509017494 conn(0x555f1ffd8ed0 msgr2=0x555f1ffdac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 shutdown_connections 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 1 -- 172.21.15.122:0/1509017494 wait complete. 2024-04-02T13:39:38.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.374+0000 7ff1135be740 20 client.17073 trim_cache size 0 max 16384 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 Processor -- start 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 -- start start 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1fff5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffd8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1acd70 con 0x555f1ffe9eb0 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ae940 con 0x555f1ffdedb0 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff1135be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ad440 con 0x555f1ffde490 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37834/0 (socket says 172.21.15.122:37834) 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 -- 172.21.15.122:0/3140854546 learned_addr learned my addr 172.21.15.122:0/3140854546 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff103fff640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffd8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10a59c640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1fff5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 -- 172.21.15.122:0/3140854546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 msgr2=0x555f1fff5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1fff5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 -- 172.21.15.122:0/3140854546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 msgr2=0x555f1ffd8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffd8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10ad9d640 1 -- 172.21.15.122:0/3140854546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f1ffdebe0 con 0x555f1ffe9eb0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.387+0000 7ff10a59c640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1fff5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff104009a70 tx=0x7ff104005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575e1e6d6ac8345e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff0f3fff640 1 -- 172.21.15.122:0/3140854546 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104009480 con 0x555f1ffe9eb0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff0f3fff640 1 -- 172.21.15.122:0/3140854546 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff104009e20 con 0x555f1ffe9eb0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff0f3fff640 1 -- 172.21.15.122:0/3140854546 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104003b80 con 0x555f1ffe9eb0 2024-04-02T13:39:38.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 -- 172.21.15.122:0/3140854546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 msgr2=0x555f1ffe9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7ff104009a70 tx=0x7ff104005c80 comp rx=0 tx=0).stop 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 -- 172.21.15.122:0/3140854546 shutdown_connections 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffe9eb0 0x555f1ffe9330 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffdedb0 0x555f1ffd8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.388+0000 7ff1135be740 1 --2- 172.21.15.122:0/3140854546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1fff5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- 172.21.15.122:0/3140854546 >> 172.21.15.122:0/3140854546 conn(0x555f1ffd8ed0 msgr2=0x555f1ffe5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- 172.21.15.122:0/3140854546 shutdown_connections 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- 172.21.15.122:0/3140854546 wait complete. 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- start start 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 Processor -- start 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffe9eb0 0x555f1ffea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1ae210 con 0x555f1ffdedb0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1adb40 con 0x555f1ffe9eb0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1aad50 con 0x555f1ffde490 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff1135be740 10 client.0 _ll_register_callbacks cb 0x7ff1040115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.389+0000 7ff103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff10a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffebb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:35900/0 (socket says 172.21.15.122:35900) 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff10a59c640 1 -- 172.21.15.122:0/232743550 learned_addr learned my addr 172.21.15.122:0/232743550 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37838/0 (socket says 172.21.15.122:37838) 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff10ad9d640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffe9eb0 0x555f1ffea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 -- 172.21.15.122:0/232743550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 msgr2=0x555f1ffebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 -- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffe9eb0 msgr2=0x555f1ffea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffe9eb0 0x555f1ffea7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 -- 172.21.15.122:0/232743550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f1ffeb000 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff103fff640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005d80 tx=0x7ff0f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be93ecaec8f6c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff0f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4005020 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff0f40051c0 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff104009e20 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0f4032050 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff1135be740 10 client.17079 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.390+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f1fff20d0 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff104009e20 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7ff0f401f640 con 0x555f1ffdedb0 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 10 client.17079 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 20 client.17079 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:38.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f20066c30 0x555f20069020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 4 client.17079 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f1ff4a040 con 0x555f20066c30 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff10a59c640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f20066c30 0x555f20069020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff10a59c640 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f20066c30 0x555f20069020 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7addbab4faa45d98 server_cookie=124d0e79b8aef1b1 in_seq=0 out_seq=0 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.391+0000 7ff0f1ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff0f1ffb640 1 -- 172.21.15.122:0/232743550 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f1ff4a040 con 0x555f20066c30 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff0f1ffb640 10 client.17079 handle_command_reply: tid=0 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 client.17079 shutdown 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 msgr2=0x555f1ffea290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005d80 tx=0x7ff0f4004730 comp rx=0 tx=0).stop 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 shutdown_connections 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555f20066c30 0x555f20069020 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f1ffe9eb0 0x555f1ffea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff0f1ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:38.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f1ffdedb0 0x555f1ffea290 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 --2- 172.21.15.122:0/232743550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f1ffde490 0x555f1ffebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 >> 172.21.15.122:0/232743550 conn(0x555f1ffd8ed0 msgr2=0x555f1ffe6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 shutdown_connections 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 1 -- 172.21.15.122:0/232743550 wait complete. 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.392+0000 7ff1135be740 20 client.17079 trim_cache size 0 max 16384 2024-04-02T13:39:38.402 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 msgr2=0x7ff0f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 0x7ff0f80a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff10c1050c0 tx=0x7ff0f400b390 comp rx=0 tx=0).stop 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 msgr2=0x7ff10c1972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc009230 tx=0x7ff0fc02cba0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 shutdown_connections 2024-04-02T13:39:38.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0f80a3d20 0x7ff0f80a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff10c1069a0 0x7ff10c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff10c101350 0x7ff10c1977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 --2- 172.21.15.122:0/4004682186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff10c100990 0x7ff10c1972b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 >> 172.21.15.122:0/4004682186 conn(0x7ff10c077ad0 msgr2=0x7ff10c103240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 shutdown_connections 2024-04-02T13:39:38.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:38.393+0000 7ff1109d4640 1 -- 172.21.15.122:0/4004682186 wait complete. 2024-04-02T13:39:38.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:38 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:38.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:38 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1555015701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:38.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:38 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:38.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:38 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1555015701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:39:38.950 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128449 2.59 MB/sec execute 59 sec latency 3351.036 ms 2024-04-02T13:39:39.409 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:39:39.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:39:39.409 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:39:39.438 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:39 smithi122 ceph-mon[32114]: pgmap v137: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 900 KiB/s wr, 80 op/s 2024-04-02T13:39:39.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:39 smithi187 ceph-mon[36064]: pgmap v137: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 900 KiB/s wr, 80 op/s 2024-04-02T13:39:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:39 smithi188 ceph-mon[36117]: pgmap v137: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 900 KiB/s wr, 80 op/s 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.522+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1515045374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4841003c0 msgr2=0x7fe484100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.522+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1515045374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4841003c0 0x7fe484100820 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe47400aa10 tx=0x7fe47402cf70 comp rx=0 tx=0).stop 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1515045374 shutdown_connections 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1515045374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe484100d60 0x7fe48410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1515045374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4841003c0 0x7fe484100820 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1515045374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484105130 0x7fe484105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1515045374 >> 172.21.15.122:0/1515045374 conn(0x7fe484075d40 msgr2=0x7fe484076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1515045374 shutdown_connections 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1515045374 wait complete. 2024-04-02T13:39:39.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 Processor -- start 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- start start 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 0x7fe484198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484100d60 0x7fe484198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410fcf0 con 0x7fe484105130 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410fb70 con 0x7fe484100d60 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48c1a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe48410fe70 con 0x7fe4841003c0 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.523+0000 7fe48a745640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37086/0 (socket says 172.21.15.122:37086) 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 -- 172.21.15.122:0/1491531877 learned_addr learned my addr 172.21.15.122:0/1491531877 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe489743640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484100d60 0x7fe484198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe489f44640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 0x7fe484198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 -- 172.21.15.122:0/1491531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 msgr2=0x7fe484198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 0x7fe484198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 -- 172.21.15.122:0/1491531877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484100d60 msgr2=0x7fe484198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484100d60 0x7fe484198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 -- 172.21.15.122:0/1491531877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48419d310 con 0x7fe484105130 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe489f44640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 0x7fe484198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48a745640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00eaa0 tx=0x7fe47c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe47affd640 1 -- 172.21.15.122:0/1491531877 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c00cdb0 con 0x7fe484105130 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe47affd640 1 -- 172.21.15.122:0/1491531877 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe47c005020 con 0x7fe484105130 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48419d600 con 0x7fe484105130 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe47affd640 1 -- 172.21.15.122:0/1491531877 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0116b0 con 0x7fe484105130 2024-04-02T13:39:39.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.524+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4841a4e90 con 0x7fe484105130 2024-04-02T13:39:39.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.526+0000 7fe47affd640 1 -- 172.21.15.122:0/1491531877 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe47c005560 con 0x7fe484105130 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 Processor -- start 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 -- start start 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe47affd640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 0x7fe4700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f05280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f07bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563872e60770 con 0x563872efc2c0 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563872e08960 con 0x563872efcbe0 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.527+0000 7fe48ed91740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563872e5c8c0 con 0x563872f057c0 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489743640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 0x7fe4700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37092/0 (socket says 172.21.15.122:37092) 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 -- 172.21.15.122:0/3678852000 learned_addr learned my addr 172.21.15.122:0/3678852000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe47affd640 1 -- 172.21.15.122:0/1491531877 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe47c0d0da0 con 0x7fe484105130 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe48a745640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f07bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489743640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f05280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 -- 172.21.15.122:0/3678852000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 msgr2=0x563872f07bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f07bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 -- 172.21.15.122:0/3678852000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 msgr2=0x563872f05280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f05280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 -- 172.21.15.122:0/3678852000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563872c3c7e0 con 0x563872efc2c0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489743640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f05280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489f44640 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe480009d50 tx=0x7fe4800117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dade7a721cfb6d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe489743640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 0x7fe4700a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe474006010 tx=0x7fe474005bb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe46ffff640 1 -- 172.21.15.122:0/3678852000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480012650 con 0x563872efc2c0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe46ffff640 1 -- 172.21.15.122:0/3678852000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe480009e40 con 0x563872efc2c0 2024-04-02T13:39:39.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.528+0000 7fe46ffff640 1 -- 172.21.15.122:0/3678852000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48000fc60 con 0x563872efc2c0 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- 172.21.15.122:0/3678852000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 msgr2=0x563872efc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fe480009d50 tx=0x7fe4800117b0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- 172.21.15.122:0/3678852000 shutdown_connections 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f07bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f05280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3678852000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872efc6a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- 172.21.15.122:0/3678852000 >> 172.21.15.122:0/3678852000 conn(0x563872ef69f0 msgr2=0x563872ef6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- 172.21.15.122:0/3678852000 shutdown_connections 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- 172.21.15.122:0/3678852000 wait complete. 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.529+0000 7fe48ed91740 1 -- start start 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 Processor -- start 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563872f085b0 con 0x563872efc2c0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563872e1bd20 con 0x563872efcbe0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563872f08730 con 0x563872f057c0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe48ed91740 10 client.0 _ll_register_callbacks cb 0x563872de8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37102/0 (socket says 172.21.15.122:37102) 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 -- 172.21.15.122:0/1600779604 learned_addr learned my addr 172.21.15.122:0/1600779604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489743640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 -- 172.21.15.122:0/1600779604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 msgr2=0x563872f031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 -- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 msgr2=0x563872f02ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f02ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489f44640 1 -- 172.21.15.122:0/1600779604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563872c9cf70 con 0x563872efc2c0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.530+0000 7fe489743640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe489f44640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe480005940 tx=0x7fe480005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d2d3aa5342ca57 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480028030 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe48000fbb0 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563872e6dcb0 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe48000fdb0 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 10 client.17091 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563872b34170 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563872e6dcb0 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe480004c90 con 0x563872efc2c0 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 10 client.17091 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 20 client.17091 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f15a40 0x563872f17e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 4 client.17091 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563872e68000 con 0x563872f15a40 2024-04-02T13:39:39.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.531+0000 7fe489743640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f15a40 0x563872f17e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.532+0000 7fe46d7fa640 10 client.17091 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.532+0000 7fe489743640 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f15a40 0x563872f17e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4755298732df65ca server_cookie=b9ad771cf01b1108 in_seq=0 out_seq=0 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe46d7fa640 1 -- 172.21.15.122:0/1600779604 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563872e68000 con 0x563872f15a40 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe46d7fa640 10 client.17091 handle_command_reply: tid=0 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 client.17091 shutdown 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 msgr2=0x563872f02760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe480005940 tx=0x7fe480005710 comp rx=0 tx=0).stop 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 shutdown_connections 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f15a40 0x563872f17e30 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872f057c0 0x563872f031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe46d7fa640 0 client.17091 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872efcbe0 0x563872f02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 --2- 172.21.15.122:0/1600779604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f02760 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 >> 172.21.15.122:0/1600779604 conn(0x563872ef69f0 msgr2=0x563872ef8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 shutdown_connections 2024-04-02T13:39:39.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 1 -- 172.21.15.122:0/1600779604 wait complete. 2024-04-02T13:39:39.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.533+0000 7fe48ed91740 20 client.17091 trim_cache size 0 max 16384 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 Processor -- start 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 -- start start 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f1d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841aefc0 con 0x563872efc2c0 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841b0b90 con 0x563872f057c0 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.550+0000 7fe48ed91740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841af690 con 0x563872efcbe0 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37112/0 (socket says 172.21.15.122:37112) 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 -- 172.21.15.122:0/3576915530 learned_addr learned my addr 172.21.15.122:0/3576915530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe48a745640 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f1d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 -- 172.21.15.122:0/3576915530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 msgr2=0x563872f114a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 -- 172.21.15.122:0/3576915530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 msgr2=0x563872f1d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f1d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 -- 172.21.15.122:0/3576915530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563872f181a0 con 0x563872efc2c0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe489f44640 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe4800117f0 tx=0x7fe48000ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323d182fc914036 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe46f7fe640 1 -- 172.21.15.122:0/3576915530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48003c070 con 0x563872efc2c0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe46f7fe640 1 -- 172.21.15.122:0/3576915530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe480029040 con 0x563872efc2c0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.551+0000 7fe46f7fe640 1 -- 172.21.15.122:0/3576915530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480028030 con 0x563872efc2c0 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- 172.21.15.122:0/3576915530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 msgr2=0x563872f04e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe4800117f0 tx=0x7fe48000ff50 comp rx=0 tx=0).stop 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- 172.21.15.122:0/3576915530 shutdown_connections 2024-04-02T13:39:39.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f1d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f114a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3576915530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f04e90 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- 172.21.15.122:0/3576915530 >> 172.21.15.122:0/3576915530 conn(0x563872ef69f0 msgr2=0x563872ef5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- 172.21.15.122:0/3576915530 shutdown_connections 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- 172.21.15.122:0/3576915530 wait complete. 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- start start 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 Processor -- start 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f09970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f09eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841b0460 con 0x563872efc2c0 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841afd90 con 0x563872f057c0 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.552+0000 7fe48ed91740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4841acfa0 con 0x563872efcbe0 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe48ed91740 10 client.0 _ll_register_callbacks cb 0x7fe480009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:39.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37128/0 (socket says 172.21.15.122:37128) 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 -- 172.21.15.122:0/3194697073 learned_addr learned my addr 172.21.15.122:0/3194697073 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489743640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f09970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe48a745640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f09eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 -- 172.21.15.122:0/3194697073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 msgr2=0x563872f09970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f09970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 -- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 msgr2=0x563872f09eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f09eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 -- 172.21.15.122:0/3194697073 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563872f7ae90 con 0x563872efc2c0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe489f44640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe480003d80 tx=0x7fe480005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52fceac16972ce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe46d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480044060 con 0x563872efc2c0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fe480003880 con 0x563872efc2c0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe480029040 con 0x563872efc2c0 2024-04-02T13:39:39.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe48003c070 con 0x563872efc2c0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe48ed91740 10 client.17097 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563872f7c190 con 0x563872efc2c0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.553+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe480029040 con 0x563872efc2c0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7fe4800124e0 con 0x563872efc2c0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 10 client.17097 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 20 client.17097 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f7c6c0 0x563872f7eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 4 client.17097 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563872de5be0 con 0x563872f7c6c0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe489743640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f7c6c0 0x563872f7eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe46d7fa640 10 client.17097 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.554+0000 7fe489743640 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f7c6c0 0x563872f7eae0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ed7c5b9d497f90 server_cookie=44f7b10caf33e074 in_seq=0 out_seq=0 2024-04-02T13:39:39.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.555+0000 7fe46d7fa640 1 -- 172.21.15.122:0/3194697073 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563872de5be0 con 0x563872f7c6c0 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.555+0000 7fe46d7fa640 10 client.17097 handle_command_reply: tid=0 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.555+0000 7fe48ed91740 1 client.17097 shutdown 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.555+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 msgr2=0x563872f09430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.555+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe480003d80 tx=0x7fe480005b30 comp rx=0 tx=0).stop 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 shutdown_connections 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563872f7c6c0 0x563872f7eae0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563872f057c0 0x563872f09eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563872efcbe0 0x563872f09970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 --2- 172.21.15.122:0/3194697073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563872efc2c0 0x563872f09430 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe46d7fa640 0 client.17097 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 >> 172.21.15.122:0/3194697073 conn(0x563872ef69f0 msgr2=0x563872f02550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 shutdown_connections 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 1 -- 172.21.15.122:0/3194697073 wait complete. 2024-04-02T13:39:39.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48ed91740 20 client.17097 trim_cache size 0 max 16384 2024-04-02T13:39:39.565 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:39:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "3129f011-afa0-413d-a491-aedc2bdf99ec", 2024-04-02T13:39:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:39:39.565 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 msgr2=0x7fe4700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 0x7fe4700a6260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe474006010 tx=0x7fe474005bb0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 msgr2=0x7fe48419cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00eaa0 tx=0x7fe47c00ef60 comp rx=0 tx=0).stop 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 shutdown_connections 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe4700a3da0 0x7fe4700a6260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe484105130 0x7fe48419cbe0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe484100d60 0x7fe484198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 --2- 172.21.15.122:0/1491531877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4841003c0 0x7fe484198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 >> 172.21.15.122:0/1491531877 conn(0x7fe484075d40 msgr2=0x7fe4840fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 shutdown_connections 2024-04-02T13:39:39.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.556+0000 7fe48c1a8640 1 -- 172.21.15.122:0/1491531877 wait complete. 2024-04-02T13:39:39.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:39:39.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:39:39.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- 172.21.15.122:0/904089619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 msgr2=0x7f2b64100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 --2- 172.21.15.122:0/904089619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b64100e30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009930 tx=0x7f2b4c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:39:39.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- 172.21.15.122:0/904089619 shutdown_connections 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 --2- 172.21.15.122:0/904089619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b64101370 0x7f2b6410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 --2- 172.21.15.122:0/904089619 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b64100e30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 --2- 172.21.15.122:0/904089619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b641069c0 0x7f2b64106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- 172.21.15.122:0/904089619 >> 172.21.15.122:0/904089619 conn(0x7f2b64076d80 msgr2=0x7f2b64077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- 172.21.15.122:0/904089619 shutdown_connections 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- 172.21.15.122:0/904089619 wait complete. 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 Processor -- start 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.718+0000 7f2b694d9640 1 -- start start 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b64101370 0x7f2b64199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b641069c0 0x7f2b641a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64110dd0 con 0x7f2b641009b0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64110c50 con 0x7f2b641069c0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64110f50 con 0x7f2b64101370 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37152/0 (socket says 172.21.15.122:37152) 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1054515817 learned_addr learned my addr 172.21.15.122:0/1054515817 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b64101370 0x7f2b64199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b637fe640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b641069c0 0x7f2b641a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1054515817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b64101370 msgr2=0x7f2b64199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b64101370 0x7f2b64199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1054515817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b641069c0 msgr2=0x7f2b641a4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b641069c0 0x7f2b641a4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1054515817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b641a47b0 con 0x7f2b641009b0 2024-04-02T13:39:39.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b4a0 tx=0x7f2b5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b60ff9640 1 -- 172.21.15.122:0/1054515817 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54014070 con 0x7f2b641009b0 2024-04-02T13:39:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b60ff9640 1 -- 172.21.15.122:0/1054515817 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b54010af0 con 0x7f2b641009b0 2024-04-02T13:39:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641a4980 con 0x7f2b641009b0 2024-04-02T13:39:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b60ff9640 1 -- 172.21.15.122:0/1054515817 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b54017d10 con 0x7f2b641009b0 2024-04-02T13:39:39.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.719+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b641a4ef0 con 0x7f2b641009b0 2024-04-02T13:39:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.721+0000 7f2b60ff9640 1 -- 172.21.15.122:0/1054515817 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2b54017070 con 0x7f2b641009b0 2024-04-02T13:39:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 Processor -- start 2024-04-02T13:39:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 -- start start 2024-04-02T13:39:39.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b60ff9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 0x7f2b500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b60ff9640 1 -- 172.21.15.122:0/1054515817 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2b540cfd50 con 0x7f2b641009b0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b637fe640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 0x7f2b500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c39bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b7c2f2830 con 0x558b7c38e5c0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b7c29a820 con 0x558b7c38eee0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b7c2ee7e0 con 0x558b7c39c1e0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37162/0 (socket says 172.21.15.122:37162) 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1407774296 learned_addr learned my addr 172.21.15.122:0/1407774296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c39bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b637fe640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 0x7f2b500a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00adb0 tx=0x7f2b5c00a2f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1407774296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 msgr2=0x558b7c39e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.722+0000 7f2b637fe640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1407774296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 msgr2=0x558b7c39bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c39bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 -- 172.21.15.122:0/1407774296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7c0d07d0 con 0x558b7c38e5c0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c39bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b637fe640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2b5409ba00 tx=0x7f2b540047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821cf387ecff0cac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b597fa640 1 -- 172.21.15.122:0/1407774296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401e070 con 0x558b7c38e5c0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b597fa640 1 -- 172.21.15.122:0/1407774296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b540bc0b0 con 0x558b7c38e5c0 2024-04-02T13:39:39.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.723+0000 7f2b597fa640 1 -- 172.21.15.122:0/1407774296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b540980e0 con 0x558b7c38e5c0 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/1407774296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 msgr2=0x558b7c38e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2b5409ba00 tx=0x7f2b540047d0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/1407774296 shutdown_connections 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c39bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/1407774296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c38e9a0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/1407774296 >> 172.21.15.122:0/1407774296 conn(0x558b7c389000 msgr2=0x558b7c3893e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/1407774296 shutdown_connections 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/1407774296 wait complete. 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- start start 2024-04-02T13:39:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 Processor -- start 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c38e5c0 0x558b7c39b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c398d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558b7c39ef70 con 0x558b7c39c1e0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558b7c2add30 con 0x558b7c38eee0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558b7c39f0f0 con 0x558b7c38e5c0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b6c1a2740 10 client.0 _ll_register_callbacks cb 0x558b7c2ad720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37178/0 (socket says 172.21.15.122:37178) 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c38e5c0 0x558b7c39b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.724+0000 7f2b637fe640 1 -- 172.21.15.122:0/4141404116 learned_addr learned my addr 172.21.15.122:0/4141404116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c398d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 -- 172.21.15.122:0/4141404116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c38e5c0 msgr2=0x558b7c39b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c38e5c0 0x558b7c39b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 -- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 msgr2=0x558b7c398d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c398d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 -- 172.21.15.122:0/4141404116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7c0063f0 con 0x558b7c39c1e0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c398d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b637fe640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c010db0 tx=0x7f2b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db79bd89bd31d982 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c01d420 con 0x558b7c39c1e0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b540bc0b0 con 0x558b7c39c1e0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2b5c01d5c0 con 0x558b7c39c1e0 2024-04-02T13:39:39.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c02e050 con 0x558b7c39c1e0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b6c1a2740 10 client.17109 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b7bfc60f0 con 0x558b7c39c1e0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.725+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b540bc0b0 con 0x558b7c39c1e0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2b5c025b70 con 0x558b7c39c1e0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 10 client.17109 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 10 client.17109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 20 client.17109 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c3ac0f0 0x558b7c3ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 4 client.17109 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b7c2ffd70 con 0x558b7c3ac0f0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c3ac0f0 0x558b7c3ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c3ac0f0 0x558b7c3ae510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c1ef9bf3ad739b server_cookie=37de269d53fa7ea2 in_seq=0 out_seq=0 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.726+0000 7f2b4a7fc640 10 client.17109 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4141404116 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b7c2ffd70 con 0x558b7c3ac0f0 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b4a7fc640 10 client.17109 handle_command_reply: tid=0 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 client.17109 shutdown 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 msgr2=0x558b7c3992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c010db0 tx=0x7f2b5c004a40 comp rx=0 tx=0).stop 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 shutdown_connections 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c3ac0f0 0x558b7c3ae510 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c39c1e0 0x558b7c3992c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c398d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b4a7fc640 0 client.17109 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4141404116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c38e5c0 0x558b7c39b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 >> 172.21.15.122:0/4141404116 conn(0x558b7c389000 msgr2=0x558b7c38ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 shutdown_connections 2024-04-02T13:39:39.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4141404116 wait complete. 2024-04-02T13:39:39.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.727+0000 7f2b6c1a2740 20 client.17109 trim_cache size 0 max 16384 2024-04-02T13:39:39.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.744+0000 7f2b6c1a2740 1 Processor -- start 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.744+0000 7f2b6c1a2740 1 -- start start 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 0x558b7c3a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641acdd0 con 0x558b7c38eee0 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ae9a0 con 0x558b7c38e5c0 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ad4a0 con 0x558b7c39c1e0 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37194/0 (socket says 172.21.15.122:37194) 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 -- 172.21.15.122:0/2835390544 learned_addr learned my addr 172.21.15.122:0/2835390544 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 0x558b7c3a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b637fe640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 -- 172.21.15.122:0/2835390544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 msgr2=0x558b7c39b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 -- 172.21.15.122:0/2835390544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 msgr2=0x558b7c3a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 0x558b7c3a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 -- 172.21.15.122:0/2835390544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b7c394950 con 0x558b7c38eee0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 0x558b7c3a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b637fe640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009900 tx=0x7f2b4c02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c81128f49fac32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b58ff9640 1 -- 172.21.15.122:0/2835390544 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c02cc40 con 0x558b7c38eee0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b58ff9640 1 -- 172.21.15.122:0/2835390544 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c0046c0 con 0x558b7c38eee0 2024-04-02T13:39:39.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.745+0000 7f2b58ff9640 1 -- 172.21.15.122:0/2835390544 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c005480 con 0x558b7c38eee0 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/2835390544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 msgr2=0x558b7c38ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009900 tx=0x7f2b4c02c710 comp rx=0 tx=0).stop 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/2835390544 shutdown_connections 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c39b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38eee0 0x558b7c38ab10 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/2835390544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38e5c0 0x558b7c3a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/2835390544 >> 172.21.15.122:0/2835390544 conn(0x558b7c389000 msgr2=0x558b7c38a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/2835390544 shutdown_connections 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/2835390544 wait complete. 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 -- start start 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.746+0000 7f2b6c1a2740 1 Processor -- start 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c395820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c395f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641ae270 con 0x558b7c38e5c0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641adba0 con 0x558b7c38eee0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b641aadb0 con 0x558b7c39c1e0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 10 client.0 _ll_register_callbacks cb 0x7f2b5c010ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37202/0 (socket says 172.21.15.122:37202) 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 -- 172.21.15.122:0/4277395113 learned_addr learned my addr 172.21.15.122:0/4277395113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b637fe640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c395f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c395820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 -- 172.21.15.122:0/4277395113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 msgr2=0x558b7c395f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c395f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 -- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 msgr2=0x558b7c395820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c395820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 -- 172.21.15.122:0/4277395113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7c411630 con 0x558b7c38e5c0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c395820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b637fe640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c395f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b62ffd640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cd2d0 tx=0x7f2b5400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9aa81492ba76d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401e040 con 0x558b7c38e5c0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2b540cd390 con 0x558b7c38e5c0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b4c0046c0 con 0x558b7c38e5c0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2b540bc0b0 con 0x558b7c38e5c0 2024-04-02T13:39:39.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 10 client.17115 fetch_fsmap learned FSMap version 34 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.747+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558b7c412700 con 0x558b7c38e5c0 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b4c0046c0 con 0x558b7c38e5c0 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 34) ==== 8374+0+0 (secure 0 0 0) 0x7f2b540b8020 con 0x558b7c38e5c0 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 10 client.17115 fetch_fsmap finished waiting for FSMap version 34 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 10 client.17115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 20 client.17115 populate_metadata read hostname 'smithi122' 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c412b50 0x558b7c414f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 4 client.17115 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b7c2fa0c0 con 0x558b7c412b50 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c412b50 0x558b7c414f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b4a7fc640 10 client.17115 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.748+0000 7f2b5bfff640 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c412b50 0x558b7c414f60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd95794d4a4b8f27 server_cookie=92c582597ecddc77 in_seq=0 out_seq=0 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b4a7fc640 1 -- 172.21.15.122:0/4277395113 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558b7c2fa0c0 con 0x558b7c412b50 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b4a7fc640 10 client.17115 handle_command_reply: tid=0 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 client.17115 shutdown 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 msgr2=0x558b7c3a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f2b540cd2d0 tx=0x7f2b5400bf90 comp rx=0 tx=0).stop 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 shutdown_connections 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558b7c412b50 0x558b7c414f60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558b7c39c1e0 0x558b7c395f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b4a7fc640 0 client.17115 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558b7c38eee0 0x558b7c395820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 --2- 172.21.15.122:0/4277395113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558b7c38e5c0 0x558b7c3a48f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 >> 172.21.15.122:0/4277395113 conn(0x558b7c389000 msgr2=0x558b7c38ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 shutdown_connections 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 1 -- 172.21.15.122:0/4277395113 wait complete. 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b6c1a2740 20 client.17115 trim_cache size 0 max 16384 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:39:39.761 INFO:teuthology.orchestra.run.smithi122.stdout: "3129f011-afa0-413d-a491-aedc2bdf99ec": { 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "3129f011-afa0-413d-a491-aedc2bdf99ec", 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:39:39.762 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:39:39.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 msgr2=0x7f2b500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 0x7f2b500a6260 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00adb0 tx=0x7f2b5c00a2f0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 msgr2=0x7f2b641991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400b4a0 tx=0x7f2b5400b960 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 shutdown_connections 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2b500a3da0 0x7f2b500a6260 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2b641069c0 0x7f2b641a4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b64101370 0x7f2b64199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 --2- 172.21.15.122:0/1054515817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2b641009b0 0x7f2b641991f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.749+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 >> 172.21.15.122:0/1054515817 conn(0x7f2b64076d80 msgr2=0x7f2b640fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.750+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 shutdown_connections 2024-04-02T13:39:39.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:39:39.750+0000 7f2b694d9640 1 -- 172.21.15.122:0/1054515817 wait complete. 2024-04-02T13:39:39.764 INFO:tasks.cephfs.filesystem:scrub status for tag:3129f011-afa0-413d-a491-aedc2bdf99ec - {'path': '/', 'tag': '3129f011-afa0-413d-a491-aedc2bdf99ec', 'options': 'recursive,force'} 2024-04-02T13:39:39.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128518 2.55 MB/sec execute 60 sec latency 1150.693 ms 2024-04-02T13:39:40.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:40.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:40.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:40.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:40.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:40.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:40.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:40.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:40.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:40.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128518 2.51 MB/sec execute 61 sec latency 1297.592 ms 2024-04-02T13:39:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:39:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:41 smithi187 ceph-mon[36064]: pgmap v138: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 786 KiB/s wr, 74 op/s 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:39:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:41 smithi188 ceph-mon[36117]: pgmap v138: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 786 KiB/s wr, 74 op/s 2024-04-02T13:39:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:39:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:39:41.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:41 smithi122 ceph-mon[32114]: pgmap v138: 129 pgs: 129 active+clean; 84 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 786 KiB/s wr, 74 op/s 2024-04-02T13:39:41.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128518 2.47 MB/sec execute 62 sec latency 2297.708 ms 2024-04-02T13:39:42.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 128518 2.43 MB/sec execute 63 sec latency 3297.818 ms 2024-04-02T13:39:43.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:43 smithi187 ceph-mon[36064]: pgmap v139: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-02T13:39:43.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:43 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:43 smithi188 ceph-mon[36117]: pgmap v139: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-02T13:39:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:43 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:43.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:43 smithi122 ceph-mon[32114]: pgmap v139: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.1 MiB/s wr, 99 op/s 2024-04-02T13:39:43.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:43 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:43.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 130135 2.50 MB/sec execute 64 sec latency 3748.384 ms 2024-04-02T13:39:44.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 131813 2.64 MB/sec execute 65 sec latency 9.867 ms 2024-04-02T13:39:45.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:45 smithi187 ceph-mon[36064]: pgmap v140: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 861 KiB/s wr, 76 op/s 2024-04-02T13:39:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:45 smithi188 ceph-mon[36117]: pgmap v140: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 861 KiB/s wr, 76 op/s 2024-04-02T13:39:45.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:45 smithi122 ceph-mon[32114]: pgmap v140: 129 pgs: 129 active+clean; 84 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 861 KiB/s wr, 76 op/s 2024-04-02T13:39:45.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 131999 2.61 MB/sec execute 66 sec latency 852.657 ms 2024-04-02T13:39:46.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 131999 2.57 MB/sec execute 67 sec latency 1852.703 ms 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: pgmap v141: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: pgmap v141: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:39:47.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: pgmap v141: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.0 MiB/s wr, 112 op/s 2024-04-02T13:39:47.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:47.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:47.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:47.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:39:47.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 131999 2.53 MB/sec execute 68 sec latency 2852.810 ms 2024-04-02T13:39:48.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 133256 2.57 MB/sec execute 69 sec latency 3308.124 ms 2024-04-02T13:39:49.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:49 smithi187 ceph-mon[36064]: pgmap v142: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 656 KiB/s wr, 73 op/s 2024-04-02T13:39:49.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:49 smithi188 ceph-mon[36117]: pgmap v142: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 656 KiB/s wr, 73 op/s 2024-04-02T13:39:49.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:49 smithi122 ceph-mon[32114]: pgmap v142: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 656 KiB/s wr, 73 op/s 2024-04-02T13:39:49.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135221 2.73 MB/sec execute 70 sec latency 14.400 ms 2024-04-02T13:39:50.951 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135611 2.70 MB/sec execute 71 sec latency 681.732 ms 2024-04-02T13:39:51.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:51 smithi187 ceph-mon[36064]: pgmap v143: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 540 KiB/s wr, 66 op/s 2024-04-02T13:39:51.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:51 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:51 smithi188 ceph-mon[36117]: pgmap v143: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 540 KiB/s wr, 66 op/s 2024-04-02T13:39:51.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:51 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:51.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:51 smithi122 ceph-mon[32114]: pgmap v143: 129 pgs: 129 active+clean; 86 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 540 KiB/s wr, 66 op/s 2024-04-02T13:39:51.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:51 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:39:51.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135611 2.66 MB/sec execute 72 sec latency 1681.819 ms 2024-04-02T13:39:52.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 135611 2.62 MB/sec execute 73 sec latency 2681.919 ms 2024-04-02T13:39:53.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:53 smithi187 ceph-mon[36064]: pgmap v144: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-04-02T13:39:53.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:53 smithi188 ceph-mon[36117]: pgmap v144: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-04-02T13:39:53.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:53 smithi122 ceph-mon[32114]: pgmap v144: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-04-02T13:39:53.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 137210 2.68 MB/sec execute 74 sec latency 3131.108 ms 2024-04-02T13:39:54.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 138843 2.80 MB/sec execute 75 sec latency 11.600 ms 2024-04-02T13:39:55.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:55 smithi187 ceph-mon[36064]: pgmap v145: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 757 KiB/s wr, 102 op/s 2024-04-02T13:39:55.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:55 smithi188 ceph-mon[36117]: pgmap v145: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 757 KiB/s wr, 102 op/s 2024-04-02T13:39:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:55 smithi122 ceph-mon[32114]: pgmap v145: 129 pgs: 129 active+clean; 89 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 757 KiB/s wr, 102 op/s 2024-04-02T13:39:55.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139034 2.77 MB/sec execute 76 sec latency 882.675 ms 2024-04-02T13:39:56.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139034 2.73 MB/sec execute 77 sec latency 1882.781 ms 2024-04-02T13:39:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:57 smithi187 ceph-mon[36064]: pgmap v146: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T13:39:57.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:57 smithi188 ceph-mon[36117]: pgmap v146: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T13:39:57.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:57 smithi122 ceph-mon[32114]: pgmap v146: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.2 MiB/s wr, 159 op/s 2024-04-02T13:39:57.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139034 2.69 MB/sec execute 78 sec latency 2882.866 ms 2024-04-02T13:39:58.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139090 2.66 MB/sec execute 79 sec latency 3338.637 ms 2024-04-02T13:39:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:39:59 smithi187 ceph-mon[36064]: pgmap v147: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-04-02T13:39:59.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:39:59 smithi188 ceph-mon[36117]: pgmap v147: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-04-02T13:39:59.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:39:59 smithi122 ceph-mon[32114]: pgmap v147: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-04-02T13:39:59.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139090 2.63 MB/sec execute 80 sec latency 1490.989 ms 2024-04-02T13:40:00.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:00 smithi187 ceph-mon[36064]: overall HEALTH_OK 2024-04-02T13:40:00.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:00 smithi188 ceph-mon[36117]: overall HEALTH_OK 2024-04-02T13:40:00.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:00 smithi122 ceph-mon[32114]: overall HEALTH_OK 2024-04-02T13:40:00.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 139360 2.61 MB/sec execute 81 sec latency 2288.300 ms 2024-04-02T13:40:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:01 smithi187 ceph-mon[36064]: pgmap v148: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 117 op/s 2024-04-02T13:40:01.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:01 smithi188 ceph-mon[36117]: pgmap v148: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 117 op/s 2024-04-02T13:40:01.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:01 smithi122 ceph-mon[32114]: pgmap v148: 129 pgs: 129 active+clean; 90 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1017 KiB/s wr, 117 op/s 2024-04-02T13:40:01.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 140193 2.63 MB/sec execute 82 sec latency 58.944 ms 2024-04-02T13:40:02.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142302 2.76 MB/sec execute 83 sec latency 11.350 ms 2024-04-02T13:40:03.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:03 smithi187 ceph-mon[36064]: pgmap v149: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-02T13:40:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:03 smithi188 ceph-mon[36117]: pgmap v149: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-02T13:40:03.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:03 smithi122 ceph-mon[32114]: pgmap v149: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-02T13:40:03.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142633 2.73 MB/sec execute 84 sec latency 505.736 ms 2024-04-02T13:40:04.952 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142633 2.70 MB/sec execute 85 sec latency 1505.813 ms 2024-04-02T13:40:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:05 smithi187 ceph-mon[36064]: pgmap v150: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 814 KiB/s wr, 96 op/s 2024-04-02T13:40:05.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:05 smithi188 ceph-mon[36117]: pgmap v150: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 814 KiB/s wr, 96 op/s 2024-04-02T13:40:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:05 smithi122 ceph-mon[32114]: pgmap v150: 129 pgs: 129 active+clean; 90 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 591 KiB/s rd, 814 KiB/s wr, 96 op/s 2024-04-02T13:40:05.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142633 2.67 MB/sec execute 86 sec latency 2505.918 ms 2024-04-02T13:40:06.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142633 2.64 MB/sec execute 87 sec latency 3506.028 ms 2024-04-02T13:40:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:07 smithi187 ceph-mon[36064]: pgmap v151: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T13:40:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:07 smithi188 ceph-mon[36117]: pgmap v151: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T13:40:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:07 smithi122 ceph-mon[32114]: pgmap v151: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-04-02T13:40:07.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 142633 2.61 MB/sec execute 88 sec latency 4506.087 ms 2024-04-02T13:40:08.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 144051 2.64 MB/sec execute 89 sec latency 4960.638 ms 2024-04-02T13:40:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:09 smithi187 ceph-mon[36064]: pgmap v152: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:09.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:09 smithi188 ceph-mon[36117]: pgmap v152: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:09 smithi122 ceph-mon[32114]: pgmap v152: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:09.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:09.766 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2301917197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 msgr2=0x7fda28106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2301917197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28106e10 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fda24009900 tx=0x7fda2402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2301917197 shutdown_connections 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2301917197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda281016a0 0x7fda2810d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2301917197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda28100ce0 0x7fda28101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2301917197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28106e10 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2301917197 >> 172.21.15.122:0/2301917197 conn(0x7fda280779e0 msgr2=0x7fda28077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2301917197 shutdown_connections 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.875+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2301917197 wait complete. 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 Processor -- start 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 -- start start 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda28100ce0 0x7fda28102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda281016a0 0x7fda281029a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda28110d80 con 0x7fda28106a30 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda28110c00 con 0x7fda281016a0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2ef7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda28110f00 con 0x7fda28100ce0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda281016a0 0x7fda281029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda28100ce0 0x7fda28102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53822/0 (socket says 172.21.15.122:53822) 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 -- 172.21.15.122:0/2182349160 learned_addr learned my addr 172.21.15.122:0/2182349160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 -- 172.21.15.122:0/2182349160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda28100ce0 msgr2=0x7fda28102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda28100ce0 0x7fda28102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 -- 172.21.15.122:0/2182349160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda281016a0 msgr2=0x7fda281029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda281016a0 0x7fda281029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 -- 172.21.15.122:0/2182349160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda281a67f0 con 0x7fda28106a30 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.876+0000 7fda2d518640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fda2000eb10 tx=0x7fda2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.877+0000 7fda1dffb640 1 -- 172.21.15.122:0/2182349160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20004b20 con 0x7fda28106a30 2024-04-02T13:40:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.877+0000 7fda1dffb640 1 -- 172.21.15.122:0/2182349160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda20004cc0 con 0x7fda28106a30 2024-04-02T13:40:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.877+0000 7fda1dffb640 1 -- 172.21.15.122:0/2182349160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20010760 con 0x7fda28106a30 2024-04-02T13:40:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.877+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda281a6ae0 con 0x7fda28106a30 2024-04-02T13:40:09.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.877+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda281a7030 con 0x7fda28106a30 2024-04-02T13:40:09.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.879+0000 7fda1dffb640 1 -- 172.21.15.122:0/2182349160 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fda20005020 con 0x7fda28106a30 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 Processor -- start 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 -- start start 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda1dffb640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 0x7fda100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316cb1f0 0x563c316d3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316d3dd0 0x563c316d61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda1dffb640 1 -- 172.21.15.122:0/2182349160 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fda200d11c0 con 0x7fda28106a30 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda1ffff640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 0x7fda100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563c3162e830 con 0x563c316ca8d0 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563c315d6820 con 0x563c316d3dd0 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda31b67740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563c3162a7e0 con 0x563c316cb1f0 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53824/0 (socket says 172.21.15.122:53824) 2024-04-02T13:40:09.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 -- 172.21.15.122:0/2933027330 learned_addr learned my addr 172.21.15.122:0/2933027330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2d518640 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316d3dd0 0x563c316d61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda1ffff640 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316cb1f0 0x563c316d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 -- 172.21.15.122:0/2933027330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316cb1f0 msgr2=0x563c316d3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316cb1f0 0x563c316d3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 -- 172.21.15.122:0/2933027330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316d3dd0 msgr2=0x563c316d61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316d3dd0 0x563c316d61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 -- 172.21.15.122:0/2933027330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3140c7d0 con 0x563c316ca8d0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda1ffff640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 0x7fda100a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda28103ef0 tx=0x7fda18013880 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda2cd17640 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fda2402cfd0 tx=0x7fda24005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d37502b4003ceb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.880+0000 7fda0effd640 1 -- 172.21.15.122:0/2933027330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24042070 con 0x563c316ca8d0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda0effd640 1 -- 172.21.15.122:0/2933027330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24004030 con 0x563c316ca8d0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda0effd640 1 -- 172.21.15.122:0/2933027330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24004c20 con 0x563c316ca8d0 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 -- 172.21.15.122:0/2933027330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 msgr2=0x563c316cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fda2402cfd0 tx=0x7fda24005c00 comp rx=0 tx=0).stop 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 -- 172.21.15.122:0/2933027330 shutdown_connections 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316d3dd0 0x563c316d61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316cb1f0 0x563c316d3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 --2- 172.21.15.122:0/2933027330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316cacb0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 -- 172.21.15.122:0/2933027330 >> 172.21.15.122:0/2933027330 conn(0x563c316c5000 msgr2=0x563c316c53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.881+0000 7fda31b67740 1 -- 172.21.15.122:0/2933027330 shutdown_connections 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 -- 172.21.15.122:0/2933027330 wait complete. 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 -- start start 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 Processor -- start 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563c316d6b60 con 0x563c316ca8d0 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563c315e9d30 con 0x563c316cb1f0 2024-04-02T13:40:09.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563c316d6ce0 con 0x563c316d3dd0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda31b67740 10 client.0 _ll_register_callbacks cb 0x563c315e9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53838/0 (socket says 172.21.15.122:53838) 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.882+0000 7fda2cd17640 1 -- 172.21.15.122:0/3568875962 learned_addr learned my addr 172.21.15.122:0/3568875962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2d518640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 -- 172.21.15.122:0/3568875962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 msgr2=0x563c316d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 -- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 msgr2=0x563c316d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 -- 172.21.15.122:0/3568875962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c313423f0 con 0x563c316ca8d0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda1ffff640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda2cd17640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fda24004840 tx=0x7fda24004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e488bc7e0b535d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda07fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:09.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24042070 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fda2402ca90 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda24004030 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda24058050 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda31b67740 10 client.17127 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563c313020f0 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.883+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24004030 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fda2404e030 con 0x563c316ca8d0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 10 client.17127 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 10 client.17127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 20 client.17127 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c316e3b40 0x563c316e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 4 client.17127 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c3163bd70 con 0x563c316e3b40 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda1ffff640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c316e3b40 0x563c316e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda07fff640 10 client.17127 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.884+0000 7fda1ffff640 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c316e3b40 0x563c316e5f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344b0f3c067ed448 server_cookie=1c63bb2d3d9cc26d in_seq=0 out_seq=0 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda07fff640 1 -- 172.21.15.122:0/3568875962 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c3163bd70 con 0x563c316e3b40 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda07fff640 10 client.17127 handle_command_reply: tid=0 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 client.17127 shutdown 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 msgr2=0x563c316d35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fda24004840 tx=0x7fda24004490 comp rx=0 tx=0).stop 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 shutdown_connections 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c316e3b40 0x563c316e5f30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda07fff640 0 client.17127 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 --2- 172.21.15.122:0/3568875962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316d35e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 >> 172.21.15.122:0/3568875962 conn(0x563c316c5000 msgr2=0x563c316c6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 shutdown_connections 2024-04-02T13:40:09.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 1 -- 172.21.15.122:0/3568875962 wait complete. 2024-04-02T13:40:09.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.885+0000 7fda31b67740 20 client.17127 trim_cache size 0 max 16384 2024-04-02T13:40:09.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.904+0000 7fda31b67740 1 Processor -- start 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.904+0000 7fda31b67740 1 -- start start 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda281b10f0 con 0x563c316ca8d0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda281b2c30 con 0x563c316cb1f0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda31b67740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda281b17c0 con 0x563c316d3dd0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53850/0 (socket says 172.21.15.122:53850) 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 -- 172.21.15.122:0/1877551822 learned_addr learned my addr 172.21.15.122:0/1877551822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda1ffff640 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2d518640 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 -- 172.21.15.122:0/1877551822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 msgr2=0x563c316d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 -- 172.21.15.122:0/1877551822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 msgr2=0x563c316cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 -- 172.21.15.122:0/1877551822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c316c4e30 con 0x563c316ca8d0 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda2cd17640 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fda24005940 tx=0x7fda24005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffcfe323536bb28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda0e7fc640 1 -- 172.21.15.122:0/1877551822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda24042040 con 0x563c316ca8d0 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda0e7fc640 1 -- 172.21.15.122:0/1877551822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24003e20 con 0x563c316ca8d0 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.905+0000 7fda0e7fc640 1 -- 172.21.15.122:0/1877551822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2402ca90 con 0x563c316ca8d0 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 -- 172.21.15.122:0/1877551822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 msgr2=0x563c316dfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fda24005940 tx=0x7fda24005c00 comp rx=0 tx=0).stop 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 -- 172.21.15.122:0/1877551822 shutdown_connections 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316cb1f0 0x563c316cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 --2- 172.21.15.122:0/1877551822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316ca8d0 0x563c316dfa50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 -- 172.21.15.122:0/1877551822 >> 172.21.15.122:0/1877551822 conn(0x563c316c5000 msgr2=0x563c316cf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 -- 172.21.15.122:0/1877551822 shutdown_connections 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.906+0000 7fda31b67740 1 -- 172.21.15.122:0/1877551822 wait complete. 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 -- start start 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 Processor -- start 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 0x563c316dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 0x563c316cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda281b2560 con 0x563c316cb1f0 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda281b1e90 con 0x563c316ca8d0 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda281af0d0 con 0x563c316d3dd0 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda31b67740 10 client.0 _ll_register_callbacks cb 0x7fda24005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 0x563c316cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda2d518640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda2cd17640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 0x563c316dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 0x563c316cd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53864/0 (socket says 172.21.15.122:53864) 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda2d518640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42312/0 (socket says 172.21.15.122:42312) 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda1ffff640 1 -- 172.21.15.122:0/190101990 learned_addr learned my addr 172.21.15.122:0/190101990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda2d518640 1 -- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 msgr2=0x563c316dc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.907+0000 7fda2d518640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 0x563c316dc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda2d518640 1 -- 172.21.15.122:0/190101990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 msgr2=0x563c316cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda2d518640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 0x563c316cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda2d518640 1 -- 172.21.15.122:0/190101990 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c316cdf40 con 0x563c316d3dd0 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda2cd17640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 0x563c316dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda2d518640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda200ce580 tx=0x7fda2000efd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5d690b91c17ec4f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda200a0040 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda24003e20 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fda20005310 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fda2009cdd0 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 10 client.26278 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563c31749de0 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda24003e20 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fda2009c290 con 0x563c316d3dd0 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 10 client.26278 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 20 client.26278 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c3174a490 0x563c3174c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 4 client.26278 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.908+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c316360c0 con 0x563c3174a490 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda2cd17640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c3174a490 0x563c3174c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda07fff640 10 client.26278 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda2cd17640 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c3174a490 0x563c3174c8b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b65ec986b8891e48 server_cookie=f5a21b44f4d3ff0a in_seq=0 out_seq=0 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda07fff640 1 -- 172.21.15.122:0/190101990 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c316360c0 con 0x563c3174a490 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda07fff640 10 client.26278 handle_command_reply: tid=0 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 client.26278 shutdown 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 msgr2=0x563c316cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fda200ce580 tx=0x7fda2000efd0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 shutdown_connections 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563c3174a490 0x563c3174c8b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563c316d3dd0 0x563c316cd800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563c316cb1f0 0x563c316cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 --2- 172.21.15.122:0/190101990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563c316ca8d0 0x563c316dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda07fff640 0 client.26278 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 >> 172.21.15.122:0/190101990 conn(0x563c316c5000 msgr2=0x563c316c39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 shutdown_connections 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 1 -- 172.21.15.122:0/190101990 wait complete. 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.909+0000 7fda31b67740 20 client.26278 trim_cache size 0 max 16384 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:40:09.920 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:40:09.921 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 msgr2=0x7fda100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 0x7fda100a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fda28103ef0 tx=0x7fda18013880 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 msgr2=0x7fda28102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fda2000eb10 tx=0x7fda2000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 shutdown_connections 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda100a3d20 0x7fda100a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda28106a30 0x7fda28102ee0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda281016a0 0x7fda281029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 --2- 172.21.15.122:0/2182349160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda28100ce0 0x7fda28102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 >> 172.21.15.122:0/2182349160 conn(0x7fda280779e0 msgr2=0x7fda2806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 shutdown_connections 2024-04-02T13:40:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:09.910+0000 7fda2ef7b640 1 -- 172.21.15.122:0/2182349160 wait complete. 2024-04-02T13:40:09.923 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:40:09.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:40:09.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 145881 2.76 MB/sec execute 90 sec latency 12.195 ms 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 -- 172.21.15.122:0/1052585010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6558106a10 msgr2=0x7f6558106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 --2- 172.21.15.122:0/1052585010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6558106a10 0x7f6558106df0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f6548009930 tx=0x7f654802bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 -- 172.21.15.122:0/1052585010 shutdown_connections 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 --2- 172.21.15.122:0/1052585010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65581013c0 0x7f655810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 --2- 172.21.15.122:0/1052585010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558100a00 0x7f6558100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 --2- 172.21.15.122:0/1052585010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6558106a10 0x7f6558106df0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 -- 172.21.15.122:0/1052585010 >> 172.21.15.122:0/1052585010 conn(0x7f6558077b00 msgr2=0x7f6558077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 -- 172.21.15.122:0/1052585010 shutdown_connections 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.030+0000 7f655e293640 1 -- 172.21.15.122:0/1052585010 wait complete. 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 Processor -- start 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 -- start start 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6558100a00 0x7f6558198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 0x7f655819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6558110df0 con 0x7f65581013c0 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6558110c70 con 0x7f6558100a00 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655e293640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6558110f70 con 0x7f6558106a10 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f65577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f65577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53882/0 (socket says 172.21.15.122:53882) 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f65577fe640 1 -- 172.21.15.122:0/2132851612 learned_addr learned my addr 172.21.15.122:0/2132851612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f655c830640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 0x7f655819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.031+0000 7f6557fff640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6558100a00 0x7f6558198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 -- 172.21.15.122:0/2132851612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 msgr2=0x7f655819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 0x7f655819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 -- 172.21.15.122:0/2132851612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6558100a00 msgr2=0x7f6558198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6558100a00 0x7f6558198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 -- 172.21.15.122:0/2132851612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655819d430 con 0x7f65581013c0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f655c830640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 0x7f655819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65577fe640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f654c00e9f0 tx=0x7f654c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c00cd80 con 0x7f65581013c0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655819d720 con 0x7f65581013c0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c010910 con 0x7f65581013c0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0188f0 con 0x7f65581013c0 2024-04-02T13:40:10.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.032+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65581a4fb0 con 0x7f65581013c0 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.033+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65580688f0 con 0x7f65581013c0 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.034+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f654c010ac0 con 0x7f65581013c0 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.035+0000 7f65557fa640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 0x7f65400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.035+0000 7f6557fff640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 0x7f65400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.035+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f654c0d0f80 con 0x7f65581013c0 2024-04-02T13:40:10.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.035+0000 7f6557fff640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 0x7f65400a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6548005ab0 tx=0x7f6548005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.040+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f654c09cbe0 con 0x7f65581013c0 2024-04-02T13:40:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.268+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6558109f80 con 0x7f65581013c0 2024-04-02T13:40:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.268+0000 7f65557fa640 1 -- 172.21.15.122:0/2132851612 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11942 (secure 0 0 0) 0x7f654c0a0ee0 con 0x7f65581013c0 2024-04-02T13:40:10.271 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:40:10.271 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:39:49.562463+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:40:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 36 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 msgr2=0x7f65400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 0x7f65400a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6548005ab0 tx=0x7f6548005fd0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 msgr2=0x7f6558198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f654c00e9f0 tx=0x7f654c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 shutdown_connections 2024-04-02T13:40:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65400a3d20 0x7f65400a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6558106a10 0x7f655819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65581013c0 0x7f6558198980 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 --2- 172.21.15.122:0/2132851612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6558100a00 0x7f6558198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 >> 172.21.15.122:0/2132851612 conn(0x7f6558077b00 msgr2=0x7f6558069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 shutdown_connections 2024-04-02T13:40:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.271+0000 7f655e293640 1 -- 172.21.15.122:0/2132851612 wait complete. 2024-04-02T13:40:10.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 -- 172.21.15.122:0/1594328309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c100970 msgr2=0x7f588c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 --2- 172.21.15.122:0/1594328309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c100970 0x7f588c100df0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f587c009930 tx=0x7f587c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 -- 172.21.15.122:0/1594328309 shutdown_connections 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 --2- 172.21.15.122:0/1594328309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 0x7f588c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 --2- 172.21.15.122:0/1594328309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c100970 0x7f588c100df0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 --2- 172.21.15.122:0/1594328309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c106980 0x7f588c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 -- 172.21.15.122:0/1594328309 >> 172.21.15.122:0/1594328309 conn(0x7f588c077b00 msgr2=0x7f588c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 -- 172.21.15.122:0/1594328309 shutdown_connections 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.426+0000 7f58943af640 1 -- 172.21.15.122:0/1594328309 wait complete. 2024-04-02T13:40:10.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 Processor -- start 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 -- start start 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c100970 0x7f588c1972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 0x7f588c197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f588c110d90 con 0x7f588c106980 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f588c110c10 con 0x7f588c100970 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f58943af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f588c110f10 con 0x7f588c101330 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53896/0 (socket says 172.21.15.122:53896) 2024-04-02T13:40:10.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 -- 172.21.15.122:0/2304197475 learned_addr learned my addr 172.21.15.122:0/2304197475 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589214b640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c100970 0x7f588c1972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589194a640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 0x7f588c197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 -- 172.21.15.122:0/2304197475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 msgr2=0x7f588c197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 0x7f588c197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 -- 172.21.15.122:0/2304197475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c100970 msgr2=0x7f588c1972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c100970 0x7f588c1972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.427+0000 7f589294c640 1 -- 172.21.15.122:0/2304197475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588c19c2c0 con 0x7f588c106980 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f589294c640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f588400cad0 tx=0x7f588400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f58837fe640 1 -- 172.21.15.122:0/2304197475 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588400ae20 con 0x7f588c106980 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588c19c5b0 con 0x7f588c106980 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f58837fe640 1 -- 172.21.15.122:0/2304197475 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5884005020 con 0x7f588c106980 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f58837fe640 1 -- 172.21.15.122:0/2304197475 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588400f6f0 con 0x7f588c106980 2024-04-02T13:40:10.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.428+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588c1a3e40 con 0x7f588c106980 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.430+0000 7f58837fe640 1 -- 172.21.15.122:0/2304197475 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f58840051d0 con 0x7f588c106980 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.430+0000 7f5896f9a740 1 Processor -- start 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 -- start start 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f58837fe640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 0x7f58780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5623d2e607b0 con 0x5623d2efcb50 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5623d2dff570 con 0x5623d2f07eb0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f5896f9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623d2e5c760 con 0x5623d2efc1a0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589194a640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 0x7f58780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f58837fe640 1 -- 172.21.15.122:0/2304197475 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f58840d0c60 con 0x7f588c106980 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f07970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53900/0 (socket says 172.21.15.122:53900) 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589194a640 1 -- 172.21.15.122:0/146329330 learned_addr learned my addr 172.21.15.122:0/146329330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589214b640 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 -- 172.21.15.122:0/146329330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 msgr2=0x5623d2efc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 -- 172.21.15.122:0/146329330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 msgr2=0x5623d2f07970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f07970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 -- 172.21.15.122:0/146329330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d2b743f0 con 0x5623d2f07eb0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589194a640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 0x7f58780a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f587c02cfd0 tx=0x7f587c005e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.431+0000 7f589294c640 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f0a2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f58840ce020 tx=0x7f58840142d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d16b9158959920 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5880bfe640 1 -- 172.21.15.122:0/146329330 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588409c870 con 0x5623d2f07eb0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5880bfe640 1 -- 172.21.15.122:0/146329330 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58840a0c20 con 0x5623d2f07eb0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5880bfe640 1 -- 172.21.15.122:0/146329330 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840ce2a0 con 0x5623d2f07eb0 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 -- 172.21.15.122:0/146329330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 msgr2=0x5623d2f0a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f0a2a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f58840ce020 tx=0x7f58840142d0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 -- 172.21.15.122:0/146329330 shutdown_connections 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f0a2a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f07970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 --2- 172.21.15.122:0/146329330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 -- 172.21.15.122:0/146329330 >> 172.21.15.122:0/146329330 conn(0x5623d2ef6ed0 msgr2=0x5623d2ef72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 -- 172.21.15.122:0/146329330 shutdown_connections 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.432+0000 7f5896f9a740 1 -- 172.21.15.122:0/146329330 wait complete. 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 -- start start 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 Processor -- start 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2f07910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5623d2f0aca0 con 0x5623d2efcb50 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5623d2e1be80 con 0x5623d2f07eb0 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5623d2f0ae20 con 0x5623d2efc1a0 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f5896f9a740 10 client.0 _ll_register_callbacks cb 0x5623d2e57aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53916/0 (socket says 172.21.15.122:53916) 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 -- 172.21.15.122:0/1638973984 learned_addr learned my addr 172.21.15.122:0/1638973984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589294c640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589214b640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2f07910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 -- 172.21.15.122:0/1638973984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 msgr2=0x5623d2f07910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2f07910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 -- 172.21.15.122:0/1638973984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 msgr2=0x5623d2f053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589194a640 1 -- 172.21.15.122:0/1638973984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d2c3e7d0 con 0x5623d2efcb50 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589214b640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2f07910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.433+0000 7f589294c640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:10.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f589194a640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f587c009230 tx=0x7f587c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71f2f6d444c3bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c005020 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f587c0051c0 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58840a0c20 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f587c0053c0 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 10 client.17151 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5623d2f16d50 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58840a0c20 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f587c042440 con 0x5623d2efcb50 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 10 client.17151 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 10 client.17151 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 20 client.17151 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f181b0 0x5623d2f1a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 4 client.17151 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.434+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623d2e6dcf0 con 0x5623d2f181b0 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f589214b640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f181b0 0x5623d2f1a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f589214b640 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f181b0 0x5623d2f1a600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac5b6a6419e0aae4 server_cookie=deb6e29e76d81241 in_seq=0 out_seq=0 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5875ffb640 10 client.17151 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5875ffb640 1 -- 172.21.15.122:0/1638973984 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623d2e6dcf0 con 0x5623d2f181b0 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5875ffb640 10 client.17151 handle_command_reply: tid=0 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5896f9a740 1 client.17151 shutdown 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 msgr2=0x5623d2f04ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f587c009230 tx=0x7f587c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 shutdown_connections 2024-04-02T13:40:10.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.435+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f181b0 0x5623d2f1a600 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f04ca0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 --2- 172.21.15.122:0/1638973984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2f07910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 >> 172.21.15.122:0/1638973984 conn(0x5623d2ef6ed0 msgr2=0x5623d2ef8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5875ffb640 0 client.17151 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 shutdown_connections 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 1 -- 172.21.15.122:0/1638973984 wait complete. 2024-04-02T13:40:10.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.436+0000 7f5896f9a740 20 client.17151 trim_cache size 0 max 16384 2024-04-02T13:40:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 Processor -- start 2024-04-02T13:40:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 -- start start 2024-04-02T13:40:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f03c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ace30 con 0x5623d2efcb50 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ae970 con 0x5623d2f07eb0 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5896f9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ad500 con 0x5623d2efc1a0 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53932/0 (socket says 172.21.15.122:53932) 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 -- 172.21.15.122:0/2014207999 learned_addr learned my addr 172.21.15.122:0/2014207999 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 -- 172.21.15.122:0/2014207999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 msgr2=0x5623d2efd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 -- 172.21.15.122:0/2014207999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 msgr2=0x5623d2f03c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f03c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 -- 172.21.15.122:0/2014207999 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623d2f008a0 con 0x5623d2efcb50 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f589194a640 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f587c02c170 tx=0x7f587c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40ec5509b4000a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.449+0000 7f5877fff640 1 -- 172.21.15.122:0/2014207999 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c0044b0 con 0x5623d2efcb50 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5877fff640 1 -- 172.21.15.122:0/2014207999 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c02cc40 con 0x5623d2efcb50 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5877fff640 1 -- 172.21.15.122:0/2014207999 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c034920 con 0x5623d2efcb50 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- 172.21.15.122:0/2014207999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 msgr2=0x5623d2f13ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f587c02c170 tx=0x7f587c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- 172.21.15.122:0/2014207999 shutdown_connections 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2f07eb0 0x5623d2f03c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efcb50 0x5623d2f13ba0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 --2- 172.21.15.122:0/2014207999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2efc1a0 0x5623d2efd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- 172.21.15.122:0/2014207999 >> 172.21.15.122:0/2014207999 conn(0x5623d2ef6ed0 msgr2=0x5623d2f03570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- 172.21.15.122:0/2014207999 shutdown_connections 2024-04-02T13:40:10.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- 172.21.15.122:0/2014207999 wait complete. 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.450+0000 7f5896f9a740 1 -- start start 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 Processor -- start 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2efcb50 0x5623d2f105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2f07eb0 0x5623d2f01130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1ae2a0 con 0x5623d2efc1a0 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1adbd0 con 0x5623d2efcb50 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f588c1aae10 con 0x5623d2f07eb0 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 10 client.0 _ll_register_callbacks cb 0x7f587c02c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53940/0 (socket says 172.21.15.122:53940) 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 -- 172.21.15.122:0/3309266986 learned_addr learned my addr 172.21.15.122:0/3309266986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589194a640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2efcb50 0x5623d2f105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 -- 172.21.15.122:0/3309266986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2f07eb0 msgr2=0x5623d2f01130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2f07eb0 0x5623d2f01130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 -- 172.21.15.122:0/3309266986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2efcb50 msgr2=0x5623d2f105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2efcb50 0x5623d2f105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 -- 172.21.15.122:0/3309266986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623d2f01960 con 0x5623d2efc1a0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f589214b640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5888005fd0 tx=0x7f58880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7b0fa8de99651d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5875ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588801d900 con 0x5623d2efc1a0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f587c02cc40 con 0x5623d2efc1a0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.451+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f5888004020 con 0x5623d2efc1a0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f588802c050 con 0x5623d2efc1a0 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 10 client.17157 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5623d2f7e210 con 0x5623d2efc1a0 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c02cc40 con 0x5623d2efc1a0 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f5888010430 con 0x5623d2efc1a0 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 10 client.17157 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 10 client.17157 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 20 client.17157 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f7e930 0x5623d2f80d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 4 client.17157 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623d2e68040 con 0x5623d2f7e930 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f589194a640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f7e930 0x5623d2f80d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f5875ffb640 10 client.17157 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:10.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.452+0000 7f589194a640 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f7e930 0x5623d2f80d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d48efedd9dc46f0c server_cookie=a47536c584eb9b79 in_seq=0 out_seq=0 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5875ffb640 1 -- 172.21.15.122:0/3309266986 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623d2e68040 con 0x5623d2f7e930 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5875ffb640 10 client.17157 handle_command_reply: tid=0 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 client.17157 shutdown 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 msgr2=0x5623d2f100b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f5888005fd0 tx=0x7f58880118d0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 shutdown_connections 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5623d2f7e930 0x5623d2f80d40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5623d2f07eb0 0x5623d2f01130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5623d2efcb50 0x5623d2f105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 --2- 172.21.15.122:0/3309266986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5623d2efc1a0 0x5623d2f100b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5875ffb640 0 client.17157 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 >> 172.21.15.122:0/3309266986 conn(0x5623d2ef6ed0 msgr2=0x5623d2ef8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 shutdown_connections 2024-04-02T13:40:10.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 1 -- 172.21.15.122:0/3309266986 wait complete. 2024-04-02T13:40:10.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f5896f9a740 20 client.17157 trim_cache size 0 max 16384 2024-04-02T13:40:10.463 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 msgr2=0x7f58780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 0x7f58780a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f587c02cfd0 tx=0x7f587c005e80 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 msgr2=0x7f588c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f588400cad0 tx=0x7f588400cf90 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 shutdown_connections 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58780a3d20 0x7f58780a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f588c106980 0x7f588c19bb90 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f588c101330 0x7f588c197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 --2- 172.21.15.122:0/2304197475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f588c100970 0x7f588c1972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 >> 172.21.15.122:0/2304197475 conn(0x7f588c077b00 msgr2=0x7f588c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 shutdown_connections 2024-04-02T13:40:10.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.453+0000 7f58943af640 1 -- 172.21.15.122:0/2304197475 wait complete. 2024-04-02T13:40:10.466 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:40:10.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 -- 172.21.15.122:0/4237362484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1003c0 msgr2=0x7f9f3c100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 --2- 172.21.15.122:0/4237362484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c100820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009930 tx=0x7f9f2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 -- 172.21.15.122:0/4237362484 shutdown_connections 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 --2- 172.21.15.122:0/4237362484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c100d60 0x7f9f3c10d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 --2- 172.21.15.122:0/4237362484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c100820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 --2- 172.21.15.122:0/4237362484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c105510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 -- 172.21.15.122:0/4237362484 >> 172.21.15.122:0/4237362484 conn(0x7f9f3c075d40 msgr2=0x7f9f3c076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 -- 172.21.15.122:0/4237362484 shutdown_connections 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.570+0000 7f9f443cd640 1 -- 172.21.15.122:0/4237362484 wait complete. 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 Processor -- start 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 -- start start 2024-04-02T13:40:10.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c100d60 0x7f9f3c198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c111f10 con 0x7f9f3c105130 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c111d90 con 0x7f9f3c1003c0 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f443cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c112090 con 0x7f9f3c100d60 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f42169640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53956/0 (socket says 172.21.15.122:53956) 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 -- 172.21.15.122:0/2689743041 learned_addr learned my addr 172.21.15.122:0/2689743041 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f41968640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c100d60 0x7f9f3c198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 -- 172.21.15.122:0/2689743041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c100d60 msgr2=0x7f9f3c198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c100d60 0x7f9f3c198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 -- 172.21.15.122:0/2689743041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c1003c0 msgr2=0x7f9f3c198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.571+0000 7f9f4296a640 1 -- 172.21.15.122:0/2689743041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c19d300 con 0x7f9f3c105130 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f4296a640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f337fe640 1 -- 172.21.15.122:0/2689743041 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004b20 con 0x7f9f3c105130 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c19d5f0 con 0x7f9f3c105130 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f337fe640 1 -- 172.21.15.122:0/2689743041 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f34004cc0 con 0x7f9f3c105130 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f337fe640 1 -- 172.21.15.122:0/2689743041 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34010760 con 0x7f9f3c105130 2024-04-02T13:40:10.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.572+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a4e80 con 0x7f9f3c105130 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.574+0000 7f9f337fe640 1 -- 172.21.15.122:0/2689743041 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9f34005020 con 0x7f9f3c105130 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.574+0000 7f9f46fb3740 1 Processor -- start 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 -- start start 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f337fe640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 0x7f9f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fe74b7f7b0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe74b1e570 con 0x55fe74c24ca0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe74b7b760 con 0x55fe74c1b7a0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f4296a640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 0x7f9f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53962/0 (socket says 172.21.15.122:53962) 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 -- 172.21.15.122:0/4023048008 learned_addr learned my addr 172.21.15.122:0/4023048008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f41968640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f337fe640 1 -- 172.21.15.122:0/2689743041 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9f340d1240 con 0x7f9f3c105130 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f41968640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 -- 172.21.15.122:0/4023048008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 msgr2=0x55fe74c1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 -- 172.21.15.122:0/4023048008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 msgr2=0x55fe74c27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 -- 172.21.15.122:0/4023048008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe748933f0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f41968640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f4296a640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 0x7f9f280a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f3409cde0 tx=0x7f9f3400efd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.575+0000 7f9f42169640 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009cf0 tx=0x7f9f38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6633285fb8afb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f30bfe640 1 -- 172.21.15.122:0/4023048008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3801e070 con 0x55fe74c1c0c0 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f30bfe640 1 -- 172.21.15.122:0/4023048008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f38019b10 con 0x55fe74c1c0c0 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f30bfe640 1 -- 172.21.15.122:0/4023048008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3800ea10 con 0x55fe74c1c0c0 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 -- 172.21.15.122:0/4023048008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 msgr2=0x55fe74c24760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009cf0 tx=0x7f9f38011810 comp rx=0 tx=0).stop 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 -- 172.21.15.122:0/4023048008 shutdown_connections 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c24760 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/4023048008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.576+0000 7f9f46fb3740 1 -- 172.21.15.122:0/4023048008 >> 172.21.15.122:0/4023048008 conn(0x55fe74c15ed0 msgr2=0x55fe74c162b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- 172.21.15.122:0/4023048008 shutdown_connections 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- 172.21.15.122:0/4023048008 wait complete. 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- start start 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 Processor -- start 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1b7a0 0x55fe74c24710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 0x55fe74c221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fe74c27a90 con 0x55fe74c1c0c0 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe74b3ae80 con 0x55fe74c1b7a0 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe74c27c10 con 0x55fe74c24ca0 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f46fb3740 10 client.0 _ll_register_callbacks cb 0x55fe74b76aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53974/0 (socket says 172.21.15.122:53974) 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 -- 172.21.15.122:0/2592403393 learned_addr learned my addr 172.21.15.122:0/2592403393 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f41968640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1b7a0 0x55fe74c24710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f41968640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 0x55fe74c221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 -- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 msgr2=0x55fe74c221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 0x55fe74c221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 -- 172.21.15.122:0/2592403393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1b7a0 msgr2=0x55fe74c24710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1b7a0 0x55fe74c24710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f42169640 1 -- 172.21.15.122:0/2592403393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe7495d7d0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.577+0000 7f9f41968640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 0x55fe74c221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f42169640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097b0 tx=0x7f9f38019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc22cd277110c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3801e050 con 0x55fe74c1c0c0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9f38005240 con 0x55fe74c1c0c0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f38019b10 con 0x55fe74c1c0c0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9f38033050 con 0x55fe74c1c0c0 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 10 client.17169 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fe74c33b20 con 0x55fe74c1c0c0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f38019b10 con 0x55fe74c1c0c0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9f380046c0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 10 client.17169 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 10 client.17169 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 20 client.17169 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c34f80 0x55fe74c37370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 4 client.17169 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.578+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe74b8ccf0 con 0x55fe74c34f80 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.579+0000 7f9f41968640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c34f80 0x55fe74c37370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.579+0000 7f9f25ffb640 10 client.17169 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.579+0000 7f9f41968640 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c34f80 0x55fe74c37370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b0757df9216f972b server_cookie=a7d57934c0bda005 in_seq=0 out_seq=0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f25ffb640 1 -- 172.21.15.122:0/2592403393 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe74b8ccf0 con 0x55fe74c34f80 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f25ffb640 10 client.17169 handle_command_reply: tid=0 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 client.17169 shutdown 2024-04-02T13:40:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 msgr2=0x55fe74c21aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f9f380097b0 tx=0x7f9f38019660 comp rx=0 tx=0).stop 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 shutdown_connections 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c24ca0 0x55fe74c221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c34f80 0x55fe74c37370 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1c0c0 0x55fe74c21aa0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f25ffb640 0 client.17169 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/2592403393 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1b7a0 0x55fe74c24710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 >> 172.21.15.122:0/2592403393 conn(0x55fe74c15ed0 msgr2=0x55fe74c17c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 shutdown_connections 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 1 -- 172.21.15.122:0/2592403393 wait complete. 2024-04-02T13:40:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.580+0000 7f9f46fb3740 20 client.17169 trim_cache size 0 max 16384 2024-04-02T13:40:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 Processor -- start 2024-04-02T13:40:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 -- start start 2024-04-02T13:40:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1c0c0 0x55fe74c20a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1af020 con 0x55fe74c1b7a0 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1b0b60 con 0x55fe74c24ca0 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1af6f0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53980/0 (socket says 172.21.15.122:53980) 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 -- 172.21.15.122:0/3684231116 learned_addr learned my addr 172.21.15.122:0/3684231116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f42169640 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1c0c0 0x55fe74c20a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 -- 172.21.15.122:0/3684231116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1c0c0 msgr2=0x55fe74c20a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1c0c0 0x55fe74c20a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 -- 172.21.15.122:0/3684231116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 msgr2=0x55fe74c241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.593+0000 7f9f41968640 1 -- 172.21.15.122:0/3684231116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe74c22550 con 0x55fe74c1b7a0 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f41968640 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02c820 tx=0x7f9f2c045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5230af9cad8b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f27fff640 1 -- 172.21.15.122:0/3684231116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004040 con 0x55fe74c1b7a0 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f27fff640 1 -- 172.21.15.122:0/3684231116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c004ba0 con 0x55fe74c1b7a0 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f27fff640 1 -- 172.21.15.122:0/3684231116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005240 con 0x55fe74c1b7a0 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 -- 172.21.15.122:0/3684231116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 msgr2=0x55fe74c30950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02c820 tx=0x7f9f2c045040 comp rx=0 tx=0).stop 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 -- 172.21.15.122:0/3684231116 shutdown_connections 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c24ca0 0x55fe74c241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1c0c0 0x55fe74c20a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/3684231116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c1b7a0 0x55fe74c30950 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 -- 172.21.15.122:0/3684231116 >> 172.21.15.122:0/3684231116 conn(0x55fe74c15ed0 msgr2=0x55fe74c20370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 -- 172.21.15.122:0/3684231116 shutdown_connections 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.594+0000 7f9f46fb3740 1 -- 172.21.15.122:0/3684231116 wait complete. 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 -- start start 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 Processor -- start 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1c0c0 0x55fe74c255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1b0490 con 0x55fe74c24ca0 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1afdc0 con 0x55fe74c1c0c0 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ad000 con 0x55fe74c1b7a0 2024-04-02T13:40:10.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f46fb3740 10 client.0 _ll_register_callbacks cb 0x7f9f3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53990/0 (socket says 172.21.15.122:53990) 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 -- 172.21.15.122:0/336960765 learned_addr learned my addr 172.21.15.122:0/336960765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f42169640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1c0c0 0x55fe74c255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 -- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 msgr2=0x55fe74c25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 -- 172.21.15.122:0/336960765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1c0c0 msgr2=0x55fe74c255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1c0c0 0x55fe74c255c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 -- 172.21.15.122:0/336960765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe74c2be60 con 0x55fe74c24ca0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f41968640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02c170 tx=0x7f9f2c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3e8530283bb6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.595+0000 7f9f25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c004040 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9f2c049040 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c004ba0 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c0041e0 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 10 client.17175 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fe74c2d160 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c004ba0 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9f2c003850 con 0x55fe74c24ca0 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 10 client.17175 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 10 client.17175 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 20 client.17175 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c9b320 0x55fe74c9d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 4 client.17175 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:10.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe74b87040 con 0x55fe74c9b320 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.596+0000 7f9f42169640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c9b320 0x55fe74c9d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f25ffb640 10 client.17175 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f42169640 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c9b320 0x55fe74c9d710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f107183d4d7843a6 server_cookie=479a27a414d9b173 in_seq=0 out_seq=0 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f25ffb640 1 -- 172.21.15.122:0/336960765 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe74b87040 con 0x55fe74c9b320 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f25ffb640 10 client.17175 handle_command_reply: tid=0 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 client.17175 shutdown 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 msgr2=0x55fe74c25b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c02c170 tx=0x7f9f2c02e040 comp rx=0 tx=0).stop 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 shutdown_connections 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fe74c9b320 0x55fe74c9d710 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe74c24ca0 0x55fe74c25b00 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f25ffb640 0 client.17175 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe74c1c0c0 0x55fe74c255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 --2- 172.21.15.122:0/336960765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe74c1b7a0 0x55fe74c25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 >> 172.21.15.122:0/336960765 conn(0x55fe74c15ed0 msgr2=0x55fe74c146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 shutdown_connections 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 1 -- 172.21.15.122:0/336960765 wait complete. 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f46fb3740 20 client.17175 trim_cache size 0 max 16384 2024-04-02T13:40:10.609 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:10.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 msgr2=0x7f9f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 0x7f9f280a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9f3409cde0 tx=0x7f9f3400efd0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 msgr2=0x7f9f3c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400eb10 tx=0x7f9f3400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 shutdown_connections 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f280a3d20 0x7f9f280a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f3c105130 0x7f9f3c19cbd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f3c100d60 0x7f9f3c198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.597+0000 7f9f443cd640 1 --2- 172.21.15.122:0/2689743041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f3c1003c0 0x7f9f3c198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.598+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 >> 172.21.15.122:0/2689743041 conn(0x7f9f3c075d40 msgr2=0x7f9f3c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.598+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 shutdown_connections 2024-04-02T13:40:10.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.598+0000 7f9f443cd640 1 -- 172.21.15.122:0/2689743041 wait complete. 2024-04-02T13:40:10.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:40:10.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 -- 172.21.15.122:0/4056112229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1009b0 msgr2=0x7fa4fc100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 --2- 172.21.15.122:0/4056112229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc100e30 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009cd0 tx=0x7fa4f002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 -- 172.21.15.122:0/4056112229 shutdown_connections 2024-04-02T13:40:10.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 --2- 172.21.15.122:0/4056112229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc101370 0x7fa4fc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 --2- 172.21.15.122:0/4056112229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc100e30 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 --2- 172.21.15.122:0/4056112229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 -- 172.21.15.122:0/4056112229 >> 172.21.15.122:0/4056112229 conn(0x7fa4fc076d80 msgr2=0x7fa4fc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 -- 172.21.15.122:0/4056112229 shutdown_connections 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 -- 172.21.15.122:0/4056112229 wait complete. 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.716+0000 7fa501482640 1 Processor -- start 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 -- start start 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc101370 0x7fa4fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc110dd0 con 0x7fa4fc1069c0 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc110c50 con 0x7fa4fc1009b0 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc110f50 con 0x7fa4fc101370 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53998/0 (socket says 172.21.15.122:53998) 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 -- 172.21.15.122:0/732508504 learned_addr learned my addr 172.21.15.122:0/732508504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc101370 0x7fa4fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4faffd640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 -- 172.21.15.122:0/732508504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc101370 msgr2=0x7fa4fc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc101370 0x7fa4fc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 -- 172.21.15.122:0/732508504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc1009b0 msgr2=0x7fa4fc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 -- 172.21.15.122:0/732508504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4fc19d3c0 con 0x7fa4fc1069c0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4fb7fe640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00d950 tx=0x7fa4ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4e3fff640 1 -- 172.21.15.122:0/732508504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec017030 con 0x7fa4fc1069c0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa4e3fff640 1 -- 172.21.15.122:0/732508504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec004020 con 0x7fa4fc1069c0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.717+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4fc19d6b0 con 0x7fa4fc1069c0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.718+0000 7fa4e3fff640 1 -- 172.21.15.122:0/732508504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec003c00 con 0x7fa4fc1069c0 2024-04-02T13:40:10.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.718+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4fc1a4f40 con 0x7fa4fc1069c0 2024-04-02T13:40:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa4e3fff640 1 -- 172.21.15.122:0/732508504 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa4ec01a070 con 0x7fa4fc1069c0 2024-04-02T13:40:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 Processor -- start 2024-04-02T13:40:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 -- start start 2024-04-02T13:40:10.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d6cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560736cd07b0 con 0x560736d75ca0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560736c6f570 con 0x560736d6c7a0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa50405f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560736ccc760 con 0x560736d6d0c0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa4e3fff640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 0x7fa4e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.720+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54002/0 (socket says 172.21.15.122:54002) 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42474/0 (socket says 172.21.15.122:42474) 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/2500018665 learned_addr learned my addr 172.21.15.122:0/2500018665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4fb7fe640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 0x7fa4e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d6cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4e3fff640 1 -- 172.21.15.122:0/732508504 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa4ec0d0440 con 0x7fa4fc1069c0 2024-04-02T13:40:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 -- 172.21.15.122:0/2500018665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 msgr2=0x560736d6cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d6cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 -- 172.21.15.122:0/2500018665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 msgr2=0x560736d78090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 -- 172.21.15.122:0/2500018665 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607369e43f0 con 0x560736d6d0c0 2024-04-02T13:40:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:10.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4fb7fe640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 0x7fa4e40a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0cd8b0 tx=0x7fa4ec06eb40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4faffd640 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4011eb0 tx=0x7fa4f400b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e93f3f86cc7d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4e17fa640 1 -- 172.21.15.122:0/2500018665 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4015070 con 0x560736d6d0c0 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4e17fa640 1 -- 172.21.15.122:0/2500018665 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4005500 con 0x560736d6d0c0 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.721+0000 7fa4e17fa640 1 -- 172.21.15.122:0/2500018665 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4004c30 con 0x560736d6d0c0 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- 172.21.15.122:0/2500018665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 msgr2=0x560736d75760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4011eb0 tx=0x7fa4f400b770 comp rx=0 tx=0).stop 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- 172.21.15.122:0/2500018665 shutdown_connections 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d78090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d75760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 --2- 172.21.15.122:0/2500018665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d6cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- 172.21.15.122:0/2500018665 >> 172.21.15.122:0/2500018665 conn(0x560736d66ed0 msgr2=0x560736d672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- 172.21.15.122:0/2500018665 shutdown_connections 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- 172.21.15.122:0/2500018665 wait complete. 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 -- start start 2024-04-02T13:40:10.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.722+0000 7fa50405f740 1 Processor -- start 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d75630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560736d78a90 con 0x560736d75ca0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560736c8be80 con 0x560736d6c7a0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560736d78c10 con 0x560736d6d0c0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 10 client.0 _ll_register_callbacks cb 0x560736cc7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54012/0 (socket says 172.21.15.122:54012) 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/1275832651 learned_addr learned my addr 172.21.15.122:0/1275832651 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d729c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d75630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/1275832651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 msgr2=0x560736d729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d729c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 msgr2=0x560736d75630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d75630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/1275832651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560736aae7d0 con 0x560736d75ca0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0005cc0 tx=0x7fa4f004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46522a1edf4f170b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4deff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0039040 con 0x560736d75ca0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa4f00329a0 con 0x560736d75ca0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f4005500 con 0x560736d75ca0 2024-04-02T13:40:10.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4f005c050 con 0x560736d75ca0 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 10 client.17187 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.723+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560736d84a20 con 0x560736d75ca0 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f4005500 con 0x560736d75ca0 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f0034440 con 0x560736d75ca0 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 10 client.17187 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 10 client.17187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 20 client.17187 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736d85ee0 0x560736d882d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 4 client.17187 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560736cddcf0 con 0x560736d85ee0 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736d85ee0 0x560736d882d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736d85ee0 0x560736d882d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7c9c25b5ec9caa9 server_cookie=84e3911b5d0b16a8 in_seq=0 out_seq=0 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.724+0000 7fa4deff5640 10 client.17187 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa4deff5640 1 -- 172.21.15.122:0/1275832651 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560736cddcf0 con 0x560736d85ee0 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa4deff5640 10 client.17187 handle_command_reply: tid=0 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 client.17187 shutdown 2024-04-02T13:40:10.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 msgr2=0x560736d72f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0005cc0 tx=0x7fa4f004a040 comp rx=0 tx=0).stop 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 shutdown_connections 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d72f40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736d85ee0 0x560736d882d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 --2- 172.21.15.122:0/1275832651 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d75630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa4deff5640 0 client.17187 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 >> 172.21.15.122:0/1275832651 conn(0x560736d66ed0 msgr2=0x560736d68c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 shutdown_connections 2024-04-02T13:40:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 1 -- 172.21.15.122:0/1275832651 wait complete. 2024-04-02T13:40:10.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.725+0000 7fa50405f740 20 client.17187 trim_cache size 0 max 16384 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.738+0000 7fa50405f740 1 Processor -- start 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.738+0000 7fa50405f740 1 -- start start 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d81950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1acd90 con 0x560736d75ca0 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ae8d0 con 0x560736d6c7a0 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa50405f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ad460 con 0x560736d6d0c0 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54022/0 (socket says 172.21.15.122:54022) 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/78514429 learned_addr learned my addr 172.21.15.122:0/78514429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d81950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/78514429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 msgr2=0x560736d719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4faffd640 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/78514429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 msgr2=0x560736d81950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d81950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 -- 172.21.15.122:0/78514429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560736d6c590 con 0x560736d75ca0 2024-04-02T13:40:10.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009600 tx=0x7fa4f0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23953a67a60a590 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4e0ff9640 1 -- 172.21.15.122:0/78514429 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0039040 con 0x560736d75ca0 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4e0ff9640 1 -- 172.21.15.122:0/78514429 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0003e50 con 0x560736d75ca0 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.739+0000 7fa4e0ff9640 1 -- 172.21.15.122:0/78514429 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0004040 con 0x560736d75ca0 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- 172.21.15.122:0/78514429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 msgr2=0x560736d750f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009600 tx=0x7fa4f0000f40 comp rx=0 tx=0).stop 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- 172.21.15.122:0/78514429 shutdown_connections 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d75ca0 0x560736d750f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d6d0c0 0x560736d719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 --2- 172.21.15.122:0/78514429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d81950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- 172.21.15.122:0/78514429 >> 172.21.15.122:0/78514429 conn(0x560736d66ed0 msgr2=0x560736d71290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- 172.21.15.122:0/78514429 shutdown_connections 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- 172.21.15.122:0/78514429 wait complete. 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 -- start start 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.740+0000 7fa50405f740 1 Processor -- start 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d769b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d75ca0 0x560736d77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1ae200 con 0x560736d6d0c0 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1adb30 con 0x560736d6c7a0 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4fc1aad70 con 0x560736d75ca0 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 10 client.0 _ll_register_callbacks cb 0x7fa4f0047b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54036/0 (socket says 172.21.15.122:54036) 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 -- 172.21.15.122:0/1333707959 learned_addr learned my addr 172.21.15.122:0/1333707959 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d75ca0 0x560736d77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 -- 172.21.15.122:0/1333707959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d75ca0 msgr2=0x560736d77430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d75ca0 0x560736d77430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 -- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 msgr2=0x560736d769b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d769b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 -- 172.21.15.122:0/1333707959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560736d7bf70 con 0x560736d6d0c0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4faffd640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4011eb0 tx=0x7fa4f400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da027b8649f3476 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4deff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f402e060 con 0x560736d6d0c0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa4f4011460 con 0x560736d6d0c0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f0003e50 con 0x560736d6d0c0 2024-04-02T13:40:10.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4f4035050 con 0x560736d6d0c0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 10 client.17193 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.741+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560736dec600 con 0x560736d6d0c0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f0003e50 con 0x560736d6d0c0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa4f4010620 con 0x560736d6d0c0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 10 client.17193 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 10 client.17193 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 20 client.17193 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736decaa0 0x560736deeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 4 client.17193 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560736cd8040 con 0x560736decaa0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736decaa0 0x560736deeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4deff5640 10 client.17193 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:10.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4fa7fc640 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736decaa0 0x560736deeec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a9cea81dd5ea94e server_cookie=db9d5e7802d82872 in_seq=0 out_seq=0 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4deff5640 1 -- 172.21.15.122:0/1333707959 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560736cd8040 con 0x560736decaa0 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa4deff5640 10 client.17193 handle_command_reply: tid=0 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.742+0000 7fa50405f740 1 client.17193 shutdown 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 msgr2=0x560736d76ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4011eb0 tx=0x7fa4f400bd90 comp rx=0 tx=0).stop 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 shutdown_connections 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560736d75ca0 0x560736d77430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560736d6d0c0 0x560736d76ef0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x560736decaa0 0x560736deeec0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 --2- 172.21.15.122:0/1333707959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560736d6c7a0 0x560736d769b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa4deff5640 0 client.17193 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 >> 172.21.15.122:0/1333707959 conn(0x560736d66ed0 msgr2=0x560736d71b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 shutdown_connections 2024-04-02T13:40:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 1 -- 172.21.15.122:0/1333707959 wait complete. 2024-04-02T13:40:10.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa50405f740 20 client.17193 trim_cache size 0 max 16384 2024-04-02T13:40:10.759 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:10.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 msgr2=0x7fa4e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 0x7fa4e40a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0cd8b0 tx=0x7fa4ec06eb40 comp rx=0 tx=0).stop 2024-04-02T13:40:10.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 msgr2=0x7fa4fc19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec00d950 tx=0x7fa4ec00de10 comp rx=0 tx=0).stop 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 shutdown_connections 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa4e40a3d20 0x7fa4e40a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa4fc1069c0 0x7fa4fc19cc90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4fc101370 0x7fa4fc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 --2- 172.21.15.122:0/732508504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa4fc1009b0 0x7fa4fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 >> 172.21.15.122:0/732508504 conn(0x7fa4fc076d80 msgr2=0x7fa4fc104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 shutdown_connections 2024-04-02T13:40:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.743+0000 7fa501482640 1 -- 172.21.15.122:0/732508504 wait complete. 2024-04-02T13:40:10.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:40:10.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 -- 172.21.15.122:0/342178422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 msgr2=0x7f9b84103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 --2- 172.21.15.122:0/342178422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84103120 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009900 tx=0x7f9b7402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 -- 172.21.15.122:0/342178422 shutdown_connections 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 --2- 172.21.15.122:0/342178422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 0x7f9b8410e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 --2- 172.21.15.122:0/342178422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 0x7f9b8410bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 --2- 172.21.15.122:0/342178422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84103120 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 -- 172.21.15.122:0/342178422 >> 172.21.15.122:0/342178422 conn(0x7f9b84076d80 msgr2=0x7f9b84077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 -- 172.21.15.122:0/342178422 shutdown_connections 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.865+0000 7f9b89300640 1 -- 172.21.15.122:0/342178422 wait complete. 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 Processor -- start 2024-04-02T13:40:10.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 -- start start 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 0x7f9b84100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 0x7f9b84101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84111ea0 con 0x7f9b84102d40 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84111d20 con 0x7f9b84103660 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b89300640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84112020 con 0x7f9b8410c110 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54062/0 (socket says 172.21.15.122:54062) 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 -- 172.21.15.122:0/54699504 learned_addr learned my addr 172.21.15.122:0/54699504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b837fe640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 0x7f9b84101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 0x7f9b84100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 -- 172.21.15.122:0/54699504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 msgr2=0x7f9b84101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 0x7f9b84101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 -- 172.21.15.122:0/54699504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 msgr2=0x7f9b84100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 0x7f9b84100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.866+0000 7f9b82ffd640 1 -- 172.21.15.122:0/54699504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b841a68c0 con 0x7f9b84102d40 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005940 tx=0x7f9b7402cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b80ff9640 1 -- 172.21.15.122:0/54699504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74041070 con 0x7f9b84102d40 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b80ff9640 1 -- 172.21.15.122:0/54699504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b74004450 con 0x7f9b84102d40 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b841a6b50 con 0x7f9b84102d40 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b80ff9640 1 -- 172.21.15.122:0/54699504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74003df0 con 0x7f9b84102d40 2024-04-02T13:40:10.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.867+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a7040 con 0x7f9b84102d40 2024-04-02T13:40:10.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.869+0000 7f9b80ff9640 1 -- 172.21.15.122:0/54699504 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9b74044070 con 0x7f9b84102d40 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.869+0000 7f9b8bfbe740 1 Processor -- start 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 -- start start 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f506760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f506ca0 0x55e12f509090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e12f4617b0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e12f400570 con 0x55e12f4fe0c0 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e12f45d760 con 0x55e12f506ca0 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b82ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f506760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54078/0 (socket says 172.21.15.122:54078) 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b837fe640 1 -- 172.21.15.122:0/4113181824 learned_addr learned my addr 172.21.15.122:0/4113181824 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b80ff9640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 0x7f9b700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f506ca0 0x55e12f509090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b837fe640 1 -- 172.21.15.122:0/4113181824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f506ca0 msgr2=0x55e12f509090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.870+0000 7f9b837fe640 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f506ca0 0x55e12f509090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b837fe640 1 -- 172.21.15.122:0/4113181824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 msgr2=0x55e12f506760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b837fe640 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f506760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b837fe640 1 -- 172.21.15.122:0/4113181824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12f1753f0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f506760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b80ff9640 1 -- 172.21.15.122:0/54699504 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9b740f25b0 con 0x7f9b84102d40 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b837fe640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 0x7f9b700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b837fe640 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c010470 tx=0x7f9b7c010930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6419f7b63976667 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b797fa640 1 -- 172.21.15.122:0/4113181824 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c01d400 con 0x55e12f4fd7a0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b797fa640 1 -- 172.21.15.122:0/4113181824 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c0134c0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.871+0000 7f9b797fa640 1 -- 172.21.15.122:0/4113181824 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0118e0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/4113181824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 msgr2=0x55e12f4fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b837fe640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 0x7f9b700a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c021000 tx=0x7f9b7c01e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c010470 tx=0x7f9b7c010930 comp rx=0 tx=0).stop 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/4113181824 shutdown_connections 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f506ca0 0x55e12f509090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f506760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/4113181824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f4fdb80 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/4113181824 >> 172.21.15.122:0/4113181824 conn(0x55e12f4f7ed0 msgr2=0x55e12f4f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/4113181824 shutdown_connections 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/4113181824 wait complete. 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 -- start start 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 Processor -- start 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.872+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f506740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f503ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e12f509a90 con 0x55e12f506ca0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e12f41ce80 con 0x55e12f4fe0c0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e12f509c10 con 0x55e12f4fd7a0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 10 client.0 _ll_register_callbacks cb 0x55e12f458aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54094/0 (socket says 172.21.15.122:54094) 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 -- 172.21.15.122:0/2750044586 learned_addr learned my addr 172.21.15.122:0/2750044586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f506740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f503ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 -- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 msgr2=0x55e12f506740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f506740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 -- 172.21.15.122:0/2750044586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 msgr2=0x55e12f503ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f503ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 -- 172.21.15.122:0/2750044586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12f23f7d0 con 0x55e12f506ca0 2024-04-02T13:40:10.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b837fe640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c011d90 tx=0x7f9b7c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8094d3f3543341 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c003b80 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9b7c003d20 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c0134c0 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c027480 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 10 client.17205 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.873+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e12f515b20 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7c0134c0 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9b7c00f040 con 0x55e12f506ca0 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 10 client.17205 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 10 client.17205 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 20 client.17205 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f519320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 4 client.17205 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:10.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e12f46ecf0 con 0x55e12f516f30 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.874+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f519320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b6a7fc640 10 client.17205 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f519320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86d01b7776386cdf server_cookie=c8481381c6d445dc in_seq=0 out_seq=0 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/2750044586 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e12f46ecf0 con 0x55e12f516f30 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b6a7fc640 10 client.17205 handle_command_reply: tid=0 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 client.17205 shutdown 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 msgr2=0x55e12f5041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c011d90 tx=0x7f9b7c011a10 comp rx=0 tx=0).stop 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 shutdown_connections 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f519320 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f506ca0 0x55e12f5041c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b6a7fc640 0 client.17205 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f4fe0c0 0x55e12f503ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/2750044586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f506740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 >> 172.21.15.122:0/2750044586 conn(0x55e12f4f7ed0 msgr2=0x55e12f4f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 shutdown_connections 2024-04-02T13:40:10.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.875+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/2750044586 wait complete. 2024-04-02T13:40:10.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.876+0000 7f9b8bfbe740 20 client.17205 trim_cache size 0 max 16384 2024-04-02T13:40:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 Processor -- start 2024-04-02T13:40:10.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 -- start start 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fe0c0 0x55e12f502ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f506200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b11e0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b2d20 con 0x55e12f506ca0 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b18b0 con 0x55e12f4fe0c0 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54100/0 (socket says 172.21.15.122:54100) 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 -- 172.21.15.122:0/1930981960 learned_addr learned my addr 172.21.15.122:0/1930981960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fe0c0 0x55e12f502ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b837fe640 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f506200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 -- 172.21.15.122:0/1930981960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fe0c0 msgr2=0x55e12f502ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fe0c0 0x55e12f502ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 -- 172.21.15.122:0/1930981960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 msgr2=0x55e12f506200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f506200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.889+0000 7f9b82ffd640 1 -- 172.21.15.122:0/1930981960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e12f502020 con 0x55e12f4fd7a0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9b740be080 tx=0x7f9b740ba750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c062f4433f9c3a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b78ff9640 1 -- 172.21.15.122:0/1930981960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7404b040 con 0x55e12f4fd7a0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b78ff9640 1 -- 172.21.15.122:0/1930981960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7402ce30 con 0x55e12f4fd7a0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b78ff9640 1 -- 172.21.15.122:0/1930981960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b740035f0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1930981960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 msgr2=0x55e12f512950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9b740be080 tx=0x7f9b740ba750 comp rx=0 tx=0).stop 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1930981960 shutdown_connections 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f506200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fe0c0 0x55e12f502ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1930981960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fd7a0 0x55e12f512950 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1930981960 >> 172.21.15.122:0/1930981960 conn(0x55e12f4f7ed0 msgr2=0x55e12f5023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.890+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1930981960 shutdown_connections 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1930981960 wait complete. 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 -- start start 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 Processor -- start 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f50f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f507c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b2650 con 0x55e12f4fe0c0 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b1f80 con 0x55e12f506ca0 2024-04-02T13:40:10.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841af1c0 con 0x55e12f4fd7a0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b8bfbe740 10 client.0 _ll_register_callbacks cb 0x7f9b7c020020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54108/0 (socket says 172.21.15.122:54108) 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 -- 172.21.15.122:0/1561087944 learned_addr learned my addr 172.21.15.122:0/1561087944 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f50f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b837fe640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f507c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 -- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 msgr2=0x55e12f50f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f50f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 -- 172.21.15.122:0/1561087944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 msgr2=0x55e12f507c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f507c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.891+0000 7f9b7bfff640 1 -- 172.21.15.122:0/1561087944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e12f508270 con 0x55e12f4fe0c0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b7bfff640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55e12f4f9060 tx=0x7f9b6c010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc943f079f51898 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c00faf0 con 0x55e12f4fe0c0 2024-04-02T13:40:10.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9b6c00fc90 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7402ce30 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9b6c02b050 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 10 client.17211 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e12f57d410 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b7402ce30 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9b6c01c370 con 0x55e12f4fe0c0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 10 client.17211 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 10 client.17211 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 20 client.17211 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f57fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 4 client.17211 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.892+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e12f469040 con 0x55e12f516f30 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f57fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b82ffd640 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f57fb90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53871e01496acd36 server_cookie=9c01a4e26fdbfc8a in_seq=0 out_seq=0 2024-04-02T13:40:10.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b6a7fc640 10 client.17211 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b6a7fc640 1 -- 172.21.15.122:0/1561087944 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e12f469040 con 0x55e12f516f30 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b6a7fc640 10 client.17211 handle_command_reply: tid=0 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 client.17211 shutdown 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 msgr2=0x55e12f5076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55e12f4f9060 tx=0x7f9b6c010b40 comp rx=0 tx=0).stop 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 shutdown_connections 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e12f516f30 0x55e12f57fb90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e12f506ca0 0x55e12f507c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b6a7fc640 0 client.17211 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e12f4fe0c0 0x55e12f5076c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 --2- 172.21.15.122:0/1561087944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e12f4fd7a0 0x55e12f50f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 >> 172.21.15.122:0/1561087944 conn(0x55e12f4f7ed0 msgr2=0x55e12f4f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 shutdown_connections 2024-04-02T13:40:10.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 1 -- 172.21.15.122:0/1561087944 wait complete. 2024-04-02T13:40:10.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.893+0000 7f9b8bfbe740 20 client.17211 trim_cache size 0 max 16384 2024-04-02T13:40:10.903 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 msgr2=0x7f9b700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 0x7f9b700a61e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c021000 tx=0x7f9b7c01e000 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 msgr2=0x7f9b84100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9b74005940 tx=0x7f9b7402cc40 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 shutdown_connections 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9b700a3d20 0x7f9b700a61e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8410c110 0x7f9b84101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b84103660 0x7f9b84100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 --2- 172.21.15.122:0/54699504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b84102d40 0x7f9b84100990 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 >> 172.21.15.122:0/54699504 conn(0x7f9b84076d80 msgr2=0x7f9b840fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 shutdown_connections 2024-04-02T13:40:10.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:10.894+0000 7f9b89300640 1 -- 172.21.15.122:0/54699504 wait complete. 2024-04-02T13:40:10.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:40:10.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 146126 2.73 MB/sec execute 91 sec latency 784.911 ms 2024-04-02T13:40:11.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 -- 172.21.15.122:0/806657684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc100970 msgr2=0x7f41cc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 --2- 172.21.15.122:0/806657684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc100970 0x7f41cc100df0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009930 tx=0x7f41c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:11.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 -- 172.21.15.122:0/806657684 shutdown_connections 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 --2- 172.21.15.122:0/806657684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc101330 0x7f41cc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 --2- 172.21.15.122:0/806657684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc100970 0x7f41cc100df0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 --2- 172.21.15.122:0/806657684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc106980 0x7f41cc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 -- 172.21.15.122:0/806657684 >> 172.21.15.122:0/806657684 conn(0x7f41cc077b00 msgr2=0x7f41cc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 -- 172.21.15.122:0/806657684 shutdown_connections 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 -- 172.21.15.122:0/806657684 wait complete. 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.009+0000 7f41d3e31640 1 Processor -- start 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- start start 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc100970 0x7f41cc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc106980 0x7f41cc19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110d90 con 0x7f41cc101330 2024-04-02T13:40:11.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110c10 con 0x7f41cc106980 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110f10 con 0x7f41cc100970 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54120/0 (socket says 172.21.15.122:54120) 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 -- 172.21.15.122:0/3048291621 learned_addr learned my addr 172.21.15.122:0/3048291621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc100970 0x7f41cc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d23ce640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc106980 0x7f41cc19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 -- 172.21.15.122:0/3048291621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc100970 msgr2=0x7f41cc197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc100970 0x7f41cc197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 -- 172.21.15.122:0/3048291621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc106980 msgr2=0x7f41cc19bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc106980 0x7f41cc19bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 -- 172.21.15.122:0/3048291621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc19c260 con 0x7f41cc101330 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d13cc640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009230 tx=0x7f41c002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41caffd640 1 -- 172.21.15.122:0/3048291621 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0041070 con 0x7f41cc101330 2024-04-02T13:40:11.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41caffd640 1 -- 172.21.15.122:0/3048291621 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c0004450 con 0x7f41cc101330 2024-04-02T13:40:11.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41caffd640 1 -- 172.21.15.122:0/3048291621 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0034440 con 0x7f41cc101330 2024-04-02T13:40:11.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc19c4f0 con 0x7f41cc101330 2024-04-02T13:40:11.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.010+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc1a3de0 con 0x7f41cc101330 2024-04-02T13:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.013+0000 7f41caffd640 1 -- 172.21.15.122:0/3048291621 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f41c0044070 con 0x7f41cc101330 2024-04-02T13:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.013+0000 7f41d6a18740 1 Processor -- start 2024-04-02T13:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.013+0000 7f41d6a18740 1 -- start start 2024-04-02T13:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfb9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56114cf1d7b0 con 0x56114cfb9d80 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56114cebc570 con 0x56114cfb9460 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d6a18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56114cf19760 con 0x56114cfc4eb0 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41caffd640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 0x7f41b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54134/0 (socket says 172.21.15.122:54134) 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 -- 172.21.15.122:0/3455357744 learned_addr learned my addr 172.21.15.122:0/3455357744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d23ce640 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfb9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d23ce640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 0x7f41b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41caffd640 1 -- 172.21.15.122:0/3048291621 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f41c00f25d0 con 0x7f41cc101330 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d13cc640 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 -- 172.21.15.122:0/3455357744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 msgr2=0x56114cfc72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 -- 172.21.15.122:0/3455357744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 msgr2=0x56114cfb9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfb9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 -- 172.21.15.122:0/3455357744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114cc313f0 con 0x56114cfb9d80 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f41c4010370 tx=0x7f41c4010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2bd3e7f21d5463 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41d23ce640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 0x7f41b40a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005fd0 tx=0x7f41bc004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.014+0000 7f41b3fff640 1 -- 172.21.15.122:0/3455357744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4016070 con 0x56114cfb9d80 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41b3fff640 1 -- 172.21.15.122:0/3455357744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4004d20 con 0x56114cfb9d80 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41b3fff640 1 -- 172.21.15.122:0/3455357744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c40054e0 con 0x56114cfb9d80 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 -- 172.21.15.122:0/3455357744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 msgr2=0x56114cfc4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f41c4010370 tx=0x7f41c4010830 comp rx=0 tx=0).stop 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 -- 172.21.15.122:0/3455357744 shutdown_connections 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc4970 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 --2- 172.21.15.122:0/3455357744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfb9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 -- 172.21.15.122:0/3455357744 >> 172.21.15.122:0/3455357744 conn(0x56114cfb3ed0 msgr2=0x56114cfb42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 -- 172.21.15.122:0/3455357744 shutdown_connections 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.015+0000 7f41d6a18740 1 -- 172.21.15.122:0/3455357744 wait complete. 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 -- start start 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 Processor -- start 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfc4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfc1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56114cfc7ca0 con 0x56114cfc4eb0 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56114ced8e80 con 0x56114cfb9d80 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56114cfc7e20 con 0x56114cfb9460 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d6a18740 10 client.0 _ll_register_callbacks cb 0x56114cf14aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d23ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d23ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54138/0 (socket says 172.21.15.122:54138) 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d23ce640 1 -- 172.21.15.122:0/306155140 learned_addr learned my addr 172.21.15.122:0/306155140 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d13cc640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfc1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfc4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.016+0000 7f41d23ce640 1 -- 172.21.15.122:0/306155140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 msgr2=0x56114cfc4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d23ce640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfc4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d23ce640 1 -- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 msgr2=0x56114cfc1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d23ce640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfc1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d23ce640 1 -- 172.21.15.122:0/306155140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114ccfb7d0 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d13cc640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfc1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d23ce640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010fd0 tx=0x7f41bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e0429a52ebaec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc005220 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f41bc0053c0 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c4004d20 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41bc00e980 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 10 client.17223 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56114cfd3d90 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4004d20 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f41bc01c040 con 0x56114cfc4eb0 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 10 client.17223 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:11.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 20 client.17223 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114cfd5250 0x56114cfd76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 4 client.17223 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.017+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56114cf2acf0 con 0x56114cfd5250 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.018+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114cfd5250 0x56114cfd76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.018+0000 7f41b17fa640 10 client.17223 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.018+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114cfd5250 0x56114cfd76a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7acb5f7f10a95de9 server_cookie=4c9abe1253ffbe06 in_seq=0 out_seq=0 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41b17fa640 1 -- 172.21.15.122:0/306155140 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56114cf2acf0 con 0x56114cfd5250 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41b17fa640 10 client.17223 handle_command_reply: tid=0 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 client.17223 shutdown 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 msgr2=0x56114cfc23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010fd0 tx=0x7f41bc0046c0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 shutdown_connections 2024-04-02T13:40:11.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114cfd5250 0x56114cfd76a0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfc23d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41b17fa640 0 client.17223 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfc1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 --2- 172.21.15.122:0/306155140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfc4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 >> 172.21.15.122:0/306155140 conn(0x56114cfb3ed0 msgr2=0x56114cfb5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 shutdown_connections 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 1 -- 172.21.15.122:0/306155140 wait complete. 2024-04-02T13:40:11.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.019+0000 7f41d6a18740 20 client.17223 trim_cache size 0 max 16384 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 Processor -- start 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 -- start start 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfd0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1acd20 con 0x56114cfb9d80 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ae860 con 0x56114cfb9460 2024-04-02T13:40:11.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d6a18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ad3f0 con 0x56114cfc4eb0 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54152/0 (socket says 172.21.15.122:54152) 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.032+0000 7f41d13cc640 1 -- 172.21.15.122:0/1482096283 learned_addr learned my addr 172.21.15.122:0/1482096283 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfd0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 -- 172.21.15.122:0/1482096283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 msgr2=0x56114cfc4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 -- 172.21.15.122:0/1482096283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 msgr2=0x56114cfd0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfd0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 -- 172.21.15.122:0/1482096283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56114cfb9bb0 con 0x56114cfb9d80 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d13cc640 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f41c00efaa0 tx=0x7f41c0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad349c99fd985d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41b37fe640 1 -- 172.21.15.122:0/1482096283 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c004b040 con 0x56114cfb9d80 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41b37fe640 1 -- 172.21.15.122:0/1482096283 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c00be990 con 0x56114cfb9d80 2024-04-02T13:40:11.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41b37fe640 1 -- 172.21.15.122:0/1482096283 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c00efbb0 con 0x56114cfb9d80 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d6a18740 1 -- 172.21.15.122:0/1482096283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 msgr2=0x56114cfc43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.033+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f41c00efaa0 tx=0x7f41c0003a90 comp rx=0 tx=0).stop 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- 172.21.15.122:0/1482096283 shutdown_connections 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfc4eb0 0x56114cfc4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfb9d80 0x56114cfc43c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1482096283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9460 0x56114cfd0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- 172.21.15.122:0/1482096283 >> 172.21.15.122:0/1482096283 conn(0x56114cfb3ed0 msgr2=0x56114cfc0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- 172.21.15.122:0/1482096283 shutdown_connections 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- 172.21.15.122:0/1482096283 wait complete. 2024-04-02T13:40:11.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- start start 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 Processor -- start 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfbe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfbec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ae190 con 0x56114cfc4eb0 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1adac0 con 0x56114cfb9d80 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1aad00 con 0x56114cfb9460 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d6a18740 10 client.0 _ll_register_callbacks cb 0x7f41bc0136a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d23ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfbec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d13cc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfbec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50740/0 (socket says 172.21.15.122:50740) 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d23ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54154/0 (socket says 172.21.15.122:54154) 2024-04-02T13:40:11.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d13cc640 1 -- 172.21.15.122:0/1947003633 learned_addr learned my addr 172.21.15.122:0/1947003633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.034+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfbe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 -- 172.21.15.122:0/1947003633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 msgr2=0x56114cfbe710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfbe710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 -- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 msgr2=0x56114cfbec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfbec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 -- 172.21.15.122:0/1947003633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56114cfcc6a0 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d23ce640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010840 tx=0x7f41bc010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515a8afdd94d8178 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc039020 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f41bc004040 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41c00be990 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41bc044050 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d6a18740 10 client.17229 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56114cfcd980 con 0x56114cfc4eb0 2024-04-02T13:40:11.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.035+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c00be990 con 0x56114cfc4eb0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f41bc004360 con 0x56114cfc4eb0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 10 client.17229 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 20 client.17229 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114d03bcf0 0x56114d03e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 4 client.17229 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56114cf25040 con 0x56114d03bcf0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114d03bcf0 0x56114d03e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41b17fa640 10 client.17229 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d1bcd640 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114d03bcf0 0x56114d03e110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e022f4742d35bd93 server_cookie=b16a6098b830b8c1 in_seq=0 out_seq=0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41b17fa640 1 -- 172.21.15.122:0/1947003633 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56114cf25040 con 0x56114d03bcf0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41b17fa640 10 client.17229 handle_command_reply: tid=0 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 1 client.17229 shutdown 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 msgr2=0x56114cfcc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.036+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010840 tx=0x7f41bc010bf0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 shutdown_connections 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56114d03bcf0 0x56114d03e110 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56114cfc4eb0 0x56114cfcc0b0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41b17fa640 0 client.17229 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56114cfb9d80 0x56114cfbec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 --2- 172.21.15.122:0/1947003633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56114cfb9460 0x56114cfbe710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 >> 172.21.15.122:0/1947003633 conn(0x56114cfb3ed0 msgr2=0x56114cfb2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 shutdown_connections 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 1 -- 172.21.15.122:0/1947003633 wait complete. 2024-04-02T13:40:11.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d6a18740 20 client.17229 trim_cache size 0 max 16384 2024-04-02T13:40:11.047 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 msgr2=0x7f41b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 0x7f41b40a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f41bc005fd0 tx=0x7f41bc004730 comp rx=0 tx=0).stop 2024-04-02T13:40:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 msgr2=0x7f41cc1977b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:11.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009230 tx=0x7f41c002cc70 comp rx=0 tx=0).stop 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 shutdown_connections 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41b40a3d20 0x7f41b40a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f41cc106980 0x7f41cc19bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc101330 0x7f41cc1977b0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 --2- 172.21.15.122:0/3048291621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41cc100970 0x7f41cc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 >> 172.21.15.122:0/3048291621 conn(0x7f41cc077b00 msgr2=0x7f41cc103120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 shutdown_connections 2024-04-02T13:40:11.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:11.037+0000 7f41d3e31640 1 -- 172.21.15.122:0/3048291621 wait complete. 2024-04-02T13:40:11.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:11 smithi187 ceph-mon[36064]: pgmap v153: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:11.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:11 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2132851612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:11 smithi188 ceph-mon[36117]: pgmap v153: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:11 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2132851612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:11 smithi122 ceph-mon[32114]: pgmap v153: 129 pgs: 129 active+clean; 92 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 649 KiB/s wr, 101 op/s 2024-04-02T13:40:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:11 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2132851612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:11.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 146126 2.70 MB/sec execute 92 sec latency 1784.943 ms 2024-04-02T13:40:12.051 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:40:12.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:40:12.051 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:40:12.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 -- 172.21.15.122:0/3971140436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4100990 msgr2=0x7fd9e4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/3971140436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4100990 0x7fd9e4100e10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009930 tx=0x7fd9d802bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 -- 172.21.15.122:0/3971140436 shutdown_connections 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/3971140436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4101350 0x7fd9e410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/3971140436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4100990 0x7fd9e4100e10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/3971140436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 0x7fd9e4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 -- 172.21.15.122:0/3971140436 >> 172.21.15.122:0/3971140436 conn(0x7fd9e4077ad0 msgr2=0x7fd9e4077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 -- 172.21.15.122:0/3971140436 shutdown_connections 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.157+0000 7fd9e92a7640 1 -- 172.21.15.122:0/3971140436 wait complete. 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 Processor -- start 2024-04-02T13:40:12.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 -- start start 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 0x7fd9e4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 0x7fd9e419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4111eb0 con 0x7fd9e4101350 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4111d30 con 0x7fd9e4100990 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e92a7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e4112030 con 0x7fd9e41069a0 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54174/0 (socket says 172.21.15.122:54174) 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 -- 172.21.15.122:0/4005433854 learned_addr learned my addr 172.21.15.122:0/4005433854 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 0x7fd9e4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 0x7fd9e419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 -- 172.21.15.122:0/4005433854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 msgr2=0x7fd9e419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 0x7fd9e419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 -- 172.21.15.122:0/4005433854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 msgr2=0x7fd9e4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 0x7fd9e4198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e27fc640 1 -- 172.21.15.122:0/4005433854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e419d380 con 0x7fd9e4101350 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.158+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 0x7fd9e4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8004f50 tx=0x7fd9d802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9cbfff640 1 -- 172.21.15.122:0/4005433854 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8041070 con 0x7fd9e4101350 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9cbfff640 1 -- 172.21.15.122:0/4005433854 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d80044d0 con 0x7fd9e4101350 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e419d610 con 0x7fd9e4101350 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9cbfff640 1 -- 172.21.15.122:0/4005433854 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8034440 con 0x7fd9e4101350 2024-04-02T13:40:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.159+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e41a4f00 con 0x7fd9e4101350 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9cbfff640 1 -- 172.21.15.122:0/4005433854 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd9d8044070 con 0x7fd9e4101350 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9ebf68740 1 Processor -- start 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9cbfff640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 0x7fd9cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9ebf68740 1 -- start start 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9cbfff640 1 -- 172.21.15.122:0/4005433854 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9d80f2730 con 0x7fd9e4101350 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc446f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.161+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 0x7fd9cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 0x564dc447abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564dc43d3770 con 0x564dc446fbe0 2024-04-02T13:40:12.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564dc437b960 con 0x564dc446f2c0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564dc43cf8c0 con 0x564dc44787c0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54180/0 (socket says 172.21.15.122:54180) 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 -- 172.21.15.122:0/1844312622 learned_addr learned my addr 172.21.15.122:0/1844312622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 0x564dc447abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc446f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 -- 172.21.15.122:0/1844312622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 msgr2=0x564dc447abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 0x564dc447abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 -- 172.21.15.122:0/1844312622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 msgr2=0x564dc446f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc446f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 -- 172.21.15.122:0/1844312622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc41af7e0 con 0x564dc446fbe0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 0x564dc447abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 0x7fd9cc0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007db0 tx=0x7fd9dc00c560 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009430 tx=0x7fd9d8049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43d1d177f90c913 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9c97fa640 1 -- 172.21.15.122:0/1844312622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d804b040 con 0x564dc446fbe0 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9c97fa640 1 -- 172.21.15.122:0/1844312622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d80be450 con 0x564dc446fbe0 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.162+0000 7fd9c97fa640 1 -- 172.21.15.122:0/1844312622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80f2340 con 0x564dc446fbe0 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- 172.21.15.122:0/1844312622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 msgr2=0x564dc4478280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009430 tx=0x7fd9d8049000 comp rx=0 tx=0).stop 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- 172.21.15.122:0/1844312622 shutdown_connections 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc44787c0 0x564dc447abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc4478280 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/1844312622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc446f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- 172.21.15.122:0/1844312622 >> 172.21.15.122:0/1844312622 conn(0x564dc44699f0 msgr2=0x564dc4469dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- 172.21.15.122:0/1844312622 shutdown_connections 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- 172.21.15.122:0/1844312622 wait complete. 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 -- start start 2024-04-02T13:40:12.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.163+0000 7fd9ebf68740 1 Processor -- start 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc4475790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc4475cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564dc447b370 con 0x564dc44787c0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564dc438ed20 con 0x564dc446f2c0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564dc447b4f0 con 0x564dc446fbe0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 10 client.0 _ll_register_callbacks cb 0x564dc435ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54192/0 (socket says 172.21.15.122:54192) 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 -- 172.21.15.122:0/3142769061 learned_addr learned my addr 172.21.15.122:0/3142769061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc4475cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 -- 172.21.15.122:0/3142769061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 msgr2=0x564dc4475cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc4475cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc4475790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 -- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 msgr2=0x564dc4475790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc4475790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 -- 172.21.15.122:0/3142769061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc420ff70 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc015e50 tx=0x7fd9dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b912b8cc1412248 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc024060 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9dc01f3f0 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d80be450 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd9dc02b050 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 10 client.17241 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.164+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564dc40a7170 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d80be450 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd9dc01e070 con 0x564dc44787c0 2024-04-02T13:40:12.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 10 client.17241 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 20 client.17241 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44887e0 0x564dc448ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 4 client.17241 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dc43e0cb0 con 0x564dc44887e0 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.165+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44887e0 0x564dc448ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.166+0000 7fd9c6ff5640 10 client.17241 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.166+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44887e0 0x564dc448ac60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d045e27df727173 server_cookie=2202e8c7d1f6065a in_seq=0 out_seq=0 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.166+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3142769061 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564dc43e0cb0 con 0x564dc44887e0 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.166+0000 7fd9c6ff5640 10 client.17241 handle_command_reply: tid=0 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.166+0000 7fd9ebf68740 1 client.17241 shutdown 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 msgr2=0x564dc4476210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc015e50 tx=0x7fd9dc004a40 comp rx=0 tx=0).stop 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 shutdown_connections 2024-04-02T13:40:12.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44887e0 0x564dc448ac60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc44787c0 0x564dc4476210 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc4475cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9c6ff5640 0 client.17241 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3142769061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc446f2c0 0x564dc4475790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 >> 172.21.15.122:0/3142769061 conn(0x564dc44699f0 msgr2=0x564dc446b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 shutdown_connections 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3142769061 wait complete. 2024-04-02T13:40:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.167+0000 7fd9ebf68740 20 client.17241 trim_cache size 0 max 16384 2024-04-02T13:40:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 Processor -- start 2024-04-02T13:40:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 -- start start 2024-04-02T13:40:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc44841a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc4490530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41aef10 con 0x564dc446f2c0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41b0a50 con 0x564dc44787c0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41af5e0 con 0x564dc446fbe0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54204/0 (socket says 172.21.15.122:54204) 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 -- 172.21.15.122:0/3420871702 learned_addr learned my addr 172.21.15.122:0/3420871702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc4490530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 -- 172.21.15.122:0/3420871702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 msgr2=0x564dc44841a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc44841a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 -- 172.21.15.122:0/3420871702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 msgr2=0x564dc4490530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc4490530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e2ffd640 1 -- 172.21.15.122:0/3420871702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc4473ce0 con 0x564dc446f2c0 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.184+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc4490530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x564dc448b4a0 tx=0x7fd9d4005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461535829c848db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9c8ff9640 1 -- 172.21.15.122:0/3420871702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d401b070 con 0x564dc446f2c0 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9c8ff9640 1 -- 172.21.15.122:0/3420871702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d4017dc0 con 0x564dc446f2c0 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9c8ff9640 1 -- 172.21.15.122:0/3420871702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d400c660 con 0x564dc446f2c0 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3420871702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 msgr2=0x564dc4477ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x564dc448b4a0 tx=0x7fd9d4005d10 comp rx=0 tx=0).stop 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3420871702 shutdown_connections 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc4490530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446fbe0 0x564dc44841a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3420871702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446f2c0 0x564dc4477ec0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3420871702 >> 172.21.15.122:0/3420871702 conn(0x564dc44699f0 msgr2=0x564dc4468ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3420871702 shutdown_connections 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.185+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3420871702 wait complete. 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 -- start start 2024-04-02T13:40:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 Processor -- start 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446f2c0 0x564dc4480960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc447d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41b0380 con 0x564dc446fbe0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41afcb0 con 0x564dc44787c0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e41acef0 con 0x564dc446f2c0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9ebf68740 10 client.0 _ll_register_callbacks cb 0x7fd9dc008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54220/0 (socket says 172.21.15.122:54220) 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 -- 172.21.15.122:0/3473992224 learned_addr learned my addr 172.21.15.122:0/3473992224 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446f2c0 0x564dc4480960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc447d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 -- 172.21.15.122:0/3473992224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446f2c0 msgr2=0x564dc4480960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446f2c0 0x564dc4480960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 -- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 msgr2=0x564dc447d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc447d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 -- 172.21.15.122:0/3473992224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc447d910 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e37fe640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc447d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9e27fc640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80bc700 tx=0x7fd9d8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922be3e63650f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.186+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d804b040 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fd9d80f2340 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9d4017dc0 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd9d80c2800 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 10 client.17247 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564dc44efc90 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9d4017dc0 con 0x564dc446fbe0 2024-04-02T13:40:12.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd9d80c6c30 con 0x564dc446fbe0 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 10 client.17247 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 10 client.17247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 20 client.17247 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44f0320 0x564dc44f2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 4 client.17247 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564dc43db000 con 0x564dc44f0320 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.187+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44f0320 0x564dc44f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.188+0000 7fd9e2ffd640 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44f0320 0x564dc44f2740 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9b5c67b93303a5 server_cookie=79e4849dabecf4f3 in_seq=0 out_seq=0 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.188+0000 7fd9c6ff5640 10 client.17247 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9c6ff5640 1 -- 172.21.15.122:0/3473992224 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564dc43db000 con 0x564dc44f0320 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9c6ff5640 10 client.17247 handle_command_reply: tid=0 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 client.17247 shutdown 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 msgr2=0x564dc447cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80bc700 tx=0x7fd9d8009560 comp rx=0 tx=0).stop 2024-04-02T13:40:12.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 shutdown_connections 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564dc44f0320 0x564dc44f2740 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564dc44787c0 0x564dc447d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564dc446fbe0 0x564dc447cce0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 --2- 172.21.15.122:0/3473992224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564dc446f2c0 0x564dc4480960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 >> 172.21.15.122:0/3473992224 conn(0x564dc44699f0 msgr2=0x564dc4475750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9c6ff5640 0 client.17247 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 shutdown_connections 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 1 -- 172.21.15.122:0/3473992224 wait complete. 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9ebf68740 20 client.17247 trim_cache size 0 max 16384 2024-04-02T13:40:12.198 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:40:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "948b1006-2c83-4e26-9cee-b69c5082885a", 2024-04-02T13:40:12.199 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:40:12.199 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 msgr2=0x7fd9cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 0x7fd9cc0a61e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc007db0 tx=0x7fd9dc00c560 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 msgr2=0x7fd9e41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8004f50 tx=0x7fd9d802ce20 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.189+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 shutdown_connections 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd9cc0a3d20 0x7fd9cc0a61e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9e41069a0 0x7fd9e419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9e4101350 0x7fd9e41988d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 --2- 172.21.15.122:0/4005433854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9e4100990 0x7fd9e4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 >> 172.21.15.122:0/4005433854 conn(0x7fd9e4077ad0 msgr2=0x7fd9e4103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 shutdown_connections 2024-04-02T13:40:12.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.190+0000 7fd9e92a7640 1 -- 172.21.15.122:0/4005433854 wait complete. 2024-04-02T13:40:12.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:12.203 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 -- 172.21.15.122:0/3532646280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 msgr2=0x7f7f4c069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3532646280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c069180 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009930 tx=0x7f7f4002bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 -- 172.21.15.122:0/3532646280 shutdown_connections 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3532646280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c0696c0 0x7f7f4c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3532646280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c069180 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3532646280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c107b50 0x7f7f4c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 -- 172.21.15.122:0/3532646280 >> 172.21.15.122:0/3532646280 conn(0x7f7f4c077b00 msgr2=0x7f7f4c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 -- 172.21.15.122:0/3532646280 shutdown_connections 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.326+0000 7f7f540d3640 1 -- 172.21.15.122:0/3532646280 wait complete. 2024-04-02T13:40:12.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 Processor -- start 2024-04-02T13:40:12.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 -- start start 2024-04-02T13:40:12.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c0696c0 0x7f7f4c100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107b50 0x7f7f4c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c111ea0 con 0x7f7f4c068d00 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c111d20 con 0x7f7f4c107b50 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f540d3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112020 con 0x7f7f4c0696c0 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54238/0 (socket says 172.21.15.122:54238) 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f51e6f640 1 -- 172.21.15.122:0/3292568535 learned_addr learned my addr 172.21.15.122:0/3292568535 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f52670640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107b50 0x7f7f4c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.327+0000 7f7f5166e640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c0696c0 0x7f7f4c100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 -- 172.21.15.122:0/3292568535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c0696c0 msgr2=0x7f7f4c100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c0696c0 0x7f7f4c100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 -- 172.21.15.122:0/3292568535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107b50 msgr2=0x7f7f4c101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107b50 0x7f7f4c101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 -- 172.21.15.122:0/3292568535 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c1a6880 con 0x7f7f4c068d00 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e9f0 tx=0x7f7f4800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f3effd640 1 -- 172.21.15.122:0/3292568535 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800cd80 con 0x7f7f4c068d00 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c1a6b70 con 0x7f7f4c068d00 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a7090 con 0x7f7f4c068d00 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.328+0000 7f7f3effd640 1 -- 172.21.15.122:0/3292568535 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f48005500 con 0x7f7f4c068d00 2024-04-02T13:40:12.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.329+0000 7f7f3effd640 1 -- 172.21.15.122:0/3292568535 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48010690 con 0x7f7f4c068d00 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.331+0000 7f7f3effd640 1 -- 172.21.15.122:0/3292568535 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7f48010830 con 0x7f7f4c068d00 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.331+0000 7f7f3effd640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 0x7f7f380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.331+0000 7f7f56d8c740 1 Processor -- start 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f3effd640 1 -- 172.21.15.122:0/3292568535 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7f480d0e90 con 0x7f7f4c068d00 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f5166e640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 0x7f7f380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 -- start start 2024-04-02T13:40:12.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd83890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56037dcde830 con 0x56037dd7b1f0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56037dc86820 con 0x56037dd83dd0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56037dcda7e0 con 0x56037dd7a8d0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f52670640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:42720/0 (socket says 172.21.15.122:42720) 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 -- 172.21.15.122:0/433762827 learned_addr learned my addr 172.21.15.122:0/433762827 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f5166e640 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd83890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f52670640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd861c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50822/0 (socket says 172.21.15.122:50822) 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f5166e640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 0x7f7f380a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005ab0 tx=0x7f7f40000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 -- 172.21.15.122:0/433762827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 msgr2=0x56037dd861c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd861c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 -- 172.21.15.122:0/433762827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 msgr2=0x56037dd83890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd83890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.332+0000 7f7f51e6f640 1 -- 172.21.15.122:0/433762827 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037dabc7d0 con 0x56037dd7a8d0 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.333+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f480a51e0 tx=0x7f7f4809c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe70e241506cc290 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.333+0000 7f7f37fff640 1 -- 172.21.15.122:0/433762827 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480a5710 con 0x56037dd7a8d0 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.333+0000 7f7f37fff640 1 -- 172.21.15.122:0/433762827 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f480ce4d0 con 0x56037dd7a8d0 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.333+0000 7f7f37fff640 1 -- 172.21.15.122:0/433762827 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4809bf50 con 0x56037dd7a8d0 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- 172.21.15.122:0/433762827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 msgr2=0x56037dd7acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f480a51e0 tx=0x7f7f4809c970 comp rx=0 tx=0).stop 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- 172.21.15.122:0/433762827 shutdown_connections 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd861c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd83890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/433762827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7acb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- 172.21.15.122:0/433762827 >> 172.21.15.122:0/433762827 conn(0x56037dd75000 msgr2=0x56037dd753e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- 172.21.15.122:0/433762827 shutdown_connections 2024-04-02T13:40:12.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- 172.21.15.122:0/433762827 wait complete. 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.334+0000 7f7f56d8c740 1 -- start start 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 Processor -- start 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd7b1f0 0x56037dd80ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd83dd0 0x56037dd81010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56037dd86b60 con 0x56037dd7a8d0 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56037dc99d30 con 0x56037dd7b1f0 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56037dd86ce0 con 0x56037dd83dd0 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f56d8c740 10 client.0 _ll_register_callbacks cb 0x56037dc99720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f51e6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54248/0 (socket says 172.21.15.122:54248) 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f51e6f640 1 -- 172.21.15.122:0/2400739682 learned_addr learned my addr 172.21.15.122:0/2400739682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.335+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd7b1f0 0x56037dd80ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f52670640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd83dd0 0x56037dd81010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 -- 172.21.15.122:0/2400739682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd83dd0 msgr2=0x56037dd81010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd83dd0 0x56037dd81010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 -- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd7b1f0 msgr2=0x56037dd80ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd7b1f0 0x56037dd80ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 -- 172.21.15.122:0/2400739682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037d9f23f0 con 0x56037dd7a8d0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b3040 tx=0x7f7f4809c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2418f2313350d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480cc060 con 0x56037dd7a8d0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7f4809c150 con 0x56037dd7a8d0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f480ce4d0 con 0x56037dd7a8d0 2024-04-02T13:40:12.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7f480a5500 con 0x56037dd7a8d0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f56d8c740 10 client.17259 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56037d9b20f0 con 0x56037dd7a8d0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.336+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f480ce4d0 con 0x56037dd7a8d0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7f480a2070 con 0x56037dd7a8d0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 10 client.17259 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 20 client.17259 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037dd9c460 0x56037dd9e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 4 client.17259 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56037dcebd70 con 0x56037dd9c460 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037dd9c460 0x56037dd9e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f357fa640 10 client.17259 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.337+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037dd9c460 0x56037dd9e850 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b26ec44842789aad server_cookie=468353af95ca4e28 in_seq=0 out_seq=0 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f357fa640 1 -- 172.21.15.122:0/2400739682 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56037dcebd70 con 0x56037dd9c460 2024-04-02T13:40:12.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f357fa640 10 client.17259 handle_command_reply: tid=0 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f56d8c740 1 client.17259 shutdown 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 msgr2=0x56037dd83720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b3040 tx=0x7f7f4809c550 comp rx=0 tx=0).stop 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.338+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 shutdown_connections 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037dd9c460 0x56037dd9e850 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd83dd0 0x56037dd81010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd7b1f0 0x56037dd80ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2400739682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7a8d0 0x56037dd83720 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f357fa640 0 client.17259 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 >> 172.21.15.122:0/2400739682 conn(0x56037dd75000 msgr2=0x56037dd76d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 shutdown_connections 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2400739682 wait complete. 2024-04-02T13:40:12.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.339+0000 7f7f56d8c740 20 client.17259 trim_cache size 0 max 16384 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 Processor -- start 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 -- start start 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd83720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ace00 con 0x56037dd7b1f0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae940 con 0x56037dd83dd0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ad4d0 con 0x56037dd7a8d0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54264/0 (socket says 172.21.15.122:54264) 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 -- 172.21.15.122:0/742035280 learned_addr learned my addr 172.21.15.122:0/742035280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f52670640 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd83720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 -- 172.21.15.122:0/742035280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 msgr2=0x56037dd7b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 -- 172.21.15.122:0/742035280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 msgr2=0x56037dd83720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd83720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 -- 172.21.15.122:0/742035280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56037dd74e30 con 0x56037dd7b1f0 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.356+0000 7f7f5166e640 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009900 tx=0x7f7f4003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588f49deec39d8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f377fe640 1 -- 172.21.15.122:0/742035280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4002cc30 con 0x56037dd7b1f0 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f377fe640 1 -- 172.21.15.122:0/742035280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40004ba0 con 0x56037dd7b1f0 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f377fe640 1 -- 172.21.15.122:0/742035280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400035f0 con 0x56037dd7b1f0 2024-04-02T13:40:12.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 -- 172.21.15.122:0/742035280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 msgr2=0x56037dd831e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009900 tx=0x7f7f4003f000 comp rx=0 tx=0).stop 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 -- 172.21.15.122:0/742035280 shutdown_connections 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd83720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd831e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/742035280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd7b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 -- 172.21.15.122:0/742035280 >> 172.21.15.122:0/742035280 conn(0x56037dd75000 msgr2=0x56037dd74240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 -- 172.21.15.122:0/742035280 shutdown_connections 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.357+0000 7f7f56d8c740 1 -- 172.21.15.122:0/742035280 wait complete. 2024-04-02T13:40:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 -- start start 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 Processor -- start 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd9f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd9fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ae270 con 0x56037dd7b1f0 2024-04-02T13:40:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1adba0 con 0x56037dd83dd0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aade0 con 0x56037dd7a8d0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f56d8c740 10 client.0 _ll_register_callbacks cb 0x7f7f4809c640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54266/0 (socket says 172.21.15.122:54266) 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 -- 172.21.15.122:0/2772078762 learned_addr learned my addr 172.21.15.122:0/2772078762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd9f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f52670640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd9fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 -- 172.21.15.122:0/2772078762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 msgr2=0x56037dd9f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd9f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 -- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 msgr2=0x56037dd9fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd9fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 -- 172.21.15.122:0/2772078762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56037de01810 con 0x56037dd7b1f0 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f52670640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd9fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f5166e640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002c170 tx=0x7f7f400045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d027f31f1cf889f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.358+0000 7f7f357fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f40065060 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7f4002ca30 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f40004ba0 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7f40005150 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 10 client.17265 fetch_fsmap learned FSMap version 36 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56037de02ae0 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f40004ba0 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7f40003c20 con 0x56037dd7b1f0 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 10 client.17265 fetch_fsmap finished waiting for FSMap version 36 2024-04-02T13:40:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 20 client.17265 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037de03230 0x56037de05650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 4 client.17265 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56037dce60c0 con 0x56037de03230 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.359+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037de03230 0x56037de05650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f357fa640 10 client.17265 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f51e6f640 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037de03230 0x56037de05650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258c56aa2aeab4f4 server_cookie=a603ca7cedb9369d in_seq=0 out_seq=0 2024-04-02T13:40:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f357fa640 1 -- 172.21.15.122:0/2772078762 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56037dce60c0 con 0x56037de03230 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f357fa640 10 client.17265 handle_command_reply: tid=0 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 client.17265 shutdown 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 msgr2=0x56037dd9f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7f4002c170 tx=0x7f7f400045c0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 shutdown_connections 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56037de03230 0x56037de05650 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56037dd83dd0 0x56037dd9fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56037dd7b1f0 0x56037dd9f990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f357fa640 0 client.17265 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 --2- 172.21.15.122:0/2772078762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56037dd7a8d0 0x56037dd9f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 >> 172.21.15.122:0/2772078762 conn(0x56037dd75000 msgr2=0x56037dd739b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 shutdown_connections 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 1 -- 172.21.15.122:0/2772078762 wait complete. 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.360+0000 7f7f56d8c740 20 client.17265 trim_cache size 0 max 16384 2024-04-02T13:40:12.375 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "948b1006-2c83-4e26-9cee-b69c5082885a": { 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "948b1006-2c83-4e26-9cee-b69c5082885a", 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:40:12.376 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:12.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 msgr2=0x7f7f380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 0x7f7f380a6260 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f40005ab0 tx=0x7f7f40000bc0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 msgr2=0x7f7f4c1009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7f4800e9f0 tx=0x7f7f4800edf0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 shutdown_connections 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7f380a3da0 0x7f7f380a6260 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f4c107b50 0x7f7f4c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4c0696c0 0x7f7f4c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 --2- 172.21.15.122:0/3292568535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f4c068d00 0x7f7f4c1009c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:12.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 >> 172.21.15.122:0/3292568535 conn(0x7f7f4c077b00 msgr2=0x7f7f4c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:12.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 shutdown_connections 2024-04-02T13:40:12.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:12.361+0000 7f7f540d3640 1 -- 172.21.15.122:0/3292568535 wait complete. 2024-04-02T13:40:12.381 INFO:tasks.cephfs.filesystem:scrub status for tag:948b1006-2c83-4e26-9cee-b69c5082885a - {'path': '/', 'tag': '948b1006-2c83-4e26-9cee-b69c5082885a', 'options': 'recursive,force'} 2024-04-02T13:40:12.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 146250 2.67 MB/sec execute 93 sec latency 2053.886 ms 2024-04-02T13:40:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: pgmap v154: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-04-02T13:40:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:40:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:13 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: pgmap v154: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:13 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: pgmap v154: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:13 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:13.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 147862 2.72 MB/sec execute 94 sec latency 1062.127 ms 2024-04-02T13:40:14.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 149533 2.82 MB/sec execute 95 sec latency 10.397 ms 2024-04-02T13:40:15.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:15 smithi187 ceph-mon[36064]: pgmap v155: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 911 KiB/s wr, 114 op/s 2024-04-02T13:40:15.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:15 smithi188 ceph-mon[36117]: pgmap v155: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 911 KiB/s wr, 114 op/s 2024-04-02T13:40:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:15 smithi122 ceph-mon[32114]: pgmap v155: 129 pgs: 129 active+clean; 96 MiB data, 897 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 911 KiB/s wr, 114 op/s 2024-04-02T13:40:15.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 149723 2.79 MB/sec execute 96 sec latency 835.563 ms 2024-04-02T13:40:16.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 149723 2.76 MB/sec execute 97 sec latency 1835.607 ms 2024-04-02T13:40:17.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:17 smithi187 ceph-mon[36064]: pgmap v156: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-02T13:40:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:17 smithi188 ceph-mon[36117]: pgmap v156: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-02T13:40:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:17 smithi122 ceph-mon[32114]: pgmap v156: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-04-02T13:40:17.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 149723 2.73 MB/sec execute 98 sec latency 2835.687 ms 2024-04-02T13:40:18.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 150668 2.75 MB/sec execute 99 sec latency 3288.167 ms 2024-04-02T13:40:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:19 smithi187 ceph-mon[36064]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:19.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:19 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:19.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:19 smithi188 ceph-mon[36117]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:19 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:19 smithi122 ceph-mon[32114]: pgmap v157: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:19 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:40:19.953 INFO:tasks.workunit.client.0.smithi122.stdout: 1 152802 2.86 MB/sec execute 100 sec latency 15.374 ms 2024-04-02T13:40:20.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:20 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:20.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:20 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:20.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:20 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:40:20.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:20 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:20.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:20 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:20.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:20 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:40:20.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:20 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:40:20.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:20 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:40:20.573 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:20 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:40:20.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 153341 2.84 MB/sec execute 101 sec latency 448.369 ms 2024-04-02T13:40:21.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:21 smithi187 ceph-mon[36064]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:21.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:21 smithi188 ceph-mon[36117]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:21 smithi122 ceph-mon[32114]: pgmap v158: 129 pgs: 129 active+clean; 98 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 889 KiB/s wr, 103 op/s 2024-04-02T13:40:21.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 153341 2.81 MB/sec execute 102 sec latency 1448.476 ms 2024-04-02T13:40:22.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:22.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:22.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:22.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:22.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:22.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:22.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:22.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:22.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:22.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:22.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 153341 2.79 MB/sec execute 103 sec latency 2448.605 ms 2024-04-02T13:40:23.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:23 smithi187 ceph-mon[36064]: pgmap v159: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-02T13:40:23.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:23 smithi188 ceph-mon[36117]: pgmap v159: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-02T13:40:23.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:23 smithi122 ceph-mon[32114]: pgmap v159: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-02T13:40:23.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 154326 2.81 MB/sec execute 104 sec latency 2897.404 ms 2024-04-02T13:40:24.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156461 2.91 MB/sec execute 105 sec latency 13.638 ms 2024-04-02T13:40:25.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:25 smithi187 ceph-mon[36064]: pgmap v160: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 850 KiB/s wr, 109 op/s 2024-04-02T13:40:25.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:25 smithi188 ceph-mon[36117]: pgmap v160: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 850 KiB/s wr, 109 op/s 2024-04-02T13:40:25.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:25 smithi122 ceph-mon[32114]: pgmap v160: 129 pgs: 129 active+clean; 101 MiB data, 984 MiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 850 KiB/s wr, 109 op/s 2024-04-02T13:40:25.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156815 2.89 MB/sec execute 106 sec latency 633.172 ms 2024-04-02T13:40:26.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156815 2.86 MB/sec execute 107 sec latency 1633.247 ms 2024-04-02T13:40:27.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:27 smithi187 ceph-mon[36064]: pgmap v161: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:40:27.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:27 smithi188 ceph-mon[36117]: pgmap v161: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:40:27.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:27 smithi122 ceph-mon[32114]: pgmap v161: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-02T13:40:27.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156815 2.83 MB/sec execute 108 sec latency 2633.283 ms 2024-04-02T13:40:28.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156883 2.81 MB/sec execute 109 sec latency 3088.108 ms 2024-04-02T13:40:29.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:29 smithi187 ceph-mon[36064]: pgmap v162: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:29.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:29 smithi188 ceph-mon[36117]: pgmap v162: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:29 smithi122 ceph-mon[32114]: pgmap v162: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:29.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156883 2.78 MB/sec execute 110 sec latency 1494.198 ms 2024-04-02T13:40:30.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156883 2.76 MB/sec execute 111 sec latency 2494.320 ms 2024-04-02T13:40:31.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:31 smithi187 ceph-mon[36064]: pgmap v163: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:31.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:31 smithi188 ceph-mon[36117]: pgmap v163: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:31.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:31 smithi122 ceph-mon[32114]: pgmap v163: 129 pgs: 129 active+clean; 103 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 115 op/s 2024-04-02T13:40:31.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156883 2.73 MB/sec execute 112 sec latency 3494.431 ms 2024-04-02T13:40:32.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 156883 2.71 MB/sec execute 113 sec latency 4494.527 ms 2024-04-02T13:40:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:33 smithi187 ceph-mon[36064]: pgmap v164: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-04-02T13:40:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:33 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:33.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:33 smithi188 ceph-mon[36117]: pgmap v164: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-04-02T13:40:33.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:33 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:33 smithi122 ceph-mon[32114]: pgmap v164: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-04-02T13:40:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:33 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:33.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 157983 2.73 MB/sec execute 114 sec latency 4941.409 ms 2024-04-02T13:40:34.954 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160048 2.82 MB/sec execute 115 sec latency 11.961 ms 2024-04-02T13:40:35.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:35 smithi188 ceph-mon[36117]: pgmap v165: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 802 KiB/s wr, 76 op/s 2024-04-02T13:40:35.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:35 smithi122 ceph-mon[32114]: pgmap v165: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 802 KiB/s wr, 76 op/s 2024-04-02T13:40:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:35 smithi187 ceph-mon[36064]: pgmap v165: 129 pgs: 129 active+clean; 104 MiB data, 979 MiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 802 KiB/s wr, 76 op/s 2024-04-02T13:40:35.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160362 2.80 MB/sec execute 116 sec latency 686.026 ms 2024-04-02T13:40:36.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160362 2.78 MB/sec execute 117 sec latency 1686.126 ms 2024-04-02T13:40:37.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:37 smithi188 ceph-mon[36117]: pgmap v166: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-04-02T13:40:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:37 smithi122 ceph-mon[32114]: pgmap v166: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-04-02T13:40:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:37 smithi187 ceph-mon[36064]: pgmap v166: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 129 op/s 2024-04-02T13:40:37.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160362 2.75 MB/sec execute 118 sec latency 2686.189 ms 2024-04-02T13:40:38.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:38 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:38.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:38 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:38.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:38 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:40:38.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160430 2.73 MB/sec execute 119 sec latency 3277.383 ms 2024-04-02T13:40:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:39 smithi188 ceph-mon[36117]: pgmap v167: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:39.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:39 smithi122 ceph-mon[32114]: pgmap v167: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:39 smithi187 ceph-mon[36064]: pgmap v167: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:39.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160430 2.71 MB/sec execute 120 sec latency 1356.305 ms 2024-04-02T13:40:40.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160430 2.69 MB/sec execute 121 sec latency 2356.398 ms 2024-04-02T13:40:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:41 smithi188 ceph-mon[36117]: pgmap v168: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:41 smithi122 ceph-mon[32114]: pgmap v168: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:41 smithi187 ceph-mon[36064]: pgmap v168: 129 pgs: 129 active+clean; 106 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 745 KiB/s wr, 73 op/s 2024-04-02T13:40:41.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160430 2.67 MB/sec execute 122 sec latency 3356.478 ms 2024-04-02T13:40:42.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:42.383 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 -- 172.21.15.122:0/1882161248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9868100990 msgr2=0x7f9868100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 --2- 172.21.15.122:0/1882161248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9868100990 0x7f9868100e10 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f985c009930 tx=0x7f985c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 -- 172.21.15.122:0/1882161248 shutdown_connections 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 --2- 172.21.15.122:0/1882161248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868101350 0x7f986810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 --2- 172.21.15.122:0/1882161248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9868100990 0x7f9868100e10 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 --2- 172.21.15.122:0/1882161248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f98681069a0 0x7f9868106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 -- 172.21.15.122:0/1882161248 >> 172.21.15.122:0/1882161248 conn(0x7f9868077ad0 msgr2=0x7f9868077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 -- 172.21.15.122:0/1882161248 shutdown_connections 2024-04-02T13:40:42.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.496+0000 7f986ea14640 1 -- 172.21.15.122:0/1882161248 wait complete. 2024-04-02T13:40:42.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 Processor -- start 2024-04-02T13:40:42.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 -- start start 2024-04-02T13:40:42.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868100990 0x7f9868199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9868101350 0x7f9868199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110db0 con 0x7f98681069a0 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110c30 con 0x7f9868101350 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986ea14640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9868110f30 con 0x7f9868100990 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55232/0 (socket says 172.21.15.122:55232) 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f986cfb1640 1 -- 172.21.15.122:0/1421618320 learned_addr learned my addr 172.21.15.122:0/1421618320 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.497+0000 7f98677fe640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9868101350 0x7f9868199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f9867fff640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868100990 0x7f9868199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 -- 172.21.15.122:0/1421618320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868100990 msgr2=0x7f9868199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868100990 0x7f9868199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 -- 172.21.15.122:0/1421618320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9868101350 msgr2=0x7f9868199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9868101350 0x7f9868199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 -- 172.21.15.122:0/1421618320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98681a46f0 con 0x7f98681069a0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f986000eaa0 tx=0x7f986000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f98657fa640 1 -- 172.21.15.122:0/1421618320 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000cdb0 con 0x7f98681069a0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98681a48c0 con 0x7f98681069a0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f98657fa640 1 -- 172.21.15.122:0/1421618320 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9860005020 con 0x7f98681069a0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f98657fa640 1 -- 172.21.15.122:0/1421618320 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600116b0 con 0x7f98681069a0 2024-04-02T13:40:42.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.498+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98681a4de0 con 0x7f98681069a0 2024-04-02T13:40:42.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.500+0000 7f98657fa640 1 -- 172.21.15.122:0/1421618320 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9860005560 con 0x7f98681069a0 2024-04-02T13:40:42.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 Processor -- start 2024-04-02T13:40:42.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 -- start start 2024-04-02T13:40:42.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98657fa640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 0x7f98500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16828d0 0x55e8c1682cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c168b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e8c15e6830 con 0x55e8c168bdd0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e8c158e820 con 0x55e8c16831f0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98715ff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e8c15e27e0 con 0x55e8c16828d0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98677fe640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 0x7f98500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98657fa640 1 -- 172.21.15.122:0/1421618320 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f98600d0dc0 con 0x7f98681069a0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f98677fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c168b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55236/0 (socket says 172.21.15.122:55236) 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 -- 172.21.15.122:0/1025589014 learned_addr learned my addr 172.21.15.122:0/1025589014 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f9867fff640 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16828d0 0x55e8c1682cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 -- 172.21.15.122:0/1025589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16828d0 msgr2=0x55e8c1682cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16828d0 0x55e8c1682cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 -- 172.21.15.122:0/1025589014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 msgr2=0x55e8c168b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c168b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.501+0000 7f986cfb1640 1 -- 172.21.15.122:0/1025589014 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c13c47d0 con 0x55e8c168bdd0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f986cfb1640 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9860098e30 tx=0x7f98600103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835a776e43462213 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98677fe640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 0x7f98500a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f985c005ab0 tx=0x7f985c02c5d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f984e7fc640 1 -- 172.21.15.122:0/1025589014 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986009d0d0 con 0x55e8c168bdd0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f984e7fc640 1 -- 172.21.15.122:0/1025589014 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986009c9d0 con 0x55e8c168bdd0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f984e7fc640 1 -- 172.21.15.122:0/1025589014 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600ce400 con 0x55e8c168bdd0 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 -- 172.21.15.122:0/1025589014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 msgr2=0x55e8c168e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9860098e30 tx=0x7f98600103d0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 -- 172.21.15.122:0/1025589014 shutdown_connections 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c168e1c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c168b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 --2- 172.21.15.122:0/1025589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16828d0 0x55e8c1682cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.502+0000 7f98715ff740 1 -- 172.21.15.122:0/1025589014 >> 172.21.15.122:0/1025589014 conn(0x55e8c167d000 msgr2=0x55e8c167d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- 172.21.15.122:0/1025589014 shutdown_connections 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- 172.21.15.122:0/1025589014 wait complete. 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- start start 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 Processor -- start 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c1688a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c168bdd0 0x55e8c1688fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e8c168eb60 con 0x55e8c16828d0 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e8c15a1d30 con 0x55e8c168bdd0 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e8c168ece0 con 0x55e8c16831f0 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f98715ff740 10 client.0 _ll_register_callbacks cb 0x55e8c15a1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:42.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.503+0000 7f9867fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55240/0 (socket says 172.21.15.122:55240) 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 -- 172.21.15.122:0/3307741693 learned_addr learned my addr 172.21.15.122:0/3307741693 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98677fe640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c1688a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f986cfb1640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c168bdd0 0x55e8c1688fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 -- 172.21.15.122:0/3307741693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 msgr2=0x55e8c1688a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c1688a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 -- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c168bdd0 msgr2=0x55e8c1688fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c168bdd0 0x55e8c1688fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 -- 172.21.15.122:0/3307741693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c12fa3f0 con 0x55e8c16828d0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f9867fff640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9858010840 tx=0x7f9858010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c909f262340ceb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98477fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800cd20 con 0x55e8c16828d0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f985801c480 con 0x55e8c16828d0 2024-04-02T13:40:42.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f986009c9d0 con 0x55e8c16828d0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9858025050 con 0x55e8c16828d0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98715ff740 10 client.17277 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e8c12ba0f0 con 0x55e8c16828d0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.504+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f986009c9d0 con 0x55e8c16828d0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f9858011740 con 0x55e8c16828d0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 10 client.17277 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 20 client.17277 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c169be60 0x55e8c169e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 4 client.17277 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8c15f3d70 con 0x55e8c169be60 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98677fe640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c169be60 0x55e8c169e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98477fe640 10 client.17277 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.505+0000 7f98677fe640 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c169be60 0x55e8c169e2c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330c882e2b50225 server_cookie=64ed64d1a73808d4 in_seq=0 out_seq=0 2024-04-02T13:40:42.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98477fe640 1 -- 172.21.15.122:0/3307741693 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e8c15f3d70 con 0x55e8c169be60 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98477fe640 10 client.17277 handle_command_reply: tid=0 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 client.17277 shutdown 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 msgr2=0x55e8c168b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9858010840 tx=0x7f9858010d00 comp rx=0 tx=0).stop 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 shutdown_connections 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c169be60 0x55e8c169e2c0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c168bdd0 0x55e8c1688fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c1688a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98477fe640 0 client.17277 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.506+0000 7f98715ff740 1 --2- 172.21.15.122:0/3307741693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c168b6d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.507+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 >> 172.21.15.122:0/3307741693 conn(0x55e8c167d000 msgr2=0x55e8c167ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.507+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 shutdown_connections 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.507+0000 7f98715ff740 1 -- 172.21.15.122:0/3307741693 wait complete. 2024-04-02T13:40:42.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.507+0000 7f98715ff740 20 client.17277 trim_cache size 0 max 16384 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 Processor -- start 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 -- start start 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c1694310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 0x55e8c1683e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98681aef30 con 0x55e8c16828d0 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98681b0b30 con 0x55e8c16831f0 2024-04-02T13:40:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f98715ff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98681af600 con 0x55e8c168bdd0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f9867fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f9867fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55244/0 (socket says 172.21.15.122:55244) 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.524+0000 7f9867fff640 1 -- 172.21.15.122:0/2503963882 learned_addr learned my addr 172.21.15.122:0/2503963882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f986cfb1640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 0x55e8c1683e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 -- 172.21.15.122:0/2503963882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 msgr2=0x55e8c1683e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 0x55e8c1683e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 -- 172.21.15.122:0/2503963882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 msgr2=0x55e8c1694310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f98677fe640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c1694310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c1694310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 -- 172.21.15.122:0/2503963882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8c1683020 con 0x55e8c16828d0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f986cfb1640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 0x55e8c1683e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f9867fff640 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9858010810 tx=0x7f9858005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c45d945ea5ffee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f984dffb640 1 -- 172.21.15.122:0/2503963882 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985800e600 con 0x55e8c16828d0 2024-04-02T13:40:42.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f984dffb640 1 -- 172.21.15.122:0/2503963882 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858005070 con 0x55e8c16828d0 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.525+0000 7f984dffb640 1 -- 172.21.15.122:0/2503963882 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580035f0 con 0x55e8c16828d0 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- 172.21.15.122:0/2503963882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 msgr2=0x55e8c167c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9858010810 tx=0x7f9858005690 comp rx=0 tx=0).stop 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- 172.21.15.122:0/2503963882 shutdown_connections 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c168bdd0 0x55e8c1683e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16831f0 0x55e8c1694310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- 172.21.15.122:0/2503963882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c16828d0 0x55e8c167c240 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- 172.21.15.122:0/2503963882 >> 172.21.15.122:0/2503963882 conn(0x55e8c167d000 msgr2=0x55e8c1683710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- 172.21.15.122:0/2503963882 shutdown_connections 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- 172.21.15.122:0/2503963882 wait complete. 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- start start 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 Processor -- start 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 0x55e8c16fa400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c16fa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f98681b03a0 con 0x55e8c168bdd0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f98681afcd0 con 0x55e8c16828d0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.526+0000 7f98715ff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98681acf10 con 0x55e8c16831f0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98715ff740 10 client.0 _ll_register_callbacks cb 0x7f98580103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55246/0 (socket says 172.21.15.122:55246) 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 -- 172.21.15.122:0/2673320043 learned_addr learned my addr 172.21.15.122:0/2673320043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f9867fff640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 0x55e8c16fa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98677fe640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c16fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 -- 172.21.15.122:0/2673320043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 msgr2=0x55e8c16fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c16fa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 -- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 msgr2=0x55e8c16fa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 0x55e8c16fa400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 -- 172.21.15.122:0/2673320043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8c17059c0 con 0x55e8c168bdd0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f9867fff640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 0x55e8c16fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f986cfb1640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9860009270 tx=0x7f9860010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcfca861eb732d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98477fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98600bb060 con 0x55e8c168bdd0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f9860099040 con 0x55e8c168bdd0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9858005070 con 0x55e8c168bdd0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f98600b5030 con 0x55e8c168bdd0 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98715ff740 10 client.17283 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e8c1706ac0 con 0x55e8c168bdd0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.527+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9858005070 con 0x55e8c168bdd0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f986007dce0 con 0x55e8c168bdd0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 10 client.17283 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 20 client.17283 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c1706df0 0x55e8c1709210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 4 client.17283 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8c15ee0c0 con 0x55e8c1706df0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f9867fff640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c1706df0 0x55e8c1709210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f98477fe640 10 client.17283 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.528+0000 7f9867fff640 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c1706df0 0x55e8c1709210 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695851fd257dde6f server_cookie=acc7c6feac967f7c in_seq=0 out_seq=0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98477fe640 1 -- 172.21.15.122:0/2673320043 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e8c15ee0c0 con 0x55e8c1706df0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98477fe640 10 client.17283 handle_command_reply: tid=0 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 client.17283 shutdown 2024-04-02T13:40:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 msgr2=0x55e8c1705480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9860009270 tx=0x7f9860010f60 comp rx=0 tx=0).stop 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 shutdown_connections 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e8c1706df0 0x55e8c1709210 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e8c168bdd0 0x55e8c1705480 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e8c16831f0 0x55e8c16fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 --2- 172.21.15.122:0/2673320043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e8c16828d0 0x55e8c16fa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98477fe640 0 client.17283 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 >> 172.21.15.122:0/2673320043 conn(0x55e8c167d000 msgr2=0x55e8c167c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 shutdown_connections 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 1 -- 172.21.15.122:0/2673320043 wait complete. 2024-04-02T13:40:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f98715ff740 20 client.17283 trim_cache size 0 max 16384 2024-04-02T13:40:42.541 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:42.541 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:40:42.541 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:40:42.541 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 msgr2=0x7f98500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 0x7f98500a61e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f985c005ab0 tx=0x7f985c02c5d0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 msgr2=0x7f98681a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.529+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f986000eaa0 tx=0x7f986000ef60 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 shutdown_connections 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f98500a3d20 0x7f98500a61e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f98681069a0 0x7f98681a41b0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9868101350 0x7f9868199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 --2- 172.21.15.122:0/1421618320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9868100990 0x7f9868199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 >> 172.21.15.122:0/1421618320 conn(0x7f9868077ad0 msgr2=0x7f9868103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 shutdown_connections 2024-04-02T13:40:42.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.530+0000 7f986ea14640 1 -- 172.21.15.122:0/1421618320 wait complete. 2024-04-02T13:40:42.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:40:42.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:40:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 -- 172.21.15.122:0/2332424656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 msgr2=0x7f0764107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 --2- 172.21.15.122:0/2332424656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764107e80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0754009930 tx=0x7f075402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 -- 172.21.15.122:0/2332424656 shutdown_connections 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 --2- 172.21.15.122:0/2332424656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 0x7f076410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 --2- 172.21.15.122:0/2332424656 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 --2- 172.21.15.122:0/2332424656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764107e80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 -- 172.21.15.122:0/2332424656 >> 172.21.15.122:0/2332424656 conn(0x7f07640fd5a0 msgr2=0x7f07640ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 -- 172.21.15.122:0/2332424656 shutdown_connections 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.643+0000 7f0769493640 1 -- 172.21.15.122:0/2332424656 wait complete. 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 Processor -- start 2024-04-02T13:40:42.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 -- start start 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 0x7f0764074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0764111eb0 con 0x7f0764107aa0 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0764111d30 con 0x7f0764101a90 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0769493640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0764112030 con 0x7f0764102450 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55260/0 (socket says 172.21.15.122:55260) 2024-04-02T13:40:42.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 -- 172.21.15.122:0/769743423 learned_addr learned my addr 172.21.15.122:0/769743423 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f075bfff640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 0x7f0764074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0762ffd640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 -- 172.21.15.122:0/769743423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 msgr2=0x7f0764074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 0x7f0764074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 -- 172.21.15.122:0/769743423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 msgr2=0x7f0764073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 -- 172.21.15.122:0/769743423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07641a68f0 con 0x7f0764107aa0 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0762ffd640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f07637fe640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b970 tx=0x7f075c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.644+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c019070 con 0x7f0764107aa0 2024-04-02T13:40:42.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.645+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f075c00cb40 con 0x7f0764107aa0 2024-04-02T13:40:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.645+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07641a6be0 con 0x7f0764107aa0 2024-04-02T13:40:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.645+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c007650 con 0x7f0764107aa0 2024-04-02T13:40:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.645+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07641a70e0 con 0x7f0764107aa0 2024-04-02T13:40:42.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.646+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0764068d40 con 0x7f0764107aa0 2024-04-02T13:40:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.647+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f075c01c070 con 0x7f0764107aa0 2024-04-02T13:40:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.647+0000 7f0760ff9640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 0x7f07500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.647+0000 7f0762ffd640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 0x7f07500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.647+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f075c0d0090 con 0x7f0764107aa0 2024-04-02T13:40:42.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.648+0000 7f0762ffd640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 0x7f07500a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0754036040 tx=0x7f0754000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:42.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.652+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f075c09c4b0 con 0x7f0764107aa0 2024-04-02T13:40:42.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.884+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f076410a000 con 0x7f0764107aa0 2024-04-02T13:40:42.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.885+0000 7f0760ff9640 1 -- 172.21.15.122:0/769743423 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11949 (secure 0 0 0) 0x7f075c09bdb0 con 0x7f0764107aa0 2024-04-02T13:40:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:40:42.888 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:40:36.405088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:40:42.889 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 38 2024-04-02T13:40:42.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 msgr2=0x7f07500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 0x7f07500a6210 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0754036040 tx=0x7f0754000bc0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 msgr2=0x7f0764074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b970 tx=0x7f075c00be30 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 shutdown_connections 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f07500a3d50 0x7f07500a6210 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0764107aa0 0x7f0764074880 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0764102450 0x7f0764074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 --2- 172.21.15.122:0/769743423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0764101a90 0x7f0764073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 >> 172.21.15.122:0/769743423 conn(0x7f07640fd5a0 msgr2=0x7f07640fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 shutdown_connections 2024-04-02T13:40:42.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:42.887+0000 7f0769493640 1 -- 172.21.15.122:0/769743423 wait complete. 2024-04-02T13:40:42.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:40:42.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 160430 2.64 MB/sec execute 123 sec latency 4356.512 ms 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 -- 172.21.15.122:0/1325674890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b300696c0 msgr2=0x7f7b3010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 --2- 172.21.15.122:0/1325674890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b300696c0 0x7f7b3010e960 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009930 tx=0x7f7b2002bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 -- 172.21.15.122:0/1325674890 shutdown_connections 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 --2- 172.21.15.122:0/1325674890 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b300696c0 0x7f7b3010e960 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 --2- 172.21.15.122:0/1325674890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 0x7f7b30069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 --2- 172.21.15.122:0/1325674890 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b30107b50 0x7f7b30107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 -- 172.21.15.122:0/1325674890 >> 172.21.15.122:0/1325674890 conn(0x7f7b30077b00 msgr2=0x7f7b30077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 -- 172.21.15.122:0/1325674890 shutdown_connections 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.002+0000 7f7b3852b640 1 -- 172.21.15.122:0/1325674890 wait complete. 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 Processor -- start 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 -- start start 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 0x7f7b301983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 0x7f7b301988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30111ea0 con 0x7f7b30107b50 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30111d20 con 0x7f7b300696c0 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b3852b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30112020 con 0x7f7b30068d00 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55272/0 (socket says 172.21.15.122:55272) 2024-04-02T13:40:43.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 -- 172.21.15.122:0/2399811404 learned_addr learned my addr 172.21.15.122:0/2399811404 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 -- 172.21.15.122:0/2399811404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 msgr2=0x7f7b301983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 0x7f7b301988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b362c7640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 0x7f7b301983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 0x7f7b301983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 -- 172.21.15.122:0/2399811404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 msgr2=0x7f7b301988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 0x7f7b301988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 -- 172.21.15.122:0/2399811404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3019d340 con 0x7f7b30107b50 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 0x7f7b301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.003+0000 7f7b36ac8640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7b2002efd0 tx=0x7f7b200045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.004+0000 7f7b2f7fe640 1 -- 172.21.15.122:0/2399811404 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20041070 con 0x7f7b30107b50 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.004+0000 7f7b2f7fe640 1 -- 172.21.15.122:0/2399811404 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2002cd40 con 0x7f7b30107b50 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.004+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3019d5d0 con 0x7f7b30107b50 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.004+0000 7f7b2f7fe640 1 -- 172.21.15.122:0/2399811404 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20003e50 con 0x7f7b30107b50 2024-04-02T13:40:43.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.004+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301a4ec0 con 0x7f7b30107b50 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b2f7fe640 1 -- 172.21.15.122:0/2399811404 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7b20044070 con 0x7f7b30107b50 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 Processor -- start 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 -- start start 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038c3ca0 0x55c5038c6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c50381e7b0 con 0x55c5038bb0c0 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c5037bd570 con 0x55c5038ba7a0 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c50381a760 con 0x55c5038c3ca0 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55274/0 (socket says 172.21.15.122:55274) 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 -- 172.21.15.122:0/1869642195 learned_addr learned my addr 172.21.15.122:0/1869642195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038c3ca0 0x55c5038c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b2f7fe640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 0x7f7b180a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 -- 172.21.15.122:0/1869642195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038c3ca0 msgr2=0x55c5038c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038c3ca0 0x55c5038c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 -- 172.21.15.122:0/1869642195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 msgr2=0x55c5038bab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038bab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.007+0000 7f7b362c7640 1 -- 172.21.15.122:0/1869642195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5035323f0 con 0x55c5038bb0c0 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b36ac8640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 0x7f7b180a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800baa0 tx=0x7f7b2800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5c98e77503fe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b2f7fe640 1 -- 172.21.15.122:0/2399811404 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b200f2ae0 con 0x7f7b30107b50 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b2cbfe640 1 -- 172.21.15.122:0/1869642195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28017420 con 0x55c5038bb0c0 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b2cbfe640 1 -- 172.21.15.122:0/1869642195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28017aa0 con 0x55c5038bb0c0 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.008+0000 7f7b2cbfe640 1 -- 172.21.15.122:0/1869642195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800fb20 con 0x55c5038bb0c0 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1869642195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 msgr2=0x55c5038c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b36ac8640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 0x7f7b180a6380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b200bac60 tx=0x7f7b200e9910 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800baa0 tx=0x7f7b2800bf60 comp rx=0 tx=0).stop 2024-04-02T13:40:43.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1869642195 shutdown_connections 2024-04-02T13:40:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038c3ca0 0x55c5038c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038bb0c0 0x55c5038c3760 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1869642195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038bab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1869642195 >> 172.21.15.122:0/1869642195 conn(0x55c5038b4ed0 msgr2=0x55c5038b52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1869642195 shutdown_connections 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1869642195 wait complete. 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- start start 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 Processor -- start 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038c3ca0 0x55c5038c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c5038c6a90 con 0x55c5038ba7a0 2024-04-02T13:40:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c5037d9e80 con 0x55c5038c3ca0 2024-04-02T13:40:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c5038c6c10 con 0x55c5038bb0c0 2024-04-02T13:40:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b3b1ef740 10 client.0 _ll_register_callbacks cb 0x55c503815aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.009+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55278/0 (socket says 172.21.15.122:55278) 2024-04-02T13:40:43.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 -- 172.21.15.122:0/1784012597 learned_addr learned my addr 172.21.15.122:0/1784012597 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038c3ca0 0x55c5038c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 -- 172.21.15.122:0/1784012597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 msgr2=0x55c5038c09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038c09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 -- 172.21.15.122:0/1784012597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038c3ca0 msgr2=0x55c5038c1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038c3ca0 0x55c5038c1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 -- 172.21.15.122:0/1784012597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5035fc7d0 con 0x55c5038ba7a0 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7b24013470 tx=0x7f7b24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf86949a5af56985 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2400f790 con 0x55c5038ba7a0 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b2400f930 con 0x55c5038ba7a0 2024-04-02T13:40:43.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b28017aa0 con 0x55c5038ba7a0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7b24025050 con 0x55c5038ba7a0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b3b1ef740 10 client.17301 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c5038d2b20 con 0x55c5038ba7a0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b28017aa0 con 0x55c5038ba7a0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.010+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7b2401d020 con 0x55c5038ba7a0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 10 client.17301 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 10 client.17301 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 20 client.17301 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c5038d3fe0 0x55c5038d6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 4 client.17301 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c50382bcf0 con 0x55c5038d3fe0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c5038d3fe0 0x55c5038d6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b362c7640 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c5038d3fe0 0x55c5038d6450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49d8061ae392e726 server_cookie=9e793989577e6c58 in_seq=0 out_seq=0 2024-04-02T13:40:43.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.011+0000 7f7b15ffb640 10 client.17301 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b15ffb640 1 -- 172.21.15.122:0/1784012597 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c50382bcf0 con 0x55c5038d3fe0 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b15ffb640 10 client.17301 handle_command_reply: tid=0 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 client.17301 shutdown 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 msgr2=0x55c5038c3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f7b24013470 tx=0x7f7b24004a40 comp rx=0 tx=0).stop 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 shutdown_connections 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c5038d3fe0 0x55c5038d6450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038c3ca0 0x55c5038c1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b15ffb640 0 client.17301 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038c09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/1784012597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038ba7a0 0x55c5038c3660 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 >> 172.21.15.122:0/1784012597 conn(0x55c5038b4ed0 msgr2=0x55c5038b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 shutdown_connections 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/1784012597 wait complete. 2024-04-02T13:40:43.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.012+0000 7f7b3b1ef740 20 client.17301 trim_cache size 0 max 16384 2024-04-02T13:40:43.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 Processor -- start 2024-04-02T13:40:43.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 -- start start 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301af000 con 0x55c5038c3ca0 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b0c00 con 0x55c5038ba7a0 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301af6d0 con 0x55c5038bb0c0 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55290/0 (socket says 172.21.15.122:55290) 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 -- 172.21.15.122:0/2863269541 learned_addr learned my addr 172.21.15.122:0/2863269541 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 -- 172.21.15.122:0/2863269541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 msgr2=0x55c5038cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.025+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b35ac6640 1 -- 172.21.15.122:0/2863269541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 msgr2=0x55c5038cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b35ac6640 1 -- 172.21.15.122:0/2863269541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5038b07a0 con 0x55c5038c3ca0 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7b2401ece0 tx=0x7f7b2400a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf21a1c015aaf18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b17fff640 1 -- 172.21.15.122:0/2863269541 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2401e480 con 0x55c5038c3ca0 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b17fff640 1 -- 172.21.15.122:0/2863269541 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2400f590 con 0x55c5038c3ca0 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b17fff640 1 -- 172.21.15.122:0/2863269541 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24029620 con 0x55c5038c3ca0 2024-04-02T13:40:43.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/2863269541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 msgr2=0x55c5038c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f7b2401ece0 tx=0x7f7b2400a670 comp rx=0 tx=0).stop 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/2863269541 shutdown_connections 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c3120 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038bb0c0 0x55c5038cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/2863269541 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038ba7a0 0x55c5038cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/2863269541 >> 172.21.15.122:0/2863269541 conn(0x55c5038b4ed0 msgr2=0x55c5038bfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/2863269541 shutdown_connections 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.026+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/2863269541 wait complete. 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 -- start start 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 Processor -- start 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038ba7a0 0x55c5038c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 0x55c5038c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301b0470 con 0x55c5038c3ca0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301afda0 con 0x55c5038bb0c0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b301acfe0 con 0x55c5038ba7a0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b3b1ef740 10 client.0 _ll_register_callbacks cb 0x7f7b24012420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55300/0 (socket says 172.21.15.122:55300) 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 -- 172.21.15.122:0/3847676386 learned_addr learned my addr 172.21.15.122:0/3847676386 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038ba7a0 0x55c5038c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b362c7640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 0x55c5038c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 -- 172.21.15.122:0/3847676386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038ba7a0 msgr2=0x55c5038c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038ba7a0 0x55c5038c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 -- 172.21.15.122:0/3847676386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 msgr2=0x55c5038c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 0x55c5038c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b35ac6640 1 -- 172.21.15.122:0/3847676386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5038c55e0 con 0x55c5038c3ca0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.027+0000 7f7b362c7640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 0x55c5038c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b35ac6640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7b24012d60 tx=0x7f7b2400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd82bcfd4e83bd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2401e480 con 0x55c5038c3ca0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b2401e620 con 0x55c5038c3ca0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b2400f590 con 0x55c5038c3ca0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7b2401e820 con 0x55c5038c3ca0 2024-04-02T13:40:43.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 10 client.17307 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c50393ef10 con 0x55c5038c3ca0 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b2400f590 con 0x55c5038c3ca0 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7b2400a020 con 0x55c5038c3ca0 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 10 client.17307 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 10 client.17307 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 20 client.17307 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c50393f580 0x55c5039419a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 4 client.17307 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c503826040 con 0x55c50393f580 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.028+0000 7f7b362c7640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c50393f580 0x55c5039419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b15ffb640 10 client.17307 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:43.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b362c7640 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c50393f580 0x55c5039419a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f2bfa5b3bd34cd2b server_cookie=c2f5d6f92814e772 in_seq=0 out_seq=0 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b15ffb640 1 -- 172.21.15.122:0/3847676386 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c503826040 con 0x55c50393f580 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b15ffb640 10 client.17307 handle_command_reply: tid=0 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 client.17307 shutdown 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 msgr2=0x55c5038c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f7b24012d60 tx=0x7f7b2400feb0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 shutdown_connections 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c50393f580 0x55c5039419a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c5038c3ca0 0x55c5038c4de0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c5038bb0c0 0x55c5038c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 --2- 172.21.15.122:0/3847676386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c5038ba7a0 0x55c5038c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b15ffb640 0 client.17307 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 >> 172.21.15.122:0/3847676386 conn(0x55c5038b4ed0 msgr2=0x55c5038b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 shutdown_connections 2024-04-02T13:40:43.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 1 -- 172.21.15.122:0/3847676386 wait complete. 2024-04-02T13:40:43.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3b1ef740 20 client.17307 trim_cache size 0 max 16384 2024-04-02T13:40:43.052 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:43.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 msgr2=0x7f7b180a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 0x7f7b180a6380 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7b200bac60 tx=0x7f7b200e9910 comp rx=0 tx=0).stop 2024-04-02T13:40:43.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 msgr2=0x7f7b3019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7b2002efd0 tx=0x7f7b200045e0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 shutdown_connections 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.029+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b180a3ec0 0x7f7b180a6380 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b30107b50 0x7f7b3019cc10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b300696c0 0x7f7b301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 --2- 172.21.15.122:0/2399811404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b30068d00 0x7f7b301983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 >> 172.21.15.122:0/2399811404 conn(0x7f7b30077b00 msgr2=0x7f7b300fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 shutdown_connections 2024-04-02T13:40:43.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.030+0000 7f7b3852b640 1 -- 172.21.15.122:0/2399811404 wait complete. 2024-04-02T13:40:43.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 -- 172.21.15.122:0/1184417238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0106a30 msgr2=0x7f7eb0106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 --2- 172.21.15.122:0/1184417238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0106a30 0x7f7eb0106e10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009900 tx=0x7f7ea002bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 -- 172.21.15.122:0/1184417238 shutdown_connections 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 --2- 172.21.15.122:0/1184417238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 0x7f7eb010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 --2- 172.21.15.122:0/1184417238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0100ce0 0x7f7eb0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 --2- 172.21.15.122:0/1184417238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0106a30 0x7f7eb0106e10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 -- 172.21.15.122:0/1184417238 >> 172.21.15.122:0/1184417238 conn(0x7f7eb00779e0 msgr2=0x7f7eb0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 -- 172.21.15.122:0/1184417238 shutdown_connections 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.155+0000 7f7eb599d640 1 -- 172.21.15.122:0/1184417238 wait complete. 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 Processor -- start 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 -- start start 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 0x7f7eb0199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0106a30 0x7f7eb01a4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0110d80 con 0x7f7eb0100ce0 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0110c00 con 0x7f7eb01016a0 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eb599d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb0110f00 con 0x7f7eb0106a30 2024-04-02T13:40:43.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55322/0 (socket says 172.21.15.122:55322) 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3050893108 learned_addr learned my addr 172.21.15.122:0/3050893108 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 0x7f7eb0199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.156+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0106a30 0x7f7eb01a4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3050893108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0106a30 msgr2=0x7f7eb01a4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0106a30 0x7f7eb01a4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3050893108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 msgr2=0x7f7eb0199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 0x7f7eb0199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3050893108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb01a4780 con 0x7f7eb0100ce0 2024-04-02T13:40:43.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eb499b640 1 -- 172.21.15.122:0/3050893108 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0041070 con 0x7f7eb0100ce0 2024-04-02T13:40:43.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb01a4950 con 0x7f7eb0100ce0 2024-04-02T13:40:43.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eb499b640 1 -- 172.21.15.122:0/3050893108 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea0004450 con 0x7f7eb0100ce0 2024-04-02T13:40:43.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eb499b640 1 -- 172.21.15.122:0/3050893108 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0003e00 con 0x7f7eb0100ce0 2024-04-02T13:40:43.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.157+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb01a4db0 con 0x7f7eb0100ce0 2024-04-02T13:40:43.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.159+0000 7f7eb499b640 1 -- 172.21.15.122:0/3050893108 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7ea0044070 con 0x7f7eb0100ce0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.159+0000 7f7eb8586740 1 Processor -- start 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 -- start start 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d05a007b0 con 0x556d05a9c7a0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d0599f570 con 0x556d05aa5ca0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb8586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d059fc760 con 0x556d05a9d0c0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb499b640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 0x7f7e980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55326/0 (socket says 172.21.15.122:55326) 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/96067107 learned_addr learned my addr 172.21.15.122:0/96067107 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 0x7f7e980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eb499b640 1 -- 172.21.15.122:0/3050893108 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7ea00f2660 con 0x7f7eb0100ce0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/96067107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 msgr2=0x556d05aa5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/96067107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 msgr2=0x556d05aa8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/96067107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d057143f0 con 0x556d05a9c7a0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8012a10 tx=0x7f7ea8012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f51018ec163802 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.160+0000 7f7e95ffb640 1 -- 172.21.15.122:0/96067107 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8022030 con 0x556d05a9c7a0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7e95ffb640 1 -- 172.21.15.122:0/96067107 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea8010c50 con 0x556d05a9c7a0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7e95ffb640 1 -- 172.21.15.122:0/96067107 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8011440 con 0x556d05a9c7a0 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- 172.21.15.122:0/96067107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 msgr2=0x556d05a9cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 0x7f7e980a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8022000 tx=0x7f7ea801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8012a10 tx=0x7f7ea8012ed0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- 172.21.15.122:0/96067107 shutdown_connections 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 --2- 172.21.15.122:0/96067107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05a9cb80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- 172.21.15.122:0/96067107 >> 172.21.15.122:0/96067107 conn(0x556d05a96ed0 msgr2=0x556d05a972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- 172.21.15.122:0/96067107 shutdown_connections 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- 172.21.15.122:0/96067107 wait complete. 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.161+0000 7f7eb8586740 1 -- start start 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 Processor -- start 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9c7a0 0x556d05aa56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d05aa8a90 con 0x556d05aa5ca0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d059bbe80 con 0x556d05a9c7a0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d05aa8c10 con 0x556d05a9d0c0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eb8586740 10 client.0 _ll_register_callbacks cb 0x556d059f7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55330/0 (socket says 172.21.15.122:55330) 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/1104258747 learned_addr learned my addr 172.21.15.122:0/1104258747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9c7a0 0x556d05aa56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 msgr2=0x556d05aa2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/1104258747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9c7a0 msgr2=0x556d05aa56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9c7a0 0x556d05aa56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eaf7fe640 1 -- 172.21.15.122:0/1104258747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d057de7d0 con 0x556d05aa5ca0 2024-04-02T13:40:43.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.162+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80120e0 tx=0x7f7ea8009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2ab15027181e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8011440 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7ea8004c00 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea8010c50 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7ea8010e00 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 10 client.17319 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d05ab4ae0 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea8010c50 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7ea8005170 con 0x556d05aa5ca0 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 10 client.17319 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 10 client.17319 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 20 client.17319 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05ab5e30 0x556d05ab8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 4 client.17319 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d05a0dcf0 con 0x556d05ab5e30 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.163+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05ab5e30 0x556d05ab8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.164+0000 7f7e8effd640 10 client.17319 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.164+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05ab5e30 0x556d05ab8220 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74be42bf1b28c834 server_cookie=92138083b468d1e4 in_seq=0 out_seq=0 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7e8effd640 1 -- 172.21.15.122:0/1104258747 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d05a0dcf0 con 0x556d05ab5e30 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7e8effd640 10 client.17319 handle_command_reply: tid=0 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 client.17319 shutdown 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 msgr2=0x556d05aa3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80120e0 tx=0x7f7ea8009f00 comp rx=0 tx=0).stop 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 shutdown_connections 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05ab5e30 0x556d05ab8220 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05aa5ca0 0x556d05aa3040 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7e8effd640 0 client.17319 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1104258747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9c7a0 0x556d05aa56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 >> 172.21.15.122:0/1104258747 conn(0x556d05a96ed0 msgr2=0x556d05a98c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 shutdown_connections 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 1 -- 172.21.15.122:0/1104258747 wait complete. 2024-04-02T13:40:43.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.165+0000 7f7eb8586740 20 client.17319 trim_cache size 0 max 16384 2024-04-02T13:40:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 Processor -- start 2024-04-02T13:40:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 -- start start 2024-04-02T13:40:43.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9d0c0 0x556d05aa5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05aa5ca0 0x556d05aa56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01aef30 con 0x556d05a9c7a0 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01b0b30 con 0x556d05a9d0c0 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.178+0000 7f7eb8586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01af600 con 0x556d05aa5ca0 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55332/0 (socket says 172.21.15.122:55332) 2024-04-02T13:40:43.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 -- 172.21.15.122:0/1578710099 learned_addr learned my addr 172.21.15.122:0/1578710099 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9d0c0 0x556d05aa5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 -- 172.21.15.122:0/1578710099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05aa5ca0 msgr2=0x556d05aa56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05aa5ca0 0x556d05aa56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 -- 172.21.15.122:0/1578710099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9d0c0 msgr2=0x556d05aa5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9d0c0 0x556d05aa5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 -- 172.21.15.122:0/1578710099 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d05a9cef0 con 0x556d05a9c7a0 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ba7e0 tx=0x7f7ea004d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7743d3a2313143 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7e957fa640 1 -- 172.21.15.122:0/1578710099 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00f1040 con 0x556d05a9c7a0 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7e957fa640 1 -- 172.21.15.122:0/1578710099 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea00f2040 con 0x556d05a9c7a0 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.179+0000 7f7e957fa640 1 -- 172.21.15.122:0/1578710099 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea004b040 con 0x556d05a9c7a0 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- 172.21.15.122:0/1578710099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 msgr2=0x556d05ab19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00ba7e0 tx=0x7f7ea004d000 comp rx=0 tx=0).stop 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- 172.21.15.122:0/1578710099 shutdown_connections 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05aa5ca0 0x556d05aa56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05a9d0c0 0x556d05aa5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 --2- 172.21.15.122:0/1578710099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05ab19b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- 172.21.15.122:0/1578710099 >> 172.21.15.122:0/1578710099 conn(0x556d05a96ed0 msgr2=0x556d05aa1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- 172.21.15.122:0/1578710099 shutdown_connections 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- 172.21.15.122:0/1578710099 wait complete. 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 -- start start 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.180+0000 7f7eb8586740 1 Processor -- start 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01b03a0 con 0x556d05a9c7a0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01afcd0 con 0x556d05aa5ca0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb01acf10 con 0x556d05a9d0c0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 10 client.0 _ll_register_callbacks cb 0x7f7ea8012810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55334/0 (socket says 172.21.15.122:55334) 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3593530366 learned_addr learned my addr 172.21.15.122:0/3593530366 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 msgr2=0x556d05aa7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3593530366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 msgr2=0x556d05aa7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 -- 172.21.15.122:0/3593530366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d05aac270 con 0x556d05a9c7a0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaf7fe640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eaeffd640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00f0020 tx=0x7f7ea00c6ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5866583aecbd1354 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7e8effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00f1070 con 0x556d05a9c7a0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7ea00035f0 con 0x556d05a9c7a0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea00f2040 con 0x556d05a9c7a0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7ea004b040 con 0x556d05a9c7a0 2024-04-02T13:40:43.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 10 client.17325 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556d05b1c720 con 0x556d05a9c7a0 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.181+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea00f2040 con 0x556d05a9c7a0 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7ea00dd0d0 con 0x556d05a9c7a0 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 10 client.17325 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 10 client.17325 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 20 client.17325 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05b1cbf0 0x556d05b1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 4 client.17325 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:43.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d05a08040 con 0x556d05b1cbf0 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05b1cbf0 0x556d05b1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7e8effd640 10 client.17325 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eae7fc640 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05b1cbf0 0x556d05b1efe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=162bca08510b185a server_cookie=9bc3e3f3c4482bda in_seq=0 out_seq=0 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7e8effd640 1 -- 172.21.15.122:0/3593530366 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d05a08040 con 0x556d05b1cbf0 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7e8effd640 10 client.17325 handle_command_reply: tid=0 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.182+0000 7f7eb8586740 1 client.17325 shutdown 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 msgr2=0x556d05aa6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00f0020 tx=0x7f7ea00c6ff0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 shutdown_connections 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d05aa5ca0 0x556d05aa7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d05a9d0c0 0x556d05aa7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556d05b1cbf0 0x556d05b1efe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 --2- 172.21.15.122:0/3593530366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d05a9c7a0 0x556d05aa6be0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7e8effd640 0 client.17325 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 >> 172.21.15.122:0/3593530366 conn(0x556d05a96ed0 msgr2=0x556d05a956a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 shutdown_connections 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 1 -- 172.21.15.122:0/3593530366 wait complete. 2024-04-02T13:40:43.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb8586740 20 client.17325 trim_cache size 0 max 16384 2024-04-02T13:40:43.200 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:43.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 msgr2=0x7f7e980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 0x7f7e980a61e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8022000 tx=0x7f7ea801e000 comp rx=0 tx=0).stop 2024-04-02T13:40:43.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 msgr2=0x7f7eb01991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0006010 tx=0x7f7ea002cb30 comp rx=0 tx=0).stop 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 shutdown_connections 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7e980a3d20 0x7f7e980a61e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7eb0106a30 0x7f7eb01a4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7eb01016a0 0x7f7eb0199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 --2- 172.21.15.122:0/3050893108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7eb0100ce0 0x7f7eb01991c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.183+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 >> 172.21.15.122:0/3050893108 conn(0x7f7eb00779e0 msgr2=0x7f7eb00ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.184+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 shutdown_connections 2024-04-02T13:40:43.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.184+0000 7f7eb599d640 1 -- 172.21.15.122:0/3050893108 wait complete. 2024-04-02T13:40:43.203 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:40:43.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 -- 172.21.15.122:0/2308244861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 msgr2=0x7fa8c4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/2308244861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4106d60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4009930 tx=0x7fa8b402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 -- 172.21.15.122:0/2308244861 shutdown_connections 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/2308244861 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/2308244861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 0x7fa8c4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/2308244861 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4106d60 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 -- 172.21.15.122:0/2308244861 >> 172.21.15.122:0/2308244861 conn(0x7fa8c4077b00 msgr2=0x7fa8c4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 -- 172.21.15.122:0/2308244861 shutdown_connections 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.304+0000 7fa8c90b1640 1 -- 172.21.15.122:0/2308244861 wait complete. 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 Processor -- start 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 -- start start 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 0x7fa8c4102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c4103500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4110d90 con 0x7fa8c4106980 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4110c10 con 0x7fa8c4101330 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c90b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c4110f10 con 0x7fa8c4100970 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55354/0 (socket says 172.21.15.122:55354) 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.305+0000 7fa8c359e640 1 -- 172.21.15.122:0/1838443811 learned_addr learned my addr 172.21.15.122:0/1838443811 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 0x7fa8c4102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c4103500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 -- 172.21.15.122:0/1838443811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 msgr2=0x7fa8c4102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 0x7fa8c4102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 -- 172.21.15.122:0/1838443811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 msgr2=0x7fa8c4103500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c4103500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 -- 172.21.15.122:0/1838443811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41a68e0 con 0x7fa8c4106980 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c359e640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c0d99640 1 -- 172.21.15.122:0/1838443811 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00cdb0 con 0x7fa8c4106980 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c0d99640 1 -- 172.21.15.122:0/1838443811 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8bc005020 con 0x7fa8c4106980 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c0d99640 1 -- 172.21.15.122:0/1838443811 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0116b0 con 0x7fa8c4106980 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41a6bd0 con 0x7fa8c4106980 2024-04-02T13:40:43.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.306+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c41a7120 con 0x7fa8c4106980 2024-04-02T13:40:43.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.308+0000 7fa8c0d99640 1 -- 172.21.15.122:0/1838443811 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa8bc005560 con 0x7fa8c4106980 2024-04-02T13:40:43.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.308+0000 7fa8cbd73740 1 Processor -- start 2024-04-02T13:40:43.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.308+0000 7fa8cbd73740 1 -- start start 2024-04-02T13:40:43.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b49ca0 0x5591c8b4c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5591c8aa47b0 con 0x5591c8b407a0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5591c8a43570 con 0x5591c8b410c0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5591c8aa0760 con 0x5591c8b49ca0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55356/0 (socket says 172.21.15.122:55356) 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2858309604 learned_addr learned my addr 172.21.15.122:0/2858309604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8c0d99640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 0x7fa8b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8c359e640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 0x7fa8b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2858309604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b49ca0 msgr2=0x5591c8b4c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b49ca0 0x5591c8b4c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2858309604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 msgr2=0x5591c8b49760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2858309604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c87b83f0 con 0x5591c8b407a0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.309+0000 7fa8c0d99640 1 -- 172.21.15.122:0/1838443811 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa8bc0d0d70 con 0x7fa8c4106980 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00aac0 tx=0x7fa8ac00af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988ac1e3195661a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8b97fa640 1 -- 172.21.15.122:0/2858309604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac00f040 con 0x5591c8b407a0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8c359e640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 0x7fa8b00a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc098de0 tx=0x7fa8bc097040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8b97fa640 1 -- 172.21.15.122:0/2858309604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac018070 con 0x5591c8b407a0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8b97fa640 1 -- 172.21.15.122:0/2858309604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0123c0 con 0x5591c8b407a0 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2858309604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 msgr2=0x5591c8b40b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.310+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00aac0 tx=0x7fa8ac00af80 comp rx=0 tx=0).stop 2024-04-02T13:40:43.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2858309604 shutdown_connections 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b49ca0 0x5591c8b4c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2858309604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b407a0 0x5591c8b40b80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2858309604 >> 172.21.15.122:0/2858309604 conn(0x5591c8b3aed0 msgr2=0x5591c8b3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2858309604 shutdown_connections 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2858309604 wait complete. 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- start start 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 Processor -- start 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b49ca0 0x5591c8b47050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5591c8b4ca30 con 0x5591c8b410c0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5591c8a5fe80 con 0x5591c8b49ca0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5591c8b4cbb0 con 0x5591c8b407a0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8cbd73740 10 client.0 _ll_register_callbacks cb 0x5591c8a9baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8c2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8c2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55370/0 (socket says 172.21.15.122:55370) 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8c2d9d640 1 -- 172.21.15.122:0/2092380467 learned_addr learned my addr 172.21.15.122:0/2092380467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b49ca0 0x5591c8b47050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8c2d9d640 1 -- 172.21.15.122:0/2092380467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 msgr2=0x5591c8b496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.311+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8c2d9d640 1 -- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b49ca0 msgr2=0x5591c8b47050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b49ca0 0x5591c8b47050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8c2d9d640 1 -- 172.21.15.122:0/2092380467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c88827d0 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa8b402cfd0 tx=0x7fa8b402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0c14b66a1a1dec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4004340 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa8b40046c0 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ac018070 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8b40044e0 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8cbd73740 10 client.17337 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5591c8b58a60 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac018070 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fa8b40035f0 con 0x5591c8b410c0 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8cbd73740 10 client.17337 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.312+0000 7fa8cbd73740 10 client.17337 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8cbd73740 20 client.17337 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8b59f70 0x5591c8b5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8cbd73740 4 client.17337 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591c8ab1cf0 con 0x5591c8b59f70 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8b59f70 0x5591c8b5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8aa7fc640 10 client.17337 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.313+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8b59f70 0x5591c8b5c3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b5c04677edba0ba server_cookie=752d3e5d85ff9ff7 in_seq=0 out_seq=0 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2092380467 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591c8ab1cf0 con 0x5591c8b59f70 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8aa7fc640 10 client.17337 handle_command_reply: tid=0 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 client.17337 shutdown 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 msgr2=0x5591c8b46a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa8b402cfd0 tx=0x7fa8b402c7e0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 shutdown_connections 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8b59f70 0x5591c8b5c3d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b49ca0 0x5591c8b47050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b410c0 0x5591c8b46a40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2092380467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8aa7fc640 0 client.17337 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 >> 172.21.15.122:0/2092380467 conn(0x5591c8b3aed0 msgr2=0x5591c8b3cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 shutdown_connections 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2092380467 wait complete. 2024-04-02T13:40:43.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.314+0000 7fa8cbd73740 20 client.17337 trim_cache size 0 max 16384 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.327+0000 7fa8cbd73740 1 Processor -- start 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.327+0000 7fa8cbd73740 1 -- start start 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b558f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b11b0 con 0x5591c8b49ca0 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b2db0 con 0x5591c8b410c0 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b1880 con 0x5591c8b407a0 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55380/0 (socket says 172.21.15.122:55380) 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 -- 172.21.15.122:0/3506608512 learned_addr learned my addr 172.21.15.122:0/3506608512 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 -- 172.21.15.122:0/3506608512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 msgr2=0x5591c8b558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 -- 172.21.15.122:0/3506608512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 msgr2=0x5591c8b49170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 -- 172.21.15.122:0/3506608512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591c8b40ef0 con 0x5591c8b49ca0 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac007fd0 tx=0x7fa8ac013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3996c6b4d0def6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8b8ff9640 1 -- 172.21.15.122:0/3506608512 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac02a070 con 0x5591c8b49ca0 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8b8ff9640 1 -- 172.21.15.122:0/3506608512 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac018030 con 0x5591c8b49ca0 2024-04-02T13:40:43.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.328+0000 7fa8b8ff9640 1 -- 172.21.15.122:0/3506608512 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac025520 con 0x5591c8b49ca0 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- 172.21.15.122:0/3506608512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 msgr2=0x5591c8b496b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac007fd0 tx=0x7fa8ac013b80 comp rx=0 tx=0).stop 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- 172.21.15.122:0/3506608512 shutdown_connections 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b496b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b410c0 0x5591c8b49170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/3506608512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b407a0 0x5591c8b558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- 172.21.15.122:0/3506608512 >> 172.21.15.122:0/3506608512 conn(0x5591c8b3aed0 msgr2=0x5591c8b3b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- 172.21.15.122:0/3506608512 shutdown_connections 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- 172.21.15.122:0/3506608512 wait complete. 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 -- start start 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.329+0000 7fa8cbd73740 1 Processor -- start 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b407a0 0x5591c8b51e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 0x5591c8b52340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b2620 con 0x5591c8b49ca0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41b1f50 con 0x5591c8b407a0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c41af190 con 0x5591c8b410c0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 10 client.0 _ll_register_callbacks cb 0x7fa8b4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55388/0 (socket says 172.21.15.122:55388) 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2169807596 learned_addr learned my addr 172.21.15.122:0/2169807596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 0x5591c8b52340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2169807596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 msgr2=0x5591c8b52340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 0x5591c8b52340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b407a0 msgr2=0x5591c8b51e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b407a0 0x5591c8b51e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 -- 172.21.15.122:0/2169807596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591c8b44490 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 0x5591c8b52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8bbfff640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac001eb0 tx=0x7fa8ac025f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d806e429f95f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8aa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac02a040 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fa8ac013760 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ac018030 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa8ac00a370 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 10 client.17343 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5591c8b4ab70 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.330+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ac018030 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fa8ac025610 con 0x5591c8b49ca0 2024-04-02T13:40:43.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 10 client.17343 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 10 client.17343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 20 client.17343 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8bc0510 0x5591c8bc2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 4 client.17343 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591c8aac040 con 0x5591c8bc0510 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8bc0510 0x5591c8bc2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8c2d9d640 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8bc0510 0x5591c8bc2900 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=377c513b2781a9ea server_cookie=286c73122e3c25f7 in_seq=0 out_seq=0 2024-04-02T13:40:43.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.331+0000 7fa8aa7fc640 10 client.17343 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8aa7fc640 1 -- 172.21.15.122:0/2169807596 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591c8aac040 con 0x5591c8bc0510 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8aa7fc640 10 client.17343 handle_command_reply: tid=0 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 client.17343 shutdown 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 msgr2=0x5591c8b43cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac001eb0 tx=0x7fa8ac025f80 comp rx=0 tx=0).stop 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 shutdown_connections 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5591c8b49ca0 0x5591c8b43cc0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5591c8b410c0 0x5591c8b52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5591c8bc0510 0x5591c8bc2900 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 --2- 172.21.15.122:0/2169807596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5591c8b407a0 0x5591c8b51e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 >> 172.21.15.122:0/2169807596 conn(0x5591c8b3aed0 msgr2=0x5591c8b45220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8aa7fc640 0 client.17343 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:40:43.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 shutdown_connections 2024-04-02T13:40:43.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 1 -- 172.21.15.122:0/2169807596 wait complete. 2024-04-02T13:40:43.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8cbd73740 20 client.17343 trim_cache size 0 max 16384 2024-04-02T13:40:43.346 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 msgr2=0x7fa8b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 0x7fa8b00a6260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc098de0 tx=0x7fa8bc097040 comp rx=0 tx=0).stop 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 msgr2=0x7fa8c4103a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00eaa0 tx=0x7fa8bc00ef60 comp rx=0 tx=0).stop 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 shutdown_connections 2024-04-02T13:40:43.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8b00a3da0 0x7fa8b00a6260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8c4106980 0x7fa8c4103a40 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8c4101330 0x7fa8c4103500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 --2- 172.21.15.122:0/1838443811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8c4100970 0x7fa8c4102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.332+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 >> 172.21.15.122:0/1838443811 conn(0x7fa8c4077b00 msgr2=0x7fa8c4068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.333+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 shutdown_connections 2024-04-02T13:40:43.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.333+0000 7fa8c90b1640 1 -- 172.21.15.122:0/1838443811 wait complete. 2024-04-02T13:40:43.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:40:43.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 -- 172.21.15.122:0/2102838439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d41058a0 msgr2=0x7f58d4105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 --2- 172.21.15.122:0/2102838439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d41058a0 0x7f58d4105c80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009900 tx=0x7f58d002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 -- 172.21.15.122:0/2102838439 shutdown_connections 2024-04-02T13:40:43.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 --2- 172.21.15.122:0/2102838439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d4101360 0x7f58d410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 --2- 172.21.15.122:0/2102838439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 --2- 172.21.15.122:0/2102838439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d41058a0 0x7f58d4105c80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 -- 172.21.15.122:0/2102838439 >> 172.21.15.122:0/2102838439 conn(0x7f58d4077a90 msgr2=0x7f58d4077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 -- 172.21.15.122:0/2102838439 shutdown_connections 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.459+0000 7f58dc21b640 1 -- 172.21.15.122:0/2102838439 wait complete. 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 Processor -- start 2024-04-02T13:40:43.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 -- start start 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 0x7f58d41988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d41058a0 0x7f58d419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4110d60 con 0x7f58d4101360 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4110be0 con 0x7f58d41009c0 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58dc21b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4110ee0 con 0x7f58d41058a0 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 0x7f58d41988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 0x7f58d41988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55408/0 (socket says 172.21.15.122:55408) 2024-04-02T13:40:43.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58d97b6640 1 -- 172.21.15.122:0/4204514767 learned_addr learned my addr 172.21.15.122:0/4204514767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.460+0000 7f58da7b8640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d41058a0 0x7f58d419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 -- 172.21.15.122:0/4204514767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d41058a0 msgr2=0x7f58d419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d41058a0 0x7f58d419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 -- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 msgr2=0x7f58d41988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 0x7f58d41988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 -- 172.21.15.122:0/4204514767 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d419d390 con 0x7f58d41009c0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d41983a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000c00 tx=0x7f58d002cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58caffd640 1 -- 172.21.15.122:0/4204514767 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0041070 con 0x7f58d41009c0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d419d620 con 0x7f58d41009c0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58caffd640 1 -- 172.21.15.122:0/4204514767 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d0004450 con 0x7f58d41009c0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58caffd640 1 -- 172.21.15.122:0/4204514767 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d0003e50 con 0x7f58d41009c0 2024-04-02T13:40:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.461+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d41a4f10 con 0x7f58d41009c0 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.464+0000 7f58caffd640 1 -- 172.21.15.122:0/4204514767 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f58d0044070 con 0x7f58d41009c0 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.464+0000 7f58dee02740 1 Processor -- start 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 -- start start 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58caffd640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 0x7f58c00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbae8057b0 con 0x55cbae8a17a0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbae7a4570 con 0x55cbae8a20c0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58dee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbae801760 con 0x55cbae8aaca0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58caffd640 1 -- 172.21.15.122:0/4204514767 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f58d00f2410 con 0x7f58d41009c0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58da7b8640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 0x7f58c00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55420/0 (socket says 172.21.15.122:55420) 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 -- 172.21.15.122:0/1820648515 learned_addr learned my addr 172.21.15.122:0/1820648515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58da7b8640 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d97b6640 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 -- 172.21.15.122:0/1820648515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 msgr2=0x55cbae8ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 -- 172.21.15.122:0/1820648515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 msgr2=0x55cbae8aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.465+0000 7f58d9fb7640 1 -- 172.21.15.122:0/1820648515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbae5193f0 con 0x55cbae8a17a0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58da7b8640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 0x7f58c00a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc009ad0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f58d00c6870 tx=0x7f58d00b9e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d7b28e72b9f4ad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58bffff640 1 -- 172.21.15.122:0/1820648515 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d004b070 con 0x55cbae8a17a0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58bffff640 1 -- 172.21.15.122:0/1820648515 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d00be1e0 con 0x55cbae8a17a0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58bffff640 1 -- 172.21.15.122:0/1820648515 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00f2160 con 0x55cbae8a17a0 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58dee02740 1 -- 172.21.15.122:0/1820648515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 msgr2=0x55cbae8a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.466+0000 7f58dee02740 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f58d00c6870 tx=0x7f58d00b9e80 comp rx=0 tx=0).stop 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- 172.21.15.122:0/1820648515 shutdown_connections 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- 172.21.15.122:0/1820648515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a17a0 0x55cbae8a1b80 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- 172.21.15.122:0/1820648515 >> 172.21.15.122:0/1820648515 conn(0x55cbae89bed0 msgr2=0x55cbae89c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- 172.21.15.122:0/1820648515 shutdown_connections 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- 172.21.15.122:0/1820648515 wait complete. 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- start start 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 Processor -- start 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a17a0 0x55cbae8aa610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8a8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbae8ad850 con 0x55cbae8a20c0 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbae7c0e80 con 0x55cbae8a17a0 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.467+0000 7f58dee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbae8ad9d0 con 0x55cbae8aaca0 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58dee02740 10 client.0 _ll_register_callbacks cb 0x55cbae7fcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55426/0 (socket says 172.21.15.122:55426) 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58da7b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8a8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 -- 172.21.15.122:0/1223172035 learned_addr learned my addr 172.21.15.122:0/1223172035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a17a0 0x55cbae8aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 -- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 msgr2=0x55cbae8a8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8a8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 -- 172.21.15.122:0/1223172035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a17a0 msgr2=0x55cbae8aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a17a0 0x55cbae8aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 -- 172.21.15.122:0/1223172035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbae5e37d0 con 0x55cbae8a20c0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58da7b8640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58d97b6640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f58c4012840 tx=0x7f58c4012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca544df3a045415 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c400f920 con 0x55cbae8a20c0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f58c400fac0 con 0x55cbae8a20c0 2024-04-02T13:40:43.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.468+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d00be1e0 con 0x55cbae8a20c0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f58c400fcc0 con 0x55cbae8a20c0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 10 client.17355 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbae8b9800 con 0x55cbae8a20c0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58d00be1e0 con 0x55cbae8a20c0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f58c4003b40 con 0x55cbae8a20c0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 10 client.17355 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 10 client.17355 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 20 client.17355 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae8bad60 0x55cbae8bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 4 client.17355 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbae812cf0 con 0x55cbae8bad60 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.469+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae8bad60 0x55cbae8bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.470+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae8bad60 0x55cbae8bd1a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f99a07fa789f5013 server_cookie=2e7a67a6926cdc9a in_seq=0 out_seq=0 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.470+0000 7f58bd7fa640 10 client.17355 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:43.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.470+0000 7f58bd7fa640 1 -- 172.21.15.122:0/1223172035 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbae812cf0 con 0x55cbae8bad60 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.470+0000 7f58bd7fa640 10 client.17355 handle_command_reply: tid=0 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.470+0000 7f58dee02740 1 client.17355 shutdown 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 msgr2=0x55cbae8a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f58c4012840 tx=0x7f58c4012c40 comp rx=0 tx=0).stop 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 shutdown_connections 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae8bad60 0x55cbae8bd1a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8aaca0 0x55cbae8a8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8a20c0 0x55cbae8a7a10 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58bd7fa640 0 client.17355 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 --2- 172.21.15.122:0/1223172035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a17a0 0x55cbae8aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 >> 172.21.15.122:0/1223172035 conn(0x55cbae89bed0 msgr2=0x55cbae89dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 shutdown_connections 2024-04-02T13:40:43.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 1 -- 172.21.15.122:0/1223172035 wait complete. 2024-04-02T13:40:43.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.471+0000 7f58dee02740 20 client.17355 trim_cache size 0 max 16384 2024-04-02T13:40:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 Processor -- start 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 -- start start 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41acdc0 con 0x55cbae8aaca0 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ae9c0 con 0x55cbae8a20c0 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58dee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ad490 con 0x55cbae8a17a0 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58da7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58da7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55442/0 (socket says 172.21.15.122:55442) 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:42218/0 (socket says 172.21.15.122:42218) 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58da7b8640 1 -- 172.21.15.122:0/632371480 learned_addr learned my addr 172.21.15.122:0/632371480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 -- 172.21.15.122:0/632371480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 msgr2=0x55cbae8b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 -- 172.21.15.122:0/632371480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 msgr2=0x55cbae8c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58d97b6640 1 -- 172.21.15.122:0/632371480 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbae8a1ef0 con 0x55cbae8a20c0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.485+0000 7f58da7b8640 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58d97b6640 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c4009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afc5f6041c1feaa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58bf7fe640 1 -- 172.21.15.122:0/632371480 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c402b070 con 0x55cbae8a20c0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58bf7fe640 1 -- 172.21.15.122:0/632371480 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c40055c0 con 0x55cbae8a20c0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58bf7fe640 1 -- 172.21.15.122:0/632371480 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4003b30 con 0x55cbae8a20c0 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 -- 172.21.15.122:0/632371480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 msgr2=0x55cbae8a71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f58c4000c00 tx=0x7f58c4009f40 comp rx=0 tx=0).stop 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 -- 172.21.15.122:0/632371480 shutdown_connections 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8a71c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 --2- 172.21.15.122:0/632371480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 -- 172.21.15.122:0/632371480 >> 172.21.15.122:0/632371480 conn(0x55cbae89bed0 msgr2=0x55cbae8a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 -- 172.21.15.122:0/632371480 shutdown_connections 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.486+0000 7f58dee02740 1 -- 172.21.15.122:0/632371480 wait complete. 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 -- start start 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 Processor -- start 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8abb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41ae230 con 0x55cbae8aaca0 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41adb60 con 0x55cbae8a20c0 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41aada0 con 0x55cbae8a17a0 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58dee02740 10 client.0 _ll_register_callbacks cb 0x7f58c4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58da7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58d9fb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58d9fb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:55956/0 (socket says 172.21.15.122:55956) 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58da7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55456/0 (socket says 172.21.15.122:55456) 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58d97b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:42224/0 (socket says 172.21.15.122:42224) 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.487+0000 7f58d9fb7640 1 -- 172.21.15.122:0/2392991280 learned_addr learned my addr 172.21.15.122:0/2392991280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 -- 172.21.15.122:0/2392991280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 msgr2=0x55cbae8ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 -- 172.21.15.122:0/2392991280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 msgr2=0x55cbae8abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 -- 172.21.15.122:0/2392991280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbae8ac1a0 con 0x55cbae8a17a0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58d9fb7640 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58d00bdee0 tx=0x7f58d002cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1cd4f92b5145f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d004b040 con 0x55cbae8a17a0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c40055c0 con 0x55cbae8a17a0 2024-04-02T13:40:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f58d00f2160 con 0x55cbae8a17a0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f58d00dd0d0 con 0x55cbae8a17a0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58dee02740 10 client.26428 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cbae9212e0 con 0x55cbae8a17a0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.488+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58c40055c0 con 0x55cbae8a17a0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f58d00de040 con 0x55cbae8a17a0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 10 client.26428 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 10 client.26428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 20 client.26428 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae9217b0 0x55cbae923bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 4 client.26428 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbae80d040 con 0x55cbae9217b0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.489+0000 7f58d97b6640 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae9217b0 0x55cbae923bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58bd7fa640 10 client.26428 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58d97b6640 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae9217b0 0x55cbae923bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0d7bdc4047ae93e server_cookie=af416677a0598600 in_seq=0 out_seq=0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58bd7fa640 1 -- 172.21.15.122:0/2392991280 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbae80d040 con 0x55cbae9217b0 2024-04-02T13:40:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58bd7fa640 10 client.26428 handle_command_reply: tid=0 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 client.26428 shutdown 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 msgr2=0x55cbae8b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58d00bdee0 tx=0x7f58d002cf60 comp rx=0 tx=0).stop 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 shutdown_connections 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55cbae9217b0 0x55cbae923bd0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbae8aaca0 0x55cbae8abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58bd7fa640 0 client.26428 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:40:43.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbae8a20c0 0x55cbae8ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 --2- 172.21.15.122:0/2392991280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbae8a17a0 0x55cbae8b2d70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 >> 172.21.15.122:0/2392991280 conn(0x55cbae89bed0 msgr2=0x55cbae8a90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 shutdown_connections 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 1 -- 172.21.15.122:0/2392991280 wait complete. 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.490+0000 7f58dee02740 20 client.26428 trim_cache size 0 max 16384 2024-04-02T13:40:43.502 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:43.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 msgr2=0x7f58c00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 0x7f58c00a6380 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f58cc005fd0 tx=0x7f58cc009ad0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 msgr2=0x7f58d41983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d41983a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000c00 tx=0x7f58d002cc90 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 shutdown_connections 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58c00a3ec0 0x7f58c00a6380 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d41058a0 0x7f58d419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58d4101360 0x7f58d41988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 --2- 172.21.15.122:0/4204514767 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f58d41009c0 0x7f58d41983a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 >> 172.21.15.122:0/4204514767 conn(0x7f58d4077a90 msgr2=0x7f58d406a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 shutdown_connections 2024-04-02T13:40:43.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.491+0000 7f58dc21b640 1 -- 172.21.15.122:0/4204514767 wait complete. 2024-04-02T13:40:43.505 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:43 smithi122 ceph-mon[32114]: pgmap v169: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 841 KiB/s wr, 88 op/s 2024-04-02T13:40:43.505 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:43 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/769743423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:43.507 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:40:43.516 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:43 smithi188 ceph-mon[36117]: pgmap v169: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 841 KiB/s wr, 88 op/s 2024-04-02T13:40:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:43 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/769743423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:43.536 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:43 smithi187 ceph-mon[36064]: pgmap v169: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 841 KiB/s wr, 88 op/s 2024-04-02T13:40:43.536 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:43 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/769743423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 -- 172.21.15.122:0/2429289403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570101330 msgr2=0x7f257010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 --2- 172.21.15.122:0/2429289403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570101330 0x7f257010d840 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2568009cd0 tx=0x7f256802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 -- 172.21.15.122:0/2429289403 shutdown_connections 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 --2- 172.21.15.122:0/2429289403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570101330 0x7f257010d840 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 --2- 172.21.15.122:0/2429289403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570100970 0x7f2570100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 --2- 172.21.15.122:0/2429289403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570106980 0x7f2570106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 -- 172.21.15.122:0/2429289403 >> 172.21.15.122:0/2429289403 conn(0x7f2570077b00 msgr2=0x7f2570077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 -- 172.21.15.122:0/2429289403 shutdown_connections 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.618+0000 7f2575463640 1 -- 172.21.15.122:0/2429289403 wait complete. 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 Processor -- start 2024-04-02T13:40:43.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 -- start start 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570101330 0x7f2570197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570106980 0x7f257019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2570110d90 con 0x7f2570100970 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2570110c10 con 0x7f2570101330 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f2575463640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2570110f10 con 0x7f2570106980 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f256effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f256effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55472/0 (socket says 172.21.15.122:55472) 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f256effd640 1 -- 172.21.15.122:0/1604463148 learned_addr learned my addr 172.21.15.122:0/1604463148 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f256e7fc640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570101330 0x7f2570197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.619+0000 7f256f7fe640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570106980 0x7f257019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 -- 172.21.15.122:0/1604463148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570106980 msgr2=0x7f257019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570106980 0x7f257019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 -- 172.21.15.122:0/1604463148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570101330 msgr2=0x7f2570197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570101330 0x7f2570197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 -- 172.21.15.122:0/1604463148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257019c240 con 0x7f2570100970 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f256effd640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9f0 tx=0x7f256000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f2557fff640 1 -- 172.21.15.122:0/1604463148 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256000cd80 con 0x7f2570100970 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257019c530 con 0x7f2570100970 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f2557fff640 1 -- 172.21.15.122:0/1604463148 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2560010910 con 0x7f2570100970 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f2557fff640 1 -- 172.21.15.122:0/1604463148 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560018990 con 0x7f2570100970 2024-04-02T13:40:43.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.620+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25701a3dc0 con 0x7f2570100970 2024-04-02T13:40:43.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2557fff640 1 -- 172.21.15.122:0/1604463148 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2560010ac0 con 0x7f2570100970 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 Processor -- start 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 -- start start 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2557fff640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 0x7f25580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b40702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5616b3fc67b0 con 0x5616b4062460 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5616b3f65570 con 0x5616b406deb0 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.623+0000 7f2578054740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5616b3fc2760 con 0x5616b4062d80 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:55974/0 (socket says 172.21.15.122:55974) 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256effd640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 0x7f25580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 -- 172.21.15.122:0/3622864251 learned_addr learned my addr 172.21.15.122:0/3622864251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55476/0 (socket says 172.21.15.122:55476) 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256f7fe640 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b40702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f2557fff640 1 -- 172.21.15.122:0/1604463148 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f25600d0e10 con 0x7f2570100970 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 -- 172.21.15.122:0/3622864251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 msgr2=0x5616b40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b40702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 -- 172.21.15.122:0/3622864251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 msgr2=0x5616b4062840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 -- 172.21.15.122:0/3622864251 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b3cda3f0 con 0x5616b4062d80 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256f7fe640 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256effd640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 0x7f25580a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25600ce280 tx=0x7f2560018420 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f256e7fc640 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f256400bc90 tx=0x7f25640107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3f38516e1abaf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f25557fa640 1 -- 172.21.15.122:0/3622864251 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564015a50 con 0x5616b4062d80 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f25557fa640 1 -- 172.21.15.122:0/3622864251 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f256400be40 con 0x5616b4062d80 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.624+0000 7f25557fa640 1 -- 172.21.15.122:0/3622864251 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564004aa0 con 0x5616b4062d80 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- 172.21.15.122:0/3622864251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 msgr2=0x5616b406d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f256400bc90 tx=0x7f25640107a0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- 172.21.15.122:0/3622864251 shutdown_connections 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b40702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b406d970 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 --2- 172.21.15.122:0/3622864251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4062840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- 172.21.15.122:0/3622864251 >> 172.21.15.122:0/3622864251 conn(0x5616b405ced0 msgr2=0x5616b405d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- 172.21.15.122:0/3622864251 shutdown_connections 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- 172.21.15.122:0/3622864251 wait complete. 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 -- start start 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.625+0000 7f2578054740 1 Processor -- start 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b4062460 0x5616b40690d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b406deb0 0x5616b4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5616b4063940 con 0x5616b4062d80 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5616b3f81e80 con 0x5616b4062460 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5616b4063ac0 con 0x5616b406deb0 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 10 client.0 _ll_register_callbacks cb 0x5616b3fbdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55488/0 (socket says 172.21.15.122:55488) 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 -- 172.21.15.122:0/240713155 learned_addr learned my addr 172.21.15.122:0/240713155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256f7fe640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b4062460 0x5616b40690d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256f7fe640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b406deb0 0x5616b4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 -- 172.21.15.122:0/240713155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b406deb0 msgr2=0x5616b4069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b406deb0 0x5616b4069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 -- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b4062460 msgr2=0x5616b40690d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b4062460 0x5616b40690d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 -- 172.21.15.122:0/240713155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b3da47d0 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f256e7fc640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2564006010 tx=0x7f2564015a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ced33bd9e2e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564005250 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f25640053f0 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.626+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5616b3fd3cf0 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2564004020 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 10 client.17373 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616b407aad0 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5616b3fd3cf0 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f2564004250 con 0x5616b4062d80 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 10 client.17373 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 10 client.17373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 20 client.17373 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b407be80 0x5616b407e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 4 client.17373 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b3fce040 con 0x5616b407be80 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f256f7fe640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b407be80 0x5616b407e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f256f7fe640 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b407be80 0x5616b407e270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb291fb2051f02ce server_cookie=2464a35004a9cdc1 in_seq=0 out_seq=0 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.627+0000 7f2552ff5640 10 client.17373 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2552ff5640 1 -- 172.21.15.122:0/240713155 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616b3fce040 con 0x5616b407be80 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2552ff5640 10 client.17373 handle_command_reply: tid=0 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 client.17373 shutdown 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 msgr2=0x5616b4069610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2564006010 tx=0x7f2564015a80 comp rx=0 tx=0).stop 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 shutdown_connections 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b407be80 0x5616b407e270 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b406deb0 0x5616b4069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2552ff5640 0 client.17373 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b4069610 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 --2- 172.21.15.122:0/240713155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b4062460 0x5616b40690d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 >> 172.21.15.122:0/240713155 conn(0x5616b405ced0 msgr2=0x5616b405ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 shutdown_connections 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 1 -- 172.21.15.122:0/240713155 wait complete. 2024-04-02T13:40:43.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.628+0000 7f2578054740 20 client.17373 trim_cache size 0 max 16384 2024-04-02T13:40:43.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 Processor -- start 2024-04-02T13:40:43.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 -- start start 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062460 0x5616b406d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4083a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25701ace40 con 0x5616b4062d80 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25701aea40 con 0x5616b406deb0 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f2578054740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25701ad510 con 0x5616b4062460 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55498/0 (socket says 172.21.15.122:55498) 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 -- 172.21.15.122:0/4153837046 learned_addr learned my addr 172.21.15.122:0/4153837046 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256f7fe640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062460 0x5616b406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256f7fe640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4083a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 -- 172.21.15.122:0/4153837046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062460 msgr2=0x5616b406d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062460 0x5616b406d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 -- 172.21.15.122:0/4153837046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 msgr2=0x5616b4083a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4083a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256e7fc640 1 -- 172.21.15.122:0/4153837046 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b407e830 con 0x5616b4062d80 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.642+0000 7f256f7fe640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4083a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f256e7fc640 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2564000c00 tx=0x7f2564017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85987e5326bb5192 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2554ff9640 1 -- 172.21.15.122:0/4153837046 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256403c070 con 0x5616b4062d80 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2554ff9640 1 -- 172.21.15.122:0/4153837046 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2564003cc0 con 0x5616b4062d80 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2554ff9640 1 -- 172.21.15.122:0/4153837046 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564015d50 con 0x5616b4062d80 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 -- 172.21.15.122:0/4153837046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 msgr2=0x5616b40680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2564000c00 tx=0x7f2564017040 comp rx=0 tx=0).stop 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 -- 172.21.15.122:0/4153837046 shutdown_connections 2024-04-02T13:40:43.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4083a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062d80 0x5616b40680d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 --2- 172.21.15.122:0/4153837046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062460 0x5616b406d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 -- 172.21.15.122:0/4153837046 >> 172.21.15.122:0/4153837046 conn(0x5616b405ced0 msgr2=0x5616b40679c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 -- 172.21.15.122:0/4153837046 shutdown_connections 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.643+0000 7f2578054740 1 -- 172.21.15.122:0/4153837046 wait complete. 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 -- start start 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 Processor -- start 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b4077470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25701ae2b0 con 0x5616b4062460 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25701adbe0 con 0x5616b406deb0 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25701aae20 con 0x5616b4062d80 2024-04-02T13:40:43.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f2578054740 10 client.0 _ll_register_callbacks cb 0x7f2560005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55502/0 (socket says 172.21.15.122:55502) 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 -- 172.21.15.122:0/904587098 learned_addr learned my addr 172.21.15.122:0/904587098 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256e7fc640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b4077470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 -- 172.21.15.122:0/904587098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 msgr2=0x5616b4077470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b4077470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 -- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 msgr2=0x5616b4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.644+0000 7f256f7fe640 1 -- 172.21.15.122:0/904587098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b405f1d0 con 0x5616b4062460 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f256f7fe640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2568005cc0 tx=0x7f256400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882e5b5d16659a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:43.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2568039040 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f2568003970 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2564003cc0 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f256805f050 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 10 client.17379 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5616b4075400 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2564003cc0 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f256803f040 con 0x5616b4062460 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 10 client.17379 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 10 client.17379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 20 client.17379 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b40e24b0 0x5616b40e48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 4 client.17379 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.645+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5616b3f4b6b0 con 0x5616b40e24b0 2024-04-02T13:40:43.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f256e7fc640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b40e24b0 0x5616b40e48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f256e7fc640 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b40e24b0 0x5616b40e48d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ffdfb517708f13 server_cookie=f16f8f24e083197c in_seq=0 out_seq=0 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2552ff5640 10 client.17379 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2552ff5640 1 -- 172.21.15.122:0/904587098 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5616b3f4b6b0 con 0x5616b40e24b0 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2552ff5640 10 client.17379 handle_command_reply: tid=0 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 client.17379 shutdown 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 msgr2=0x5616b4076f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f2568005cc0 tx=0x7f256400b630 comp rx=0 tx=0).stop 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 shutdown_connections 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5616b40e24b0 0x5616b40e48d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5616b406deb0 0x5616b4075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2552ff5640 0 client.17379 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5616b4062d80 0x5616b4077470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 --2- 172.21.15.122:0/904587098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5616b4062460 0x5616b4076f30 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 >> 172.21.15.122:0/904587098 conn(0x5616b405ced0 msgr2=0x5616b4068250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 shutdown_connections 2024-04-02T13:40:43.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 1 -- 172.21.15.122:0/904587098 wait complete. 2024-04-02T13:40:43.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.646+0000 7f2578054740 20 client.17379 trim_cache size 0 max 16384 2024-04-02T13:40:43.657 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 msgr2=0x7f25580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 0x7f25580a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25600ce280 tx=0x7f2560018420 comp rx=0 tx=0).stop 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 msgr2=0x7f2570197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f256000e9f0 tx=0x7f256000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 shutdown_connections 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25580a3d20 0x7f25580a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2570106980 0x7f257019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2570101330 0x7f2570197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 --2- 172.21.15.122:0/1604463148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2570100970 0x7f2570197250 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 >> 172.21.15.122:0/1604463148 conn(0x7f2570077b00 msgr2=0x7f2570103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 shutdown_connections 2024-04-02T13:40:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:43.647+0000 7f2575463640 1 -- 172.21.15.122:0/1604463148 wait complete. 2024-04-02T13:40:43.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 161334 2.66 MB/sec execute 124 sec latency 4807.635 ms 2024-04-02T13:40:44.662 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:40:44.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:40:44.662 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- 172.21.15.122:0/1168931588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 msgr2=0x7f1730107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 --2- 172.21.15.122:0/1168931588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f1730107f30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f172c009930 tx=0x7f172c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- 172.21.15.122:0/1168931588 shutdown_connections 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 --2- 172.21.15.122:0/1168931588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 0x7f173010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 --2- 172.21.15.122:0/1168931588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 --2- 172.21.15.122:0/1168931588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f1730107f30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- 172.21.15.122:0/1168931588 >> 172.21.15.122:0/1168931588 conn(0x7f1730077b00 msgr2=0x7f1730077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- 172.21.15.122:0/1168931588 shutdown_connections 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- 172.21.15.122:0/1168931588 wait complete. 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 Processor -- start 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.770+0000 7f1737788640 1 -- start start 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 0x7f1730198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1730111ea0 con 0x7f1730107b50 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1730111d20 con 0x7f1730068d00 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1737788640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1730112020 con 0x7f17300696c0 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735d25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1734d23640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 0x7f1730198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:42300/0 (socket says 172.21.15.122:42300) 2024-04-02T13:40:44.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735d25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55520/0 (socket says 172.21.15.122:55520) 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 -- 172.21.15.122:0/1148705346 learned_addr learned my addr 172.21.15.122:0/1148705346 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 -- 172.21.15.122:0/1148705346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 msgr2=0x7f1730198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 0x7f1730198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 -- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 msgr2=0x7f173019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 -- 172.21.15.122:0/1148705346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173019d330 con 0x7f1730068d00 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735d25640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f1735524640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f172c009230 tx=0x7f172c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.771+0000 7f17267fc640 1 -- 172.21.15.122:0/1148705346 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c041070 con 0x7f1730068d00 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.772+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173019d5c0 con 0x7f1730068d00 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.772+0000 7f17267fc640 1 -- 172.21.15.122:0/1148705346 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c0054e0 con 0x7f1730068d00 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.772+0000 7f17267fc640 1 -- 172.21.15.122:0/1148705346 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c003c90 con 0x7f1730068d00 2024-04-02T13:40:44.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.772+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17301a4eb0 con 0x7f1730068d00 2024-04-02T13:40:44.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.774+0000 7f17267fc640 1 -- 172.21.15.122:0/1148705346 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f172c044070 con 0x7f1730068d00 2024-04-02T13:40:44.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.774+0000 7f173a368740 1 Processor -- start 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 -- start start 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f17267fc640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 0x7f171c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 0x559987b9b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987b9dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559987af6770 con 0x559987b92be0 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735d25640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 0x7f171c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559987a9e960 con 0x559987b922c0 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f173a368740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559987af28c0 con 0x559987b9b7c0 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f17267fc640 1 -- 172.21.15.122:0/1148705346 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f172c0f2620 con 0x7f1730068d00 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735524640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 0x559987b9b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735524640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b926a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:42310/0 (socket says 172.21.15.122:42310) 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735524640 1 -- 172.21.15.122:0/3188319610 learned_addr learned my addr 172.21.15.122:0/3188319610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735d25640 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987b9dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 -- 172.21.15.122:0/3188319610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 msgr2=0x559987b9dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987b9dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 -- 172.21.15.122:0/3188319610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 msgr2=0x559987b926a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b926a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 -- 172.21.15.122:0/3188319610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599878d27e0 con 0x559987b92be0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1735d25640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 0x7f171c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f172800fb40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.775+0000 7f1734d23640 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 0x559987b9b280 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1720009bf0 tx=0x7f1720011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e31378e49a9517f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f171b7fe640 1 -- 172.21.15.122:0/3188319610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720004050 con 0x559987b92be0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f171b7fe640 1 -- 172.21.15.122:0/3188319610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1720005500 con 0x559987b92be0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f171b7fe640 1 -- 172.21.15.122:0/3188319610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1720004e00 con 0x559987b92be0 2024-04-02T13:40:44.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 -- 172.21.15.122:0/3188319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 msgr2=0x559987b9b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 0x559987b9b280 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f1720009bf0 tx=0x7f1720011810 comp rx=0 tx=0).stop 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 -- 172.21.15.122:0/3188319610 shutdown_connections 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987b9dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b92be0 0x559987b9b280 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 --2- 172.21.15.122:0/3188319610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b926a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 -- 172.21.15.122:0/3188319610 >> 172.21.15.122:0/3188319610 conn(0x559987b8c9f0 msgr2=0x559987b8cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 -- 172.21.15.122:0/3188319610 shutdown_connections 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.776+0000 7f173a368740 1 -- 172.21.15.122:0/3188319610 wait complete. 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 -- start start 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 Processor -- start 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b987c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b92be0 0x559987b98d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559987b9e5b0 con 0x559987b9b7c0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559987ab1d20 con 0x559987b922c0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559987b9e730 con 0x559987b92be0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f173a368740 10 client.0 _ll_register_callbacks cb 0x559987a7ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1735d25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1734d23640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b92be0 0x559987b98d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1735d25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55546/0 (socket says 172.21.15.122:55546) 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1735d25640 1 -- 172.21.15.122:0/3468693370 learned_addr learned my addr 172.21.15.122:0/3468693370 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1735524640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.777+0000 7f1735d25640 1 -- 172.21.15.122:0/3468693370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b92be0 msgr2=0x559987b98d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1735d25640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b92be0 0x559987b98d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1735d25640 1 -- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 msgr2=0x559987b987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1735d25640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1735d25640 1 -- 172.21.15.122:0/3468693370 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559987932f70 con 0x559987b9b7c0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1735d25640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1728003bd0 tx=0x7f1728004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441dc697ed3c3ac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728017040 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f17280047c0 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1720005500 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1728003e40 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 10 client.17391 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5599877ca170 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1720005500 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f1728024070 con 0x559987b9b7c0 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 10 client.17391 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 10 client.17391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 20 client.17391 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987babe10 0x559987bae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 4 client.17391 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:44.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.778+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559987b03cb0 con 0x559987babe10 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.779+0000 7f1735524640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987babe10 0x559987bae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.779+0000 7f1718ff9640 10 client.17391 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.779+0000 7f1735524640 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987babe10 0x559987bae270 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b77caa07d527ac9 server_cookie=199087ecd955ed62 in_seq=0 out_seq=0 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f1718ff9640 1 -- 172.21.15.122:0/3468693370 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559987b03cb0 con 0x559987babe10 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f1718ff9640 10 client.17391 handle_command_reply: tid=0 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 client.17391 shutdown 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 msgr2=0x559987b99240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f1728003bd0 tx=0x7f1728004ab0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 shutdown_connections 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987babe10 0x559987bae270 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b9b7c0 0x559987b99240 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b92be0 0x559987b98d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f1718ff9640 0 client.17391 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 --2- 172.21.15.122:0/3468693370 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b922c0 0x559987b987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 >> 172.21.15.122:0/3468693370 conn(0x559987b8c9f0 msgr2=0x559987b8e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 shutdown_connections 2024-04-02T13:40:44.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 1 -- 172.21.15.122:0/3468693370 wait complete. 2024-04-02T13:40:44.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.780+0000 7f173a368740 20 client.17391 trim_cache size 0 max 16384 2024-04-02T13:40:44.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 Processor -- start 2024-04-02T13:40:44.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 -- start start 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987ba74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17301aef30 con 0x559987b922c0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17301b0b30 con 0x559987b92be0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f173a368740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17301af600 con 0x559987b9b7c0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f1735524640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f1735524640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55554/0 (socket says 172.21.15.122:55554) 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f1735524640 1 -- 172.21.15.122:0/312219110 learned_addr learned my addr 172.21.15.122:0/312219110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f1735d25640 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987ba74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.798+0000 7f1734d23640 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 -- 172.21.15.122:0/312219110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 msgr2=0x559987ba74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987ba74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 -- 172.21.15.122:0/312219110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 msgr2=0x559987b97700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b97700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 -- 172.21.15.122:0/312219110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559987b91640 con 0x559987b922c0 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f1735524640 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ba0d0 tx=0x7f172c003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ffee4b3cef01c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f171affd640 1 -- 172.21.15.122:0/312219110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c04b040 con 0x559987b922c0 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f171affd640 1 -- 172.21.15.122:0/312219110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f172c02ccd0 con 0x559987b922c0 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.799+0000 7f171affd640 1 -- 172.21.15.122:0/312219110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0efc60 con 0x559987b922c0 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- 172.21.15.122:0/312219110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 msgr2=0x559987b9aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ba0d0 tx=0x7f172c003760 comp rx=0 tx=0).stop 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- 172.21.15.122:0/312219110 shutdown_connections 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987ba74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b97700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 --2- 172.21.15.122:0/312219110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9aef0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- 172.21.15.122:0/312219110 >> 172.21.15.122:0/312219110 conn(0x559987b8c9f0 msgr2=0x559987b8bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- 172.21.15.122:0/312219110 shutdown_connections 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- 172.21.15.122:0/312219110 wait complete. 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 -- start start 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.800+0000 7f173a368740 1 Processor -- start 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987c113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17301b03a0 con 0x559987b922c0 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17301afcd0 con 0x559987b92be0 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17301acf10 con 0x559987b9b7c0 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 10 client.0 _ll_register_callbacks cb 0x7f172800feb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55564/0 (socket says 172.21.15.122:55564) 2024-04-02T13:40:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 -- 172.21.15.122:0/2972390822 learned_addr learned my addr 172.21.15.122:0/2972390822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1734d23640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735d25640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987c113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 -- 172.21.15.122:0/2972390822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 msgr2=0x559987c113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987c113f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 -- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 msgr2=0x559987b9ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b9ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 -- 172.21.15.122:0/2972390822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559987c11930 con 0x559987b922c0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1735524640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ba850 tx=0x7f172c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69afdf700c2e9b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1718ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f172c0d6070 con 0x559987b922c0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f172c0efc60 con 0x559987b922c0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559987afe000 con 0x559987b922c0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f172c04b050 con 0x559987b922c0 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 10 client.17397 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:44.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.801+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559987c12be0 con 0x559987b922c0 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559987afe000 con 0x559987b922c0 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f172c0be650 con 0x559987b922c0 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 10 client.17397 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 20 client.17397 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987c13250 0x559987c15670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 4 client.17397 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559987a7bbe0 con 0x559987c13250 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.802+0000 7f1734d23640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987c13250 0x559987c15670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.803+0000 7f1734d23640 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987c13250 0x559987c15670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f89623b0b2f9a7 server_cookie=14ee5f6b07bbad0d in_seq=0 out_seq=0 2024-04-02T13:40:44.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.803+0000 7f1718ff9640 10 client.17397 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f1718ff9640 1 -- 172.21.15.122:0/2972390822 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559987a7bbe0 con 0x559987c13250 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f1718ff9640 10 client.17397 handle_command_reply: tid=0 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 client.17397 shutdown 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 msgr2=0x559987b9fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f172c0ba850 tx=0x7f172c049000 comp rx=0 tx=0).stop 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 shutdown_connections 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559987c13250 0x559987c15670 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559987b9b7c0 0x559987c113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559987b92be0 0x559987b9ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f1718ff9640 0 client.17397 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 --2- 172.21.15.122:0/2972390822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559987b922c0 0x559987b9fa70 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 >> 172.21.15.122:0/2972390822 conn(0x559987b8c9f0 msgr2=0x559987b98780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 shutdown_connections 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 1 -- 172.21.15.122:0/2972390822 wait complete. 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.804+0000 7f173a368740 20 client.17397 trim_cache size 0 max 16384 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "b671cc78-9e6f-4f0d-9c43-cc71f2584430", 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:40:44.812 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 msgr2=0x7f171c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 0x7f171c0a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1728009fd0 tx=0x7f172800fb40 comp rx=0 tx=0).stop 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 msgr2=0x7f1730198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f172c009230 tx=0x7f172c004270 comp rx=0 tx=0).stop 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 shutdown_connections 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f171c0a3d20 0x7f171c0a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1730107b50 0x7f173019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17300696c0 0x7f1730198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 --2- 172.21.15.122:0/1148705346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1730068d00 0x7f1730198340 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 >> 172.21.15.122:0/1148705346 conn(0x7f1730077b00 msgr2=0x7f17300fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 shutdown_connections 2024-04-02T13:40:44.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.805+0000 7f1737788640 1 -- 172.21.15.122:0/1148705346 wait complete. 2024-04-02T13:40:44.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:40:44.819 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:40:44.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 -- 172.21.15.122:0/3911352786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c101a80 msgr2=0x7ff37c101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 --2- 172.21.15.122:0/3911352786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c101a80 0x7ff37c101f00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff364009930 tx=0x7ff36402bc30 comp rx=0 tx=0).stop 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 -- 172.21.15.122:0/3911352786 shutdown_connections 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 --2- 172.21.15.122:0/3911352786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c102440 0x7ff37c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 --2- 172.21.15.122:0/3911352786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c101a80 0x7ff37c101f00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 --2- 172.21.15.122:0/3911352786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 -- 172.21.15.122:0/3911352786 >> 172.21.15.122:0/3911352786 conn(0x7ff37c0fd590 msgr2=0x7ff37c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 -- 172.21.15.122:0/3911352786 shutdown_connections 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 -- 172.21.15.122:0/3911352786 wait complete. 2024-04-02T13:40:44.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.932+0000 7ff38111b640 1 Processor -- start 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 -- start start 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c101a80 0x7ff37c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c111ea0 con 0x7ff37c102440 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c111d20 con 0x7ff37c107a90 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff38111b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c112020 con 0x7ff37c101a80 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55584/0 (socket says 172.21.15.122:55584) 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 -- 172.21.15.122:0/3757862343 learned_addr learned my addr 172.21.15.122:0/3757862343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37b59e640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c101a80 0x7ff37c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 -- 172.21.15.122:0/3757862343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c101a80 msgr2=0x7ff37c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c101a80 0x7ff37c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 -- 172.21.15.122:0/3757862343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 msgr2=0x7ff37c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 -- 172.21.15.122:0/3757862343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37c19d3f0 con 0x7ff37c102440 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37b59e640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.933+0000 7ff37259c640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff364009230 tx=0x7ff36402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.934+0000 7ff378d99640 1 -- 172.21.15.122:0/3757862343 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364041070 con 0x7ff37c102440 2024-04-02T13:40:44.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.934+0000 7ff378d99640 1 -- 172.21.15.122:0/3757862343 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff364004450 con 0x7ff37c102440 2024-04-02T13:40:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.934+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37c19d680 con 0x7ff37c102440 2024-04-02T13:40:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.934+0000 7ff378d99640 1 -- 172.21.15.122:0/3757862343 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364003e50 con 0x7ff37c102440 2024-04-02T13:40:44.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.934+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37c1a4f70 con 0x7ff37c102440 2024-04-02T13:40:44.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.936+0000 7ff378d99640 1 -- 172.21.15.122:0/3757862343 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff364044070 con 0x7ff37c102440 2024-04-02T13:40:44.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.936+0000 7ff378d99640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 0x7ff3680a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 Processor -- start 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 -- start start 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff378d99640 1 -- 172.21.15.122:0/3757862343 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff3640f2320 con 0x7ff37c102440 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 0x7ff3680a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x55661653fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x556616548890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5566164a3830 con 0x55661653f8d0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55661644b820 con 0x556616548dd0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff383dd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55661649f7e0 con 0x5566165401f0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x55661653fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x55661653fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55592/0 (socket says 172.21.15.122:55592) 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37ad9d640 1 -- 172.21.15.122:0/911755449 learned_addr learned my addr 172.21.15.122:0/911755449 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37259c640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x556616548890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 -- 172.21.15.122:0/911755449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 msgr2=0x556616548890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x556616548890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 0x7ff3680a6380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0134b0 tx=0x7ff36c013970 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 -- 172.21.15.122:0/911755449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 msgr2=0x55661653fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x55661653fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37b59e640 1 -- 172.21.15.122:0/911755449 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566162817d0 con 0x556616548dd0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.937+0000 7ff37259c640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x556616548890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.938+0000 7ff37b59e640 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654b1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff374005fd0 tx=0x7ff374011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2d3e7eec0f36ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.938+0000 7ff37159a640 1 -- 172.21.15.122:0/911755449 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004d20 con 0x556616548dd0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.938+0000 7ff37159a640 1 -- 172.21.15.122:0/911755449 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37400f920 con 0x556616548dd0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.938+0000 7ff37159a640 1 -- 172.21.15.122:0/911755449 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004440 con 0x556616548dd0 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- 172.21.15.122:0/911755449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 msgr2=0x55661654b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654b1c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff374005fd0 tx=0x7ff374011bf0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- 172.21.15.122:0/911755449 shutdown_connections 2024-04-02T13:40:44.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654b1c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x556616548890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- 172.21.15.122:0/911755449 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x55661653fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- 172.21.15.122:0/911755449 >> 172.21.15.122:0/911755449 conn(0x55661653a000 msgr2=0x55661653a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- 172.21.15.122:0/911755449 shutdown_connections 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- 172.21.15.122:0/911755449 wait complete. 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 -- start start 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 Processor -- start 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 0x5566165485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.939+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556616548dd0 0x556616545eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55661654bbc0 con 0x5566165401f0 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55661645ed30 con 0x55661653f8d0 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55661654bd40 con 0x556616548dd0 2024-04-02T13:40:44.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 10 client.0 _ll_register_callbacks cb 0x55661645e720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55606/0 (socket says 172.21.15.122:55606) 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 0x5566165485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37259c640 1 -- 172.21.15.122:0/2633346844 learned_addr learned my addr 172.21.15.122:0/2633346844 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37b59e640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556616548dd0 0x556616545eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2633346844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556616548dd0 msgr2=0x556616545eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556616548dd0 0x556616545eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2633346844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 msgr2=0x556616545970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2633346844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566161b73f0 con 0x55661653f8d0 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37259c640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:44.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 0x5566165485e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff36c013e70 tx=0x7ff36c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a5e53c944381e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff3627fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c003cf0 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37400f920 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.940+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff36c010430 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff36c003e90 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 10 client.26672 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5566161770f0 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37400f920 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7ff36c01f070 con 0x55661653f8d0 2024-04-02T13:40:44.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 10 client.26672 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 10 client.26672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 20 client.26672 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556616558e80 0x55661655b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 4 client.26672 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566164b0d70 con 0x556616558e80 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.941+0000 7ff37259c640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556616558e80 0x55661655b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.942+0000 7ff3627fc640 10 client.26672 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.942+0000 7ff37259c640 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556616558e80 0x55661655b300 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1e1346868ee077b server_cookie=8523b8d443ea3d64 in_seq=0 out_seq=0 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.942+0000 7ff3627fc640 1 -- 172.21.15.122:0/2633346844 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566164b0d70 con 0x556616558e80 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.942+0000 7ff3627fc640 10 client.26672 handle_command_reply: tid=0 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.942+0000 7ff383dd6740 1 client.26672 shutdown 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 msgr2=0x5566165485e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 0x5566165485e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff36c013e70 tx=0x7ff36c004730 comp rx=0 tx=0).stop 2024-04-02T13:40:44.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 shutdown_connections 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556616558e80 0x55661655b300 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556616548dd0 0x556616545eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566165401f0 0x556616545970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2633346844 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55661653f8d0 0x5566165485e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff3627fc640 0 client.26672 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 >> 172.21.15.122:0/2633346844 conn(0x55661653a000 msgr2=0x55661653bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 shutdown_connections 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 1 -- 172.21.15.122:0/2633346844 wait complete. 2024-04-02T13:40:44.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.943+0000 7ff383dd6740 20 client.26672 trim_cache size 0 max 16384 2024-04-02T13:40:44.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 163524 2.75 MB/sec execute 125 sec latency 26.835 ms 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.962+0000 7ff383dd6740 1 Processor -- start 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.962+0000 7ff383dd6740 1 -- start start 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661653f8d0 0x556616554ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566165401f0 0x556616544ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1aef70 con 0x556616548dd0 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1b0b70 con 0x5566165401f0 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff383dd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1af640 con 0x55661653f8d0 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55612/0 (socket says 172.21.15.122:55612) 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566165401f0 0x556616544ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 -- 172.21.15.122:0/2529336466 learned_addr learned my addr 172.21.15.122:0/2529336466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661653f8d0 0x556616554ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 -- 172.21.15.122:0/2529336466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661653f8d0 msgr2=0x556616554ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661653f8d0 0x556616554ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 -- 172.21.15.122:0/2529336466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566165401f0 msgr2=0x556616544ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566165401f0 0x556616544ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 -- 172.21.15.122:0/2529336466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556616539e30 con 0x556616548dd0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff37b59e640 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff374011730 tx=0x7ff374011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62866f4a45d0b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff370d99640 1 -- 172.21.15.122:0/2529336466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374011050 con 0x556616548dd0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff370d99640 1 -- 172.21.15.122:0/2529336466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37401f040 con 0x556616548dd0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.963+0000 7ff370d99640 1 -- 172.21.15.122:0/2529336466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3740053d0 con 0x556616548dd0 2024-04-02T13:40:44.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 -- 172.21.15.122:0/2529336466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 msgr2=0x5566165480a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff374011730 tx=0x7ff374011520 comp rx=0 tx=0).stop 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 -- 172.21.15.122:0/2529336466 shutdown_connections 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556616548dd0 0x5566165480a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566165401f0 0x556616544ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2529336466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55661653f8d0 0x556616554ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 -- 172.21.15.122:0/2529336466 >> 172.21.15.122:0/2529336466 conn(0x55661653a000 msgr2=0x556616540ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 -- 172.21.15.122:0/2529336466 shutdown_connections 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.964+0000 7ff383dd6740 1 -- 172.21.15.122:0/2529336466 wait complete. 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 -- start start 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 Processor -- start 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x556616551310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1b03e0 con 0x55661653f8d0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1afd10 con 0x556616548dd0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff37c1acf50 con 0x5566165401f0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff383dd6740 10 client.0 _ll_register_callbacks cb 0x7ff36c004350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff37259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff37259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:56132/0 (socket says 172.21.15.122:56132) 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff37ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x556616551310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff37259c640 1 -- 172.21.15.122:0/2142108232 learned_addr learned my addr 172.21.15.122:0/2142108232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.965+0000 7ff37b59e640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2142108232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 msgr2=0x55661654c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 msgr2=0x55661654cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 -- 172.21.15.122:0/2142108232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661654d3c0 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37259c640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff37ad9d640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x556616551310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff36c020480 tx=0x7ff36c012360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e547246c2e378889 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c020900 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7ff36c020cc0 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff37401f040 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff36c003e90 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 10 client.17415 fetch_fsmap learned FSMap version 38 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5566165bf420 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff37401f040 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7ff36c01f040 con 0x55661653f8d0 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 10 client.17415 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:40:44.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 10 client.17415 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 20 client.17415 populate_metadata read hostname 'smithi122' 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5566165bfc20 0x5566165c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 4 client.17415 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.966+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566164ab0c0 con 0x5566165bfc20 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff37259c640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5566165bfc20 0x5566165c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff3627fc640 10 client.17415 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff37259c640 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5566165bfc20 0x5566165c2040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a213616644740cec server_cookie=b741aaeb9374f57 in_seq=0 out_seq=0 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff3627fc640 1 -- 172.21.15.122:0/2142108232 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5566164ab0c0 con 0x5566165bfc20 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff3627fc640 10 client.17415 handle_command_reply: tid=0 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 client.17415 shutdown 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 msgr2=0x556616551310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x556616551310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff36c020480 tx=0x7ff36c012360 comp rx=0 tx=0).stop 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 shutdown_connections 2024-04-02T13:40:44.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5566165bfc20 0x5566165c2040 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556616548dd0 0x55661654cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff3627fc640 0 client.17415 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566165401f0 0x55661654c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 --2- 172.21.15.122:0/2142108232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55661653f8d0 0x556616551310 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.967+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 >> 172.21.15.122:0/2142108232 conn(0x55661653a000 msgr2=0x556616544ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 shutdown_connections 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff383dd6740 1 -- 172.21.15.122:0/2142108232 wait complete. 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff383dd6740 20 client.17415 trim_cache size 0 max 16384 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:40:44.978 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "b671cc78-9e6f-4f0d-9c43-cc71f2584430": { 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "b671cc78-9e6f-4f0d-9c43-cc71f2584430", 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:40:44.979 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 msgr2=0x7ff3680a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 0x7ff3680a6380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0134b0 tx=0x7ff36c013970 comp rx=0 tx=0).stop 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 msgr2=0x7ff37c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff364009230 tx=0x7ff36402cb30 comp rx=0 tx=0).stop 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 shutdown_connections 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3680a3ec0 0x7ff3680a6380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff37c107a90 0x7ff37c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff37c102440 0x7ff37c198940 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 --2- 172.21.15.122:0/3757862343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff37c101a80 0x7ff37c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:40:44.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 >> 172.21.15.122:0/3757862343 conn(0x7ff37c0fd590 msgr2=0x7ff37c0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:40:44.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 shutdown_connections 2024-04-02T13:40:44.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:40:44.968+0000 7ff38111b640 1 -- 172.21.15.122:0/3757862343 wait complete. 2024-04-02T13:40:44.982 INFO:tasks.cephfs.filesystem:scrub status for tag:b671cc78-9e6f-4f0d-9c43-cc71f2584430 - {'path': '/', 'tag': 'b671cc78-9e6f-4f0d-9c43-cc71f2584430', 'options': 'recursive,force'} 2024-04-02T13:40:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:45 smithi188 ceph-mon[36117]: pgmap v170: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 557 KiB/s wr, 67 op/s 2024-04-02T13:40:45.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:45 smithi122 ceph-mon[32114]: pgmap v170: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 557 KiB/s wr, 67 op/s 2024-04-02T13:40:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:45 smithi187 ceph-mon[36064]: pgmap v170: 129 pgs: 129 active+clean; 106 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 557 KiB/s wr, 67 op/s 2024-04-02T13:40:45.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 163906 2.73 MB/sec execute 126 sec latency 629.080 ms 2024-04-02T13:40:46.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:46 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:46 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:46 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:40:46.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 163906 2.71 MB/sec execute 127 sec latency 1629.126 ms 2024-04-02T13:40:47.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:47 smithi122 ceph-mon[32114]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:40:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:47 smithi187 ceph-mon[36064]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:40:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:47 smithi188 ceph-mon[36117]: pgmap v171: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:40:47.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 163906 2.69 MB/sec execute 128 sec latency 2629.204 ms 2024-04-02T13:40:48.955 INFO:tasks.workunit.client.0.smithi122.stdout: 1 164823 2.71 MB/sec execute 129 sec latency 3086.923 ms 2024-04-02T13:40:49.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:49 smithi122 ceph-mon[32114]: pgmap v172: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:49 smithi187 ceph-mon[36064]: pgmap v172: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:49.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:49 smithi188 ceph-mon[36117]: pgmap v172: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:49.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 167045 2.79 MB/sec execute 130 sec latency 13.622 ms 2024-04-02T13:40:50.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 167399 2.77 MB/sec execute 131 sec latency 607.359 ms 2024-04-02T13:40:51.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:51 smithi122 ceph-mon[32114]: pgmap v173: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:51 smithi187 ceph-mon[36064]: pgmap v173: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:51 smithi188 ceph-mon[36117]: pgmap v173: 129 pgs: 129 active+clean; 108 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 682 KiB/s wr, 71 op/s 2024-04-02T13:40:51.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 167399 2.75 MB/sec execute 132 sec latency 1607.419 ms 2024-04-02T13:40:52.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:52.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:52.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:52.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:52.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:40:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:40:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:40:52.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 167399 2.73 MB/sec execute 133 sec latency 2607.507 ms 2024-04-02T13:40:53.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:53 smithi122 ceph-mon[32114]: pgmap v174: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-04-02T13:40:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:53 smithi187 ceph-mon[36064]: pgmap v174: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-04-02T13:40:53.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:53 smithi188 ceph-mon[36117]: pgmap v174: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-04-02T13:40:53.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 168304 2.75 MB/sec execute 134 sec latency 3057.389 ms 2024-04-02T13:40:54.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170525 2.83 MB/sec execute 135 sec latency 12.773 ms 2024-04-02T13:40:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:55 smithi122 ceph-mon[32114]: pgmap v175: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-02T13:40:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:55 smithi187 ceph-mon[36064]: pgmap v175: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-02T13:40:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:55 smithi188 ceph-mon[36117]: pgmap v175: 129 pgs: 129 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-04-02T13:40:55.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170939 2.81 MB/sec execute 136 sec latency 562.463 ms 2024-04-02T13:40:56.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170939 2.79 MB/sec execute 137 sec latency 1562.523 ms 2024-04-02T13:40:57.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:57 smithi122 ceph-mon[32114]: pgmap v176: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-02T13:40:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:57 smithi187 ceph-mon[36064]: pgmap v176: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-02T13:40:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:57 smithi188 ceph-mon[36117]: pgmap v176: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-04-02T13:40:57.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170939 2.77 MB/sec execute 138 sec latency 2562.628 ms 2024-04-02T13:40:58.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170995 2.75 MB/sec execute 139 sec latency 3016.807 ms 2024-04-02T13:40:59.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:40:59 smithi122 ceph-mon[32114]: pgmap v177: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:40:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:40:59 smithi187 ceph-mon[36064]: pgmap v177: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:40:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:40:59 smithi188 ceph-mon[36117]: pgmap v177: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:40:59.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 170995 2.73 MB/sec execute 140 sec latency 1496.172 ms 2024-04-02T13:41:00.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 171156 2.72 MB/sec execute 141 sec latency 2376.125 ms 2024-04-02T13:41:01.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:01 smithi122 ceph-mon[32114]: pgmap v178: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:41:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:01 smithi187 ceph-mon[36064]: pgmap v178: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:41:01.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:01 smithi188 ceph-mon[36117]: pgmap v178: 129 pgs: 129 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-04-02T13:41:01.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 171726 2.73 MB/sec execute 142 sec latency 64.224 ms 2024-04-02T13:41:02.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 173983 2.80 MB/sec execute 143 sec latency 13.716 ms 2024-04-02T13:41:03.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:03 smithi122 ceph-mon[32114]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-02T13:41:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:03 smithi187 ceph-mon[36064]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-02T13:41:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:03 smithi188 ceph-mon[36117]: pgmap v179: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-02T13:41:03.956 INFO:tasks.workunit.client.0.smithi122.stdout: 1 174613 2.79 MB/sec execute 144 sec latency 352.232 ms 2024-04-02T13:41:04.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 174613 2.77 MB/sec execute 145 sec latency 1352.275 ms 2024-04-02T13:41:05.569 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:05 smithi188 ceph-mon[36117]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-04-02T13:41:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:05 smithi122 ceph-mon[32114]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-04-02T13:41:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:05 smithi187 ceph-mon[36064]: pgmap v180: 129 pgs: 129 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 856 KiB/s wr, 93 op/s 2024-04-02T13:41:05.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 174613 2.75 MB/sec execute 146 sec latency 2352.382 ms 2024-04-02T13:41:06.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 174613 2.73 MB/sec execute 147 sec latency 3352.426 ms 2024-04-02T13:41:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:07 smithi122 ceph-mon[32114]: pgmap v181: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-04-02T13:41:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:07 smithi187 ceph-mon[36064]: pgmap v181: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-04-02T13:41:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:07 smithi188 ceph-mon[36117]: pgmap v181: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-04-02T13:41:07.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 174613 2.71 MB/sec execute 148 sec latency 4352.539 ms 2024-04-02T13:41:08.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 175515 2.73 MB/sec execute 149 sec latency 4798.187 ms 2024-04-02T13:41:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:09 smithi122 ceph-mon[32114]: pgmap v182: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:09 smithi187 ceph-mon[36064]: pgmap v182: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:09.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:09 smithi188 ceph-mon[36117]: pgmap v182: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:09.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 177704 2.80 MB/sec execute 150 sec latency 13.438 ms 2024-04-02T13:41:10.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178088 2.79 MB/sec execute 151 sec latency 598.759 ms 2024-04-02T13:41:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:11 smithi122 ceph-mon[32114]: pgmap v183: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:11 smithi187 ceph-mon[36064]: pgmap v183: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:11 smithi188 ceph-mon[36117]: pgmap v183: 129 pgs: 129 active+clean; 115 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 690 KiB/s wr, 105 op/s 2024-04-02T13:41:11.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178088 2.77 MB/sec execute 152 sec latency 1598.861 ms 2024-04-02T13:41:12.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178088 2.75 MB/sec execute 153 sec latency 2598.953 ms 2024-04-02T13:41:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:13 smithi122 ceph-mon[32114]: pgmap v184: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T13:41:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:13 smithi187 ceph-mon[36064]: pgmap v184: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T13:41:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:13 smithi188 ceph-mon[36117]: pgmap v184: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-04-02T13:41:13.957 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178156 2.73 MB/sec execute 154 sec latency 3066.714 ms 2024-04-02T13:41:14.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178156 2.71 MB/sec execute 155 sec latency 1476.316 ms 2024-04-02T13:41:14.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:14.983 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:15.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2865071636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4106980 msgr2=0x7f8aa4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2865071636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4106980 0x7f8aa4106d60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0009930 tx=0x7f8aa002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2865071636 shutdown_connections 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2865071636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4101330 0x7f8aa410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2865071636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4100970 0x7f8aa4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2865071636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4106980 0x7f8aa4106d60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2865071636 >> 172.21.15.122:0/2865071636 conn(0x7f8aa4077b00 msgr2=0x7f8aa4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2865071636 shutdown_connections 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2865071636 wait complete. 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 Processor -- start 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.095+0000 7f8aac5e9640 1 -- start start 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4100970 0x7f8aa4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4106980 0x7f8aa419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110d60 con 0x7f8aa4101330 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110be0 con 0x7f8aa4106980 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa4110ee0 con 0x7f8aa4100970 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49632/0 (socket says 172.21.15.122:49632) 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 -- 172.21.15.122:0/2786469832 learned_addr learned my addr 172.21.15.122:0/2786469832 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4106980 0x7f8aa419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aaa385640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4100970 0x7f8aa4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 -- 172.21.15.122:0/2786469832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4100970 msgr2=0x7f8aa4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4100970 0x7f8aa4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 -- 172.21.15.122:0/2786469832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4106980 msgr2=0x7f8aa419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4106980 0x7f8aa419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 -- 172.21.15.122:0/2786469832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa419c230 con 0x7f8aa4101330 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c920 tx=0x7f8a9400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8a9b7fe640 1 -- 172.21.15.122:0/2786469832 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94007c70 con 0x7f8aa4101330 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8a9b7fe640 1 -- 172.21.15.122:0/2786469832 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a94007e10 con 0x7f8aa4101330 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa419c520 con 0x7f8aa4101330 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8a9b7fe640 1 -- 172.21.15.122:0/2786469832 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94018890 con 0x7f8aa4101330 2024-04-02T13:41:15.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.096+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa4068c60 con 0x7f8aa4101330 2024-04-02T13:41:15.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.099+0000 7f8a9b7fe640 1 -- 172.21.15.122:0/2786469832 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8a94018a70 con 0x7f8aa4101330 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.099+0000 7f8aaf1d3740 1 Processor -- start 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.099+0000 7f8aaf1d3740 1 -- start start 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a1718e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 0x559a171911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a170e9830 con 0x559a171858d0 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559a17091820 con 0x559a1718edd0 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a170e57e0 con 0x559a171861f0 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8a9b7fe640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49648/0 (socket says 172.21.15.122:49648) 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 -- 172.21.15.122:0/2713751354 learned_addr learned my addr 172.21.15.122:0/2713751354 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaa385640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a1718e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8a9b7fe640 1 -- 172.21.15.122:0/2786469832 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8a940d0d90 con 0x7f8aa4101330 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 0x559a171911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 -- 172.21.15.122:0/2713751354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 msgr2=0x559a1718e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a1718e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 -- 172.21.15.122:0/2713751354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 msgr2=0x559a171911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 0x559a171911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 -- 172.21.15.122:0/2713751354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a16ec77d0 con 0x559a171858d0 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 0x559a171911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:15.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.100+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ee90 tx=0x7f8a9c01b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e4cb635637bcc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaab86640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 0x7f8a900a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c01bbb0 tx=0x7f8a9c01f000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8a98bfe640 1 -- 172.21.15.122:0/2713751354 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00e7e0 con 0x559a171858d0 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8a98bfe640 1 -- 172.21.15.122:0/2713751354 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c009b70 con 0x559a171858d0 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8a98bfe640 1 -- 172.21.15.122:0/2713751354 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00ccb0 con 0x559a171858d0 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/2713751354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 msgr2=0x559a17185cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00ee90 tx=0x7f8a9c01b6b0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/2713751354 shutdown_connections 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a1718edd0 0x559a171911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a1718e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/2713751354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a17185cb0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.101+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/2713751354 >> 172.21.15.122:0/2713751354 conn(0x559a17180000 msgr2=0x559a171803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/2713751354 shutdown_connections 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/2713751354 wait complete. 2024-04-02T13:41:15.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- start start 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 Processor -- start 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1718bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559a17191b60 con 0x559a171858d0 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559a170a4d30 con 0x559a171861f0 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559a17191ce0 con 0x559a1718edd0 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaf1d3740 10 client.0 _ll_register_callbacks cb 0x559a170a4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaa385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaa385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49650/0 (socket says 172.21.15.122:49650) 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaa385640 1 -- 172.21.15.122:0/635251361 learned_addr learned my addr 172.21.15.122:0/635251361 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1718bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.102+0000 7f8aaab86640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 -- 172.21.15.122:0/635251361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 msgr2=0x559a1718c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 -- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 msgr2=0x559a1718bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1718bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 -- 172.21.15.122:0/635251361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a16dfd3f0 con 0x559a171858d0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaa385640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036040 tx=0x7f8aa002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e3167e2a295ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0005230 con 0x559a171858d0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8aa00053d0 con 0x559a171858d0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c009b70 con 0x559a171858d0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8aa0004490 con 0x559a171858d0 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 10 client.17427 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559a16dbd0f0 con 0x559a171858d0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c009b70 con 0x559a171858d0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f8aa0003820 con 0x559a171858d0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 10 client.17427 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 20 client.17427 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a1719eea0 0x559a171a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 4 client.17427 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.103+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a170f6d70 con 0x559a1719eea0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.104+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a1719eea0 0x559a171a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.104+0000 7f8a8dffb640 10 client.17427 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.104+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a1719eea0 0x559a171a12c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a54f47d9dab1c820 server_cookie=7318b5cc6a2df0eb in_seq=0 out_seq=0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8a8dffb640 1 -- 172.21.15.122:0/635251361 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a170f6d70 con 0x559a1719eea0 2024-04-02T13:41:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8a8dffb640 10 client.17427 handle_command_reply: tid=0 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 client.17427 shutdown 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 msgr2=0x559a1718e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0036040 tx=0x7f8aa002c2b0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 shutdown_connections 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a1719eea0 0x559a171a12c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1718bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8a8dffb640 0 client.17427 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/635251361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1718e770 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 >> 172.21.15.122:0/635251361 conn(0x559a17180000 msgr2=0x559a17181d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 shutdown_connections 2024-04-02T13:41:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/635251361 wait complete. 2024-04-02T13:41:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.105+0000 7f8aaf1d3740 20 client.17427 trim_cache size 0 max 16384 2024-04-02T13:41:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.122+0000 7f8aaf1d3740 1 Processor -- start 2024-04-02T13:41:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 -- start start 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1717f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41acd60 con 0x559a171858d0 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ae960 con 0x559a171861f0 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ad430 con 0x559a1718edd0 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49652/0 (socket says 172.21.15.122:49652) 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 -- 172.21.15.122:0/925415922 learned_addr learned my addr 172.21.15.122:0/925415922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1717f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 -- 172.21.15.122:0/925415922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 msgr2=0x559a1718e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 -- 172.21.15.122:0/925415922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 msgr2=0x559a1717f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1717f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 -- 172.21.15.122:0/925415922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a1718be70 con 0x559a171858d0 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8aaa385640 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8aa002c820 tx=0x7f8aa002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa9cad2c9e27260 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8a8ffff640 1 -- 172.21.15.122:0/925415922 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00035f0 con 0x559a171858d0 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8a8ffff640 1 -- 172.21.15.122:0/925415922 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0005550 con 0x559a171858d0 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.123+0000 7f8a8ffff640 1 -- 172.21.15.122:0/925415922 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004a80 con 0x559a171858d0 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/925415922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 msgr2=0x559a1719aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8aa002c820 tx=0x7f8aa002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/925415922 shutdown_connections 2024-04-02T13:41:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a1718edd0 0x559a1718e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171861f0 0x559a1717f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/925415922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a171858d0 0x559a1719aa20 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/925415922 >> 172.21.15.122:0/925415922 conn(0x559a17180000 msgr2=0x559a1717e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/925415922 shutdown_connections 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/925415922 wait complete. 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 -- start start 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.124+0000 7f8aaf1d3740 1 Processor -- start 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171858d0 0x559a171974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41ae1d0 con 0x559a1718edd0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41adb00 con 0x559a171858d0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa41aad40 con 0x559a171861f0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaf1d3740 10 client.0 _ll_register_callbacks cb 0x7f8aa002cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aa9b84640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aa9b84640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54502/0 (socket says 172.21.15.122:54502) 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49656/0 (socket says 172.21.15.122:49656) 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aa9b84640 1 -- 172.21.15.122:0/705983602 learned_addr learned my addr 172.21.15.122:0/705983602 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaa385640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171858d0 0x559a171974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 -- 172.21.15.122:0/705983602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 msgr2=0x559a17192810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 -- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171858d0 msgr2=0x559a171974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171858d0 0x559a171974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 -- 172.21.15.122:0/705983602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a17204440 con 0x559a1718edd0 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aa9b84640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8aaab86640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006010 tx=0x7f8a9c0059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492dc68213d1393d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.125+0000 7f8a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c034020 con 0x559a1718edd0 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8a9c009b70 con 0x559a1718edd0 2024-04-02T13:41:15.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa0005550 con 0x559a1718edd0 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c00e100 con 0x559a1718edd0 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 10 client.17433 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559a172055f0 con 0x559a1718edd0 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aa0005550 con 0x559a1718edd0 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f8a9c024020 con 0x559a1718edd0 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 10 client.17433 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 10 client.17433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 20 client.17433 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a17205b30 0x559a17207f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 4 client.17433 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a170f10c0 con 0x559a17205b30 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.126+0000 7f8aaa385640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a17205b30 0x559a17207f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8a8dffb640 10 client.17433 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaa385640 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a17205b30 0x559a17207f50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a70ab728c4eec6f server_cookie=d809e18c81498634 in_seq=0 out_seq=0 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8a8dffb640 1 -- 172.21.15.122:0/705983602 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a170f10c0 con 0x559a17205b30 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8a8dffb640 10 client.17433 handle_command_reply: tid=0 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 client.17433 shutdown 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 msgr2=0x559a17192f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c006010 tx=0x7f8a9c0059f0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 shutdown_connections 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559a17205b30 0x559a17207f50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559a1718edd0 0x559a17192f00 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8a8dffb640 0 client.17433 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559a171861f0 0x559a17192810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 --2- 172.21.15.122:0/705983602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559a171858d0 0x559a171974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 >> 172.21.15.122:0/705983602 conn(0x559a17180000 msgr2=0x559a1717f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 shutdown_connections 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 1 -- 172.21.15.122:0/705983602 wait complete. 2024-04-02T13:41:15.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.127+0000 7f8aaf1d3740 20 client.17433 trim_cache size 0 max 16384 2024-04-02T13:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:41:15.140 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 msgr2=0x7f8a900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 0x7f8a900a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c01bbb0 tx=0x7f8a9c01f000 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 msgr2=0x7f8aa4197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8a9400c920 tx=0x7f8a9400cde0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 shutdown_connections 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a900a3d20 0x7f8a900a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8aa4106980 0x7f8aa419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa4101330 0x7f8aa4197780 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 --2- 172.21.15.122:0/2786469832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8aa4100970 0x7f8aa4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 >> 172.21.15.122:0/2786469832 conn(0x7f8aa4077b00 msgr2=0x7f8aa4103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 shutdown_connections 2024-04-02T13:41:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.128+0000 7f8aac5e9640 1 -- 172.21.15.122:0/2786469832 wait complete. 2024-04-02T13:41:15.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:41:15.143 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:41:15.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 -- 172.21.15.122:0/2507821644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0100a80 msgr2=0x7f8ea0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/2507821644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0100a80 0x7f8ea0100f00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009930 tx=0x7f8e9402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 -- 172.21.15.122:0/2507821644 shutdown_connections 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/2507821644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0101440 0x7f8ea010d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/2507821644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0100a80 0x7f8ea0100f00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/2507821644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0106a40 0x7f8ea0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 -- 172.21.15.122:0/2507821644 >> 172.21.15.122:0/2507821644 conn(0x7f8ea0077b60 msgr2=0x7f8ea0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 -- 172.21.15.122:0/2507821644 shutdown_connections 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.242+0000 7f8ea66e3640 1 -- 172.21.15.122:0/2507821644 wait complete. 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 Processor -- start 2024-04-02T13:41:15.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 -- start start 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 0x7f8ea0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0106a40 0x7f8ea019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0110e50 con 0x7f8ea0101440 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0110cd0 con 0x7f8ea0106a40 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea66e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea0110fd0 con 0x7f8ea0100a80 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49666/0 (socket says 172.21.15.122:49666) 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 -- 172.21.15.122:0/3208542344 learned_addr learned my addr 172.21.15.122:0/3208542344 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9ffff640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 0x7f8ea0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8ea4c80640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0106a40 0x7f8ea019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 -- 172.21.15.122:0/3208542344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 msgr2=0x7f8ea0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 0x7f8ea0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 -- 172.21.15.122:0/3208542344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0106a40 msgr2=0x7f8ea019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0106a40 0x7f8ea019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9f7fe640 1 -- 172.21.15.122:0/3208542344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea019d420 con 0x7f8ea0101440 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.243+0000 7f8e9ffff640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 0x7f8ea0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8e9f7fe640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e9402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94041070 con 0x7f8ea0101440 2024-04-02T13:41:15.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e94004450 con 0x7f8ea0101440 2024-04-02T13:41:15.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea019d6b0 con 0x7f8ea0101440 2024-04-02T13:41:15.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94034440 con 0x7f8ea0101440 2024-04-02T13:41:15.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.244+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea01a4fa0 con 0x7f8ea0101440 2024-04-02T13:41:15.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.245+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ea0102170 con 0x7f8ea0101440 2024-04-02T13:41:15.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.246+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8e94044070 con 0x7f8ea0101440 2024-04-02T13:41:15.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.247+0000 7f8e9d7fa640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 0x7f8e880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.247+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8e940f2970 con 0x7f8ea0101440 2024-04-02T13:41:15.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.247+0000 7f8e9ffff640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 0x7f8e880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.247+0000 7f8e9ffff640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 0x7f8e880a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e90005e00 tx=0x7f8e90005c70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.251+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8e940bed90 con 0x7f8ea0101440 2024-04-02T13:41:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.467+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ea0109fb0 con 0x7f8ea0101440 2024-04-02T13:41:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.468+0000 7f8e9d7fa640 1 -- 172.21.15.122:0/3208542344 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11949 (secure 0 0 0) 0x7f8e940be690 con 0x7f8ea0101440 2024-04-02T13:41:15.471 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:41:15.471 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:40:36.405088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:41:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 38 2024-04-02T13:41:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 msgr2=0x7f8e880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 0x7f8e880a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8e90005e00 tx=0x7f8e90005c70 comp rx=0 tx=0).stop 2024-04-02T13:41:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 msgr2=0x7f8ea0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e9402cc70 comp rx=0 tx=0).stop 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 shutdown_connections 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e880a3d20 0x7f8e880a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ea0106a40 0x7f8ea019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ea0101440 0x7f8ea0198970 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 --2- 172.21.15.122:0/3208542344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ea0100a80 0x7f8ea0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 >> 172.21.15.122:0/3208542344 conn(0x7f8ea0077b60 msgr2=0x7f8ea006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 shutdown_connections 2024-04-02T13:41:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.470+0000 7f8ea66e3640 1 -- 172.21.15.122:0/3208542344 wait complete. 2024-04-02T13:41:15.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:41:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:15 smithi122 ceph-mon[32114]: pgmap v185: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 131 op/s 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- 172.21.15.122:0/3980738781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8101af0 msgr2=0x7fedd810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 --2- 172.21.15.122:0/3980738781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8101af0 0x7fedd810d820 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fedd0009ca0 tx=0x7fedd002c140 comp rx=0 tx=0).stop 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- 172.21.15.122:0/3980738781 shutdown_connections 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 --2- 172.21.15.122:0/3980738781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd8101af0 0x7fedd810d820 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 --2- 172.21.15.122:0/3980738781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101150 0x7fedd81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 --2- 172.21.15.122:0/3980738781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd81059b0 0x7fedd8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- 172.21.15.122:0/3980738781 >> 172.21.15.122:0/3980738781 conn(0x7fedd8076fc0 msgr2=0x7fedd80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- 172.21.15.122:0/3980738781 shutdown_connections 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- 172.21.15.122:0/3980738781 wait complete. 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 Processor -- start 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.586+0000 7feddf0eb640 1 -- start start 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 0x7fedd81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 0x7fedd8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8110f50 con 0x7fedd81059b0 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8110dd0 con 0x7fedd8101af0 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81110d0 con 0x7fedd8101150 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49684/0 (socket says 172.21.15.122:49684) 2024-04-02T13:41:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 -- 172.21.15.122:0/1319745506 learned_addr learned my addr 172.21.15.122:0/1319745506 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 -- 172.21.15.122:0/1319745506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 msgr2=0x7fedd81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 0x7fedd8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddce87640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 0x7fedd81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 0x7fedd81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 -- 172.21.15.122:0/1319745506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 msgr2=0x7fedd8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 0x7fedd8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 -- 172.21.15.122:0/1319745506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd81a48b0 con 0x7fedd81059b0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddce87640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 0x7fedd81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 0x7fedd8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddd688640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fedd002c750 tx=0x7fedd00346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7fedd5ffb640 1 -- 172.21.15.122:0/1319745506 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0039050 con 0x7fedd81059b0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7fedd5ffb640 1 -- 172.21.15.122:0/1319745506 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd0045070 con 0x7fedd81059b0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd81a4a80 con 0x7fedd81059b0 2024-04-02T13:41:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.587+0000 7fedd5ffb640 1 -- 172.21.15.122:0/1319745506 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd0040450 con 0x7fedd81059b0 2024-04-02T13:41:15.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.588+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a4ee0 con 0x7fedd81059b0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd5ffb640 1 -- 172.21.15.122:0/1319745506 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fedd0004040 con 0x7fedd81059b0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 Processor -- start 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 -- start start 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc65b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55efc651d7b0 con 0x55efc65c2ca0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55efc64bc570 con 0x55efc65b97a0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fede1da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55efc6519760 con 0x55efc65ba0c0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49700/0 (socket says 172.21.15.122:49700) 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd7fff640 1 -- 172.21.15.122:0/1904622201 learned_addr learned my addr 172.21.15.122:0/1904622201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc65b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7feddce87640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.590+0000 7fedd5ffb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 0x7fedc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 -- 172.21.15.122:0/1904622201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 msgr2=0x55efc65c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 -- 172.21.15.122:0/1904622201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 msgr2=0x55efc65b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc65b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7feddd688640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 0x7fedc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 -- 172.21.15.122:0/1904622201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc62313f0 con 0x55efc65c2ca0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7feddce87640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd5ffb640 1 -- 172.21.15.122:0/1319745506 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fedd00f37d0 con 0x7fedd81059b0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedd7fff640 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006fd0 tx=0x7fedc400c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a585e9f75f7a757 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedc2ffd640 1 -- 172.21.15.122:0/1904622201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401c070 con 0x55efc65c2ca0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7feddd688640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 0x7fedc80a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fedd00bb950 tx=0x7fedd0089f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedc2ffd640 1 -- 172.21.15.122:0/1904622201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4017af0 con 0x55efc65c2ca0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.591+0000 7fedc2ffd640 1 -- 172.21.15.122:0/1904622201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc400f040 con 0x55efc65c2ca0 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- 172.21.15.122:0/1904622201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 msgr2=0x55efc65c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fedc4006fd0 tx=0x7fedc400c230 comp rx=0 tx=0).stop 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- 172.21.15.122:0/1904622201 shutdown_connections 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c5090 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 --2- 172.21.15.122:0/1904622201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc65b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- 172.21.15.122:0/1904622201 >> 172.21.15.122:0/1904622201 conn(0x55efc65b3ed0 msgr2=0x55efc65b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- 172.21.15.122:0/1904622201 shutdown_connections 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- 172.21.15.122:0/1904622201 wait complete. 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 -- start start 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.592+0000 7fede1da9740 1 Processor -- start 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 0x55efc65c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55efc65c5a90 con 0x55efc65ba0c0 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55efc64d8e80 con 0x55efc65c2ca0 2024-04-02T13:41:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55efc65c5c10 con 0x55efc65b97a0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 10 client.0 _ll_register_callbacks cb 0x55efc6514aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49710/0 (socket says 172.21.15.122:49710) 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 -- 172.21.15.122:0/3295871571 learned_addr learned my addr 172.21.15.122:0/3295871571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedd7fff640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 0x55efc65c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 -- 172.21.15.122:0/3295871571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 msgr2=0x55efc65c2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 0x55efc65c2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 -- 172.21.15.122:0/3295871571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 msgr2=0x55efc65c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 -- 172.21.15.122:0/3295871571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc62fb7d0 con 0x55efc65ba0c0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedd7fff640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 0x55efc65c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7feddce87640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00df70 tx=0x7fedcc00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e07652bb72273a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc014690 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fedcc014830 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc4017af0 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fedcc023050 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 10 client.17451 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.593+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55efc65d1ba0 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc4017af0 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fedcc01c3e0 con 0x55efc65ba0c0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 10 client.17451 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 10 client.17451 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 20 client.17451 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc65d30b0 0x55efc65d5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 4 client.17451 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc652acf0 con 0x55efc65d30b0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fedd7fff640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc65d30b0 0x55efc65d5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fedb7fff640 10 client.17451 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.594+0000 7fedd7fff640 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc65d30b0 0x55efc65d5510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3936968978d09e3b server_cookie=f1b809448bf48561 in_seq=0 out_seq=0 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fedb7fff640 1 -- 172.21.15.122:0/3295871571 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efc652acf0 con 0x55efc65d30b0 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fedb7fff640 10 client.17451 handle_command_reply: tid=0 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 client.17451 shutdown 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 msgr2=0x55efc65bfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00df70 tx=0x7fedcc00f670 comp rx=0 tx=0).stop 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 shutdown_connections 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc65d30b0 0x55efc65d5510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65ba0c0 0x55efc65bfaf0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 --2- 172.21.15.122:0/3295871571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65b97a0 0x55efc65c2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fedb7fff640 0 client.17451 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 >> 172.21.15.122:0/3295871571 conn(0x55efc65b3ed0 msgr2=0x55efc65b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 shutdown_connections 2024-04-02T13:41:15.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 1 -- 172.21.15.122:0/3295871571 wait complete. 2024-04-02T13:41:15.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.595+0000 7fede1da9740 20 client.17451 trim_cache size 0 max 16384 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 Processor -- start 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 -- start start 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65ba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81aeed0 con 0x55efc65b97a0 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81b0ad0 con 0x55efc65c2ca0 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.608+0000 7fede1da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81af5a0 con 0x55efc65ba0c0 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49726/0 (socket says 172.21.15.122:49726) 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 -- 172.21.15.122:0/4262380307 learned_addr learned my addr 172.21.15.122:0/4262380307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 -- 172.21.15.122:0/4262380307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 msgr2=0x55efc65ba4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65ba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 -- 172.21.15.122:0/4262380307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 msgr2=0x55efc65cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 -- 172.21.15.122:0/4262380307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc65b9ef0 con 0x55efc65b97a0 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedd7fff640 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fedc4008010 tx=0x7fedc40166a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289d8e7add748bde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedc27fc640 1 -- 172.21.15.122:0/4262380307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc401c040 con 0x55efc65b97a0 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedc27fc640 1 -- 172.21.15.122:0/4262380307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc401d040 con 0x55efc65b97a0 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.609+0000 7fedc27fc640 1 -- 172.21.15.122:0/4262380307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4027310 con 0x55efc65b97a0 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- 172.21.15.122:0/4262380307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 msgr2=0x55efc65ce950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fedc4008010 tx=0x7fedc40166a0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- 172.21.15.122:0/4262380307 shutdown_connections 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65c2ca0 0x55efc65cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc65ba4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- 172.21.15.122:0/4262380307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65b97a0 0x55efc65ce950 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- 172.21.15.122:0/4262380307 >> 172.21.15.122:0/4262380307 conn(0x55efc65b3ed0 msgr2=0x55efc65be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- 172.21.15.122:0/4262380307 shutdown_connections 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- 172.21.15.122:0/4262380307 wait complete. 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- start start 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 Processor -- start 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc6635da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc66362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81b0340 con 0x55efc65c2ca0 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81afc70 con 0x55efc65b97a0 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81aceb0 con 0x55efc65ba0c0 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.610+0000 7fede1da9740 10 client.0 _ll_register_callbacks cb 0x7fedcc00d220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49736/0 (socket says 172.21.15.122:49736) 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 -- 172.21.15.122:0/701457550 learned_addr learned my addr 172.21.15.122:0/701457550 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7feddce87640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc66362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc6635da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 -- 172.21.15.122:0/701457550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 msgr2=0x55efc66362e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc66362e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 -- 172.21.15.122:0/701457550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 msgr2=0x55efc6635da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc6635da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 -- 172.21.15.122:0/701457550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc65c1fb0 con 0x55efc65c2ca0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedd7fff640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c9b0 tx=0x7fedc4016e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8361f6f11be40f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc4006bf0 con 0x55efc65c2ca0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fedc40270d0 con 0x55efc65c2ca0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc401d040 con 0x55efc65c2ca0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fedc40272d0 con 0x55efc65c2ca0 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fede1da9740 10 client.17457 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55efc663d5c0 con 0x55efc65c2ca0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.611+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc401d040 con 0x55efc65c2ca0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fedc4015040 con 0x55efc65c2ca0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 10 client.17457 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 10 client.17457 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 20 client.17457 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc663d940 0x55efc663fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 4 client.17457 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55efc6525040 con 0x55efc663d940 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7feddce87640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc663d940 0x55efc663fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7feddce87640 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc663d940 0x55efc663fd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c418b2a1528f5f2c server_cookie=25c87a38027ea63e in_seq=0 out_seq=0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fedb7fff640 10 client.17457 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fedb7fff640 1 -- 172.21.15.122:0/701457550 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55efc6525040 con 0x55efc663d940 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fedb7fff640 10 client.17457 handle_command_reply: tid=0 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 client.17457 shutdown 2024-04-02T13:41:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 msgr2=0x55efc65c1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fedc400c9b0 tx=0x7fedc4016e90 comp rx=0 tx=0).stop 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 shutdown_connections 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55efc663d940 0x55efc663fd60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55efc65c2ca0 0x55efc65c1870 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fedb7fff640 0 client.17457 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55efc65ba0c0 0x55efc66362e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 --2- 172.21.15.122:0/701457550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55efc65b97a0 0x55efc6635da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.612+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 >> 172.21.15.122:0/701457550 conn(0x55efc65b3ed0 msgr2=0x55efc65be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 shutdown_connections 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7fede1da9740 1 -- 172.21.15.122:0/701457550 wait complete. 2024-04-02T13:41:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7fede1da9740 20 client.17457 trim_cache size 0 max 16384 2024-04-02T13:41:15.623 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 msgr2=0x7fedc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 0x7fedc80a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fedd00bb950 tx=0x7fedd0089f90 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 msgr2=0x7fedd81a4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fedd002c750 tx=0x7fedd00346f0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 shutdown_connections 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fedc80a3d20 0x7fedc80a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fedd81059b0 0x7fedd81a4370 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fedd8101af0 0x7fedd8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 --2- 172.21.15.122:0/1319745506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fedd8101150 0x7fedd81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 >> 172.21.15.122:0/1319745506 conn(0x7fedd8076fc0 msgr2=0x7fedd8103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 shutdown_connections 2024-04-02T13:41:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.613+0000 7feddf0eb640 1 -- 172.21.15.122:0/1319745506 wait complete. 2024-04-02T13:41:15.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:41:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 -- 172.21.15.122:0/112593326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 msgr2=0x7fed6010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 --2- 172.21.15.122:0/112593326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed6010e960 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fed58009cd0 tx=0x7fed5802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 -- 172.21.15.122:0/112593326 shutdown_connections 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 --2- 172.21.15.122:0/112593326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed6010e960 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 --2- 172.21.15.122:0/112593326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 --2- 172.21.15.122:0/112593326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 0x7fed60107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 -- 172.21.15.122:0/112593326 >> 172.21.15.122:0/112593326 conn(0x7fed60078f00 msgr2=0x7fed60079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 -- 172.21.15.122:0/112593326 shutdown_connections 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.726+0000 7fed66382640 1 -- 172.21.15.122:0/112593326 wait complete. 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 Processor -- start 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 -- start start 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 0x7fed60074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111e80 con 0x7fed60102450 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed60111d00 con 0x7fed60101a90 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed66382640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed60112000 con 0x7fed60107aa0 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed5f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed5f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49762/0 (socket says 172.21.15.122:49762) 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed5f7fe640 1 -- 172.21.15.122:0/75506774 learned_addr learned my addr 172.21.15.122:0/75506774 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed5ffff640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.727+0000 7fed6491f640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 0x7fed60074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 -- 172.21.15.122:0/75506774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 msgr2=0x7fed60074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 0x7fed60074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 -- 172.21.15.122:0/75506774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 msgr2=0x7fed60073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 -- 172.21.15.122:0/75506774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed601a6880 con 0x7fed60102450 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5ffff640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5f7fe640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fed5400d950 tx=0x7fed5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5d7fa640 1 -- 172.21.15.122:0/75506774 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54017070 con 0x7fed60102450 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed601a6b70 con 0x7fed60102450 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5d7fa640 1 -- 172.21.15.122:0/75506774 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed54009c30 con 0x7fed60102450 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed5d7fa640 1 -- 172.21.15.122:0/75506774 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5400f6b0 con 0x7fed60102450 2024-04-02T13:41:15.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.728+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed601a70c0 con 0x7fed60102450 2024-04-02T13:41:15.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:15 smithi187 ceph-mon[36064]: pgmap v185: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 131 op/s 2024-04-02T13:41:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.730+0000 7fed5d7fa640 1 -- 172.21.15.122:0/75506774 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fed540109d0 con 0x7fed60102450 2024-04-02T13:41:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.730+0000 7fed68f6d740 1 Processor -- start 2024-04-02T13:41:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.730+0000 7fed68f6d740 1 -- start start 2024-04-02T13:41:15.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b088b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5d7fa640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 0x7fed480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b094090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa4afec7b0 con 0x55aa4b0890c0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aa4af8b570 con 0x55aa4b0887a0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed68f6d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aa4afe8760 con 0x55aa4b091ca0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5f7fe640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 0x7fed480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49776/0 (socket says 172.21.15.122:49776) 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 -- 172.21.15.122:0/679655159 learned_addr learned my addr 172.21.15.122:0/679655159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5d7fa640 1 -- 172.21.15.122:0/75506774 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed540d0850 con 0x7fed60102450 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed6491f640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b088b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed6491f640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b094090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 -- 172.21.15.122:0/679655159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 msgr2=0x55aa4b094090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b094090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 -- 172.21.15.122:0/679655159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 msgr2=0x55aa4b088b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b088b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 -- 172.21.15.122:0/679655159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4ad003f0 con 0x55aa4b0890c0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed6491f640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b094090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5ffff640 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed5000cc20 tx=0x7fed5000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad48ff6271cbf8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed5f7fe640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 0x7fed480a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fed54096fd0 tx=0x7fed54098040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed467fc640 1 -- 172.21.15.122:0/679655159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50005220 con 0x55aa4b0890c0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed467fc640 1 -- 172.21.15.122:0/679655159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50016070 con 0x55aa4b0890c0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.731+0000 7fed467fc640 1 -- 172.21.15.122:0/679655159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50010700 con 0x55aa4b0890c0 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- 172.21.15.122:0/679655159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 msgr2=0x55aa4b091760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed5000cc20 tx=0x7fed5000e720 comp rx=0 tx=0).stop 2024-04-02T13:41:15.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- 172.21.15.122:0/679655159 shutdown_connections 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b094090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b091760 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 --2- 172.21.15.122:0/679655159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b088b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- 172.21.15.122:0/679655159 >> 172.21.15.122:0/679655159 conn(0x55aa4b082ed0 msgr2=0x55aa4b0832b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- 172.21.15.122:0/679655159 shutdown_connections 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- 172.21.15.122:0/679655159 wait complete. 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.732+0000 7fed68f6d740 1 -- start start 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 Processor -- start 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b08ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aa4b094a90 con 0x55aa4b0890c0 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aa4afa7e80 con 0x55aa4b0887a0 2024-04-02T13:41:15.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aa4b094c10 con 0x55aa4b091ca0 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed68f6d740 10 client.0 _ll_register_callbacks cb 0x55aa4afe3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49786/0 (socket says 172.21.15.122:49786) 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 -- 172.21.15.122:0/2380430399 learned_addr learned my addr 172.21.15.122:0/2380430399 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed6491f640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b08ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed6491f640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 -- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 msgr2=0x55aa4b08ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b08ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 -- 172.21.15.122:0/2380430399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 msgr2=0x55aa4b091620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b091620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 -- 172.21.15.122:0/2380430399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4adca7d0 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec20 tx=0x7fed50010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2dd1086b35f5538 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.733+0000 7fed3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5002a060 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed5000c730 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed50016070 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed50031050 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 10 client.17469 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aa4b0a0a40 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50016070 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fed5000f3c0 con 0x55aa4b0890c0 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 10 client.17469 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 10 client.17469 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 20 client.17469 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b0a1ea0 0x55aa4b0a4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 4 client.17469 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa4aff9cf0 con 0x55aa4b0a1ea0 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.734+0000 7fed6491f640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b0a1ea0 0x55aa4b0a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.735+0000 7fed3f7fe640 10 client.17469 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.735+0000 7fed6491f640 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b0a1ea0 0x55aa4b0a4290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b30ace5dc8bc8604 server_cookie=a1d57628032d52c1 in_seq=0 out_seq=0 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.735+0000 7fed3f7fe640 1 -- 172.21.15.122:0/2380430399 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa4aff9cf0 con 0x55aa4b0a1ea0 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.735+0000 7fed3f7fe640 10 client.17469 handle_command_reply: tid=0 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 client.17469 shutdown 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 msgr2=0x55aa4b08e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fed5000ec20 tx=0x7fed50010b40 comp rx=0 tx=0).stop 2024-04-02T13:41:15.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 shutdown_connections 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b08ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b0a1ea0 0x55aa4b0a4290 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b08e9b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2380430399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b091620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed3f7fe640 0 client.17469 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 >> 172.21.15.122:0/2380430399 conn(0x55aa4b082ed0 msgr2=0x55aa4b084c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 shutdown_connections 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 1 -- 172.21.15.122:0/2380430399 wait complete. 2024-04-02T13:41:15.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.736+0000 7fed68f6d740 20 client.17469 trim_cache size 0 max 16384 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 Processor -- start 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 -- start start 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b09d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b091620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1140 con 0x55aa4b0890c0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b2d40 con 0x55aa4b0887a0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed68f6d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1810 con 0x55aa4b091ca0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49790/0 (socket says 172.21.15.122:49790) 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 -- 172.21.15.122:0/2857386515 learned_addr learned my addr 172.21.15.122:0/2857386515 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed6491f640 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b09d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 -- 172.21.15.122:0/2857386515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 msgr2=0x55aa4b091620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b091620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 -- 172.21.15.122:0/2857386515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 msgr2=0x55aa4b09d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b09d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.749+0000 7fed5ffff640 1 -- 172.21.15.122:0/2857386515 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa4b08f510 con 0x55aa4b0890c0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed5ffff640 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fed50010470 tx=0x7fed5000c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c0a71bd7350cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed45ffb640 1 -- 172.21.15.122:0/2857386515 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5002a040 con 0x55aa4b0890c0 2024-04-02T13:41:15.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed45ffb640 1 -- 172.21.15.122:0/2857386515 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50016070 con 0x55aa4b0890c0 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed45ffb640 1 -- 172.21.15.122:0/2857386515 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50004020 con 0x55aa4b0890c0 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 -- 172.21.15.122:0/2857386515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 msgr2=0x55aa4b0910e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fed50010470 tx=0x7fed5000c390 comp rx=0 tx=0).stop 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 -- 172.21.15.122:0/2857386515 shutdown_connections 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b091ca0 0x55aa4b091620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b0890c0 0x55aa4b0910e0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 --2- 172.21.15.122:0/2857386515 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0887a0 0x55aa4b09d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 -- 172.21.15.122:0/2857386515 >> 172.21.15.122:0/2857386515 conn(0x55aa4b082ed0 msgr2=0x55aa4b08e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 -- 172.21.15.122:0/2857386515 shutdown_connections 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.750+0000 7fed68f6d740 1 -- 172.21.15.122:0/2857386515 wait complete. 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 -- start start 2024-04-02T13:41:15.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 Processor -- start 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b0887a0 0x55aa4b099ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0890c0 0x55aa4b09a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b25b0 con 0x55aa4b091ca0 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed601b1ee0 con 0x55aa4b0890c0 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed601af120 con 0x55aa4b0887a0 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed68f6d740 10 client.0 _ll_register_callbacks cb 0x7fed50014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed6491f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed6491f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49796/0 (socket says 172.21.15.122:49796) 2024-04-02T13:41:15.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed6491f640 1 -- 172.21.15.122:0/3798224724 learned_addr learned my addr 172.21.15.122:0/3798224724 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed5ffff640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0890c0 0x55aa4b09a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.751+0000 7fed6491f640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b0887a0 0x55aa4b099ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 -- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b0887a0 msgr2=0x55aa4b099ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b0887a0 0x55aa4b099ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 -- 172.21.15.122:0/3798224724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0890c0 msgr2=0x55aa4b09a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0890c0 0x55aa4b09a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 -- 172.21.15.122:0/3798224724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa4b08c210 con 0x55aa4b091ca0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed6491f640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fed58005f30 tx=0x7fed58032460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92fd92fa3fb7516 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58039040 con 0x55aa4b091ca0 2024-04-02T13:41:15.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fed580329a0 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed50016070 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fed58058050 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 10 client.17475 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aa4b0929c0 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed50016070 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fed58051060 con 0x55aa4b091ca0 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 10 client.17475 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 10 client.17475 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 20 client.17475 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b110a80 0x55aa4b112e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 4 client.17475 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:15.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.752+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa4aff4040 con 0x55aa4b110a80 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed5ffff640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b110a80 0x55aa4b112e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed5ffff640 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b110a80 0x55aa4b112e70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=34ed7c19d327e71b server_cookie=4d130da3877df16b in_seq=0 out_seq=0 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed3f7fe640 10 client.17475 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed3f7fe640 1 -- 172.21.15.122:0/3798224724 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa4aff4040 con 0x55aa4b110a80 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed3f7fe640 10 client.17475 handle_command_reply: tid=0 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 client.17475 shutdown 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 msgr2=0x55aa4b08bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fed58005f30 tx=0x7fed58032460 comp rx=0 tx=0).stop 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 shutdown_connections 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55aa4b110a80 0x55aa4b112e70 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aa4b091ca0 0x55aa4b08bc50 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aa4b0890c0 0x55aa4b09a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed3f7fe640 0 client.17475 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 --2- 172.21.15.122:0/3798224724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aa4b0887a0 0x55aa4b099ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 >> 172.21.15.122:0/3798224724 conn(0x55aa4b082ed0 msgr2=0x55aa4b08d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 shutdown_connections 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 1 -- 172.21.15.122:0/3798224724 wait complete. 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.753+0000 7fed68f6d740 20 client.17475 trim_cache size 0 max 16384 2024-04-02T13:41:15.766 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 msgr2=0x7fed480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 0x7fed480a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fed54096fd0 tx=0x7fed54098040 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 msgr2=0x7fed60074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fed5400d950 tx=0x7fed5400de10 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 shutdown_connections 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fed480a3d20 0x7fed480a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed60107aa0 0x7fed60074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed60102450 0x7fed60074340 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 --2- 172.21.15.122:0/75506774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed60101a90 0x7fed60073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 >> 172.21.15.122:0/75506774 conn(0x7fed60078f00 msgr2=0x7fed60104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 shutdown_connections 2024-04-02T13:41:15.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.754+0000 7fed66382640 1 -- 172.21.15.122:0/75506774 wait complete. 2024-04-02T13:41:15.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:41:15.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:15 smithi188 ceph-mon[36117]: pgmap v185: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 910 KiB/s wr, 131 op/s 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 -- 172.21.15.122:0/4182820524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 msgr2=0x7f0d6010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 --2- 172.21.15.122:0/4182820524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d6010d840 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009cd0 tx=0x7f0d5802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 -- 172.21.15.122:0/4182820524 shutdown_connections 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 --2- 172.21.15.122:0/4182820524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d6010d840 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 --2- 172.21.15.122:0/4182820524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 0x7f0d60100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 --2- 172.21.15.122:0/4182820524 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 0x7f0d60106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 -- 172.21.15.122:0/4182820524 >> 172.21.15.122:0/4182820524 conn(0x7f0d60077b00 msgr2=0x7f0d60077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 -- 172.21.15.122:0/4182820524 shutdown_connections 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.870+0000 7f0d64d38640 1 -- 172.21.15.122:0/4182820524 wait complete. 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 Processor -- start 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 -- start start 2024-04-02T13:41:15.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 0x7f0d601982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 0x7f0d6019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60110d90 con 0x7f0d60101330 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60110c10 con 0x7f0d60106980 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d64d38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60110f10 con 0x7f0d60100970 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49816/0 (socket says 172.21.15.122:49816) 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d57fff640 1 -- 172.21.15.122:0/1403925698 learned_addr learned my addr 172.21.15.122:0/1403925698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 0x7f0d6019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.871+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 0x7f0d601982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 -- 172.21.15.122:0/1403925698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 msgr2=0x7f0d601982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 0x7f0d601982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 -- 172.21.15.122:0/1403925698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 msgr2=0x7f0d6019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 0x7f0d6019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 -- 172.21.15.122:0/1403925698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6019d2e0 con 0x7f0d60101330 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b4a0 tx=0x7f0d4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d577fe640 1 -- 172.21.15.122:0/1403925698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d48015020 con 0x7f0d60101330 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d577fe640 1 -- 172.21.15.122:0/1403925698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48011d10 con 0x7f0d60101330 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6019d5d0 con 0x7f0d60101330 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d577fe640 1 -- 172.21.15.122:0/1403925698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4800f2e0 con 0x7f0d60101330 2024-04-02T13:41:15.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.872+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a4e60 con 0x7f0d60101330 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.874+0000 7f0d577fe640 1 -- 172.21.15.122:0/1403925698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0d48018020 con 0x7f0d60101330 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.874+0000 7f0d679f3740 1 Processor -- start 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 -- start start 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54260c0 0x55e2a542e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5431090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e2a53897b0 con 0x55e2a54257a0 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5328570 con 0x55e2a54260c0 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d679f3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5385760 con 0x55e2a542eca0 2024-04-02T13:41:15.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49822/0 (socket says 172.21.15.122:49822) 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d577fe640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 0x7f0d4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 -- 172.21.15.122:0/4075652285 learned_addr learned my addr 172.21.15.122:0/4075652285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54260c0 0x55e2a542e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 0x7f0d4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 -- 172.21.15.122:0/4075652285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 msgr2=0x55e2a5431090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d577fe640 1 -- 172.21.15.122:0/1403925698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f0d480d02b0 con 0x7f0d60101330 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d57fff640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5431090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5431090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 -- 172.21.15.122:0/4075652285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54260c0 msgr2=0x55e2a542e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54260c0 0x55e2a542e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 -- 172.21.15.122:0/4075652285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a509d3f0 con 0x55e2a54257a0 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d57fff640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5431090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.875+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009a60 tx=0x7f0d58032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68790af7239b43ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.876+0000 7f0d54ff9640 1 -- 172.21.15.122:0/4075652285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58050070 con 0x55e2a54257a0 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.876+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 0x7f0d4c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d5804d000 tx=0x7f0d58048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.876+0000 7f0d54ff9640 1 -- 172.21.15.122:0/4075652285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d58003b70 con 0x55e2a54257a0 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.876+0000 7f0d54ff9640 1 -- 172.21.15.122:0/4075652285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d58039040 con 0x55e2a54257a0 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- 172.21.15.122:0/4075652285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 msgr2=0x55e2a5425b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009a60 tx=0x7f0d58032d10 comp rx=0 tx=0).stop 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- 172.21.15.122:0/4075652285 shutdown_connections 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5431090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54260c0 0x55e2a542e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- 172.21.15.122:0/4075652285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54257a0 0x55e2a5425b80 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- 172.21.15.122:0/4075652285 >> 172.21.15.122:0/4075652285 conn(0x55e2a541fed0 msgr2=0x55e2a54202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- 172.21.15.122:0/4075652285 shutdown_connections 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- 172.21.15.122:0/4075652285 wait complete. 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- start start 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 Processor -- start 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a542e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a542c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5431a90 con 0x55e2a54260c0 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5344e80 con 0x55e2a542eca0 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e2a5431c10 con 0x55e2a54257a0 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d679f3740 10 client.0 _ll_register_callbacks cb 0x55e2a5380aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49838/0 (socket says 172.21.15.122:49838) 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d57fff640 1 -- 172.21.15.122:0/1874030035 learned_addr learned my addr 172.21.15.122:0/1874030035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.877+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a542e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a542c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 -- 172.21.15.122:0/1874030035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 msgr2=0x55e2a542e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a542e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 -- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 msgr2=0x55e2a542c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a542c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 -- 172.21.15.122:0/1874030035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a51677d0 con 0x55e2a54260c0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a542c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d57fff640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0d480989a0 tx=0x7f0d4809bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6957e1e9ec03bff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4809c6d0 con 0x55e2a54260c0 2024-04-02T13:41:15.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d58003b70 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f0d48098640 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0d4809bfd0 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d679f3740 10 client.17487 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2a543da80 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d58003b70 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f0d4809b420 con 0x55e2a54260c0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d679f3740 10 client.17487 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.878+0000 7f0d679f3740 10 client.17487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d679f3740 20 client.17487 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a543eee0 0x55e2a54412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d679f3740 4 client.17487 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a5396cf0 con 0x55e2a543eee0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a543eee0 0x55e2a54412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a543eee0 0x55e2a54412f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d2ac5c5e991a7767 server_cookie=225ea70b5ed790bb in_seq=0 out_seq=0 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.879+0000 7f0d45ffb640 10 client.17487 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d45ffb640 1 -- 172.21.15.122:0/1874030035 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2a5396cf0 con 0x55e2a543eee0 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d45ffb640 10 client.17487 handle_command_reply: tid=0 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 client.17487 shutdown 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 msgr2=0x55e2a542ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0d480989a0 tx=0x7f0d4809bb90 comp rx=0 tx=0).stop 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 shutdown_connections 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a543eee0 0x55e2a54412f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a542c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d45ffb640 0 client.17487 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542ba80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 --2- 172.21.15.122:0/1874030035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a542e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 >> 172.21.15.122:0/1874030035 conn(0x55e2a541fed0 msgr2=0x55e2a5421c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 shutdown_connections 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.880+0000 7f0d679f3740 1 -- 172.21.15.122:0/1874030035 wait complete. 2024-04-02T13:41:15.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.881+0000 7f0d679f3740 20 client.17487 trim_cache size 0 max 16384 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.893+0000 7f0d679f3740 1 Processor -- start 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.893+0000 7f0d679f3740 1 -- start start 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a54412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a5446b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601aef20 con 0x55e2a54260c0 2024-04-02T13:41:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601b0b20 con 0x55e2a542eca0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d679f3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601af5f0 con 0x55e2a54257a0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49846/0 (socket says 172.21.15.122:49846) 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 -- 172.21.15.122:0/3385454742 learned_addr learned my addr 172.21.15.122:0/3385454742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 -- 172.21.15.122:0/3385454742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 msgr2=0x55e2a54412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a54412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 -- 172.21.15.122:0/3385454742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 msgr2=0x55e2a5446b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a5446b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a5446b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 -- 172.21.15.122:0/3385454742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a5425ef0 con 0x55e2a54260c0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a5446b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0d48098430 tx=0x7f0d48010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a7c359a776f2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d47fff640 1 -- 172.21.15.122:0/3385454742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480a0380 con 0x55e2a54260c0 2024-04-02T13:41:15.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d47fff640 1 -- 172.21.15.122:0/3385454742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48064070 con 0x55e2a54260c0 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.894+0000 7f0d47fff640 1 -- 172.21.15.122:0/3385454742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480b5280 con 0x55e2a54260c0 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- 172.21.15.122:0/3385454742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 msgr2=0x55e2a543a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0d48098430 tx=0x7f0d48010720 comp rx=0 tx=0).stop 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- 172.21.15.122:0/3385454742 shutdown_connections 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a542eca0 0x55e2a5446b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a543a980 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3385454742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a54257a0 0x55e2a54412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- 172.21.15.122:0/3385454742 >> 172.21.15.122:0/3385454742 conn(0x55e2a541fed0 msgr2=0x55e2a542a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- 172.21.15.122:0/3385454742 shutdown_connections 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- 172.21.15.122:0/3385454742 wait complete. 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- start start 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 Processor -- start 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54257a0 0x55e2a54373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5430020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601b0390 con 0x55e2a54260c0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601afcc0 con 0x55e2a54257a0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601acf00 con 0x55e2a542eca0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.895+0000 7f0d679f3740 10 client.0 _ll_register_callbacks cb 0x7f0d480982e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49850/0 (socket says 172.21.15.122:49850) 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 -- 172.21.15.122:0/3804338710 learned_addr learned my addr 172.21.15.122:0/3804338710 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54257a0 0x55e2a54373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d5ed9d640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5430020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 -- 172.21.15.122:0/3804338710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 msgr2=0x55e2a5430020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5430020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 -- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54257a0 msgr2=0x55e2a54373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:15.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54257a0 0x55e2a54373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 -- 172.21.15.122:0/3804338710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a5434e80 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d57fff640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0d4809bf60 tx=0x7f0d480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716700cb7d3019e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480852b0 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f0d4809b420 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d48064070 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0d4809b660 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d679f3740 10 client.17493 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e2a54a5440 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d48064070 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.896+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f0d480b5290 con 0x55e2a54260c0 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 10 client.17493 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 10 client.17493 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 20 client.17493 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:15.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a54a5870 0x55e2a54a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 4 client.17493 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a5391040 con 0x55e2a54a5870 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a54a5870 0x55e2a54a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d45ffb640 10 client.17493 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d5e59c640 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a54a5870 0x55e2a54a7c60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc7d3e855335d5bf server_cookie=f989c501a1a70c6 in_seq=0 out_seq=0 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d45ffb640 1 -- 172.21.15.122:0/3804338710 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a5391040 con 0x55e2a54a5870 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d45ffb640 10 client.17493 handle_command_reply: tid=0 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 client.17493 shutdown 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 msgr2=0x55e2a542f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f0d4809bf60 tx=0x7f0d480047d0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 shutdown_connections 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2a542eca0 0x55e2a5430020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e2a54260c0 0x55e2a542f8e0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55e2a54a5870 0x55e2a54a7c60 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 --2- 172.21.15.122:0/3804338710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e2a54257a0 0x55e2a54373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 >> 172.21.15.122:0/3804338710 conn(0x55e2a541fed0 msgr2=0x55e2a54211d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.897+0000 7f0d45ffb640 0 client.17493 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 shutdown_connections 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d679f3740 1 -- 172.21.15.122:0/3804338710 wait complete. 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d679f3740 20 client.17493 trim_cache size 0 max 16384 2024-04-02T13:41:15.908 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 msgr2=0x7f0d4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 0x7f0d4c0a61e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d5804d000 tx=0x7f0d58048000 comp rx=0 tx=0).stop 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 msgr2=0x7f0d60198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800b4a0 tx=0x7f0d4800b8a0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 shutdown_connections 2024-04-02T13:41:15.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d4c0a3d20 0x7f0d4c0a61e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d60106980 0x7f0d6019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d60101330 0x7f0d60198830 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 --2- 172.21.15.122:0/1403925698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d60100970 0x7f0d601982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 >> 172.21.15.122:0/1403925698 conn(0x7f0d60077b00 msgr2=0x7f0d600ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 shutdown_connections 2024-04-02T13:41:15.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:15.898+0000 7f0d64d38640 1 -- 172.21.15.122:0/1403925698 wait complete. 2024-04-02T13:41:15.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:41:15.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178156 2.70 MB/sec execute 156 sec latency 2476.363 ms 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.012+0000 7f366eac7640 1 -- 172.21.15.122:0/2911918614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 msgr2=0x7f366810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.012+0000 7f366eac7640 1 --2- 172.21.15.122:0/2911918614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f366810d840 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3660009cd0 tx=0x7f366002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- 172.21.15.122:0/2911918614 shutdown_connections 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- 172.21.15.122:0/2911918614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f366810d840 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- 172.21.15.122:0/2911918614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668100970 0x7f3668100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- 172.21.15.122:0/2911918614 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668106980 0x7f3668106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- 172.21.15.122:0/2911918614 >> 172.21.15.122:0/2911918614 conn(0x7f3668077b00 msgr2=0x7f3668077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- 172.21.15.122:0/2911918614 shutdown_connections 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- 172.21.15.122:0/2911918614 wait complete. 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 Processor -- start 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- start start 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668100970 0x7f36681982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668106980 0x7f366819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3668111e50 con 0x7f3668101330 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3668111cd0 con 0x7f3668100970 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366eac7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3668111fd0 con 0x7f3668106980 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f3667fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f3667fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49860/0 (socket says 172.21.15.122:49860) 2024-04-02T13:41:16.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f3667fff640 1 -- 172.21.15.122:0/2106963589 learned_addr learned my addr 172.21.15.122:0/2106963589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.013+0000 7f366c863640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668100970 0x7f36681982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f366d064640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668106980 0x7f366819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 -- 172.21.15.122:0/2106963589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668106980 msgr2=0x7f366819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668106980 0x7f366819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 -- 172.21.15.122:0/2106963589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668100970 msgr2=0x7f36681982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668100970 0x7f36681982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 -- 172.21.15.122:0/2106963589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366819d2c0 con 0x7f3668101330 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3667fff640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f365400d950 tx=0x7f365400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3665ffb640 1 -- 172.21.15.122:0/2106963589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654017070 con 0x7f3668101330 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3665ffb640 1 -- 172.21.15.122:0/2106963589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365400de80 con 0x7f3668101330 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366819d5b0 con 0x7f3668101330 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f3665ffb640 1 -- 172.21.15.122:0/2106963589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400f620 con 0x7f3668101330 2024-04-02T13:41:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.014+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681a4e40 con 0x7f3668101330 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.016+0000 7f3665ffb640 1 -- 172.21.15.122:0/2106963589 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3654009c30 con 0x7f3668101330 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.016+0000 7f3671781740 1 Processor -- start 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 -- start start 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170124d0 0x5565170128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3665ffb640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 0x7f36580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x5565170224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556516f767b0 con 0x556517012df0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556516f15570 con 0x5565170200b0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3671781740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556516f72760 con 0x5565170124d0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3667fff640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 0x7f36580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49864/0 (socket says 172.21.15.122:49864) 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 -- 172.21.15.122:0/3133482062 learned_addr learned my addr 172.21.15.122:0/3133482062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3665ffb640 1 -- 172.21.15.122:0/2106963589 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f36540d0840 con 0x7f3668101330 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366d064640 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170124d0 0x5565170128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 -- 172.21.15.122:0/3133482062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170124d0 msgr2=0x5565170128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366d064640 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x5565170224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170124d0 0x5565170128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 -- 172.21.15.122:0/3133482062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 msgr2=0x5565170224a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x5565170224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 -- 172.21.15.122:0/3133482062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556516c8a3f0 con 0x556517012df0 2024-04-02T13:41:16.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f3667fff640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 0x7f36580a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36540ce200 tx=0x7f3654000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.017+0000 7f366c863640 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f365c00ac90 tx=0x7f365c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf1d7365f0ae1a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3652ffd640 1 -- 172.21.15.122:0/3133482062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c01d070 con 0x556517012df0 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3652ffd640 1 -- 172.21.15.122:0/3133482062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c018b20 con 0x556517012df0 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3652ffd640 1 -- 172.21.15.122:0/3133482062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c010b40 con 0x556517012df0 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 -- 172.21.15.122:0/3133482062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 msgr2=0x55651701fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f365c00ac90 tx=0x7f365c00f790 comp rx=0 tx=0).stop 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 -- 172.21.15.122:0/3133482062 shutdown_connections 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x5565170224a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701fb70 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 --2- 172.21.15.122:0/3133482062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170124d0 0x5565170128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 -- 172.21.15.122:0/3133482062 >> 172.21.15.122:0/3133482062 conn(0x55651700ced0 msgr2=0x55651700d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 -- 172.21.15.122:0/3133482062 shutdown_connections 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.018+0000 7f3671781740 1 -- 172.21.15.122:0/3133482062 wait complete. 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 -- start start 2024-04-02T13:41:16.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 Processor -- start 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x55651701cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x55651701d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556517022e70 con 0x5565170124d0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556516f31e80 con 0x5565170200b0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556517022ff0 con 0x556517012df0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f3671781740 10 client.0 _ll_register_callbacks cb 0x556516f6daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49876/0 (socket says 172.21.15.122:49876) 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 -- 172.21.15.122:0/4257607997 learned_addr learned my addr 172.21.15.122:0/4257607997 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x55651701d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366c863640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x55651701cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 -- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 msgr2=0x55651701cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x55651701cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 -- 172.21.15.122:0/4257607997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 msgr2=0x55651701d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x55651701d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.019+0000 7f366d064640 1 -- 172.21.15.122:0/4257607997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556516d547d0 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f366d064640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3660005fe0 tx=0x7f3660032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7820ad29f2104f9b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660048060 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f3660003970 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365c018b20 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f366004f050 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 10 client.17505 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55651702ef40 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c018b20 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f3660041440 con 0x5565170124d0 2024-04-02T13:41:16.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 10 client.17505 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 10 client.17505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 20 client.17505 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517030400 0x556517032810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 4 client.17505 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556516f83cf0 con 0x556517030400 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.020+0000 7f366c863640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517030400 0x556517032810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.021+0000 7f3647fff640 10 client.17505 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.021+0000 7f366c863640 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517030400 0x556517032810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac230bf8b1445d17 server_cookie=15ffe1b5cc77b8d4 in_seq=0 out_seq=0 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.021+0000 7f3647fff640 1 -- 172.21.15.122:0/4257607997 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556516f83cf0 con 0x556517030400 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.021+0000 7f3647fff640 10 client.17505 handle_command_reply: tid=0 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.021+0000 7f3671781740 1 client.17505 shutdown 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 msgr2=0x55651701fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3660005fe0 tx=0x7f3660032440 comp rx=0 tx=0).stop 2024-04-02T13:41:16.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 shutdown_connections 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517030400 0x556517032810 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170200b0 0x55651701d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3647fff640 0 client.17505 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x55651701cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 --2- 172.21.15.122:0/4257607997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170124d0 0x55651701fae0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 >> 172.21.15.122:0/4257607997 conn(0x55651700ced0 msgr2=0x55651700ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 shutdown_connections 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 1 -- 172.21.15.122:0/4257607997 wait complete. 2024-04-02T13:41:16.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.022+0000 7f3671781740 20 client.17505 trim_cache size 0 max 16384 2024-04-02T13:41:16.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 Processor -- start 2024-04-02T13:41:16.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 -- start start 2024-04-02T13:41:16.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x5565170327f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170200b0 0x55651700bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36681acd40 con 0x556517012df0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ae940 con 0x5565170124d0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f3671781740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ad410 con 0x5565170200b0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49888/0 (socket says 172.21.15.122:49888) 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 -- 172.21.15.122:0/2236983294 learned_addr learned my addr 172.21.15.122:0/2236983294 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366d064640 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170200b0 0x55651700bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 -- 172.21.15.122:0/2236983294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170200b0 msgr2=0x55651700bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170200b0 0x55651700bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 -- 172.21.15.122:0/2236983294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 msgr2=0x5565170327f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x5565170327f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 -- 172.21.15.122:0/2236983294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651701d990 con 0x556517012df0 2024-04-02T13:41:16.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f366c863640 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f365c0107d0 tx=0x7f365c0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3244b9bdb68133 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.036+0000 7f36527fc640 1 -- 172.21.15.122:0/2236983294 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c01d040 con 0x556517012df0 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f36527fc640 1 -- 172.21.15.122:0/2236983294 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c02e070 con 0x556517012df0 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f36527fc640 1 -- 172.21.15.122:0/2236983294 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c005040 con 0x556517012df0 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- 172.21.15.122:0/2236983294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 msgr2=0x55651701c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f365c0107d0 tx=0x7f365c0106c0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- 172.21.15.122:0/2236983294 shutdown_connections 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565170200b0 0x55651700bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556517012df0 0x55651701c800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 --2- 172.21.15.122:0/2236983294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x5565170327f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- 172.21.15.122:0/2236983294 >> 172.21.15.122:0/2236983294 conn(0x55651700ced0 msgr2=0x55651700d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- 172.21.15.122:0/2236983294 shutdown_connections 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- 172.21.15.122:0/2236983294 wait complete. 2024-04-02T13:41:16.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 -- start start 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.037+0000 7f3671781740 1 Processor -- start 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x556517019210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x556517019750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ae1b0 con 0x5565170200b0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36681adae0 con 0x5565170124d0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36681aad20 con 0x556517012df0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3671781740 10 client.0 _ll_register_callbacks cb 0x7f3660037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49896/0 (socket says 172.21.15.122:49896) 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 -- 172.21.15.122:0/684058337 learned_addr learned my addr 172.21.15.122:0/684058337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x556517019210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366c863640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x556517019750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 -- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 msgr2=0x556517019750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x556517019750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 -- 172.21.15.122:0/684058337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 msgr2=0x556517019210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x556517019210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 -- 172.21.15.122:0/684058337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556517012c20 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f366d064640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f36600419b0 tx=0x7f3660000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7457c47e0aed0601 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3647fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.038+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660048040 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f3660041440 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365c02e070 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3660063050 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 10 client.17511 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556517028510 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c02e070 con 0x5565170200b0 2024-04-02T13:41:16.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f3660004200 con 0x5565170200b0 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 10 client.17511 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 10 client.17511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 20 client.17511 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517096950 0x556517098d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 4 client.17511 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556516f7e040 con 0x556517096950 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.039+0000 7f366c863640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517096950 0x556517098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3647fff640 10 client.17511 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f366c863640 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517096950 0x556517098d40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24fb94ba6dd516a5 server_cookie=2ba1ec83f8d69a36 in_seq=0 out_seq=0 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3647fff640 1 -- 172.21.15.122:0/684058337 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556516f7e040 con 0x556517096950 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3647fff640 10 client.17511 handle_command_reply: tid=0 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 client.17511 shutdown 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 msgr2=0x556517019c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f36600419b0 tx=0x7f3660000f40 comp rx=0 tx=0).stop 2024-04-02T13:41:16.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 shutdown_connections 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x556517096950 0x556517098d40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5565170200b0 0x556517019c90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3647fff640 0 client.17511 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556517012df0 0x556517019750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 --2- 172.21.15.122:0/684058337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5565170124d0 0x556517019210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 >> 172.21.15.122:0/684058337 conn(0x55651700ced0 msgr2=0x55651700eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 shutdown_connections 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 1 -- 172.21.15.122:0/684058337 wait complete. 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.040+0000 7f3671781740 20 client.17511 trim_cache size 0 max 16384 2024-04-02T13:41:16.050 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:16.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 msgr2=0x7f36580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 0x7f36580a61e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36540ce200 tx=0x7f3654000bc0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 msgr2=0x7f3668198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f365400d950 tx=0x7f365400dd50 comp rx=0 tx=0).stop 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 shutdown_connections 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36580a3d20 0x7f36580a61e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3668106980 0x7f366819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3668101330 0x7f3668198810 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 --2- 172.21.15.122:0/2106963589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3668100970 0x7f36681982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 >> 172.21.15.122:0/2106963589 conn(0x7f3668077b00 msgr2=0x7f3668103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 shutdown_connections 2024-04-02T13:41:16.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.041+0000 7f366eac7640 1 -- 172.21.15.122:0/2106963589 wait complete. 2024-04-02T13:41:16.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:41:16.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1792129976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c1013e0 msgr2=0x7fbd1c10d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1792129976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c10d8f0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009cd0 tx=0x7fbd1402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1792129976 shutdown_connections 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1792129976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c10d8f0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1792129976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 0x7fbd1c100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1792129976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c106a30 0x7fbd1c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1792129976 >> 172.21.15.122:0/1792129976 conn(0x7fbd1c077b60 msgr2=0x7fbd1c077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1792129976 shutdown_connections 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.156+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1792129976 wait complete. 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 Processor -- start 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 -- start start 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 0x7fbd1c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c110e40 con 0x7fbd1c106a30 2024-04-02T13:41:16.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c110cc0 con 0x7fbd1c1013e0 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd21ca5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c110fc0 con 0x7fbd1c100a20 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49916/0 (socket says 172.21.15.122:49916) 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 -- 172.21.15.122:0/1385402984 learned_addr learned my addr 172.21.15.122:0/1385402984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd13fff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 0x7fbd1c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 -- 172.21.15.122:0/1385402984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 msgr2=0x7fbd1c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 0x7fbd1c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 -- 172.21.15.122:0/1385402984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c1013e0 msgr2=0x7fbd1c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 -- 172.21.15.122:0/1385402984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1c19d3b0 con 0x7fbd1c106a30 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.157+0000 7fbd1bfff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbd140348f0 tx=0x7fbd14032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.158+0000 7fbd197fa640 1 -- 172.21.15.122:0/1385402984 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14041020 con 0x7fbd1c106a30 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.158+0000 7fbd197fa640 1 -- 172.21.15.122:0/1385402984 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd14034cd0 con 0x7fbd1c106a30 2024-04-02T13:41:16.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.158+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1c19d640 con 0x7fbd1c106a30 2024-04-02T13:41:16.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.158+0000 7fbd197fa640 1 -- 172.21.15.122:0/1385402984 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14039040 con 0x7fbd1c106a30 2024-04-02T13:41:16.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.158+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd1c1a4f30 con 0x7fbd1c106a30 2024-04-02T13:41:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd197fa640 1 -- 172.21.15.122:0/1385402984 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbd14044020 con 0x7fbd1c106a30 2024-04-02T13:41:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 Processor -- start 2024-04-02T13:41:16.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 -- start start 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c877a0 0x55b476c87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c880c0 0x55b476c90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b476beb7b0 con 0x55b476c90ca0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b476b8a570 con 0x55b476c880c0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd2496c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b476be7760 con 0x55b476c877a0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.160+0000 7fbd197fa640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 0x7fbd080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49922/0 (socket says 172.21.15.122:49922) 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 -- 172.21.15.122:0/2815302851 learned_addr learned my addr 172.21.15.122:0/2815302851 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c877a0 0x55b476c87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd1bfff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 0x7fbd080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd197fa640 1 -- 172.21.15.122:0/1385402984 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbd140f2870 con 0x7fbd1c106a30 2024-04-02T13:41:16.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c880c0 0x55b476c90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 -- 172.21.15.122:0/2815302851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c877a0 msgr2=0x55b476c87b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c877a0 0x55b476c87b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 -- 172.21.15.122:0/2815302851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c880c0 msgr2=0x55b476c90760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c880c0 0x55b476c90760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 -- 172.21.15.122:0/2815302851 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4768ff3f0 con 0x55b476c90ca0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd13fff640 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011730 tx=0x7fbd0400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a0941ec237152f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd1bfff640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 0x7fbd080a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd140be520 tx=0x7fbd140a9040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd11ffb640 1 -- 172.21.15.122:0/2815302851 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004020 con 0x55b476c90ca0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd11ffb640 1 -- 172.21.15.122:0/2815302851 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04005500 con 0x55b476c90ca0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.161+0000 7fbd11ffb640 1 -- 172.21.15.122:0/2815302851 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004bb0 con 0x55b476c90ca0 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- 172.21.15.122:0/2815302851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 msgr2=0x55b476c93090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011730 tx=0x7fbd0400a6a0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- 172.21.15.122:0/2815302851 shutdown_connections 2024-04-02T13:41:16.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c93090 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c880c0 0x55b476c90760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 --2- 172.21.15.122:0/2815302851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c877a0 0x55b476c87b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- 172.21.15.122:0/2815302851 >> 172.21.15.122:0/2815302851 conn(0x55b476c81ed0 msgr2=0x55b476c822b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- 172.21.15.122:0/2815302851 shutdown_connections 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- 172.21.15.122:0/2815302851 wait complete. 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 -- start start 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.162+0000 7fbd2496c740 1 Processor -- start 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c877a0 0x55b476c90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c8dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b476c93a90 con 0x55b476c90ca0 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b476ba6e80 con 0x55b476c877a0 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b476c93c10 con 0x55b476c880c0 2024-04-02T13:41:16.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 10 client.0 _ll_register_callbacks cb 0x55b476be2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49924/0 (socket says 172.21.15.122:49924) 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 -- 172.21.15.122:0/590139072 learned_addr learned my addr 172.21.15.122:0/590139072 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c8dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c877a0 0x55b476c90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 -- 172.21.15.122:0/590139072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 msgr2=0x55b476c8dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c8dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 -- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c877a0 msgr2=0x55b476c90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c877a0 0x55b476c90740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 -- 172.21.15.122:0/590139072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4769c97d0 con 0x55b476c90ca0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd13fff640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011700 tx=0x7fbd0401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce58a21e00979ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbcfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04022070 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbd04004bc0 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.163+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd04005500 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbd04033050 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 10 client.17523 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b476c9fb50 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04005500 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fbd0401e470 con 0x55b476c90ca0 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 10 client.17523 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 20 client.17523 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476ca0fb0 0x55b476ca33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 4 client.17523 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b476bf8cf0 con 0x55b476ca0fb0 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.164+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476ca0fb0 0x55b476ca33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.165+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476ca0fb0 0x55b476ca33a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72d06ed1d8f73cb8 server_cookie=b62de15aa3973bb8 in_seq=0 out_seq=0 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.165+0000 7fbcfaffd640 10 client.17523 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.165+0000 7fbcfaffd640 1 -- 172.21.15.122:0/590139072 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b476bf8cf0 con 0x55b476ca0fb0 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.165+0000 7fbcfaffd640 10 client.17523 handle_command_reply: tid=0 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.165+0000 7fbd2496c740 1 client.17523 shutdown 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 msgr2=0x55b476c8e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbd04011700 tx=0x7fbd0401a040 comp rx=0 tx=0).stop 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 shutdown_connections 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476ca0fb0 0x55b476ca33a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c90ca0 0x55b476c8e1c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c8dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 --2- 172.21.15.122:0/590139072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c877a0 0x55b476c90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbcfaffd640 0 client.17523 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 >> 172.21.15.122:0/590139072 conn(0x55b476c81ed0 msgr2=0x55b476c83c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 shutdown_connections 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 1 -- 172.21.15.122:0/590139072 wait complete. 2024-04-02T13:41:16.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.166+0000 7fbd2496c740 20 client.17523 trim_cache size 0 max 16384 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 Processor -- start 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 -- start start 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c90200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c90740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1aef10 con 0x55b476c877a0 2024-04-02T13:41:16.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b0b10 con 0x55b476c90ca0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd2496c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1af5e0 con 0x55b476c880c0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49938/0 (socket says 172.21.15.122:49938) 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 -- 172.21.15.122:0/273218486 learned_addr learned my addr 172.21.15.122:0/273218486 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 -- 172.21.15.122:0/273218486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 msgr2=0x55b476c90200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c90200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 -- 172.21.15.122:0/273218486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 msgr2=0x55b476c90740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c90740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.179+0000 7fbd13fff640 1 -- 172.21.15.122:0/273218486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b476c87ef0 con 0x55b476c877a0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd13fff640 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd04013fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd50e9a120f50329 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd117fa640 1 -- 172.21.15.122:0/273218486 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04022030 con 0x55b476c877a0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd117fa640 1 -- 172.21.15.122:0/273218486 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0400a570 con 0x55b476c877a0 2024-04-02T13:41:16.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd117fa640 1 -- 172.21.15.122:0/273218486 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0401eab0 con 0x55b476c877a0 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 -- 172.21.15.122:0/273218486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 msgr2=0x55b476c9c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd04013fb0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 -- 172.21.15.122:0/273218486 shutdown_connections 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c90740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c90200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 --2- 172.21.15.122:0/273218486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c9c980 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 -- 172.21.15.122:0/273218486 >> 172.21.15.122:0/273218486 conn(0x55b476c81ed0 msgr2=0x55b476c8c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 -- 172.21.15.122:0/273218486 shutdown_connections 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.180+0000 7fbd2496c740 1 -- 172.21.15.122:0/273218486 wait complete. 2024-04-02T13:41:16.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 -- start start 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 Processor -- start 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c8a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1b0380 con 0x55b476c877a0 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1afcb0 con 0x55b476c90ca0 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd1c1acef0 con 0x55b476c880c0 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd2496c740 10 client.0 _ll_register_callbacks cb 0x7fbd0400adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49948/0 (socket says 172.21.15.122:49948) 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 -- 172.21.15.122:0/3583040091 learned_addr learned my addr 172.21.15.122:0/3583040091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 -- 172.21.15.122:0/3583040091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 msgr2=0x55b476c993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 -- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 msgr2=0x55b476c8a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c8a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd13fff640 1 -- 172.21.15.122:0/3583040091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b476ca3710 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.181+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd13fff640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005590 tx=0x7fbd04005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4390045739db120c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04022030 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbd0401e470 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0400a570 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbd0404b050 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 10 client.17529 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b476d0b220 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0400a570 con 0x55b476c877a0 2024-04-02T13:41:16.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fbd04004ba0 con 0x55b476c877a0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 10 client.17529 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 20 client.17529 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476d0b8b0 0x55b476d0dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 4 client.17529 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.182+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b476bf3040 con 0x55b476d0b8b0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476d0b8b0 0x55b476d0dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbcfaffd640 10 client.17529 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd1b7fe640 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476d0b8b0 0x55b476d0dcd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcd4c48ab61c853 server_cookie=1484dbbc6ad950b0 in_seq=0 out_seq=0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbcfaffd640 1 -- 172.21.15.122:0/3583040091 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b476bf3040 con 0x55b476d0b8b0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbcfaffd640 10 client.17529 handle_command_reply: tid=0 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 client.17529 shutdown 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 msgr2=0x55b476c98e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fbd04005590 tx=0x7fbd04005630 comp rx=0 tx=0).stop 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 shutdown_connections 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b476d0b8b0 0x55b476d0dcd0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b476c90ca0 0x55b476c8a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbcfaffd640 0 client.17529 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b476c880c0 0x55b476c993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 --2- 172.21.15.122:0/3583040091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b476c877a0 0x55b476c98e90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 >> 172.21.15.122:0/3583040091 conn(0x55b476c81ed0 msgr2=0x55b476c83300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 shutdown_connections 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.183+0000 7fbd2496c740 1 -- 172.21.15.122:0/3583040091 wait complete. 2024-04-02T13:41:16.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd2496c740 20 client.17529 trim_cache size 0 max 16384 2024-04-02T13:41:16.195 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 msgr2=0x7fbd080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 0x7fbd080a61e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd140be520 tx=0x7fbd140a9040 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 msgr2=0x7fbd1c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbd140348f0 tx=0x7fbd14032ec0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 shutdown_connections 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbd080a3d20 0x7fbd080a61e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbd1c106a30 0x7fbd1c19cc80 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbd1c1013e0 0x7fbd1c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 --2- 172.21.15.122:0/1385402984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd1c100a20 0x7fbd1c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 >> 172.21.15.122:0/1385402984 conn(0x7fbd1c077b60 msgr2=0x7fbd1c06a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:16.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 shutdown_connections 2024-04-02T13:41:16.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:16.184+0000 7fbd21ca5640 1 -- 172.21.15.122:0/1385402984 wait complete. 2024-04-02T13:41:16.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:16 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3208542344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:16.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:16 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3208542344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:16.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:16 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3208542344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:16.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 178156 2.68 MB/sec execute 157 sec latency 3476.407 ms 2024-04-02T13:41:17.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:41:17.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:41:17.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.317+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/2255262973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 msgr2=0x7f8f2810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.317+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/2255262973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2810e590 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009ca0 tx=0x7f8f2002c180 comp rx=0 tx=0).stop 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/2255262973 shutdown_connections 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/2255262973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2810e590 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/2255262973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 0x7f8f2810bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/2255262973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 0x7f8f28103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/2255262973 >> 172.21.15.122:0/2255262973 conn(0x7f8f28077b60 msgr2=0x7f8f28077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/2255262973 shutdown_connections 2024-04-02T13:41:17.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/2255262973 wait complete. 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.318+0000 7f8f2c9a8640 1 Processor -- start 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 -- start start 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 0x7f8f28198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 0x7f8f281988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28111ef0 con 0x7f8f2810c160 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28111d70 con 0x7f8f281036b0 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2c9a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f28112070 con 0x7f8f28102d90 2024-04-02T13:41:17.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49960/0 (socket says 172.21.15.122:49960) 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 -- 172.21.15.122:0/78458280 learned_addr learned my addr 172.21.15.122:0/78458280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 0x7f8f281988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f2659c640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 0x7f8f28198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 -- 172.21.15.122:0/78458280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 msgr2=0x7f8f28198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 0x7f8f28198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 -- 172.21.15.122:0/78458280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 msgr2=0x7f8f281988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 0x7f8f281988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 -- 172.21.15.122:0/78458280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2819d360 con 0x7f8f2810c160 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.319+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8f20037040 tx=0x7f8f20035cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.320+0000 7f8f177fe640 1 -- 172.21.15.122:0/78458280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f20039040 con 0x7f8f2810c160 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.320+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2819d5f0 con 0x7f8f2810c160 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.320+0000 7f8f177fe640 1 -- 172.21.15.122:0/78458280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f20044070 con 0x7f8f2810c160 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.320+0000 7f8f177fe640 1 -- 172.21.15.122:0/78458280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200328d0 con 0x7f8f2810c160 2024-04-02T13:41:17.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.320+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f281a4ee0 con 0x7f8f2810c160 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.322+0000 7f8f177fe640 1 -- 172.21.15.122:0/78458280 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8f20004040 con 0x7f8f2810c160 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.322+0000 7f8f2f58f740 1 Processor -- start 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.322+0000 7f8f2f58f740 1 -- start start 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e7876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f177fe640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 0x7f8f100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 0x55fe9e792bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e6eb770 con 0x55fe9e787be0 2024-04-02T13:41:17.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e693960 con 0x55fe9e7907c0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e6e78c0 con 0x55fe9e7872c0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2659c640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 0x7f8f100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49962/0 (socket says 172.21.15.122:49962) 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 -- 172.21.15.122:0/3143188667 learned_addr learned my addr 172.21.15.122:0/3143188667 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f177fe640 1 -- 172.21.15.122:0/78458280 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8f200f2ed0 con 0x7f8f2810c160 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e7876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 0x55fe9e792bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 -- 172.21.15.122:0/3143188667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 msgr2=0x55fe9e7876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e7876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 -- 172.21.15.122:0/3143188667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 msgr2=0x55fe9e792bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 0x55fe9e792bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 -- 172.21.15.122:0/3143188667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9e4c77e0 con 0x55fe9e787be0 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 0x55fe9e792bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:17.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f2659c640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 0x7f8f100a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005d00 tx=0x7f8f1800b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009f90 tx=0x7f8f1c009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdd2a983abdcb99 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f14ff9640 1 -- 172.21.15.122:0/3143188667 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c004050 con 0x55fe9e787be0 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f14ff9640 1 -- 172.21.15.122:0/3143188667 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c005500 con 0x55fe9e787be0 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.323+0000 7f8f14ff9640 1 -- 172.21.15.122:0/3143188667 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c004db0 con 0x55fe9e787be0 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 -- 172.21.15.122:0/3143188667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 msgr2=0x55fe9e790280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009f90 tx=0x7f8f1c009960 comp rx=0 tx=0).stop 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 -- 172.21.15.122:0/3143188667 shutdown_connections 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7907c0 0x55fe9e792bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e790280 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/3143188667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e7876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 -- 172.21.15.122:0/3143188667 >> 172.21.15.122:0/3143188667 conn(0x55fe9e7819f0 msgr2=0x55fe9e781dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 -- 172.21.15.122:0/3143188667 shutdown_connections 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.324+0000 7f8f2f58f740 1 -- 172.21.15.122:0/3143188667 wait complete. 2024-04-02T13:41:17.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 -- start start 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 Processor -- start 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e787be0 0x55fe9e78dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e7935b0 con 0x55fe9e7907c0 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e6a6d20 con 0x55fe9e7872c0 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fe9e793730 con 0x55fe9e787be0 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2f58f740 10 client.0 _ll_register_callbacks cb 0x55fe9e673a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49968/0 (socket says 172.21.15.122:49968) 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 -- 172.21.15.122:0/818294648 learned_addr learned my addr 172.21.15.122:0/818294648 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 -- 172.21.15.122:0/818294648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e787be0 msgr2=0x55fe9e78dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e787be0 0x55fe9e78dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 -- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 msgr2=0x55fe9e78d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2659c640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 -- 172.21.15.122:0/818294648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9e527f70 con 0x55fe9e7907c0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f2659c640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8f200f0400 tx=0x7f8f200be610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b472ce87e8e21199 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.325+0000 7f8f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200c8720 con 0x55fe9e7907c0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8f200df110 con 0x55fe9e7907c0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c005500 con 0x55fe9e7907c0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8f200bebf0 con 0x55fe9e7907c0 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 10 client.17541 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fe9e3bf170 con 0x55fe9e7907c0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c005500 con 0x55fe9e7907c0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f8f2003c030 con 0x55fe9e7907c0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 10 client.17541 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 10 client.17541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 20 client.17541 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e7a0c70 0x55fe9e7a30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 4 client.17541 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe9e6f8cb0 con 0x55fe9e7a0c70 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.326+0000 7f8f2659c640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e7a0c70 0x55fe9e7a30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.327+0000 7f8f0dffb640 10 client.17541 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.327+0000 7f8f2659c640 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e7a0c70 0x55fe9e7a30f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeb868bf5eeb0644 server_cookie=ebdd55a2f0d0ee31 in_seq=0 out_seq=0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.327+0000 7f8f0dffb640 1 -- 172.21.15.122:0/818294648 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe9e6f8cb0 con 0x55fe9e7a0c70 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.327+0000 7f8f0dffb640 10 client.17541 handle_command_reply: tid=0 2024-04-02T13:41:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 client.17541 shutdown 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 msgr2=0x55fe9e78e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8f200f0400 tx=0x7f8f200be610 comp rx=0 tx=0).stop 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 shutdown_connections 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e7a0c70 0x55fe9e7a30f0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e78e1f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e787be0 0x55fe9e78dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f0dffb640 0 client.17541 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/818294648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 >> 172.21.15.122:0/818294648 conn(0x55fe9e7819f0 msgr2=0x55fe9e783740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 shutdown_connections 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 1 -- 172.21.15.122:0/818294648 wait complete. 2024-04-02T13:41:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.328+0000 7f8f2f58f740 20 client.17541 trim_cache size 0 max 16384 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 Processor -- start 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 -- start start 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7907c0 0x55fe9e7a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281aee10 con 0x55fe9e787be0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b0a10 con 0x55fe9e7872c0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.345+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281af4e0 con 0x55fe9e7907c0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49980/0 (socket says 172.21.15.122:49980) 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 -- 172.21.15.122:0/2951683412 learned_addr learned my addr 172.21.15.122:0/2951683412 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7907c0 0x55fe9e7a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f2659c640 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 -- 172.21.15.122:0/2951683412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7907c0 msgr2=0x55fe9e7a89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7907c0 0x55fe9e7a89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 -- 172.21.15.122:0/2951683412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 msgr2=0x55fe9e78fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 -- 172.21.15.122:0/2951683412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe9e7a3720 con 0x55fe9e787be0 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c01b000 tx=0x7f8f1c005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e891826db331b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f0ffff640 1 -- 172.21.15.122:0/2951683412 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0055c0 con 0x55fe9e787be0 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f0ffff640 1 -- 172.21.15.122:0/2951683412 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c003ad0 con 0x55fe9e787be0 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.346+0000 7f8f0ffff640 1 -- 172.21.15.122:0/2951683412 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c01e7d0 con 0x55fe9e787be0 2024-04-02T13:41:17.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- 172.21.15.122:0/2951683412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 msgr2=0x55fe9e79c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c01b000 tx=0x7f8f1c005220 comp rx=0 tx=0).stop 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- 172.21.15.122:0/2951683412 shutdown_connections 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7907c0 0x55fe9e7a89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e787be0 0x55fe9e79c4d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/2951683412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e7872c0 0x55fe9e78fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- 172.21.15.122:0/2951683412 >> 172.21.15.122:0/2951683412 conn(0x55fe9e7819f0 msgr2=0x55fe9e780ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- 172.21.15.122:0/2951683412 shutdown_connections 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- 172.21.15.122:0/2951683412 wait complete. 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- start start 2024-04-02T13:41:17.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 Processor -- start 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e798bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e787be0 0x55fe9e7959b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281b0280 con 0x55fe9e7907c0 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281afbb0 con 0x55fe9e787be0 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f281acdf0 con 0x55fe9e7872c0 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f2f58f740 10 client.0 _ll_register_callbacks cb 0x7f8f20004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.347+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49988/0 (socket says 172.21.15.122:49988) 2024-04-02T13:41:17.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 -- 172.21.15.122:0/446494435 learned_addr learned my addr 172.21.15.122:0/446494435 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2659c640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e798bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f25d9b640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e787be0 0x55fe9e7959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 -- 172.21.15.122:0/446494435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 msgr2=0x55fe9e798bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e798bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 -- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e787be0 msgr2=0x55fe9e7959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e787be0 0x55fe9e7959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 -- 172.21.15.122:0/446494435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe9e781820 con 0x55fe9e7907c0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f26d9d640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8f200049d0 tx=0x7f8f20004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5f5c5b5fd4d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f200be730 con 0x55fe9e7907c0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f8f200bee50 con 0x55fe9e7907c0 2024-04-02T13:41:17.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c003ad0 con 0x55fe9e7907c0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8f200c2fa0 con 0x55fe9e7907c0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 10 client.17547 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fe9e807d50 con 0x55fe9e7907c0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c003ad0 con 0x55fe9e7907c0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f8f20004bd0 con 0x55fe9e7907c0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 10 client.17547 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 10 client.17547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 20 client.17547 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e808360 0x55fe9e80a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 4 client.17547 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.348+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe9e6f3000 con 0x55fe9e808360 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.349+0000 7f8f2659c640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e808360 0x55fe9e80a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.349+0000 7f8f0dffb640 10 client.17547 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.349+0000 7f8f2659c640 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e808360 0x55fe9e80a780 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=861f05eaa5010a67 server_cookie=c1d6c6ffefed9bac in_seq=0 out_seq=0 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f0dffb640 1 -- 172.21.15.122:0/446494435 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe9e6f3000 con 0x55fe9e808360 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f0dffb640 10 client.17547 handle_command_reply: tid=0 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 client.17547 shutdown 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 msgr2=0x55fe9e795ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8f200049d0 tx=0x7f8f20004560 comp rx=0 tx=0).stop 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 shutdown_connections 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fe9e808360 0x55fe9e80a780 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fe9e7907c0 0x55fe9e795ef0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fe9e787be0 0x55fe9e7959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 --2- 172.21.15.122:0/446494435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fe9e7872c0 0x55fe9e798bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f0dffb640 0 client.17547 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 >> 172.21.15.122:0/446494435 conn(0x55fe9e7819f0 msgr2=0x55fe9e782d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 shutdown_connections 2024-04-02T13:41:17.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.350+0000 7f8f2f58f740 1 -- 172.21.15.122:0/446494435 wait complete. 2024-04-02T13:41:17.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2f58f740 20 client.17547 trim_cache size 0 max 16384 2024-04-02T13:41:17.366 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:17.366 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:41:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "20e3952f-c49c-4988-bff2-b3c06fd2bd01", 2024-04-02T13:41:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:41:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:17.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 msgr2=0x7f8f100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 0x7f8f100a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8f18005d00 tx=0x7f8f1800b410 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 msgr2=0x7f8f2819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f8f20037040 tx=0x7f8f20035cf0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 shutdown_connections 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f100a3d20 0x7f8f100a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f2810c160 0x7f8f2819cc30 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f281036b0 0x7f8f281988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 --2- 172.21.15.122:0/78458280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f28102d90 0x7f8f28198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 >> 172.21.15.122:0/78458280 conn(0x7f8f28077b60 msgr2=0x7f8f280fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 shutdown_connections 2024-04-02T13:41:17.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.351+0000 7f8f2c9a8640 1 -- 172.21.15.122:0/78458280 wait complete. 2024-04-02T13:41:17.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:17.370 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:17.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3027419784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b81010b0 msgr2=0x7fb1b8101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3027419784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b81010b0 0x7fb1b8101530 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009900 tx=0x7fb1a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3027419784 shutdown_connections 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3027419784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8101a70 0x7fb1b810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3027419784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b81010b0 0x7fb1b8101530 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3027419784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 0x7fb1b8105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3027419784 >> 172.21.15.122:0/3027419784 conn(0x7fb1b8076f20 msgr2=0x7fb1b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3027419784 shutdown_connections 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3027419784 wait complete. 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.483+0000 7fb1bff5f640 1 Processor -- start 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 -- start start 2024-04-02T13:41:17.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b81010b0 0x7fb1b80fc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 0x7fb1b80fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110e00 con 0x7fb1b8101a70 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110c80 con 0x7fb1b81010b0 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bff5f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110f80 con 0x7fb1b81059d0 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50010/0 (socket says 172.21.15.122:50010) 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/3716141838 learned_addr learned my addr 172.21.15.122:0/3716141838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b81010b0 0x7fb1b80fc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 0x7fb1b80fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/3716141838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 msgr2=0x7fb1b80fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 0x7fb1b80fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/3716141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b81010b0 msgr2=0x7fb1b80fc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b81010b0 0x7fb1b80fc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/3716141838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b81a6830 con 0x7fb1b8101a70 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb1a802efd0 tx=0x7fb1a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.484+0000 7fb1aeffd640 1 -- 172.21.15.122:0/3716141838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8041070 con 0x7fb1b8101a70 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.485+0000 7fb1aeffd640 1 -- 172.21.15.122:0/3716141838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a802ccc0 con 0x7fb1b8101a70 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.485+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b81a6ac0 con 0x7fb1b8101a70 2024-04-02T13:41:17.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.485+0000 7fb1aeffd640 1 -- 172.21.15.122:0/3716141838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8003d10 con 0x7fb1b8101a70 2024-04-02T13:41:17.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.485+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81a7000 con 0x7fb1b8101a70 2024-04-02T13:41:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.487+0000 7fb1aeffd640 1 -- 172.21.15.122:0/3716141838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb1a8044070 con 0x7fb1b8101a70 2024-04-02T13:41:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.487+0000 7fb1c2c20740 1 Processor -- start 2024-04-02T13:41:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 -- start start 2024-04-02T13:41:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1aeffd640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 0x7fb1a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 0x557f78e55890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f78db0830 con 0x557f78e55dd0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f78d58820 con 0x557f78e4c8d0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557f78dac7e0 con 0x557f78e4d1f0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1aeffd640 1 -- 172.21.15.122:0/3716141838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb1a80f23e0 con 0x7fb1b8101a70 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 0x7fb1a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1bdcfb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50020/0 (socket says 172.21.15.122:50020) 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1bdcfb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:58362/0 (socket says 172.21.15.122:58362) 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3839963018 learned_addr learned my addr 172.21.15.122:0/3839963018 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 0x557f78e55890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3839963018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 msgr2=0x557f78e55890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 0x557f78e55890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3839963018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 msgr2=0x557f78e4ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.488+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3839963018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f78b8e7d0 con 0x557f78e55dd0 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 0x557f78e55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 0x7fb1a40a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011d00 tx=0x7fb1b0011440 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011670 tx=0x7fb1b001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528d42ae25f5ab92 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1a3fff640 1 -- 172.21.15.122:0/3839963018 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0024070 con 0x557f78e55dd0 2024-04-02T13:41:17.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1a3fff640 1 -- 172.21.15.122:0/3839963018 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0011df0 con 0x557f78e55dd0 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.489+0000 7fb1a3fff640 1 -- 172.21.15.122:0/3839963018 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0029460 con 0x557f78e55dd0 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3839963018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 msgr2=0x557f78e581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011670 tx=0x7fb1b001d000 comp rx=0 tx=0).stop 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3839963018 shutdown_connections 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e55dd0 0x557f78e581c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4d1f0 0x557f78e55890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3839963018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e4ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3839963018 >> 172.21.15.122:0/3839963018 conn(0x557f78e47000 msgr2=0x557f78e473e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3839963018 shutdown_connections 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3839963018 wait complete. 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 -- start start 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.490+0000 7fb1c2c20740 1 Processor -- start 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 0x557f78e52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f78e58b60 con 0x557f78e4c8d0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f78d6bd30 con 0x557f78e4d1f0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557f78e58ce0 con 0x557f78e55dd0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1c2c20740 10 client.0 _ll_register_callbacks cb 0x557f78d6b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bdcfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 0x557f78e52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bdcfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50034/0 (socket says 172.21.15.122:50034) 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54850/0 (socket says 172.21.15.122:54850) 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bdcfb640 1 -- 172.21.15.122:0/3982353096 learned_addr learned my addr 172.21.15.122:0/3982353096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 msgr2=0x557f78e52a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 0x557f78e52a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3982353096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 msgr2=0x557f78e556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1be4fc640 1 -- 172.21.15.122:0/3982353096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f78ac43f0 con 0x557f78e55dd0 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 0x557f78e52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.491+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x557f78e48190 tx=0x7fb1b001f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b594bcc399b0353 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0036060 con 0x557f78e55dd0 2024-04-02T13:41:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb1b001f470 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b0011df0 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb1b003d050 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1c2c20740 10 client.26578 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f78a840f0 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.492+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0011df0 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fb1b00052a0 con 0x557f78e55dd0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 10 client.26578 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 10 client.26578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 20 client.26578 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78e65d30 0x557f78e68120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 4 client.26578 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f78dbdd70 con 0x557f78e65d30 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78e65d30 0x557f78e68120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1a17fa640 10 client.26578 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.493+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78e65d30 0x557f78e68120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd80e413fbfb9d7a server_cookie=a37474e1d77be289 in_seq=0 out_seq=0 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1a17fa640 1 -- 172.21.15.122:0/3982353096 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f78dbdd70 con 0x557f78e65d30 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1a17fa640 10 client.26578 handle_command_reply: tid=0 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 client.26578 shutdown 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 msgr2=0x557f78e52f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x557f78e48190 tx=0x7fb1b001f9f0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 shutdown_connections 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78e65d30 0x557f78e68120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e52f70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1a17fa640 0 client.26578 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4d1f0 0x557f78e52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/3982353096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4c8d0 0x557f78e556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 >> 172.21.15.122:0/3982353096 conn(0x557f78e47000 msgr2=0x557f78e48d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 shutdown_connections 2024-04-02T13:41:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.494+0000 7fb1c2c20740 1 -- 172.21.15.122:0/3982353096 wait complete. 2024-04-02T13:41:17.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.495+0000 7fb1c2c20740 20 client.26578 trim_cache size 0 max 16384 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 Processor -- start 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 -- start start 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e61a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e55160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b1090 con 0x557f78e4d1f0 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b2c90 con 0x557f78e4c8d0 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b1760 con 0x557f78e55dd0 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50042/0 (socket says 172.21.15.122:50042) 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/4049745160 learned_addr learned my addr 172.21.15.122:0/4049745160 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e61a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e55160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/4049745160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 msgr2=0x557f78e55160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e55160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/4049745160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 msgr2=0x557f78e61a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e61a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/4049745160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f78e46e30 con 0x557f78e4d1f0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e55160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.513+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80ba610 tx=0x7fb1a8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e2c4aa9c316a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1a37fe640 1 -- 172.21.15.122:0/4049745160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80d6070 con 0x557f78e4d1f0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1a37fe640 1 -- 172.21.15.122:0/4049745160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a80be1b0 con 0x557f78e4d1f0 2024-04-02T13:41:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1a37fe640 1 -- 172.21.15.122:0/4049745160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a804b040 con 0x557f78e4d1f0 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1c2c20740 1 -- 172.21.15.122:0/4049745160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 msgr2=0x557f78e48ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.514+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80ba610 tx=0x7fb1a8005d50 comp rx=0 tx=0).stop 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- 172.21.15.122:0/4049745160 shutdown_connections 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e55dd0 0x557f78e55160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e48ce0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/4049745160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e4c8d0 0x557f78e61a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- 172.21.15.122:0/4049745160 >> 172.21.15.122:0/4049745160 conn(0x557f78e47000 msgr2=0x557f78e51970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- 172.21.15.122:0/4049745160 shutdown_connections 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- 172.21.15.122:0/4049745160 wait complete. 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- start start 2024-04-02T13:41:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 Processor -- start 2024-04-02T13:41:17.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4c8d0 0x557f78e5e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e55dd0 0x557f78e59a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b2500 con 0x557f78e4d1f0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b1e30 con 0x557f78e55dd0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81af070 con 0x557f78e4c8d0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1c2c20740 10 client.0 _ll_register_callbacks cb 0x7fb1b0011710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1bd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50046/0 (socket says 172.21.15.122:50046) 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/2497008996 learned_addr learned my addr 172.21.15.122:0/2497008996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4c8d0 0x557f78e5e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.515+0000 7fb1be4fc640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e55dd0 0x557f78e59a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/2497008996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4c8d0 msgr2=0x557f78e5e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4c8d0 0x557f78e5e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e55dd0 msgr2=0x557f78e59a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e55dd0 0x557f78e59a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 -- 172.21.15.122:0/2497008996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f78e5a0a0 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1bd4fa640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80efda0 tx=0x7fb1a80bdb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb27b28a0a107b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80d6030 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fb1a80c2170 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a80be1b0 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb1a804b070 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1c2c20740 10 client.17565 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x557f78ecbe30 con 0x557f78e4d1f0 2024-04-02T13:41:17.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.516+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a80be1b0 con 0x557f78e4d1f0 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fb1a80c68e0 con 0x557f78e4d1f0 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 10 client.17565 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 10 client.17565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 20 client.17565 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78ecc4f0 0x557f78ece910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 4 client.17565 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557f78db80c0 con 0x557f78ecc4f0 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78ecc4f0 0x557f78ece910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1a17fa640 10 client.17565 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1bdcfb640 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78ecc4f0 0x557f78ece910 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62ceb744578a7340 server_cookie=e7d1ad684e5c17fc in_seq=0 out_seq=0 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1a17fa640 1 -- 172.21.15.122:0/2497008996 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557f78db80c0 con 0x557f78ecc4f0 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1a17fa640 10 client.17565 handle_command_reply: tid=0 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 1 client.17565 shutdown 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 msgr2=0x557f78e594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.517+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80efda0 tx=0x7fb1a80bdb10 comp rx=0 tx=0).stop 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 shutdown_connections 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557f78ecc4f0 0x557f78ece910 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f78e55dd0 0x557f78e59a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f78e4d1f0 0x557f78e594f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 --2- 172.21.15.122:0/2497008996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f78e4c8d0 0x557f78e5e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1a17fa640 0 client.17565 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 >> 172.21.15.122:0/2497008996 conn(0x557f78e47000 msgr2=0x557f78e46240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 shutdown_connections 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 1 -- 172.21.15.122:0/2497008996 wait complete. 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1c2c20740 20 client.17565 trim_cache size 0 max 16384 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "20e3952f-c49c-4988-bff2-b3c06fd2bd01": { 2024-04-02T13:41:17.529 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:41:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "20e3952f-c49c-4988-bff2-b3c06fd2bd01", 2024-04-02T13:41:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:41:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:41:17.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:41:17.530 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 msgr2=0x7fb1a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 0x7fb1a40a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0011d00 tx=0x7fb1b0011440 comp rx=0 tx=0).stop 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 msgr2=0x7fb1b80fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb1a802efd0 tx=0x7fb1a80042f0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 shutdown_connections 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a40a3da0 0x7fb1a40a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b81059d0 0x7fb1b80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101a70 0x7fb1b80fc8b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 --2- 172.21.15.122:0/3716141838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b81010b0 0x7fb1b80fc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:17.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 >> 172.21.15.122:0/3716141838 conn(0x7fb1b8076f20 msgr2=0x7fb1b81003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:17.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 shutdown_connections 2024-04-02T13:41:17.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:17.518+0000 7fb1bff5f640 1 -- 172.21.15.122:0/3716141838 wait complete. 2024-04-02T13:41:17.532 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:17 smithi122 ceph-mon[32114]: pgmap v186: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 952 KiB/s wr, 144 op/s 2024-04-02T13:41:17.533 INFO:tasks.cephfs.filesystem:scrub status for tag:20e3952f-c49c-4988-bff2-b3c06fd2bd01 - {'path': '/', 'tag': '20e3952f-c49c-4988-bff2-b3c06fd2bd01', 'options': 'recursive,force'} 2024-04-02T13:41:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:17 smithi187 ceph-mon[36064]: pgmap v186: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 952 KiB/s wr, 144 op/s 2024-04-02T13:41:17.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:17 smithi188 ceph-mon[36117]: pgmap v186: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 952 KiB/s wr, 144 op/s 2024-04-02T13:41:17.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 179240 2.69 MB/sec execute 158 sec latency 3900.150 ms 2024-04-02T13:41:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:41:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:18.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:41:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:18.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181296 2.76 MB/sec execute 159 sec latency 30.840 ms 2024-04-02T13:41:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:19 smithi187 ceph-mon[36064]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:19.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:19 smithi188 ceph-mon[36117]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:19 smithi122 ceph-mon[32114]: pgmap v187: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:19.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181579 2.75 MB/sec execute 160 sec latency 706.159 ms 2024-04-02T13:41:20.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:20 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:20.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:20 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:20.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:20 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:41:20.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181579 2.73 MB/sec execute 161 sec latency 1706.218 ms 2024-04-02T13:41:21.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:21 smithi187 ceph-mon[36064]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:21.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:21 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:21.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:21 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:21.740 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:21 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:41:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:21 smithi188 ceph-mon[36117]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:21 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:21 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:21 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:41:21.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:21 smithi122 ceph-mon[32114]: pgmap v188: 129 pgs: 129 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 577 KiB/s wr, 70 op/s 2024-04-02T13:41:21.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:21 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:41:21.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:21 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:41:21.832 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:21 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:41:21.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181579 2.71 MB/sec execute 162 sec latency 2706.294 ms 2024-04-02T13:41:22.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181579 2.70 MB/sec execute 163 sec latency 3706.397 ms 2024-04-02T13:41:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:23 smithi187 ceph-mon[36064]: pgmap v189: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:41:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:23 smithi188 ceph-mon[36117]: pgmap v189: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:41:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:23 smithi122 ceph-mon[32114]: pgmap v189: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 123 op/s 2024-04-02T13:41:23.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181635 2.68 MB/sec execute 164 sec latency 4154.720 ms 2024-04-02T13:41:24.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181635 2.67 MB/sec execute 165 sec latency 1505.057 ms 2024-04-02T13:41:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:25 smithi187 ceph-mon[36064]: pgmap v190: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 626 KiB/s wr, 66 op/s 2024-04-02T13:41:25.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:25 smithi188 ceph-mon[36117]: pgmap v190: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 626 KiB/s wr, 66 op/s 2024-04-02T13:41:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:25 smithi122 ceph-mon[32114]: pgmap v190: 129 pgs: 129 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 626 KiB/s wr, 66 op/s 2024-04-02T13:41:25.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181635 2.65 MB/sec execute 166 sec latency 2505.165 ms 2024-04-02T13:41:26.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181635 2.63 MB/sec execute 167 sec latency 3505.278 ms 2024-04-02T13:41:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: pgmap v191: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 898 KiB/s wr, 83 op/s 2024-04-02T13:41:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: pgmap v191: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 898 KiB/s wr, 83 op/s 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: pgmap v191: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 898 KiB/s wr, 83 op/s 2024-04-02T13:41:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:27.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181635 2.62 MB/sec execute 168 sec latency 4505.323 ms 2024-04-02T13:41:28.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181703 2.60 MB/sec execute 169 sec latency 4955.065 ms 2024-04-02T13:41:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:29 smithi187 ceph-mon[36064]: pgmap v192: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:29.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:29 smithi188 ceph-mon[36117]: pgmap v192: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:29 smithi122 ceph-mon[32114]: pgmap v192: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:29.958 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181703 2.59 MB/sec execute 170 sec latency 1483.846 ms 2024-04-02T13:41:30.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181703 2.57 MB/sec execute 171 sec latency 2483.967 ms 2024-04-02T13:41:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:31 smithi187 ceph-mon[36064]: pgmap v193: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:31 smithi188 ceph-mon[36117]: pgmap v193: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:31 smithi122 ceph-mon[32114]: pgmap v193: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 856 KiB/s wr, 70 op/s 2024-04-02T13:41:31.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181703 2.56 MB/sec execute 172 sec latency 3484.065 ms 2024-04-02T13:41:32.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 181703 2.54 MB/sec execute 173 sec latency 4484.155 ms 2024-04-02T13:41:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:33 smithi187 ceph-mon[36064]: pgmap v194: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T13:41:33.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:33 smithi188 ceph-mon[36117]: pgmap v194: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T13:41:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:33 smithi122 ceph-mon[32114]: pgmap v194: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-04-02T13:41:33.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 182617 2.56 MB/sec execute 174 sec latency 4930.805 ms 2024-04-02T13:41:34.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 184763 2.62 MB/sec execute 175 sec latency 13.056 ms 2024-04-02T13:41:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:35 smithi187 ceph-mon[36064]: pgmap v195: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 572 KiB/s wr, 41 op/s 2024-04-02T13:41:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:35 smithi188 ceph-mon[36117]: pgmap v195: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 572 KiB/s wr, 41 op/s 2024-04-02T13:41:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:35 smithi122 ceph-mon[32114]: pgmap v195: 129 pgs: 129 active+clean; 123 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 572 KiB/s wr, 41 op/s 2024-04-02T13:41:35.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 185986 2.63 MB/sec execute 176 sec latency 11.619 ms 2024-04-02T13:41:36.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188200 2.70 MB/sec execute 177 sec latency 16.468 ms 2024-04-02T13:41:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:37 smithi187 ceph-mon[36064]: pgmap v196: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-04-02T13:41:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:37 smithi188 ceph-mon[36117]: pgmap v196: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-04-02T13:41:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:37 smithi122 ceph-mon[32114]: pgmap v196: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.1 MiB/s wr, 105 op/s 2024-04-02T13:41:37.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188728 2.69 MB/sec execute 178 sec latency 380.565 ms 2024-04-02T13:41:38.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188796 2.67 MB/sec execute 179 sec latency 823.712 ms 2024-04-02T13:41:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:39 smithi187 ceph-mon[36064]: pgmap v197: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:39 smithi188 ceph-mon[36117]: pgmap v197: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:39 smithi122 ceph-mon[32114]: pgmap v197: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:39.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188796 2.66 MB/sec execute 180 sec latency 1501.164 ms 2024-04-02T13:41:40.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188796 2.64 MB/sec execute 181 sec latency 2501.283 ms 2024-04-02T13:41:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:41 smithi187 ceph-mon[36064]: pgmap v198: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:41 smithi188 ceph-mon[36117]: pgmap v198: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:41 smithi122 ceph-mon[32114]: pgmap v198: 129 pgs: 129 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 890 KiB/s wr, 88 op/s 2024-04-02T13:41:41.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188796 2.63 MB/sec execute 182 sec latency 3501.403 ms 2024-04-02T13:41:42.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 188796 2.61 MB/sec execute 183 sec latency 4501.499 ms 2024-04-02T13:41:43.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:43 smithi187 ceph-mon[36064]: pgmap v199: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-04-02T13:41:43.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:43 smithi188 ceph-mon[36117]: pgmap v199: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-04-02T13:41:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:43 smithi122 ceph-mon[32114]: pgmap v199: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-04-02T13:41:43.959 INFO:tasks.workunit.client.0.smithi122.stdout: 1 189710 2.63 MB/sec execute 184 sec latency 4952.312 ms 2024-04-02T13:41:44.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 191875 2.68 MB/sec execute 185 sec latency 13.961 ms 2024-04-02T13:41:45.622 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:45 smithi122 ceph-mon[32114]: pgmap v200: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 827 KiB/s wr, 107 op/s 2024-04-02T13:41:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:45 smithi187 ceph-mon[36064]: pgmap v200: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 827 KiB/s wr, 107 op/s 2024-04-02T13:41:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:45 smithi188 ceph-mon[36117]: pgmap v200: 129 pgs: 129 active+clean; 126 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 827 KiB/s wr, 107 op/s 2024-04-02T13:41:45.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 192213 2.67 MB/sec execute 186 sec latency 625.457 ms 2024-04-02T13:41:46.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 192213 2.66 MB/sec execute 187 sec latency 1625.513 ms 2024-04-02T13:41:47.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:47.535 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:47.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 -- 172.21.15.122:0/165163647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 msgr2=0x7f04c4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 --2- 172.21.15.122:0/165163647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c4101160 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009ca0 tx=0x7f04b802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 -- 172.21.15.122:0/165163647 shutdown_connections 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 --2- 172.21.15.122:0/165163647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 0x7f04c410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 --2- 172.21.15.122:0/165163647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c4101160 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 --2- 172.21.15.122:0/165163647 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 0x7f04c4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 -- 172.21.15.122:0/165163647 >> 172.21.15.122:0/165163647 conn(0x7f04c40779e0 msgr2=0x7f04c4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 -- 172.21.15.122:0/165163647 shutdown_connections 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 -- 172.21.15.122:0/165163647 wait complete. 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.648+0000 7f04c968f640 1 Processor -- start 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 -- start start 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 0x7f04c419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 0x7f04c419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4110d80 con 0x7f04c4100ce0 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4110c00 con 0x7f04c4106a30 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c968f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04c4110f00 con 0x7f04c41016a0 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44434/0 (socket says 172.21.15.122:44434) 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 -- 172.21.15.122:0/4189174546 learned_addr learned my addr 172.21.15.122:0/4189174546 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c27fc640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 0x7f04c419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c37fe640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 0x7f04c419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 -- 172.21.15.122:0/4189174546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 msgr2=0x7f04c419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 0x7f04c419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 -- 172.21.15.122:0/4189174546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 msgr2=0x7f04c419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 0x7f04c419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 -- 172.21.15.122:0/4189174546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c419f4a0 con 0x7f04c4100ce0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.649+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00d950 tx=0x7f04bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.650+0000 7f04abfff640 1 -- 172.21.15.122:0/4189174546 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc017070 con 0x7f04c4100ce0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.650+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c419f790 con 0x7f04c4100ce0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.650+0000 7f04abfff640 1 -- 172.21.15.122:0/4189174546 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc009c30 con 0x7f04c4100ce0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.650+0000 7f04abfff640 1 -- 172.21.15.122:0/4189174546 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00f6b0 con 0x7f04c4100ce0 2024-04-02T13:41:47.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.650+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c41a7020 con 0x7f04c4100ce0 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04abfff640 1 -- 172.21.15.122:0/4189174546 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f04bc0109d0 con 0x7f04c4100ce0 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 Processor -- start 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 -- start start 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 0x55fd29be6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fd29b4a830 con 0x55fd29befdd0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fd29af2820 con 0x55fd29be68d0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.652+0000 7f04cc277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fd29b467e0 con 0x55fd29be71f0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04abfff640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44442/0 (socket says 172.21.15.122:44442) 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 -- 172.21.15.122:0/1217488990 learned_addr learned my addr 172.21.15.122:0/1217488990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 0x55fd29be6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c37fe640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04abfff640 1 -- 172.21.15.122:0/4189174546 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f04bc0d0770 con 0x7f04c4100ce0 2024-04-02T13:41:47.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 -- 172.21.15.122:0/1217488990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 msgr2=0x55fd29bef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 -- 172.21.15.122:0/1217488990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 msgr2=0x55fd29be6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 0x55fd29be6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 -- 172.21.15.122:0/1217488990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd299287d0 con 0x55fd29befdd0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 0x55fd29be6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f04b802c700 tx=0x7f04b80099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0eddca03caa5883 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04c37fe640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04b4012910 tx=0x7f04b400b2e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04a97fa640 1 -- 172.21.15.122:0/1217488990 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8043070 con 0x55fd29befdd0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04a97fa640 1 -- 172.21.15.122:0/1217488990 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8032c90 con 0x55fd29befdd0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.653+0000 7f04a97fa640 1 -- 172.21.15.122:0/1217488990 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b8039040 con 0x55fd29befdd0 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- 172.21.15.122:0/1217488990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 msgr2=0x55fd29bf21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f04b802c700 tx=0x7f04b80099f0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- 172.21.15.122:0/1217488990 shutdown_connections 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29bf21c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 --2- 172.21.15.122:0/1217488990 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be68d0 0x55fd29be6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- 172.21.15.122:0/1217488990 >> 172.21.15.122:0/1217488990 conn(0x55fd29be1000 msgr2=0x55fd29be13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- 172.21.15.122:0/1217488990 shutdown_connections 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- 172.21.15.122:0/1217488990 wait complete. 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 -- start start 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.654+0000 7f04cc277740 1 Processor -- start 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 0x55fd29beced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fd29bf2b60 con 0x55fd29be68d0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fd29b05d30 con 0x55fd29be71f0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fd29bf2ce0 con 0x55fd29befdd0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 10 client.0 _ll_register_callbacks cb 0x55fd29b05720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44446/0 (socket says 172.21.15.122:44446) 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 -- 172.21.15.122:0/1838370630 learned_addr learned my addr 172.21.15.122:0/1838370630 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 0x55fd29beced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 -- 172.21.15.122:0/1838370630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 msgr2=0x55fd29beced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 0x55fd29beced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 -- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 msgr2=0x55fd29bec990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bec990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 -- 172.21.15.122:0/1838370630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd2985e3f0 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 0x55fd29beced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009610 tx=0x7f04bc0073b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910fe2428f61cf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc020040 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f04bc0d0310 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.655+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04b8032c90 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f04bc0cdf50 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 10 client.17577 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd2981e0f0 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04b8032c90 con 0x55fd29be68d0 2024-04-02T13:41:47.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f04bc0d0b60 con 0x55fd29be68d0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 10 client.17577 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 20 client.17577 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 4 client.17577 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd29b57d70 con 0x55fd29bffcc0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.656+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04a6ff5640 10 client.17577 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c02140 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ccae96b483d0d1 server_cookie=d9b1fcaf01c40760 in_seq=0 out_seq=0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1838370630 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fd29b57d70 con 0x55fd29bffcc0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04a6ff5640 10 client.17577 handle_command_reply: tid=0 2024-04-02T13:41:47.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04cc277740 1 client.17577 shutdown 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 msgr2=0x55fd29bef600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009610 tx=0x7f04bc0073b0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.657+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 shutdown_connections 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c02140 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29befdd0 0x55fd29beced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bec990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04a6ff5640 0 client.17577 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 --2- 172.21.15.122:0/1838370630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bef600 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 >> 172.21.15.122:0/1838370630 conn(0x55fd29be1000 msgr2=0x55fd29be2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 shutdown_connections 2024-04-02T13:41:47.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 1 -- 172.21.15.122:0/1838370630 wait complete. 2024-04-02T13:41:47.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.658+0000 7f04cc277740 20 client.17577 trim_cache size 0 max 16384 2024-04-02T13:41:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.675+0000 7f04cc277740 1 Processor -- start 2024-04-02T13:41:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.675+0000 7f04cc277740 1 -- start start 2024-04-02T13:41:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29befdd0 0x55fd29bef600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b1060 con 0x55fd29be68d0 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b2c60 con 0x55fd29befdd0 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04cc277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b1730 con 0x55fd29be71f0 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44462/0 (socket says 172.21.15.122:44462) 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 -- 172.21.15.122:0/3567838546 learned_addr learned my addr 172.21.15.122:0/3567838546 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c37fe640 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29befdd0 0x55fd29bef600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 -- 172.21.15.122:0/3567838546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 msgr2=0x55fd29bef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 -- 172.21.15.122:0/3567838546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29befdd0 msgr2=0x55fd29bef600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29befdd0 0x55fd29bef600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 -- 172.21.15.122:0/3567838546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd29bdcc50 con 0x55fd29be68d0 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f04bc09c360 tx=0x7f04bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf3538f221db3e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04a8ff9640 1 -- 172.21.15.122:0/3567838546 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0cdf50 con 0x55fd29be68d0 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04a8ff9640 1 -- 172.21.15.122:0/3567838546 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc0ba070 con 0x55fd29be68d0 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.676+0000 7f04a8ff9640 1 -- 172.21.15.122:0/3567838546 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc0d0570 con 0x55fd29be68d0 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- 172.21.15.122:0/3567838546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 msgr2=0x55fd29bfba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f04bc09c360 tx=0x7f04bc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- 172.21.15.122:0/3567838546 shutdown_connections 2024-04-02T13:41:47.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29befdd0 0x55fd29bef600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be71f0 0x55fd29bef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- 172.21.15.122:0/3567838546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29be68d0 0x55fd29bfba50 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- 172.21.15.122:0/3567838546 >> 172.21.15.122:0/3567838546 conn(0x55fd29be1000 msgr2=0x55fd29beb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- 172.21.15.122:0/3567838546 shutdown_connections 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- 172.21.15.122:0/3567838546 wait complete. 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- start start 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 Processor -- start 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 0x55fd29bf7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bf84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b24d0 con 0x55fd29befdd0 2024-04-02T13:41:47.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41b1e00 con 0x55fd29be71f0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.677+0000 7f04cc277740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04c41af040 con 0x55fd29be68d0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04cc277740 10 client.0 _ll_register_callbacks cb 0x7f04bc09bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44468/0 (socket says 172.21.15.122:44468) 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 -- 172.21.15.122:0/1081738 learned_addr learned my addr 172.21.15.122:0/1081738 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c27fc640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bf84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 0x55fd29bf7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 -- 172.21.15.122:0/1081738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 msgr2=0x55fd29bf7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 0x55fd29bf7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 -- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 msgr2=0x55fd29bf84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bf84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 -- 172.21.15.122:0/1081738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd29be9120 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 0x55fd29bf7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04c37fe640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f04b40040e0 tx=0x7f04b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2563b6c5d55a8f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b40035f0 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f04b4003790 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc0ba070 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f04b4003990 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04cc277740 10 client.17583 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.678+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55fd29c65a80 con 0x55fd29befdd0 2024-04-02T13:41:47.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc0ba070 con 0x55fd29befdd0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f04b4028070 con 0x55fd29befdd0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 10 client.17583 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 20 client.17583 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 4 client.17583 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fd29b520c0 con 0x55fd29bffcc0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04a6ff5640 10 client.17583 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.679+0000 7f04c2ffd640 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c683b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6d99d22b68a405 server_cookie=a6d20486b108daaf in_seq=0 out_seq=0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04a6ff5640 1 -- 172.21.15.122:0/1081738 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fd29b520c0 con 0x55fd29bffcc0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04a6ff5640 10 client.17583 handle_command_reply: tid=0 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 client.17583 shutdown 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 msgr2=0x55fd29be8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f04b40040e0 tx=0x7f04b40046c0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 shutdown_connections 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fd29bffcc0 0x55fd29c683b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fd29befdd0 0x55fd29be8be0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04a6ff5640 0 client.17583 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fd29be71f0 0x55fd29bf84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 --2- 172.21.15.122:0/1081738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fd29be68d0 0x55fd29bf7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 >> 172.21.15.122:0/1081738 conn(0x55fd29be1000 msgr2=0x55fd29be7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 shutdown_connections 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 1 -- 172.21.15.122:0/1081738 wait complete. 2024-04-02T13:41:47.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04cc277740 20 client.17583 trim_cache size 0 max 16384 2024-04-02T13:41:47.694 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:47.694 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:41:47.694 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:41:47.694 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:47.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 msgr2=0x7f04ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f04b4012910 tx=0x7f04b400b2e0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 msgr2=0x7f04c419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00d950 tx=0x7f04bc00de10 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 shutdown_connections 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04ac0a3d20 0x7f04ac0a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04c4106a30 0x7f04c419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04c41016a0 0x7f04c419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 --2- 172.21.15.122:0/4189174546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04c4100ce0 0x7f04c419a4b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 >> 172.21.15.122:0/4189174546 conn(0x7f04c40779e0 msgr2=0x7f04c406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.680+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 shutdown_connections 2024-04-02T13:41:47.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.681+0000 7f04c968f640 1 -- 172.21.15.122:0/4189174546 wait complete. 2024-04-02T13:41:47.699 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:47 smithi122 ceph-mon[32114]: pgmap v201: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T13:41:47.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:41:47.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:41:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:47 smithi187 ceph-mon[36064]: pgmap v201: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T13:41:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:47 smithi188 ceph-mon[36117]: pgmap v201: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-04-02T13:41:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.799+0000 7f822afe2640 1 -- 172.21.15.122:0/3077002401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8224105460 msgr2=0x7f822410d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- 172.21.15.122:0/3077002401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8224105460 0x7f822410d030 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f8214009900 tx=0x7f821402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- 172.21.15.122:0/3077002401 shutdown_connections 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- 172.21.15.122:0/3077002401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8224105460 0x7f822410d030 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- 172.21.15.122:0/3077002401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8224100d90 0x7f8224104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- 172.21.15.122:0/3077002401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82241003e0 0x7f82241007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- 172.21.15.122:0/3077002401 >> 172.21.15.122:0/3077002401 conn(0x7f8224076f90 msgr2=0x7f82240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- 172.21.15.122:0/3077002401 shutdown_connections 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- 172.21.15.122:0/3077002401 wait complete. 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 Processor -- start 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- start start 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 0x7f8224196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822419aa90 0x7f822419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f822410fcf0 con 0x7f82241003e0 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f822410fb70 con 0x7f8224105460 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f822afe2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f822410fe70 con 0x7f822419aa90 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f8228d7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f8228d7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44494/0 (socket says 172.21.15.122:44494) 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.800+0000 7f8228d7e640 1 -- 172.21.15.122:0/3491480376 learned_addr learned my addr 172.21.15.122:0/3491480376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8223fff640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 0x7f8224196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f822957f640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822419aa90 0x7f822419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 -- 172.21.15.122:0/3491480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822419aa90 msgr2=0x7f822419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822419aa90 0x7f822419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 -- 172.21.15.122:0/3491480376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 msgr2=0x7f8224196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 0x7f8224196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 -- 172.21.15.122:0/3491480376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822419b580 con 0x7f82241003e0 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8223fff640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 0x7f8224196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8228d7e640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f821c00de60 tx=0x7f821c00b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00e860 con 0x7f82241003e0 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822419b870 con 0x7f82241003e0 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f821c016430 con 0x7f82241003e0 2024-04-02T13:41:47.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c015640 con 0x7f82241003e0 2024-04-02T13:41:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.801+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241a30b0 con 0x7f82241003e0 2024-04-02T13:41:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.802+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224068d40 con 0x7f82241003e0 2024-04-02T13:41:47.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.803+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f821c0165e0 con 0x7f82241003e0 2024-04-02T13:41:47.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.804+0000 7f8221ffb640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 0x7f820c0a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:47.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.804+0000 7f8223fff640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 0x7f820c0a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:47.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.804+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f821c0d1150 con 0x7f82241003e0 2024-04-02T13:41:47.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.804+0000 7f8223fff640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 0x7f820c0a83f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:47.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:47.808+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f821c09d570 con 0x7f82241003e0 2024-04-02T13:41:47.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 192213 2.64 MB/sec execute 188 sec latency 2625.560 ms 2024-04-02T13:41:48.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.025+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8224100d90 con 0x7f82241003e0 2024-04-02T13:41:48.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.026+0000 7f8221ffb640 1 -- 172.21.15.122:0/3491480376 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11949 (secure 0 0 0) 0x7f821c09ce70 con 0x7f82241003e0 2024-04-02T13:41:48.029 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:41:48.030 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:40:36.405088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:41:48.030 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 38 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 msgr2=0x7f820c0a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 0x7f820c0a83f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8218006fd0 tx=0x7f8218008040 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 msgr2=0x7f82241961d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f821c00de60 tx=0x7f821c00b730 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 shutdown_connections 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f820c0a5f30 0x7f820c0a83f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822419aa90 0x7f822419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8224105460 0x7f8224196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 --2- 172.21.15.122:0/3491480376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f82241003e0 0x7f82241961d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 >> 172.21.15.122:0/3491480376 conn(0x7f8224076f90 msgr2=0x7f8224103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 shutdown_connections 2024-04-02T13:41:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.028+0000 7f822afe2640 1 -- 172.21.15.122:0/3491480376 wait complete. 2024-04-02T13:41:48.042 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:41:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- 172.21.15.122:0/32338407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 msgr2=0x7f76d4106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- 172.21.15.122:0/32338407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d4106d80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009930 tx=0x7f76d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- 172.21.15.122:0/32338407 shutdown_connections 2024-04-02T13:41:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- 172.21.15.122:0/32338407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- 172.21.15.122:0/32338407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 0x7f76d4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- 172.21.15.122:0/32338407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d4106d80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- 172.21.15.122:0/32338407 >> 172.21.15.122:0/32338407 conn(0x7f76d4077ad0 msgr2=0x7f76d4077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- 172.21.15.122:0/32338407 shutdown_connections 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- 172.21.15.122:0/32338407 wait complete. 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 Processor -- start 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 -- start start 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 0x7f76d4199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.142+0000 7f76db996640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d4199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110d80 con 0x7f76d41069a0 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110c00 con 0x7f76d4101350 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76d4110f00 con 0x7f76d4100990 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44510/0 (socket says 172.21.15.122:44510) 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 -- 172.21.15.122:0/1422670356 learned_addr learned my addr 172.21.15.122:0/1422670356 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9732640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 0x7f76d4199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d4199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 -- 172.21.15.122:0/1422670356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 msgr2=0x7f76d4199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 0x7f76d4199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 -- 172.21.15.122:0/1422670356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 msgr2=0x7f76d4199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d4199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 -- 172.21.15.122:0/1422670356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d41a47c0 con 0x7f76d41069a0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76d9f33640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00eaa0 tx=0x7f76cc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76ca7fc640 1 -- 172.21.15.122:0/1422670356 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00cdb0 con 0x7f76d41069a0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76ca7fc640 1 -- 172.21.15.122:0/1422670356 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc005020 con 0x7f76d41069a0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d41a4990 con 0x7f76d41069a0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76ca7fc640 1 -- 172.21.15.122:0/1422670356 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0116b0 con 0x7f76d41069a0 2024-04-02T13:41:48.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.143+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d41a4e60 con 0x7f76d41069a0 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.145+0000 7f76ca7fc640 1 -- 172.21.15.122:0/1422670356 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f76cc005560 con 0x7f76d41069a0 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 Processor -- start 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76ca7fc640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 0x7f76c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 -- start start 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76ca7fc640 1 -- 172.21.15.122:0/1422670356 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f76cc0d0df0 con 0x7f76d41069a0 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 0x7f76c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c114d0 0x563031c118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563031b757b0 con 0x563031c1f0b0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563031b14570 con 0x563031c114d0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76de57d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563031b71760 con 0x563031c11df0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44526/0 (socket says 172.21.15.122:44526) 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76d9f33640 1 -- 172.21.15.122:0/171951744 learned_addr learned my addr 172.21.15.122:0/171951744 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.146+0000 7f76d9732640 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c114d0 0x563031c118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d8f31640 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 -- 172.21.15.122:0/171951744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 msgr2=0x563031c1eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 0x7f76c00a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f76d406ac60 tx=0x7f76c4009f20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 -- 172.21.15.122:0/171951744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c114d0 msgr2=0x563031c118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c114d0 0x563031c118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 -- 172.21.15.122:0/171951744 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630318893f0 con 0x563031c1f0b0 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76d9f33640 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00efd0 tx=0x7f76cc09cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1b42b6fd0a3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76bf7fe640 1 -- 172.21.15.122:0/171951744 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc01f040 con 0x563031c1f0b0 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76bf7fe640 1 -- 172.21.15.122:0/171951744 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc0051d0 con 0x563031c1f0b0 2024-04-02T13:41:48.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.147+0000 7f76bf7fe640 1 -- 172.21.15.122:0/171951744 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0872c0 con 0x563031c1f0b0 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- 172.21.15.122:0/171951744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 msgr2=0x563031c214a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00efd0 tx=0x7f76cc09cae0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- 172.21.15.122:0/171951744 shutdown_connections 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c214a0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- 172.21.15.122:0/171951744 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c114d0 0x563031c118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- 172.21.15.122:0/171951744 >> 172.21.15.122:0/171951744 conn(0x563031c0bed0 msgr2=0x563031c0c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- 172.21.15.122:0/171951744 shutdown_connections 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- 172.21.15.122:0/171951744 wait complete. 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- start start 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 Processor -- start 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563031c21e70 con 0x563031c114d0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563031b30e80 con 0x563031c1f0b0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.148+0000 7f76de57d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563031c21ff0 con 0x563031c11df0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76de57d740 10 client.0 _ll_register_callbacks cb 0x563031b6caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44538/0 (socket says 172.21.15.122:44538) 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 -- 172.21.15.122:0/1910646024 learned_addr learned my addr 172.21.15.122:0/1910646024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9f33640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 -- 172.21.15.122:0/1910646024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 msgr2=0x563031c1bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 -- 172.21.15.122:0/1910646024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 msgr2=0x563031c1c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c1c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 -- 172.21.15.122:0/1910646024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630319537d0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76d9732640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f76d002cfd0 tx=0x7f76d002cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ad5b2e87c88d7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004340 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f76d00046c0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76cc0051d0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76d00044e0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76de57d740 10 client.17601 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563031c2dfa0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.149+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc0051d0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f76d00035f0 con 0x563031c114d0 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 10 client.17601 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 10 client.17601 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:41:48.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 20 client.17601 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c2f4b0 0x563031c31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 4 client.17601 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563031b82cf0 con 0x563031c2f4b0 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c2f4b0 0x563031c31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76bcff9640 10 client.17601 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.150+0000 7f76d8f31640 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c2f4b0 0x563031c31950 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d9a1385fe876db79 server_cookie=8fd6c7ccfc560758 in_seq=0 out_seq=0 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76bcff9640 1 -- 172.21.15.122:0/1910646024 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563031b82cf0 con 0x563031c2f4b0 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76bcff9640 10 client.17601 handle_command_reply: tid=0 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 client.17601 shutdown 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 msgr2=0x563031c1eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f76d002cfd0 tx=0x7f76d002cdc0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 shutdown_connections 2024-04-02T13:41:48.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c2f4b0 0x563031c31950 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c1c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c11df0 0x563031c1bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 --2- 172.21.15.122:0/1910646024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c114d0 0x563031c1eae0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 >> 172.21.15.122:0/1910646024 conn(0x563031c0bed0 msgr2=0x563031c0dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76bcff9640 0 client.17601 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 shutdown_connections 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 1 -- 172.21.15.122:0/1910646024 wait complete. 2024-04-02T13:41:48.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.151+0000 7f76de57d740 20 client.17601 trim_cache size 0 max 16384 2024-04-02T13:41:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 Processor -- start 2024-04-02T13:41:48.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 -- start start 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c1b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c11df0 0x563031c0da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41acde0 con 0x563031c1f0b0 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ae9e0 con 0x563031c11df0 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.164+0000 7f76de57d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ad4b0 con 0x563031c114d0 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44544/0 (socket says 172.21.15.122:44544) 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 -- 172.21.15.122:0/544431444 learned_addr learned my addr 172.21.15.122:0/544431444 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 -- 172.21.15.122:0/544431444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 msgr2=0x563031c1b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c1b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 -- 172.21.15.122:0/544431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c11df0 msgr2=0x563031c0da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c11df0 0x563031c0da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 -- 172.21.15.122:0/544431444 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563031c1c990 con 0x563031c1f0b0 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76d9f33640 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0a0f10 tx=0x7f76cc09c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a8ae1935dabb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76beffd640 1 -- 172.21.15.122:0/544431444 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc0b5030 con 0x563031c1f0b0 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76beffd640 1 -- 172.21.15.122:0/544431444 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc09d0a0 con 0x563031c1f0b0 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.165+0000 7f76beffd640 1 -- 172.21.15.122:0/544431444 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc01f050 con 0x563031c1f0b0 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- 172.21.15.122:0/544431444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 msgr2=0x563031c1e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0a0f10 tx=0x7f76cc09c610 comp rx=0 tx=0).stop 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- 172.21.15.122:0/544431444 shutdown_connections 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c1f0b0 0x563031c1e610 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c11df0 0x563031c0da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- 172.21.15.122:0/544431444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c1b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- 172.21.15.122:0/544431444 >> 172.21.15.122:0/544431444 conn(0x563031c0bed0 msgr2=0x563031c0d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- 172.21.15.122:0/544431444 shutdown_connections 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- 172.21.15.122:0/544431444 wait complete. 2024-04-02T13:41:48.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- start start 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 Processor -- start 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c18190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c18c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41ae250 con 0x563031c11df0 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41adb80 con 0x563031c1f0b0 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76d41aadc0 con 0x563031c114d0 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.166+0000 7f76de57d740 10 client.0 _ll_register_callbacks cb 0x7f76d0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44548/0 (socket says 172.21.15.122:44548) 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 -- 172.21.15.122:0/789178630 learned_addr learned my addr 172.21.15.122:0/789178630 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 -- 172.21.15.122:0/789178630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 msgr2=0x563031c18190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c18190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 -- 172.21.15.122:0/789178630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 msgr2=0x563031c18c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c18c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 -- 172.21.15.122:0/789178630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563031c26360 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76d8f31640 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009fd0 tx=0x7f76c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d083e847c61356a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c402c030 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f76c4004020 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76cc09d0a0 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f76c4036050 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 10 client.17607 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563031c276b0 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc09d0a0 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f76c400f040 con 0x563031c11df0 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 10 client.17607 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 10 client.17607 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.167+0000 7f76de57d740 20 client.17607 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c95ba0 0x563031c97fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 4 client.17607 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563031b7d040 con 0x563031c95ba0 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76d9732640 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c95ba0 0x563031c97fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76bcff9640 10 client.17607 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76d9732640 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c95ba0 0x563031c97fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bb6deb0d6d1b2b7 server_cookie=c171fbf1bee2bf39 in_seq=0 out_seq=0 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76bcff9640 1 -- 172.21.15.122:0/789178630 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563031b7d040 con 0x563031c95ba0 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76bcff9640 10 client.17607 handle_command_reply: tid=0 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 client.17607 shutdown 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 msgr2=0x563031c186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009fd0 tx=0x7f76c40046c0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 shutdown_connections 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563031c95ba0 0x563031c97fc0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563031c1f0b0 0x563031c18c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563031c11df0 0x563031c186d0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76bcff9640 0 client.17607 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 --2- 172.21.15.122:0/789178630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563031c114d0 0x563031c18190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 >> 172.21.15.122:0/789178630 conn(0x563031c0bed0 msgr2=0x563031c0dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 shutdown_connections 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 1 -- 172.21.15.122:0/789178630 wait complete. 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.168+0000 7f76de57d740 20 client.17607 trim_cache size 0 max 16384 2024-04-02T13:41:48.178 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 msgr2=0x7f76c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 0x7f76c00a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f76d406ac60 tx=0x7f76c4009f20 comp rx=0 tx=0).stop 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 msgr2=0x7f76d41a4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00eaa0 tx=0x7f76cc00ef60 comp rx=0 tx=0).stop 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 shutdown_connections 2024-04-02T13:41:48.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76c00a3d20 0x7f76c00a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76d41069a0 0x7f76d41a4280 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76d4101350 0x7f76d4199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 --2- 172.21.15.122:0/1422670356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76d4100990 0x7f76d4199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 >> 172.21.15.122:0/1422670356 conn(0x7f76d4077ad0 msgr2=0x7f76d4103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 shutdown_connections 2024-04-02T13:41:48.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.169+0000 7f76db996640 1 -- 172.21.15.122:0/1422670356 wait complete. 2024-04-02T13:41:48.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:41:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- 172.21.15.122:0/1859703514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 msgr2=0x7f17a4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1859703514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a4101f10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1794009930 tx=0x7f179402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- 172.21.15.122:0/1859703514 shutdown_connections 2024-04-02T13:41:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1859703514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 0x7f17a410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1859703514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a4101f10 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1859703514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 0x7f17a4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- 172.21.15.122:0/1859703514 >> 172.21.15.122:0/1859703514 conn(0x7f17a4078f00 msgr2=0x7f17a4079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- 172.21.15.122:0/1859703514 shutdown_connections 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- 172.21.15.122:0/1859703514 wait complete. 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 Processor -- start 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.283+0000 7f17ab37b640 1 -- start start 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 0x7f17a4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 0x7f17a419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4111eb0 con 0x7f17a4101a90 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4111d30 con 0x7f17a4102450 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4112030 con 0x7f17a4107aa0 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44574/0 (socket says 172.21.15.122:44574) 2024-04-02T13:41:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 -- 172.21.15.122:0/1227387713 learned_addr learned my addr 172.21.15.122:0/1227387713 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9918640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 0x7f17a419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a8916640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 0x7f17a4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 -- 172.21.15.122:0/1227387713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 msgr2=0x7f17a419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 0x7f17a419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 -- 172.21.15.122:0/1227387713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 msgr2=0x7f17a4198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 0x7f17a4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 -- 172.21.15.122:0/1227387713 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a419d3c0 con 0x7f17a4101a90 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17a9117640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f17a000d770 tx=0x7f17a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f179a7fc640 1 -- 172.21.15.122:0/1227387713 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0017070 con 0x7f17a4101a90 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f179a7fc640 1 -- 172.21.15.122:0/1227387713 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0005500 con 0x7f17a4101a90 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a419d6b0 con 0x7f17a4101a90 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f179a7fc640 1 -- 172.21.15.122:0/1227387713 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004e00 con 0x7f17a4101a90 2024-04-02T13:41:48.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.284+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a4f40 con 0x7f17a4101a90 2024-04-02T13:41:48.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f179a7fc640 1 -- 172.21.15.122:0/1227387713 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f17a0004fa0 con 0x7f17a4101a90 2024-04-02T13:41:48.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 Processor -- start 2024-04-02T13:41:48.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 -- start start 2024-04-02T13:41:48.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dfa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6dfd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f179a7fc640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 0x7f17900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6d557b0 con 0x55ecc6df17a0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6cf4570 con 0x55ecc6dfaca0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17adf65740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6d51760 con 0x55ecc6df20c0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17a9918640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17a9918640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44584/0 (socket says 172.21.15.122:44584) 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.287+0000 7f17a9918640 1 -- 172.21.15.122:0/3313134225 learned_addr learned my addr 172.21.15.122:0/3313134225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 0x7f17900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a8916640 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6dfd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f179a7fc640 1 -- 172.21.15.122:0/1227387713 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f17a00d05f0 con 0x7f17a4101a90 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9117640 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dfa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 -- 172.21.15.122:0/3313134225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 msgr2=0x55ecc6dfa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dfa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 -- 172.21.15.122:0/3313134225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 msgr2=0x55ecc6dfd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6dfd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 -- 172.21.15.122:0/3313134225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6a693f0 con 0x55ecc6df17a0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f179c012d00 tx=0x7f179c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedf590837228b0f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f17a9918640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 0x7f17900a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f179c009fd0 tx=0x7f179c017000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f178f7fe640 1 -- 172.21.15.122:0/3313134225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c022070 con 0x55ecc6df17a0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f178f7fe640 1 -- 172.21.15.122:0/3313134225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c009c80 con 0x55ecc6df17a0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.288+0000 7f178f7fe640 1 -- 172.21.15.122:0/3313134225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c01e900 con 0x55ecc6df17a0 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- 172.21.15.122:0/3313134225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 msgr2=0x55ecc6df1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f179c012d00 tx=0x7f179c010740 comp rx=0 tx=0).stop 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- 172.21.15.122:0/3313134225 shutdown_connections 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6dfd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dfa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 --2- 172.21.15.122:0/3313134225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df1b80 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- 172.21.15.122:0/3313134225 >> 172.21.15.122:0/3313134225 conn(0x55ecc6debed0 msgr2=0x55ecc6dec2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- 172.21.15.122:0/3313134225 shutdown_connections 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- 172.21.15.122:0/3313134225 wait complete. 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 -- start start 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.289+0000 7f17adf65740 1 Processor -- start 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6df20c0 0x55ecc6df8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6dfaca0 0x55ecc6df8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6dfda30 con 0x55ecc6df17a0 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6d10e80 con 0x55ecc6df20c0 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6dfdbb0 con 0x55ecc6dfaca0 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 10 client.0 _ll_register_callbacks cb 0x55ecc6d4caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44600/0 (socket says 172.21.15.122:44600) 2024-04-02T13:41:48.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 -- 172.21.15.122:0/2196628337 learned_addr learned my addr 172.21.15.122:0/2196628337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9918640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6dfaca0 0x55ecc6df8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 -- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6dfaca0 msgr2=0x55ecc6df8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a8916640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6df20c0 0x55ecc6df8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6dfaca0 0x55ecc6df8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 -- 172.21.15.122:0/2196628337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6df20c0 msgr2=0x55ecc6df8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6df20c0 0x55ecc6df8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 -- 172.21.15.122:0/2196628337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc6b337d0 con 0x55ecc6df17a0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17a9117640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005760 tx=0x7f17a009bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db967da8fbd13f95 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f178cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0014030 con 0x55ecc6df17a0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f17a00b5640 con 0x55ecc6df17a0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.290+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179c009c80 con 0x55ecc6df17a0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f17a00ba050 con 0x55ecc6df17a0 2024-04-02T13:41:48.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 10 client.17619 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ecc6e09b90 con 0x55ecc6df17a0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0062020 con 0x55ecc6df17a0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f17a00cdc30 con 0x55ecc6df17a0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 10 client.17619 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 10 client.17619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 20 client.17619 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e0b040 0x55ecc6e0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 4 client.17619 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc6d62cf0 con 0x55ecc6e0b040 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.291+0000 7f17a8916640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e0b040 0x55ecc6e0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.292+0000 7f17a8916640 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e0b040 0x55ecc6e0d490 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fe59d18b7c5a3c9 server_cookie=eddd26d06ccdafe5 in_seq=0 out_seq=0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.292+0000 7f178cff9640 10 client.17619 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.292+0000 7f178cff9640 1 -- 172.21.15.122:0/2196628337 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecc6d62cf0 con 0x55ecc6e0b040 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.292+0000 7f178cff9640 10 client.17619 handle_command_reply: tid=0 2024-04-02T13:41:48.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.292+0000 7f17adf65740 1 client.17619 shutdown 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 msgr2=0x55ecc6df7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f17a0005760 tx=0x7f17a009bc20 comp rx=0 tx=0).stop 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 shutdown_connections 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6dfaca0 0x55ecc6df8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6df20c0 0x55ecc6df8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e0b040 0x55ecc6e0d490 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 --2- 172.21.15.122:0/2196628337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6df7b00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f178cff9640 0 client.17619 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 >> 172.21.15.122:0/2196628337 conn(0x55ecc6debed0 msgr2=0x55ecc6dedc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 shutdown_connections 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 1 -- 172.21.15.122:0/2196628337 wait complete. 2024-04-02T13:41:48.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.293+0000 7f17adf65740 20 client.17619 trim_cache size 0 max 16384 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 Processor -- start 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 -- start start 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6e06920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6deda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41aef60 con 0x55ecc6df17a0 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41b0b60 con 0x55ecc6dfaca0 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17adf65740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41af630 con 0x55ecc6df20c0 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44612/0 (socket says 172.21.15.122:44612) 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 -- 172.21.15.122:0/3839685941 learned_addr learned my addr 172.21.15.122:0/3839685941 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 -- 172.21.15.122:0/3839685941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 msgr2=0x55ecc6e06920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6e06920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 -- 172.21.15.122:0/3839685941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 msgr2=0x55ecc6deda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6deda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 -- 172.21.15.122:0/3839685941 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc6df2900 con 0x55ecc6df17a0 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.306+0000 7f17a9117640 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f17a009bf00 tx=0x7f17a000dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec109ccabd2240d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f178effd640 1 -- 172.21.15.122:0/3839685941 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a0014030 con 0x55ecc6df17a0 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f178effd640 1 -- 172.21.15.122:0/3839685941 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0062070 con 0x55ecc6df17a0 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f178effd640 1 -- 172.21.15.122:0/3839685941 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a00d0040 con 0x55ecc6df17a0 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- 172.21.15.122:0/3839685941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 msgr2=0x55ecc6dfa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f17a009bf00 tx=0x7f17a000dea0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- 172.21.15.122:0/3839685941 shutdown_connections 2024-04-02T13:41:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6deda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6e06920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 --2- 172.21.15.122:0/3839685941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfa230 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- 172.21.15.122:0/3839685941 >> 172.21.15.122:0/3839685941 conn(0x55ecc6debed0 msgr2=0x55ecc6ded300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- 172.21.15.122:0/3839685941 shutdown_connections 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- 172.21.15.122:0/3839685941 wait complete. 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.307+0000 7f17adf65740 1 -- start start 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 Processor -- start 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6e70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41b03d0 con 0x55ecc6df17a0 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41afd00 con 0x55ecc6dfaca0 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41acf40 con 0x55ecc6df20c0 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17adf65740 10 client.0 _ll_register_callbacks cb 0x7f17a009bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44628/0 (socket says 172.21.15.122:44628) 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 -- 172.21.15.122:0/45078098 learned_addr learned my addr 172.21.15.122:0/45078098 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a8916640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 -- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 msgr2=0x55ecc6dff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 -- 172.21.15.122:0/45078098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 msgr2=0x55ecc6e70580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6e70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 -- 172.21.15.122:0/45078098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc6e70ac0 con 0x55ecc6df17a0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f17a9117640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f17a00048e0 tx=0x7f17a0009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc33cd7b8959160 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.308+0000 7f178cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a007f070 con 0x55ecc6df17a0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f17a00a4be0 con 0x55ecc6df17a0 2024-04-02T13:41:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a0062070 con 0x55ecc6df17a0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f17a00d0040 con 0x55ecc6df17a0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 10 client.17625 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55ecc6e71cc0 con 0x55ecc6df17a0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17a0062070 con 0x55ecc6df17a0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f17a009c460 con 0x55ecc6df17a0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 10 client.17625 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 10 client.17625 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 20 client.17625 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e721e0 0x55ecc6e745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 4 client.17625 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ecc6d5d040 con 0x55ecc6e721e0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.309+0000 7f17a8916640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e721e0 0x55ecc6e745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f178cff9640 10 client.17625 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17a8916640 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e721e0 0x55ecc6e745d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f8dbe91077ea11d server_cookie=26c092fa3ee11550 in_seq=0 out_seq=0 2024-04-02T13:41:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f178cff9640 1 -- 172.21.15.122:0/45078098 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ecc6d5d040 con 0x55ecc6e721e0 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f178cff9640 10 client.17625 handle_command_reply: tid=0 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 client.17625 shutdown 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 msgr2=0x55ecc6dfee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f17a00048e0 tx=0x7f17a0009ef0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 shutdown_connections 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ecc6dfaca0 0x55ecc6e70580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ecc6df20c0 0x55ecc6dff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ecc6e721e0 0x55ecc6e745d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 --2- 172.21.15.122:0/45078098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ecc6df17a0 0x55ecc6dfee80 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f178cff9640 0 client.17625 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 >> 172.21.15.122:0/45078098 conn(0x55ecc6debed0 msgr2=0x55ecc6df8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 shutdown_connections 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 1 -- 172.21.15.122:0/45078098 wait complete. 2024-04-02T13:41:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.310+0000 7f17adf65740 20 client.17625 trim_cache size 0 max 16384 2024-04-02T13:41:48.321 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 msgr2=0x7f17900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 0x7f17900a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f179c009fd0 tx=0x7f179c017000 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 msgr2=0x7f17a41983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f17a000d770 tx=0x7f17a000dc30 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 shutdown_connections 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f17900a3d20 0x7f17900a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4107aa0 0x7f17a419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4102450 0x7f17a4198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 --2- 172.21.15.122:0/1227387713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4101a90 0x7f17a41983d0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 >> 172.21.15.122:0/1227387713 conn(0x7f17a4078f00 msgr2=0x7f17a41044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 shutdown_connections 2024-04-02T13:41:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.311+0000 7f17ab37b640 1 -- 172.21.15.122:0/1227387713 wait complete. 2024-04-02T13:41:48.324 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:41:48.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 -- 172.21.15.122:0/2096540988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28107b50 msgr2=0x7fbf28107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 --2- 172.21.15.122:0/2096540988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28107b50 0x7fbf28107f30 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbf20009cd0 tx=0x7fbf2002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 -- 172.21.15.122:0/2096540988 shutdown_connections 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 --2- 172.21.15.122:0/2096540988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf280696c0 0x7fbf2810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 --2- 172.21.15.122:0/2096540988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf28068d00 0x7fbf28069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 --2- 172.21.15.122:0/2096540988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28107b50 0x7fbf28107f30 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 -- 172.21.15.122:0/2096540988 >> 172.21.15.122:0/2096540988 conn(0x7fbf28077b00 msgr2=0x7fbf28077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 -- 172.21.15.122:0/2096540988 shutdown_connections 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.431+0000 7fbf30709640 1 -- 172.21.15.122:0/2096540988 wait complete. 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 Processor -- start 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 -- start start 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf280696c0 0x7fbf28198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf28107b50 0x7fbf2819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28111e70 con 0x7fbf28068d00 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28111cf0 con 0x7fbf28107b50 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf30709640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf28111ff0 con 0x7fbf280696c0 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44636/0 (socket says 172.21.15.122:44636) 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/331482958 learned_addr learned my addr 172.21.15.122:0/331482958 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf280696c0 0x7fbf28198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf28107b50 0x7fbf2819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/331482958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf280696c0 msgr2=0x7fbf28198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf280696c0 0x7fbf28198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/331482958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf28107b50 msgr2=0x7fbf2819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf28107b50 0x7fbf2819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.432+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/331482958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2819d3c0 con 0x7fbf28068d00 2024-04-02T13:41:48.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004af0 tx=0x7fbf20032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf1f7fe640 1 -- 172.21.15.122:0/331482958 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20042020 con 0x7fbf28068d00 2024-04-02T13:41:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2819d650 con 0x7fbf28068d00 2024-04-02T13:41:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf1f7fe640 1 -- 172.21.15.122:0/331482958 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf20034640 con 0x7fbf28068d00 2024-04-02T13:41:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf1f7fe640 1 -- 172.21.15.122:0/331482958 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20039040 con 0x7fbf28068d00 2024-04-02T13:41:48.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.433+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf281a4f40 con 0x7fbf28068d00 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf1f7fe640 1 -- 172.21.15.122:0/331482958 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbf20045020 con 0x7fbf28068d00 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 Processor -- start 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 -- start start 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f72020760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f72020ca0 0x556f72023090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f71f7b7b0 con 0x556f720177a0 2024-04-02T13:41:48.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556f71f1a570 con 0x556f720180c0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf333c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556f71f77760 con 0x556f72020ca0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf2eca6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf2eca6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44650/0 (socket says 172.21.15.122:44650) 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2067083429 learned_addr learned my addr 172.21.15.122:0/2067083429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f72020760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f72020ca0 0x556f72023090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.436+0000 7fbf1f7fe640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 0x7fbf140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf1f7fe640 1 -- 172.21.15.122:0/331482958 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf200f2a20 con 0x7fbf28068d00 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2067083429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f72020ca0 msgr2=0x556f72023090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f72020ca0 0x556f72023090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2067083429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 msgr2=0x556f72020760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f72020760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2067083429 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f71c8f3f0 con 0x556f720177a0 2024-04-02T13:41:48.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 0x7fbf140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.437+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009d20 tx=0x7fbf24013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b35d6bccb568d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.438+0000 7fbf1cff9640 1 -- 172.21.15.122:0/2067083429 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400f6e0 con 0x556f720177a0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.438+0000 7fbf1cff9640 1 -- 172.21.15.122:0/2067083429 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf2400fe50 con 0x556f720177a0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.438+0000 7fbf1cff9640 1 -- 172.21.15.122:0/2067083429 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24025ab0 con 0x556f720177a0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.438+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 0x7fbf140a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf24020000 tx=0x7fbf2401d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- 172.21.15.122:0/2067083429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 msgr2=0x556f72017b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbf24009d20 tx=0x7fbf24013830 comp rx=0 tx=0).stop 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- 172.21.15.122:0/2067083429 shutdown_connections 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f72020ca0 0x556f72023090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f72020760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2067083429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f72017b80 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- 172.21.15.122:0/2067083429 >> 172.21.15.122:0/2067083429 conn(0x556f72011ed0 msgr2=0x556f720122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- 172.21.15.122:0/2067083429 shutdown_connections 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- 172.21.15.122:0/2067083429 wait complete. 2024-04-02T13:41:48.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 -- start start 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.439+0000 7fbf333c5740 1 Processor -- start 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720177a0 0x556f72020680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f7201da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556f72023a90 con 0x556f72020ca0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556f71f36e80 con 0x556f720180c0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556f72023c10 con 0x556f720177a0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 10 client.0 _ll_register_callbacks cb 0x556f71f72aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44656/0 (socket says 172.21.15.122:44656) 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2098725641 learned_addr learned my addr 172.21.15.122:0/2098725641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f7201da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720177a0 0x556f72020680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2098725641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720177a0 msgr2=0x556f72020680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720177a0 0x556f72020680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 msgr2=0x556f7201da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f7201da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 -- 172.21.15.122:0/2098725641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f71d597d0 con 0x556f72020ca0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf240256d0 tx=0x7fbf2401beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48edb2873712016d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24013280 con 0x556f72020ca0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf24025430 con 0x556f72020ca0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.440+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf2400fe50 con 0x556f72020ca0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbf2403a050 con 0x556f72020ca0 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 10 client.17637 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556f7202faf0 con 0x556f72020ca0 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf2400fe50 con 0x556f72020ca0 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fbf240108f0 con 0x556f72020ca0 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 10 client.17637 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 10 client.17637 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 20 client.17637 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72030e90 0x556f720332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 4 client.17637 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f71f88cf0 con 0x556f72030e90 2024-04-02T13:41:48.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.441+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72030e90 0x556f720332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72030e90 0x556f720332a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52a72be56fca7c38 server_cookie=80c6df98fdea3cdd in_seq=0 out_seq=0 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf11ffb640 10 client.17637 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf11ffb640 1 -- 172.21.15.122:0/2098725641 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f71f88cf0 con 0x556f72030e90 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf11ffb640 10 client.17637 handle_command_reply: tid=0 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf333c5740 1 client.17637 shutdown 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 msgr2=0x556f7201e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbf240256d0 tx=0x7fbf2401beb0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 shutdown_connections 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.442+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f72020ca0 0x556f7201e100 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72030e90 0x556f720332a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f720180c0 0x556f7201da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 --2- 172.21.15.122:0/2098725641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720177a0 0x556f72020680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 >> 172.21.15.122:0/2098725641 conn(0x556f72011ed0 msgr2=0x556f72013c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf11ffb640 0 client.17637 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 shutdown_connections 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 1 -- 172.21.15.122:0/2098725641 wait complete. 2024-04-02T13:41:48.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.443+0000 7fbf333c5740 20 client.17637 trim_cache size 0 max 16384 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.456+0000 7fbf333c5740 1 Processor -- start 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.456+0000 7fbf333c5740 1 -- start start 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f72032ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f720139c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281ace40 con 0x556f720177a0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281aea40 con 0x556f72020ca0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf333c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281ad510 con 0x556f720180c0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44658/0 (socket says 172.21.15.122:44658) 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/1421036664 learned_addr learned my addr 172.21.15.122:0/1421036664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f720139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/1421036664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 msgr2=0x556f72032ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f72032ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/1421036664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 msgr2=0x556f720139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f720139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/1421036664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f72019670 con 0x556f720177a0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f720139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbf200bb110 tx=0x7fbf200f4f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19836e91a1930807 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf13fff640 1 -- 172.21.15.122:0/1421036664 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200be280 con 0x556f720177a0 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf13fff640 1 -- 172.21.15.122:0/1421036664 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf200bee40 con 0x556f720177a0 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.457+0000 7fbf13fff640 1 -- 172.21.15.122:0/1421036664 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200be0a0 con 0x556f720177a0 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- 172.21.15.122:0/1421036664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 msgr2=0x556f7202c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbf200bb110 tx=0x7fbf200f4f30 comp rx=0 tx=0).stop 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- 172.21.15.122:0/1421036664 shutdown_connections 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f720139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f72032ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 --2- 172.21.15.122:0/1421036664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7202c950 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- 172.21.15.122:0/1421036664 >> 172.21.15.122:0/1421036664 conn(0x556f72011ed0 msgr2=0x556f720132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- 172.21.15.122:0/1421036664 shutdown_connections 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- 172.21.15.122:0/1421036664 wait complete. 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 -- start start 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.458+0000 7fbf333c5740 1 Processor -- start 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f7201a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f72027e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281ae2b0 con 0x556f720177a0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281adbe0 con 0x556f72020ca0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbf281aae20 con 0x556f720180c0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 10 client.0 _ll_register_callbacks cb 0x7fbf200f7040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43718/0 (socket says 172.21.15.122:43718) 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/184381247 learned_addr learned my addr 172.21.15.122:0/184381247 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2eca6640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f72027e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f7201a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/184381247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 msgr2=0x556f7201a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f7201a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 msgr2=0x556f72027e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f72027e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 -- 172.21.15.122:0/184381247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f720283a0 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf2e4a5640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbf200baba0 tx=0x7fbf200be190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddc41671723a12c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf200c2af0 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fbf200f0060 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.459+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf200bee40 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fbf200c73b0 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 10 client.17643 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556f72029610 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf200bee40 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fbf200c6f20 con 0x556f720177a0 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 10 client.17643 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 10 client.17643 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:41:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 20 client.17643 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72097720 0x556f72099b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 4 client.17643 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f71f83040 con 0x556f72097720 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.460+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72097720 0x556f72099b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf11ffb640 10 client.17643 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf2dca4640 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72097720 0x556f72099b10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=107a97deff275604 server_cookie=70fe2394b5ad0292 in_seq=0 out_seq=0 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf11ffb640 1 -- 172.21.15.122:0/184381247 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f71f83040 con 0x556f72097720 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf11ffb640 10 client.17643 handle_command_reply: tid=0 2024-04-02T13:41:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 client.17643 shutdown 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 msgr2=0x556f7201a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fbf200baba0 tx=0x7fbf200be190 comp rx=0 tx=0).stop 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 shutdown_connections 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556f72097720 0x556f72099b10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556f72020ca0 0x556f72027e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf11ffb640 0 client.17643 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556f720180c0 0x556f7201a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 --2- 172.21.15.122:0/184381247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556f720177a0 0x556f7201a0c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 >> 172.21.15.122:0/184381247 conn(0x556f72011ed0 msgr2=0x556f7201f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 shutdown_connections 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 1 -- 172.21.15.122:0/184381247 wait complete. 2024-04-02T13:41:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.461+0000 7fbf333c5740 20 client.17643 trim_cache size 0 max 16384 2024-04-02T13:41:48.473 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:48.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 msgr2=0x7fbf140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 0x7fbf140a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf24020000 tx=0x7fbf2401d000 comp rx=0 tx=0).stop 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 msgr2=0x7fbf281983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbf20004af0 tx=0x7fbf20032ec0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 shutdown_connections 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf140a3d20 0x7fbf140a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbf28107b50 0x7fbf2819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf280696c0 0x7fbf28198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 --2- 172.21.15.122:0/331482958 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbf28068d00 0x7fbf281983d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 >> 172.21.15.122:0/331482958 conn(0x7fbf28077b00 msgr2=0x7fbf280fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 shutdown_connections 2024-04-02T13:41:48.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.462+0000 7fbf30709640 1 -- 172.21.15.122:0/331482958 wait complete. 2024-04-02T13:41:48.476 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:41:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 -- 172.21.15.122:0/566294276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06aa30 msgr2=0x7f43ec10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 --2- 172.21.15.122:0/566294276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06aa30 0x7f43ec10e960 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f43e4009cd0 tx=0x7f43e402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 -- 172.21.15.122:0/566294276 shutdown_connections 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 --2- 172.21.15.122:0/566294276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06aa30 0x7f43ec10e960 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 --2- 172.21.15.122:0/566294276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec06a070 0x7f43ec06a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 --2- 172.21.15.122:0/566294276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec107b30 0x7f43ec107f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 -- 172.21.15.122:0/566294276 >> 172.21.15.122:0/566294276 conn(0x7f43ec0779e0 msgr2=0x7f43ec077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 -- 172.21.15.122:0/566294276 shutdown_connections 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 -- 172.21.15.122:0/566294276 wait complete. 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.590+0000 7f43f291c640 1 Processor -- start 2024-04-02T13:41:48.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 -- start start 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec06aa30 0x7f43ec198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec107b30 0x7f43ec19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec111e80 con 0x7f43ec06a070 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec111d00 con 0x7f43ec107b30 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f291c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec112000 con 0x7f43ec06aa30 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43734/0 (socket says 172.21.15.122:43734) 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 -- 172.21.15.122:0/2926961008 learned_addr learned my addr 172.21.15.122:0/2926961008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43f0eb9640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec107b30 0x7f43ec19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec06aa30 0x7f43ec198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 -- 172.21.15.122:0/2926961008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec06aa30 msgr2=0x7f43ec198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec06aa30 0x7f43ec198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 -- 172.21.15.122:0/2926961008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec107b30 msgr2=0x7f43ec19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec107b30 0x7f43ec19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 -- 172.21.15.122:0/2926961008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec19d2d0 con 0x7f43ec06a070 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.591+0000 7f43ebfff640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c920 tx=0x7f43dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.592+0000 7f43e97fa640 1 -- 172.21.15.122:0/2926961008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc007c70 con 0x7f43ec06a070 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.592+0000 7f43e97fa640 1 -- 172.21.15.122:0/2926961008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc007e10 con 0x7f43ec06a070 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.592+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec19d5c0 con 0x7f43ec06a070 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.592+0000 7f43e97fa640 1 -- 172.21.15.122:0/2926961008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc018890 con 0x7f43ec06a070 2024-04-02T13:41:48.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.592+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1a4e50 con 0x7f43ec06a070 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.594+0000 7f43e97fa640 1 -- 172.21.15.122:0/2926961008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f43dc018a70 con 0x7f43ec06a070 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43e97fa640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 0x7f43d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 0x7f43d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43e97fa640 1 -- 172.21.15.122:0/2926961008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f43dc0d0c60 con 0x7f43ec06a070 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 Processor -- start 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 -- start start 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 0x7f43d40a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43ec107130 tx=0x7f43e0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d993040760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993043090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d992f9b7b0 con 0x55d9930377a0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d992f3a570 con 0x55d9930380c0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43f55e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d992f97760 con 0x55d993040ca0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43744/0 (socket says 172.21.15.122:43744) 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43ebfff640 1 -- 172.21.15.122:0/739324355 learned_addr learned my addr 172.21.15.122:0/739324355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.595+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d993040760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 -- 172.21.15.122:0/739324355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 msgr2=0x55d993043090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993043090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 -- 172.21.15.122:0/739324355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 msgr2=0x55d993040760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d993040760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 -- 172.21.15.122:0/739324355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d992caf3f0 con 0x55d9930377a0 2024-04-02T13:41:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d993040760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43ebfff640 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f43dc099240 tx=0x7f43dc09d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3677652ed77c445 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43d27fc640 1 -- 172.21.15.122:0/739324355 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0a54e0 con 0x55d9930377a0 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43d27fc640 1 -- 172.21.15.122:0/739324355 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc018d60 con 0x55d9930377a0 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.596+0000 7f43d27fc640 1 -- 172.21.15.122:0/739324355 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0870b0 con 0x55d9930377a0 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- 172.21.15.122:0/739324355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 msgr2=0x55d993037b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f43dc099240 tx=0x7f43dc09d030 comp rx=0 tx=0).stop 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- 172.21.15.122:0/739324355 shutdown_connections 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993043090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d993040760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- 172.21.15.122:0/739324355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993037b80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- 172.21.15.122:0/739324355 >> 172.21.15.122:0/739324355 conn(0x55d993031ed0 msgr2=0x55d9930322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- 172.21.15.122:0/739324355 shutdown_connections 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- 172.21.15.122:0/739324355 wait complete. 2024-04-02T13:41:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- start start 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 Processor -- start 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d99303db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d99303e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.597+0000 7f43f55e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d993043a90 con 0x55d9930377a0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43f55e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d992f56e80 con 0x55d9930380c0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43f55e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d993043c10 con 0x55d993040ca0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43f55e5740 10 client.0 _ll_register_callbacks cb 0x55d992f92aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d99303db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35228/0 (socket says 172.21.15.122:35228) 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 -- 172.21.15.122:0/3529477288 learned_addr learned my addr 172.21.15.122:0/3529477288 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43f0eb9640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d99303e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 -- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 msgr2=0x55d99303e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d99303e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 -- 172.21.15.122:0/3529477288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 msgr2=0x55d99303db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d99303db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 -- 172.21.15.122:0/3529477288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d992d797d0 con 0x55d9930380c0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43ebfff640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d99303db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43e0012990 tx=0x7f43e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0610e51866489c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e001fb40 con 0x55d9930380c0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f43e001fce0 con 0x55d9930380c0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.598+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43dc018d60 con 0x55d9930380c0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f43e002e640 con 0x55d9930380c0 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 10 client.26843 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d993053ff0 con 0x55d9930380c0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43dc018d60 con 0x55d9930380c0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f43e00294c0 con 0x55d9930380c0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 10 client.26843 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 10 client.26843 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 20 client.26843 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930554b0 0x55d993057900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 4 client.26843 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d992fa8cf0 con 0x55d9930554b0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.599+0000 7f43ebfff640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930554b0 0x55d993057900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.600+0000 7f43cb7fe640 10 client.26843 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.600+0000 7f43ebfff640 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930554b0 0x55d993057900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9bd28291c3df646 server_cookie=5b35e689f28ea60c in_seq=0 out_seq=0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.600+0000 7f43cb7fe640 1 -- 172.21.15.122:0/3529477288 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d992fa8cf0 con 0x55d9930554b0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.600+0000 7f43cb7fe640 10 client.26843 handle_command_reply: tid=0 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.600+0000 7f43f55e5740 1 client.26843 shutdown 2024-04-02T13:41:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 msgr2=0x55d99303e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f43e0012990 tx=0x7f43e0004730 comp rx=0 tx=0).stop 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 shutdown_connections 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930554b0 0x55d993057900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d99303e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43cb7fe640 0 client.26843 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99303e050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 --2- 172.21.15.122:0/3529477288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d99303db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 >> 172.21.15.122:0/3529477288 conn(0x55d993031ed0 msgr2=0x55d993033c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 shutdown_connections 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 1 -- 172.21.15.122:0/3529477288 wait complete. 2024-04-02T13:41:48.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.601+0000 7f43f55e5740 20 client.26843 trim_cache size 0 max 16384 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 Processor -- start 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 -- start start 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9930377a0 0x55d993040240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 0x55d99303caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1acda0 con 0x55d993040ca0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ae9a0 con 0x55d9930380c0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f55e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ad470 con 0x55d9930377a0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f0eb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 0x55d99303caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35240/0 (socket says 172.21.15.122:35240) 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43eb7fe640 1 -- 172.21.15.122:0/2913857842 learned_addr learned my addr 172.21.15.122:0/2913857842 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43f0eb9640 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 0x55d99303caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43752/0 (socket says 172.21.15.122:43752) 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.617+0000 7f43ebfff640 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9930377a0 0x55d993040240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 -- 172.21.15.122:0/2913857842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9930377a0 msgr2=0x55d993040240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9930377a0 0x55d993040240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 -- 172.21.15.122:0/2913857842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 msgr2=0x55d99303caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 0x55d99303caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 -- 172.21.15.122:0/2913857842 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99302d7a0 con 0x55d9930380c0 2024-04-02T13:41:48.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43e0029930 tx=0x7f43e0012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8526d3975b81c2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43d1ffb640 1 -- 172.21.15.122:0/2913857842 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e002e400 con 0x55d9930380c0 2024-04-02T13:41:48.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43d1ffb640 1 -- 172.21.15.122:0/2913857842 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e002ecc0 con 0x55d9930380c0 2024-04-02T13:41:48.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.618+0000 7f43d1ffb640 1 -- 172.21.15.122:0/2913857842 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e001f790 con 0x55d9930380c0 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- 172.21.15.122:0/2913857842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 msgr2=0x55d99304c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43e0029930 tx=0x7f43e0012f90 comp rx=0 tx=0).stop 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- 172.21.15.122:0/2913857842 shutdown_connections 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d993040ca0 0x55d99303caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d99304c950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 --2- 172.21.15.122:0/2913857842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d9930377a0 0x55d993040240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- 172.21.15.122:0/2913857842 >> 172.21.15.122:0/2913857842 conn(0x55d993031ed0 msgr2=0x55d99303c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- 172.21.15.122:0/2913857842 shutdown_connections 2024-04-02T13:41:48.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- 172.21.15.122:0/2913857842 wait complete. 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 -- start start 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.619+0000 7f43f55e5740 1 Processor -- start 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993048e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993039dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1ae210 con 0x55d9930377a0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1adb40 con 0x55d9930380c0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1aad80 con 0x55d993040ca0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 10 client.0 _ll_register_callbacks cb 0x55d993040bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43ebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993048e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f0eb9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993039dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35254/0 (socket says 172.21.15.122:35254) 2024-04-02T13:41:48.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f0eb9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993039dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:60808/0 (socket says 172.21.15.122:60808) 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 -- 172.21.15.122:0/205779099 learned_addr learned my addr 172.21.15.122:0/205779099 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 -- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 msgr2=0x55d993039dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993039dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 -- 172.21.15.122:0/205779099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 msgr2=0x55d993048e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993048e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 -- 172.21.15.122:0/205779099 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99303a3b0 con 0x55d9930380c0 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43eb7fe640 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43e00117b0 tx=0x7f43e0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4d815f959ea93c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0029d50 con 0x55d9930380c0 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43e002ecc0 con 0x55d9930380c0 2024-04-02T13:41:48.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.620+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f43e002e7e0 con 0x55d9930380c0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f43e002e410 con 0x55d9930380c0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 10 client.26849 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d9930bb340 con 0x55d9930380c0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43e002ecc0 con 0x55d9930380c0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f43e001f9c0 con 0x55d9930380c0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 10 client.26849 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 10 client.26849 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 20 client.26849 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930bb870 0x55d9930bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 4 client.26849 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d993038950 con 0x55d9930bb870 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.621+0000 7f43ebfff640 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930bb870 0x55d9930bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43ebfff640 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930bb870 0x55d9930bdc90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e318ecbd6666e10 server_cookie=5a95cbcd5698756b in_seq=0 out_seq=0 2024-04-02T13:41:48.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43cb7fe640 10 client.26849 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43cb7fe640 1 -- 172.21.15.122:0/205779099 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d993038950 con 0x55d9930bb870 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43cb7fe640 10 client.26849 handle_command_reply: tid=0 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 client.26849 shutdown 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 msgr2=0x55d9930493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f43e00117b0 tx=0x7f43e0014000 comp rx=0 tx=0).stop 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 shutdown_connections 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d9930bb870 0x55d9930bdc90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d993040ca0 0x55d993039dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43cb7fe640 0 client.26849 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d9930380c0 0x55d9930493a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 --2- 172.21.15.122:0/205779099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d9930377a0 0x55d993048e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 >> 172.21.15.122:0/205779099 conn(0x55d993031ed0 msgr2=0x55d993030f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 shutdown_connections 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 1 -- 172.21.15.122:0/205779099 wait complete. 2024-04-02T13:41:48.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.622+0000 7f43f55e5740 20 client.26849 trim_cache size 0 max 16384 2024-04-02T13:41:48.632 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 msgr2=0x7f43d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 0x7f43d40a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43ec107130 tx=0x7f43e0005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 msgr2=0x7f43ec1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f43dc00c920 tx=0x7f43dc00cde0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 shutdown_connections 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f43d40a3d20 0x7f43d40a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f43ec107b30 0x7f43ec19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43ec06aa30 0x7f43ec198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 --2- 172.21.15.122:0/2926961008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43ec06a070 0x7f43ec1982e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 >> 172.21.15.122:0/2926961008 conn(0x7f43ec0779e0 msgr2=0x7f43ec0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 shutdown_connections 2024-04-02T13:41:48.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.623+0000 7f43f291c640 1 -- 172.21.15.122:0/2926961008 wait complete. 2024-04-02T13:41:48.636 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3491480376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:48.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:41:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3491480376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:48.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 -- 172.21.15.122:0/3689348304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 msgr2=0x7fcdec107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/3689348304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec107e90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009930 tx=0x7fcde002bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 -- 172.21.15.122:0/3689348304 shutdown_connections 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/3689348304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec102460 0x7fcdec10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/3689348304 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec101aa0 0x7fcdec101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/3689348304 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec107e90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 -- 172.21.15.122:0/3689348304 >> 172.21.15.122:0/3689348304 conn(0x7fcdec0fd530 msgr2=0x7fcdec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 -- 172.21.15.122:0/3689348304 shutdown_connections 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.746+0000 7fcdf2d09640 1 -- 172.21.15.122:0/3689348304 wait complete. 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 Processor -- start 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 -- start start 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 0x7fcdec1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec111e90 con 0x7fcdec107ab0 2024-04-02T13:41:48.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec111d10 con 0x7fcdec102460 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf2d09640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec112010 con 0x7fcdec101aa0 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf0aa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf0aa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:60822/0 (socket says 172.21.15.122:60822) 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/2958300842 learned_addr learned my addr 172.21.15.122:0/2958300842 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43778/0 (socket says 172.21.15.122:43778) 2024-04-02T13:41:48.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.747+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 0x7fcdec1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/2958300842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 msgr2=0x7fcdec1a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 0x7fcdec1a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 msgr2=0x7fcdec10dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/2958300842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec10e340 con 0x7fcdec101aa0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 0x7fcdec1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009230 tx=0x7fcde002cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcde9ffb640 1 -- 172.21.15.122:0/2958300842 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0041070 con 0x7fcdec101aa0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec10e5d0 con 0x7fcdec101aa0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcde9ffb640 1 -- 172.21.15.122:0/2958300842 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde0004450 con 0x7fcdec101aa0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcde9ffb640 1 -- 172.21.15.122:0/2958300842 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde0034440 con 0x7fcdec101aa0 2024-04-02T13:41:48.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.748+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1ab500 con 0x7fcdec101aa0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 Processor -- start 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcde9ffb640 1 -- 172.21.15.122:0/2958300842 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcde0044070 con 0x7fcdec101aa0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 -- start start 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b500c0 0x55c4e7b58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7ab37b0 con 0x55c4e7b500c0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7a52570 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7aaf760 con 0x55c4e7b58ca0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf0aa5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b500c0 0x55c4e7b58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35300/0 (socket says 172.21.15.122:35300) 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 -- 172.21.15.122:0/2745702168 learned_addr learned my addr 172.21.15.122:0/2745702168 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 -- 172.21.15.122:0/2745702168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b58ca0 msgr2=0x55c4e7b5b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 -- 172.21.15.122:0/2745702168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b500c0 msgr2=0x55c4e7b58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b500c0 0x55c4e7b58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcde9ffb640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 0x7fcddc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 -- 172.21.15.122:0/2745702168 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e77c73f0 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 0x7fcddc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcde40107a0 tx=0x7fcde4010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed94d635d6cd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.751+0000 7fcde9ffb640 1 -- 172.21.15.122:0/2958300842 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcde00f2460 con 0x7fcdec101aa0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdd6ffd640 1 -- 172.21.15.122:0/2745702168 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4003b70 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdd6ffd640 1 -- 172.21.15.122:0/2745702168 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4010e50 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdd6ffd640 1 -- 172.21.15.122:0/2745702168 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4004640 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2745702168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 msgr2=0x55c4e7b4fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 0x7fcddc0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55c4e7b4b060 tx=0x7fcdd8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcde40107a0 tx=0x7fcde4010c60 comp rx=0 tx=0).stop 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2745702168 shutdown_connections 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b500c0 0x55c4e7b58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2745702168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b4fb80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2745702168 >> 172.21.15.122:0/2745702168 conn(0x55c4e7b49ed0 msgr2=0x55c4e7b4a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2745702168 shutdown_connections 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.752+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2745702168 wait complete. 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 -- start start 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 Processor -- start 2024-04-02T13:41:48.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b55b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b56060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7b5ba90 con 0x55c4e7b58ca0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7a6ee80 con 0x55c4e7b500c0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7b5bc10 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf59c5740 10 client.0 _ll_register_callbacks cb 0x55c4e7aaaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf0aa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b55b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43786/0 (socket says 172.21.15.122:43786) 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdf12a6640 1 -- 172.21.15.122:0/4276891306 learned_addr learned my addr 172.21.15.122:0/4276891306 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.753+0000 7fcdebfff640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b56060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 msgr2=0x55c4e7b56060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b56060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/4276891306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 msgr2=0x55c4e7b565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 -- 172.21.15.122:0/4276891306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e78917d0 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b55b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ef930 tx=0x7fcde00f5ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc8b9653ee493edf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde004b070 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcde00035f0 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde4010e50 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcde00ba4d0 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf59c5740 10 client.26656 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c4e7b5bb40 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.754+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde4010e50 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fcde00c6960 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 10 client.26656 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 10 client.26656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 20 client.26656 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7b690f0 0x55c4e7b6b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 4 client.26656 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e7ac0cf0 con 0x55c4e7b690f0 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdebfff640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7b690f0 0x55c4e7b6b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdcbfff640 10 client.26656 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.755+0000 7fcdebfff640 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7b690f0 0x55c4e7b6b550 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb413e16a3a4d6fb server_cookie=f0214c4f73b085ee in_seq=0 out_seq=0 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdcbfff640 1 -- 172.21.15.122:0/4276891306 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4e7ac0cf0 con 0x55c4e7b690f0 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdcbfff640 10 client.26656 handle_command_reply: tid=0 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 client.26656 shutdown 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 msgr2=0x55c4e7b55b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b55b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcde00ef930 tx=0x7fcde00f5ba0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 shutdown_connections 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7b690f0 0x55c4e7b6b550 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdcbfff640 0 client.26656 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b56060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/4276891306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b55b20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 >> 172.21.15.122:0/4276891306 conn(0x55c4e7b49ed0 msgr2=0x55c4e7b4bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 shutdown_connections 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 1 -- 172.21.15.122:0/4276891306 wait complete. 2024-04-02T13:41:48.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.756+0000 7fcdf59c5740 20 client.26656 trim_cache size 0 max 16384 2024-04-02T13:41:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3491480376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 Processor -- start 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 -- start start 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b58250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 0x55c4e7b64950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b54b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b55a0 con 0x55c4e7b58ca0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b71a0 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b5c70 con 0x55c4e7b500c0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b54b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 0x55c4e7b64950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b54b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43798/0 (socket says 172.21.15.122:43798) 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf12a6640 1 -- 172.21.15.122:0/2568395991 learned_addr learned my addr 172.21.15.122:0/2568395991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.771+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b58250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 -- 172.21.15.122:0/2568395991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 msgr2=0x55c4e7b58250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b58250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 -- 172.21.15.122:0/2568395991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 msgr2=0x55c4e7b54b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b54b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 -- 172.21.15.122:0/2568395991 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e7b5b970 con 0x55c4e7b500c0 2024-04-02T13:41:48.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdebfff640 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 0x55c4e7b64950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c4e7b6bd90 tx=0x7fcdd801b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b7f9f19214f1c44 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdd67fc640 1 -- 172.21.15.122:0/2568395991 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd800fd80 con 0x55c4e7b500c0 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdd67fc640 1 -- 172.21.15.122:0/2568395991 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd802b900 con 0x55c4e7b500c0 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.772+0000 7fcdd67fc640 1 -- 172.21.15.122:0/2568395991 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd8005020 con 0x55c4e7b500c0 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2568395991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 msgr2=0x55c4e7b64950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 0x55c4e7b64950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55c4e7b6bd90 tx=0x7fcdd801b000 comp rx=0 tx=0).stop 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2568395991 shutdown_connections 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b54b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b500c0 0x55c4e7b64950 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/2568395991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b58250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2568395991 >> 172.21.15.122:0/2568395991 conn(0x55c4e7b49ed0 msgr2=0x55c4e7b543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2568395991 shutdown_connections 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- 172.21.15.122:0/2568395991 wait complete. 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 -- start start 2024-04-02T13:41:48.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.773+0000 7fcdf59c5740 1 Processor -- start 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b60de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b6a10 con 0x55c4e7b58ca0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b6340 con 0x55c4e7b500c0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec1b3580 con 0x55c4e7b4f7a0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf59c5740 10 client.0 _ll_register_callbacks cb 0x7fcde0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43814/0 (socket says 172.21.15.122:43814) 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdebfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35318/0 (socket says 172.21.15.122:35318) 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 -- 172.21.15.122:0/3753658492 learned_addr learned my addr 172.21.15.122:0/3753658492 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b60de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 -- 172.21.15.122:0/3753658492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 msgr2=0x55c4e7b60de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b60de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 -- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 msgr2=0x55c4e7b61320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 -- 172.21.15.122:0/3753658492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e7b5ccc0 con 0x55c4e7b58ca0 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdebfff640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdf12a6640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcde40107a0 tx=0x7fcde4009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37378516df27f996 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:48.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.774+0000 7fcdcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde402c060 con 0x55c4e7b58ca0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdd802b900 con 0x55c4e7b58ca0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7fcde400ed40 con 0x55c4e7b58ca0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcde4033050 con 0x55c4e7b58ca0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 10 client.17679 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c4e7bcf320 con 0x55c4e7b58ca0 2024-04-02T13:41:48.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdd802b900 con 0x55c4e7b58ca0 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fcde4003e00 con 0x55c4e7b58ca0 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 10 client.17679 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 20 client.17679 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7bcf930 0x55c4e7bd1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 4 client.17679 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:41:48.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4e7abb040 con 0x55c4e7bcf930 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.775+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7bcf930 0x55c4e7bd1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf0aa5640 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7bcf930 0x55c4e7bd1d50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebe80a79ee7ec39 server_cookie=51b75d2a73af0834 in_seq=0 out_seq=0 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdcbfff640 10 client.17679 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdcbfff640 1 -- 172.21.15.122:0/3753658492 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4e7abb040 con 0x55c4e7bcf930 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdcbfff640 10 client.17679 handle_command_reply: tid=0 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 client.17679 shutdown 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 msgr2=0x55c4e7b5c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcde40107a0 tx=0x7fcde4009db0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 shutdown_connections 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4e7bcf930 0x55c4e7bd1d50 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4e7b58ca0 0x55c4e7b5c4c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4e7b500c0 0x55c4e7b61320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdcbfff640 0 client.17679 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 --2- 172.21.15.122:0/3753658492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4e7b4f7a0 0x55c4e7b60de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 >> 172.21.15.122:0/3753658492 conn(0x55c4e7b49ed0 msgr2=0x55c4e7b54c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 shutdown_connections 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 1 -- 172.21.15.122:0/3753658492 wait complete. 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.776+0000 7fcdf59c5740 20 client.17679 trim_cache size 0 max 16384 2024-04-02T13:41:48.788 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 msgr2=0x7fcddc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 0x7fcddc0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55c4e7b4b060 tx=0x7fcdd8005f90 comp rx=0 tx=0).stop 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 msgr2=0x7fcdec1a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009230 tx=0x7fcde002cba0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 shutdown_connections 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcddc0a3d20 0x7fcddc0a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdec107ab0 0x7fcdec10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdec102460 0x7fcdec1a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 --2- 172.21.15.122:0/2958300842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdec101aa0 0x7fcdec1a46b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:48.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 >> 172.21.15.122:0/2958300842 conn(0x7fcdec0fd530 msgr2=0x7fcdec0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:48.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 shutdown_connections 2024-04-02T13:41:48.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:48.777+0000 7fcdf2d09640 1 -- 172.21.15.122:0/2958300842 wait complete. 2024-04-02T13:41:48.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 193156 2.66 MB/sec execute 189 sec latency 3073.414 ms 2024-04-02T13:41:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:49 smithi187 ceph-mon[36064]: pgmap v202: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:49 smithi188 ceph-mon[36117]: pgmap v202: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:49.792 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:41:49.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:41:49.792 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:41:49.817 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:49 smithi122 ceph-mon[32114]: pgmap v202: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.912+0000 7f7ba7975640 1 -- 172.21.15.122:0/2305848034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 msgr2=0x7f7ba0114710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.912+0000 7f7ba7975640 1 --2- 172.21.15.122:0/2305848034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba0114710 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000dd10 tx=0x7f7b9002b990 comp rx=0 tx=0).stop 2024-04-02T13:41:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 -- 172.21.15.122:0/2305848034 shutdown_connections 2024-04-02T13:41:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 --2- 172.21.15.122:0/2305848034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0114c50 0x7f7ba0117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 --2- 172.21.15.122:0/2305848034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba0114710 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 --2- 172.21.15.122:0/2305848034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0108ce0 0x7f7ba01090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 -- 172.21.15.122:0/2305848034 >> 172.21.15.122:0/2305848034 conn(0x7f7ba006cbe0 msgr2=0x7f7ba006cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 -- 172.21.15.122:0/2305848034 shutdown_connections 2024-04-02T13:41:49.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 -- 172.21.15.122:0/2305848034 wait complete. 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 Processor -- start 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.913+0000 7f7ba7975640 1 -- start start 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0108ce0 0x7f7ba01a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba01a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01184b0 con 0x7f7ba0108ce0 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0118330 con 0x7f7ba0114c50 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba7975640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0118630 con 0x7f7ba0109600 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0108ce0 0x7f7ba01a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba01a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35334/0 (socket says 172.21.15.122:35334) 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 -- 172.21.15.122:0/755260096 learned_addr learned my addr 172.21.15.122:0/755260096 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 -- 172.21.15.122:0/755260096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 msgr2=0x7f7ba01a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba01a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 -- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0108ce0 msgr2=0x7f7ba01a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0108ce0 0x7f7ba01a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.914+0000 7f7ba5f12640 1 -- 172.21.15.122:0/755260096 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba019f000 con 0x7f7ba0114c50 2024-04-02T13:41:49.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.915+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c014340 tx=0x7f7b9c014800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.915+0000 7f7b967fc640 1 -- 172.21.15.122:0/755260096 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c004020 con 0x7f7ba0114c50 2024-04-02T13:41:49.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.915+0000 7f7b967fc640 1 -- 172.21.15.122:0/755260096 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c0041c0 con 0x7f7ba0114c50 2024-04-02T13:41:49.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.915+0000 7f7b967fc640 1 -- 172.21.15.122:0/755260096 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0035f0 con 0x7f7ba0114c50 2024-04-02T13:41:49.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.916+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba019f290 con 0x7f7ba0114c50 2024-04-02T13:41:49.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.916+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01ab510 con 0x7f7ba0114c50 2024-04-02T13:41:49.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.921+0000 7f7baa638740 1 Processor -- start 2024-04-02T13:41:49.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.921+0000 7f7b967fc640 1 -- 172.21.15.122:0/755260096 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7b9c023030 con 0x7f7ba0114c50 2024-04-02T13:41:49.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.922+0000 7f7baa638740 1 -- start start 2024-04-02T13:41:49.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.922+0000 7f7b967fc640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 0x7f7b8c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 0x556fb44146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb441d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7b967fc640 1 -- 172.21.15.122:0/755260096 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b9c0d82b0 con 0x7f7ba0114c50 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba4f10640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 0x7f7b8c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 0x556fb44146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 0x556fb441fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556fb4378770 con 0x556fb4414be0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556fb4320960 con 0x556fb441d7c0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 0x556fb44146a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:60886/0 (socket says 172.21.15.122:60886) 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb441d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba5711640 1 -- 172.21.15.122:0/3547706548 learned_addr learned my addr 172.21.15.122:0/3547706548 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7baa638740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556fb43748c0 con 0x556fb44142c0 2024-04-02T13:41:49.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.923+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 0x556fb441fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3547706548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 msgr2=0x556fb44146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 0x556fb44146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3547706548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 msgr2=0x556fb441d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb441d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3547706548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb41547e0 con 0x556fb441d7c0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba4f10640 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb441d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba4f10640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 0x7f7b8c0a63b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ba01169c0 tx=0x7f7b9000d4c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 0x556fb441fbb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0d5610 tx=0x7f7b9c0a3aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2faed6164017d891 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.924+0000 7f7b8b7fe640 1 -- 172.21.15.122:0/3547706548 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0bc020 con 0x556fb441d7c0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.925+0000 7f7b8b7fe640 1 -- 172.21.15.122:0/3547706548 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c0a3e60 con 0x556fb441d7c0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.925+0000 7f7b8b7fe640 1 -- 172.21.15.122:0/3547706548 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0a04d0 con 0x556fb441d7c0 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.925+0000 7f7baa638740 1 -- 172.21.15.122:0/3547706548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 msgr2=0x556fb441fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.925+0000 7f7baa638740 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 0x556fb441fbb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0d5610 tx=0x7f7b9c0a3aa0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 -- 172.21.15.122:0/3547706548 shutdown_connections 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb441d7c0 0x556fb441fbb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb441d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 --2- 172.21.15.122:0/3547706548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb44142c0 0x556fb44146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 -- 172.21.15.122:0/3547706548 >> 172.21.15.122:0/3547706548 conn(0x556fb440e9f0 msgr2=0x556fb440edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 -- 172.21.15.122:0/3547706548 shutdown_connections 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.926+0000 7f7baa638740 1 -- 172.21.15.122:0/3547706548 wait complete. 2024-04-02T13:41:49.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.927+0000 7f7baa638740 1 -- start start 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.927+0000 7f7baa638740 1 Processor -- start 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.927+0000 7f7baa638740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.927+0000 7f7baa638740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb441ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7baa638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb441b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7baa638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556fb44205b0 con 0x556fb441d7c0 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7baa638740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556fb4333d20 con 0x556fb44142c0 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7baa638740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556fb4420730 con 0x556fb4414be0 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7baa638740 10 client.0 _ll_register_callbacks cb 0x556fb4300a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35350/0 (socket says 172.21.15.122:35350) 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5711640 1 -- 172.21.15.122:0/1977594676 learned_addr learned my addr 172.21.15.122:0/1977594676 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:49.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba4f10640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb441ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb441b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 -- 172.21.15.122:0/1977594676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 msgr2=0x556fb441ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb441ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 -- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 msgr2=0x556fb441a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 -- 172.21.15.122:0/1977594676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb41b4f70 con 0x556fb441d7c0 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb441b2a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0acc90 tx=0x7f7b9c011100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e1a47a0a66c105 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:49.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.928+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0bc050 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b9c0d5890 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b9c0a3e60 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c0d5a90 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 10 client.17691 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556fb404c170 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c0a3e60 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7b9c0a2d20 con 0x556fb441d7c0 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 10 client.17691 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 10 client.17691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 20 client.17691 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb442da00 0x556fb442fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 4 client.17691 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:49.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fb4385cb0 con 0x556fb442da00 2024-04-02T13:41:49.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.929+0000 7f7ba5711640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb442da00 0x556fb442fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.930+0000 7f7b88ff9640 10 client.17691 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:49.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.930+0000 7f7ba5711640 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb442da00 0x556fb442fdf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34890546cd4f94b6 server_cookie=ff962912b87070cc in_seq=0 out_seq=0 2024-04-02T13:41:49.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7b88ff9640 1 -- 172.21.15.122:0/1977594676 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fb4385cb0 con 0x556fb442da00 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7b88ff9640 10 client.17691 handle_command_reply: tid=0 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 client.17691 shutdown 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 msgr2=0x556fb441b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb441b2a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0acc90 tx=0x7f7b9c011100 comp rx=0 tx=0).stop 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 shutdown_connections 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb442da00 0x556fb442fdf0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb441b2a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb441ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7b88ff9640 0 client.17691 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:49.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 --2- 172.21.15.122:0/1977594676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 >> 172.21.15.122:0/1977594676 conn(0x556fb440e9f0 msgr2=0x556fb4410740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 shutdown_connections 2024-04-02T13:41:49.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 1 -- 172.21.15.122:0/1977594676 wait complete. 2024-04-02T13:41:49.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.931+0000 7f7baa638740 20 client.17691 trim_cache size 0 max 16384 2024-04-02T13:41:49.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.955+0000 7f7baa638740 1 Processor -- start 2024-04-02T13:41:49.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.955+0000 7f7baa638740 1 -- start start 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb4429470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb44354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b5620 con 0x556fb441d7c0 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b7220 con 0x556fb44142c0 2024-04-02T13:41:49.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7baa638740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b5cf0 con 0x556fb4414be0 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb4429470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb4429470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:60908/0 (socket says 172.21.15.122:60908) 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7ba4f10640 1 -- 172.21.15.122:0/3858019692 learned_addr learned my addr 172.21.15.122:0/3858019692 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.956+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb44354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 -- 172.21.15.122:0/3858019692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 msgr2=0x556fb4429470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb4429470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 -- 172.21.15.122:0/3858019692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 msgr2=0x556fb44354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb44354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 -- 172.21.15.122:0/3858019692 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fb440f370 con 0x556fb44142c0 2024-04-02T13:41:49.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 195313 2.71 MB/sec execute 190 sec latency 22.736 ms 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.957+0000 7f7ba5711640 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441cee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x556fb44303f0 tx=0x7f7b98009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84bc399b2fe54b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7b8affd640 1 -- 172.21.15.122:0/3858019692 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98004d20 con 0x556fb44142c0 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7b8affd640 1 -- 172.21.15.122:0/3858019692 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b98005500 con 0x556fb44142c0 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7b8affd640 1 -- 172.21.15.122:0/3858019692 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980108c0 con 0x556fb44142c0 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 -- 172.21.15.122:0/3858019692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 msgr2=0x556fb441cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441cee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x556fb44303f0 tx=0x7f7b98009500 comp rx=0 tx=0).stop 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 -- 172.21.15.122:0/3858019692 shutdown_connections 2024-04-02T13:41:49.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb441d7c0 0x556fb44354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb4414be0 0x556fb4429470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.958+0000 7f7baa638740 1 --2- 172.21.15.122:0/3858019692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb441cee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.959+0000 7f7baa638740 1 -- 172.21.15.122:0/3858019692 >> 172.21.15.122:0/3858019692 conn(0x556fb440e9f0 msgr2=0x556fb441b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.959+0000 7f7baa638740 1 -- 172.21.15.122:0/3858019692 shutdown_connections 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.959+0000 7f7baa638740 1 -- 172.21.15.122:0/3858019692 wait complete. 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 -- start start 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 Processor -- start 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb4425d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb44220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 0x556fb4422610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b6a90 con 0x556fb4414be0 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b63c0 con 0x556fb44142c0 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba01b3600 con 0x556fb441d7c0 2024-04-02T13:41:49.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.960+0000 7f7baa638740 10 client.0 _ll_register_callbacks cb 0x7f7b9c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:49.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 0x556fb4422610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb4425d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb44220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb4425d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35372/0 (socket says 172.21.15.122:35372) 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba4f10640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb44220d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43854/0 (socket says 172.21.15.122:43854) 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5711640 1 -- 172.21.15.122:0/3929827097 learned_addr learned my addr 172.21.15.122:0/3929827097 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 msgr2=0x556fb4425d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb4425d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3929827097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 msgr2=0x556fb44220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb44220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 -- 172.21.15.122:0/3929827097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fb4422df0 con 0x556fb441d7c0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7ba5f12640 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 0x556fb4422610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0a07e0 tx=0x7f7b9c0bd4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e38f76123103067 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.961+0000 7f7b88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0acad0 con 0x556fb441d7c0 2024-04-02T13:41:49.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f7b9c004370 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b98005500 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c0a4560 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7baa638740 10 client.26680 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.962+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556fb4494d80 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b98005500 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f7b9c0a3450 con 0x556fb441d7c0 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 10 client.26680 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 20 client.26680 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb4495390 0x556fb44977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 4 client.26680 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.963+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556fb4380000 con 0x556fb4495390 2024-04-02T13:41:49.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.964+0000 7f7ba5711640 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb4495390 0x556fb44977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.965+0000 7f7ba5711640 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb4495390 0x556fb44977b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709ee1dadb821612 server_cookie=9b882c65241730ec in_seq=0 out_seq=0 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.965+0000 7f7b88ff9640 10 client.26680 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7b88ff9640 1 -- 172.21.15.122:0/3929827097 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556fb4380000 con 0x556fb4495390 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7b88ff9640 10 client.26680 handle_command_reply: tid=0 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 client.26680 shutdown 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 msgr2=0x556fb4422610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 0x556fb4422610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0a07e0 tx=0x7f7b9c0bd4b0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 shutdown_connections 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556fb4495390 0x556fb44977b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556fb441d7c0 0x556fb4422610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556fb4414be0 0x556fb44220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7b88ff9640 0 client.26680 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 --2- 172.21.15.122:0/3929827097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556fb44142c0 0x556fb4425d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 >> 172.21.15.122:0/3929827097 conn(0x556fb440e9f0 msgr2=0x556fb441a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 shutdown_connections 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 1 -- 172.21.15.122:0/3929827097 wait complete. 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.966+0000 7f7baa638740 20 client.26680 trim_cache size 0 max 16384 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:49.971 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:41:49.972 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "20f6b675-efe7-4ff0-b40a-25dcd5fca171", 2024-04-02T13:41:49.972 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:41:49.972 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 msgr2=0x7f7b8c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 0x7f7b8c0a63b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ba01169c0 tx=0x7f7b9000d4c0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 msgr2=0x7f7ba019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c014340 tx=0x7f7b9c014800 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 shutdown_connections 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7b8c0a3ef0 0x7f7b8c0a63b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba0114c50 0x7f7ba019e830 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ba0109600 0x7f7ba01a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 --2- 172.21.15.122:0/755260096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ba0108ce0 0x7f7ba01a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.967+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 >> 172.21.15.122:0/755260096 conn(0x7f7ba006cbe0 msgr2=0x7f7ba0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.968+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 shutdown_connections 2024-04-02T13:41:49.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:49.968+0000 7f7ba7975640 1 -- 172.21.15.122:0/755260096 wait complete. 2024-04-02T13:41:49.982 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:41:49.982 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:41:50.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 -- 172.21.15.122:0/113603409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960101a90 msgr2=0x7f6960101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 --2- 172.21.15.122:0/113603409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960101a90 0x7f6960101f10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6954009930 tx=0x7f695402bc30 comp rx=0 tx=0).stop 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 -- 172.21.15.122:0/113603409 shutdown_connections 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 --2- 172.21.15.122:0/113603409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960102450 0x7f696010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 --2- 172.21.15.122:0/113603409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960101a90 0x7f6960101f10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 --2- 172.21.15.122:0/113603409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f6960107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 -- 172.21.15.122:0/113603409 >> 172.21.15.122:0/113603409 conn(0x7f6960078ec0 msgr2=0x7f69600792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 -- 172.21.15.122:0/113603409 shutdown_connections 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.130+0000 7f6965493640 1 -- 172.21.15.122:0/113603409 wait complete. 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 Processor -- start 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 -- start start 2024-04-02T13:41:50.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960101a90 0x7f69601983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f696019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111eb0 con 0x7f6960102450 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6960111d30 con 0x7f6960101a90 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f6965493640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6960112030 con 0x7f6960107aa0 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43872/0 (socket says 172.21.15.122:43872) 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 -- 172.21.15.122:0/4040328623 learned_addr learned my addr 172.21.15.122:0/4040328623 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695effd640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960101a90 0x7f69601983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695f7fe640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f696019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 -- 172.21.15.122:0/4040328623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 msgr2=0x7f696019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f696019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 -- 172.21.15.122:0/4040328623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960101a90 msgr2=0x7f69601983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960101a90 0x7f69601983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695e7fc640 1 -- 172.21.15.122:0/4040328623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696019d3c0 con 0x7f6960102450 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.131+0000 7f695f7fe640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f696019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f695e7fc640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6954009230 tx=0x7f695402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f6947fff640 1 -- 172.21.15.122:0/4040328623 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954041070 con 0x7f6960102450 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696019d650 con 0x7f6960102450 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f6947fff640 1 -- 172.21.15.122:0/4040328623 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6954004450 con 0x7f6960102450 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f6947fff640 1 -- 172.21.15.122:0/4040328623 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954034440 con 0x7f6960102450 2024-04-02T13:41:50.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.132+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69601a4f40 con 0x7f6960102450 2024-04-02T13:41:50.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.134+0000 7f6947fff640 1 -- 172.21.15.122:0/4040328623 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6954044070 con 0x7f6960102450 2024-04-02T13:41:50.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 Processor -- start 2024-04-02T13:41:50.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 -- start start 2024-04-02T13:41:50.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6947fff640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786244cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55678624ddd0 0x5567862501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567861a8830 con 0x5567862451f0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556786150820 con 0x5567862448d0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6968084740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5567861a47e0 con 0x55678624ddd0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695f7fe640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f6947fff640 1 -- 172.21.15.122:0/4040328623 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f69540f2570 con 0x7f6960102450 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43882/0 (socket says 172.21.15.122:43882) 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 -- 172.21.15.122:0/3055678064 learned_addr learned my addr 172.21.15.122:0/3055678064 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695f7fe640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55678624ddd0 0x5567862501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 -- 172.21.15.122:0/3055678064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55678624ddd0 msgr2=0x5567862501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55678624ddd0 0x5567862501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 -- 172.21.15.122:0/3055678064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 msgr2=0x556786244cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695effd640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786244cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786244cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695e7fc640 1 -- 172.21.15.122:0/3055678064 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556785f867d0 con 0x5567862451f0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.135+0000 7f695effd640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786244cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f695f7fe640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 0x7f69480a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6958009370 tx=0x7f6958009f00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f695e7fc640 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f69540be220 tx=0x7f6954009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661569dc775e2032 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f69457fa640 1 -- 172.21.15.122:0/3055678064 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695404b070 con 0x5567862451f0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f69457fa640 1 -- 172.21.15.122:0/3055678064 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69540be990 con 0x5567862451f0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f69457fa640 1 -- 172.21.15.122:0/3055678064 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69540bc660 con 0x5567862451f0 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 -- 172.21.15.122:0/3055678064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 msgr2=0x55678624d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f69540be220 tx=0x7f6954009260 comp rx=0 tx=0).stop 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 -- 172.21.15.122:0/3055678064 shutdown_connections 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55678624ddd0 0x5567862501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862451f0 0x55678624d890 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 --2- 172.21.15.122:0/3055678064 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786244cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.136+0000 7f6968084740 1 -- 172.21.15.122:0/3055678064 >> 172.21.15.122:0/3055678064 conn(0x55678623f000 msgr2=0x55678623f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- 172.21.15.122:0/3055678064 shutdown_connections 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- 172.21.15.122:0/3055678064 wait complete. 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- start start 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 Processor -- start 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 0x55678624d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862451f0 0x55678624a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5567861a8830 con 0x55678624ddd0 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556786163d30 con 0x5567862451f0 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556786250980 con 0x5567862448d0 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f6968084740 10 client.0 _ll_register_callbacks cb 0x556786163720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f695effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 0x55678624d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43894/0 (socket says 172.21.15.122:43894) 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f695f7fe640 1 -- 172.21.15.122:0/1586851976 learned_addr learned my addr 172.21.15.122:0/1586851976 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.137+0000 7f695e7fc640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862451f0 0x55678624a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 -- 172.21.15.122:0/1586851976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 msgr2=0x55678624d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 0x55678624d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 -- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862451f0 msgr2=0x55678624a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862451f0 0x55678624a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 -- 172.21.15.122:0/1586851976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556785ebc3f0 con 0x55678624ddd0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695effd640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 0x55678624d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6958014370 tx=0x7f6958004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635dfabc507a5c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958029400 con 0x55678624ddd0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69540be990 con 0x55678624ddd0 2024-04-02T13:41:50.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f69580213f0 con 0x55678624ddd0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f69580215f0 con 0x55678624ddd0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 10 client.17709 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556785e7c0f0 con 0x55678624ddd0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69540be990 con 0x55678624ddd0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f69580297d0 con 0x55678624ddd0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 10 client.17709 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 10 client.17709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 20 client.17709 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55678625d8e0 0x55678625fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 4 client.17709 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.138+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567861b5d70 con 0x55678625d8e0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.139+0000 7f695effd640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55678625d8e0 0x55678625fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.139+0000 7f6942ff5640 10 client.17709 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.139+0000 7f695effd640 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55678625d8e0 0x55678625fd80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d604c1bf4c886be0 server_cookie=28d88105989bc949 in_seq=0 out_seq=0 2024-04-02T13:41:50.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6942ff5640 1 -- 172.21.15.122:0/1586851976 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567861b5d70 con 0x55678625d8e0 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6942ff5640 10 client.17709 handle_command_reply: tid=0 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 client.17709 shutdown 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 msgr2=0x55678624ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6958014370 tx=0x7f6958004ab0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 shutdown_connections 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55678625d8e0 0x55678625fd80 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x55678624ae80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6942ff5640 0 client.17709 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862451f0 0x55678624a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 --2- 172.21.15.122:0/1586851976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862448d0 0x55678624d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 >> 172.21.15.122:0/1586851976 conn(0x55678623f000 msgr2=0x556786240d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 shutdown_connections 2024-04-02T13:41:50.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 1 -- 172.21.15.122:0/1586851976 wait complete. 2024-04-02T13:41:50.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.140+0000 7f6968084740 20 client.17709 trim_cache size 0 max 16384 2024-04-02T13:41:50.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 Processor -- start 2024-04-02T13:41:50.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 -- start start 2024-04-02T13:41:50.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x55678624d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55678624ddd0 0x55678624d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69601af010 con 0x5567862448d0 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69601b0c10 con 0x55678624ddd0 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f6968084740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69601af6e0 con 0x5567862451f0 2024-04-02T13:41:50.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43898/0 (socket says 172.21.15.122:43898) 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 -- 172.21.15.122:0/1235065622 learned_addr learned my addr 172.21.15.122:0/1235065622 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695e7fc640 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x55678624d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55678624ddd0 0x55678624d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 -- 172.21.15.122:0/1235065622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 msgr2=0x55678624d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x55678624d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 -- 172.21.15.122:0/1235065622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55678624ddd0 msgr2=0x55678624d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55678624ddd0 0x55678624d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 -- 172.21.15.122:0/1235065622 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55678623ee30 con 0x5567862448d0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.159+0000 7f695effd640 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x5567862605c0 tx=0x7f69500058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783036eb6a0c261a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6944ff9640 1 -- 172.21.15.122:0/1235065622 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950003d60 con 0x5567862448d0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6944ff9640 1 -- 172.21.15.122:0/1235065622 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695001e530 con 0x5567862448d0 2024-04-02T13:41:50.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6944ff9640 1 -- 172.21.15.122:0/1235065622 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950012a90 con 0x5567862448d0 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 -- 172.21.15.122:0/1235065622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 msgr2=0x55678624a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x5567862605c0 tx=0x7f69500058f0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 -- 172.21.15.122:0/1235065622 shutdown_connections 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55678624ddd0 0x55678624d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x55678624d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 --2- 172.21.15.122:0/1235065622 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567862448d0 0x55678624a220 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 -- 172.21.15.122:0/1235065622 >> 172.21.15.122:0/1235065622 conn(0x55678623f000 msgr2=0x556786240cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 -- 172.21.15.122:0/1235065622 shutdown_connections 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.160+0000 7f6968084740 1 -- 172.21.15.122:0/1235065622 wait complete. 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 -- start start 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 Processor -- start 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x556786256110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f69601b0480 con 0x55678624ddd0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69601afdb0 con 0x5567862448d0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69601acff0 con 0x5567862451f0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f6968084740 10 client.0 _ll_register_callbacks cb 0x556786163720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x556786256110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43908/0 (socket says 172.21.15.122:43908) 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 -- 172.21.15.122:0/1278274266 learned_addr learned my addr 172.21.15.122:0/1278274266 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35436/0 (socket says 172.21.15.122:35436) 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 -- 172.21.15.122:0/1278274266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 msgr2=0x556786256110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x556786256110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 -- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 msgr2=0x556786255bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:41:50.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695f7fe640 1 -- 172.21.15.122:0/1278274266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556786247180 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.161+0000 7f695effd640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f695f7fe640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6958004a80 tx=0x7f6958029630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a881ae3d159610 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6942ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958029e60 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9437+0+0 (secure 0 0 0) 0x7f6958029930 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695001e530 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f6958009910 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6968084740 10 client.17715 fetch_fsmap learned FSMap version 38 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5567862c3750 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.162+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695001e530 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f6958036770 con 0x55678624ddd0 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 10 client.17715 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:41:50.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 20 client.17715 populate_metadata read hostname 'smithi122' 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567862c3f60 0x5567862c6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 4 client.17715 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567861b00c0 con 0x5567862c3f60 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f695effd640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567862c3f60 0x5567862c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f695effd640 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567862c3f60 0x5567862c6380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4252ae077c537a server_cookie=20b839f73aeacb42 in_seq=0 out_seq=0 2024-04-02T13:41:50.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6942ff5640 10 client.17715 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:50.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6942ff5640 1 -- 172.21.15.122:0/1278274266 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5567861b00c0 con 0x5567862c3f60 2024-04-02T13:41:50.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6942ff5640 10 client.17715 handle_command_reply: tid=0 2024-04-02T13:41:50.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.163+0000 7f6968084740 1 client.17715 shutdown 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 msgr2=0x556786246bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6958004a80 tx=0x7f6958029630 comp rx=0 tx=0).stop 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 shutdown_connections 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567862c3f60 0x5567862c6380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55678624ddd0 0x556786246bf0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567862451f0 0x556786256110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6942ff5640 0 client.17715 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 --2- 172.21.15.122:0/1278274266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567862448d0 0x556786255bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 >> 172.21.15.122:0/1278274266 conn(0x55678623f000 msgr2=0x556786240430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 shutdown_connections 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 1 -- 172.21.15.122:0/1278274266 wait complete. 2024-04-02T13:41:50.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6968084740 20 client.17715 trim_cache size 0 max 16384 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "20f6b675-efe7-4ff0-b40a-25dcd5fca171": { 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "20f6b675-efe7-4ff0-b40a-25dcd5fca171", 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:41:50.175 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:41:50.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 msgr2=0x7f69480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 0x7f69480a61e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6958009370 tx=0x7f6958009f00 comp rx=0 tx=0).stop 2024-04-02T13:41:50.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 msgr2=0x7f6960198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:41:50.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6954009230 tx=0x7f695402cc70 comp rx=0 tx=0).stop 2024-04-02T13:41:50.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 shutdown_connections 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f69480a3d20 0x7f69480a61e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6960107aa0 0x7f696019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6960102450 0x7f6960198910 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 --2- 172.21.15.122:0/4040328623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6960101a90 0x7f69601983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 >> 172.21.15.122:0/4040328623 conn(0x7f6960078ec0 msgr2=0x7f6960104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 shutdown_connections 2024-04-02T13:41:50.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:41:50.164+0000 7f6965493640 1 -- 172.21.15.122:0/4040328623 wait complete. 2024-04-02T13:41:50.178 INFO:tasks.cephfs.filesystem:scrub status for tag:20f6b675-efe7-4ff0-b40a-25dcd5fca171 - {'path': '/', 'tag': '20f6b675-efe7-4ff0-b40a-25dcd5fca171', 'options': 'recursive,force'} 2024-04-02T13:41:50.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 195819 2.70 MB/sec execute 191 sec latency 407.176 ms 2024-04-02T13:41:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: pgmap v203: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:41:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:41:51.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: pgmap v203: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: pgmap v203: 129 pgs: 129 active+clean; 128 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 739 KiB/s wr, 95 op/s 2024-04-02T13:41:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:41:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:41:51.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 195819 2.69 MB/sec execute 192 sec latency 1407.305 ms 2024-04-02T13:41:52.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 195819 2.67 MB/sec execute 193 sec latency 2407.385 ms 2024-04-02T13:41:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:53 smithi187 ceph-mon[36064]: pgmap v204: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-02T13:41:53.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:53 smithi188 ceph-mon[36117]: pgmap v204: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-02T13:41:53.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:53 smithi122 ceph-mon[32114]: pgmap v204: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-02T13:41:53.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 196782 2.69 MB/sec execute 194 sec latency 2853.557 ms 2024-04-02T13:41:54.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 198941 2.74 MB/sec execute 195 sec latency 13.339 ms 2024-04-02T13:41:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:55 smithi187 ceph-mon[36064]: pgmap v205: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-04-02T13:41:55.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:55 smithi188 ceph-mon[36117]: pgmap v205: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-04-02T13:41:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:55 smithi122 ceph-mon[32114]: pgmap v205: 129 pgs: 129 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-04-02T13:41:55.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199305 2.73 MB/sec execute 196 sec latency 607.673 ms 2024-04-02T13:41:56.960 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199305 2.72 MB/sec execute 197 sec latency 1607.784 ms 2024-04-02T13:41:57.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: pgmap v206: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-02T13:41:57.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:57.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:57.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:57.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:57.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: pgmap v206: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:41:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: pgmap v206: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-04-02T13:41:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:41:57.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199305 2.70 MB/sec execute 198 sec latency 2607.859 ms 2024-04-02T13:41:58.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199429 2.69 MB/sec execute 199 sec latency 3057.740 ms 2024-04-02T13:41:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:41:59 smithi187 ceph-mon[36064]: pgmap v207: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:41:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:41:59 smithi188 ceph-mon[36117]: pgmap v207: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:41:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:41:59 smithi122 ceph-mon[32114]: pgmap v207: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:41:59.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199429 2.68 MB/sec execute 200 sec latency 1437.626 ms 2024-04-02T13:42:00.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199495 2.67 MB/sec execute 201 sec latency 2398.268 ms 2024-04-02T13:42:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:01 smithi187 ceph-mon[36064]: pgmap v208: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:42:01.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:01 smithi188 ceph-mon[36117]: pgmap v208: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:42:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:01 smithi122 ceph-mon[32114]: pgmap v208: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 899 KiB/s wr, 102 op/s 2024-04-02T13:42:01.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 199878 2.66 MB/sec execute 202 sec latency 61.384 ms 2024-04-02T13:42:02.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 202008 2.71 MB/sec execute 203 sec latency 14.192 ms 2024-04-02T13:42:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:03 smithi187 ceph-mon[36064]: pgmap v209: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 958 KiB/s wr, 117 op/s 2024-04-02T13:42:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:03 smithi187 ceph-mon[36064]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:42:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:03 smithi188 ceph-mon[36117]: pgmap v209: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 958 KiB/s wr, 117 op/s 2024-04-02T13:42:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:03 smithi188 ceph-mon[36117]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:42:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:03 smithi122 ceph-mon[32114]: pgmap v209: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 958 KiB/s wr, 117 op/s 2024-04-02T13:42:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:03 smithi122 ceph-mon[32114]: osdmap e96: 12 total, 12 up, 12 in 2024-04-02T13:42:03.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 203258 2.73 MB/sec execute 204 sec latency 14.433 ms 2024-04-02T13:42:04.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 205160 2.76 MB/sec execute 205 sec latency 21.883 ms 2024-04-02T13:42:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:05 smithi187 ceph-mon[36064]: pgmap v211: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 387 KiB/s wr, 69 op/s 2024-04-02T13:42:05.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:05 smithi188 ceph-mon[36117]: pgmap v211: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 387 KiB/s wr, 69 op/s 2024-04-02T13:42:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:05 smithi122 ceph-mon[32114]: pgmap v211: 129 pgs: 129 active+clean; 134 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 387 KiB/s wr, 69 op/s 2024-04-02T13:42:05.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 206474 2.78 MB/sec execute 206 sec latency 11.933 ms 2024-04-02T13:42:06.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 206520 2.77 MB/sec execute 207 sec latency 959.649 ms 2024-04-02T13:42:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:07 smithi187 ceph-mon[36064]: pgmap v212: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:07 smithi188 ceph-mon[36117]: pgmap v212: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:07 smithi122 ceph-mon[32114]: pgmap v212: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:07.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 206520 2.76 MB/sec execute 208 sec latency 1959.688 ms 2024-04-02T13:42:08.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 207451 2.77 MB/sec execute 209 sec latency 2405.243 ms 2024-04-02T13:42:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:09 smithi187 ceph-mon[36064]: pgmap v213: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:09 smithi188 ceph-mon[36117]: pgmap v213: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:09 smithi122 ceph-mon[32114]: pgmap v213: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:09.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 209577 2.82 MB/sec execute 210 sec latency 27.233 ms 2024-04-02T13:42:10.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 210918 2.83 MB/sec execute 211 sec latency 14.179 ms 2024-04-02T13:42:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:11 smithi187 ceph-mon[36064]: pgmap v214: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:11 smithi187 ceph-mon[36064]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:42:11.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:11 smithi188 ceph-mon[36117]: pgmap v214: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:11 smithi188 ceph-mon[36117]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:42:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:11 smithi122 ceph-mon[32114]: pgmap v214: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-02T13:42:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:11 smithi122 ceph-mon[32114]: osdmap e97: 12 total, 12 up, 12 in 2024-04-02T13:42:11.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 213103 2.88 MB/sec execute 212 sec latency 13.579 ms 2024-04-02T13:42:12.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 214469 2.90 MB/sec execute 213 sec latency 13.908 ms 2024-04-02T13:42:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:13 smithi187 ceph-mon[36064]: pgmap v216: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:42:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:13 smithi188 ceph-mon[36117]: pgmap v216: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:42:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:13 smithi122 ceph-mon[32114]: pgmap v216: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-02T13:42:13.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 216683 2.95 MB/sec execute 214 sec latency 14.661 ms 2024-04-02T13:42:14.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 217145 2.93 MB/sec execute 215 sec latency 478.392 ms 2024-04-02T13:42:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:15 smithi187 ceph-mon[36064]: pgmap v217: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-02T13:42:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:15 smithi188 ceph-mon[36117]: pgmap v217: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-02T13:42:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:15 smithi122 ceph-mon[32114]: pgmap v217: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-02T13:42:15.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 217145 2.92 MB/sec execute 216 sec latency 1478.492 ms 2024-04-02T13:42:16.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 217145 2.91 MB/sec execute 217 sec latency 2478.582 ms 2024-04-02T13:42:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:17 smithi187 ceph-mon[36064]: pgmap v218: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:17 smithi188 ceph-mon[36117]: pgmap v218: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:17 smithi122 ceph-mon[32114]: pgmap v218: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:17.961 INFO:tasks.workunit.client.0.smithi122.stdout: 1 217145 2.89 MB/sec execute 218 sec latency 3478.633 ms 2024-04-02T13:42:18.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 218114 2.90 MB/sec execute 219 sec latency 3925.973 ms 2024-04-02T13:42:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:19 smithi187 ceph-mon[36064]: pgmap v219: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:19.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:19 smithi188 ceph-mon[36117]: pgmap v219: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:19 smithi122 ceph-mon[32114]: pgmap v219: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:19.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 220260 2.95 MB/sec execute 220 sec latency 16.079 ms 2024-04-02T13:42:20.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:20.180 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 -- 172.21.15.122:0/2486359586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 msgr2=0x7f3e3c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 --2- 172.21.15.122:0/2486359586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c100e20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 -- 172.21.15.122:0/2486359586 shutdown_connections 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 --2- 172.21.15.122:0/2486359586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c101360 0x7f3e3c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 --2- 172.21.15.122:0/2486359586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c100e20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 --2- 172.21.15.122:0/2486359586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c1058a0 0x7f3e3c105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 -- 172.21.15.122:0/2486359586 >> 172.21.15.122:0/2486359586 conn(0x7f3e3c077a90 msgr2=0x7f3e3c077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 -- 172.21.15.122:0/2486359586 shutdown_connections 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.287+0000 7f3e439cd640 1 -- 172.21.15.122:0/2486359586 wait complete. 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 Processor -- start 2024-04-02T13:42:20.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 -- start start 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c101360 0x7f3e3c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c1058a0 0x7f3e3c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c110d60 con 0x7f3e3c1009c0 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c110be0 con 0x7f3e3c101360 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e439cd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c110ee0 con 0x7f3e3c1058a0 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40956/0 (socket says 172.21.15.122:40956) 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 -- 172.21.15.122:0/3828968008 learned_addr learned my addr 172.21.15.122:0/3828968008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e40f68640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c101360 0x7f3e3c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c1058a0 0x7f3e3c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 -- 172.21.15.122:0/3828968008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c1058a0 msgr2=0x7f3e3c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c1058a0 0x7f3e3c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 -- 172.21.15.122:0/3828968008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c101360 msgr2=0x7f3e3c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c101360 0x7f3e3c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.288+0000 7f3e41769640 1 -- 172.21.15.122:0/3828968008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c19d3e0 con 0x7f3e3c1009c0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e41769640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9f0 tx=0x7f3e3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e327fc640 1 -- 172.21.15.122:0/3828968008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800cd80 con 0x7f3e3c1009c0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3c19d6d0 con 0x7f3e3c1009c0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e327fc640 1 -- 172.21.15.122:0/3828968008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38010910 con 0x7f3e3c1009c0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e327fc640 1 -- 172.21.15.122:0/3828968008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e380188f0 con 0x7f3e3c1009c0 2024-04-02T13:42:20.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.289+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3c1a4f60 con 0x7f3e3c1009c0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e327fc640 1 -- 172.21.15.122:0/3828968008 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3e38010ac0 con 0x7f3e3c1009c0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 Processor -- start 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 -- start start 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a322710830 con 0x55a3227ac8d0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a3226b8820 con 0x55a3227ad1f0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e465b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a32270c7e0 con 0x55a3227b5dd0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40958/0 (socket says 172.21.15.122:40958) 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 -- 172.21.15.122:0/1681449155 learned_addr learned my addr 172.21.15.122:0/1681449155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41769640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e40f68640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e327fc640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 0x7f3e280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 -- 172.21.15.122:0/1681449155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 msgr2=0x55a3227b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 -- 172.21.15.122:0/1681449155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 msgr2=0x55a3227b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41f6a640 1 -- 172.21.15.122:0/1681449155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3224ee7d0 con 0x55a3227ac8d0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.292+0000 7f3e41769640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e327fc640 1 -- 172.21.15.122:0/3828968008 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3e380d0e70 con 0x7f3e3c1009c0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 0x7f3e280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009fd0 tx=0x7f3e340118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f12a8f027f2812 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e277fe640 1 -- 172.21.15.122:0/1681449155 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34004250 con 0x55a3227ac8d0 2024-04-02T13:42:20.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e277fe640 1 -- 172.21.15.122:0/1681449155 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e34003db0 con 0x55a3227ac8d0 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.293+0000 7f3e277fe640 1 -- 172.21.15.122:0/1681449155 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3401bc20 con 0x55a3227ac8d0 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- 172.21.15.122:0/1681449155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 msgr2=0x55a3227accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 0x7f3e280a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e34020000 tx=0x7f3e34021000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009fd0 tx=0x7f3e340118d0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- 172.21.15.122:0/1681449155 shutdown_connections 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1681449155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227accb0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- 172.21.15.122:0/1681449155 >> 172.21.15.122:0/1681449155 conn(0x55a3227a7000 msgr2=0x55a3227a73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- 172.21.15.122:0/1681449155 shutdown_connections 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- 172.21.15.122:0/1681449155 wait complete. 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 -- start start 2024-04-02T13:42:20.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.294+0000 7f3e465b4740 1 Processor -- start 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 0x55a3227b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a3227b8bc0 con 0x55a3227b5dd0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a3226cbd30 con 0x55a3227ad1f0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a3227b8d40 con 0x55a3227ac8d0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 10 client.0 _ll_register_callbacks cb 0x55a3226cb720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41769640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 0x55a3227b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40974/0 (socket says 172.21.15.122:40974) 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 -- 172.21.15.122:0/3931406804 learned_addr learned my addr 172.21.15.122:0/3931406804 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41769640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 0x55a3227b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36380/0 (socket says 172.21.15.122:36380) 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e40f68640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 -- 172.21.15.122:0/3931406804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 msgr2=0x55a3227b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 0x55a3227b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 -- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 msgr2=0x55a3227b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 -- 172.21.15.122:0/3931406804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3224243f0 con 0x55a3227b5dd0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3e34003fa0 tx=0x7f3e3401b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bc3661e03e99a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34034020 con 0x55a3227b5dd0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3e3401bba0 con 0x55a3227b5dd0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e34003db0 con 0x55a3227b5dd0 2024-04-02T13:42:20.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e3403f050 con 0x55a3227b5dd0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 10 client.17727 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.295+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a3223e40f0 con 0x55a3227b5dd0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e34003db0 con 0x55a3227b5dd0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f3e3402f960 con 0x55a3227b5dd0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 10 client.17727 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 20 client.17727 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a3227c5cc0 0x55a3227c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 4 client.17727 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32271dd70 con 0x55a3227c5cc0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e41769640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a3227c5cc0 0x55a3227c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e24ff9640 10 client.17727 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:20.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.296+0000 7f3e41769640 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a3227c5cc0 0x55a3227c80b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab6e4142f978a34 server_cookie=c1ce243c8224016 in_seq=0 out_seq=0 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e24ff9640 1 -- 172.21.15.122:0/3931406804 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a32271dd70 con 0x55a3227c5cc0 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e24ff9640 10 client.17727 handle_command_reply: tid=0 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 client.17727 shutdown 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 msgr2=0x55a3227b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3e34003fa0 tx=0x7f3e3401b8a0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 shutdown_connections 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a3227c5cc0 0x55a3227c80b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227b5dd0 0x55a3227b2f10 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e24ff9640 0 client.17727 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 --2- 172.21.15.122:0/3931406804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ac8d0 0x55a3227b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 >> 172.21.15.122:0/3931406804 conn(0x55a3227a7000 msgr2=0x55a3227a8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 shutdown_connections 2024-04-02T13:42:20.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 1 -- 172.21.15.122:0/3931406804 wait complete. 2024-04-02T13:42:20.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.297+0000 7f3e465b4740 20 client.17727 trim_cache size 0 max 16384 2024-04-02T13:42:20.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 Processor -- start 2024-04-02T13:42:20.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 -- start start 2024-04-02T13:42:20.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1af100 con 0x55a3227ac8d0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1b0d00 con 0x55a3227ad1f0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.317+0000 7f3e465b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1af7d0 con 0x55a3227b5dd0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40976/0 (socket says 172.21.15.122:40976) 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 -- 172.21.15.122:0/1608254757 learned_addr learned my addr 172.21.15.122:0/1608254757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e40f68640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41f6a640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 -- 172.21.15.122:0/1608254757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 msgr2=0x55a3227b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 -- 172.21.15.122:0/1608254757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 msgr2=0x55a3227a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227a8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 -- 172.21.15.122:0/1608254757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3227a6e30 con 0x55a3227ac8d0 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e40f68640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e41769640 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3e38098f30 tx=0x7f3e380d3380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de185a75a957fc6a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e26ffd640 1 -- 172.21.15.122:0/1608254757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3809cad0 con 0x55a3227ac8d0 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e26ffd640 1 -- 172.21.15.122:0/1608254757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e380a0dd0 con 0x55a3227ac8d0 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.318+0000 7f3e26ffd640 1 -- 172.21.15.122:0/1608254757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3809cd30 con 0x55a3227ac8d0 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- 172.21.15.122:0/1608254757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 msgr2=0x55a3227a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f3e38098f30 tx=0x7f3e380d3380 comp rx=0 tx=0).stop 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- 172.21.15.122:0/1608254757 shutdown_connections 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227b5dd0 0x55a3227b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227ad1f0 0x55a3227a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 --2- 172.21.15.122:0/1608254757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227a6240 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- 172.21.15.122:0/1608254757 >> 172.21.15.122:0/1608254757 conn(0x55a3227a7000 msgr2=0x55a3227a8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- 172.21.15.122:0/1608254757 shutdown_connections 2024-04-02T13:42:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- 172.21.15.122:0/1608254757 wait complete. 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 -- start start 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.319+0000 7f3e465b4740 1 Processor -- start 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ad1f0 0x55a3227afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227b5dd0 0x55a3227b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1b0570 con 0x55a3227ac8d0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1afea0 con 0x55a3227b5dd0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3c1ad100 con 0x55a3227ad1f0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 10 client.0 _ll_register_callbacks cb 0x7f3e34009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40984/0 (socket says 172.21.15.122:40984) 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 -- 172.21.15.122:0/2793979169 learned_addr learned my addr 172.21.15.122:0/2793979169 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41f6a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227b5dd0 0x55a3227b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e40f68640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ad1f0 0x55a3227afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 -- 172.21.15.122:0/2793979169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ad1f0 msgr2=0x55a3227afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ad1f0 0x55a3227afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 -- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227b5dd0 msgr2=0x55a3227b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227b5dd0 0x55a3227b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 -- 172.21.15.122:0/2793979169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a32282b1a0 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e41769640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3e380bd210 tx=0x7f3e3809c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8396a69fa0d21b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3809cad0 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3e38099140 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e380a0dd0 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.320+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f3e380a5690 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 10 client.17733 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a32282c3f0 con 0x55a3227ac8d0 2024-04-02T13:42:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e380a0dd0 con 0x55a3227ac8d0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f3e380a5200 con 0x55a3227ac8d0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 10 client.17733 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 10 client.17733 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 20 client.17733 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a32282cbc0 0x55a32282efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 4 client.17733 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a3227180c0 con 0x55a32282cbc0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.321+0000 7f3e40f68640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a32282cbc0 0x55a32282efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e24ff9640 10 client.17733 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e40f68640 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a32282cbc0 0x55a32282efe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=502adacc1a5afbcc server_cookie=744486faace2f36a in_seq=0 out_seq=0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e24ff9640 1 -- 172.21.15.122:0/2793979169 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a3227180c0 con 0x55a32282cbc0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e24ff9640 10 client.17733 handle_command_reply: tid=0 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 client.17733 shutdown 2024-04-02T13:42:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 msgr2=0x55a3227af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f3e380bd210 tx=0x7f3e3809c4f0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 shutdown_connections 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55a32282cbc0 0x55a32282efe0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3227b5dd0 0x55a3227b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3227ad1f0 0x55a3227afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 --2- 172.21.15.122:0/2793979169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3227ac8d0 0x55a3227af600 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e24ff9640 0 client.17733 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 >> 172.21.15.122:0/2793979169 conn(0x55a3227a7000 msgr2=0x55a3227adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 shutdown_connections 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 1 -- 172.21.15.122:0/2793979169 wait complete. 2024-04-02T13:42:20.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.322+0000 7f3e465b4740 20 client.17733 trim_cache size 0 max 16384 2024-04-02T13:42:20.332 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:42:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:42:20.332 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 msgr2=0x7f3e280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 0x7f3e280a6260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e34020000 tx=0x7f3e34021000 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 msgr2=0x7f3e3c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9f0 tx=0x7f3e3800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 shutdown_connections 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e280a3da0 0x7f3e280a6260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e3c1058a0 0x7f3e3c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e3c101360 0x7f3e3c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 --2- 172.21.15.122:0/3828968008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e3c1009c0 0x7f3e3c1983f0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 >> 172.21.15.122:0/3828968008 conn(0x7f3e3c077a90 msgr2=0x7f3e3c06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 shutdown_connections 2024-04-02T13:42:20.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.323+0000 7f3e439cd640 1 -- 172.21.15.122:0/3828968008 wait complete. 2024-04-02T13:42:20.335 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:42:20.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:42:20.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- 172.21.15.122:0/3999444664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 msgr2=0x7fdcc8107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 --2- 172.21.15.122:0/3999444664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc8107ee0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009900 tx=0x7fdcb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:20.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- 172.21.15.122:0/3999444664 shutdown_connections 2024-04-02T13:42:20.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 --2- 172.21.15.122:0/3999444664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 0x7fdcc810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 --2- 172.21.15.122:0/3999444664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 0x7fdcc806a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 --2- 172.21.15.122:0/3999444664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc8107ee0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- 172.21.15.122:0/3999444664 >> 172.21.15.122:0/3999444664 conn(0x7fdcc8076800 msgr2=0x7fdcc8076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- 172.21.15.122:0/3999444664 shutdown_connections 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- 172.21.15.122:0/3999444664 wait complete. 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 Processor -- start 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.447+0000 7fdcce60d640 1 -- start start 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 0x7fdcc81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 0x7fdcc81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8111ea0 con 0x7fdcc8107b00 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8111d20 con 0x7fdcc806a150 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcce60d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc8112020 con 0x7fdcc806aaf0 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41004/0 (socket says 172.21.15.122:41004) 2024-04-02T13:42:20.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 -- 172.21.15.122:0/1307488065 learned_addr learned my addr 172.21.15.122:0/1307488065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcc77fe640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 0x7fdcc81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcc7fff640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 0x7fdcc81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 -- 172.21.15.122:0/1307488065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 msgr2=0x7fdcc81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 0x7fdcc81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 -- 172.21.15.122:0/1307488065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 msgr2=0x7fdcc81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 0x7fdcc81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 -- 172.21.15.122:0/1307488065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc819d390 con 0x7fdcc8107b00 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.448+0000 7fdcccbaa640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000eb10 tx=0x7fdcc000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.449+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0004b20 con 0x7fdcc8107b00 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.449+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcc0004cc0 con 0x7fdcc8107b00 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.449+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc819d680 con 0x7fdcc8107b00 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.449+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0010760 con 0x7fdcc8107b00 2024-04-02T13:42:20.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.449+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81a4f10 con 0x7fdcc8107b00 2024-04-02T13:42:20.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.450+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc81031c0 con 0x7fdcc8107b00 2024-04-02T13:42:20.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.451+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdcc0005020 con 0x7fdcc8107b00 2024-04-02T13:42:20.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.451+0000 7fdcc57fa640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.452+0000 7fdcc7fff640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.452+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fdcc00d10f0 con 0x7fdcc8107b00 2024-04-02T13:42:20.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.452+0000 7fdcc7fff640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009230 tx=0x7fdcb8005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.457+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdcc009d3a0 con 0x7fdcc8107b00 2024-04-02T13:42:20.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.705+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdcc81996f0 con 0x7fdcc8107b00 2024-04-02T13:42:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.706+0000 7fdcc57fa640 1 -- 172.21.15.122:0/1307488065 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11949 (secure 0 0 0) 0x7fdcc009cca0 con 0x7fdcc8107b00 2024-04-02T13:42:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:42:20.708 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:40:36.405088+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24512,14628,24460,24487,24485]},"id":1}]} 2024-04-02T13:42:20.708 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 38 2024-04-02T13:42:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 msgr2=0x7fdcb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009230 tx=0x7fdcb8005fd0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 msgr2=0x7fdcc819cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fdcc000eb10 tx=0x7fdcc000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 shutdown_connections 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdcb00a3d20 0x7fdcb00a61e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc8107b00 0x7fdcc819cc60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcc806aaf0 0x7fdcc81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 --2- 172.21.15.122:0/1307488065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdcc806a150 0x7fdcc81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 >> 172.21.15.122:0/1307488065 conn(0x7fdcc8076800 msgr2=0x7fdcc80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 shutdown_connections 2024-04-02T13:42:20.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.708+0000 7fdcce60d640 1 -- 172.21.15.122:0/1307488065 wait complete. 2024-04-02T13:42:20.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 -- 172.21.15.122:0/4280554774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff41009c0 msgr2=0x7faff4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 --2- 172.21.15.122:0/4280554774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff41009c0 0x7faff4100e20 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fafe8009900 tx=0x7fafe802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 -- 172.21.15.122:0/4280554774 shutdown_connections 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 --2- 172.21.15.122:0/4280554774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff4101360 0x7faff410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 --2- 172.21.15.122:0/4280554774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff41009c0 0x7faff4100e20 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 --2- 172.21.15.122:0/4280554774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41058a0 0x7faff4105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 -- 172.21.15.122:0/4280554774 >> 172.21.15.122:0/4280554774 conn(0x7faff4077a90 msgr2=0x7faff4077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 -- 172.21.15.122:0/4280554774 shutdown_connections 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.825+0000 7faffa904640 1 -- 172.21.15.122:0/4280554774 wait complete. 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 Processor -- start 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 -- start start 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 0x7faff41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff41058a0 0x7faff419bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faff4110d90 con 0x7faff4101360 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faff4110c10 con 0x7faff41058a0 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faffa904640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faff4110f10 con 0x7faff41009c0 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 0x7faff41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41014/0 (socket says 172.21.15.122:41014) 2024-04-02T13:42:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 -- 172.21.15.122:0/4064156580 learned_addr learned my addr 172.21.15.122:0/4064156580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff8ea1640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff41058a0 0x7faff419bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 -- 172.21.15.122:0/4064156580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 msgr2=0x7faff41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 0x7faff41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 -- 172.21.15.122:0/4064156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff41058a0 msgr2=0x7faff419bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff41058a0 0x7faff419bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff37fe640 1 -- 172.21.15.122:0/4064156580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff419c290 con 0x7faff4101360 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.826+0000 7faff3fff640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 0x7faff41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faff37fe640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005940 tx=0x7fafe802cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faff17fa640 1 -- 172.21.15.122:0/4064156580 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8041070 con 0x7faff4101360 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff419c520 con 0x7faff4101360 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faff17fa640 1 -- 172.21.15.122:0/4064156580 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe8004450 con 0x7faff4101360 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faff17fa640 1 -- 172.21.15.122:0/4064156580 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe8034440 con 0x7faff4101360 2024-04-02T13:42:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.827+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff41a3e10 con 0x7faff4101360 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.829+0000 7faff17fa640 1 -- 172.21.15.122:0/4064156580 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fafe8044070 con 0x7faff4101360 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 Processor -- start 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 -- start start 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff17fa640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026640c0 0x55920266c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592025c77b0 con 0x5592026637a0 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559202566570 con 0x55920266cca0 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faffd5ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592025c3760 con 0x5592026640c0 2024-04-02T13:42:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff8ea1640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41018/0 (socket says 172.21.15.122:41018) 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 -- 172.21.15.122:0/237957377 learned_addr learned my addr 172.21.15.122:0/237957377 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff17fa640 1 -- 172.21.15.122:0/4064156580 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fafe80f2660 con 0x7faff4101360 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff37fe640 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026640c0 0x55920266c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff8ea1640 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 -- 172.21.15.122:0/237957377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026640c0 msgr2=0x55920266c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026640c0 0x55920266c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 -- 172.21.15.122:0/237957377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 msgr2=0x55920266f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.830+0000 7faff3fff640 1 -- 172.21.15.122:0/237957377 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592022db3f0 con 0x5592026637a0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.831+0000 7faff8ea1640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafec005fd0 tx=0x7fafec003f50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.831+0000 7faff3fff640 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fafe4010480 tx=0x7fafe400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b103978a01982ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.831+0000 7fafda7fc640 1 -- 172.21.15.122:0/237957377 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe401c680 con 0x5592026637a0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.831+0000 7fafda7fc640 1 -- 172.21.15.122:0/237957377 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe401cd00 con 0x5592026637a0 2024-04-02T13:42:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.831+0000 7fafda7fc640 1 -- 172.21.15.122:0/237957377 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4011e50 con 0x5592026637a0 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- 172.21.15.122:0/237957377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 msgr2=0x559202663b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fafe4010480 tx=0x7fafe400bf90 comp rx=0 tx=0).stop 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- 172.21.15.122:0/237957377 shutdown_connections 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026640c0 0x55920266c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 --2- 172.21.15.122:0/237957377 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x559202663b80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- 172.21.15.122:0/237957377 >> 172.21.15.122:0/237957377 conn(0x55920265ded0 msgr2=0x55920265e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- 172.21.15.122:0/237957377 shutdown_connections 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- 172.21.15.122:0/237957377 wait complete. 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 -- start start 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.832+0000 7faffd5ca740 1 Processor -- start 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592026640c0 0x559202669a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55920266cca0 0x55920266a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55920266fa90 con 0x5592026637a0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559202582e80 con 0x5592026640c0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55920266fc10 con 0x55920266cca0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faffd5ca740 10 client.0 _ll_register_callbacks cb 0x5592025beaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41026/0 (socket says 172.21.15.122:41026) 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 -- 172.21.15.122:0/3782072329 learned_addr learned my addr 172.21.15.122:0/3782072329 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff37fe640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592026640c0 0x559202669a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff8ea1640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55920266cca0 0x55920266a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 -- 172.21.15.122:0/3782072329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55920266cca0 msgr2=0x55920266a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55920266cca0 0x55920266a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 -- 172.21.15.122:0/3782072329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592026640c0 msgr2=0x559202669a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592026640c0 0x559202669a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 -- 172.21.15.122:0/3782072329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592023a57d0 con 0x5592026637a0 2024-04-02T13:42:20.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7faff3fff640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fafe4011770 tx=0x7fafe400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f396ce7e96a1f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7fafd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.833+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe402a060 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fafe4011dc0 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe401cd00 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fafe4031050 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 10 client.17751 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55920267bb00 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe401cd00 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fafe401c5a0 con 0x5592026637a0 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 10 client.17751 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 10 client.17751 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 20 client.17751 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55920267cf60 0x55920267f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 4 client.17751 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592025d4cf0 con 0x55920267cf60 2024-04-02T13:42:20.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.834+0000 7faff37fe640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55920267cf60 0x55920267f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7fafd37fe640 10 client.17751 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7faff37fe640 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55920267cf60 0x55920267f350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c66016dc037c8b91 server_cookie=6e48c5189de63bf3 in_seq=0 out_seq=0 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7fafd37fe640 1 -- 172.21.15.122:0/3782072329 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592025d4cf0 con 0x55920267cf60 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7fafd37fe640 10 client.17751 handle_command_reply: tid=0 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7faffd5ca740 1 client.17751 shutdown 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 msgr2=0x55920266c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fafe4011770 tx=0x7fafe400bd40 comp rx=0 tx=0).stop 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.835+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 shutdown_connections 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55920267cf60 0x55920267f350 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55920266cca0 0x55920266a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592026640c0 0x559202669a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3782072329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026637a0 0x55920266c700 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7fafd37fe640 0 client.17751 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 >> 172.21.15.122:0/3782072329 conn(0x55920265ded0 msgr2=0x55920265fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 shutdown_connections 2024-04-02T13:42:20.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 1 -- 172.21.15.122:0/3782072329 wait complete. 2024-04-02T13:42:20.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.836+0000 7faffd5ca740 20 client.17751 trim_cache size 0 max 16384 2024-04-02T13:42:20.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.849+0000 7faffd5ca740 1 Processor -- start 2024-04-02T13:42:20.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.849+0000 7faffd5ca740 1 -- start start 2024-04-02T13:42:20.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x559202678950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ace20 con 0x5592026640c0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faff41aea20 con 0x55920266cca0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faffd5ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ad4f0 con 0x5592026637a0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41032/0 (socket says 172.21.15.122:41032) 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 -- 172.21.15.122:0/3070316340 learned_addr learned my addr 172.21.15.122:0/3070316340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff8ea1640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff3fff640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x559202678950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 -- 172.21.15.122:0/3070316340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 msgr2=0x559202678950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x559202678950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 -- 172.21.15.122:0/3070316340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 msgr2=0x55920266c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 -- 172.21.15.122:0/3070316340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559202663ef0 con 0x5592026640c0 2024-04-02T13:42:20.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff3fff640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x559202678950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.850+0000 7faff37fe640 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ba830 tx=0x7fafe802bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81522fd2706f1db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7fafd9ffb640 1 -- 172.21.15.122:0/3070316340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe804b040 con 0x5592026640c0 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7fafd9ffb640 1 -- 172.21.15.122:0/3070316340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe80f2040 con 0x5592026640c0 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7fafd9ffb640 1 -- 172.21.15.122:0/3070316340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe80baa40 con 0x5592026640c0 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 -- 172.21.15.122:0/3070316340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 msgr2=0x559202668a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ba830 tx=0x7fafe802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 -- 172.21.15.122:0/3070316340 shutdown_connections 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x559202668a70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 --2- 172.21.15.122:0/3070316340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x559202678950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 -- 172.21.15.122:0/3070316340 >> 172.21.15.122:0/3070316340 conn(0x55920265ded0 msgr2=0x559202668360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 -- 172.21.15.122:0/3070316340 shutdown_connections 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.851+0000 7faffd5ca740 1 -- 172.21.15.122:0/3070316340 wait complete. 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 -- start start 2024-04-02T13:42:20.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 Processor -- start 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x55920266d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faff41ae290 con 0x5592026640c0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7faff41adbc0 con 0x55920266cca0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faff41aae20 con 0x5592026637a0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faffd5ca740 10 client.0 _ll_register_callbacks cb 0x7fafe400be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faff3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x55920266d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faff37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41034/0 (socket says 172.21.15.122:41034) 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faff37fe640 1 -- 172.21.15.122:0/2703920969 learned_addr learned my addr 172.21.15.122:0/2703920969 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.852+0000 7faff8ea1640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 -- 172.21.15.122:0/2703920969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 msgr2=0x55920266d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x55920266d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 -- 172.21.15.122:0/2703920969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 msgr2=0x55920266df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 -- 172.21.15.122:0/2703920969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592026740b0 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faff37fe640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ba050 tx=0x7fafe8003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862e642e86af161f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7fafd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe804b040 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fafe80baa40 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafe80f2040 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fafe80dc050 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faffd5ca740 10 client.17757 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.853+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559202675320 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafe80f2040 con 0x5592026640c0 2024-04-02T13:42:20.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fafe80c6b00 con 0x5592026640c0 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 10 client.17757 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 10 client.17757 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 20 client.17757 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592026e3300 0x5592026e5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 4 client.17757 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592025cf040 con 0x5592026e3300 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faff3fff640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592026e3300 0x5592026e5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7fafd37fe640 10 client.17757 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.854+0000 7faff3fff640 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592026e3300 0x5592026e5720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=820837e54afca81e server_cookie=79bf0caaf4b6ef8e in_seq=0 out_seq=0 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7fafd37fe640 1 -- 172.21.15.122:0/2703920969 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592025cf040 con 0x5592026e3300 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7fafd37fe640 10 client.17757 handle_command_reply: tid=0 2024-04-02T13:42:20.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 client.17757 shutdown 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 msgr2=0x55920266d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fafe80ba050 tx=0x7fafe8003990 comp rx=0 tx=0).stop 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 shutdown_connections 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592026e3300 0x5592026e5720 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55920266cca0 0x55920266df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592026640c0 0x55920266d9d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7fafd37fe640 0 client.17757 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 --2- 172.21.15.122:0/2703920969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592026637a0 0x55920266d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 >> 172.21.15.122:0/2703920969 conn(0x55920265ded0 msgr2=0x55920265cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 shutdown_connections 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 1 -- 172.21.15.122:0/2703920969 wait complete. 2024-04-02T13:42:20.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.855+0000 7faffd5ca740 20 client.17757 trim_cache size 0 max 16384 2024-04-02T13:42:20.865 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:20.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 msgr2=0x7fafdc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fafec005fd0 tx=0x7fafec003f50 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 msgr2=0x7faff4197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fafe8005940 tx=0x7fafe802cd10 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 shutdown_connections 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fafdc0a3d20 0x7fafdc0a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7faff41058a0 0x7faff419bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faff4101360 0x7faff4197830 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 --2- 172.21.15.122:0/4064156580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faff41009c0 0x7faff41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 >> 172.21.15.122:0/4064156580 conn(0x7faff4077a90 msgr2=0x7faff40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 shutdown_connections 2024-04-02T13:42:20.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.856+0000 7faffa904640 1 -- 172.21.15.122:0/4064156580 wait complete. 2024-04-02T13:42:20.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:42:20.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 220756 2.94 MB/sec execute 221 sec latency 432.606 ms 2024-04-02T13:42:20.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 -- 172.21.15.122:0/1713637424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c8107b50 msgr2=0x7f39c8107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 --2- 172.21.15.122:0/1713637424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c8107b50 0x7f39c8107f30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009930 tx=0x7f39b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 -- 172.21.15.122:0/1713637424 shutdown_connections 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 --2- 172.21.15.122:0/1713637424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c80696c0 0x7f39c810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 --2- 172.21.15.122:0/1713637424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8068d00 0x7f39c8069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 --2- 172.21.15.122:0/1713637424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c8107b50 0x7f39c8107f30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 -- 172.21.15.122:0/1713637424 >> 172.21.15.122:0/1713637424 conn(0x7f39c8077b00 msgr2=0x7f39c8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 -- 172.21.15.122:0/1713637424 shutdown_connections 2024-04-02T13:42:20.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.994+0000 7f39cd167640 1 -- 172.21.15.122:0/1713637424 wait complete. 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.995+0000 7f39cd167640 1 Processor -- start 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.995+0000 7f39cd167640 1 -- start start 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 0x7f39c8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8107b50 0x7f39c819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8111ea0 con 0x7f39c80696c0 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8111d20 con 0x7f39c8068d00 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39cd167640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8112020 con 0x7f39c8107b50 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41048/0 (socket says 172.21.15.122:41048) 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c659c640 1 -- 172.21.15.122:0/1248150636 learned_addr learned my addr 172.21.15.122:0/1248150636 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 0x7f39c8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c759e640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8107b50 0x7f39c819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 -- 172.21.15.122:0/1248150636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8107b50 msgr2=0x7f39c819cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8107b50 0x7f39c819cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 -- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 msgr2=0x7f39c8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 -- 172.21.15.122:0/1248150636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c819d3a0 con 0x7f39c8068d00 2024-04-02T13:42:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c659c640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 0x7f39c8198400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.996+0000 7f39affff640 1 -- 172.21.15.122:0/1248150636 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8041070 con 0x7f39c8068d00 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.997+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c819d630 con 0x7f39c8068d00 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.997+0000 7f39affff640 1 -- 172.21.15.122:0/1248150636 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b80054e0 con 0x7f39c8068d00 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.997+0000 7f39affff640 1 -- 172.21.15.122:0/1248150636 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b8003bf0 con 0x7f39c8068d00 2024-04-02T13:42:21.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.997+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c81a4f20 con 0x7f39c8068d00 2024-04-02T13:42:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39affff640 1 -- 172.21.15.122:0/1248150636 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f39b8044070 con 0x7f39c8068d00 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 Processor -- start 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 -- start start 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 0x55a4be5d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 0x55a4be5dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a4be5347b0 con 0x55a4be5d10c0 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a4be4d3570 con 0x55a4be5d9ca0 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a4be530760 con 0x55a4be5d07a0 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:20.999+0000 7f39c6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 0x55a4be5d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36494/0 (socket says 172.21.15.122:36494) 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 -- 172.21.15.122:0/2097377850 learned_addr learned my addr 172.21.15.122:0/2097377850 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c659c640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 0x55a4be5dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39affff640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 0x7f39b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 -- 172.21.15.122:0/2097377850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 msgr2=0x55a4be5d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 -- 172.21.15.122:0/2097377850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 msgr2=0x55a4be5dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 0x55a4be5dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 -- 172.21.15.122:0/2097377850 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4be2483f0 con 0x55a4be5d10c0 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c659c640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 0x55a4be5dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 0x7f39b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39affff640 1 -- 172.21.15.122:0/1248150636 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f39b80f25a0 con 0x7f39c8068d00 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 0x55a4be5d9760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ef9f0 tx=0x7f39b80b3780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7032153f0f5b1c8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39ad7fa640 1 -- 172.21.15.122:0/2097377850 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b80b8e60 con 0x55a4be5d10c0 2024-04-02T13:42:21.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39c759e640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 0x7f39b00a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39c0003df0 tx=0x7f39c0004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39ad7fa640 1 -- 172.21.15.122:0/2097377850 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b8004c80 con 0x55a4be5d10c0 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.000+0000 7f39ad7fa640 1 -- 172.21.15.122:0/2097377850 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b808cab0 con 0x55a4be5d10c0 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2097377850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 msgr2=0x55a4be5d9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 0x55a4be5d9760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ef9f0 tx=0x7f39b80b3780 comp rx=0 tx=0).stop 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2097377850 shutdown_connections 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d9ca0 0x55a4be5dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d10c0 0x55a4be5d9760 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2097377850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d07a0 0x55a4be5d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2097377850 >> 172.21.15.122:0/2097377850 conn(0x55a4be5caed0 msgr2=0x55a4be5cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2097377850 shutdown_connections 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2097377850 wait complete. 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 -- start start 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.001+0000 7f39cfe2f740 1 Processor -- start 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d10c0 0x55a4be5d6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a4be5dc850 con 0x55a4be5d07a0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a4be4efe80 con 0x55a4be5d10c0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a4be5dc9d0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 10 client.0 _ll_register_callbacks cb 0x55a4be52baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41068/0 (socket says 172.21.15.122:41068) 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d10c0 0x55a4be5d6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 -- 172.21.15.122:0/3345613849 learned_addr learned my addr 172.21.15.122:0/3345613849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c759e640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 -- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d9ca0 msgr2=0x55a4be5d71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 -- 172.21.15.122:0/3345613849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d10c0 msgr2=0x55a4be5d6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d10c0 0x55a4be5d6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 -- 172.21.15.122:0/3345613849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4be3127d0 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f39b80efa50 tx=0x7f39b8004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5324d1a7e029665 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b804b040 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f39b80d29a0 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.002+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39b8004c80 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f39b80b8e60 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 10 client.17769 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a4be5dc850 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39b8004c80 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f39b80be550 con 0x55a4be5d07a0 2024-04-02T13:42:21.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 10 client.17769 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 10 client.17769 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 20 client.17769 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be5e9af0 0x55a4be5ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 4 client.17769 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4be541cf0 con 0x55a4be5e9af0 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.003+0000 7f39c659c640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be5e9af0 0x55a4be5ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39c659c640 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be5e9af0 0x55a4be5ebf20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d738301376db4d63 server_cookie=cab2449fd0a8d502 in_seq=0 out_seq=0 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39aaff5640 10 client.17769 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39aaff5640 1 -- 172.21.15.122:0/3345613849 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a4be541cf0 con 0x55a4be5e9af0 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39aaff5640 10 client.17769 handle_command_reply: tid=0 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 client.17769 shutdown 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 msgr2=0x55a4be5d9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f39b80efa50 tx=0x7f39b8004bf0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 shutdown_connections 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d10c0 0x55a4be5d6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be5e9af0 0x55a4be5ebf20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/3345613849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d07a0 0x55a4be5d9740 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39aaff5640 0 client.17769 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.004+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 >> 172.21.15.122:0/3345613849 conn(0x55a4be5caed0 msgr2=0x55a4be5ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.005+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 shutdown_connections 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.005+0000 7f39cfe2f740 1 -- 172.21.15.122:0/3345613849 wait complete. 2024-04-02T13:42:21.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.005+0000 7f39cfe2f740 20 client.17769 trim_cache size 0 max 16384 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 Processor -- start 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 -- start start 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81af0d0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81b0cd0 con 0x55a4be5d07a0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81af7a0 con 0x55a4be5d10c0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41072/0 (socket says 172.21.15.122:41072) 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 -- 172.21.15.122:0/2725087315 learned_addr learned my addr 172.21.15.122:0/2725087315 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c659c640 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 -- 172.21.15.122:0/2725087315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 msgr2=0x55a4be5e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 -- 172.21.15.122:0/2725087315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 msgr2=0x55a4be5d6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.018+0000 7f39c759e640 1 -- 172.21.15.122:0/2725087315 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4be5d5020 con 0x55a4be5d9ca0 2024-04-02T13:42:21.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39c759e640 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f39c00117f0 tx=0x7f39c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f470eb027105b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39acff9640 1 -- 172.21.15.122:0/2725087315 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c001e0d0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39acff9640 1 -- 172.21.15.122:0/2725087315 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c001e270 con 0x55a4be5d9ca0 2024-04-02T13:42:21.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39acff9640 1 -- 172.21.15.122:0/2725087315 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00053c0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2725087315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 msgr2=0x55a4be5d9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.019+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f39c00117f0 tx=0x7f39c0019000 comp rx=0 tx=0).stop 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2725087315 shutdown_connections 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d9200 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2725087315 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2725087315 >> 172.21.15.122:0/2725087315 conn(0x55a4be5caed0 msgr2=0x55a4be5d1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2725087315 shutdown_connections 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2725087315 wait complete. 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- start start 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 Processor -- start 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81b0540 con 0x55a4be5d9ca0 2024-04-02T13:42:21.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81afe70 con 0x55a4be5d07a0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39c81ad0d0 con 0x55a4be5d10c0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39cfe2f740 10 client.0 _ll_register_callbacks cb 0x7f39bc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39c759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39c759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41086/0 (socket says 172.21.15.122:41086) 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39c759e640 1 -- 172.21.15.122:0/2992211467 learned_addr learned my addr 172.21.15.122:0/2992211467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39c659c640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.020+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 -- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 msgr2=0x55a4be5d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 -- 172.21.15.122:0/2992211467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 msgr2=0x55a4be5d38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 -- 172.21.15.122:0/2992211467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4be5dfce0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39c759e640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f39c001e810 tx=0x7f39c0011610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668b5d8775271119 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0038020 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39c001e270 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f39c0004d80 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f39c0043050 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 10 client.17775 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a4be6502c0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c001e270 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f39c00056e0 con 0x55a4be5d9ca0 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 10 client.17775 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 10 client.17775 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.021+0000 7f39cfe2f740 20 client.17775 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be650a10 0x55a4be652e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 4 client.17775 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:21.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4be53c040 con 0x55a4be650a10 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be650a10 0x55a4be652e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39aaff5640 10 client.17775 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39c6d9d640 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be650a10 0x55a4be652e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9644d510a8f17b3 server_cookie=1f12625763a39551 in_seq=0 out_seq=0 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39aaff5640 1 -- 172.21.15.122:0/2992211467 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4be53c040 con 0x55a4be650a10 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39aaff5640 10 client.17775 handle_command_reply: tid=0 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 client.17775 shutdown 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 msgr2=0x55a4be5d4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f39c001e810 tx=0x7f39c0011610 comp rx=0 tx=0).stop 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 shutdown_connections 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55a4be650a10 0x55a4be652e30 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a4be5d9ca0 0x55a4be5d4350 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39aaff5640 0 client.17775 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a4be5d10c0 0x55a4be5d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.022+0000 7f39cfe2f740 1 --2- 172.21.15.122:0/2992211467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a4be5d07a0 0x55a4be5d38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 >> 172.21.15.122:0/2992211467 conn(0x55a4be5caed0 msgr2=0x55a4be5c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 shutdown_connections 2024-04-02T13:42:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cfe2f740 1 -- 172.21.15.122:0/2992211467 wait complete. 2024-04-02T13:42:21.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cfe2f740 20 client.17775 trim_cache size 0 max 16384 2024-04-02T13:42:21.032 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 msgr2=0x7f39b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 0x7f39b00a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39c0003df0 tx=0x7f39c0004a40 comp rx=0 tx=0).stop 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 msgr2=0x7f39c8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 0x7f39c8198400 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f39b8009230 tx=0x7f39b8004270 comp rx=0 tx=0).stop 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 shutdown_connections 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39b00a3d20 0x7f39b00a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39c8107b50 0x7f39c819cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39c80696c0 0x7f39c8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 --2- 172.21.15.122:0/1248150636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39c8068d00 0x7f39c8198400 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 >> 172.21.15.122:0/1248150636 conn(0x7f39c8077b00 msgr2=0x7f39c80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 shutdown_connections 2024-04-02T13:42:21.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.023+0000 7f39cd167640 1 -- 172.21.15.122:0/1248150636 wait complete. 2024-04-02T13:42:21.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:42:21.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 -- 172.21.15.122:0/499178302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 msgr2=0x7f85c8069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 --2- 172.21.15.122:0/499178302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8069180 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009930 tx=0x7f85b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:21.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 -- 172.21.15.122:0/499178302 shutdown_connections 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 --2- 172.21.15.122:0/499178302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c80696c0 0x7f85c810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 --2- 172.21.15.122:0/499178302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8069180 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 --2- 172.21.15.122:0/499178302 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c8107b50 0x7f85c8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 -- 172.21.15.122:0/499178302 >> 172.21.15.122:0/499178302 conn(0x7f85c8077b00 msgr2=0x7f85c8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 -- 172.21.15.122:0/499178302 shutdown_connections 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.139+0000 7f85cf71c640 1 -- 172.21.15.122:0/499178302 wait complete. 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.140+0000 7f85cf71c640 1 Processor -- start 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.140+0000 7f85cf71c640 1 -- start start 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c80696c0 0x7f85c81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c8107b50 0x7f85c819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8111e70 con 0x7f85c8068d00 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8111cf0 con 0x7f85c80696c0 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85c8111ff0 con 0x7f85c8107b50 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41104/0 (socket says 172.21.15.122:41104) 2024-04-02T13:42:21.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 -- 172.21.15.122:0/294216197 learned_addr learned my addr 172.21.15.122:0/294216197 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cccb7640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c80696c0 0x7f85c81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c8107b50 0x7f85c819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 -- 172.21.15.122:0/294216197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c8107b50 msgr2=0x7f85c819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c8107b50 0x7f85c819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 -- 172.21.15.122:0/294216197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c80696c0 msgr2=0x7f85c81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c80696c0 0x7f85c81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 -- 172.21.15.122:0/294216197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c819d330 con 0x7f85c8068d00 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f85c400d950 tx=0x7f85c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85be7fc640 1 -- 172.21.15.122:0/294216197 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c4017030 con 0x7f85c8068d00 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c819d620 con 0x7f85c8068d00 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85be7fc640 1 -- 172.21.15.122:0/294216197 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c4004020 con 0x7f85c8068d00 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c81a4eb0 con 0x7f85c8068d00 2024-04-02T13:42:21.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.141+0000 7f85be7fc640 1 -- 172.21.15.122:0/294216197 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c40047a0 con 0x7f85c8068d00 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 Processor -- start 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 -- start start 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 0x55aee9100830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aee90647b0 con 0x55aee910e0b0 2024-04-02T13:42:21.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aee9003570 con 0x55aee9100450 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85d23e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aee9060760 con 0x55aee9100d70 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.144+0000 7f85be7fc640 1 -- 172.21.15.122:0/294216197 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f85c40041d0 con 0x7f85c8068d00 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cccb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cccb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41116/0 (socket says 172.21.15.122:41116) 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cccb7640 1 -- 172.21.15.122:0/2041328792 learned_addr learned my addr 172.21.15.122:0/2041328792 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 0x55aee9100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 -- 172.21.15.122:0/2041328792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 msgr2=0x55aee910db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 -- 172.21.15.122:0/2041328792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 msgr2=0x55aee91104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 -- 172.21.15.122:0/2041328792 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee8d783f0 con 0x55aee9100450 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cccb7640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 0x55aee9100830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009f30 tx=0x7f85c0010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6563e3b7700ee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85b37fe640 1 -- 172.21.15.122:0/2041328792 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0016070 con 0x55aee9100450 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85be7fc640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 0x7f85b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85b37fe640 1 -- 172.21.15.122:0/2041328792 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c0004d20 con 0x55aee9100450 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85b37fe640 1 -- 172.21.15.122:0/2041328792 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0012790 con 0x55aee9100450 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.145+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 0x7f85b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85be7fc640 1 -- 172.21.15.122:0/294216197 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f85c40d05c0 con 0x7f85c8068d00 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- 172.21.15.122:0/2041328792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 msgr2=0x55aee9100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85cd4b8640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 0x7f85b40a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c40057b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 0x55aee9100830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009f30 tx=0x7f85c0010800 comp rx=0 tx=0).stop 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- 172.21.15.122:0/2041328792 shutdown_connections 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee910e0b0 0x55aee91104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2041328792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100450 0x55aee9100830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- 172.21.15.122:0/2041328792 >> 172.21.15.122:0/2041328792 conn(0x55aee90faed0 msgr2=0x55aee90fb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- 172.21.15.122:0/2041328792 shutdown_connections 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- 172.21.15.122:0/2041328792 wait complete. 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.146+0000 7f85d23e7740 1 -- start start 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 Processor -- start 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100450 0x55aee910db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee910b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55aee9110ef0 con 0x55aee9100d70 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55aee901fe80 con 0x55aee910e0b0 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55aee9111070 con 0x55aee9100450 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85d23e7740 10 client.0 _ll_register_callbacks cb 0x55aee905baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41126/0 (socket says 172.21.15.122:41126) 2024-04-02T13:42:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 -- 172.21.15.122:0/2734819195 learned_addr learned my addr 172.21.15.122:0/2734819195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100450 0x55aee910db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee910b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 -- 172.21.15.122:0/2734819195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100450 msgr2=0x55aee910db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100450 0x55aee910db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 -- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 msgr2=0x55aee910b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee910b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.147+0000 7f85cccb7640 1 -- 172.21.15.122:0/2734819195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee8e427d0 con 0x55aee9100d70 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85cccb7640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009230 tx=0x7f85b8004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6c5603115517c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b802cab0 con 0x55aee9100d70 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f85b802cc50 con 0x55aee9100d70 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c0004d20 con 0x55aee9100d70 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f85b80046c0 con 0x55aee9100d70 2024-04-02T13:42:21.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 10 client.17787 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aee911cfe0 con 0x55aee9100d70 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c0004d20 con 0x55aee9100d70 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f85b80035f0 con 0x55aee9100d70 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 10 client.17787 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 10 client.17787 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 20 client.17787 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee911e540 0x55aee9120960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 4 client.17787 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.148+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aee9071cf0 con 0x55aee911e540 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.149+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee911e540 0x55aee9120960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.149+0000 7f85b0ff9640 10 client.17787 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.149+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee911e540 0x55aee9120960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=993435d180a42bc0 server_cookie=eda75d95574853a0 in_seq=0 out_seq=0 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85b0ff9640 1 -- 172.21.15.122:0/2734819195 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aee9071cf0 con 0x55aee911e540 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85b0ff9640 10 client.17787 handle_command_reply: tid=0 2024-04-02T13:42:21.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 client.17787 shutdown 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 msgr2=0x55aee910ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009230 tx=0x7f85b8004610 comp rx=0 tx=0).stop 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 shutdown_connections 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee911e540 0x55aee9120960 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee910b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85b0ff9640 0 client.17787 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100d70 0x55aee910ae90 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 --2- 172.21.15.122:0/2734819195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100450 0x55aee910db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 >> 172.21.15.122:0/2734819195 conn(0x55aee90faed0 msgr2=0x55aee90fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 shutdown_connections 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 1 -- 172.21.15.122:0/2734819195 wait complete. 2024-04-02T13:42:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.150+0000 7f85d23e7740 20 client.17787 trim_cache size 0 max 16384 2024-04-02T13:42:21.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 Processor -- start 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 -- start start 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 0x55aee9119e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee910e0b0 0x55aee9126230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81acf20 con 0x55aee9100450 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81aeb20 con 0x55aee9100d70 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85d23e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81ad5f0 con 0x55aee910e0b0 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85cdcb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.163+0000 7f85cdcb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41138/0 (socket says 172.21.15.122:41138) 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 -- 172.21.15.122:0/4184835024 learned_addr learned my addr 172.21.15.122:0/4184835024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cccb7640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 0x55aee9119e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee910e0b0 0x55aee9126230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 -- 172.21.15.122:0/4184835024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee910e0b0 msgr2=0x55aee9126230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee910e0b0 0x55aee9126230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 -- 172.21.15.122:0/4184835024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 msgr2=0x55aee9119e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 0x55aee9119e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 -- 172.21.15.122:0/4184835024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aee90ff890 con 0x55aee9100450 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cccb7640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 0x55aee9119e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009640 tx=0x7f85c0005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572b107abbe31491 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85b2ffd640 1 -- 172.21.15.122:0/4184835024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0016040 con 0x55aee9100450 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85b2ffd640 1 -- 172.21.15.122:0/4184835024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c002d040 con 0x55aee9100450 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.164+0000 7f85b2ffd640 1 -- 172.21.15.122:0/4184835024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0023040 con 0x55aee9100450 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- 172.21.15.122:0/4184835024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 msgr2=0x55aee910ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009640 tx=0x7f85c0005a40 comp rx=0 tx=0).stop 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- 172.21.15.122:0/4184835024 shutdown_connections 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee910e0b0 0x55aee9126230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee9100d70 0x55aee9119e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 --2- 172.21.15.122:0/4184835024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910ae30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- 172.21.15.122:0/4184835024 >> 172.21.15.122:0/4184835024 conn(0x55aee90faed0 msgr2=0x55aee910a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- 172.21.15.122:0/4184835024 shutdown_connections 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- 172.21.15.122:0/4184835024 wait complete. 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 -- start start 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.165+0000 7f85d23e7740 1 Processor -- start 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee9115360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81ae390 con 0x55aee9100450 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81adcc0 con 0x55aee910e0b0 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85c81aaf20 con 0x55aee9100d70 2024-04-02T13:42:21.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85d23e7740 10 client.0 _ll_register_callbacks cb 0x7f85b8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41154/0 (socket says 172.21.15.122:41154) 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 -- 172.21.15.122:0/144520954 learned_addr learned my addr 172.21.15.122:0/144520954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cccb7640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee9115360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 -- 172.21.15.122:0/144520954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 msgr2=0x55aee910edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 -- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 msgr2=0x55aee9115360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee9115360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 -- 172.21.15.122:0/144520954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aee91158a0 con 0x55aee9100450 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85cdcb9640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c0010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c3a50596012f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.166+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0016070 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f85c002d650 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c002d040 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f85c0003a70 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 10 client.17793 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55aee9116ba0 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c002d040 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f85c000f040 con 0x55aee9100450 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 10 client.17793 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 10 client.17793 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 20 client.17793 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee9184ca0 0x55aee9187090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 4 client.17793 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aee906c040 con 0x55aee9184ca0 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.167+0000 7f85cccb7640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee9184ca0 0x55aee9187090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85b0ff9640 10 client.17793 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85cccb7640 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee9184ca0 0x55aee9187090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27e10b6bafe7c76 server_cookie=682f8e367386b25e in_seq=0 out_seq=0 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85b0ff9640 1 -- 172.21.15.122:0/144520954 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aee906c040 con 0x55aee9184ca0 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85b0ff9640 10 client.17793 handle_command_reply: tid=0 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 client.17793 shutdown 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 msgr2=0x55aee910e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f85c0005fd0 tx=0x7f85c0010e10 comp rx=0 tx=0).stop 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 shutdown_connections 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55aee9184ca0 0x55aee9187090 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55aee910e0b0 0x55aee9115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85b0ff9640 0 client.17793 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55aee9100d70 0x55aee910edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 --2- 172.21.15.122:0/144520954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55aee9100450 0x55aee910e880 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 >> 172.21.15.122:0/144520954 conn(0x55aee90faed0 msgr2=0x55aee90f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 shutdown_connections 2024-04-02T13:42:21.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 1 -- 172.21.15.122:0/144520954 wait complete. 2024-04-02T13:42:21.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.168+0000 7f85d23e7740 20 client.17793 trim_cache size 0 max 16384 2024-04-02T13:42:21.178 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 msgr2=0x7f85b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 0x7f85b40a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009fd0 tx=0x7f85c40057b0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 msgr2=0x7f85c8198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f85c400d950 tx=0x7f85c400de10 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 shutdown_connections 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85b40a3d20 0x7f85b40a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85c8107b50 0x7f85c819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85c80696c0 0x7f85c81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 --2- 172.21.15.122:0/294216197 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85c8068d00 0x7f85c8198390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 >> 172.21.15.122:0/294216197 conn(0x7f85c8077b00 msgr2=0x7f85c80fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 shutdown_connections 2024-04-02T13:42:21.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.169+0000 7f85cf71c640 1 -- 172.21.15.122:0/294216197 wait complete. 2024-04-02T13:42:21.181 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:42:21.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 -- 172.21.15.122:0/275821008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100bc0 msgr2=0x7f99801066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 --2- 172.21.15.122:0/275821008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100bc0 0x7f99801066f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9974009900 tx=0x7f997402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 -- 172.21.15.122:0/275821008 shutdown_connections 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 --2- 172.21.15.122:0/275821008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9980106dd0 0x7f998010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 --2- 172.21.15.122:0/275821008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100bc0 0x7f99801066f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 --2- 172.21.15.122:0/275821008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980100210 0x7f99801005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 -- 172.21.15.122:0/275821008 >> 172.21.15.122:0/275821008 conn(0x7f9980076f20 msgr2=0x7f99800fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 -- 172.21.15.122:0/275821008 shutdown_connections 2024-04-02T13:42:21.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.286+0000 7f99855a8640 1 -- 172.21.15.122:0/275821008 wait complete. 2024-04-02T13:42:21.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 Processor -- start 2024-04-02T13:42:21.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 -- start start 2024-04-02T13:42:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 0x7f9980198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 0x7f998019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9980111e80 con 0x7f9980100210 2024-04-02T13:42:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9980111d00 con 0x7f9980106dd0 2024-04-02T13:42:21.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f99855a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9980112000 con 0x7f998019cc80 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41178/0 (socket says 172.21.15.122:41178) 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997effd640 1 -- 172.21.15.122:0/2978270676 learned_addr learned my addr 172.21.15.122:0/2978270676 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 0x7f998019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997e7fc640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 0x7f9980198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 -- 172.21.15.122:0/2978270676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 msgr2=0x7f9980198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 0x7f9980198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 -- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 msgr2=0x7f99801983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997f7fe640 1 -- 172.21.15.122:0/2978270676 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998019d770 con 0x7f998019cc80 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997effd640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.287+0000 7f997e7fc640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 0x7f9980198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f997f7fe640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 0x7f998019d130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f997800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f9967fff640 1 -- 172.21.15.122:0/2978270676 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997800ce20 con 0x7f998019cc80 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f9967fff640 1 -- 172.21.15.122:0/2978270676 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9978005020 con 0x7f998019cc80 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f9967fff640 1 -- 172.21.15.122:0/2978270676 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978010720 con 0x7f998019cc80 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998019da60 con 0x7f998019cc80 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.288+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99801a51d0 con 0x7f998019cc80 2024-04-02T13:42:21.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.290+0000 7f9967fff640 1 -- 172.21.15.122:0/2978270676 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9978005540 con 0x7f998019cc80 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.291+0000 7f9967fff640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 0x7f99680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.291+0000 7f9967fff640 1 -- 172.21.15.122:0/2978270676 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f99780d1130 con 0x7f998019cc80 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.291+0000 7f997effd640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 0x7f99680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.291+0000 7f9988191740 1 Processor -- start 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.291+0000 7f9988191740 1 -- start start 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997effd640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 0x7f99680a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9970011550 tx=0x7f9970005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8461b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a846d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631a83c57b0 con 0x5631a84620c0 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631a8364570 con 0x5631a846aca0 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f9988191740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631a83c1760 con 0x5631a84617a0 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41182/0 (socket says 172.21.15.122:41182) 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 -- 172.21.15.122:0/1000292591 learned_addr learned my addr 172.21.15.122:0/1000292591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997effd640 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8461b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 -- 172.21.15.122:0/1000292591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 msgr2=0x5631a8461b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8461b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 -- 172.21.15.122:0/1000292591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 msgr2=0x5631a846d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a846d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 -- 172.21.15.122:0/1000292591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a80d93f0 con 0x5631a84620c0 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9980074220 tx=0x7f9974005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84cdf7a3494451e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.292+0000 7f99657fa640 1 -- 172.21.15.122:0/1000292591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974004a70 con 0x5631a84620c0 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f99657fa640 1 -- 172.21.15.122:0/1000292591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9974004c10 con 0x5631a84620c0 2024-04-02T13:42:21.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f99657fa640 1 -- 172.21.15.122:0/1000292591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974041c60 con 0x5631a84620c0 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 -- 172.21.15.122:0/1000292591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 msgr2=0x5631a846a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9980074220 tx=0x7f9974005e60 comp rx=0 tx=0).stop 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 -- 172.21.15.122:0/1000292591 shutdown_connections 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a846d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a846a760 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 --2- 172.21.15.122:0/1000292591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8461b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 -- 172.21.15.122:0/1000292591 >> 172.21.15.122:0/1000292591 conn(0x5631a845bed0 msgr2=0x5631a845c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 -- 172.21.15.122:0/1000292591 shutdown_connections 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.293+0000 7f9988191740 1 -- 172.21.15.122:0/1000292591 wait complete. 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 -- start start 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 Processor -- start 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 0x5631a846a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 0x5631a8467ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a84681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631a846da30 con 0x5631a84617a0 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631a8380e80 con 0x5631a846aca0 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631a846dbb0 con 0x5631a84620c0 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 0x5631a846a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f9988191740 10 client.0 _ll_register_callbacks cb 0x5631a83bcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 0x5631a846a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41198/0 (socket says 172.21.15.122:41198) 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997effd640 1 -- 172.21.15.122:0/3723377993 learned_addr learned my addr 172.21.15.122:0/3723377993 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 0x5631a8467ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997f7fe640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a84681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 -- 172.21.15.122:0/3723377993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 msgr2=0x5631a84681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a84681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 -- 172.21.15.122:0/3723377993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 msgr2=0x5631a846a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 0x5631a846a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.294+0000 7f997e7fc640 1 -- 172.21.15.122:0/3723377993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a81a37d0 con 0x5631a84620c0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f997e7fc640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 0x5631a8467ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9974036040 tx=0x7f997402cd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c86de4062cbf1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974052060 con 0x5631a84620c0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f9974041bf0 con 0x5631a84620c0 2024-04-02T13:42:21.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9974004c10 con 0x5631a84620c0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9974059050 con 0x5631a84620c0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 10 client.26785 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5631a846da30 con 0x5631a84620c0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9974004c10 con 0x5631a84620c0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f997404a600 con 0x5631a84620c0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 10 client.26785 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 10 client.26785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 20 client.26785 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a847ac70 0x5631a847d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 4 client.26785 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.295+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a83d2cf0 con 0x5631a847ac70 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.296+0000 7f997effd640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a847ac70 0x5631a847d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.296+0000 7f9962ff5640 10 client.26785 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.296+0000 7f997effd640 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a847ac70 0x5631a847d060 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f5b2be497d2fc24 server_cookie=b3b0023682054cc2 in_seq=0 out_seq=0 2024-04-02T13:42:21.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9962ff5640 1 -- 172.21.15.122:0/3723377993 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631a83d2cf0 con 0x5631a847ac70 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9962ff5640 10 client.26785 handle_command_reply: tid=0 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 client.26785 shutdown 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 msgr2=0x5631a8467ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 0x5631a8467ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9974036040 tx=0x7f997402cd60 comp rx=0 tx=0).stop 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 shutdown_connections 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a847ac70 0x5631a847d060 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a84681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84620c0 0x5631a8467ab0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9962ff5640 0 client.26785 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 --2- 172.21.15.122:0/3723377993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84617a0 0x5631a846a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 >> 172.21.15.122:0/3723377993 conn(0x5631a845bed0 msgr2=0x5631a845dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 shutdown_connections 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 1 -- 172.21.15.122:0/3723377993 wait complete. 2024-04-02T13:42:21.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.297+0000 7f9988191740 20 client.26785 trim_cache size 0 max 16384 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 Processor -- start 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 -- start start 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8476920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a845c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99801acdc0 con 0x5631a84620c0 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99801ae9c0 con 0x5631a846aca0 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.310+0000 7f9988191740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99801ad490 con 0x5631a84617a0 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8476920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41200/0 (socket says 172.21.15.122:41200) 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 -- 172.21.15.122:0/1547436652 learned_addr learned my addr 172.21.15.122:0/1547436652 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997f7fe640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a845c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 -- 172.21.15.122:0/1547436652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 msgr2=0x5631a8476920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8476920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 -- 172.21.15.122:0/1547436652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 msgr2=0x5631a845c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a845c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 -- 172.21.15.122:0/1547436652 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8461ef0 con 0x5631a84620c0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997effd640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8476920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997f7fe640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a845c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f997e7fc640 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9974000c00 tx=0x7f9974005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c985721f89fc7ace server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f9964ff9640 1 -- 172.21.15.122:0/1547436652 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974052070 con 0x5631a84620c0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f9964ff9640 1 -- 172.21.15.122:0/1547436652 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997402cbb0 con 0x5631a84620c0 2024-04-02T13:42:21.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.311+0000 7f9964ff9640 1 -- 172.21.15.122:0/1547436652 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974005210 con 0x5631a84620c0 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- 172.21.15.122:0/1547436652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 msgr2=0x5631a847d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9974000c00 tx=0x7f9974005b50 comp rx=0 tx=0).stop 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- 172.21.15.122:0/1547436652 shutdown_connections 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a846aca0 0x5631a845c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a847d060 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 --2- 172.21.15.122:0/1547436652 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a84617a0 0x5631a8476920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- 172.21.15.122:0/1547436652 >> 172.21.15.122:0/1547436652 conn(0x5631a845bed0 msgr2=0x5631a8467340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- 172.21.15.122:0/1547436652 shutdown_connections 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- 172.21.15.122:0/1547436652 wait complete. 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 -- start start 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.312+0000 7f9988191740 1 Processor -- start 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 0x5631a8465790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a846aca0 0x5631a8464460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99801ae230 con 0x5631a84620c0 2024-04-02T13:42:21.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99801adb60 con 0x5631a84617a0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99801aadc0 con 0x5631a846aca0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 10 client.0 _ll_register_callbacks cb 0x7f9970009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41206/0 (socket says 172.21.15.122:41206) 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 0x5631a8465790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 -- 172.21.15.122:0/976697666 learned_addr learned my addr 172.21.15.122:0/976697666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997f7fe640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a846aca0 0x5631a8464460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 -- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a846aca0 msgr2=0x5631a8464460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a846aca0 0x5631a8464460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 -- 172.21.15.122:0/976697666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 msgr2=0x5631a8465790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 0x5631a8465790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 -- 172.21.15.122:0/976697666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8471e30 con 0x5631a84620c0 2024-04-02T13:42:21.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997effd640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 0x5631a8465790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f997e7fc640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9974004ed0 tx=0x7f9974041970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9df49db9d5a0b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9962ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9974052070 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f997402cbb0 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.313+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f997404a470 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f997406e050 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 10 client.17811 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5631a8471e30 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997402cbb0 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f9974065070 con 0x5631a84620c0 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 10 client.17811 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 10 client.17811 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 20 client.17811 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a84732a0 0x5631a84e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 4 client.17811 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631a83cd040 con 0x5631a84732a0 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.314+0000 7f997effd640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a84732a0 0x5631a84e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f997effd640 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a84732a0 0x5631a84e3000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=85b33c115dc588 server_cookie=a1488b86be60db01 in_seq=0 out_seq=0 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9962ff5640 10 client.17811 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9962ff5640 1 -- 172.21.15.122:0/976697666 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631a83cd040 con 0x5631a84732a0 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9962ff5640 10 client.17811 handle_command_reply: tid=0 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 client.17811 shutdown 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 msgr2=0x5631a8463d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f9974004ed0 tx=0x7f9974041970 comp rx=0 tx=0).stop 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 shutdown_connections 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5631a84732a0 0x5631a84e3000 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631a846aca0 0x5631a8464460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631a84620c0 0x5631a8463d20 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 --2- 172.21.15.122:0/976697666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631a84617a0 0x5631a8465790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9962ff5640 0 client.17811 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 >> 172.21.15.122:0/976697666 conn(0x5631a845bed0 msgr2=0x5631a8466ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 shutdown_connections 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 1 -- 172.21.15.122:0/976697666 wait complete. 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f9988191740 20 client.17811 trim_cache size 0 max 16384 2024-04-02T13:42:21.329 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 msgr2=0x7f99680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 0x7f99680a6260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9970011550 tx=0x7f9970005f90 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 msgr2=0x7f998019d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.315+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 0x7f998019d130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9978009fd0 tx=0x7f997800ef10 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 shutdown_connections 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99680a3da0 0x7f99680a6260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f998019cc80 0x7f998019d130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9980106dd0 0x7f9980198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 --2- 172.21.15.122:0/2978270676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9980100210 0x7f99801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 >> 172.21.15.122:0/2978270676 conn(0x7f9980076f20 msgr2=0x7f9980104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 shutdown_connections 2024-04-02T13:42:21.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.316+0000 7f99855a8640 1 -- 172.21.15.122:0/2978270676 wait complete. 2024-04-02T13:42:21.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/4013346379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc106980 msgr2=0x7fb4dc106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/4013346379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc106980 0x7fb4dc106d60 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009930 tx=0x7fb4d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/4013346379 shutdown_connections 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/4013346379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc101330 0x7fb4dc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/4013346379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc100970 0x7fb4dc100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/4013346379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc106980 0x7fb4dc106d60 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/4013346379 >> 172.21.15.122:0/4013346379 conn(0x7fb4dc077b00 msgr2=0x7fb4dc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/4013346379 shutdown_connections 2024-04-02T13:42:21.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.437+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/4013346379 wait complete. 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 Processor -- start 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 -- start start 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc100970 0x7fb4dc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc106980 0x7fb4dc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc110d90 con 0x7fb4dc101330 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc110c10 con 0x7fb4dc106980 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e2cd6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc110f10 con 0x7fb4dc100970 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41234/0 (socket says 172.21.15.122:41234) 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 -- 172.21.15.122:0/538874529 learned_addr learned my addr 172.21.15.122:0/538874529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc100970 0x7fb4dc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4e1273640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc106980 0x7fb4dc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 -- 172.21.15.122:0/538874529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc100970 msgr2=0x7fb4dc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc100970 0x7fb4dc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 -- 172.21.15.122:0/538874529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc106980 msgr2=0x7fb4dc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc106980 0x7fb4dc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 -- 172.21.15.122:0/538874529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4dc19d3f0 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.438+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c890 tx=0x7fb4c800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.439+0000 7fb4d9ffb640 1 -- 172.21.15.122:0/538874529 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8004220 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.439+0000 7fb4d9ffb640 1 -- 172.21.15.122:0/538874529 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c80043c0 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.439+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4dc19d6e0 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.439+0000 7fb4d9ffb640 1 -- 172.21.15.122:0/538874529 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c80049e0 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.439+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4dc1a4f70 con 0x7fb4dc101330 2024-04-02T13:42:21.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.441+0000 7fb4d9ffb640 1 -- 172.21.15.122:0/538874529 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb4c80079b0 con 0x7fb4dc101330 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.441+0000 7fb4d9ffb640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 0x7fb4cc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.442+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 0x7fb4cc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.442+0000 7fb4d9ffb640 1 -- 172.21.15.122:0/538874529 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb4c80d0c20 con 0x7fb4dc101330 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.442+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 0x7fb4cc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005ab0 tx=0x7fb4d0004f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.442+0000 7fb4e5997740 1 Processor -- start 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.442+0000 7fb4e5997740 1 -- start start 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d629b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d632760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b30d58d7b0 con 0x55b30d632ca0 2024-04-02T13:42:21.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b30d52c570 con 0x55b30d62a0c0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e5997740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b30d589760 con 0x55b30d6297a0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41238/0 (socket says 172.21.15.122:41238) 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 -- 172.21.15.122:0/3183605314 learned_addr learned my addr 172.21.15.122:0/3183605314 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d629b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d632760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 -- 172.21.15.122:0/3183605314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 msgr2=0x55b30d629b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d629b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 -- 172.21.15.122:0/3183605314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 msgr2=0x55b30d632760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d632760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 -- 172.21.15.122:0/3183605314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30d2a13f0 con 0x55b30d632ca0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009f20 tx=0x7fb4d400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2c2080c564dbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4c6ffd640 1 -- 172.21.15.122:0/3183605314 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401a070 con 0x55b30d632ca0 2024-04-02T13:42:21.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4c6ffd640 1 -- 172.21.15.122:0/3183605314 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d40158f0 con 0x55b30d632ca0 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.443+0000 7fb4c6ffd640 1 -- 172.21.15.122:0/3183605314 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400a900 con 0x55b30d632ca0 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- 172.21.15.122:0/3183605314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 msgr2=0x55b30d635090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009f20 tx=0x7fb4d400c8b0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- 172.21.15.122:0/3183605314 shutdown_connections 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d635090 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d632760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3183605314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d629b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- 172.21.15.122:0/3183605314 >> 172.21.15.122:0/3183605314 conn(0x55b30d623ed0 msgr2=0x55b30d6242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- 172.21.15.122:0/3183605314 shutdown_connections 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- 172.21.15.122:0/3183605314 wait complete. 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.444+0000 7fb4e5997740 1 -- start start 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 Processor -- start 2024-04-02T13:42:21.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d632630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d62f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b30d635a90 con 0x55b30d632ca0 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b30d548e80 con 0x55b30d62a0c0 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b30d635c10 con 0x55b30d6297a0 2024-04-02T13:42:21.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e5997740 10 client.0 _ll_register_callbacks cb 0x55b30d584aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41254/0 (socket says 172.21.15.122:41254) 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 -- 172.21.15.122:0/3731365963 learned_addr learned my addr 172.21.15.122:0/3731365963 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d62f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d632630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 -- 172.21.15.122:0/3731365963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 msgr2=0x55b30d632630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d632630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 -- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 msgr2=0x55b30d62f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d62f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4e1273640 1 -- 172.21.15.122:0/3731365963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30d36b7d0 con 0x55b30d632ca0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.445+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d62f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e1273640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400a4e0 tx=0x7fb4d4015cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f7e0e033bdffe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401a040 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb4d4012040 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d40158f0 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4d4015e20 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 10 client.17823 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b30d641a50 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d40158f0 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fb4d401f3b0 con 0x55b30d632ca0 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 10 client.17823 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 20 client.17823 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d642eb0 0x55b30d6452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 4 client.17823 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b30d59acf0 con 0x55b30d642eb0 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.446+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d642eb0 0x55b30d6452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.447+0000 7fb4bbfff640 10 client.17823 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.447+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d642eb0 0x55b30d6452a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d711a38031df4b1 server_cookie=cf7da4a6c5967299 in_seq=0 out_seq=0 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.447+0000 7fb4bbfff640 1 -- 172.21.15.122:0/3731365963 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b30d59acf0 con 0x55b30d642eb0 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.447+0000 7fb4bbfff640 10 client.17823 handle_command_reply: tid=0 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 client.17823 shutdown 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 msgr2=0x55b30d62ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400a4e0 tx=0x7fb4d4015cb0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 shutdown_connections 2024-04-02T13:42:21.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d642eb0 0x55b30d6452a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d632ca0 0x55b30d62ff40 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d62a0c0 0x55b30d62f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 --2- 172.21.15.122:0/3731365963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d6297a0 0x55b30d632630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 >> 172.21.15.122:0/3731365963 conn(0x55b30d623ed0 msgr2=0x55b30d625c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4bbfff640 0 client.17823 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 shutdown_connections 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 1 -- 172.21.15.122:0/3731365963 wait complete. 2024-04-02T13:42:21.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.448+0000 7fb4e5997740 20 client.17823 trim_cache size 0 max 16384 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 Processor -- start 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 -- start start 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d63e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d632630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1af200 con 0x55b30d62a0c0 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1b0e00 con 0x55b30d6297a0 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4e5997740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1af8d0 con 0x55b30d632ca0 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41264/0 (socket says 172.21.15.122:41264) 2024-04-02T13:42:21.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.461+0000 7fb4dbfff640 1 -- 172.21.15.122:0/1311060837 learned_addr learned my addr 172.21.15.122:0/1311060837 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d63e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 -- 172.21.15.122:0/1311060837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 msgr2=0x55b30d632630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d632630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 -- 172.21.15.122:0/1311060837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 msgr2=0x55b30d63e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d63e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 -- 172.21.15.122:0/1311060837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b30d62df10 con 0x55b30d62a0c0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d63e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb4c809c4d0 tx=0x7fb4c80d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e371d1590083de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4c67fc640 1 -- 172.21.15.122:0/1311060837 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c801d040 con 0x55b30d62a0c0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4c67fc640 1 -- 172.21.15.122:0/1311060837 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c80a0b80 con 0x55b30d62a0c0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.462+0000 7fb4c67fc640 1 -- 172.21.15.122:0/1311060837 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8007ca0 con 0x55b30d62a0c0 2024-04-02T13:42:21.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- 172.21.15.122:0/1311060837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 msgr2=0x55b30d6320f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb4c809c4d0 tx=0x7fb4c80d0040 comp rx=0 tx=0).stop 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- 172.21.15.122:0/1311060837 shutdown_connections 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d632630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d6320f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 --2- 172.21.15.122:0/1311060837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d63e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- 172.21.15.122:0/1311060837 >> 172.21.15.122:0/1311060837 conn(0x55b30d623ed0 msgr2=0x55b30d6242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- 172.21.15.122:0/1311060837 shutdown_connections 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- 172.21.15.122:0/1311060837 wait complete. 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 -- start start 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.463+0000 7fb4e5997740 1 Processor -- start 2024-04-02T13:42:21.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d6336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d639e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1b0670 con 0x55b30d62a0c0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1affa0 con 0x55b30d6297a0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4dc1ad200 con 0x55b30d632ca0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e5997740 10 client.0 _ll_register_callbacks cb 0x7fb4d4018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41278/0 (socket says 172.21.15.122:41278) 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 -- 172.21.15.122:0/2452945634 learned_addr learned my addr 172.21.15.122:0/2452945634 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d6336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4e1273640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d639e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 -- 172.21.15.122:0/2452945634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 msgr2=0x55b30d639e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d639e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 -- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 msgr2=0x55b30d6336a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d6336a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 -- 172.21.15.122:0/2452945634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b30d63a3d0 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4dbfff640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a308a1908eee34 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.464+0000 7fb4bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c801d040 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb4c809aad0 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c80a0b80 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb4c80a5440 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 10 client.17829 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b30d63b6d0 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c80a0b80 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7fb4c809bdb0 con 0x55b30d62a0c0 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 10 client.17829 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:21.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 10 client.17829 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 20 client.17829 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d6a9650 0x55b30d6aba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 4 client.17829 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b30d595040 con 0x55b30d6a9650 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.465+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d6a9650 0x55b30d6aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4bbfff640 10 client.17829 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e0a72640 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d6a9650 0x55b30d6aba70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae1f61966408d19 server_cookie=5fe01f1d66b67ff8 in_seq=0 out_seq=0 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4bbfff640 1 -- 172.21.15.122:0/2452945634 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b30d595040 con 0x55b30d6a9650 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4bbfff640 10 client.17829 handle_command_reply: tid=0 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 client.17829 shutdown 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 msgr2=0x55b30d633be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8000c00 tx=0x7fb4c80046c0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 shutdown_connections 2024-04-02T13:42:21.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b30d6a9650 0x55b30d6aba70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b30d632ca0 0x55b30d639e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4bbfff640 0 client.17829 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b30d62a0c0 0x55b30d633be0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 --2- 172.21.15.122:0/2452945634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b30d6297a0 0x55b30d6336a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 >> 172.21.15.122:0/2452945634 conn(0x55b30d623ed0 msgr2=0x55b30d62f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 shutdown_connections 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 1 -- 172.21.15.122:0/2452945634 wait complete. 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.466+0000 7fb4e5997740 20 client.17829 trim_cache size 0 max 16384 2024-04-02T13:42:21.479 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 msgr2=0x7fb4cc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 0x7fb4cc0a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005ab0 tx=0x7fb4d0004f10 comp rx=0 tx=0).stop 2024-04-02T13:42:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 msgr2=0x7fb4dc198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800c890 tx=0x7fb4c800cd50 comp rx=0 tx=0).stop 2024-04-02T13:42:21.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 shutdown_connections 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4cc0a3d20 0x7fb4cc0a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb4dc106980 0x7fb4dc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb4dc101330 0x7fb4dc198940 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 --2- 172.21.15.122:0/538874529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4dc100970 0x7fb4dc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 >> 172.21.15.122:0/538874529 conn(0x7fb4dc077b00 msgr2=0x7fb4dc068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:21.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 shutdown_connections 2024-04-02T13:42:21.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:21.467+0000 7fb4e2cd6640 1 -- 172.21.15.122:0/538874529 wait complete. 2024-04-02T13:42:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:21 smithi187 ceph-mon[36064]: pgmap v220: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:21 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1307488065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:21 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:21 smithi188 ceph-mon[36117]: pgmap v220: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:21 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1307488065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:21 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:21 smithi122 ceph-mon[32114]: pgmap v220: 129 pgs: 129 active+clean; 151 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-04-02T13:42:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:21 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1307488065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:21 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:42:21.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 220756 2.93 MB/sec execute 222 sec latency 1432.636 ms 2024-04-02T13:42:22.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:42:22.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:42:22.484 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:42:22.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 -- 172.21.15.122:0/2516131900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c44106e30 msgr2=0x7f4c4410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 --2- 172.21.15.122:0/2516131900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c44106e30 0x7f4c4410ea40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009900 tx=0x7f4c3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:22.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 -- 172.21.15.122:0/2516131900 shutdown_connections 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 --2- 172.21.15.122:0/2516131900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c44106e30 0x7f4c4410ea40 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 --2- 172.21.15.122:0/2516131900 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44101020 0x7f4c44106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 --2- 172.21.15.122:0/2516131900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44100670 0x7f4c44100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 -- 172.21.15.122:0/2516131900 >> 172.21.15.122:0/2516131900 conn(0x7f4c44075c60 msgr2=0x7f4c44076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 -- 172.21.15.122:0/2516131900 shutdown_connections 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 -- 172.21.15.122:0/2516131900 wait complete. 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.599+0000 7f4c490b1640 1 Processor -- start 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 -- start start 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 0x7f4c44198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 0x7f4c441988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44111f20 con 0x7f4c4419cc20 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c44111da0 con 0x7f4c44100670 2024-04-02T13:42:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c490b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441120a0 con 0x7f4c44106e30 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41302/0 (socket says 172.21.15.122:41302) 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 -- 172.21.15.122:0/872365024 learned_addr learned my addr 172.21.15.122:0/872365024 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4259c640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 0x7f4c441988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 0x7f4c44198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 -- 172.21.15.122:0/872365024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 msgr2=0x7f4c441988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 0x7f4c441988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 -- 172.21.15.122:0/872365024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 msgr2=0x7f4c44198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 0x7f4c44198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 -- 172.21.15.122:0/872365024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4419d710 con 0x7f4c4419cc20 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4259c640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 0x7f4c441988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 0x7f4c44198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.600+0000 7f4c4359e640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009230 tx=0x7f4c3c02cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.601+0000 7f4c2bfff640 1 -- 172.21.15.122:0/872365024 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c041070 con 0x7f4c4419cc20 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.601+0000 7f4c2bfff640 1 -- 172.21.15.122:0/872365024 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c3c004450 con 0x7f4c4419cc20 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.601+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4419d9a0 con 0x7f4c4419cc20 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.601+0000 7f4c2bfff640 1 -- 172.21.15.122:0/872365024 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c003d90 con 0x7f4c4419cc20 2024-04-02T13:42:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.601+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c441a5160 con 0x7f4c4419cc20 2024-04-02T13:42:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.603+0000 7f4c2bfff640 1 -- 172.21.15.122:0/872365024 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4c3c044070 con 0x7f4c4419cc20 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.603+0000 7f4c4bd73740 1 Processor -- start 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.603+0000 7f4c4bd73740 1 -- start start 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccbc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eaccc20770 con 0x55eacccbcbe0 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eaccbc8960 con 0x55eacccc57c0 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eaccc1c8c0 con 0x55eacccbc2c0 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c2bfff640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 0x7f4c2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41306/0 (socket says 172.21.15.122:41306) 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 -- 172.21.15.122:0/317557567 learned_addr learned my addr 172.21.15.122:0/317557567 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4359e640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 0x7f4c2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4259c640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccbc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c2bfff640 1 -- 172.21.15.122:0/872365024 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f4c3c0f27d0 con 0x7f4c4419cc20 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4259c640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 -- 172.21.15.122:0/317557567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 msgr2=0x55eacccbc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccbc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 -- 172.21.15.122:0/317557567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 msgr2=0x55eacccc7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 -- 172.21.15.122:0/317557567 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacc9fc7e0 con 0x55eacccbcbe0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4259c640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400f390 tx=0x7f4c3400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1256ad5ea7c5cd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c4359e640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 0x7f4c2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0be3c0 tx=0x7f4c3c08fe70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.604+0000 7f4c297fa640 1 -- 172.21.15.122:0/317557567 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34003b70 con 0x55eacccbcbe0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c297fa640 1 -- 172.21.15.122:0/317557567 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34003d10 con 0x55eacccbcbe0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c297fa640 1 -- 172.21.15.122:0/317557567 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c34011a60 con 0x55eacccbcbe0 2024-04-02T13:42:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 -- 172.21.15.122:0/317557567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 msgr2=0x55eacccc5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400f390 tx=0x7f4c3400f850 comp rx=0 tx=0).stop 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 -- 172.21.15.122:0/317557567 shutdown_connections 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbcbe0 0x55eacccc5280 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/317557567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccbc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 -- 172.21.15.122:0/317557567 >> 172.21.15.122:0/317557567 conn(0x55eacccb69f0 msgr2=0x55eacccb6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 -- 172.21.15.122:0/317557567 shutdown_connections 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.605+0000 7f4c4bd73740 1 -- 172.21.15.122:0/317557567 wait complete. 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 -- start start 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 Processor -- start 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eacccc2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eacccc85b0 con 0x55eacccbc2c0 2024-04-02T13:42:22.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eaccbdbd20 con 0x55eacccc57c0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eacccc8730 con 0x55eacccbcbe0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4bd73740 10 client.0 _ll_register_callbacks cb 0x55eaccba8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41318/0 (socket says 172.21.15.122:41318) 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 -- 172.21.15.122:0/322189864 learned_addr learned my addr 172.21.15.122:0/322189864 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 -- 172.21.15.122:0/322189864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 msgr2=0x55eacccc2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eacccc2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eacccc2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 -- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 msgr2=0x55eacccc31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 -- 172.21.15.122:0/322189864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eacca5cf70 con 0x55eacccbc2c0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.606+0000 7f4c4259c640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800bfa0 tx=0x7f4c38013ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733308aeea33cdff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:22.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38013110 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f4c38004050 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c34003d10 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4c38022050 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 10 client.17841 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eacc8f4170 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c34003d10 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f4c38011440 con 0x55eacccbc2c0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 10 client.17841 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 20 client.17841 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eacccd5b20 0x55eacccd7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 4 client.17841 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.607+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eaccc2dcb0 con 0x55eacccd5b20 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.608+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eacccd5b20 0x55eacccd7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.608+0000 7f4c26ff5640 10 client.17841 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.608+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eacccd5b20 0x55eacccd7f70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b91c3cd6f5c7f4 server_cookie=71d9b6ec518ba2e4 in_seq=0 out_seq=0 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c26ff5640 1 -- 172.21.15.122:0/322189864 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eaccc2dcb0 con 0x55eacccd5b20 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c26ff5640 10 client.17841 handle_command_reply: tid=0 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 client.17841 shutdown 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 msgr2=0x55eacccc2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f4c3800bfa0 tx=0x7f4c38013ee0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 shutdown_connections 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eacccd5b20 0x55eacccd7f70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccc57c0 0x55eacccc31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eacccc2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c26ff5640 0 client.17841 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/322189864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccbc2c0 0x55eacccc2750 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 >> 172.21.15.122:0/322189864 conn(0x55eacccb69f0 msgr2=0x55eacccb8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 shutdown_connections 2024-04-02T13:42:22.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 1 -- 172.21.15.122:0/322189864 wait complete. 2024-04-02T13:42:22.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.609+0000 7f4c4bd73740 20 client.17841 trim_cache size 0 max 16384 2024-04-02T13:42:22.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.626+0000 7f4c4bd73740 1 Processor -- start 2024-04-02T13:42:22.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.626+0000 7f4c4bd73740 1 -- start start 2024-04-02T13:42:22.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccc4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbcbe0 0x55eacccd14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441af080 con 0x55eacccc57c0 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441b0c80 con 0x55eacccbcbe0 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441af750 con 0x55eacccbc2c0 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41328/0 (socket says 172.21.15.122:41328) 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 -- 172.21.15.122:0/895971904 learned_addr learned my addr 172.21.15.122:0/895971904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 -- 172.21.15.122:0/895971904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 msgr2=0x55eacccc4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccc4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 -- 172.21.15.122:0/895971904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbcbe0 msgr2=0x55eacccd14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbcbe0 0x55eacccd14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 -- 172.21.15.122:0/895971904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eacccc1720 con 0x55eacccc57c0 2024-04-02T13:42:22.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c4259c640 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001eb0 tx=0x7f4c38013670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca846dd04e843fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c28ff9640 1 -- 172.21.15.122:0/895971904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3802b020 con 0x55eacccc57c0 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c28ff9640 1 -- 172.21.15.122:0/895971904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c3800b780 con 0x55eacccc57c0 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.627+0000 7f4c28ff9640 1 -- 172.21.15.122:0/895971904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3801c8f0 con 0x55eacccc57c0 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- 172.21.15.122:0/895971904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 msgr2=0x55eacccdd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001eb0 tx=0x7f4c38013670 comp rx=0 tx=0).stop 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- 172.21.15.122:0/895971904 shutdown_connections 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccdd7f0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbcbe0 0x55eacccd14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/895971904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbc2c0 0x55eacccc4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- 172.21.15.122:0/895971904 >> 172.21.15.122:0/895971904 conn(0x55eacccb69f0 msgr2=0x55eacccb8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- 172.21.15.122:0/895971904 shutdown_connections 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- 172.21.15.122:0/895971904 wait complete. 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- start start 2024-04-02T13:42:22.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 Processor -- start 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbc2c0 0x55eaccccd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eaccccdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441b04f0 con 0x55eacccc57c0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441afe20 con 0x55eacccbc2c0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c441ad080 con 0x55eacccbcbe0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4bd73740 10 client.0 _ll_register_callbacks cb 0x7f4c3800b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.628+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41336/0 (socket says 172.21.15.122:41336) 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 -- 172.21.15.122:0/857209937 learned_addr learned my addr 172.21.15.122:0/857209937 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eaccccdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbc2c0 0x55eaccccd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 -- 172.21.15.122:0/857209937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 msgr2=0x55eaccccdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eaccccdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 -- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbc2c0 msgr2=0x55eaccccd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbc2c0 0x55eaccccd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 -- 172.21.15.122:0/857209937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eaccccb050 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4259c640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4c38005710 tx=0x7f4c38005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb9d251047117c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c38011c30 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f4c38005020 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c3800b780 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f4c3802b020 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 10 client.17847 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55eaccd3ca30 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c3800b780 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f4c38031430 con 0x55eacccc57c0 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 10 client.17847 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:22.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 10 client.17847 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 20 client.17847 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.629+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eaccd3d110 0x55eaccd3f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.630+0000 7f4c4bd73740 4 client.17847 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.630+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eaccc28000 con 0x55eaccd3d110 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.630+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eaccd3d110 0x55eaccd3f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.630+0000 7f4c42d9d640 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eaccd3d110 0x55eaccd3f530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f699c38177819705 server_cookie=c3544cc599aa9a70 in_seq=0 out_seq=0 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.630+0000 7f4c26ff5640 10 client.17847 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c26ff5640 1 -- 172.21.15.122:0/857209937 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eaccc28000 con 0x55eaccd3d110 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c26ff5640 10 client.17847 handle_command_reply: tid=0 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 client.17847 shutdown 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 msgr2=0x55eacccca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f4c38005710 tx=0x7f4c38005da0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 shutdown_connections 2024-04-02T13:42:22.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55eaccd3d110 0x55eaccd3f530 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eacccc57c0 0x55eacccca9b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eacccbcbe0 0x55eaccccdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 --2- 172.21.15.122:0/857209937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eacccbc2c0 0x55eaccccd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 >> 172.21.15.122:0/857209937 conn(0x55eacccb69f0 msgr2=0x55eacccb5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.631+0000 7f4c26ff5640 0 client.17847 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 shutdown_connections 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c4bd73740 1 -- 172.21.15.122:0/857209937 wait complete. 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c4bd73740 20 client.17847 trim_cache size 0 max 16384 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:22.640 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:42:22.641 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "1c562b1b-f96c-4117-b9d1-5dbd827e6f06", 2024-04-02T13:42:22.641 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:42:22.641 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 msgr2=0x7f4c2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 0x7f4c2c0a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0be3c0 tx=0x7f4c3c08fe70 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 msgr2=0x7f4c4419d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009230 tx=0x7f4c3c02cba0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 shutdown_connections 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4c2c0a3d20 0x7f4c2c0a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c4419cc20 0x7f4c4419d0d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c44106e30 0x7f4c441988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 --2- 172.21.15.122:0/872365024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c44100670 0x7f4c44198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 >> 172.21.15.122:0/872365024 conn(0x7f4c44075c60 msgr2=0x7f4c440fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 shutdown_connections 2024-04-02T13:42:22.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.632+0000 7f4c490b1640 1 -- 172.21.15.122:0/872365024 wait complete. 2024-04-02T13:42:22.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:22.647 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:22.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:22 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:22.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:22 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:22.672 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:22 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:42:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:22 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:22 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:22 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:42:22.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:22 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:42:22.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:22 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:42:22.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:22 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:42:22.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 -- 172.21.15.122:0/4089188511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c0696c0 msgr2=0x7f223c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 --2- 172.21.15.122:0/4089188511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c0696c0 0x7f223c10e960 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f2234009cd0 tx=0x7f223402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 -- 172.21.15.122:0/4089188511 shutdown_connections 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 --2- 172.21.15.122:0/4089188511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c0696c0 0x7f223c10e960 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 --2- 172.21.15.122:0/4089188511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c068d00 0x7f223c069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 --2- 172.21.15.122:0/4089188511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c107b50 0x7f223c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 -- 172.21.15.122:0/4089188511 >> 172.21.15.122:0/4089188511 conn(0x7f223c077b00 msgr2=0x7f223c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 -- 172.21.15.122:0/4089188511 shutdown_connections 2024-04-02T13:42:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.761+0000 7f22418d9640 1 -- 172.21.15.122:0/4089188511 wait complete. 2024-04-02T13:42:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 Processor -- start 2024-04-02T13:42:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 -- start start 2024-04-02T13:42:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 0x7f223c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 0x7f223c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f223c111ea0 con 0x7f223c068d00 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f223c111d20 con 0x7f223c107b50 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22418d9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f223c112020 con 0x7f223c0696c0 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f223affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f223affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41348/0 (socket says 172.21.15.122:41348) 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f223affd640 1 -- 172.21.15.122:0/174259547 learned_addr learned my addr 172.21.15.122:0/174259547 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f22327fc640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 0x7f223c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.762+0000 7f223b7fe640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 0x7f223c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 -- 172.21.15.122:0/174259547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 msgr2=0x7f223c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 0x7f223c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 -- 172.21.15.122:0/174259547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 msgr2=0x7f223c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 0x7f223c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 -- 172.21.15.122:0/174259547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223c19d2c0 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22327fc640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 0x7f223c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223b7fe640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 0x7f223c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f223affd640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9f0 tx=0x7f222c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22408d7640 1 -- 172.21.15.122:0/174259547 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c00cd80 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223c19d5b0 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223c1a4e40 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22408d7640 1 -- 172.21.15.122:0/174259547 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c005500 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.763+0000 7f22408d7640 1 -- 172.21.15.122:0/174259547 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c010690 con 0x7f223c068d00 2024-04-02T13:42:22.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.765+0000 7f22408d7640 1 -- 172.21.15.122:0/174259547 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f222c0046c0 con 0x7f223c068d00 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 Processor -- start 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 -- start start 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f22408d7640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 0x7f22280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f223b7fe640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 0x7f22280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f22408d7640 1 -- 172.21.15.122:0/174259547 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f222c0d0f50 con 0x7f223c068d00 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30e1f0 0x55843b316890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b316dd0 0x55843b3191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55843b271830 con 0x55843b30d8d0 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55843b219820 con 0x55843b316dd0 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f224459e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55843b26d7e0 con 0x55843b30e1f0 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f223affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f223affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41354/0 (socket says 172.21.15.122:41354) 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f223affd640 1 -- 172.21.15.122:0/2632914460 learned_addr learned my addr 172.21.15.122:0/2632914460 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f22327fc640 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30e1f0 0x55843b316890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.766+0000 7f223b7fe640 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b316dd0 0x55843b3191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223b7fe640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 0x7f22280a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f223402c630 tx=0x7f2234005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 -- 172.21.15.122:0/2632914460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30e1f0 msgr2=0x55843b316890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30e1f0 0x55843b316890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 -- 172.21.15.122:0/2632914460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b316dd0 msgr2=0x55843b3191c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b316dd0 0x55843b3191c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 -- 172.21.15.122:0/2632914460 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843b04f7d0 con 0x55843b30d8d0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f223affd640 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f222c0a5240 tx=0x7f222c09c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e0b8ed1396df489 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f2231ffb640 1 -- 172.21.15.122:0/2632914460 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0a0eb0 con 0x55843b30d8d0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f2231ffb640 1 -- 172.21.15.122:0/2632914460 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c0b6070 con 0x55843b30d8d0 2024-04-02T13:42:22.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.767+0000 7f2231ffb640 1 -- 172.21.15.122:0/2632914460 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0a52e0 con 0x55843b30d8d0 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- 172.21.15.122:0/2632914460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 msgr2=0x55843b30dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f222c0a5240 tx=0x7f222c09c460 comp rx=0 tx=0).stop 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- 172.21.15.122:0/2632914460 shutdown_connections 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b316dd0 0x55843b3191c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30e1f0 0x55843b316890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 --2- 172.21.15.122:0/2632914460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30d8d0 0x55843b30dcb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- 172.21.15.122:0/2632914460 >> 172.21.15.122:0/2632914460 conn(0x55843b308000 msgr2=0x55843b3083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- 172.21.15.122:0/2632914460 shutdown_connections 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- 172.21.15.122:0/2632914460 wait complete. 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.768+0000 7f224459e740 1 -- start start 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 Processor -- start 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b3166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b316dd0 0x55843b313fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55843b319bc0 con 0x55843b30e1f0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55843b22cd30 con 0x55843b30d8d0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55843b319d40 con 0x55843b316dd0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f224459e740 10 client.0 _ll_register_callbacks cb 0x55843b22c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f223affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b3166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b316dd0 0x55843b313fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41356/0 (socket says 172.21.15.122:41356) 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f223affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b3166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:32788/0 (socket says 172.21.15.122:32788) 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 -- 172.21.15.122:0/605133867 learned_addr learned my addr 172.21.15.122:0/605133867 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 -- 172.21.15.122:0/605133867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b316dd0 msgr2=0x55843b313fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b316dd0 0x55843b313fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 -- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 msgr2=0x55843b3166f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b3166f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.769+0000 7f22327fc640 1 -- 172.21.15.122:0/605133867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843af853f0 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f22327fc640 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2224012b10 tx=0x7f222400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8566183bcdebc813 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f221affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400fb40 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f222400fce0 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222c0b6070 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2224004050 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f224459e740 10 client.17859 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.770+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55843af450f0 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c0b6070 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f2224010430 con 0x55843b30e1f0 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 10 client.17859 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 10 client.17859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:22.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 20 client.17859 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b326e20 0x55843b3292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 4 client.17859 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843b27ed70 con 0x55843b326e20 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f223affd640 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b326e20 0x55843b3292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f221affd640 10 client.17859 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.771+0000 7f223affd640 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b326e20 0x55843b3292a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b51c177fbcda68 server_cookie=de66298290c97d7c in_seq=0 out_seq=0 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f221affd640 1 -- 172.21.15.122:0/605133867 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55843b27ed70 con 0x55843b326e20 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f221affd640 10 client.17859 handle_command_reply: tid=0 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 client.17859 shutdown 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 msgr2=0x55843b313a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2224012b10 tx=0x7f222400f3c0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 shutdown_connections 2024-04-02T13:42:22.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b326e20 0x55843b3292a0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b316dd0 0x55843b313fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f221affd640 0 client.17859 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b30e1f0 0x55843b313a80 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 --2- 172.21.15.122:0/605133867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b3166f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.772+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 >> 172.21.15.122:0/605133867 conn(0x55843b308000 msgr2=0x55843b309d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.773+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 shutdown_connections 2024-04-02T13:42:22.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.773+0000 7f224459e740 1 -- 172.21.15.122:0/605133867 wait complete. 2024-04-02T13:42:22.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.773+0000 7f224459e740 20 client.17859 trim_cache size 0 max 16384 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 Processor -- start 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 -- start start 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b3161b0 0x55843b3165f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1acee0 con 0x55843b316dd0 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1aeae0 con 0x55843b30d8d0 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f224459e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ad5b0 con 0x55843b3161b0 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f22327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f22327fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41366/0 (socket says 172.21.15.122:41366) 2024-04-02T13:42:22.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:32796/0 (socket says 172.21.15.122:32796) 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f22327fc640 1 -- 172.21.15.122:0/3795966786 learned_addr learned my addr 172.21.15.122:0/3795966786 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223b7fe640 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b3161b0 0x55843b3165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 -- 172.21.15.122:0/3795966786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b3161b0 msgr2=0x55843b3165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b3161b0 0x55843b3165f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 -- 172.21.15.122:0/3795966786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 msgr2=0x55843b3069b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.798+0000 7f223affd640 1 -- 172.21.15.122:0/3795966786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843b307860 con 0x55843b30d8d0 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.799+0000 7f22327fc640 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.799+0000 7f223affd640 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f222c0a5240 tx=0x7f222c0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adeac2ea679d16df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.799+0000 7f22317fa640 1 -- 172.21.15.122:0/3795966786 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0b6040 con 0x55843b30d8d0 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.799+0000 7f22317fa640 1 -- 172.21.15.122:0/3795966786 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c0a5770 con 0x55843b30d8d0 2024-04-02T13:42:22.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.799+0000 7f22317fa640 1 -- 172.21.15.122:0/3795966786 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222c0a52e0 con 0x55843b30d8d0 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- 172.21.15.122:0/3795966786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 msgr2=0x55843b322a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f222c0a5240 tx=0x7f222c0057b0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- 172.21.15.122:0/3795966786 shutdown_connections 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b3161b0 0x55843b3165f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 --2- 172.21.15.122:0/3795966786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b30d8d0 0x55843b322a80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- 172.21.15.122:0/3795966786 >> 172.21.15.122:0/3795966786 conn(0x55843b308000 msgr2=0x55843b30efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- 172.21.15.122:0/3795966786 shutdown_connections 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- 172.21.15.122:0/3795966786 wait complete. 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 -- start start 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.800+0000 7f224459e740 1 Processor -- start 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 0x55843b310eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1ae350 con 0x55843b316dd0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1adc80 con 0x55843b3161b0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f223c1aaee0 con 0x55843b30d8d0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f223affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 0x55843b310eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f223b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 10 client.0 _ll_register_callbacks cb 0x7f2224012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f223affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36762/0 (socket says 172.21.15.122:36762) 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f223affd640 1 -- 172.21.15.122:0/2240029268 learned_addr learned my addr 172.21.15.122:0/2240029268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:22.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 -- 172.21.15.122:0/2240029268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 msgr2=0x55843b310970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 -- 172.21.15.122:0/2240029268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 msgr2=0x55843b3113f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3113f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 -- 172.21.15.122:0/2240029268 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843b38c590 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f223affd640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f22327fc640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 0x55843b310eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22240109c0 tx=0x7f2224010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a966f434a6c2159 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f221affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222400fd50 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f2224005260 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.801+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f222c0a5770 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2224005480 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 10 client.27026 fetch_fsmap learned FSMap version 38 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55843b38d880 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f222c0a5770 con 0x55843b3161b0 2024-04-02T13:42:22.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f2224026420 con 0x55843b3161b0 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 10 client.27026 fetch_fsmap finished waiting for FSMap version 38 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 10 client.27026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 20 client.27026 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b38e140 0x55843b390560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 4 client.27026 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55843b2790c0 con 0x55843b38e140 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.802+0000 7f223affd640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b38e140 0x55843b390560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f221affd640 10 client.27026 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f223affd640 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b38e140 0x55843b390560 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7130e366bcb6c5b server_cookie=40c455ceb0aac239 in_seq=0 out_seq=0 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f221affd640 1 -- 172.21.15.122:0/2240029268 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55843b2790c0 con 0x55843b38e140 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f221affd640 10 client.27026 handle_command_reply: tid=0 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 client.27026 shutdown 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 msgr2=0x55843b310eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 0x55843b310eb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22240109c0 tx=0x7f2224010cf0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 shutdown_connections 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55843b38e140 0x55843b390560 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55843b316dd0 0x55843b3113f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55843b3161b0 0x55843b310eb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 --2- 172.21.15.122:0/2240029268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55843b30d8d0 0x55843b310970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f221affd640 0 client.27026 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 >> 172.21.15.122:0/2240029268 conn(0x55843b308000 msgr2=0x55843b30e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 shutdown_connections 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 1 -- 172.21.15.122:0/2240029268 wait complete. 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.803+0000 7f224459e740 20 client.27026 trim_cache size 0 max 16384 2024-04-02T13:42:22.818 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "1c562b1b-f96c-4117-b9d1-5dbd827e6f06": { 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "1c562b1b-f96c-4117-b9d1-5dbd827e6f06", 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:42:22.819 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:22.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 msgr2=0x7f22280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 0x7f22280a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f223402c630 tx=0x7f2234005d80 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 msgr2=0x7f223c198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f222c00e9f0 tx=0x7f222c00edf0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 shutdown_connections 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f22280a3da0 0x7f22280a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f223c107b50 0x7f223c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f223c0696c0 0x7f223c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 --2- 172.21.15.122:0/174259547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f223c068d00 0x7f223c198320 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 >> 172.21.15.122:0/174259547 conn(0x7f223c077b00 msgr2=0x7f223c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 shutdown_connections 2024-04-02T13:42:22.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:22.804+0000 7f22418d9640 1 -- 172.21.15.122:0/174259547 wait complete. 2024-04-02T13:42:22.821 INFO:tasks.cephfs.filesystem:scrub status for tag:1c562b1b-f96c-4117-b9d1-5dbd827e6f06 - {'path': '/', 'tag': '1c562b1b-f96c-4117-b9d1-5dbd827e6f06', 'options': 'recursive,force'} 2024-04-02T13:42:22.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 221218 2.92 MB/sec execute 223 sec latency 2143.794 ms 2024-04-02T13:42:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: pgmap v221: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-02T13:42:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:42:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 223626 2.98 MB/sec execute 224 sec latency 13.932 ms 2024-04-02T13:42:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: pgmap v221: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-02T13:42:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:23.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:23.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: pgmap v221: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-02T13:42:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:42:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:24.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:24 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:24.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:24.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 224764 2.98 MB/sec execute 225 sec latency 12.862 ms 2024-04-02T13:42:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:24 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:24 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:25 smithi122 ceph-mon[32114]: pgmap v222: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-02T13:42:25.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 227064 3.04 MB/sec execute 226 sec latency 25.280 ms 2024-04-02T13:42:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:25 smithi187 ceph-mon[36064]: pgmap v222: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-02T13:42:26.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:25 smithi188 ceph-mon[36117]: pgmap v222: 129 pgs: 129 active+clean; 154 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.4 MiB/s wr, 145 op/s 2024-04-02T13:42:26.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 227778 3.03 MB/sec execute 227 sec latency 261.260 ms 2024-04-02T13:42:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:27 smithi122 ceph-mon[32114]: pgmap v223: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-02T13:42:27.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 227778 3.02 MB/sec execute 228 sec latency 1261.355 ms 2024-04-02T13:42:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:27 smithi187 ceph-mon[36064]: pgmap v223: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-02T13:42:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:27 smithi188 ceph-mon[36117]: pgmap v223: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-02T13:42:28.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 228748 3.03 MB/sec execute 229 sec latency 1706.486 ms 2024-04-02T13:42:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:29 smithi122 ceph-mon[32114]: pgmap v224: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:29.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 230903 3.07 MB/sec execute 230 sec latency 14.977 ms 2024-04-02T13:42:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:29 smithi187 ceph-mon[36064]: pgmap v224: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:30.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:29 smithi188 ceph-mon[36117]: pgmap v224: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:30.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231271 3.06 MB/sec execute 231 sec latency 589.383 ms 2024-04-02T13:42:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:31 smithi122 ceph-mon[32114]: pgmap v225: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:31 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:42:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:31 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:42:31.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231271 3.05 MB/sec execute 232 sec latency 1589.489 ms 2024-04-02T13:42:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:31 smithi187 ceph-mon[36064]: pgmap v225: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:31 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:42:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:31 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:42:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:31 smithi188 ceph-mon[36117]: pgmap v225: 129 pgs: 129 active+clean; 160 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-02T13:42:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:31 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:42:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:31 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:42:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:42:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:42:32.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:42:32.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231271 3.04 MB/sec execute 233 sec latency 2589.582 ms 2024-04-02T13:42:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:42:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:42:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:42:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:42:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:42:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:42:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:33 smithi122 ceph-mon[32114]: pgmap v226: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-02T13:42:33.962 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231395 3.02 MB/sec execute 234 sec latency 3039.747 ms 2024-04-02T13:42:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:33 smithi187 ceph-mon[36064]: pgmap v226: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-02T13:42:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:33 smithi188 ceph-mon[36117]: pgmap v226: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-02T13:42:34.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231395 3.01 MB/sec execute 235 sec latency 1445.862 ms 2024-04-02T13:42:35.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:35 smithi188 ceph-mon[36117]: pgmap v227: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-02T13:42:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:35 smithi122 ceph-mon[32114]: pgmap v227: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-02T13:42:35.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231395 3.00 MB/sec execute 236 sec latency 2445.958 ms 2024-04-02T13:42:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:35 smithi187 ceph-mon[36064]: pgmap v227: 129 pgs: 129 active+clean; 161 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 904 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-04-02T13:42:36.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231395 2.99 MB/sec execute 237 sec latency 3445.998 ms 2024-04-02T13:42:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:37 smithi122 ceph-mon[32114]: pgmap v228: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-02T13:42:37.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 231395 2.97 MB/sec execute 238 sec latency 4446.051 ms 2024-04-02T13:42:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:37 smithi187 ceph-mon[36064]: pgmap v228: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-02T13:42:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:37 smithi188 ceph-mon[36117]: pgmap v228: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-02T13:42:38.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 232430 2.98 MB/sec execute 239 sec latency 4888.889 ms 2024-04-02T13:42:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:39 smithi122 ceph-mon[32114]: pgmap v229: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:39.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 234535 3.03 MB/sec execute 240 sec latency 12.166 ms 2024-04-02T13:42:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:39 smithi187 ceph-mon[36064]: pgmap v229: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:39 smithi188 ceph-mon[36117]: pgmap v229: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:40.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 234814 3.01 MB/sec execute 241 sec latency 747.450 ms 2024-04-02T13:42:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:41 smithi122 ceph-mon[32114]: pgmap v230: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:41.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 234814 3.00 MB/sec execute 242 sec latency 1747.484 ms 2024-04-02T13:42:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:41 smithi187 ceph-mon[36064]: pgmap v230: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:41 smithi188 ceph-mon[36117]: pgmap v230: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 960 KiB/s wr, 86 op/s 2024-04-02T13:42:42.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 234814 2.99 MB/sec execute 243 sec latency 2747.528 ms 2024-04-02T13:42:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:43 smithi122 ceph-mon[32114]: pgmap v231: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-04-02T13:42:43.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 235706 3.00 MB/sec execute 244 sec latency 3192.483 ms 2024-04-02T13:42:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:43 smithi187 ceph-mon[36064]: pgmap v231: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-04-02T13:42:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:43 smithi188 ceph-mon[36117]: pgmap v231: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-04-02T13:42:44.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 237977 3.04 MB/sec execute 245 sec latency 13.330 ms 2024-04-02T13:42:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:45 smithi122 ceph-mon[32114]: pgmap v232: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T13:42:45.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238419 3.03 MB/sec execute 246 sec latency 515.158 ms 2024-04-02T13:42:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:45 smithi187 ceph-mon[36064]: pgmap v232: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T13:42:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:45 smithi188 ceph-mon[36117]: pgmap v232: 129 pgs: 129 active+clean; 166 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-04-02T13:42:46.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238419 3.02 MB/sec execute 247 sec latency 1515.198 ms 2024-04-02T13:42:47.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238419 3.01 MB/sec execute 248 sec latency 2515.244 ms 2024-04-02T13:42:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:47 smithi187 ceph-mon[36064]: pgmap v233: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-04-02T13:42:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:47 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:47 smithi188 ceph-mon[36117]: pgmap v233: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-04-02T13:42:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:47 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:48.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:47 smithi122 ceph-mon[32114]: pgmap v233: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-04-02T13:42:48.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:47 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:48.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238487 3.00 MB/sec execute 249 sec latency 2960.817 ms 2024-04-02T13:42:49.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238487 2.98 MB/sec execute 250 sec latency 1502.844 ms 2024-04-02T13:42:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:49 smithi187 ceph-mon[36064]: pgmap v234: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:49 smithi188 ceph-mon[36117]: pgmap v234: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:50.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:49 smithi122 ceph-mon[32114]: pgmap v234: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:50.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238487 2.97 MB/sec execute 251 sec latency 2502.929 ms 2024-04-02T13:42:51.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238487 2.96 MB/sec execute 252 sec latency 3502.985 ms 2024-04-02T13:42:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:51 smithi187 ceph-mon[36064]: pgmap v235: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:51 smithi188 ceph-mon[36117]: pgmap v235: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:51 smithi122 ceph-mon[32114]: pgmap v235: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-04-02T13:42:52.822 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:52.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 -- 172.21.15.122:0/335186850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c4103f10 msgr2=0x7fe6c41042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/335186850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c4103f10 0x7fe6c41042f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009900 tx=0x7fe6c002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 -- 172.21.15.122:0/335186850 shutdown_connections 2024-04-02T13:42:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/335186850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4108fe0 0x7fe6c4110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/335186850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c41048c0 0x7fe6c4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/335186850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c4103f10 0x7fe6c41042f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.938+0000 7fe6cc28f640 1 -- 172.21.15.122:0/335186850 >> 172.21.15.122:0/335186850 conn(0x7fe6c40ff750 msgr2=0x7fe6c4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- 172.21.15.122:0/335186850 shutdown_connections 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- 172.21.15.122:0/335186850 wait complete. 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 Processor -- start 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- start start 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 0x7fe6c419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c419ef00 0x7fe6c419f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4114090 con 0x7fe6c4108fe0 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4113f10 con 0x7fe6c419ef00 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6cc28f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4114210 con 0x7fe6c4103f10 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55764/0 (socket says 172.21.15.122:55764) 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6c982a640 1 -- 172.21.15.122:0/598240904 learned_addr learned my addr 172.21.15.122:0/598240904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.939+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c419ef00 0x7fe6c419f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 0x7fe6c419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 -- 172.21.15.122:0/598240904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 msgr2=0x7fe6c419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 0x7fe6c419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 -- 172.21.15.122:0/598240904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c419ef00 msgr2=0x7fe6c419f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c419ef00 0x7fe6c419f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 -- 172.21.15.122:0/598240904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c419f9f0 con 0x7fe6c4108fe0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 0x7fe6c419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6c982a640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400d770 tx=0x7fe6b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6bb7fe640 1 -- 172.21.15.122:0/598240904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4017070 con 0x7fe6c4108fe0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c419fce0 con 0x7fe6c4108fe0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6bb7fe640 1 -- 172.21.15.122:0/598240904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b4009c30 con 0x7fe6c4108fe0 2024-04-02T13:42:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6bb7fe640 1 -- 172.21.15.122:0/598240904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400f620 con 0x7fe6c4108fe0 2024-04-02T13:42:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.940+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c41a7520 con 0x7fe6c4108fe0 2024-04-02T13:42:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.942+0000 7fe6bb7fe640 1 -- 172.21.15.122:0/598240904 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe6b401a070 con 0x7fe6c4108fe0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.943+0000 7fe6bb7fe640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 0x7fe6b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.943+0000 7fe6cee76740 1 Processor -- start 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.943+0000 7fe6bb7fe640 1 -- 172.21.15.122:0/598240904 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fe6b40d0710 con 0x7fe6c4108fe0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.943+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 0x7fe6b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.943+0000 7fe6cee76740 1 -- start start 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f357cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 0x55721f360890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55721f2bb830 con 0x55721f360dd0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55721f263820 con 0x55721f3581f0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6cee76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55721f2b77e0 con 0x55721f3578d0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55778/0 (socket says 172.21.15.122:55778) 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 -- 172.21.15.122:0/1857831033 learned_addr learned my addr 172.21.15.122:0/1857831033 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 0x7fe6b00a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000c00 tx=0x7fe6c002c730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f357cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6c982a640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 0x55721f360890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 -- 172.21.15.122:0/1857831033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 msgr2=0x55721f357cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f357cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 -- 172.21.15.122:0/1857831033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 msgr2=0x55721f360890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 0x55721f360890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 -- 172.21.15.122:0/1857831033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721f0997d0 con 0x55721f360dd0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f357cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6c982a640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 0x55721f360890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f70 tx=0x7fe6bc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1940e095ffae08ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6b8bfe640 1 -- 172.21.15.122:0/1857831033 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc016070 con 0x55721f360dd0 2024-04-02T13:42:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6b8bfe640 1 -- 172.21.15.122:0/1857831033 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004d20 con 0x55721f360dd0 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.944+0000 7fe6b8bfe640 1 -- 172.21.15.122:0/1857831033 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc012850 con 0x55721f360dd0 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 -- 172.21.15.122:0/1857831033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 msgr2=0x55721f3631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009f70 tx=0x7fe6bc010800 comp rx=0 tx=0).stop 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 -- 172.21.15.122:0/1857831033 shutdown_connections 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f360dd0 0x55721f3631c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3581f0 0x55721f360890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1857831033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f357cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 -- 172.21.15.122:0/1857831033 >> 172.21.15.122:0/1857831033 conn(0x55721f352000 msgr2=0x55721f3523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 -- 172.21.15.122:0/1857831033 shutdown_connections 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.945+0000 7fe6cee76740 1 -- 172.21.15.122:0/1857831033 wait complete. 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 -- start start 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 Processor -- start 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f3606c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f35df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55721f363bc0 con 0x55721f3581f0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55721f276d30 con 0x55721f3578d0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55721f363d40 con 0x55721f360dd0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6cee76740 10 client.0 _ll_register_callbacks cb 0x55721f276720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55780/0 (socket says 172.21.15.122:55780) 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 -- 172.21.15.122:0/2162688794 learned_addr learned my addr 172.21.15.122:0/2162688794 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f35df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 -- 172.21.15.122:0/2162688794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 msgr2=0x55721f35df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f35df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 -- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 msgr2=0x55721f3606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f3606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f3606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 -- 172.21.15.122:0/2162688794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721efcf3f0 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f35df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f3606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ce070 tx=0x7fe6b409c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e972627703771cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6adffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.946+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40b5060 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fe6b40d02c0 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc004d20 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6b4021040 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 10 client.17877 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55721ef8f0f0 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc004d20 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe6b40a0170 con 0x55721f3581f0 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 10 client.17877 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 10 client.17877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 20 client.17877 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f370d60 0x55721f373220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 4 client.17877 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55721f2c8d70 con 0x55721f370d60 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.947+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f370d60 0x55721f373220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.948+0000 7fe6adffb640 10 client.17877 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.948+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f370d60 0x55721f373220 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad08ee279679221 server_cookie=91084411273c8bd in_seq=0 out_seq=0 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.948+0000 7fe6adffb640 1 -- 172.21.15.122:0/2162688794 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55721f2c8d70 con 0x55721f370d60 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.948+0000 7fe6adffb640 10 client.17877 handle_command_reply: tid=0 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.948+0000 7fe6cee76740 1 client.17877 shutdown 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 msgr2=0x55721f35da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40ce070 tx=0x7fe6b409c9c0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 shutdown_connections 2024-04-02T13:42:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f370d60 0x55721f373220 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f35df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6adffb640 0 client.17877 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f35da50 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2162688794 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f3606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 >> 172.21.15.122:0/2162688794 conn(0x55721f352000 msgr2=0x55721f353d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 shutdown_connections 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 1 -- 172.21.15.122:0/2162688794 wait complete. 2024-04-02T13:42:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.949+0000 7fe6cee76740 20 client.17877 trim_cache size 0 max 16384 2024-04-02T13:42:52.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 238487 2.95 MB/sec execute 253 sec latency 4503.049 ms 2024-04-02T13:42:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.966+0000 7fe6cee76740 1 Processor -- start 2024-04-02T13:42:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.966+0000 7fe6cee76740 1 -- start start 2024-04-02T13:42:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f36cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f360180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b1380 con 0x55721f3581f0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b2f20 con 0x55721f3578d0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6cee76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b1a50 con 0x55721f360dd0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55788/0 (socket says 172.21.15.122:55788) 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 -- 172.21.15.122:0/1425667184 learned_addr learned my addr 172.21.15.122:0/1425667184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f360180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 -- 172.21.15.122:0/1425667184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 msgr2=0x55721f360180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f360180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 -- 172.21.15.122:0/1425667184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 msgr2=0x55721f36cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f36cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f36cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 -- 172.21.15.122:0/1425667184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721f351e30 con 0x55721f3581f0 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f360180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6c982a640 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe6b409bfc0 tx=0x7fe6b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcd2b0dce5856f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6affff640 1 -- 172.21.15.122:0/1425667184 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40a4f30 con 0x55721f3581f0 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6affff640 1 -- 172.21.15.122:0/1425667184 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b409b8b0 con 0x55721f3581f0 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.967+0000 7fe6affff640 1 -- 172.21.15.122:0/1425667184 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b40d02c0 con 0x55721f3581f0 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- 172.21.15.122:0/1425667184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 msgr2=0x55721f3510c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fe6b409bfc0 tx=0x7fe6b40057f0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- 172.21.15.122:0/1425667184 shutdown_connections 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f360dd0 0x55721f360180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f3510c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- 172.21.15.122:0/1425667184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f3578d0 0x55721f36cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- 172.21.15.122:0/1425667184 >> 172.21.15.122:0/1425667184 conn(0x55721f352000 msgr2=0x55721f3509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- 172.21.15.122:0/1425667184 shutdown_connections 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- 172.21.15.122:0/1425667184 wait complete. 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- start start 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 Processor -- start 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f368fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f360dd0 0x55721f35a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b27f0 con 0x55721f3581f0 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41b2120 con 0x55721f360dd0 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41af380 con 0x55721f3578d0 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.968+0000 7fe6cee76740 10 client.0 _ll_register_callbacks cb 0x7fe6b409bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55790/0 (socket says 172.21.15.122:55790) 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 -- 172.21.15.122:0/2857723525 learned_addr learned my addr 172.21.15.122:0/2857723525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6ca82c640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f360dd0 0x55721f35a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f368fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 -- 172.21.15.122:0/2857723525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 msgr2=0x55721f368fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f368fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 -- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f360dd0 msgr2=0x55721f35a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f360dd0 0x55721f35a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 -- 172.21.15.122:0/2857723525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721f3631c0 con 0x55721f3581f0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f368fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6c982a640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4098720 tx=0x7fe6b4098b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d3900465daab5d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4085070 con 0x55721f3581f0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fe6b40d04d0 con 0x55721f3581f0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6b409b8b0 con 0x55721f3581f0 2024-04-02T13:42:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6b40a4f30 con 0x55721f3581f0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 10 client.17883 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55721f3631c0 con 0x55721f3581f0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6b409b8b0 con 0x55721f3581f0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fe6b4085210 con 0x55721f3581f0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 10 client.17883 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 10 client.17883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 20 client.17883 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f3d6ef0 0x55721f3d92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 4 client.17883 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.969+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55721f2c30c0 con 0x55721f3d6ef0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f3d6ef0 0x55721f3d92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6adffb640 10 client.17883 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6ca02b640 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f3d6ef0 0x55721f3d92e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35187f20e24e714 server_cookie=e988028a1aa11b92 in_seq=0 out_seq=0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6adffb640 1 -- 172.21.15.122:0/2857723525 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55721f2c30c0 con 0x55721f3d6ef0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6adffb640 10 client.17883 handle_command_reply: tid=0 2024-04-02T13:42:52.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 client.17883 shutdown 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 msgr2=0x55721f369500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4098720 tx=0x7fe6b4098b60 comp rx=0 tx=0).stop 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 shutdown_connections 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55721f3d6ef0 0x55721f3d92e0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55721f360dd0 0x55721f35a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6adffb640 0 client.17883 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55721f3581f0 0x55721f369500 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 --2- 172.21.15.122:0/2857723525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55721f3578d0 0x55721f368fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 >> 172.21.15.122:0/2857723525 conn(0x55721f352000 msgr2=0x55721f358fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 shutdown_connections 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.970+0000 7fe6cee76740 1 -- 172.21.15.122:0/2857723525 wait complete. 2024-04-02T13:42:52.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cee76740 20 client.17883 trim_cache size 0 max 16384 2024-04-02T13:42:52.981 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:52.981 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:42:52.981 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:42:52.981 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 msgr2=0x7fe6b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 0x7fe6b00a61e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000c00 tx=0x7fe6c002c730 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 msgr2=0x7fe6c419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400d770 tx=0x7fe6b400dc30 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 shutdown_connections 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe6b00a3d20 0x7fe6b00a61e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6c419ef00 0x7fe6c419f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6c4108fe0 0x7fe6c419abd0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 --2- 172.21.15.122:0/598240904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6c4103f10 0x7fe6c419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 >> 172.21.15.122:0/598240904 conn(0x7fe6c40ff750 msgr2=0x7fe6c4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 shutdown_connections 2024-04-02T13:42:52.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:52.971+0000 7fe6cc28f640 1 -- 172.21.15.122:0/598240904 wait complete. 2024-04-02T13:42:52.983 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:42:52.984 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:42:53.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- 172.21.15.122:0/2588485673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 msgr2=0x7fac1410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 --2- 172.21.15.122:0/2588485673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1410e5e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009ca0 tx=0x7fac0c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- 172.21.15.122:0/2588485673 shutdown_connections 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 --2- 172.21.15.122:0/2588485673 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1410e5e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 --2- 172.21.15.122:0/2588485673 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac1410bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 --2- 172.21.15.122:0/2588485673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- 172.21.15.122:0/2588485673 >> 172.21.15.122:0/2588485673 conn(0x7fac14077b00 msgr2=0x7fac14077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- 172.21.15.122:0/2588485673 shutdown_connections 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- 172.21.15.122:0/2588485673 wait complete. 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 Processor -- start 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.083+0000 7fac19ab8640 1 -- start start 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac14198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac14111f40 con 0x7fac1410c140 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac14111dc0 con 0x7fac14103690 2024-04-02T13:42:53.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac141120c0 con 0x7fac14102d70 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55806/0 (socket says 172.21.15.122:55806) 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 -- 172.21.15.122:0/833952084 learned_addr learned my addr 172.21.15.122:0/833952084 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac137fe640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac12ffd640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac14198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 -- 172.21.15.122:0/833952084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 msgr2=0x7fac14198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 -- 172.21.15.122:0/833952084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 msgr2=0x7fac14198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac14198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 -- 172.21.15.122:0/833952084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1419d2f0 con 0x7fac1410c140 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac137fe640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac12ffd640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac14198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac13fff640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fac0c02c700 tx=0x7fac0c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c041020 con 0x7fac1410c140 2024-04-02T13:42:53.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1419d580 con 0x7fac1410c140 2024-04-02T13:42:53.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c034cd0 con 0x7fac1410c140 2024-04-02T13:42:53.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c039040 con 0x7fac1410c140 2024-04-02T13:42:53.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.084+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac141a4e70 con 0x7fac1410c140 2024-04-02T13:42:53.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.085+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac140688f0 con 0x7fac1410c140 2024-04-02T13:42:53.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.086+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fac0c044020 con 0x7fac1410c140 2024-04-02T13:42:53.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.087+0000 7fac10ff9640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 0x7fabfc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.087+0000 7fac137fe640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 0x7fabfc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.087+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fac0c0f2e20 con 0x7fac1410c140 2024-04-02T13:42:53.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.087+0000 7fac137fe640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 0x7fabfc0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac08009b50 tx=0x7fac08005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.091+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fac0c04b020 con 0x7fac1410c140 2024-04-02T13:42:53.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.304+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac141a5040 con 0x7fac1410c140 2024-04-02T13:42:53.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.305+0000 7fac10ff9640 1 -- 172.21.15.122:0/833952084 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11945 (secure 0 0 0) 0x7fac0c045e40 con 0x7fac1410c140 2024-04-02T13:42:53.308 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:42:53.308 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:46.408044+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:42:53.308 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 39 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 msgr2=0x7fabfc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 0x7fabfc0a61e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac08009b50 tx=0x7fac08005f90 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 msgr2=0x7fac1419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fac0c02c700 tx=0x7fac0c032f20 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 shutdown_connections 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabfc0a3d20 0x7fabfc0a61e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1410c140 0x7fac1419cbc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac14103690 0x7fac14198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 --2- 172.21.15.122:0/833952084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac14102d70 0x7fac14198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.307+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 >> 172.21.15.122:0/833952084 conn(0x7fac14077b00 msgr2=0x7fac140fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.308+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 shutdown_connections 2024-04-02T13:42:53.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.308+0000 7fac19ab8640 1 -- 172.21.15.122:0/833952084 wait complete. 2024-04-02T13:42:53.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:42:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 -- 172.21.15.122:0/1786977228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c106e80 msgr2=0x7fdf1c10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 --2- 172.21.15.122:0/1786977228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c106e80 0x7fdf1c10ea00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009ca0 tx=0x7fdf1402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 -- 172.21.15.122:0/1786977228 shutdown_connections 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 --2- 172.21.15.122:0/1786977228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c106e80 0x7fdf1c10ea00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 --2- 172.21.15.122:0/1786977228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100c20 0x7fdf1c106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 --2- 172.21.15.122:0/1786977228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c100270 0x7fdf1c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 -- 172.21.15.122:0/1786977228 >> 172.21.15.122:0/1786977228 conn(0x7fdf1c076fc0 msgr2=0x7fdf1c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 -- 172.21.15.122:0/1786977228 shutdown_connections 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 -- 172.21.15.122:0/1786977228 wait complete. 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.432+0000 7fdf24288640 1 Processor -- start 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 -- start start 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100270 0x7fdf1c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c106e80 0x7fdf1c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c111ee0 con 0x7fdf1c19cce0 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c111d60 con 0x7fdf1c106e80 2024-04-02T13:42:53.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf24288640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c112060 con 0x7fdf1c100270 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55818/0 (socket says 172.21.15.122:55818) 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 -- 172.21.15.122:0/3402678003 learned_addr learned my addr 172.21.15.122:0/3402678003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf21823640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c106e80 0x7fdf1c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22024640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100270 0x7fdf1c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 -- 172.21.15.122:0/3402678003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100270 msgr2=0x7fdf1c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100270 0x7fdf1c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 -- 172.21.15.122:0/3402678003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c106e80 msgr2=0x7fdf1c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c106e80 0x7fdf1c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 -- 172.21.15.122:0/3402678003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1c19d7d0 con 0x7fdf1c19cce0 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf22825640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402c700 tx=0x7fdf14032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf137fe640 1 -- 172.21.15.122:0/3402678003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14041020 con 0x7fdf1c19cce0 2024-04-02T13:42:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.433+0000 7fdf137fe640 1 -- 172.21.15.122:0/3402678003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf14034cd0 con 0x7fdf1c19cce0 2024-04-02T13:42:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.434+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1c19da60 con 0x7fdf1c19cce0 2024-04-02T13:42:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.434+0000 7fdf137fe640 1 -- 172.21.15.122:0/3402678003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14039040 con 0x7fdf1c19cce0 2024-04-02T13:42:53.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.434+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf1c1a5280 con 0x7fdf1c19cce0 2024-04-02T13:42:53.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.436+0000 7fdf137fe640 1 -- 172.21.15.122:0/3402678003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdf14044020 con 0x7fdf1c19cce0 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 Processor -- start 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 -- start start 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94283b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af941e77b0 con 0x55af942840c0 2024-04-02T13:42:53.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af94186570 con 0x55af942837a0 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf26e73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af941e3760 con 0x55af9428cca0 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf22024640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf21823640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94283b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf22024640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55824/0 (socket says 172.21.15.122:55824) 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf137fe640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 0x7fdf080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf22024640 1 -- 172.21.15.122:0/1154827752 learned_addr learned my addr 172.21.15.122:0/1154827752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.437+0000 7fdf21823640 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf137fe640 1 -- 172.21.15.122:0/3402678003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fdf140f2880 con 0x7fdf1c19cce0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 -- 172.21.15.122:0/1154827752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 msgr2=0x55af9428f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 -- 172.21.15.122:0/1154827752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 msgr2=0x55af94283b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94283b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 -- 172.21.15.122:0/1154827752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af93efb3f0 con 0x55af942840c0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf21823640 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94283b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22024640 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009cf0 tx=0x7fdf18011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca9aac19b6f0e52 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf22825640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 0x7fdf080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf10bfe640 1 -- 172.21.15.122:0/1154827752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18004230 con 0x55af942840c0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf10bfe640 1 -- 172.21.15.122:0/1154827752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf180043d0 con 0x55af942840c0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.438+0000 7fdf10bfe640 1 -- 172.21.15.122:0/1154827752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180197e0 con 0x55af942840c0 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- 172.21.15.122:0/1154827752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 msgr2=0x55af9428c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf22825640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 0x7fdf080a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf140ba9a0 tx=0x7fdf1402c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009cf0 tx=0x7fdf18011810 comp rx=0 tx=0).stop 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- 172.21.15.122:0/1154827752 shutdown_connections 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942840c0 0x55af9428c760 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 --2- 172.21.15.122:0/1154827752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94283b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- 172.21.15.122:0/1154827752 >> 172.21.15.122:0/1154827752 conn(0x55af9427ded0 msgr2=0x55af9427e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- 172.21.15.122:0/1154827752 shutdown_connections 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- 172.21.15.122:0/1154827752 wait complete. 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 -- start start 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.439+0000 7fdf26e73740 1 Processor -- start 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af9428a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af9428fa90 con 0x55af942837a0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af941a2e80 con 0x55af942840c0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af9428fc10 con 0x55af9428cca0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 10 client.0 _ll_register_callbacks cb 0x55af941deaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55826/0 (socket says 172.21.15.122:55826) 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 -- 172.21.15.122:0/254509642 learned_addr learned my addr 172.21.15.122:0/254509642 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf22024640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af9428a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 -- 172.21.15.122:0/254509642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 msgr2=0x55af9428a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 -- 172.21.15.122:0/254509642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 msgr2=0x55af9428a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af9428a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 -- 172.21.15.122:0/254509642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af93fc57d0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf22024640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af9428a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf21823640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0699c0 tx=0x7fdf0c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54a38e8c2972a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c012da0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fdf0c019110 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.440+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf180043d0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c004030 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 10 client.17901 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af9429bbc0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf180043d0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdf0c003bf0 con 0x55af942837a0 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 10 client.17901 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 10 client.17901 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 20 client.17901 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9429d080 0x55af9429f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 4 client.17901 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af941f4cf0 con 0x55af9429d080 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.441+0000 7fdf22024640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9429d080 0x55af9429f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.442+0000 7fdf05ffb640 10 client.17901 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.442+0000 7fdf22024640 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9429d080 0x55af9429f4d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a41d0199483676c1 server_cookie=cb7ace1928f9bf6f in_seq=0 out_seq=0 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.442+0000 7fdf05ffb640 1 -- 172.21.15.122:0/254509642 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af941f4cf0 con 0x55af9429d080 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.442+0000 7fdf05ffb640 10 client.17901 handle_command_reply: tid=0 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.442+0000 7fdf26e73740 1 client.17901 shutdown 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 msgr2=0x55af94289b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0699c0 tx=0x7fdf0c004730 comp rx=0 tx=0).stop 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 shutdown_connections 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9429d080 0x55af9429f4d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9428cca0 0x55af9428a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af9428a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 --2- 172.21.15.122:0/254509642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af942837a0 0x55af94289b00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 >> 172.21.15.122:0/254509642 conn(0x55af9427ded0 msgr2=0x55af9427fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf05ffb640 0 client.17901 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:53.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 shutdown_connections 2024-04-02T13:42:53.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 1 -- 172.21.15.122:0/254509642 wait complete. 2024-04-02T13:42:53.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.443+0000 7fdf26e73740 20 client.17901 trim_cache size 0 max 16384 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 Processor -- start 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 -- start start 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942837a0 0x55af9428c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af94298950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1acf00 con 0x55af9428cca0 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.457+0000 7fdf26e73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1aeaa0 con 0x55af942840c0 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf26e73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ad5d0 con 0x55af942837a0 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55842/0 (socket says 172.21.15.122:55842) 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 -- 172.21.15.122:0/3197816494 learned_addr learned my addr 172.21.15.122:0/3197816494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf22024640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af94298950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942837a0 0x55af9428c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 -- 172.21.15.122:0/3197816494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942837a0 msgr2=0x55af9428c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942837a0 0x55af9428c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 -- 172.21.15.122:0/3197816494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 msgr2=0x55af94298950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af94298950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 -- 172.21.15.122:0/3197816494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9429fb50 con 0x55af9428cca0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf21823640 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x55af9427ea40 tx=0x7fdf0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8c1618afcef63a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf07fff640 1 -- 172.21.15.122:0/3197816494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c019110 con 0x55af9428cca0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf07fff640 1 -- 172.21.15.122:0/3197816494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c005500 con 0x55af9428cca0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.458+0000 7fdf07fff640 1 -- 172.21.15.122:0/3197816494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0047a0 con 0x55af9428cca0 2024-04-02T13:42:53.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- 172.21.15.122:0/3197816494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 msgr2=0x55af94288c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x55af9427ea40 tx=0x7fdf0c0046c0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- 172.21.15.122:0/3197816494 shutdown_connections 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94288c60 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942840c0 0x55af94298950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 --2- 172.21.15.122:0/3197816494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942837a0 0x55af9428c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- 172.21.15.122:0/3197816494 >> 172.21.15.122:0/3197816494 conn(0x55af9427ded0 msgr2=0x55af9427f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- 172.21.15.122:0/3197816494 shutdown_connections 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- 172.21.15.122:0/3197816494 wait complete. 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 -- start start 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.459+0000 7fdf26e73740 1 Processor -- start 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94294f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 0x55af942903e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1ae370 con 0x55af9428cca0 2024-04-02T13:42:53.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1adca0 con 0x55af942837a0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1c1aaf00 con 0x55af942840c0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf26e73740 10 client.0 _ll_register_callbacks cb 0x7fdf0c00bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55856/0 (socket says 172.21.15.122:55856) 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 -- 172.21.15.122:0/2008716923 learned_addr learned my addr 172.21.15.122:0/2008716923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf22024640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 0x55af942903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94294f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 -- 172.21.15.122:0/2008716923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 msgr2=0x55af942903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 0x55af942903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 -- 172.21.15.122:0/2008716923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 msgr2=0x55af94294f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94294f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 -- 172.21.15.122:0/2008716923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af94290e90 con 0x55af9428cca0 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf22024640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 0x55af942903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf21823640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0129e0 tx=0x7fdf0c0179a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6e1479204d7f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.460+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0043d0 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fdf0c017ab0 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf0c005500 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdf0c0035f0 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 10 client.17907 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55af943033b0 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf0c005500 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fdf0c043070 con 0x55af9428cca0 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 10 client.17907 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 10 client.17907 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 20 client.17907 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af94303a80 0x55af94305ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 4 client.17907 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af941ef040 con 0x55af94303a80 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.461+0000 7fdf22024640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af94303a80 0x55af94305ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf05ffb640 10 client.17907 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf22024640 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af94303a80 0x55af94305ea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d99c40715c282503 server_cookie=eec6f656d5958136 in_seq=0 out_seq=0 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf05ffb640 1 -- 172.21.15.122:0/2008716923 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af941ef040 con 0x55af94303a80 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf05ffb640 10 client.17907 handle_command_reply: tid=0 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 client.17907 shutdown 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 msgr2=0x55af94290920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0129e0 tx=0x7fdf0c0179a0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 shutdown_connections 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af94303a80 0x55af94305ea0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9428cca0 0x55af94290920 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af942840c0 0x55af942903e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf05ffb640 0 client.17907 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:42:53.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 --2- 172.21.15.122:0/2008716923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af942837a0 0x55af94294f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 >> 172.21.15.122:0/2008716923 conn(0x55af9427ded0 msgr2=0x55af9427cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 shutdown_connections 2024-04-02T13:42:53.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 1 -- 172.21.15.122:0/2008716923 wait complete. 2024-04-02T13:42:53.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.462+0000 7fdf26e73740 20 client.17907 trim_cache size 0 max 16384 2024-04-02T13:42:53.471 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 msgr2=0x7fdf080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 0x7fdf080a6260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdf140ba9a0 tx=0x7fdf1402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 msgr2=0x7fdf1c19d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402c700 tx=0x7fdf14032f20 comp rx=0 tx=0).stop 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 shutdown_connections 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdf080a3da0 0x7fdf080a6260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf1c19cce0 0x7fdf1c19d190 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf1c106e80 0x7fdf1c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 --2- 172.21.15.122:0/3402678003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c100270 0x7fdf1c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 >> 172.21.15.122:0/3402678003 conn(0x7fdf1c076fc0 msgr2=0x7fdf1c105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 shutdown_connections 2024-04-02T13:42:53.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.463+0000 7fdf24288640 1 -- 172.21.15.122:0/3402678003 wait complete. 2024-04-02T13:42:53.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/224900754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 msgr2=0x7fa5b810d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/224900754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b810d840 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009cd0 tx=0x7fa5b002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/224900754 shutdown_connections 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/224900754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b810d840 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/224900754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8100970 0x7fa5b8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/224900754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8106980 0x7fa5b8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/224900754 >> 172.21.15.122:0/224900754 conn(0x7fa5b8077b00 msgr2=0x7fa5b8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/224900754 shutdown_connections 2024-04-02T13:42:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.599+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/224900754 wait complete. 2024-04-02T13:42:53.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 Processor -- start 2024-04-02T13:42:53.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 -- start start 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8106980 0x7fa5b81a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b8110d90 con 0x7fa5b8101330 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b8110c10 con 0x7fa5b8106980 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bfe0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b8110f10 con 0x7fa5b8100970 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.600+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58872/0 (socket says 172.21.15.122:58872) 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 -- 172.21.15.122:0/1906166638 learned_addr learned my addr 172.21.15.122:0/1906166638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55870/0 (socket says 172.21.15.122:55870) 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8106980 0x7fa5b81a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 -- 172.21.15.122:0/1906166638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8106980 msgr2=0x7fa5b81a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8106980 0x7fa5b81a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 -- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 msgr2=0x7fa5b8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 -- 172.21.15.122:0/1906166638 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b81a47d0 con 0x7fa5b8100970 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e9f0 tx=0x7fa5b400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5aeffd640 1 -- 172.21.15.122:0/1906166638 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400cd80 con 0x7fa5b8100970 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b81a49a0 con 0x7fa5b8100970 2024-04-02T13:42:53.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5aeffd640 1 -- 172.21.15.122:0/1906166638 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4005500 con 0x7fa5b8100970 2024-04-02T13:42:53.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b81a4e60 con 0x7fa5b8100970 2024-04-02T13:42:53.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.601+0000 7fa5aeffd640 1 -- 172.21.15.122:0/1906166638 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4010690 con 0x7fa5b8100970 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.603+0000 7fa5aeffd640 1 -- 172.21.15.122:0/1906166638 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa5b40046c0 con 0x7fa5b8100970 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 Processor -- start 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 -- start start 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611d70 0x55987f61eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55987f5757b0 con 0x55987f61f0b0 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55987f514570 con 0x55987f611450 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55987f571760 con 0x55987f611d70 2024-04-02T13:42:53.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5aeffd640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 0x7fa5a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611d70 0x55987f61eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5be3aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55872/0 (socket says 172.21.15.122:55872) 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5be3aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36482/0 (socket says 172.21.15.122:36482) 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.604+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2964488469 learned_addr learned my addr 172.21.15.122:0/2964488469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 0x7fa5a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5aeffd640 1 -- 172.21.15.122:0/1906166638 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fa5b40d0f70 con 0x7fa5b8100970 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2964488469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611d70 msgr2=0x55987f61eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611d70 0x55987f61eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2964488469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 msgr2=0x55987f611830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2964488469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987f2893f0 con 0x55987f61f0b0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80114f0 tx=0x7fa5a80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9053b71a929fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 0x7fa5a40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005cc0 tx=0x7fa5b0005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5a3fff640 1 -- 172.21.15.122:0/2964488469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801e070 con 0x55987f61f0b0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5a3fff640 1 -- 172.21.15.122:0/2964488469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80035f0 con 0x55987f61f0b0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.605+0000 7fa5a3fff640 1 -- 172.21.15.122:0/2964488469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800f7c0 con 0x55987f61f0b0 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2964488469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 msgr2=0x55987f6214a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80114f0 tx=0x7fa5a80119b0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2964488469 shutdown_connections 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f6214a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611d70 0x55987f61eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2964488469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611450 0x55987f611830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2964488469 >> 172.21.15.122:0/2964488469 conn(0x55987f60bed0 msgr2=0x55987f60c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2964488469 shutdown_connections 2024-04-02T13:42:53.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2964488469 wait complete. 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.606+0000 7fa5c29fa740 1 -- start start 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 Processor -- start 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f61c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55987f621ef0 con 0x55987f611d70 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55987f530e80 con 0x55987f61f0b0 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55987f622070 con 0x55987f611450 2024-04-02T13:42:53.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5c29fa740 10 client.0 _ll_register_callbacks cb 0x55987f56caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55882/0 (socket says 172.21.15.122:55882) 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58882/0 (socket says 172.21.15.122:58882) 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2542043534 learned_addr learned my addr 172.21.15.122:0/2542043534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f61c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 msgr2=0x55987f61ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2542043534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 msgr2=0x55987f61c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f61c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.607+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2542043534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987f3537d0 con 0x55987f611d70 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8011eb0 tx=0x7fa5a8004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ef56caae9c66b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801e040 con 0x55987f611d70 2024-04-02T13:42:53.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fa5a8005040 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5a80035f0 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa5a8032050 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 10 client.17919 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55987f62df60 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5a80035f0 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa5a8021030 con 0x55987f611d70 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 10 client.17919 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 10 client.17919 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 20 client.17919 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f62f3c0 0x55987f6317b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 4 client.17919 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.608+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55987f582cf0 con 0x55987f62f3c0 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.609+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f62f3c0 0x55987f6317b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.609+0000 7fa5a17fa640 10 client.17919 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.609+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f62f3c0 0x55987f6317b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef983de9a9b429e5 server_cookie=950adc93a405a18f in_seq=0 out_seq=0 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.609+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2542043534 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55987f582cf0 con 0x55987f62f3c0 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5a17fa640 10 client.17919 handle_command_reply: tid=0 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 client.17919 shutdown 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 msgr2=0x55987f61be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8011eb0 tx=0x7fa5a8004290 comp rx=0 tx=0).stop 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 shutdown_connections 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f61c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f62f3c0 0x55987f6317b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61be10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2542043534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5a17fa640 0 client.17919 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 >> 172.21.15.122:0/2542043534 conn(0x55987f60bed0 msgr2=0x55987f60dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 shutdown_connections 2024-04-02T13:42:53.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2542043534 wait complete. 2024-04-02T13:42:53.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.610+0000 7fa5c29fa740 20 client.17919 trim_cache size 0 max 16384 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.624+0000 7fa5c29fa740 1 Processor -- start 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.624+0000 7fa5c29fa740 1 -- start start 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f62ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 0x55987f61e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81acf30 con 0x55987f61f0b0 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aead0 con 0x55987f611d70 2024-04-02T13:42:53.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ad600 con 0x55987f611450 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55898/0 (socket says 172.21.15.122:55898) 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5bdba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f62ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 -- 172.21.15.122:0/1423728591 learned_addr learned my addr 172.21.15.122:0/1423728591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 0x55987f61e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 -- 172.21.15.122:0/1423728591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 msgr2=0x55987f62ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f62ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 -- 172.21.15.122:0/1423728591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 msgr2=0x55987f61e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 0x55987f61e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 -- 172.21.15.122:0/1423728591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987f611ba0 con 0x55987f61f0b0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 0x55987f61e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009cd0 tx=0x7fa5b002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f14cd93217624 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5a37fe640 1 -- 172.21.15.122:0/1423728591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0039040 con 0x55987f61f0b0 2024-04-02T13:42:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5a37fe640 1 -- 172.21.15.122:0/1423728591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0054070 con 0x55987f61f0b0 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.625+0000 7fa5a37fe640 1 -- 172.21.15.122:0/1423728591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b004f460 con 0x55987f61f0b0 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- 172.21.15.122:0/1423728591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 msgr2=0x55987f61ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009cd0 tx=0x7fa5b002e040 comp rx=0 tx=0).stop 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- 172.21.15.122:0/1423728591 shutdown_connections 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f61f0b0 0x55987f61ea80 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f611d70 0x55987f61e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/1423728591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f62ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- 172.21.15.122:0/1423728591 >> 172.21.15.122:0/1423728591 conn(0x55987f60bed0 msgr2=0x55987f60c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- 172.21.15.122:0/1423728591 shutdown_connections 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- 172.21.15.122:0/1423728591 wait complete. 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.626+0000 7fa5c29fa740 1 -- start start 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 Processor -- start 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f6202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81ae3a0 con 0x55987f611d70 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81adcd0 con 0x55987f61f0b0 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b81aaf30 con 0x55987f611450 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5c29fa740 10 client.0 _ll_register_callbacks cb 0x7fa5a801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55906/0 (socket says 172.21.15.122:55906) 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2265996285 learned_addr learned my addr 172.21.15.122:0/2265996285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5be3aa640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f6202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 msgr2=0x55987f61f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2265996285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 msgr2=0x55987f6202a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f6202a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.627+0000 7fa5bd3a8640 1 -- 172.21.15.122:0/2265996285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55987f6265e0 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5bd3a8640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55987f61d3e0 tx=0x7fa5a800f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1260dd3c61363d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801e040 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fa5a8009b50 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b0054070 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa5a8044050 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 10 client.17925 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55987f6278e0 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0054070 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fa5a8021040 con 0x55987f611d70 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 10 client.17925 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 10 client.17925 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 20 client.17925 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f695ad0 0x55987f697ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 4 client.17925 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.628+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55987f57d040 con 0x55987f695ad0 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f695ad0 0x55987f697ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5a17fa640 10 client.17925 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5bdba9640 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f695ad0 0x55987f697ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ded593a0b44b849a server_cookie=e2f97434c2962fa1 in_seq=0 out_seq=0 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5a17fa640 1 -- 172.21.15.122:0/2265996285 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55987f57d040 con 0x55987f695ad0 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5a17fa640 10 client.17925 handle_command_reply: tid=0 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 client.17925 shutdown 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 msgr2=0x55987f61fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x55987f61d3e0 tx=0x7fa5a800f3c0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 shutdown_connections 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55987f61f0b0 0x55987f6202a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55987f695ad0 0x55987f697ef0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55987f611d70 0x55987f61fd60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 --2- 172.21.15.122:0/2265996285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55987f611450 0x55987f61f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5a17fa640 0 client.17925 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 >> 172.21.15.122:0/2265996285 conn(0x55987f60bed0 msgr2=0x55987f61a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 shutdown_connections 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 1 -- 172.21.15.122:0/2265996285 wait complete. 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.629+0000 7fa5c29fa740 20 client.17925 trim_cache size 0 max 16384 2024-04-02T13:42:53.638 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 msgr2=0x7fa5a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 0x7fa5a40a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0005cc0 tx=0x7fa5b0005c50 comp rx=0 tx=0).stop 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 msgr2=0x7fa5b8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400e9f0 tx=0x7fa5b400edf0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 shutdown_connections 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5a40a3d20 0x7fa5a40a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b8106980 0x7fa5b81a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5b8101330 0x7fa5b8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 --2- 172.21.15.122:0/1906166638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5b8100970 0x7fa5b8199210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 >> 172.21.15.122:0/1906166638 conn(0x7fa5b8077b00 msgr2=0x7fa5b8103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 shutdown_connections 2024-04-02T13:42:53.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.630+0000 7fa5bfe0d640 1 -- 172.21.15.122:0/1906166638 wait complete. 2024-04-02T13:42:53.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.750+0000 7fb08440f640 1 -- 172.21.15.122:0/3760428858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c101330 msgr2=0x7fb07c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.750+0000 7fb08440f640 1 --2- 172.21.15.122:0/3760428858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c101330 0x7fb07c10d840 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb074009930 tx=0x7fb07402bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- 172.21.15.122:0/3760428858 shutdown_connections 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- 172.21.15.122:0/3760428858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c101330 0x7fb07c10d840 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- 172.21.15.122:0/3760428858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 0x7fb07c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- 172.21.15.122:0/3760428858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c106980 0x7fb07c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- 172.21.15.122:0/3760428858 >> 172.21.15.122:0/3760428858 conn(0x7fb07c077b00 msgr2=0x7fb07c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- 172.21.15.122:0/3760428858 shutdown_connections 2024-04-02T13:42:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- 172.21.15.122:0/3760428858 wait complete. 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 Processor -- start 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- start start 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 0x7fb07c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c101330 0x7fb07c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c110d90 con 0x7fb07c106980 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c110c10 con 0x7fb07c100970 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.751+0000 7fb08440f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c110f10 con 0x7fb07c101330 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55932/0 (socket says 172.21.15.122:55932) 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 -- 172.21.15.122:0/928330811 learned_addr learned my addr 172.21.15.122:0/928330811 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0821ab640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 0x7fb07c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0819aa640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c101330 0x7fb07c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 -- 172.21.15.122:0/928330811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c101330 msgr2=0x7fb07c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c101330 0x7fb07c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 -- 172.21.15.122:0/928330811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 msgr2=0x7fb07c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 0x7fb07c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 -- 172.21.15.122:0/928330811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07c1a47d0 con 0x7fb07c106980 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0829ac640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb074005940 tx=0x7fb07402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0737fe640 1 -- 172.21.15.122:0/928330811 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074041070 con 0x7fb07c106980 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07c1a49a0 con 0x7fb07c106980 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0737fe640 1 -- 172.21.15.122:0/928330811 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb074004450 con 0x7fb07c106980 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb0737fe640 1 -- 172.21.15.122:0/928330811 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb074003d30 con 0x7fb07c106980 2024-04-02T13:42:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.752+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07c1a4db0 con 0x7fb07c106980 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb0737fe640 1 -- 172.21.15.122:0/928330811 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb074044070 con 0x7fb07c106980 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 Processor -- start 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 -- start start 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583d1a667b0 con 0x5583d1b027a0 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5583d1a05570 con 0x5583d1b030c0 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.755+0000 7fb086ff1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5583d1a62760 con 0x5583d1b0bca0 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0737fe640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 0x7fb0680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b02b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55946/0 (socket says 172.21.15.122:55946) 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0819aa640 1 -- 172.21.15.122:0/1178798193 learned_addr learned my addr 172.21.15.122:0/1178798193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36534/0 (socket says 172.21.15.122:36534) 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0819aa640 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0829ac640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 0x7fb0680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0737fe640 1 -- 172.21.15.122:0/928330811 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb0740f2500 con 0x7fb07c106980 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 -- 172.21.15.122:0/1178798193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 msgr2=0x5583d1b0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 -- 172.21.15.122:0/1178798193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 msgr2=0x5583d1b02b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b02b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 -- 172.21.15.122:0/1178798193 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d177a3f0 con 0x5583d1b030c0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0821ab640 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb078009f80 tx=0x7fb0780118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb2b84129f64bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb0829ac640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 0x7fb0680a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb07402c820 tx=0x7fb07404bbd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb070bfe640 1 -- 172.21.15.122:0/1178798193 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078004d20 con 0x5583d1b030c0 2024-04-02T13:42:53.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb070bfe640 1 -- 172.21.15.122:0/1178798193 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb078004050 con 0x5583d1b030c0 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.756+0000 7fb070bfe640 1 -- 172.21.15.122:0/1178798193 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078005260 con 0x5583d1b030c0 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- 172.21.15.122:0/1178798193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 msgr2=0x5583d1b0b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb078009f80 tx=0x7fb0780118d0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- 172.21.15.122:0/1178798193 shutdown_connections 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b0b760 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 --2- 172.21.15.122:0/1178798193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b02b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- 172.21.15.122:0/1178798193 >> 172.21.15.122:0/1178798193 conn(0x5583d1afced0 msgr2=0x5583d1afd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- 172.21.15.122:0/1178798193 shutdown_connections 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- 172.21.15.122:0/1178798193 wait complete. 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 -- start start 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.757+0000 7fb086ff1740 1 Processor -- start 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b09050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b09590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583d1b0ea90 con 0x5583d1b027a0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5583d1a21e80 con 0x5583d1b030c0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5583d1b0ec10 con 0x5583d1b0bca0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 10 client.0 _ll_register_callbacks cb 0x5583d1a5daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55962/0 (socket says 172.21.15.122:55962) 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 -- 172.21.15.122:0/3488028974 learned_addr learned my addr 172.21.15.122:0/3488028974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b09050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b09590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 -- 172.21.15.122:0/3488028974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 msgr2=0x5583d1b09590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b09590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 -- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 msgr2=0x5583d1b09050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b09050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 -- 172.21.15.122:0/3488028974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d18447d0 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0095c0 tx=0x7fb06c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347274a73ac82804 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0120d0 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb06c004260 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.758+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb078004050 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb06c012270 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 10 client.17937 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583d1b1ab80 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb078004050 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb06c0049d0 con 0x5583d1b027a0 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 10 client.17937 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 10 client.17937 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:42:53.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 20 client.17937 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b1c040 0x5583d1b1e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 4 client.17937 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583d1a73cf0 con 0x5583d1b1c040 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b1c040 0x5583d1b1e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb065ffb640 10 client.17937 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.759+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b1c040 0x5583d1b1e490 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a79437ef4983efd server_cookie=9d5605a57fc3bf45 in_seq=0 out_seq=0 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb065ffb640 1 -- 172.21.15.122:0/3488028974 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583d1a73cf0 con 0x5583d1b1c040 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb065ffb640 10 client.17937 handle_command_reply: tid=0 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 client.17937 shutdown 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 msgr2=0x5583d1b08b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb06c0095c0 tx=0x7fb06c005f90 comp rx=0 tx=0).stop 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 shutdown_connections 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b0bca0 0x5583d1b09590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b1c040 0x5583d1b1e490 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b09050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3488028974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b027a0 0x5583d1b08b10 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb065ffb640 0 client.17937 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 >> 172.21.15.122:0/3488028974 conn(0x5583d1afced0 msgr2=0x5583d1afec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 shutdown_connections 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.760+0000 7fb086ff1740 1 -- 172.21.15.122:0/3488028974 wait complete. 2024-04-02T13:42:53.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.761+0000 7fb086ff1740 20 client.17937 trim_cache size 0 max 16384 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 Processor -- start 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 -- start start 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b17950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1af010 con 0x5583d1b0bca0 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1b0bb0 con 0x5583d1b030c0 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb086ff1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1af6e0 con 0x5583d1b027a0 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0821ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b17950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55968/0 (socket says 172.21.15.122:55968) 2024-04-02T13:42:53.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 -- 172.21.15.122:0/458853977 learned_addr learned my addr 172.21.15.122:0/458853977 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 -- 172.21.15.122:0/458853977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 msgr2=0x5583d1b0b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b0b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 -- 172.21.15.122:0/458853977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 msgr2=0x5583d1b17950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b17950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0819aa640 1 -- 172.21.15.122:0/458853977 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583d1b046f0 con 0x5583d1b0bca0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.775+0000 7fb0821ab640 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b17950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.776+0000 7fb0819aa640 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb06c010010 tx=0x7fb06c005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63179265b4e2ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.776+0000 7fb067fff640 1 -- 172.21.15.122:0/458853977 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c0120d0 con 0x5583d1b0bca0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.776+0000 7fb067fff640 1 -- 172.21.15.122:0/458853977 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06c004ba0 con 0x5583d1b0bca0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.776+0000 7fb067fff640 1 -- 172.21.15.122:0/458853977 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c005420 con 0x5583d1b0bca0 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- 172.21.15.122:0/458853977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 msgr2=0x5583d1b07b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb06c010010 tx=0x7fb06c005970 comp rx=0 tx=0).stop 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- 172.21.15.122:0/458853977 shutdown_connections 2024-04-02T13:42:53.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b07b40 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b17950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- 172.21.15.122:0/458853977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b0b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- 172.21.15.122:0/458853977 >> 172.21.15.122:0/458853977 conn(0x5583d1afced0 msgr2=0x5583d1afd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- 172.21.15.122:0/458853977 shutdown_connections 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- 172.21.15.122:0/458853977 wait complete. 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- start start 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 Processor -- start 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b13e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b14370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1b0480 con 0x5583d1b0bca0 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1afdb0 con 0x5583d1b030c0 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb07c1ad010 con 0x5583d1b027a0 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb086ff1740 10 client.0 _ll_register_callbacks cb 0x5583d1b0bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb0819aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:55978/0 (socket says 172.21.15.122:55978) 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb0821ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b14370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb0819aa640 1 -- 172.21.15.122:0/3431518742 learned_addr learned my addr 172.21.15.122:0/3431518742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.777+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b13e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 -- 172.21.15.122:0/3431518742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 msgr2=0x5583d1b13e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b13e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 -- 172.21.15.122:0/3431518742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 msgr2=0x5583d1b04d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 -- 172.21.15.122:0/3431518742 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583d1b052c0 con 0x5583d1b030c0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb0821ab640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b14370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb07801b040 tx=0x7fb078005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db80d25d5ee6192c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:42:53.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07802cbd0 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb06c004ba0 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb07802cd70 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb07802c3d0 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb086ff1740 10 client.27086 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583d1b81d40 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.778+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06c004ba0 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb0780055b0 con 0x5583d1b030c0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 10 client.27086 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 10 client.27086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 20 client.27086 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b82280 0x5583d1b846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 4 client.27086 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583d1a6e040 con 0x5583d1b82280 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b82280 0x5583d1b846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb065ffb640 10 client.27086 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb0819aa640 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b82280 0x5583d1b846a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a927f340572792 server_cookie=f0e67e4c15452f35 in_seq=0 out_seq=0 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb065ffb640 1 -- 172.21.15.122:0/3431518742 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583d1a6e040 con 0x5583d1b82280 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb065ffb640 10 client.27086 handle_command_reply: tid=0 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 client.27086 shutdown 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 msgr2=0x5583d1b14370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b14370 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb07801b040 tx=0x7fb078005fd0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 shutdown_connections 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583d1b0bca0 0x5583d1b04d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5583d1b82280 0x5583d1b846a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5583d1b030c0 0x5583d1b14370 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb065ffb640 0 client.27086 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 --2- 172.21.15.122:0/3431518742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5583d1b027a0 0x5583d1b13e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.779+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 >> 172.21.15.122:0/3431518742 conn(0x5583d1afced0 msgr2=0x5583d1b09360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 shutdown_connections 2024-04-02T13:42:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb086ff1740 1 -- 172.21.15.122:0/3431518742 wait complete. 2024-04-02T13:42:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb086ff1740 20 client.27086 trim_cache size 0 max 16384 2024-04-02T13:42:53.789 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 msgr2=0x7fb0680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 0x7fb0680a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb07402c820 tx=0x7fb07404bbd0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 msgr2=0x7fb07c1a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb074005940 tx=0x7fb07402cb30 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 shutdown_connections 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0680a3d20 0x7fb0680a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb07c106980 0x7fb07c1a4290 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07c101330 0x7fb07c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 --2- 172.21.15.122:0/928330811 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb07c100970 0x7fb07c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 >> 172.21.15.122:0/928330811 conn(0x7fb07c077b00 msgr2=0x7fb07c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 shutdown_connections 2024-04-02T13:42:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.780+0000 7fb08440f640 1 -- 172.21.15.122:0/928330811 wait complete. 2024-04-02T13:42:53.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:42:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/1186249883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4102460 msgr2=0x7f3fc410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/1186249883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4102460 0x7f3fc410e970 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009cd0 tx=0x7f3fbc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/1186249883 shutdown_connections 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/1186249883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4102460 0x7f3fc410e970 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/1186249883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/1186249883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/1186249883 >> 172.21.15.122:0/1186249883 conn(0x7f3fc40fd530 msgr2=0x7f3fc40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/1186249883 shutdown_connections 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.900+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/1186249883 wait complete. 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 Processor -- start 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 -- start start 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 0x7f3fc4074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4111e90 con 0x7f3fc4101aa0 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4111d10 con 0x7f3fc4107ab0 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fcbe2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc4112010 con 0x7f3fc4102460 2024-04-02T13:42:53.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 0x7f3fc4074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 0x7f3fc4074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58956/0 (socket says 172.21.15.122:58956) 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3252156840 learned_addr learned my addr 172.21.15.122:0/3252156840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3252156840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 msgr2=0x7f3fc4074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 0x7f3fc4074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.901+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3252156840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 msgr2=0x7f3fc4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3252156840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc41a6870 con 0x7f3fc4101aa0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4073e00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9f0 tx=0x7f3fc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fbaffd640 1 -- 172.21.15.122:0/3252156840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000ccc0 con 0x7f3fc4101aa0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc41a6b60 con 0x7f3fc4101aa0 2024-04-02T13:42:53.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fbaffd640 1 -- 172.21.15.122:0/3252156840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fc0010910 con 0x7f3fc4101aa0 2024-04-02T13:42:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fbaffd640 1 -- 172.21.15.122:0/3252156840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0018960 con 0x7f3fc4101aa0 2024-04-02T13:42:53.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.902+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc41a70b0 con 0x7f3fc4101aa0 2024-04-02T13:42:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.904+0000 7f3fbaffd640 1 -- 172.21.15.122:0/3252156840 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3fc0010ac0 con 0x7f3fc4101aa0 2024-04-02T13:42:53.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.904+0000 7f3fcea13740 1 Processor -- start 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 -- start start 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d623b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56063d5877b0 con 0x56063d6237a0 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56063d526570 con 0x56063d62cca0 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fcea13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56063d583760 con 0x56063d6240c0 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fbaffd640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 0x7f3fb00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fca3c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d623b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36590/0 (socket says 172.21.15.122:36590) 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3787899973 learned_addr learned my addr 172.21.15.122:0/3787899973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 0x7f3fb00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fbaffd640 1 -- 172.21.15.122:0/3252156840 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3fc00d0f60 con 0x7f3fc4101aa0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3787899973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 msgr2=0x56063d62c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3787899973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 msgr2=0x56063d623b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d623b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3787899973 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063d29b3f0 con 0x56063d62cca0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.905+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.906+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009d60 tx=0x7f3fb4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a861f5b54d0f5f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.906+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 0x7f3fb00a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005cc0 tx=0x7f3fbc005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.906+0000 7f3faffff640 1 -- 172.21.15.122:0/3787899973 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4004230 con 0x56063d62cca0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.906+0000 7f3faffff640 1 -- 172.21.15.122:0/3787899973 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb40043d0 con 0x56063d62cca0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.906+0000 7f3faffff640 1 -- 172.21.15.122:0/3787899973 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4004dc0 con 0x56063d62cca0 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- 172.21.15.122:0/3787899973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 msgr2=0x56063d62f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009d60 tx=0x7f3fb4011810 comp rx=0 tx=0).stop 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- 172.21.15.122:0/3787899973 shutdown_connections 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62f090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3787899973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d623b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- 172.21.15.122:0/3787899973 >> 172.21.15.122:0/3787899973 conn(0x56063d61ded0 msgr2=0x56063d61e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- 172.21.15.122:0/3787899973 shutdown_connections 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- 172.21.15.122:0/3787899973 wait complete. 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 -- start start 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.907+0000 7f3fcea13740 1 Processor -- start 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d629980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56063d62fa90 con 0x56063d62cca0 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56063d542e80 con 0x56063d6237a0 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56063d62fc10 con 0x56063d6240c0 2024-04-02T13:42:53.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 10 client.0 _ll_register_callbacks cb 0x56063d57eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fca3c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36606/0 (socket says 172.21.15.122:36606) 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d629980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3617574899 learned_addr learned my addr 172.21.15.122:0/3617574899 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fca3c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56004/0 (socket says 172.21.15.122:56004) 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 msgr2=0x56063d629980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d629980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3617574899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 msgr2=0x56063d629ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3617574899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063d3657d0 con 0x56063d6237a0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d629980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3fc009c860 tx=0x7f3fc0014cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69e0bc4a9b9050d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0025170 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fb40043d0 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3fc0014d20 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fc009cb60 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.908+0000 7f3fcea13740 10 client.27098 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56063d63bab0 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb40043d0 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3fc009c040 con 0x56063d6237a0 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 10 client.27098 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 10 client.27098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 20 client.27098 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d63f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 4 client.27098 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56063d594cf0 con 0x56063d63cec0 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.909+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d63f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.910+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d63f2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5afbf4ab3b09e000 server_cookie=de1eae8ae8330982 in_seq=0 out_seq=0 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.910+0000 7f3fad7fa640 10 client.27098 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.910+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3617574899 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56063d594cf0 con 0x56063d63cec0 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.910+0000 7f3fad7fa640 10 client.27098 handle_command_reply: tid=0 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.910+0000 7f3fcea13740 1 client.27098 shutdown 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 msgr2=0x56063d62c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3fc009c860 tx=0x7f3fc0014cb0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 shutdown_connections 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d63f2b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d62cca0 0x56063d629ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d629980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fad7fa640 0 client.27098 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3617574899 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62c5f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 >> 172.21.15.122:0/3617574899 conn(0x56063d61ded0 msgr2=0x56063d61fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 shutdown_connections 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 1 -- 172.21.15.122:0/3617574899 wait complete. 2024-04-02T13:42:53.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.911+0000 7f3fcea13740 20 client.27098 trim_cache size 0 max 16384 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 Processor -- start 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 -- start start 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d6389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41b1350 con 0x56063d6237a0 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41b2ef0 con 0x56063d62cca0 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fcea13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41b1a20 con 0x56063d6240c0 2024-04-02T13:42:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d6389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d6389b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56016/0 (socket says 172.21.15.122:56016) 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fc9bc6640 1 -- 172.21.15.122:0/3939177648 learned_addr learned my addr 172.21.15.122:0/3939177648 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 -- 172.21.15.122:0/3939177648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 msgr2=0x56063d62c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 -- 172.21.15.122:0/3939177648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 msgr2=0x56063d6389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d6389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.925+0000 7f3fca3c7640 1 -- 172.21.15.122:0/3939177648 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56063d63f620 con 0x56063d62cca0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.926+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62c5f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc02c630 tx=0x7f3fbc02e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378ce04c2580b7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.926+0000 7f3faf7fe640 1 -- 172.21.15.122:0/3939177648 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc039040 con 0x56063d62cca0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.926+0000 7f3faf7fe640 1 -- 172.21.15.122:0/3939177648 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fbc055070 con 0x56063d62cca0 2024-04-02T13:42:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.926+0000 7f3faf7fe640 1 -- 172.21.15.122:0/3939177648 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc004560 con 0x56063d62cca0 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- 172.21.15.122:0/3939177648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 msgr2=0x56063d62c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62c5f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc02c630 tx=0x7f3fbc02e040 comp rx=0 tx=0).stop 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- 172.21.15.122:0/3939177648 shutdown_connections 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d62cca0 0x56063d62c5f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d6240c0 0x56063d62c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3939177648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6237a0 0x56063d6389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- 172.21.15.122:0/3939177648 >> 172.21.15.122:0/3939177648 conn(0x56063d61ded0 msgr2=0x56063d61e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- 172.21.15.122:0/3939177648 shutdown_connections 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- 172.21.15.122:0/3939177648 wait complete. 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- start start 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 Processor -- start 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d62cca0 0x56063d62db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41b27c0 con 0x56063d6240c0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41b20f0 con 0x56063d6237a0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc41af350 con 0x56063d62cca0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.927+0000 7f3fcea13740 10 client.0 _ll_register_callbacks cb 0x7f3fc000ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56028/0 (socket says 172.21.15.122:56028) 2024-04-02T13:42:53.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc9bc6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36620/0 (socket says 172.21.15.122:36620) 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3881618596 learned_addr learned my addr 172.21.15.122:0/3881618596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fca3c7640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d62cca0 0x56063d62db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d62cca0 msgr2=0x56063d62db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d62cca0 0x56063d62db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3881618596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 msgr2=0x56063d62d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 -- 172.21.15.122:0/3881618596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56063d6340c0 con 0x56063d6240c0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fc93c5640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009cb0 tx=0x7f3fb4005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195b88372b2a2235 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb401e040 con 0x56063d6240c0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f3fb4005380 con 0x56063d6240c0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fbc055070 con 0x56063d6240c0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3fb403a050 con 0x56063d6240c0 2024-04-02T13:42:53.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fcea13740 10 client.17961 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56063d6353d0 con 0x56063d6240c0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.928+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fbc055070 con 0x56063d6240c0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f3fb4024040 con 0x56063d6240c0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 10 client.17961 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 10 client.17961 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 20 client.17961 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d6a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 4 client.17961 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56063d58f040 con 0x56063d63cec0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d6a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fad7fa640 10 client.17961 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fc9bc6640 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d6a54a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69c16410881d12fe server_cookie=561076c9cd8d1345 in_seq=0 out_seq=0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fad7fa640 1 -- 172.21.15.122:0/3881618596 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56063d58f040 con 0x56063d63cec0 2024-04-02T13:42:53.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fad7fa640 10 client.17961 handle_command_reply: tid=0 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.929+0000 7f3fcea13740 1 client.17961 shutdown 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 msgr2=0x56063d62d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009cb0 tx=0x7f3fb4005080 comp rx=0 tx=0).stop 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 shutdown_connections 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56063d63cec0 0x56063d6a54a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56063d62cca0 0x56063d62db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56063d6240c0 0x56063d62d5c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fad7fa640 0 client.17961 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 --2- 172.21.15.122:0/3881618596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56063d6237a0 0x56063d62d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 >> 172.21.15.122:0/3881618596 conn(0x56063d61ded0 msgr2=0x56063d628140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 shutdown_connections 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 1 -- 172.21.15.122:0/3881618596 wait complete. 2024-04-02T13:42:53.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcea13740 20 client.17961 trim_cache size 0 max 16384 2024-04-02T13:42:53.939 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:53.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 msgr2=0x7f3fb00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 0x7f3fb00a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005cc0 tx=0x7f3fbc005c50 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 msgr2=0x7f3fc4073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4073e00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9f0 tx=0x7f3fc000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 shutdown_connections 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3fb00a3d20 0x7f3fb00a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3fc4107ab0 0x7f3fc4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fc4102460 0x7f3fc4074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 --2- 172.21.15.122:0/3252156840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fc4101aa0 0x7f3fc4073e00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.930+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 >> 172.21.15.122:0/3252156840 conn(0x7f3fc40fd530 msgr2=0x7f3fc40fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.931+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 shutdown_connections 2024-04-02T13:42:53.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:53.931+0000 7f3fcbe2a640 1 -- 172.21.15.122:0/3252156840 wait complete. 2024-04-02T13:42:53.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:53 smithi122 ceph-mon[32114]: pgmap v236: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-04-02T13:42:53.942 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:53 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/833952084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:53.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:42:53.963 INFO:tasks.workunit.client.0.smithi122.stdout: 1 239557 2.96 MB/sec execute 254 sec latency 4945.364 ms 2024-04-02T13:42:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:53 smithi187 ceph-mon[36064]: pgmap v236: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-04-02T13:42:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:53 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/833952084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:53 smithi188 ceph-mon[36117]: pgmap v236: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 1.6 MiB/s wr, 146 op/s 2024-04-02T13:42:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:53 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/833952084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 -- 172.21.15.122:0/4078218738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59001069a0 msgr2=0x7f5900106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 --2- 172.21.15.122:0/4078218738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59001069a0 0x7f5900106d80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009930 tx=0x7f58fc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 -- 172.21.15.122:0/4078218738 shutdown_connections 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 --2- 172.21.15.122:0/4078218738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5900101350 0x7f590010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 --2- 172.21.15.122:0/4078218738 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900100990 0x7f5900100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 --2- 172.21.15.122:0/4078218738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59001069a0 0x7f5900106d80 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 -- 172.21.15.122:0/4078218738 >> 172.21.15.122:0/4078218738 conn(0x7f5900077ad0 msgr2=0x7f5900077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 -- 172.21.15.122:0/4078218738 shutdown_connections 2024-04-02T13:42:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.055+0000 7f5907677640 1 -- 172.21.15.122:0/4078218738 wait complete. 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 Processor -- start 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 -- start start 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900101350 0x7f590019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59001069a0 0x7f590019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5900110db0 con 0x7f5900100990 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5900110c30 con 0x7f5900101350 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5907677640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5900110f30 con 0x7f59001069a0 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56048/0 (socket says 172.21.15.122:56048) 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5905413640 1 -- 172.21.15.122:0/3094626881 learned_addr learned my addr 172.21.15.122:0/3094626881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.056+0000 7f5904c12640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900101350 0x7f590019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905c14640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59001069a0 0x7f590019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 -- 172.21.15.122:0/3094626881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59001069a0 msgr2=0x7f590019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59001069a0 0x7f590019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 -- 172.21.15.122:0/3094626881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900101350 msgr2=0x7f590019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900101350 0x7f590019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 -- 172.21.15.122:0/3094626881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590019f540 con 0x7f5900100990 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5905413640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009230 tx=0x7f58fc02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f58f67fc640 1 -- 172.21.15.122:0/3094626881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc041070 con 0x7f5900100990 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590019f7d0 con 0x7f5900100990 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f58f67fc640 1 -- 172.21.15.122:0/3094626881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58fc004450 con 0x7f5900100990 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f58f67fc640 1 -- 172.21.15.122:0/3094626881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc034440 con 0x7f5900100990 2024-04-02T13:42:54.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.057+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59001a70c0 con 0x7f5900100990 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f58f67fc640 1 -- 172.21.15.122:0/3094626881 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f58fc044070 con 0x7f5900100990 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 Processor -- start 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 -- start start 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab5d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136ab60090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56136aab87b0 con 0x56136ab550c0 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56136aa57570 con 0x56136ab5dca0 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f590a25b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56136aab4760 con 0x56136ab547a0 2024-04-02T13:42:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f58f67fc640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 0x7f58ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.060+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab5d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:59026/0 (socket says 172.21.15.122:59026) 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab5d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56052/0 (socket says 172.21.15.122:56052) 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 -- 172.21.15.122:0/1151109470 learned_addr learned my addr 172.21.15.122:0/1151109470 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5904c12640 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136ab60090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 0x7f58ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f58f67fc640 1 -- 172.21.15.122:0/3094626881 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f58fc0f24f0 con 0x7f5900100990 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 -- 172.21.15.122:0/1151109470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 msgr2=0x56136ab60090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136ab60090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 -- 172.21.15.122:0/1151109470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 msgr2=0x56136ab5d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab5d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 -- 172.21.15.122:0/1151109470 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136a7cc3f0 con 0x56136ab547a0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 0x7f58ec0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010c40 tx=0x7f58f800c000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f5905c14640 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58f8012d00 tx=0x7f58f8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1780d9edbac4228f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f58eb7fe640 1 -- 172.21.15.122:0/1151109470 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8024070 con 0x56136ab547a0 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f58eb7fe640 1 -- 172.21.15.122:0/1151109470 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f8009d20 con 0x56136ab547a0 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.061+0000 7f58eb7fe640 1 -- 172.21.15.122:0/1151109470 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8005310 con 0x56136ab547a0 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- 172.21.15.122:0/1151109470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 msgr2=0x56136ab54b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f58f8012d00 tx=0x7f58f8010740 comp rx=0 tx=0).stop 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- 172.21.15.122:0/1151109470 shutdown_connections 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136ab60090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab5d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 --2- 172.21.15.122:0/1151109470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab547a0 0x56136ab54b80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- 172.21.15.122:0/1151109470 >> 172.21.15.122:0/1151109470 conn(0x56136ab4eed0 msgr2=0x56136ab4f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- 172.21.15.122:0/1151109470 shutdown_connections 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- 172.21.15.122:0/1151109470 wait complete. 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.062+0000 7f590a25b740 1 -- start start 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 Processor -- start 2024-04-02T13:42:54.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 0x56136ab5b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56136ab60a30 con 0x56136ab547a0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56136aa73e80 con 0x56136ab550c0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56136ab60bb0 con 0x56136ab5dca0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f590a25b740 10 client.0 _ll_register_callbacks cb 0x56136aaafaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5904c12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 0x56136ab5b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56062/0 (socket says 172.21.15.122:56062) 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 -- 172.21.15.122:0/3113622134 learned_addr learned my addr 172.21.15.122:0/3113622134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905c14640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 -- 172.21.15.122:0/3113622134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 msgr2=0x56136ab5b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 -- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 msgr2=0x56136ab5b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 0x56136ab5b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5905413640 1 -- 172.21.15.122:0/3113622134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136a8967d0 con 0x56136ab547a0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.063+0000 7f5904c12640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 0x56136ab5b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f5905413640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0bdf60 tx=0x7f58fc08db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadcc8007e2de51a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc04b040 con 0x56136ab547a0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f58fc0f2170 con 0x56136ab547a0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56136aac5cf0 con 0x56136ab547a0 2024-04-02T13:42:54.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f58fc0be260 con 0x56136ab547a0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 10 client.17973 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56136ab6cb00 con 0x56136ab547a0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56136aac5cf0 con 0x56136ab547a0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f58fc0c2150 con 0x56136ab547a0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 10 client.17973 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 10 client.17973 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 20 client.17973 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136ab6def0 0x56136ab70370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 4 client.17973 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56136aac0040 con 0x56136ab6def0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.064+0000 7f5904c12640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136ab6def0 0x56136ab70370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.065+0000 7f58e8ff9640 10 client.17973 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.065+0000 7f5904c12640 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136ab6def0 0x56136ab70370 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f42abbb4760bbc server_cookie=ab0c3359afd06103 in_seq=0 out_seq=0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f58e8ff9640 1 -- 172.21.15.122:0/3113622134 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56136aac0040 con 0x56136ab6def0 2024-04-02T13:42:54.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f58e8ff9640 10 client.17973 handle_command_reply: tid=0 2024-04-02T13:42:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 client.17973 shutdown 2024-04-02T13:42:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 msgr2=0x56136ab5ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f58fc0bdf60 tx=0x7f58fc08db40 comp rx=0 tx=0).stop 2024-04-02T13:42:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 shutdown_connections 2024-04-02T13:42:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136ab6def0 0x56136ab70370 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab550c0 0x56136ab5b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f58e8ff9640 0 client.17973 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 --2- 172.21.15.122:0/3113622134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab5ab00 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 >> 172.21.15.122:0/3113622134 conn(0x56136ab4eed0 msgr2=0x56136ab50c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 shutdown_connections 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 1 -- 172.21.15.122:0/3113622134 wait complete. 2024-04-02T13:42:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.066+0000 7f590a25b740 20 client.17973 trim_cache size 0 max 16384 2024-04-02T13:42:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 Processor -- start 2024-04-02T13:42:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 -- start start 2024-04-02T13:42:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59001b1190 con 0x56136ab550c0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59001b2d30 con 0x56136ab547a0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f590a25b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59001b1860 con 0x56136ab5dca0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f5904c12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f5905413640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f5904c12640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56074/0 (socket says 172.21.15.122:56074) 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f5904c12640 1 -- 172.21.15.122:0/2315548413 learned_addr learned my addr 172.21.15.122:0/2315548413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.080+0000 7f5905413640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36660/0 (socket says 172.21.15.122:36660) 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5905c14640 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 -- 172.21.15.122:0/2315548413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 msgr2=0x56136ab5b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 -- 172.21.15.122:0/2315548413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 msgr2=0x56136ab5d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 -- 172.21.15.122:0/2315548413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136ab709f0 con 0x56136ab550c0 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5905413640 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f5904c12640 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f590006b1f0 tx=0x7f58f000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4188543fb512ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f58eaffd640 1 -- 172.21.15.122:0/2315548413 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0004220 con 0x56136ab550c0 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f58eaffd640 1 -- 172.21.15.122:0/2315548413 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f0024030 con 0x56136ab550c0 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.081+0000 7f58eaffd640 1 -- 172.21.15.122:0/2315548413 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f00047a0 con 0x56136ab550c0 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- 172.21.15.122:0/2315548413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 msgr2=0x56136ab698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f590006b1f0 tx=0x7f58f000b410 comp rx=0 tx=0).stop 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- 172.21.15.122:0/2315548413 shutdown_connections 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab5dca0 0x56136ab5b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab550c0 0x56136ab698f0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 --2- 172.21.15.122:0/2315548413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab547a0 0x56136ab5d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- 172.21.15.122:0/2315548413 >> 172.21.15.122:0/2315548413 conn(0x56136ab4eed0 msgr2=0x56136ab5aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- 172.21.15.122:0/2315548413 shutdown_connections 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- 172.21.15.122:0/2315548413 wait complete. 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 -- start start 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.082+0000 7f590a25b740 1 Processor -- start 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab550c0 0x56136ab660e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136abd4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59001b2600 con 0x56136ab547a0 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59001b1f30 con 0x56136ab5dca0 2024-04-02T13:42:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59001af190 con 0x56136ab550c0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 10 client.0 _ll_register_callbacks cb 0x7f58fc0be0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136abd4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5904c12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab550c0 0x56136ab660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56080/0 (socket says 172.21.15.122:56080) 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905413640 1 -- 172.21.15.122:0/850933745 learned_addr learned my addr 172.21.15.122:0/850933745 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 -- 172.21.15.122:0/850933745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab550c0 msgr2=0x56136ab660e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab550c0 0x56136ab660e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 -- 172.21.15.122:0/850933745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 msgr2=0x56136ab65ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 -- 172.21.15.122:0/850933745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56136abd53b0 con 0x56136ab5dca0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905413640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f5905c14640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136abd4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010fd0 tx=0x7f58f8009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59d6fecefcc5f2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f58e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8036060 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7f58f8004a00 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.083+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58f0024030 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f58f801b040 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 10 client.27122 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56136abdc7c0 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58f0024030 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7f58f800f040 con 0x56136ab5dca0 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 10 client.27122 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 10 client.27122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 20 client.27122 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136abdcc00 0x56136abdf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 4 client.27122 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:42:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56136aa3d6b0 con 0x56136abdcc00 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.084+0000 7f5905413640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136abdcc00 0x56136abdf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f58e8ff9640 10 client.27122 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f5905413640 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136abdcc00 0x56136abdf020 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c9c65ba0d44573c server_cookie=f8ef752cfe31b77d in_seq=0 out_seq=0 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f58e8ff9640 1 -- 172.21.15.122:0/850933745 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56136aa3d6b0 con 0x56136abdcc00 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f58e8ff9640 10 client.27122 handle_command_reply: tid=0 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 client.27122 shutdown 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 msgr2=0x56136abd4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136abd4cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010fd0 tx=0x7f58f8009600 comp rx=0 tx=0).stop 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 shutdown_connections 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56136abdcc00 0x56136abdf020 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56136ab5dca0 0x56136abd4cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f58e8ff9640 0 client.27122 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56136ab550c0 0x56136ab660e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 --2- 172.21.15.122:0/850933745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56136ab547a0 0x56136ab65ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 >> 172.21.15.122:0/850933745 conn(0x56136ab4eed0 msgr2=0x56136ab4df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 shutdown_connections 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 1 -- 172.21.15.122:0/850933745 wait complete. 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.085+0000 7f590a25b740 20 client.27122 trim_cache size 0 max 16384 2024-04-02T13:42:54.095 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 msgr2=0x7f58ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 0x7f58ec0a61e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f58f8010c40 tx=0x7f58f800c000 comp rx=0 tx=0).stop 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 msgr2=0x7f590019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009230 tx=0x7f58fc02cc70 comp rx=0 tx=0).stop 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 shutdown_connections 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58ec0a3d20 0x7f58ec0a61e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59001069a0 0x7f590019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5900101350 0x7f590019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 --2- 172.21.15.122:0/3094626881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5900100990 0x7f590019a550 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 >> 172.21.15.122:0/3094626881 conn(0x7f5900077ad0 msgr2=0x7f5900102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 shutdown_connections 2024-04-02T13:42:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:54.086+0000 7f5907677640 1 -- 172.21.15.122:0/3094626881 wait complete. 2024-04-02T13:42:54.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 241625 3.00 MB/sec execute 255 sec latency 12.017 ms 2024-04-02T13:42:55.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:42:55.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:42:55.101 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:42:55.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- 172.21.15.122:0/1106412556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 msgr2=0x7fcfd01042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/1106412556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd01042f0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009ca0 tx=0x7fcfc802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- 172.21.15.122:0/1106412556 shutdown_connections 2024-04-02T13:42:55.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/1106412556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 0x7fcfd0110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/1106412556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd01048c0 0x7fcfd0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/1106412556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd01042f0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- 172.21.15.122:0/1106412556 >> 172.21.15.122:0/1106412556 conn(0x7fcfd00ff750 msgr2=0x7fcfd0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- 172.21.15.122:0/1106412556 shutdown_connections 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- 172.21.15.122:0/1106412556 wait complete. 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 Processor -- start 2024-04-02T13:42:55.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.210+0000 7fcfd65ba640 1 -- start start 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 0x7fcfd019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd019ef10 0x7fcfd019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01140f0 con 0x7fcfd0103f10 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0113f70 con 0x7fcfd0108fe0 2024-04-02T13:42:55.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0114270 con 0x7fcfd019ef10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56096/0 (socket says 172.21.15.122:56096) 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 -- 172.21.15.122:0/784883239 learned_addr learned my addr 172.21.15.122:0/784883239 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd019ef10 0x7fcfd019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 0x7fcfd019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 -- 172.21.15.122:0/784883239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd019ef10 msgr2=0x7fcfd019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd019ef10 0x7fcfd019f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 -- 172.21.15.122:0/784883239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 msgr2=0x7fcfd019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 0x7fcfd019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 -- 172.21.15.122:0/784883239 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd019fa00 con 0x7fcfd0103f10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcffff640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005e30 tx=0x7fcfc8032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcd7fa640 1 -- 172.21.15.122:0/784883239 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8039040 con 0x7fcfd0103f10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcd7fa640 1 -- 172.21.15.122:0/784883239 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8044070 con 0x7fcfd0103f10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd019fc90 con 0x7fcfd0103f10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfcd7fa640 1 -- 172.21.15.122:0/784883239 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80042a0 con 0x7fcfd0103f10 2024-04-02T13:42:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.211+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd01a7450 con 0x7fcfd0103f10 2024-04-02T13:42:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.214+0000 7fcfcd7fa640 1 -- 172.21.15.122:0/784883239 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcfc8004440 con 0x7fcfd0103f10 2024-04-02T13:42:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.214+0000 7fcfcd7fa640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 0x7fcfb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.214+0000 7fcfcd7fa640 1 -- 172.21.15.122:0/784883239 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fcfc80f2d90 con 0x7fcfd0103f10 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.214+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 0x7fcfb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.214+0000 7fcfd919d740 1 Processor -- start 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 -- start start 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 0x7fcfb80a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfd010a4f0 tx=0x7fcfc400b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc1dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56309bb76770 con 0x56309bc12be0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56309bb1e960 con 0x56309bc1b7c0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd919d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56309bb728c0 con 0x56309bc122c0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56112/0 (socket says 172.21.15.122:56112) 2024-04-02T13:42:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/2762877110 learned_addr learned my addr 172.21.15.122:0/2762877110 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc1dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcffff640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/2762877110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 msgr2=0x56309bc126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/2762877110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 msgr2=0x56309bc1dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc1dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/2762877110 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309b9527e0 con 0x56309bc12be0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc1dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400bf80 tx=0x7fcfc4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5db403b23588dc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.215+0000 7fcfb67fc640 1 -- 172.21.15.122:0/2762877110 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401e620 con 0x56309bc12be0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfb67fc640 1 -- 172.21.15.122:0/2762877110 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc401eca0 con 0x56309bc12be0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfb67fc640 1 -- 172.21.15.122:0/2762877110 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401cd00 con 0x56309bc12be0 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 -- 172.21.15.122:0/2762877110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 msgr2=0x56309bc1b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400bf80 tx=0x7fcfc4013000 comp rx=0 tx=0).stop 2024-04-02T13:42:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 -- 172.21.15.122:0/2762877110 shutdown_connections 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc1dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc1b280 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 --2- 172.21.15.122:0/2762877110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 -- 172.21.15.122:0/2762877110 >> 172.21.15.122:0/2762877110 conn(0x56309bc0c9f0 msgr2=0x56309bc0cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 -- 172.21.15.122:0/2762877110 shutdown_connections 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.216+0000 7fcfd919d740 1 -- 172.21.15.122:0/2762877110 wait complete. 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 -- start start 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 Processor -- start 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 0x56309bc18780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc1b7c0 0x56309bc19200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56309bc1e5b0 con 0x56309bc12be0 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56309bb31d20 con 0x56309bc122c0 2024-04-02T13:42:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56309bc1e730 con 0x56309bc1b7c0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd919d740 10 client.0 _ll_register_callbacks cb 0x56309bafea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56128/0 (socket says 172.21.15.122:56128) 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 0x56309bc18780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/1235176157 learned_addr learned my addr 172.21.15.122:0/1235176157 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.217+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc1b7c0 0x56309bc19200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 -- 172.21.15.122:0/1235176157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc1b7c0 msgr2=0x56309bc19200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc1b7c0 0x56309bc19200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 -- 172.21.15.122:0/1235176157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 msgr2=0x56309bc18cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 -- 172.21.15.122:0/1235176157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309b9b2f70 con 0x56309bc122c0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 0x56309bc18780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80f0200 tx=0x7fcfc80be650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc66bdf388e95154 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:42:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80bea50 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fcfc8003970 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc401eca0 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfc80c2e00 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfd919d740 10 client.27131 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56309b84a170 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.218+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc401eca0 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcfc803c030 con 0x56309bc122c0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 10 client.27131 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 20 client.27131 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc2e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 4 client.27131 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56309bb83cb0 con 0x56309bc2bbb0 2024-04-02T13:42:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc2e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfaf7fe640 10 client.27131 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.219+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc2e010 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1beff242dc54aead server_cookie=98ea94323da7167f in_seq=0 out_seq=0 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1235176157 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56309bb83cb0 con 0x56309bc2bbb0 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfaf7fe640 10 client.27131 handle_command_reply: tid=0 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 client.27131 shutdown 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 msgr2=0x56309bc18780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 0x56309bc18780 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80f0200 tx=0x7fcfc80be650 comp rx=0 tx=0).stop 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 shutdown_connections 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc2e010 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc1b7c0 0x56309bc19200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc18cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfaf7fe640 0 client.27131 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1235176157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc122c0 0x56309bc18780 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 >> 172.21.15.122:0/1235176157 conn(0x56309bc0c9f0 msgr2=0x56309bc0e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 shutdown_connections 2024-04-02T13:42:55.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 1 -- 172.21.15.122:0/1235176157 wait complete. 2024-04-02T13:42:55.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.220+0000 7fcfd919d740 20 client.27131 trim_cache size 0 max 16384 2024-04-02T13:42:55.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 Processor -- start 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 -- start start 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc338e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b1220 con 0x56309bc12be0 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b2dc0 con 0x56309bc1b7c0 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd919d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b18f0 con 0x56309bc122c0 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56134/0 (socket says 172.21.15.122:56134) 2024-04-02T13:42:55.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/3267513904 learned_addr learned my addr 172.21.15.122:0/3267513904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc338e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcffff640 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/3267513904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 msgr2=0x56309bc1ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc1ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/3267513904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 msgr2=0x56309bc338e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc338e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 -- 172.21.15.122:0/3267513904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56309bc0c820 con 0x56309bc12be0 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcfc401c8b0 tx=0x7fcfc4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d75bbb70318b0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.242+0000 7fcfb5ffb640 1 -- 172.21.15.122:0/3267513904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401c910 con 0x56309bc12be0 2024-04-02T13:42:55.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfb5ffb640 1 -- 172.21.15.122:0/3267513904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc403c070 con 0x56309bc12be0 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfb5ffb640 1 -- 172.21.15.122:0/3267513904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40368d0 con 0x56309bc12be0 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 -- 172.21.15.122:0/3267513904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 msgr2=0x56309bc27500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcfc401c8b0 tx=0x7fcfc4016000 comp rx=0 tx=0).stop 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 -- 172.21.15.122:0/3267513904 shutdown_connections 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc1b7c0 0x56309bc338e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc12be0 0x56309bc27500 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 --2- 172.21.15.122:0/3267513904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc1ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 -- 172.21.15.122:0/3267513904 >> 172.21.15.122:0/3267513904 conn(0x56309bc0c9f0 msgr2=0x56309bc0bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 -- 172.21.15.122:0/3267513904 shutdown_connections 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.243+0000 7fcfd919d740 1 -- 172.21.15.122:0/3267513904 wait complete. 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 -- start start 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 Processor -- start 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc12be0 0x56309bc209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b2690 con 0x56309bc1b7c0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01b1fc0 con 0x56309bc12be0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01af220 con 0x56309bc122c0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd919d740 10 client.0 _ll_register_callbacks cb 0x7fcfc80be8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56144/0 (socket says 172.21.15.122:56144) 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 -- 172.21.15.122:0/1796291495 learned_addr learned my addr 172.21.15.122:0/1796291495 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfcf7fe640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc12be0 0x56309bc209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 -- 172.21.15.122:0/1796291495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 msgr2=0x56309bc23c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc23c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 -- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc12be0 msgr2=0x56309bc209e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.244+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc12be0 0x56309bc209e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd4b57640 1 -- 172.21.15.122:0/1796291495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56309bc21460 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd4b57640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0015a40 tx=0x7fcfc0015b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff350d696b36b4e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0023060 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fcfc000f040 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc403c070 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcfc002a050 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 10 client.17997 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56309bc92f10 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc403c070 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fcfc00133a0 con 0x56309bc1b7c0 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 10 client.17997 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 10 client.17997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:55.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 20 client.17997 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 4 client.17997 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.245+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56309bb7e000 con 0x56309bc2bbb0 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.246+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.246+0000 7fcfaf7fe640 10 client.17997 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.246+0000 7fcfcffff640 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc95760 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ada081df555dfd server_cookie=9edfe12bfcf3093a in_seq=0 out_seq=0 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfaf7fe640 1 -- 172.21.15.122:0/1796291495 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56309bb7e000 con 0x56309bc2bbb0 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfaf7fe640 10 client.17997 handle_command_reply: tid=0 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 client.17997 shutdown 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 msgr2=0x56309bc20f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0015a40 tx=0x7fcfc0015b30 comp rx=0 tx=0).stop 2024-04-02T13:42:55.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 shutdown_connections 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56309bc2bbb0 0x56309bc95760 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56309bc1b7c0 0x56309bc20f20 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56309bc12be0 0x56309bc209e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfaf7fe640 0 client.17997 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 --2- 172.21.15.122:0/1796291495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56309bc122c0 0x56309bc23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 >> 172.21.15.122:0/1796291495 conn(0x56309bc0c9f0 msgr2=0x56309bc18740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 shutdown_connections 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 1 -- 172.21.15.122:0/1796291495 wait complete. 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.248+0000 7fcfd919d740 20 client.17997 trim_cache size 0 max 16384 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:55.256 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:42:55.257 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "12263594-a8a1-48ae-86b1-b00629dfc922", 2024-04-02T13:42:55.257 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:42:55.257 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:55.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 msgr2=0x7fcfb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 0x7fcfb80a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcfd010a4f0 tx=0x7fcfc400b410 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 msgr2=0x7fcfd019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005e30 tx=0x7fcfc8032f20 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 shutdown_connections 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcfb80a3d20 0x7fcfb80a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfd019ef10 0x7fcfd019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfd0108fe0 0x7fcfd019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 --2- 172.21.15.122:0/784883239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd0103f10 0x7fcfd019a650 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 >> 172.21.15.122:0/784883239 conn(0x7fcfd00ff750 msgr2=0x7fcfd0100b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 shutdown_connections 2024-04-02T13:42:55.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.249+0000 7fcfd65ba640 1 -- 172.21.15.122:0/784883239 wait complete. 2024-04-02T13:42:55.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:42:55.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:42:55.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.385+0000 7fb57842a640 1 -- 172.21.15.122:0/3204354677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570101aa0 msgr2=0x7fb570101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.385+0000 7fb57842a640 1 --2- 172.21.15.122:0/3204354677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570101aa0 0x7fb570101f20 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fb560009930 tx=0x7fb56002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:55.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 -- 172.21.15.122:0/3204354677 shutdown_connections 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 --2- 172.21.15.122:0/3204354677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570102460 0x7fb57010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 --2- 172.21.15.122:0/3204354677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570101aa0 0x7fb570101f20 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 --2- 172.21.15.122:0/3204354677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb570107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 -- 172.21.15.122:0/3204354677 >> 172.21.15.122:0/3204354677 conn(0x7fb5700fd530 msgr2=0x7fb5700ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 -- 172.21.15.122:0/3204354677 shutdown_connections 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 -- 172.21.15.122:0/3204354677 wait complete. 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 Processor -- start 2024-04-02T13:42:55.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.386+0000 7fb57842a640 1 -- start start 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570102460 0x7fb570196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010fcb0 con 0x7fb570102460 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010fb30 con 0x7fb570101aa0 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb57842a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb57010fe30 con 0x7fb570107ab0 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5769c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5769c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:59120/0 (socket says 172.21.15.122:59120) 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36736/0 (socket says 172.21.15.122:36736) 2024-04-02T13:42:55.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5769c7640 1 -- 172.21.15.122:0/3978233599 learned_addr learned my addr 172.21.15.122:0/3978233599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5759c5640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570102460 0x7fb570196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 -- 172.21.15.122:0/3978233599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 msgr2=0x7fb57019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 -- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570102460 msgr2=0x7fb570196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570102460 0x7fb570196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 -- 172.21.15.122:0/3978233599 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57019b220 con 0x7fb570101aa0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5769c7640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.387+0000 7fb5761c6640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb56c012c10 tx=0x7fb56c010490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.388+0000 7fb5677fe640 1 -- 172.21.15.122:0/3978233599 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c010da0 con 0x7fb570101aa0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.388+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57019b4b0 con 0x7fb570101aa0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.388+0000 7fb5677fe640 1 -- 172.21.15.122:0/3978233599 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56c01d070 con 0x7fb570101aa0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.388+0000 7fb5677fe640 1 -- 172.21.15.122:0/3978233599 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0186e0 con 0x7fb570101aa0 2024-04-02T13:42:55.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.388+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a2da0 con 0x7fb570101aa0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 Processor -- start 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 -- start start 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 0x55cdac29f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdac1f7830 con 0x55cdac2938d0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cdac19f820 con 0x55cdac2941f0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb57b012740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdac1f37e0 con 0x55cdac29cdd0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5677fe640 1 -- 172.21.15.122:0/3978233599 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb56c016050 con 0x7fb570101aa0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5769c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5761c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5761c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36750/0 (socket says 172.21.15.122:36750) 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5769c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56166/0 (socket says 172.21.15.122:56166) 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5769c7640 1 -- 172.21.15.122:0/2437710916 learned_addr learned my addr 172.21.15.122:0/2437710916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.391+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 0x55cdac29f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 -- 172.21.15.122:0/2437710916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 msgr2=0x55cdac29c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 -- 172.21.15.122:0/2437710916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 msgr2=0x55cdac293cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5677fe640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 0x7fb55c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 -- 172.21.15.122:0/2437710916 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdabfd57d0 con 0x55cdac29cdd0 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5677fe640 1 -- 172.21.15.122:0/3978233599 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb56c0d49c0 con 0x7fb570101aa0 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 0x55cdac29f1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb560009230 tx=0x7fb560009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f31d6fa856d0465 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb5761c6640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 0x7fb55c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.392+0000 7fb564bfe640 1 -- 172.21.15.122:0/2437710916 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560042070 con 0x55cdac29cdd0 2024-04-02T13:42:55.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb564bfe640 1 -- 172.21.15.122:0/2437710916 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56002c690 con 0x55cdac29cdd0 2024-04-02T13:42:55.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb564bfe640 1 -- 172.21.15.122:0/2437710916 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560004ba0 con 0x55cdac29cdd0 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 -- 172.21.15.122:0/2437710916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 msgr2=0x55cdac29f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 0x55cdac29f1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb560009230 tx=0x7fb560009610 comp rx=0 tx=0).stop 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb5761c6640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 0x7fb55c0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0d1dd0 tx=0x7fb56c0d8720 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 -- 172.21.15.122:0/2437710916 shutdown_connections 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac29cdd0 0x55cdac29f1c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 --2- 172.21.15.122:0/2437710916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac293cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.393+0000 7fb57b012740 1 -- 172.21.15.122:0/2437710916 >> 172.21.15.122:0/2437710916 conn(0x55cdac28e000 msgr2=0x55cdac28e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.394+0000 7fb57b012740 1 -- 172.21.15.122:0/2437710916 shutdown_connections 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.394+0000 7fb57b012740 1 -- 172.21.15.122:0/2437710916 wait complete. 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.394+0000 7fb57b012740 1 -- start start 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.394+0000 7fb57b012740 1 Processor -- start 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac2938d0 0x55cdac29c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac29cdd0 0x55cdac299f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cdac29fb60 con 0x55cdac2941f0 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cdac1b2d30 con 0x55cdac29cdd0 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdac29fce0 con 0x55cdac2938d0 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb57b012740 10 client.0 _ll_register_callbacks cb 0x55cdac1b2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56170/0 (socket says 172.21.15.122:56170) 2024-04-02T13:42:55.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 -- 172.21.15.122:0/3417331404 learned_addr learned my addr 172.21.15.122:0/3417331404 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5769c7640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac2938d0 0x55cdac29c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5769c7640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac29cdd0 0x55cdac299f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 -- 172.21.15.122:0/3417331404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac2938d0 msgr2=0x55cdac29c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac2938d0 0x55cdac29c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 -- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac29cdd0 msgr2=0x55cdac299f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac29cdd0 0x55cdac299f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 -- 172.21.15.122:0/3417331404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdabf0b3f0 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb5759c5640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb56002c170 tx=0x7fb560009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d71d49062c2dac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.395+0000 7fb559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560042040 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb560004c70 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cdac204d70 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb560056050 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 10 client.18009 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cdabecb0f0 con 0x55cdac2941f0 2024-04-02T13:42:55.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55cdac204d70 con 0x55cdac2941f0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb560003950 con 0x55cdac2941f0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 10 client.18009 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 10 client.18009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 20 client.18009 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac2ac9a0 0x55cdac2aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 4 client.18009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.396+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdac1ff0c0 con 0x55cdac2ac9a0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.397+0000 7fb5769c7640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac2ac9a0 0x55cdac2aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.397+0000 7fb559ffb640 10 client.18009 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.397+0000 7fb5769c7640 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac2ac9a0 0x55cdac2aed90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9a19c935bd9ca7 server_cookie=58122ec34330eff3 in_seq=0 out_seq=0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb559ffb640 1 -- 172.21.15.122:0/3417331404 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdac1ff0c0 con 0x55cdac2ac9a0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb559ffb640 10 client.18009 handle_command_reply: tid=0 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 client.18009 shutdown 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 msgr2=0x55cdac299a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fb56002c170 tx=0x7fb560009260 comp rx=0 tx=0).stop 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 shutdown_connections 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac2ac9a0 0x55cdac2aed90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac29cdd0 0x55cdac299f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2941f0 0x55cdac299a20 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 --2- 172.21.15.122:0/3417331404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac2938d0 0x55cdac29c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 >> 172.21.15.122:0/3417331404 conn(0x55cdac28e000 msgr2=0x55cdac28fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb559ffb640 0 client.18009 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 shutdown_connections 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 1 -- 172.21.15.122:0/3417331404 wait complete. 2024-04-02T13:42:55.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.398+0000 7fb57b012740 20 client.18009 trim_cache size 0 max 16384 2024-04-02T13:42:55.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.417+0000 7fb57b012740 1 Processor -- start 2024-04-02T13:42:55.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.417+0000 7fb57b012740 1 -- start start 2024-04-02T13:42:55.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701acf40 con 0x55cdac2938d0 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701aeae0 con 0x55cdac2941f0 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb57b012740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701ad610 con 0x55cdac28e000 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56174/0 (socket says 172.21.15.122:56174) 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 -- 172.21.15.122:0/2646871159 learned_addr learned my addr 172.21.15.122:0/2646871159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 -- 172.21.15.122:0/2646871159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 msgr2=0x55cdac2a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 -- 172.21.15.122:0/2646871159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 msgr2=0x55cdac29c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 -- 172.21.15.122:0/2646871159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdac28de30 con 0x55cdac2938d0 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb560009230 tx=0x7fb56002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce63eacaca1044e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb55bfff640 1 -- 172.21.15.122:0/2646871159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560042040 con 0x55cdac2938d0 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb55bfff640 1 -- 172.21.15.122:0/2646871159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56002cc70 con 0x55cdac2938d0 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.418+0000 7fb55bfff640 1 -- 172.21.15.122:0/2646871159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560034760 con 0x55cdac2938d0 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 -- 172.21.15.122:0/2646871159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 msgr2=0x55cdac2982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fb560009230 tx=0x7fb56002bc30 comp rx=0 tx=0).stop 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 -- 172.21.15.122:0/2646871159 shutdown_connections 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac29c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2982f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 --2- 172.21.15.122:0/2646871159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 -- 172.21.15.122:0/2646871159 >> 172.21.15.122:0/2646871159 conn(0x55cdac29a0c0 msgr2=0x55cdac28d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 -- 172.21.15.122:0/2646871159 shutdown_connections 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.419+0000 7fb57b012740 1 -- 172.21.15.122:0/2646871159 wait complete. 2024-04-02T13:42:55.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 -- start start 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 Processor -- start 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac2a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701ae3b0 con 0x55cdac2938d0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701adce0 con 0x55cdac2941f0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701aaf40 con 0x55cdac28e000 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb57b012740 10 client.0 _ll_register_callbacks cb 0x7fb560009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5759c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56190/0 (socket says 172.21.15.122:56190) 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5759c5640 1 -- 172.21.15.122:0/2896703758 learned_addr learned my addr 172.21.15.122:0/2896703758 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac2a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 -- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 msgr2=0x55cdac2a0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac2a0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 -- 172.21.15.122:0/2896703758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 msgr2=0x55cdac2a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5769c7640 1 -- 172.21.15.122:0/2896703758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdac29f1c0 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.420+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb5769c7640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a53f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb568004a80 tx=0x7fb560005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc4834ac5fd64daf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5680228b0 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb56002cc70 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9525+0+0 (secure 0 0 0) 0x7fb568024e00 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb568021ca0 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 10 client.26968 fetch_fsmap learned FSMap version 39 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55cdac312c40 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb56002cc70 con 0x55cdac28e000 2024-04-02T13:42:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 39) ==== 8370+0+0 (secure 0 0 0) 0x7fb56801f420 con 0x55cdac28e000 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 10 client.26968 fetch_fsmap finished waiting for FSMap version 39 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 20 client.26968 populate_metadata read hostname 'smithi122' 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac29cdd0 0x55cdac315300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 4 client.26968 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdac17c560 con 0x55cdac29cdd0 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.421+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac29cdd0 0x55cdac315300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb5759c5640 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac29cdd0 0x55cdac315300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda5a06b52fad56b server_cookie=2090db698a270db3 in_seq=0 out_seq=0 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb559ffb640 10 client.26968 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb559ffb640 1 -- 172.21.15.122:0/2896703758 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cdac17c560 con 0x55cdac29cdd0 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb559ffb640 10 client.26968 handle_command_reply: tid=0 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 client.26968 shutdown 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 msgr2=0x55cdac2a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a53f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb568004a80 tx=0x7fb560005e50 comp rx=0 tx=0).stop 2024-04-02T13:42:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 shutdown_connections 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cdac29cdd0 0x55cdac315300 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cdac2941f0 0x55cdac2a0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cdac2938d0 0x55cdac2a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb559ffb640 0 client.26968 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 --2- 172.21.15.122:0/2896703758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdac28e000 0x55cdac2a53f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 >> 172.21.15.122:0/2896703758 conn(0x55cdac29a0c0 msgr2=0x55cdac28c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 shutdown_connections 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 1 -- 172.21.15.122:0/2896703758 wait complete. 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.422+0000 7fb57b012740 20 client.26968 trim_cache size 0 max 16384 2024-04-02T13:42:55.431 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "12263594-a8a1-48ae-86b1-b00629dfc922": { 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "12263594-a8a1-48ae-86b1-b00629dfc922", 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:42:55.432 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 msgr2=0x7fb55c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 0x7fb55c0a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0d1dd0 tx=0x7fb56c0d8720 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 msgr2=0x7fb570196230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb56c012c10 tx=0x7fb56c010490 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 shutdown_connections 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb55c0a3d20 0x7fb55c0a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb570107ab0 0x7fb57019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb570102460 0x7fb570196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 --2- 172.21.15.122:0/3978233599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb570101aa0 0x7fb570196230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 >> 172.21.15.122:0/3978233599 conn(0x7fb5700fd530 msgr2=0x7fb5700fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:42:55.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 shutdown_connections 2024-04-02T13:42:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:42:55.423+0000 7fb57842a640 1 -- 172.21.15.122:0/3978233599 wait complete. 2024-04-02T13:42:55.437 INFO:tasks.cephfs.filesystem:scrub status for tag:12263594-a8a1-48ae-86b1-b00629dfc922 - {'path': '/', 'tag': '12263594-a8a1-48ae-86b1-b00629dfc922', 'options': 'recursive,force'} 2024-04-02T13:42:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: pgmap v237: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-02T13:42:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 242879 3.01 MB/sec execute 256 sec latency 25.024 ms 2024-04-02T13:42:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: pgmap v237: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-02T13:42:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:42:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: pgmap v237: 129 pgs: 129 active+clean; 168 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 245032 3.05 MB/sec execute 257 sec latency 22.835 ms 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:42:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:42:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:42:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:42:57.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 245509 3.04 MB/sec execute 258 sec latency 454.378 ms 2024-04-02T13:42:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:57 smithi187 ceph-mon[36064]: pgmap v238: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-04-02T13:42:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:57 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:57 smithi188 ceph-mon[36117]: pgmap v238: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-04-02T13:42:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:57 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:57 smithi122 ceph-mon[32114]: pgmap v238: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-04-02T13:42:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:57 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:42:58.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 246309 3.05 MB/sec execute 259 sec latency 895.317 ms 2024-04-02T13:42:59.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 248137 3.07 MB/sec execute 260 sec latency 30.034 ms 2024-04-02T13:42:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:42:59 smithi187 ceph-mon[36064]: pgmap v239: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:42:59 smithi188 ceph-mon[36117]: pgmap v239: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:00.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:42:59 smithi122 ceph-mon[32114]: pgmap v239: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:00.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 249253 3.08 MB/sec execute 261 sec latency 41.074 ms 2024-04-02T13:43:01.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 249303 3.07 MB/sec execute 262 sec latency 975.638 ms 2024-04-02T13:43:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:01 smithi187 ceph-mon[36064]: pgmap v240: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:01 smithi188 ceph-mon[36117]: pgmap v240: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:01 smithi122 ceph-mon[32114]: pgmap v240: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-02T13:43:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:02.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 249303 3.06 MB/sec execute 263 sec latency 1975.688 ms 2024-04-02T13:43:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:03.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 249360 3.05 MB/sec execute 264 sec latency 2421.397 ms 2024-04-02T13:43:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:03 smithi187 ceph-mon[36064]: pgmap v241: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-02T13:43:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:03 smithi188 ceph-mon[36117]: pgmap v241: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-02T13:43:04.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:03 smithi122 ceph-mon[32114]: pgmap v241: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.7 MiB/s wr, 175 op/s 2024-04-02T13:43:04.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 250246 3.05 MB/sec execute 265 sec latency 61.533 ms 2024-04-02T13:43:05.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 252130 3.09 MB/sec execute 266 sec latency 43.903 ms 2024-04-02T13:43:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:05 smithi187 ceph-mon[36064]: pgmap v242: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-02T13:43:06.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:05 smithi188 ceph-mon[36117]: pgmap v242: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-02T13:43:06.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:05 smithi122 ceph-mon[32114]: pgmap v242: 129 pgs: 129 active+clean; 175 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-02T13:43:06.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 253332 3.10 MB/sec execute 267 sec latency 17.736 ms 2024-04-02T13:43:07.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 254888 3.11 MB/sec execute 268 sec latency 16.179 ms 2024-04-02T13:43:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:07 smithi187 ceph-mon[36064]: pgmap v243: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-02T13:43:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:07 smithi187 ceph-mon[36064]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:43:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:07 smithi188 ceph-mon[36117]: pgmap v243: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-02T13:43:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:07 smithi188 ceph-mon[36117]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:43:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:07 smithi122 ceph-mon[32114]: pgmap v243: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-02T13:43:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:07 smithi122 ceph-mon[32114]: osdmap e98: 12 total, 12 up, 12 in 2024-04-02T13:43:08.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 256087 3.13 MB/sec execute 269 sec latency 79.488 ms 2024-04-02T13:43:09.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 256087 3.12 MB/sec execute 270 sec latency 1079.549 ms 2024-04-02T13:43:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:09 smithi187 ceph-mon[36064]: pgmap v245: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:09 smithi188 ceph-mon[36117]: pgmap v245: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:09 smithi122 ceph-mon[32114]: pgmap v245: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:10.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 256087 3.10 MB/sec execute 271 sec latency 2079.586 ms 2024-04-02T13:43:11.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 256211 3.09 MB/sec execute 272 sec latency 2304.983 ms 2024-04-02T13:43:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:11 smithi187 ceph-mon[36064]: pgmap v246: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:11 smithi188 ceph-mon[36117]: pgmap v246: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:11 smithi122 ceph-mon[32114]: pgmap v246: 129 pgs: 129 active+clean; 176 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 201 op/s 2024-04-02T13:43:12.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 256211 3.08 MB/sec execute 273 sec latency 1668.555 ms 2024-04-02T13:43:13.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 257197 3.09 MB/sec execute 274 sec latency 2111.035 ms 2024-04-02T13:43:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:13 smithi187 ceph-mon[36064]: pgmap v247: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:13 smithi188 ceph-mon[36117]: pgmap v247: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:14.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:13 smithi122 ceph-mon[32114]: pgmap v247: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:14.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 259325 3.13 MB/sec execute 275 sec latency 13.986 ms 2024-04-02T13:43:15.872 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:15 smithi122 ceph-mon[32114]: pgmap v248: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:15.964 INFO:tasks.workunit.client.0.smithi122.stdout: 1 260503 3.14 MB/sec execute 276 sec latency 16.151 ms 2024-04-02T13:43:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:15 smithi187 ceph-mon[36064]: pgmap v248: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:15 smithi188 ceph-mon[36117]: pgmap v248: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-04-02T13:43:16.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 262340 3.16 MB/sec execute 277 sec latency 30.143 ms 2024-04-02T13:43:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:17 smithi188 ceph-mon[36117]: pgmap v249: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:17.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:17 smithi188 ceph-mon[36117]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:43:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:17 smithi122 ceph-mon[32114]: pgmap v249: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:17 smithi122 ceph-mon[32114]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:43:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:17 smithi187 ceph-mon[36064]: pgmap v249: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:17 smithi187 ceph-mon[36064]: osdmap e99: 12 total, 12 up, 12 in 2024-04-02T13:43:17.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 263303 3.16 MB/sec execute 278 sec latency 141.435 ms 2024-04-02T13:43:18.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 264358 3.17 MB/sec execute 279 sec latency 581.003 ms 2024-04-02T13:43:19.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:19 smithi188 ceph-mon[36117]: pgmap v251: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:19 smithi122 ceph-mon[32114]: pgmap v251: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:19 smithi187 ceph-mon[36064]: pgmap v251: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:19.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266444 3.21 MB/sec execute 280 sec latency 11.605 ms 2024-04-02T13:43:20.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266726 3.20 MB/sec execute 281 sec latency 694.741 ms 2024-04-02T13:43:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:21 smithi122 ceph-mon[32114]: pgmap v252: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:21 smithi187 ceph-mon[36064]: pgmap v252: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:21 smithi188 ceph-mon[36117]: pgmap v252: 129 pgs: 129 active+clean; 183 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-04-02T13:43:21.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266726 3.19 MB/sec execute 282 sec latency 1694.778 ms 2024-04-02T13:43:22.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:22 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:22 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:22 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:43:22.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266726 3.18 MB/sec execute 283 sec latency 2694.837 ms 2024-04-02T13:43:23.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:23 smithi122 ceph-mon[32114]: pgmap v253: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:23.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:43:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:23 smithi187 ceph-mon[36064]: pgmap v253: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:43:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:23 smithi188 ceph-mon[36117]: pgmap v253: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:43:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:43:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:43:23.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266783 3.16 MB/sec execute 284 sec latency 3138.911 ms 2024-04-02T13:43:24.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266783 3.15 MB/sec execute 285 sec latency 1506.226 ms 2024-04-02T13:43:25.438 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:25.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 -- 172.21.15.122:0/1535724099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554100970 msgr2=0x7f1554100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 --2- 172.21.15.122:0/1535724099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554100970 0x7f1554100df0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f153c009930 tx=0x7f153c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 -- 172.21.15.122:0/1535724099 shutdown_connections 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 --2- 172.21.15.122:0/1535724099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 0x7f155410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 --2- 172.21.15.122:0/1535724099 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554100970 0x7f1554100df0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 --2- 172.21.15.122:0/1535724099 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554106980 0x7f1554106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.552+0000 7f155a196640 1 -- 172.21.15.122:0/1535724099 >> 172.21.15.122:0/1535724099 conn(0x7f1554077b00 msgr2=0x7f1554077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- 172.21.15.122:0/1535724099 shutdown_connections 2024-04-02T13:43:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- 172.21.15.122:0/1535724099 wait complete. 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 Processor -- start 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- start start 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554100970 0x7f15541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 0x7f15541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1554110d90 con 0x7f1554106980 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1554110c10 con 0x7f1554100970 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.553+0000 7f155a196640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1554110f10 con 0x7f1554101330 2024-04-02T13:43:25.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39704/0 (socket says 172.21.15.122:39704) 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 -- 172.21.15.122:0/2922201566 learned_addr learned my addr 172.21.15.122:0/2922201566 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f154bfff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 0x7f15541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f15537fe640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554100970 0x7f15541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 -- 172.21.15.122:0/2922201566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 msgr2=0x7f15541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 0x7f15541988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 -- 172.21.15.122:0/2922201566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554100970 msgr2=0x7f15541983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554100970 0x7f15541983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 -- 172.21.15.122:0/2922201566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f155419d3a0 con 0x7f1554106980 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f1553fff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f154c00eb10 tx=0x7f154c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f15517fa640 1 -- 172.21.15.122:0/2922201566 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c004b20 con 0x7f1554106980 2024-04-02T13:43:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f15517fa640 1 -- 172.21.15.122:0/2922201566 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c004cc0 con 0x7f1554106980 2024-04-02T13:43:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f155419d690 con 0x7f1554106980 2024-04-02T13:43:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f15517fa640 1 -- 172.21.15.122:0/2922201566 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c010760 con 0x7f1554106980 2024-04-02T13:43:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.554+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15541a4f20 con 0x7f1554106980 2024-04-02T13:43:25.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.556+0000 7f15517fa640 1 -- 172.21.15.122:0/2922201566 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f154c005020 con 0x7f1554106980 2024-04-02T13:43:25.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 Processor -- start 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 -- start start 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f15517fa640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 0x7f15400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4d2019830 con 0x55c4d20c31e0 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1fc1820 con 0x55c4d20b5600 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f155cd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4d20157e0 con 0x55c4d20b5f20 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39716/0 (socket says 172.21.15.122:39716) 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f1553fff640 1 -- 172.21.15.122:0/2322375202 learned_addr learned my addr 172.21.15.122:0/2322375202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f154bfff640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f154bfff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 0x7f15400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f15537fe640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.557+0000 7f15517fa640 1 -- 172.21.15.122:0/2922201566 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f154c0d1350 con 0x7f1554106980 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 -- 172.21.15.122:0/2322375202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 msgr2=0x55c4d20c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 -- 172.21.15.122:0/2322375202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 msgr2=0x55c4d20b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 -- 172.21.15.122:0/2322375202 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d1df77d0 con 0x55c4d20c31e0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f154bfff640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1553fff640 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f154c0674d0 tx=0x7f154c06c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98245e0896a3ef8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f154bfff640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 0x7f15400a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f153c005ab0 tx=0x7f153c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1549ffb640 1 -- 172.21.15.122:0/2322375202 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c09c860 con 0x55c4d20c31e0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1549ffb640 1 -- 172.21.15.122:0/2322375202 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c09d600 con 0x55c4d20c31e0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f1549ffb640 1 -- 172.21.15.122:0/2322375202 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c067ca0 con 0x55c4d20c31e0 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f155cd76740 1 -- 172.21.15.122:0/2322375202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 msgr2=0x55c4d20c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.558+0000 7f155cd76740 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f154c0674d0 tx=0x7f154c06c7e0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- 172.21.15.122:0/2322375202 shutdown_connections 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c55d0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- 172.21.15.122:0/2322375202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- 172.21.15.122:0/2322375202 >> 172.21.15.122:0/2322375202 conn(0x55c4d20b0000 msgr2=0x55c4d20b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- 172.21.15.122:0/2322375202 shutdown_connections 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- 172.21.15.122:0/2322375202 wait complete. 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- start start 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 Processor -- start 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 0x55c4d20c0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c4d20c5fa0 con 0x55c4d20b5600 2024-04-02T13:43:25.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c4d1fd4d30 con 0x55c4d20c31e0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c4d20c6120 con 0x55c4d20b5f20 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f155cd76740 10 client.0 _ll_register_callbacks cb 0x55c4d1fd4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.559+0000 7f15537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39726/0 (socket says 172.21.15.122:39726) 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 -- 172.21.15.122:0/3072803312 learned_addr learned my addr 172.21.15.122:0/3072803312 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f154bfff640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1553fff640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 0x55c4d20c0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 -- 172.21.15.122:0/3072803312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 msgr2=0x55c4d20bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 -- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 msgr2=0x55c4d20c0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 0x55c4d20c0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 -- 172.21.15.122:0/3072803312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d1d2d3f0 con 0x55c4d20b5600 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1553fff640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 0x55c4d20c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f15537fe640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1544013710 tx=0x7f1544013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0b65e8fa75860c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544011dd0 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f1544004020 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c09d600 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1544026050 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f155cd76740 10 client.18027 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c4d1ced0f0 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.560+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c09d600 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f154401a030 con 0x55c4d20b5600 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 10 client.18027 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 10 client.18027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 20 client.18027 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d20d3210 0x55c4d20d56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 4 client.18027 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4d2026d70 con 0x55c4d20d3210 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.561+0000 7f154bfff640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d20d3210 0x55c4d20d56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.562+0000 7f1532ffd640 10 client.18027 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.562+0000 7f154bfff640 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d20d3210 0x55c4d20d56a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dde1135c5020916 server_cookie=4ff26ae7a8c8b605 in_seq=0 out_seq=0 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.562+0000 7f1532ffd640 1 -- 172.21.15.122:0/3072803312 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4d2026d70 con 0x55c4d20d3210 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.562+0000 7f1532ffd640 10 client.18027 handle_command_reply: tid=0 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.562+0000 7f155cd76740 1 client.18027 shutdown 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 msgr2=0x55c4d20c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1544013710 tx=0x7f1544013bd0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 shutdown_connections 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d20d3210 0x55c4d20d56a0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20c31e0 0x55c4d20c0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5f20 0x55c4d20bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f1532ffd640 0 client.18027 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 --2- 172.21.15.122:0/3072803312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5600 0x55c4d20c2ab0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 >> 172.21.15.122:0/3072803312 conn(0x55c4d20b0000 msgr2=0x55c4d20b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 shutdown_connections 2024-04-02T13:43:25.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 1 -- 172.21.15.122:0/3072803312 wait complete. 2024-04-02T13:43:25.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.563+0000 7f155cd76740 20 client.18027 trim_cache size 0 max 16384 2024-04-02T13:43:25.578 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:25 smithi122 ceph-mon[32114]: pgmap v254: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 Processor -- start 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 -- start start 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5600 0x55c4d20ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15541acf10 con 0x55c4d20c31e0 2024-04-02T13:43:25.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15541aea50 con 0x55c4d20b5f20 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f155cd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15541ad5e0 con 0x55c4d20b5600 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.580+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39742/0 (socket says 172.21.15.122:39742) 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 -- 172.21.15.122:0/3832752105 learned_addr learned my addr 172.21.15.122:0/3832752105 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f154bfff640 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 -- 172.21.15.122:0/3832752105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5600 msgr2=0x55c4d20ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5600 0x55c4d20ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 -- 172.21.15.122:0/3832752105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5f20 msgr2=0x55c4d20c2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 -- 172.21.15.122:0/3832752105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4d20b5d50 con 0x55c4d20c31e0 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f1553fff640 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f154c0ce760 tx=0x7f154c0991a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a17cbb1781d74b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f15497fa640 1 -- 172.21.15.122:0/3832752105 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c06c070 con 0x55c4d20c31e0 2024-04-02T13:43:25.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f15497fa640 1 -- 172.21.15.122:0/3832752105 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c09d600 con 0x55c4d20c31e0 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.581+0000 7f15497fa640 1 -- 172.21.15.122:0/3832752105 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154c09a550 con 0x55c4d20c31e0 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- 172.21.15.122:0/3832752105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 msgr2=0x55c4d20c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f154c0ce760 tx=0x7f154c0991a0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- 172.21.15.122:0/3832752105 shutdown_connections 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20c31e0 0x55c4d20c2ab0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5f20 0x55c4d20c2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- 172.21.15.122:0/3832752105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20b5600 0x55c4d20ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- 172.21.15.122:0/3832752105 >> 172.21.15.122:0/3832752105 conn(0x55c4d20b0000 msgr2=0x55c4d20bb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- 172.21.15.122:0/3832752105 shutdown_connections 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- 172.21.15.122:0/3832752105 wait complete. 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- start start 2024-04-02T13:43:25.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 Processor -- start 2024-04-02T13:43:25.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20c31e0 0x55c4d20c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15541ae380 con 0x55c4d20b5f20 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15541adcb0 con 0x55c4d20b5600 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.582+0000 7f155cd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15541aaf10 con 0x55c4d20c31e0 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f155cd76740 10 client.0 _ll_register_callbacks cb 0x7f1544009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39754/0 (socket says 172.21.15.122:39754) 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 -- 172.21.15.122:0/4134766863 learned_addr learned my addr 172.21.15.122:0/4134766863 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f15537fe640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1553fff640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20c31e0 0x55c4d20c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 -- 172.21.15.122:0/4134766863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20c31e0 msgr2=0x55c4d20c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20c31e0 0x55c4d20c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 -- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 msgr2=0x55c4d20cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 -- 172.21.15.122:0/4134766863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4d20c7af0 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f15537fe640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f154bfff640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f153c036040 tx=0x7f153c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441273ec1ea892a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0044e0 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f153c0048c0 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f154c09d600 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f153c004ac0 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f155cd76740 10 client.18033 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c4d2139a00 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.583+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154c09d600 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f153c050470 con 0x55c4d20b5f20 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 10 client.18033 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 10 client.18033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:25.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 20 client.18033 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d213a160 0x55c4d213c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 4 client.18033 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4d20210c0 con 0x55c4d213a160 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f15537fe640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d213a160 0x55c4d213c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f15537fe640 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d213a160 0x55c4d213c580 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d6cd9d674fc080 server_cookie=dda48aa04483a7bc in_seq=0 out_seq=0 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.584+0000 7f1532ffd640 10 client.18033 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f1532ffd640 1 -- 172.21.15.122:0/4134766863 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c4d20210c0 con 0x55c4d213a160 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f1532ffd640 10 client.18033 handle_command_reply: tid=0 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 client.18033 shutdown 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 msgr2=0x55c4d20c6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f153c036040 tx=0x7f153c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 shutdown_connections 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c4d213a160 0x55c4d213c580 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c4d20c31e0 0x55c4d20c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c4d20b5f20 0x55c4d20c6cb0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f1532ffd640 0 client.18033 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 --2- 172.21.15.122:0/4134766863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c4d20b5600 0x55c4d20cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 >> 172.21.15.122:0/4134766863 conn(0x55c4d20b0000 msgr2=0x55c4d20b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 shutdown_connections 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 1 -- 172.21.15.122:0/4134766863 wait complete. 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155cd76740 20 client.18033 trim_cache size 0 max 16384 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:43:25.595 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:43:25.596 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 msgr2=0x7f15400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 0x7f15400a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f153c005ab0 tx=0x7f153c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 msgr2=0x7f155419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.585+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f154c00eb10 tx=0x7f154c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 shutdown_connections 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15400a3d20 0x7f15400a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1554106980 0x7f155419cc70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1554101330 0x7f15541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 --2- 172.21.15.122:0/2922201566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1554100970 0x7f15541983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 >> 172.21.15.122:0/2922201566 conn(0x7f1554077b00 msgr2=0x7f1554068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 shutdown_connections 2024-04-02T13:43:25.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.586+0000 7f155a196640 1 -- 172.21.15.122:0/2922201566 wait complete. 2024-04-02T13:43:25.599 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:43:25.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 -- 172.21.15.122:0/3182094365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 msgr2=0x7f143c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 --2- 172.21.15.122:0/3182094365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c10d870 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f142c009cd0 tx=0x7f142c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 -- 172.21.15.122:0/3182094365 shutdown_connections 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 --2- 172.21.15.122:0/3182094365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c10d870 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 --2- 172.21.15.122:0/3182094365 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1009a0 0x7f143c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 --2- 172.21.15.122:0/3182094365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1069b0 0x7f143c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 -- 172.21.15.122:0/3182094365 >> 172.21.15.122:0/3182094365 conn(0x7f143c076d80 msgr2=0x7f143c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 -- 172.21.15.122:0/3182094365 shutdown_connections 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 -- 172.21.15.122:0/3182094365 wait complete. 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.700+0000 7f144143c640 1 Processor -- start 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 -- start start 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 0x7f143c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1069b0 0x7f143c1a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f143c110dc0 con 0x7f143c101360 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f143c110c40 con 0x7f143c1069b0 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f144143c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f143c110f40 con 0x7f143c1009a0 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39770/0 (socket says 172.21.15.122:39770) 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 -- 172.21.15.122:0/831490058 learned_addr learned my addr 172.21.15.122:0/831490058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143b7fe640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1069b0 0x7f143c1a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143affd640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 0x7f143c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 -- 172.21.15.122:0/831490058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 msgr2=0x7f143c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 0x7f143c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 -- 172.21.15.122:0/831490058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1069b0 msgr2=0x7f143c1a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1069b0 0x7f143c1a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 -- 172.21.15.122:0/831490058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c1a4720 con 0x7f143c101360 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143affd640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 0x7f143c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.701+0000 7f143a7fc640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f142400c920 tx=0x7f142400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.702+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424007c70 con 0x7f143c101360 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.702+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c1a48f0 con 0x7f143c101360 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.702+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1424007e10 con 0x7f143c101360 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.702+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424018890 con 0x7f143c101360 2024-04-02T13:43:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.702+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c1a4e60 con 0x7f143c101360 2024-04-02T13:43:25.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.703+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143c1020e0 con 0x7f143c101360 2024-04-02T13:43:25.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.704+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1424018a70 con 0x7f143c101360 2024-04-02T13:43:25.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.704+0000 7f1433fff640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 0x7f14280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:25.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.704+0000 7f143affd640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 0x7f14280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:25.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.704+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f14240d0d20 con 0x7f143c101360 2024-04-02T13:43:25.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.705+0000 7f143affd640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 0x7f14280a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1434005fd0 tx=0x7f143400b5d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.709+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1424014030 con 0x7f143c101360 2024-04-02T13:43:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:25 smithi187 ceph-mon[36064]: pgmap v254: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:25 smithi188 ceph-mon[36117]: pgmap v254: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-04-02T13:43:25.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.928+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f143c108f10 con 0x7f143c101360 2024-04-02T13:43:25.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.929+0000 7f1433fff640 1 -- 172.21.15.122:0/831490058 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7f142409cfd0 con 0x7f143c101360 2024-04-02T13:43:25.932 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:43:25.932 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:43:25.933 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:43:25.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 msgr2=0x7f14280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 0x7f14280a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1434005fd0 tx=0x7f143400b5d0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 msgr2=0x7f143c1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:25.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f142400c920 tx=0x7f142400cde0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 shutdown_connections 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3d20 0x7f14280a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c1069b0 0x7f143c1a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c101360 0x7f143c1996a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 --2- 172.21.15.122:0/831490058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c1009a0 0x7f143c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 >> 172.21.15.122:0/831490058 conn(0x7f143c076d80 msgr2=0x7f143c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 shutdown_connections 2024-04-02T13:43:25.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:25.931+0000 7f144143c640 1 -- 172.21.15.122:0/831490058 wait complete. 2024-04-02T13:43:25.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:43:25.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266783 3.14 MB/sec execute 286 sec latency 2506.257 ms 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 -- 172.21.15.122:0/2162716018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a01016a0 msgr2=0x7fc4a010d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2162716018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a01016a0 0x7fc4a010d7e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc498009ca0 tx=0x7fc49802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 -- 172.21.15.122:0/2162716018 shutdown_connections 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2162716018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a01016a0 0x7fc4a010d7e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2162716018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a0100ce0 0x7fc4a0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2162716018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0106a30 0x7fc4a0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 -- 172.21.15.122:0/2162716018 >> 172.21.15.122:0/2162716018 conn(0x7fc4a00779e0 msgr2=0x7fc4a0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 -- 172.21.15.122:0/2162716018 shutdown_connections 2024-04-02T13:43:26.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.046+0000 7fc4a5455640 1 -- 172.21.15.122:0/2162716018 wait complete. 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 Processor -- start 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 -- start start 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 0x7fc4a01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 0x7fc4a0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110d80 con 0x7fc4a0106a30 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110c00 con 0x7fc4a0100ce0 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc4a5455640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a0110f00 con 0x7fc4a01016a0 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc49f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39792/0 (socket says 172.21.15.122:39792) 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc49f7fe640 1 -- 172.21.15.122:0/2760785093 learned_addr learned my addr 172.21.15.122:0/2760785093 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc49effd640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 0x7fc4a01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.047+0000 7fc49f7fe640 1 -- 172.21.15.122:0/2760785093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 msgr2=0x7fc4a0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc497fff640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 0x7fc4a0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49f7fe640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 0x7fc4a0199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49f7fe640 1 -- 172.21.15.122:0/2760785093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 msgr2=0x7fc4a01990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49f7fe640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 0x7fc4a01990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49f7fe640 1 -- 172.21.15.122:0/2760785093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01a4680 con 0x7fc4a0106a30 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49effd640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 0x7fc4a01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc497fff640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 0x7fc4a0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49f7fe640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc49802c700 tx=0x7fc498032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49cff9640 1 -- 172.21.15.122:0/2760785093 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498039040 con 0x7fc4a0106a30 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49cff9640 1 -- 172.21.15.122:0/2760785093 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc498044070 con 0x7fc4a0106a30 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc49cff9640 1 -- 172.21.15.122:0/2760785093 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980042a0 con 0x7fc4a0106a30 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a01a4850 con 0x7fc4a0106a30 2024-04-02T13:43:26.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.048+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a01a4cc0 con 0x7fc4a0106a30 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.050+0000 7fc49cff9640 1 -- 172.21.15.122:0/2760785093 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc498004440 con 0x7fc4a0106a30 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.050+0000 7fc4a8040740 1 Processor -- start 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.050+0000 7fc4a8040740 1 -- start start 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2beab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558cc2b4e7b0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cc2aed570 con 0x558cc2bea7a0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc4a8040740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cc2b4a760 con 0x558cc2beb0c0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc49cff9640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 0x7fc48c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39802/0 (socket says 172.21.15.122:39802) 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 -- 172.21.15.122:0/3807592773 learned_addr learned my addr 172.21.15.122:0/3807592773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc49f7fe640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 0x7fc48c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc49effd640 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2beab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc49cff9640 1 -- 172.21.15.122:0/2760785093 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fc4980f2f40 con 0x7fc4a0106a30 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 -- 172.21.15.122:0/3807592773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 msgr2=0x558cc2bf3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 -- 172.21.15.122:0/3807592773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 msgr2=0x558cc2beab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2beab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 -- 172.21.15.122:0/3807592773 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc28623f0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.051+0000 7fc497fff640 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc488009fa0 tx=0x7fc488009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1455cd01c8b59f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc49f7fe640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 0x7fc48c0a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4980f0460 tx=0x7fc4980095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4957fa640 1 -- 172.21.15.122:0/3807592773 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880134c0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4957fa640 1 -- 172.21.15.122:0/3807592773 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48801b070 con 0x558cc2bf3ca0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4957fa640 1 -- 172.21.15.122:0/3807592773 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488012890 con 0x558cc2bf3ca0 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 -- 172.21.15.122:0/3807592773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 msgr2=0x558cc2bf6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc488009fa0 tx=0x7fc488009650 comp rx=0 tx=0).stop 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 -- 172.21.15.122:0/3807592773 shutdown_connections 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf6090 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 --2- 172.21.15.122:0/3807592773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2beab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.052+0000 7fc4a8040740 1 -- 172.21.15.122:0/3807592773 >> 172.21.15.122:0/3807592773 conn(0x558cc2be4ed0 msgr2=0x558cc2be52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- 172.21.15.122:0/3807592773 shutdown_connections 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- 172.21.15.122:0/3807592773 wait complete. 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- start start 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 Processor -- start 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2bea7a0 0x558cc2bf3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2beb0c0 0x558cc2bf0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558cc2bf6a90 con 0x558cc2bf3ca0 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cc2b09e80 con 0x558cc2beb0c0 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cc2bf6c10 con 0x558cc2bea7a0 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc4a8040740 10 client.0 _ll_register_callbacks cb 0x558cc2b45aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39806/0 (socket says 172.21.15.122:39806) 2024-04-02T13:43:26.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc497fff640 1 -- 172.21.15.122:0/4167105818 learned_addr learned my addr 172.21.15.122:0/4167105818 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc49effd640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2beb0c0 0x558cc2bf0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.053+0000 7fc497fff640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2bea7a0 0x558cc2bf3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 -- 172.21.15.122:0/4167105818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2bea7a0 msgr2=0x558cc2bf3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2bea7a0 0x558cc2bf3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 -- 172.21.15.122:0/4167105818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2beb0c0 msgr2=0x558cc2bf0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2beb0c0 0x558cc2bf0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 -- 172.21.15.122:0/4167105818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc292c7d0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc497fff640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc488009ee0 tx=0x7fc488012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252ea4f31de4b0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880137e0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fc488013980 con 0x558cc2bf3ca0 2024-04-02T13:43:26.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48801b070 con 0x558cc2bf3ca0 2024-04-02T13:43:26.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc488033050 con 0x558cc2bf3ca0 2024-04-02T13:43:26.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 10 client.18051 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558cc2c02b70 con 0x558cc2bf3ca0 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48801b070 con 0x558cc2bf3ca0 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc48800c070 con 0x558cc2bf3ca0 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 10 client.18051 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 10 client.18051 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 20 client.18051 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c04030 0x558cc2c06420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 4 client.18051 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.054+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cc2b5bcf0 con 0x558cc2c04030 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc49effd640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c04030 0x558cc2c06420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc4867fc640 10 client.18051 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc49effd640 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c04030 0x558cc2c06420 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f32de0fec569524a server_cookie=74102bd4d31c69de in_seq=0 out_seq=0 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc4867fc640 1 -- 172.21.15.122:0/4167105818 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cc2b5bcf0 con 0x558cc2c04030 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc4867fc640 10 client.18051 handle_command_reply: tid=0 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.055+0000 7fc4a8040740 1 client.18051 shutdown 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 msgr2=0x558cc2bf1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc488009ee0 tx=0x7fc488012530 comp rx=0 tx=0).stop 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 shutdown_connections 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c04030 0x558cc2c06420 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf1220 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2beb0c0 0x558cc2bf0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4867fc640 0 client.18051 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:26.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4167105818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2bea7a0 0x558cc2bf3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 >> 172.21.15.122:0/4167105818 conn(0x558cc2be4ed0 msgr2=0x558cc2be6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 shutdown_connections 2024-04-02T13:43:26.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 1 -- 172.21.15.122:0/4167105818 wait complete. 2024-04-02T13:43:26.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.056+0000 7fc4a8040740 20 client.18051 trim_cache size 0 max 16384 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 Processor -- start 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 -- start start 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01af040 con 0x558cc2bea7a0 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01b0b80 con 0x558cc2bf3ca0 2024-04-02T13:43:26.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc4a8040740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01af710 con 0x558cc2beb0c0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39810/0 (socket says 172.21.15.122:39810) 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc497fff640 1 -- 172.21.15.122:0/4146075742 learned_addr learned my addr 172.21.15.122:0/4146075742 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.069+0000 7fc49effd640 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 -- 172.21.15.122:0/4146075742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 msgr2=0x558cc2bf3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 -- 172.21.15.122:0/4146075742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bf3ca0 msgr2=0x558cc2bf3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 -- 172.21.15.122:0/4146075742 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc2beaef0 con 0x558cc2bea7a0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc497fff640 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc48802e470 tx=0x7fc4880092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7961d3f425f70e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc494ff9640 1 -- 172.21.15.122:0/4146075742 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880095b0 con 0x558cc2bea7a0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc494ff9640 1 -- 172.21.15.122:0/4146075742 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48801b040 con 0x558cc2bea7a0 2024-04-02T13:43:26.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc494ff9640 1 -- 172.21.15.122:0/4146075742 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48802eb50 con 0x558cc2bea7a0 2024-04-02T13:43:26.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.070+0000 7fc4a8040740 1 -- 172.21.15.122:0/4146075742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 msgr2=0x558cc2bff980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc48802e470 tx=0x7fc4880092a0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- 172.21.15.122:0/4146075742 shutdown_connections 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bf3ca0 0x558cc2bf3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bf3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- 172.21.15.122:0/4146075742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bea7a0 0x558cc2bff980 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- 172.21.15.122:0/4146075742 >> 172.21.15.122:0/4146075742 conn(0x558cc2be4ed0 msgr2=0x558cc2bebde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- 172.21.15.122:0/4146075742 shutdown_connections 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- 172.21.15.122:0/4146075742 wait complete. 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- start start 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 Processor -- start 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2bed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bedde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01b04b0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01afde0 con 0x558cc2bea7a0 2024-04-02T13:43:26.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a01ad040 con 0x558cc2beb0c0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc4a8040740 10 client.0 _ll_register_callbacks cb 0x7fc4880154b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc497fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39826/0 (socket says 172.21.15.122:39826) 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc497fff640 1 -- 172.21.15.122:0/1170038482 learned_addr learned my addr 172.21.15.122:0/1170038482 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc49effd640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bedde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.071+0000 7fc497fff640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2bed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 -- 172.21.15.122:0/1170038482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 msgr2=0x558cc2bedde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bedde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 -- 172.21.15.122:0/1170038482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 msgr2=0x558cc2bed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2bed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 -- 172.21.15.122:0/1170038482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc2bfa280 con 0x558cc2bf3ca0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc497fff640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc4880057a0 tx=0x7fc488009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2836fc33c32d1d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4880097b0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fc4880137e0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48801b040 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc488045050 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 10 client.18057 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558cc2c6a7f0 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc48801b040 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc488040030 con 0x558cc2bf3ca0 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 10 client.18057 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 10 client.18057 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 20 client.18057 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c6adf0 0x558cc2c6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 4 client.18057 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:26.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.072+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cc2b56040 con 0x558cc2c6adf0 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc49effd640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c6adf0 0x558cc2c6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4867fc640 10 client.18057 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc49effd640 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c6adf0 0x558cc2c6d210 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=46cfaf87bedc62e6 server_cookie=bde9206130f810ad in_seq=0 out_seq=0 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4867fc640 1 -- 172.21.15.122:0/1170038482 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cc2b56040 con 0x558cc2c6adf0 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4867fc640 10 client.18057 handle_command_reply: tid=0 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 client.18057 shutdown 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 msgr2=0x558cc2bee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc4880057a0 tx=0x7fc488009b20 comp rx=0 tx=0).stop 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 shutdown_connections 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x558cc2c6adf0 0x558cc2c6d210 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cc2bf3ca0 0x558cc2bee320 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4867fc640 0 client.18057 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc2beb0c0 0x558cc2bedde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 --2- 172.21.15.122:0/1170038482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cc2bea7a0 0x558cc2bed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 >> 172.21.15.122:0/1170038482 conn(0x558cc2be4ed0 msgr2=0x558cc2be36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 shutdown_connections 2024-04-02T13:43:26.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 1 -- 172.21.15.122:0/1170038482 wait complete. 2024-04-02T13:43:26.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.073+0000 7fc4a8040740 20 client.18057 trim_cache size 0 max 16384 2024-04-02T13:43:26.084 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 msgr2=0x7fc48c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 0x7fc48c0a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4980f0460 tx=0x7fc4980095c0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 msgr2=0x7fc4a01a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc49802c700 tx=0x7fc498032f20 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 shutdown_connections 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc48c0a3d20 0x7fc48c0a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4a0106a30 0x7fc4a01a4140 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4a01016a0 0x7fc4a0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 --2- 172.21.15.122:0/2760785093 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4a0100ce0 0x7fc4a01990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 >> 172.21.15.122:0/2760785093 conn(0x7fc4a00779e0 msgr2=0x7fc4a00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 shutdown_connections 2024-04-02T13:43:26.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.074+0000 7fc4a5455640 1 -- 172.21.15.122:0/2760785093 wait complete. 2024-04-02T13:43:26.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:43:26.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 -- 172.21.15.122:0/122353115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4301013e0 msgr2=0x7fd43010d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 --2- 172.21.15.122:0/122353115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4301013e0 0x7fd43010d8f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fd420009930 tx=0x7fd42002bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 -- 172.21.15.122:0/122353115 shutdown_connections 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 --2- 172.21.15.122:0/122353115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4301013e0 0x7fd43010d8f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 --2- 172.21.15.122:0/122353115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd430100a20 0x7fd430100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 --2- 172.21.15.122:0/122353115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd430106a30 0x7fd430106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 -- 172.21.15.122:0/122353115 >> 172.21.15.122:0/122353115 conn(0x7fd430077b60 msgr2=0x7fd430077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 -- 172.21.15.122:0/122353115 shutdown_connections 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 -- 172.21.15.122:0/122353115 wait complete. 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.229+0000 7fd434b8d640 1 Processor -- start 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- start start 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd430100a20 0x7fd430198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4301013e0 0x7fd430198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd430110e10 con 0x7fd430106a30 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd430110c90 con 0x7fd430100a20 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd430110f90 con 0x7fd4301013e0 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39852/0 (socket says 172.21.15.122:39852) 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 -- 172.21.15.122:0/3454403808 learned_addr learned my addr 172.21.15.122:0/3454403808 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 -- 172.21.15.122:0/3454403808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4301013e0 msgr2=0x7fd430198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4301013e0 0x7fd430198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 -- 172.21.15.122:0/3454403808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd430100a20 msgr2=0x7fd430198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd430100a20 0x7fd430198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 -- 172.21.15.122:0/3454403808 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43019d310 con 0x7fd430106a30 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd42ed9d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd420005940 tx=0x7fd42002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd41f7fe640 1 -- 172.21.15.122:0/3454403808 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420041070 con 0x7fd430106a30 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43019d5a0 con 0x7fd430106a30 2024-04-02T13:43:26.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd41f7fe640 1 -- 172.21.15.122:0/3454403808 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd420004450 con 0x7fd430106a30 2024-04-02T13:43:26.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd41f7fe640 1 -- 172.21.15.122:0/3454403808 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420034440 con 0x7fd430106a30 2024-04-02T13:43:26.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.230+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4301a4e90 con 0x7fd430106a30 2024-04-02T13:43:26.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.233+0000 7fd437774740 1 Processor -- start 2024-04-02T13:43:26.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.233+0000 7fd41f7fe640 1 -- 172.21.15.122:0/3454403808 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd420044070 con 0x7fd430106a30 2024-04-02T13:43:26.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.233+0000 7fd437774740 1 -- start start 2024-04-02T13:43:26.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeaca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cfeea227b0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cfee9c1570 con 0x55cfeeac7ca0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd437774740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cfeea1e760 con 0x55cfeeabf0c0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39860/0 (socket says 172.21.15.122:39860) 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 -- 172.21.15.122:0/666554255 learned_addr learned my addr 172.21.15.122:0/666554255 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd41f7fe640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 0x7fd4180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42e59c640 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42ed9d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 0x7fd4180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 -- 172.21.15.122:0/666554255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 msgr2=0x55cfeeac7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 -- 172.21.15.122:0/666554255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 msgr2=0x55cfeeaca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeaca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 -- 172.21.15.122:0/666554255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfee7363f0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd41f7fe640 1 -- 172.21.15.122:0/3454403808 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fd4200f2730 con 0x7fd430106a30 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e90 tx=0x7fd4240106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336226c79438042c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd42ed9d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 0x7fd4180a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4200efc50 tx=0x7fd4200b3880 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.234+0000 7fd41cff9640 1 -- 172.21.15.122:0/666554255 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424016070 con 0x55cfeeabe7a0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd41cff9640 1 -- 172.21.15.122:0/666554255 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd424012b00 con 0x55cfeeabe7a0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd41cff9640 1 -- 172.21.15.122:0/666554255 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424011a50 con 0x55cfeeabe7a0 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 -- 172.21.15.122:0/666554255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 msgr2=0x55cfeeabeb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e90 tx=0x7fd4240106e0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 -- 172.21.15.122:0/666554255 shutdown_connections 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeaca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 --2- 172.21.15.122:0/666554255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeabeb80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 -- 172.21.15.122:0/666554255 >> 172.21.15.122:0/666554255 conn(0x55cfeeab8ed0 msgr2=0x55cfeeab92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 -- 172.21.15.122:0/666554255 shutdown_connections 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.235+0000 7fd437774740 1 -- 172.21.15.122:0/666554255 wait complete. 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 -- start start 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 Processor -- start 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cfeeacaa90 con 0x55cfeeabf0c0 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cfee9dde80 con 0x55cfeeabe7a0 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cfeeacac10 con 0x55cfeeac7ca0 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd437774740 10 client.0 _ll_register_callbacks cb 0x55cfeea19aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39866/0 (socket says 172.21.15.122:39866) 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 -- 172.21.15.122:0/2262786589 learned_addr learned my addr 172.21.15.122:0/2262786589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 -- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 msgr2=0x55cfeeac5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 -- 172.21.15.122:0/2262786589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabe7a0 msgr2=0x55cfeeac4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 -- 172.21.15.122:0/2262786589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfee8007d0 con 0x55cfeeabf0c0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.236+0000 7fd42e59c640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d640 tx=0x7fd42800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3878af8294b34072 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428004220 con 0x55cfeeabf0c0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fd4280043c0 con 0x55cfeeabf0c0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd424012b00 con 0x55cfeeabf0c0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd428005020 con 0x55cfeeabf0c0 2024-04-02T13:43:26.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 10 client.18069 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cfeead6c20 con 0x55cfeeabf0c0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd424012b00 con 0x55cfeeabf0c0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd42800f6f0 con 0x55cfeeabf0c0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 10 client.18069 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 10 client.18069 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 20 client.18069 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeead80e0 0x55cfeeada530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 4 client.18069 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfeea2fcf0 con 0x55cfeead80e0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.237+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeead80e0 0x55cfeeada530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.238+0000 7fd415ffb640 10 client.18069 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.238+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeead80e0 0x55cfeeada530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50686ebf2edd1949 server_cookie=40814778ccf29b18 in_seq=0 out_seq=0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd415ffb640 1 -- 172.21.15.122:0/2262786589 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfeea2fcf0 con 0x55cfeead80e0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd415ffb640 10 client.18069 handle_command_reply: tid=0 2024-04-02T13:43:26.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 client.18069 shutdown 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 msgr2=0x55cfeeac5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d640 tx=0x7fd42800db00 comp rx=0 tx=0).stop 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 shutdown_connections 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeead80e0 0x55cfeeada530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac5040 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 --2- 172.21.15.122:0/2262786589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd415ffb640 0 client.18069 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 >> 172.21.15.122:0/2262786589 conn(0x55cfeeab8ed0 msgr2=0x55cfeeabac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 shutdown_connections 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 1 -- 172.21.15.122:0/2262786589 wait complete. 2024-04-02T13:43:26.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.239+0000 7fd437774740 20 client.18069 trim_cache size 0 max 16384 2024-04-02T13:43:26.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 Processor -- start 2024-04-02T13:43:26.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 -- start start 2024-04-02T13:43:26.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeead39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301aef90 con 0x55cfeeabe7a0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301b0ad0 con 0x55cfeeac7ca0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd437774740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301af660 con 0x55cfeeabf0c0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39868/0 (socket says 172.21.15.122:39868) 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 -- 172.21.15.122:0/276455428 learned_addr learned my addr 172.21.15.122:0/276455428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 -- 172.21.15.122:0/276455428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 msgr2=0x55cfeead39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeead39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 -- 172.21.15.122:0/276455428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 msgr2=0x55cfeeac3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.252+0000 7fd42dd9b640 1 -- 172.21.15.122:0/276455428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfeeabdf40 con 0x55cfeeabe7a0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd424005fd0 tx=0x7fd424011610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86080752da13aa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd417fff640 1 -- 172.21.15.122:0/276455428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424016040 con 0x55cfeeabe7a0 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd417fff640 1 -- 172.21.15.122:0/276455428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4240278e0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd417fff640 1 -- 172.21.15.122:0/276455428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424011cc0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 -- 172.21.15.122:0/276455428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 msgr2=0x55cfeeac7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fd424005fd0 tx=0x7fd424011610 comp rx=0 tx=0).stop 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 -- 172.21.15.122:0/276455428 shutdown_connections 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeabf0c0 0x55cfeead39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 --2- 172.21.15.122:0/276455428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac7230 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 -- 172.21.15.122:0/276455428 >> 172.21.15.122:0/276455428 conn(0x55cfeeab8ed0 msgr2=0x55cfeeac33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 -- 172.21.15.122:0/276455428 shutdown_connections 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.253+0000 7fd437774740 1 -- 172.21.15.122:0/276455428 wait complete. 2024-04-02T13:43:26.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 -- start start 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 Processor -- start 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301b0400 con 0x55cfeeabe7a0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301afd30 con 0x55cfeeabf0c0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4301acf90 con 0x55cfeeac7ca0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd437774740 10 client.0 _ll_register_callbacks cb 0x7fd428005eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39880/0 (socket says 172.21.15.122:39880) 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 -- 172.21.15.122:0/4130978195 learned_addr learned my addr 172.21.15.122:0/4130978195 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42e59c640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 -- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 msgr2=0x55cfeeac8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 -- 172.21.15.122:0/4130978195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabf0c0 msgr2=0x55cfeeac89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 -- 172.21.15.122:0/4130978195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfeeacf160 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd42dd9b640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e60 tx=0x7fd4240114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de1e63efc209494 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.254+0000 7fd415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424016070 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fd424011680 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4240278e0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd424011880 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 10 client.18075 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cfeead0460 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4240278e0 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fd424037030 con 0x55cfeeabe7a0 2024-04-02T13:43:26.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 10 client.18075 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 10 client.18075 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 20 client.18075 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeeb3e770 0x55cfeeb40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 4 client.18075 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cfeea2a040 con 0x55cfeeb3e770 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.255+0000 7fd42e59c640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeeb3e770 0x55cfeeb40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd415ffb640 10 client.18075 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd42e59c640 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeeb3e770 0x55cfeeb40b60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d701ada7825c55f3 server_cookie=68f19a59f7d62b8 in_seq=0 out_seq=0 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd415ffb640 1 -- 172.21.15.122:0/4130978195 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cfeea2a040 con 0x55cfeeb3e770 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd415ffb640 10 client.18075 handle_command_reply: tid=0 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 client.18075 shutdown 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 msgr2=0x55cfeeac84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd424009e60 tx=0x7fd4240114c0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 shutdown_connections 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cfeeac7ca0 0x55cfeeac8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cfeeabf0c0 0x55cfeeac89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55cfeeb3e770 0x55cfeeb40b60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 --2- 172.21.15.122:0/4130978195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cfeeabe7a0 0x55cfeeac84b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd415ffb640 0 client.18075 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 >> 172.21.15.122:0/4130978195 conn(0x55cfeeab8ed0 msgr2=0x55cfeeab76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 shutdown_connections 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 1 -- 172.21.15.122:0/4130978195 wait complete. 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.256+0000 7fd437774740 20 client.18075 trim_cache size 0 max 16384 2024-04-02T13:43:26.266 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:26.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 msgr2=0x7fd4180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 0x7fd4180a61e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd4200efc50 tx=0x7fd4200b3880 comp rx=0 tx=0).stop 2024-04-02T13:43:26.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 msgr2=0x7fd43019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fd420005940 tx=0x7fd42002cc70 comp rx=0 tx=0).stop 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 shutdown_connections 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4180a3d20 0x7fd4180a61e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd430106a30 0x7fd43019cbe0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4301013e0 0x7fd430198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 --2- 172.21.15.122:0/3454403808 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd430100a20 0x7fd430198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 >> 172.21.15.122:0/3454403808 conn(0x7fd430077b60 msgr2=0x7fd43006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 shutdown_connections 2024-04-02T13:43:26.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.257+0000 7fd434b8d640 1 -- 172.21.15.122:0/3454403808 wait complete. 2024-04-02T13:43:26.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- 172.21.15.122:0/3185912439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 msgr2=0x7f01f4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3185912439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f4106da0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009930 tx=0x7f01e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- 172.21.15.122:0/3185912439 shutdown_connections 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3185912439 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f4101370 0x7f01f410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3185912439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01f41009b0 0x7f01f4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3185912439 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f4106da0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- 172.21.15.122:0/3185912439 >> 172.21.15.122:0/3185912439 conn(0x7f01f4076d80 msgr2=0x7f01f4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- 172.21.15.122:0/3185912439 shutdown_connections 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- 172.21.15.122:0/3185912439 wait complete. 2024-04-02T13:43:26.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 Processor -- start 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.372+0000 7f01f8a55640 1 -- start start 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f41009b0 0x7f01f41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01f4101370 0x7f01f4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4110da0 con 0x7f01f41069c0 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4110c20 con 0x7f01f41009b0 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01f4110f20 con 0x7f01f4101370 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39910/0 (socket says 172.21.15.122:39910) 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 -- 172.21.15.122:0/3005845582 learned_addr learned my addr 172.21.15.122:0/3005845582 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f259c640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f41009b0 0x7f01f41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 -- 172.21.15.122:0/3005845582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01f4101370 msgr2=0x7f01f4199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01f4101370 0x7f01f4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 -- 172.21.15.122:0/3005845582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f41009b0 msgr2=0x7f01f41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f41009b0 0x7f01f41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 -- 172.21.15.122:0/3005845582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f41a47a0 con 0x7f01f41069c0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f2d9d640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00eaa0 tx=0x7f01ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01e37fe640 1 -- 172.21.15.122:0/3005845582 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00cdb0 con 0x7f01f41069c0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01e37fe640 1 -- 172.21.15.122:0/3005845582 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01ec005020 con 0x7f01f41069c0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f41a4970 con 0x7f01f41069c0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01e37fe640 1 -- 172.21.15.122:0/3005845582 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec0116b0 con 0x7f01f41069c0 2024-04-02T13:43:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.373+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f41a4e70 con 0x7f01f41069c0 2024-04-02T13:43:26.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01e37fe640 1 -- 172.21.15.122:0/3005845582 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f01ec005560 con 0x7f01f41069c0 2024-04-02T13:43:26.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 Processor -- start 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 -- start start 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631b29a67b0 con 0x5631b2a430c0 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631b2945570 con 0x5631b2a427a0 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01fb643740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631b29a2760 con 0x5631b2a4bca0 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01e37fe640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 0x7f01dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39920/0 (socket says 172.21.15.122:39920) 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.376+0000 7f01f259c640 1 -- 172.21.15.122:0/3286456178 learned_addr learned my addr 172.21.15.122:0/3286456178 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f2d9d640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 0x7f01dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01e37fe640 1 -- 172.21.15.122:0/3005845582 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f01ec0d0df0 con 0x7f01f41069c0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 -- 172.21.15.122:0/3286456178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 msgr2=0x5631b2a4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 -- 172.21.15.122:0/3286456178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 msgr2=0x5631b2a42b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a42b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 -- 172.21.15.122:0/3286456178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b26ba3f0 con 0x5631b2a430c0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f259c640 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f01e802cfd0 tx=0x7f01e8005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c16e963180f28cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01f2d9d640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 0x7f01dc0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00efd0 tx=0x7f01ec06e3e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01e0ff9640 1 -- 172.21.15.122:0/3286456178 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8042070 con 0x5631b2a430c0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01e0ff9640 1 -- 172.21.15.122:0/3286456178 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e802ca80 con 0x5631b2a430c0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.377+0000 7f01e0ff9640 1 -- 172.21.15.122:0/3286456178 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8004cb0 con 0x5631b2a430c0 2024-04-02T13:43:26.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- 172.21.15.122:0/3286456178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 msgr2=0x5631b2a4b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f01e802cfd0 tx=0x7f01e8005c00 comp rx=0 tx=0).stop 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- 172.21.15.122:0/3286456178 shutdown_connections 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a4b760 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 --2- 172.21.15.122:0/3286456178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a42b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- 172.21.15.122:0/3286456178 >> 172.21.15.122:0/3286456178 conn(0x5631b2a3ced0 msgr2=0x5631b2a3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- 172.21.15.122:0/3286456178 shutdown_connections 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- 172.21.15.122:0/3286456178 wait complete. 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 -- start start 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 Processor -- start 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.378+0000 7f01fb643740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5631b2a4ea30 con 0x5631b2a430c0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5631b2961e80 con 0x5631b2a427a0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5631b2a4ebb0 con 0x5631b2a4bca0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 10 client.0 _ll_register_callbacks cb 0x5631b299daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39932/0 (socket says 172.21.15.122:39932) 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 -- 172.21.15.122:0/4226914010 learned_addr learned my addr 172.21.15.122:0/4226914010 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 -- 172.21.15.122:0/4226914010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 msgr2=0x5631b2a49050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a49050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 -- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 msgr2=0x5631b2a4b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a4b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 -- 172.21.15.122:0/4226914010 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b27847d0 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01f259c640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f036dbe302648127 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e8042040 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f01e8004450 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e802ca80 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f01e8057050 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 10 client.18087 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5631b2a5aac0 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.379+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e802ca80 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f01e8003bc0 con 0x5631b2a430c0 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 10 client.18087 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 10 client.18087 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:43:26.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 20 client.18087 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2a5be10 0x5631b2a5e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 4 client.18087 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631b29b3cf0 con 0x5631b2a5be10 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2a5be10 0x5631b2a5e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01d9ffb640 10 client.18087 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.380+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2a5be10 0x5631b2a5e200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f15ea1c73227623e server_cookie=6288e2a7878ee83c in_seq=0 out_seq=0 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4226914010 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631b29b3cf0 con 0x5631b2a5be10 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01d9ffb640 10 client.18087 handle_command_reply: tid=0 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 client.18087 shutdown 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 msgr2=0x5631b2a48a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f01e8000c00 tx=0x7f01e80046c0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 shutdown_connections 2024-04-02T13:43:26.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a4bca0 0x5631b2a49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a430c0 0x5631b2a48a40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2a5be10 0x5631b2a5e200 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 --2- 172.21.15.122:0/4226914010 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01d9ffb640 0 client.18087 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.381+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 >> 172.21.15.122:0/4226914010 conn(0x5631b2a3ced0 msgr2=0x5631b2a3ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.382+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 shutdown_connections 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.382+0000 7f01fb643740 1 -- 172.21.15.122:0/4226914010 wait complete. 2024-04-02T13:43:26.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.382+0000 7f01fb643740 20 client.18087 trim_cache size 0 max 16384 2024-04-02T13:43:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 Processor -- start 2024-04-02T13:43:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 -- start start 2024-04-02T13:43:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a427a0 0x5631b2a4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a430c0 0x5631b2a4b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41af010 con 0x5631b2a4bca0 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b0b50 con 0x5631b2a430c0 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01fb643740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41af6e0 con 0x5631b2a427a0 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39938/0 (socket says 172.21.15.122:39938) 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 -- 172.21.15.122:0/2373165623 learned_addr learned my addr 172.21.15.122:0/2373165623 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 -- 172.21.15.122:0/2373165623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a427a0 msgr2=0x5631b2a4b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a427a0 0x5631b2a4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f259c640 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a430c0 0x5631b2a4b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 -- 172.21.15.122:0/2373165623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a430c0 msgr2=0x5631b2a4b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a430c0 0x5631b2a4b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 -- 172.21.15.122:0/2373165623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631b2a46fd0 con 0x5631b2a4bca0 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005d00 tx=0x7f01e401d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b2d26b47607c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01dbfff640 1 -- 172.21.15.122:0/2373165623 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011b70 con 0x5631b2a4bca0 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01dbfff640 1 -- 172.21.15.122:0/2373165623 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e4011d10 con 0x5631b2a4bca0 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.395+0000 7f01dbfff640 1 -- 172.21.15.122:0/2373165623 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e400f3c0 con 0x5631b2a4bca0 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- 172.21.15.122:0/2373165623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 msgr2=0x5631b2a47350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005d00 tx=0x7f01e401d750 comp rx=0 tx=0).stop 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- 172.21.15.122:0/2373165623 shutdown_connections 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a47350 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a430c0 0x5631b2a4b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- 172.21.15.122:0/2373165623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a427a0 0x5631b2a4b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- 172.21.15.122:0/2373165623 >> 172.21.15.122:0/2373165623 conn(0x5631b2a3ced0 msgr2=0x5631b2a41f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- 172.21.15.122:0/2373165623 shutdown_connections 2024-04-02T13:43:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- 172.21.15.122:0/2373165623 wait complete. 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 -- start start 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 Processor -- start 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a430c0 0x5631b2a45310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.396+0000 7f01fb643740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41b0480 con 0x5631b2a4bca0 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41afdb0 con 0x5631b2a427a0 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01f41ad010 con 0x5631b2a430c0 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 10 client.0 _ll_register_callbacks cb 0x7f01e8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39946/0 (socket says 172.21.15.122:39946) 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 -- 172.21.15.122:0/4079906487 learned_addr learned my addr 172.21.15.122:0/4079906487 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f259c640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a430c0 0x5631b2a45310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 -- 172.21.15.122:0/4079906487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a430c0 msgr2=0x5631b2a45310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a430c0 0x5631b2a45310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 -- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 msgr2=0x5631b2a61420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 -- 172.21.15.122:0/4079906487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631b2a45e40 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01f1d9b640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005fd0 tx=0x7f01e400dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c4b7bf28e79a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e40049a0 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f01e4004030 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01e4011d10 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f01e4038050 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 10 client.18093 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5631b2ac1de0 con 0x5631b2a4bca0 2024-04-02T13:43:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.397+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01e4011d10 con 0x5631b2a4bca0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f01e401c070 con 0x5631b2a4bca0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 10 client.18093 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 10 client.18093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 20 client.18093 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2ac24b0 0x5631b2ac48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 4 client.18093 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631b29ae040 con 0x5631b2ac24b0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01f259c640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2ac24b0 0x5631b2ac48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01d9ffb640 10 client.18093 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01f259c640 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2ac24b0 0x5631b2ac48d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=52f5f5d0d7e45d1c server_cookie=d04c68d64fbf77a8 in_seq=0 out_seq=0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01d9ffb640 1 -- 172.21.15.122:0/4079906487 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631b29ae040 con 0x5631b2ac24b0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01d9ffb640 10 client.18093 handle_command_reply: tid=0 2024-04-02T13:43:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.398+0000 7f01fb643740 1 client.18093 shutdown 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 msgr2=0x5631b2a45850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f01e4005fd0 tx=0x7f01e400dcb0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 shutdown_connections 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5631b2a4bca0 0x5631b2a45850 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5631b2a430c0 0x5631b2a45310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5631b2ac24b0 0x5631b2ac48d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 --2- 172.21.15.122:0/4079906487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5631b2a427a0 0x5631b2a61420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01d9ffb640 0 client.18093 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 >> 172.21.15.122:0/4079906487 conn(0x5631b2a3ced0 msgr2=0x5631b2a471e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 shutdown_connections 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 1 -- 172.21.15.122:0/4079906487 wait complete. 2024-04-02T13:43:26.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01fb643740 20 client.18093 trim_cache size 0 max 16384 2024-04-02T13:43:26.410 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 msgr2=0x7f01dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 0x7f01dc0a61e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00efd0 tx=0x7f01ec06e3e0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 msgr2=0x7f01f41a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00eaa0 tx=0x7f01ec00ef60 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 shutdown_connections 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01dc0a3d20 0x7f01dc0a61e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f01f41069c0 0x7f01f41a4260 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01f4101370 0x7f01f4199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 --2- 172.21.15.122:0/3005845582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01f41009b0 0x7f01f41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 >> 172.21.15.122:0/3005845582 conn(0x7f01f4076d80 msgr2=0x7f01f40fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 shutdown_connections 2024-04-02T13:43:26.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.399+0000 7f01f8a55640 1 -- 172.21.15.122:0/3005845582 wait complete. 2024-04-02T13:43:26.415 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:43:26.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 -- 172.21.15.122:0/3696843490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f8101180 msgr2=0x7f83f810d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/3696843490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f8101180 0x7f83f810d690 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009900 tx=0x7f83f402bc60 comp rx=0 tx=0).stop 2024-04-02T13:43:26.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 -- 172.21.15.122:0/3696843490 shutdown_connections 2024-04-02T13:43:26.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/3696843490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f8101180 0x7f83f810d690 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/3696843490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f81007c0 0x7f83f8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/3696843490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81067d0 0x7f83f8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.534+0000 7f83fe5c8640 1 -- 172.21.15.122:0/3696843490 >> 172.21.15.122:0/3696843490 conn(0x7f83f80fc2d0 msgr2=0x7f83f80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.536+0000 7f83fe5c8640 1 -- 172.21.15.122:0/3696843490 shutdown_connections 2024-04-02T13:43:26.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.538+0000 7f83fe5c8640 1 -- 172.21.15.122:0/3696843490 wait complete. 2024-04-02T13:43:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 Processor -- start 2024-04-02T13:43:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 -- start start 2024-04-02T13:43:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81007c0 0x7f83f8115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f8101180 0x7f83f810fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810e800 con 0x7f83f81067d0 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810e680 con 0x7f83f81007c0 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.541+0000 7f83fe5c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83f810e980 con 0x7f83f8101180 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.542+0000 7f83fd5c6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81007c0 0x7f83f8115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.542+0000 7f83fddc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.542+0000 7f83fddc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39960/0 (socket says 172.21.15.122:39960) 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.542+0000 7f83fddc7640 1 -- 172.21.15.122:0/213127295 learned_addr learned my addr 172.21.15.122:0/213127295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.542+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f8101180 0x7f83f810fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 -- 172.21.15.122:0/213127295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f8101180 msgr2=0x7f83f810fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f8101180 0x7f83f810fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 -- 172.21.15.122:0/213127295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81007c0 msgr2=0x7f83f8115d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81007c0 0x7f83f8115d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 -- 172.21.15.122:0/213127295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f8110a90 con 0x7f83f81067d0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.543+0000 7f83fddc7640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009850 tx=0x7f83f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.544+0000 7f83ee7fc640 1 -- 172.21.15.122:0/213127295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4041070 con 0x7f83f81067d0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.544+0000 7f83ee7fc640 1 -- 172.21.15.122:0/213127295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f402cd40 con 0x7f83f81067d0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.544+0000 7f83ee7fc640 1 -- 172.21.15.122:0/213127295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f4034440 con 0x7f83f81067d0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.544+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f81b3620 con 0x7f83f81067d0 2024-04-02T13:43:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.544+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81b3a30 con 0x7f83f81067d0 2024-04-02T13:43:26.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.548+0000 7f8402412740 1 Processor -- start 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f83ee7fc640 1 -- 172.21.15.122:0/213127295 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f83f4044070 con 0x7f83f81067d0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 -- start start 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c53290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4bab7b0 con 0x55f0a4c479a0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4b4a570 con 0x55f0a4c50ea0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.549+0000 7f8402412740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4ba7760 con 0x55f0a4c482c0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39968/0 (socket says 172.21.15.122:39968) 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 -- 172.21.15.122:0/3896052293 learned_addr learned my addr 172.21.15.122:0/3896052293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83ee7fc640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 0x7f83e40a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c53290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 -- 172.21.15.122:0/3896052293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 msgr2=0x55f0a4c50960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 -- 172.21.15.122:0/3896052293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 msgr2=0x55f0a4c53290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c53290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 -- 172.21.15.122:0/3896052293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a48bf3f0 con 0x55f0a4c479a0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fddc7640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 0x7f83e40a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83ee7fc640 1 -- 172.21.15.122:0/213127295 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f83f40f2630 con 0x7f83f81067d0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f83e8013530 tx=0x7f83e800a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff39f9647198acd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83e37fe640 1 -- 172.21.15.122:0/3896052293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8019030 con 0x55f0a4c479a0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83e37fe640 1 -- 172.21.15.122:0/3896052293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8005500 con 0x55f0a4c479a0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.550+0000 7f83e37fe640 1 -- 172.21.15.122:0/3896052293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80116b0 con 0x55f0a4c479a0 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 -- 172.21.15.122:0/3896052293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 msgr2=0x55f0a4c47d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f83e8013530 tx=0x7f83e800a720 comp rx=0 tx=0).stop 2024-04-02T13:43:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f83fddc7640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 0x7f83e40a63f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83f40be2d0 tx=0x7f83f404bbd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 -- 172.21.15.122:0/3896052293 shutdown_connections 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c53290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 --2- 172.21.15.122:0/3896052293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c479a0 0x55f0a4c47d80 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 -- 172.21.15.122:0/3896052293 >> 172.21.15.122:0/3896052293 conn(0x55f0a4c420d0 msgr2=0x55f0a4c424b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 -- 172.21.15.122:0/3896052293 shutdown_connections 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.551+0000 7f8402412740 1 -- 172.21.15.122:0/3896052293 wait complete. 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 -- start start 2024-04-02T13:43:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 Processor -- start 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c4e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4c53c90 con 0x55f0a4c482c0 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4b66e80 con 0x55f0a4c50ea0 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0a4c53e10 con 0x55f0a4c479a0 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f8402412740 10 client.0 _ll_register_callbacks cb 0x55f0a4ba2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39972/0 (socket says 172.21.15.122:39972) 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 -- 172.21.15.122:0/3355418486 learned_addr learned my addr 172.21.15.122:0/3355418486 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c4e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 -- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 msgr2=0x55f0a4c507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 -- 172.21.15.122:0/3355418486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 msgr2=0x55f0a4c4e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c4e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fd5c6640 1 -- 172.21.15.122:0/3355418486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a49897d0 con 0x55f0a4c482c0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.552+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c4e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f83f000be90 tx=0x7f83f000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b655505c5e8f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f001a020 con 0x55f0a4c482c0 2024-04-02T13:43:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f83f0016670 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8005500 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f83f0025050 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 10 client.18105 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f0a4c5fcd0 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8005500 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f83f00159d0 con 0x55f0a4c482c0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 10 client.18105 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 10 client.18105 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 20 client.18105 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c61230 0x55f0a4c63650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 4 client.18105 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.553+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0a4bb8cf0 con 0x55f0a4c61230 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.554+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c61230 0x55f0a4c63650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.554+0000 7f83e0ff9640 10 client.18105 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.554+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c61230 0x55f0a4c63650 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5426543bf068d4f server_cookie=b0a19bc1c16e0e73 in_seq=0 out_seq=0 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f83e0ff9640 1 -- 172.21.15.122:0/3355418486 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0a4bb8cf0 con 0x55f0a4c61230 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f83e0ff9640 10 client.18105 handle_command_reply: tid=0 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 client.18105 shutdown 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 msgr2=0x55f0a4c4db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f83f000be90 tx=0x7f83f000a7b0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 shutdown_connections 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c61230 0x55f0a4c63650 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c4e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c4db60 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 --2- 172.21.15.122:0/3355418486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f83e0ff9640 0 client.18105 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 >> 172.21.15.122:0/3355418486 conn(0x55f0a4c420d0 msgr2=0x55f0a4c43e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 shutdown_connections 2024-04-02T13:43:26.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 1 -- 172.21.15.122:0/3355418486 wait complete. 2024-04-02T13:43:26.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.555+0000 7f8402412740 20 client.18105 trim_cache size 0 max 16384 2024-04-02T13:43:26.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:26 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/831490058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.568+0000 7f8402412740 1 Processor -- start 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 -- start start 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c479a0 0x55f0a4c5cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc010000 con 0x55f0a4c50ea0 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc011b40 con 0x55f0a4c479a0 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f8402412740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc0106d0 con 0x55f0a4c482c0 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39974/0 (socket says 172.21.15.122:39974) 2024-04-02T13:43:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 -- 172.21.15.122:0/4083530455 learned_addr learned my addr 172.21.15.122:0/4083530455 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c479a0 0x55f0a4c5cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 -- 172.21.15.122:0/4083530455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 msgr2=0x55f0a4c50290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 -- 172.21.15.122:0/4083530455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c479a0 msgr2=0x55f0a4c5cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c479a0 0x55f0a4c5cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 -- 172.21.15.122:0/4083530455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a4c47370 con 0x55f0a4c50ea0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f83e8015010 tx=0x7f83e8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47397dd682047329 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83e2ffd640 1 -- 172.21.15.122:0/4083530455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8019040 con 0x55f0a4c50ea0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83e2ffd640 1 -- 172.21.15.122:0/4083530455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8023030 con 0x55f0a4c50ea0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.569+0000 7f83e2ffd640 1 -- 172.21.15.122:0/4083530455 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e802d7f0 con 0x55f0a4c50ea0 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- 172.21.15.122:0/4083530455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 msgr2=0x55f0a4c507d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f83e8015010 tx=0x7f83e8015040 comp rx=0 tx=0).stop 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- 172.21.15.122:0/4083530455 shutdown_connections 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c507d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c482c0 0x55f0a4c50290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 --2- 172.21.15.122:0/4083530455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c479a0 0x55f0a4c5cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- 172.21.15.122:0/4083530455 >> 172.21.15.122:0/4083530455 conn(0x55f0a4c420d0 msgr2=0x55f0a4c48ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- 172.21.15.122:0/4083530455 shutdown_connections 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- 172.21.15.122:0/4083530455 wait complete. 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 -- start start 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.570+0000 7f8402412740 1 Processor -- start 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c59460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c52860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc011470 con 0x55f0a4c482c0 2024-04-02T13:43:26.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc010da0 con 0x55f0a4c50ea0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc00e000 con 0x55f0a4c479a0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f8402412740 10 client.0 _ll_register_callbacks cb 0x7f83f0009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39982/0 (socket says 172.21.15.122:39982) 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 -- 172.21.15.122:0/2311211601 learned_addr learned my addr 172.21.15.122:0/2311211601 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c59460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 -- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 msgr2=0x55f0a4c59460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c59460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 -- 172.21.15.122:0/2311211601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 msgr2=0x55f0a4c52860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c52860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 -- 172.21.15.122:0/2311211601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a4c51280 con 0x55f0a4c482c0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83fd5c6640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55f0a4c4f130 tx=0x7f83f000bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8632e2b9d4bccb51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.571+0000 7f83e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f002d070 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8023030 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f83f0003a30 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f83f0039050 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 10 client.18111 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55f0a4c582b0 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8023030 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f83f00040b0 con 0x55f0a4c482c0 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 10 client.18111 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 10 client.18111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 20 client.18111 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c58a30 0x55f0a4cc97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 4 client.18111 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:26.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0a4bb3040 con 0x55f0a4c58a30 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.572+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c58a30 0x55f0a4cc97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f83e0ff9640 10 client.18111 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f83fcdc5640 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c58a30 0x55f0a4cc97e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac01219e4c8d632 server_cookie=6da6bd398508d7d7 in_seq=0 out_seq=0 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f83e0ff9640 1 -- 172.21.15.122:0/2311211601 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0a4bb3040 con 0x55f0a4c58a30 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f83e0ff9640 10 client.18111 handle_command_reply: tid=0 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 client.18111 shutdown 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 msgr2=0x55f0a4c52320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x55f0a4c4f130 tx=0x7f83f000bb20 comp rx=0 tx=0).stop 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 shutdown_connections 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f0a4c58a30 0x55f0a4cc97e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0a4c50ea0 0x55f0a4c52860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0a4c482c0 0x55f0a4c52320 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 --2- 172.21.15.122:0/2311211601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0a4c479a0 0x55f0a4c59460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 >> 172.21.15.122:0/2311211601 conn(0x55f0a4c420d0 msgr2=0x55f0a4c4c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f83e0ff9640 0 client.18111 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 shutdown_connections 2024-04-02T13:43:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 1 -- 172.21.15.122:0/2311211601 wait complete. 2024-04-02T13:43:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.573+0000 7f8402412740 20 client.18111 trim_cache size 0 max 16384 2024-04-02T13:43:26.584 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 msgr2=0x7f83e40a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 0x7f83e40a63f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f83f40be2d0 tx=0x7f83f404bbd0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 msgr2=0x7f83f8110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009850 tx=0x7f83f40045e0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 shutdown_connections 2024-04-02T13:43:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83e40a3f30 0x7f83e40a63f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83f81067d0 0x7f83f8110310 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f83f8101180 0x7f83f810fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 --2- 172.21.15.122:0/213127295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f83f81007c0 0x7f83f8115d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 >> 172.21.15.122:0/213127295 conn(0x7f83f80fc2d0 msgr2=0x7f83f80fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 shutdown_connections 2024-04-02T13:43:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.574+0000 7f83fe5c8640 1 -- 172.21.15.122:0/213127295 wait complete. 2024-04-02T13:43:26.590 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:43:26.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- 172.21.15.122:0/281658004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 msgr2=0x7f9918100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 --2- 172.21.15.122:0/281658004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f9918100e30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f9914009930 tx=0x7f991402bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- 172.21.15.122:0/281658004 shutdown_connections 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 --2- 172.21.15.122:0/281658004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9918101370 0x7f991810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 --2- 172.21.15.122:0/281658004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f9918100e30 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 --2- 172.21.15.122:0/281658004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f99181069c0 0x7f9918106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- 172.21.15.122:0/281658004 >> 172.21.15.122:0/281658004 conn(0x7f9918076d80 msgr2=0x7f9918077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- 172.21.15.122:0/281658004 shutdown_connections 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- 172.21.15.122:0/281658004 wait complete. 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 Processor -- start 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.691+0000 7f991feb1640 1 -- start start 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 0x7f9918198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 0x7f991819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9918110dd0 con 0x7f99181009b0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9918110c50 con 0x7f9918101370 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9918110f50 con 0x7f99181069c0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39992/0 (socket says 172.21.15.122:39992) 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 -- 172.21.15.122:0/2802845699 learned_addr learned my addr 172.21.15.122:0/2802845699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991e44e640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 0x7f991819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991d44c640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 0x7f9918198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 -- 172.21.15.122:0/2802845699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 msgr2=0x7f991819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 0x7f991819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 -- 172.21.15.122:0/2802845699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 msgr2=0x7f9918198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 0x7f9918198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 -- 172.21.15.122:0/2802845699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991819d3b0 con 0x7f99181009b0 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991d44c640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 0x7f9918198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:26.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991e44e640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 0x7f991819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991dc4d640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f990800d950 tx=0x7f990800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f990effd640 1 -- 172.21.15.122:0/2802845699 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908017070 con 0x7f99181009b0 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f990effd640 1 -- 172.21.15.122:0/2802845699 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908009c30 con 0x7f99181009b0 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991819d6a0 con 0x7f99181009b0 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f990effd640 1 -- 172.21.15.122:0/2802845699 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990800f6b0 con 0x7f99181009b0 2024-04-02T13:43:26.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.692+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99181a4f30 con 0x7f99181009b0 2024-04-02T13:43:26.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.695+0000 7f990effd640 1 -- 172.21.15.122:0/2802845699 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f99080109d0 con 0x7f99181009b0 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.695+0000 7f9922b71740 1 Processor -- start 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 -- start start 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e1ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56203e1537b0 con 0x56203e1efb30 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56203e0f2570 con 0x56203e1ef180 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f9922b71740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56203e14f760 con 0x56203e1fd0b0 2024-04-02T13:43:26.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f990effd640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 0x7f99040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40000/0 (socket says 172.21.15.122:40000) 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 -- 172.21.15.122:0/135687896 learned_addr learned my addr 172.21.15.122:0/135687896 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991e44e640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 0x7f99040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991e44e640 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e1ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991d44c640 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f990effd640 1 -- 172.21.15.122:0/2802845699 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f99080d0850 con 0x7f99181009b0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 -- 172.21.15.122:0/135687896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 msgr2=0x56203e1ff4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1ff4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 -- 172.21.15.122:0/135687896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 msgr2=0x56203e1ef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e1ef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.696+0000 7f991dc4d640 1 -- 172.21.15.122:0/135687896 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203de673f0 con 0x56203e1efb30 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f991e44e640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 0x7f99040a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9910005c40 tx=0x7f99100098e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f991dc4d640 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f99080cdb20 tx=0x7f9908000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41cce65adf44f5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f9903fff640 1 -- 172.21.15.122:0/135687896 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9908020050 con 0x56203e1efb30 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f9903fff640 1 -- 172.21.15.122:0/135687896 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908009de0 con 0x56203e1efb30 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f9903fff640 1 -- 172.21.15.122:0/135687896 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990809b850 con 0x56203e1efb30 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.697+0000 7f9922b71740 1 -- 172.21.15.122:0/135687896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 msgr2=0x56203e1fcb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f99080cdb20 tx=0x7f9908000bc0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- 172.21.15.122:0/135687896 shutdown_connections 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1ff4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fcb70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- 172.21.15.122:0/135687896 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e1ef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- 172.21.15.122:0/135687896 >> 172.21.15.122:0/135687896 conn(0x56203e1e9ed0 msgr2=0x56203e1ea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- 172.21.15.122:0/135687896 shutdown_connections 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- 172.21.15.122:0/135687896 wait complete. 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- start start 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 Processor -- start 2024-04-02T13:43:26.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e1fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56203e1ffe10 con 0x56203e1efb30 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56203e10ee80 con 0x56203e1fd0b0 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.698+0000 7f9922b71740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56203e1fff90 con 0x56203e1ef180 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f9922b71740 10 client.0 _ll_register_callbacks cb 0x56203e14aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40012/0 (socket says 172.21.15.122:40012) 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 -- 172.21.15.122:0/1710624600 learned_addr learned my addr 172.21.15.122:0/1710624600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991e44e640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e1fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 -- 172.21.15.122:0/1710624600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 msgr2=0x56203e1fcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 -- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 msgr2=0x56203e1fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e1fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 -- 172.21.15.122:0/1710624600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203df317d0 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991e44e640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e1fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f991d44c640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9914036040 tx=0x7f9914005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8883de4470dda4e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f99017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140044d0 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f9914005030 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9908009de0 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f991404e050 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f9922b71740 10 client.18123 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56203e20bf60 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.699+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9908009de0 con 0x56203e1efb30 2024-04-02T13:43:26.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f99140035f0 con 0x56203e1efb30 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 10 client.18123 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 20 client.18123 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e20d420 0x56203e20f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 4 client.18123 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56203e160cf0 con 0x56203e20d420 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e20d420 0x56203e20f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f99017fa640 10 client.18123 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.700+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e20d420 0x56203e20f8a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcec803ce2340c1 server_cookie=168acd1f8b16c27 in_seq=0 out_seq=0 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f99017fa640 1 -- 172.21.15.122:0/1710624600 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56203e160cf0 con 0x56203e20d420 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f99017fa640 10 client.18123 handle_command_reply: tid=0 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 client.18123 shutdown 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 msgr2=0x56203e1f9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f9914036040 tx=0x7f9914005d50 comp rx=0 tx=0).stop 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 shutdown_connections 2024-04-02T13:43:26.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e20d420 0x56203e20f8a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e1fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f99017fa640 0 client.18123 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1f9ed0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 --2- 172.21.15.122:0/1710624600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 >> 172.21.15.122:0/1710624600 conn(0x56203e1e9ed0 msgr2=0x56203e1ebc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 shutdown_connections 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 1 -- 172.21.15.122:0/1710624600 wait complete. 2024-04-02T13:43:26.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.701+0000 7f9922b71740 20 client.18123 trim_cache size 0 max 16384 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.718+0000 7f9922b71740 1 Processor -- start 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.718+0000 7f9922b71740 1 -- start start 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e208d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ace80 con 0x56203e1efb30 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ae9c0 con 0x56203e1ef180 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f9922b71740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ad550 con 0x56203e1fd0b0 2024-04-02T13:43:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40024/0 (socket says 172.21.15.122:40024) 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 -- 172.21.15.122:0/2657085690 learned_addr learned my addr 172.21.15.122:0/2657085690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991e44e640 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991dc4d640 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e208d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 -- 172.21.15.122:0/2657085690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 msgr2=0x56203e1fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1fc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 -- 172.21.15.122:0/2657085690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 msgr2=0x56203e208d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e208d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 -- 172.21.15.122:0/2657085690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56203e1ef8d0 con 0x56203e1efb30 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f991d44c640 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f991402c310 tx=0x7f9914009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e320632d3aafc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f99037fe640 1 -- 172.21.15.122:0/2657085690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914003be0 con 0x56203e1efb30 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f99037fe640 1 -- 172.21.15.122:0/2657085690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9914003d80 con 0x56203e1efb30 2024-04-02T13:43:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.719+0000 7f99037fe640 1 -- 172.21.15.122:0/2657085690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140046c0 con 0x56203e1efb30 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- 172.21.15.122:0/2657085690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 msgr2=0x56203e20f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f991402c310 tx=0x7f9914009780 comp rx=0 tx=0).stop 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- 172.21.15.122:0/2657085690 shutdown_connections 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1fd0b0 0x56203e1fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e20f810 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 --2- 172.21.15.122:0/2657085690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1ef180 0x56203e208d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- 172.21.15.122:0/2657085690 >> 172.21.15.122:0/2657085690 conn(0x56203e1e9ed0 msgr2=0x56203e1f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- 172.21.15.122:0/2657085690 shutdown_connections 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- 172.21.15.122:0/2657085690 wait complete. 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 -- start start 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.720+0000 7f9922b71740 1 Processor -- start 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e204250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99181ae2f0 con 0x56203e1efb30 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99181adc20 con 0x56203e1fd0b0 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99181aae80 con 0x56203e1ef180 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f9922b71740 10 client.0 _ll_register_callbacks cb 0x7f99100093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40030/0 (socket says 172.21.15.122:40030) 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 -- 172.21.15.122:0/1198687466 learned_addr learned my addr 172.21.15.122:0/1198687466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:26.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 -- 172.21.15.122:0/1198687466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 msgr2=0x56203e1fda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991e44e640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e204250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 -- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 msgr2=0x56203e204250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e204250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 -- 172.21.15.122:0/1198687466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56203e204790 con 0x56203e1efb30 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991e44e640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e204250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f991d44c640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f991402c820 tx=0x7f9914000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1918c94af316ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.721+0000 7f99017fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991405f060 con 0x56203e1efb30 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f9914005450 con 0x56203e1efb30 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56203e15b040 con 0x56203e1efb30 2024-04-02T13:43:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9914066050 con 0x56203e1efb30 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 10 client.18129 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56203e205a90 con 0x56203e1efb30 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56203e15b040 con 0x56203e1efb30 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9914004c30 con 0x56203e1efb30 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 10 client.18129 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 20 client.18129 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e273d80 0x56203e2761a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 4 client.18129 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56203e0d86b0 con 0x56203e273d80 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.722+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e273d80 0x56203e2761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f99017fa640 10 client.18129 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f991dc4d640 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e273d80 0x56203e2761a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f6fc562485f343 server_cookie=91061d290cb2c927 in_seq=0 out_seq=0 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f99017fa640 1 -- 172.21.15.122:0/1198687466 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56203e0d86b0 con 0x56203e273d80 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f99017fa640 10 client.18129 handle_command_reply: tid=0 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 client.18129 shutdown 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 msgr2=0x56203e1fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f991402c820 tx=0x7f9914000bc0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 shutdown_connections 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56203e273d80 0x56203e2761a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56203e1fd0b0 0x56203e204250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56203e1efb30 0x56203e1fdfd0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f99017fa640 0 client.18129 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 --2- 172.21.15.122:0/1198687466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56203e1ef180 0x56203e1fda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 >> 172.21.15.122:0/1198687466 conn(0x56203e1e9ed0 msgr2=0x56203e1f8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 shutdown_connections 2024-04-02T13:43:26.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 1 -- 172.21.15.122:0/1198687466 wait complete. 2024-04-02T13:43:26.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.723+0000 7f9922b71740 20 client.18129 trim_cache size 0 max 16384 2024-04-02T13:43:26.735 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 msgr2=0x7f99040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 0x7f99040a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9910005c40 tx=0x7f99100098e0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 msgr2=0x7f99181983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f990800d950 tx=0x7f990800de10 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 shutdown_connections 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f99040a3d20 0x7f99040a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99181069c0 0x7f991819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9918101370 0x7f9918198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 --2- 172.21.15.122:0/2802845699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99181009b0 0x7f99181983c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 >> 172.21.15.122:0/2802845699 conn(0x7f9918076d80 msgr2=0x7f991806a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 shutdown_connections 2024-04-02T13:43:26.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:26.724+0000 7f991feb1640 1 -- 172.21.15.122:0/2802845699 wait complete. 2024-04-02T13:43:26.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:26 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/831490058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:26 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/831490058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:26.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266783 3.13 MB/sec execute 287 sec latency 3506.337 ms 2024-04-02T13:43:27.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:27 smithi122 ceph-mon[32114]: pgmap v255: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:43:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:27 smithi187 ceph-mon[36064]: pgmap v255: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:43:27.739 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:43:27.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:43:27.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:43:27.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:27 smithi188 ceph-mon[36117]: pgmap v255: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 116 op/s 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 -- 172.21.15.122:0/4220572704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 msgr2=0x7fe09810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 --2- 172.21.15.122:0/4220572704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe09810e960 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fe090009cd0 tx=0x7fe09002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 -- 172.21.15.122:0/4220572704 shutdown_connections 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 --2- 172.21.15.122:0/4220572704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe09810e960 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 --2- 172.21.15.122:0/4220572704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 0x7fe098069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 --2- 172.21.15.122:0/4220572704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 0x7fe098107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 -- 172.21.15.122:0/4220572704 >> 172.21.15.122:0/4220572704 conn(0x7fe098077b00 msgr2=0x7fe098077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 -- 172.21.15.122:0/4220572704 shutdown_connections 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.852+0000 7fe09fce4640 1 -- 172.21.15.122:0/4220572704 wait complete. 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 Processor -- start 2024-04-02T13:43:27.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 -- start start 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 0x7fe098198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 0x7fe09819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111ea0 con 0x7fe0980696c0 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe098111d20 con 0x7fe098107b50 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09fce4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe098112020 con 0x7fe098068d00 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09d27f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09d27f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40040/0 (socket says 172.21.15.122:40040) 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09d27f640 1 -- 172.21.15.122:0/3311041959 learned_addr learned my addr 172.21.15.122:0/3311041959 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09da80640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 0x7fe098198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.853+0000 7fe09e281640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 0x7fe09819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 -- 172.21.15.122:0/3311041959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 msgr2=0x7fe098198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 0x7fe098198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 -- 172.21.15.122:0/3311041959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 msgr2=0x7fe09819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 0x7fe09819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 -- 172.21.15.122:0/3311041959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09819d2c0 con 0x7fe0980696c0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09d27f640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe08800d950 tx=0x7fe08800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe08effd640 1 -- 172.21.15.122:0/3311041959 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088017070 con 0x7fe0980696c0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09819d5b0 con 0x7fe0980696c0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe08effd640 1 -- 172.21.15.122:0/3311041959 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe088009c30 con 0x7fe0980696c0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe08effd640 1 -- 172.21.15.122:0/3311041959 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08800f6b0 con 0x7fe0980696c0 2024-04-02T13:43:27.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.854+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981a4e40 con 0x7fe0980696c0 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.856+0000 7fe08effd640 1 -- 172.21.15.122:0/3311041959 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe0880109d0 con 0x7fe0980696c0 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 Processor -- start 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 -- start start 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe08effd640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 0x7fe0840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0378280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 0x55d7e037abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe09e281640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 0x7fe0840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7e02d3770 con 0x55d7e036f2c0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d7e027b960 con 0x55d7e036fbe0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7e02cf8c0 con 0x55d7e03787c0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe08effd640 1 -- 172.21.15.122:0/3311041959 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fe0880d0750 con 0x7fe0980696c0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe09da80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe09da80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40050/0 (socket says 172.21.15.122:40050) 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe09da80640 1 -- 172.21.15.122:0/1074388248 learned_addr learned my addr 172.21.15.122:0/1074388248 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.857+0000 7fe09d27f640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0378280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09e281640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 0x55d7e037abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 -- 172.21.15.122:0/1074388248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 msgr2=0x55d7e037abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 0x55d7e037abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 -- 172.21.15.122:0/1074388248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 msgr2=0x55d7e0378280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0378280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 -- 172.21.15.122:0/1074388248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e00af7e0 con 0x55d7e036f2c0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09e281640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 0x55d7e037abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09e281640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 0x7fe0840a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe090005cc0 tx=0x7fe090005f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe09da80640 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe09400bd10 tx=0x7fe0940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2025441e5fa27d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe083fff640 1 -- 172.21.15.122:0/1074388248 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094004030 con 0x55d7e036f2c0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe083fff640 1 -- 172.21.15.122:0/1074388248 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0940041d0 con 0x55d7e036f2c0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.858+0000 7fe083fff640 1 -- 172.21.15.122:0/1074388248 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094011820 con 0x55d7e036f2c0 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- 172.21.15.122:0/1074388248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 msgr2=0x55d7e036f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe09400bd10 tx=0x7fe0940107a0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- 172.21.15.122:0/1074388248 shutdown_connections 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e03787c0 0x55d7e037abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0378280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/1074388248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e036f6a0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- 172.21.15.122:0/1074388248 >> 172.21.15.122:0/1074388248 conn(0x55d7e03699f0 msgr2=0x55d7e0369dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- 172.21.15.122:0/1074388248 shutdown_connections 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- 172.21.15.122:0/1074388248 wait complete. 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 -- start start 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.859+0000 7fe0a29a7740 1 Processor -- start 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0375740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0375c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7e037b5b0 con 0x55d7e03787c0 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d7e028ed20 con 0x55d7e036fbe0 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7e037b730 con 0x55d7e036f2c0 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0a29a7740 10 client.0 _ll_register_callbacks cb 0x55d7e025ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40066/0 (socket says 172.21.15.122:40066) 2024-04-02T13:43:27.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 -- 172.21.15.122:0/281802159 learned_addr learned my addr 172.21.15.122:0/281802159 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09da80640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0375740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09d27f640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0375c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 -- 172.21.15.122:0/281802159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 msgr2=0x55d7e0375740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0375740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 -- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 msgr2=0x55d7e0375c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0375c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 -- 172.21.15.122:0/281802159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e010ff70 con 0x55d7e03787c0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09da80640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0375740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09d27f640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0375c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe09e281640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe090009cd0 tx=0x7fe09002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47e8e2cef59ecef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0817fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe090050060 con 0x55d7e03787c0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.860+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fe090004240 con 0x55d7e03787c0 2024-04-02T13:43:27.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0940041d0 con 0x55d7e03787c0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe090057050 con 0x55d7e03787c0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 10 client.18141 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7dffa7170 con 0x55d7e03787c0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0940041d0 con 0x55d7e03787c0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe09003f040 con 0x55d7e03787c0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 10 client.18141 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 20 client.18141 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e0388a70 0x55d7e038aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 4 client.18141 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7e02e0cb0 con 0x55d7e0388a70 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.861+0000 7fe09da80640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e0388a70 0x55d7e038aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.862+0000 7fe0817fa640 10 client.18141 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.862+0000 7fe09da80640 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e0388a70 0x55d7e038aec0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f074fc095eeed66 server_cookie=c22cd1280fff453a in_seq=0 out_seq=0 2024-04-02T13:43:27.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.862+0000 7fe0817fa640 1 -- 172.21.15.122:0/281802159 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7e02e0cb0 con 0x55d7e0388a70 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.862+0000 7fe0817fa640 10 client.18141 handle_command_reply: tid=0 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.862+0000 7fe0a29a7740 1 client.18141 shutdown 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 msgr2=0x55d7e03761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe090009cd0 tx=0x7fe09002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 shutdown_connections 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e0388a70 0x55d7e038aec0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e03787c0 0x55d7e03761c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e036fbe0 0x55d7e0375c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/281802159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0375740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0817fa640 0 client.18141 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 >> 172.21.15.122:0/281802159 conn(0x55d7e03699f0 msgr2=0x55d7e036b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 shutdown_connections 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 1 -- 172.21.15.122:0/281802159 wait complete. 2024-04-02T13:43:27.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.863+0000 7fe0a29a7740 20 client.18141 trim_cache size 0 max 16384 2024-04-02T13:43:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 Processor -- start 2024-04-02T13:43:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 -- start start 2024-04-02T13:43:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0377e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e0390790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981af030 con 0x55d7e036fbe0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b0b70 con 0x55d7e03787c0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981af700 con 0x55d7e036f2c0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe09d27f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe09d27f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40072/0 (socket says 172.21.15.122:40072) 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.880+0000 7fe09d27f640 1 -- 172.21.15.122:0/2986818322 learned_addr learned my addr 172.21.15.122:0/2986818322 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09e281640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e0390790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09da80640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0377e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 -- 172.21.15.122:0/2986818322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 msgr2=0x55d7e0377e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0377e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 -- 172.21.15.122:0/2986818322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 msgr2=0x55d7e0390790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e0390790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 -- 172.21.15.122:0/2986818322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7e037abb0 con 0x55d7e036fbe0 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09e281640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e0390790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe08809c0b0 tx=0x7fe088009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8233f61c3b3f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe0837fe640 1 -- 172.21.15.122:0/2986818322 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880b5070 con 0x55d7e036fbe0 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe0837fe640 1 -- 172.21.15.122:0/2986818322 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880cdd30 con 0x55d7e036fbe0 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe0837fe640 1 -- 172.21.15.122:0/2986818322 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088020040 con 0x55d7e036fbe0 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2986818322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 msgr2=0x55d7e03844a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.881+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe08809c0b0 tx=0x7fe088009f20 comp rx=0 tx=0).stop 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2986818322 shutdown_connections 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e0390790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036fbe0 0x55d7e03844a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2986818322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036f2c0 0x55d7e0377e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2986818322 >> 172.21.15.122:0/2986818322 conn(0x55d7e03699f0 msgr2=0x55d7e036b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2986818322 shutdown_connections 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2986818322 wait complete. 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- start start 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 Processor -- start 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 0x55d7e0380ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e037d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981b04a0 con 0x55d7e036f2c0 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981afdd0 con 0x55d7e03787c0 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0981ad030 con 0x55d7e036fbe0 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe0a29a7740 10 client.0 _ll_register_callbacks cb 0x7fe090009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe09da80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe09da80640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40084/0 (socket says 172.21.15.122:40084) 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe09da80640 1 -- 172.21.15.122:0/2622463996 learned_addr learned my addr 172.21.15.122:0/2622463996 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:27.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.882+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 0x55d7e0380ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09e281640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e037d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 -- 172.21.15.122:0/2622463996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 msgr2=0x55d7e0380ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 0x55d7e0380ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 -- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 msgr2=0x55d7e037d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e037d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 -- 172.21.15.122:0/2622463996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7e037dfc0 con 0x55d7e036f2c0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 0x55d7e0380ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe09da80640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe094010ca0 tx=0x7fe09400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7be047f39783fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe09402b020 con 0x55d7e036f2c0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fe094027620 con 0x55d7e036f2c0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0880cdd30 con 0x55d7e036f2c0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe094036050 con 0x55d7e036f2c0 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 10 client.18147 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:27.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55d7e03ef970 con 0x55d7e036f2c0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0880cdd30 con 0x55d7e036f2c0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe0940038f0 con 0x55d7e036f2c0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 10 client.18147 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 20 client.18147 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e03efe10 0x55d7e03f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 4 client.18147 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.883+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d7e02db000 con 0x55d7e03efe10 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.884+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e03efe10 0x55d7e03f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.884+0000 7fe09d27f640 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e03efe10 0x55d7e03f2230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42a46bb466a588fc server_cookie=89599e7f900567c9 in_seq=0 out_seq=0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.884+0000 7fe0817fa640 10 client.18147 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.885+0000 7fe0817fa640 1 -- 172.21.15.122:0/2622463996 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d7e02db000 con 0x55d7e03efe10 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.885+0000 7fe0817fa640 10 client.18147 handle_command_reply: tid=0 2024-04-02T13:43:27.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.885+0000 7fe0a29a7740 1 client.18147 shutdown 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.885+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 msgr2=0x55d7e03809b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.885+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe094010ca0 tx=0x7fe09400be00 comp rx=0 tx=0).stop 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 shutdown_connections 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7e03efe10 0x55d7e03f2230 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7e03787c0 0x55d7e037d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0817fa640 0 client.18147 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7e036fbe0 0x55d7e0380ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 --2- 172.21.15.122:0/2622463996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7e036f2c0 0x55d7e03809b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 >> 172.21.15.122:0/2622463996 conn(0x55d7e03699f0 msgr2=0x55d7e036ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 shutdown_connections 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 1 -- 172.21.15.122:0/2622463996 wait complete. 2024-04-02T13:43:27.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe0a29a7740 20 client.18147 trim_cache size 0 max 16384 2024-04-02T13:43:27.896 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:43:27.896 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:43:27.896 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "4031c4dc-98b2-4e3c-901c-f21c689943f2", 2024-04-02T13:43:27.896 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:43:27.896 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 msgr2=0x7fe0840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 0x7fe0840a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe090005cc0 tx=0x7fe090005f10 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 msgr2=0x7fe098198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe08800d950 tx=0x7fe08800de10 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 shutdown_connections 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe0840a3d20 0x7fe0840a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe098107b50 0x7fe09819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0980696c0 0x7fe098198860 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 --2- 172.21.15.122:0/3311041959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe098068d00 0x7fe098198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:27.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 >> 172.21.15.122:0/3311041959 conn(0x7fe098077b00 msgr2=0x7fe0980fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:27.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 shutdown_connections 2024-04-02T13:43:27.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:27.886+0000 7fe09fce4640 1 -- 172.21.15.122:0/3311041959 wait complete. 2024-04-02T13:43:27.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:27.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:27.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 266841 3.12 MB/sec execute 288 sec latency 4452.236 ms 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3513895334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec101330 msgr2=0x7ffbec10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3513895334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec101330 0x7ffbec10d840 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009cd0 tx=0x7ffbe402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3513895334 shutdown_connections 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3513895334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec101330 0x7ffbec10d840 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3513895334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3513895334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec106980 0x7ffbec106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3513895334 >> 172.21.15.122:0/3513895334 conn(0x7ffbec077b00 msgr2=0x7ffbec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3513895334 shutdown_connections 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.020+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3513895334 wait complete. 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 Processor -- start 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 -- start start 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec101330 0x7ffbec198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec110d90 con 0x7ffbec106980 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec110c10 con 0x7ffbec100970 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbf10ed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec110f10 con 0x7ffbec101330 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbeb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40092/0 (socket says 172.21.15.122:40092) 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbeb59e640 1 -- 172.21.15.122:0/3067077787 learned_addr learned my addr 172.21.15.122:0/3067077787 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.021+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec101330 0x7ffbec198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 -- 172.21.15.122:0/3067077787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec101330 msgr2=0x7ffbec198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec101330 0x7ffbec198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 -- 172.21.15.122:0/3067077787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 msgr2=0x7ffbec1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 -- 172.21.15.122:0/3067077787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbec19d3b0 con 0x7ffbec106980 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbead9d640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbeb59e640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40348f0 tx=0x7ffbe4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbe8d99640 1 -- 172.21.15.122:0/3067077787 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4041020 con 0x7ffbec106980 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbe8d99640 1 -- 172.21.15.122:0/3067077787 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbe4032e80 con 0x7ffbec106980 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbec19d640 con 0x7ffbec106980 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbe8d99640 1 -- 172.21.15.122:0/3067077787 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe4039040 con 0x7ffbec106980 2024-04-02T13:43:28.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.022+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbec1a4f30 con 0x7ffbec106980 2024-04-02T13:43:28.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.024+0000 7ffbe8d99640 1 -- 172.21.15.122:0/3067077787 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ffbe4044020 con 0x7ffbec106980 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 Processor -- start 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 -- start start 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68751f0 0x557fa687d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa68801c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe8d99640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 0x7ffbd80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557fa67d8830 con 0x557fa687ddd0 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557fa6780820 con 0x557fa68748d0 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557fa67d47e0 con 0x557fa68751f0 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47666/0 (socket says 172.21.15.122:47666) 2024-04-02T13:43:28.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe3fff640 1 -- 172.21.15.122:0/1181579462 learned_addr learned my addr 172.21.15.122:0/1181579462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa68801c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbead9d640 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68751f0 0x557fa687d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbe8d99640 1 -- 172.21.15.122:0/3067077787 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7ffbe40f2690 con 0x7ffbec106980 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.025+0000 7ffbeb59e640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 0x7ffbd80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 -- 172.21.15.122:0/1181579462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68751f0 msgr2=0x557fa687d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68751f0 0x557fa687d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 -- 172.21.15.122:0/1181579462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 msgr2=0x557fa68801c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa68801c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 -- 172.21.15.122:0/1181579462 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa65b67d0 con 0x557fa68748d0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbeb59e640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 0x7ffbd80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40be2e0 tx=0x7ffbe4003bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40058c0 tx=0x7ffbd4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7574cfbd3a13c601 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe17fa640 1 -- 172.21.15.122:0/1181579462 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4004030 con 0x557fa68748d0 2024-04-02T13:43:28.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe17fa640 1 -- 172.21.15.122:0/1181579462 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd4005500 con 0x557fa68748d0 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.026+0000 7ffbe17fa640 1 -- 172.21.15.122:0/1181579462 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd4004db0 con 0x557fa68748d0 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 -- 172.21.15.122:0/1181579462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 msgr2=0x557fa6874cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40058c0 tx=0x7ffbd4005850 comp rx=0 tx=0).stop 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 -- 172.21.15.122:0/1181579462 shutdown_connections 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa68801c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68751f0 0x557fa687d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/1181579462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa6874cb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 -- 172.21.15.122:0/1181579462 >> 172.21.15.122:0/1181579462 conn(0x557fa686f000 msgr2=0x557fa686f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 -- 172.21.15.122:0/1181579462 shutdown_connections 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.027+0000 7ffbf3da8740 1 -- 172.21.15.122:0/1181579462 wait complete. 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 -- start start 2024-04-02T13:43:28.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 Processor -- start 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa687d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa687afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557fa6880bc0 con 0x557fa68751f0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557fa6793d30 con 0x557fa68748d0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557fa6880d40 con 0x557fa687ddd0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbf3da8740 10 client.0 _ll_register_callbacks cb 0x557fa6793720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa687d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40110/0 (socket says 172.21.15.122:40110) 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbead9d640 1 -- 172.21.15.122:0/3805095094 learned_addr learned my addr 172.21.15.122:0/3805095094 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:28.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.028+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa687afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 -- 172.21.15.122:0/3805095094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 msgr2=0x557fa687afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa687afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 -- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 msgr2=0x557fa687d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa687d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 -- 172.21.15.122:0/3805095094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa64ec3f0 con 0x557fa68751f0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa687d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbead9d640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc012d20 tx=0x7ffbdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04209a43e087764 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc012110 con 0x557fa68751f0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7ffbdc004050 con 0x557fa68751f0 2024-04-02T13:43:28.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd4005500 con 0x557fa68751f0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc025050 con 0x557fa68751f0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbf3da8740 10 client.18159 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557fa64ac0f0 con 0x557fa68751f0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd4005500 con 0x557fa68751f0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.029+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ffbdc010430 con 0x557fa68751f0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 10 client.18159 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 20 client.18159 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa688deb0 0x557fa68902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 4 client.18159 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fa67e5d70 con 0x557fa688deb0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa688deb0 0x557fa68902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa688deb0 0x557fa68902f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c041f50d54272d server_cookie=bae4af828bed501e in_seq=0 out_seq=0 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.030+0000 7ffbd27fc640 10 client.18159 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbd27fc640 1 -- 172.21.15.122:0/3805095094 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fa67e5d70 con 0x557fa688deb0 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbd27fc640 10 client.18159 handle_command_reply: tid=0 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 client.18159 shutdown 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 msgr2=0x557fa687aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc012d20 tx=0x7ffbdc00f790 comp rx=0 tx=0).stop 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 shutdown_connections 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa688deb0 0x557fa68902f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa687afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68751f0 0x557fa687aa70 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbd27fc640 0 client.18159 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:28.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/3805095094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68748d0 0x557fa687d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 >> 172.21.15.122:0/3805095094 conn(0x557fa686f000 msgr2=0x557fa6870d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 shutdown_connections 2024-04-02T13:43:28.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 1 -- 172.21.15.122:0/3805095094 wait complete. 2024-04-02T13:43:28.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.031+0000 7ffbf3da8740 20 client.18159 trim_cache size 0 max 16384 2024-04-02T13:43:28.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 Processor -- start 2024-04-02T13:43:28.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 -- start start 2024-04-02T13:43:28.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa687d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1af0d0 con 0x557fa687ddd0 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1b0c10 con 0x557fa68751f0 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.051+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1af7a0 con 0x557fa68748d0 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47686/0 (socket says 172.21.15.122:47686) 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46166/0 (socket says 172.21.15.122:46166) 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 -- 172.21.15.122:0/2518894596 learned_addr learned my addr 172.21.15.122:0/2518894596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:28.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa687d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 -- 172.21.15.122:0/2518894596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 msgr2=0x557fa6889ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 -- 172.21.15.122:0/2518894596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 msgr2=0x557fa687d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa687d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 -- 172.21.15.122:0/2518894596 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fa6875020 con 0x557fa68751f0 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbead9d640 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc010880 tx=0x7ffbdc012830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70587db5c1ea3418 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe0ff9640 1 -- 172.21.15.122:0/2518894596 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc02e070 con 0x557fa68751f0 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe0ff9640 1 -- 172.21.15.122:0/2518894596 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc005590 con 0x557fa68751f0 2024-04-02T13:43:28.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.052+0000 7ffbe0ff9640 1 -- 172.21.15.122:0/2518894596 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc003850 con 0x557fa68751f0 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 -- 172.21.15.122:0/2518894596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 msgr2=0x557fa686e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc010880 tx=0x7ffbdc012830 comp rx=0 tx=0).stop 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 -- 172.21.15.122:0/2518894596 shutdown_connections 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa687ddd0 0x557fa687d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa686e240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/2518894596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa68748d0 0x557fa6889ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 -- 172.21.15.122:0/2518894596 >> 172.21.15.122:0/2518894596 conn(0x557fa686f000 msgr2=0x557fa68799b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 -- 172.21.15.122:0/2518894596 shutdown_connections 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.053+0000 7ffbf3da8740 1 -- 172.21.15.122:0/2518894596 wait complete. 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 -- start start 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 Processor -- start 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa6877300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa6877840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1b0540 con 0x557fa68748d0 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1afe70 con 0x557fa68751f0 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffbec1ad0d0 con 0x557fa687ddd0 2024-04-02T13:43:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbf3da8740 10 client.0 _ll_register_callbacks cb 0x7ffbdc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40118/0 (socket says 172.21.15.122:40118) 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa6877300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbe3fff640 1 -- 172.21.15.122:0/589908202 learned_addr learned my addr 172.21.15.122:0/589908202 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa6877840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 -- 172.21.15.122:0/589908202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 msgr2=0x557fa6877840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa6877840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 -- 172.21.15.122:0/589908202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 msgr2=0x557fa68861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbead9d640 1 -- 172.21.15.122:0/589908202 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fa6877d80 con 0x557fa68751f0 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.054+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbead9d640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa6877300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x557fa6873d90 tx=0x7ffbdc00fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2348ec87041e1b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc02e040 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbdc005590 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7ffbdc010430 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffbdc005740 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbf3da8740 10 client.27257 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557fa68f4100 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc005590 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.055+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ffbdc003c10 con 0x557fa68751f0 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 10 client.27257 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 20 client.27257 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa68f4760 0x557fa68f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 4 client.27257 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557fa67e00c0 con 0x557fa68f4760 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa68f4760 0x557fa68f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbd27fc640 10 client.27257 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbe3fff640 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa68f4760 0x557fa68f6b50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cff2d779632cfd server_cookie=17eadc8898bdf324 in_seq=0 out_seq=0 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbd27fc640 1 -- 172.21.15.122:0/589908202 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557fa67e00c0 con 0x557fa68f4760 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbd27fc640 10 client.27257 handle_command_reply: tid=0 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 client.27257 shutdown 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 msgr2=0x557fa6877300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa6877300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x557fa6873d90 tx=0x7ffbdc00fd50 comp rx=0 tx=0).stop 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 shutdown_connections 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557fa68f4760 0x557fa68f6b50 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557fa687ddd0 0x557fa6877840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbd27fc640 0 client.27257 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557fa68751f0 0x557fa6877300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 --2- 172.21.15.122:0/589908202 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557fa68748d0 0x557fa68861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.056+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 >> 172.21.15.122:0/589908202 conn(0x557fa686f000 msgr2=0x557fa686e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 shutdown_connections 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf3da8740 1 -- 172.21.15.122:0/589908202 wait complete. 2024-04-02T13:43:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf3da8740 20 client.27257 trim_cache size 0 max 16384 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "4031c4dc-98b2-4e3c-901c-f21c689943f2": { 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "4031c4dc-98b2-4e3c-901c-f21c689943f2", 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:43:28.068 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:43:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 msgr2=0x7ffbd80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 0x7ffbd80a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40be2e0 tx=0x7ffbe4003bc0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 msgr2=0x7ffbec19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ffbe40348f0 tx=0x7ffbe4032c90 comp rx=0 tx=0).stop 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 shutdown_connections 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffbd80a3d20 0x7ffbd80a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbec106980 0x7ffbec19cc80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbec101330 0x7ffbec198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 --2- 172.21.15.122:0/3067077787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffbec100970 0x7ffbec1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 >> 172.21.15.122:0/3067077787 conn(0x7ffbec077b00 msgr2=0x7ffbec068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.057+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 shutdown_connections 2024-04-02T13:43:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:28.058+0000 7ffbf10ed640 1 -- 172.21.15.122:0/3067077787 wait complete. 2024-04-02T13:43:28.072 INFO:tasks.cephfs.filesystem:scrub status for tag:4031c4dc-98b2-4e3c-901c-f21c689943f2 - {'path': '/', 'tag': '4031c4dc-98b2-4e3c-901c-f21c689943f2', 'options': 'recursive,force'} 2024-04-02T13:43:28.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 268989 3.15 MB/sec execute 289 sec latency 17.838 ms 2024-04-02T13:43:29.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:43:29.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:43:29.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:29 smithi122 ceph-mon[32114]: pgmap v256: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 889 KiB/s wr, 99 op/s 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:29 smithi187 ceph-mon[36064]: pgmap v256: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 889 KiB/s wr, 99 op/s 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:43:29.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:29 smithi188 ceph-mon[36117]: pgmap v256: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 889 KiB/s wr, 99 op/s 2024-04-02T13:43:29.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 270394 3.17 MB/sec execute 290 sec latency 12.952 ms 2024-04-02T13:43:30.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 272590 3.20 MB/sec execute 291 sec latency 16.695 ms 2024-04-02T13:43:31.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:31 smithi122 ceph-mon[32114]: pgmap v257: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 866 KiB/s wr, 96 op/s 2024-04-02T13:43:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:31 smithi187 ceph-mon[36064]: pgmap v257: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 866 KiB/s wr, 96 op/s 2024-04-02T13:43:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:31 smithi188 ceph-mon[36117]: pgmap v257: 129 pgs: 129 active+clean; 188 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 866 KiB/s wr, 96 op/s 2024-04-02T13:43:31.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273818 3.21 MB/sec execute 292 sec latency 100.289 ms 2024-04-02T13:43:32.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 273818 3.20 MB/sec execute 293 sec latency 1100.396 ms 2024-04-02T13:43:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:33 smithi122 ceph-mon[32114]: pgmap v258: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T13:43:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:33 smithi187 ceph-mon[36064]: pgmap v258: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T13:43:33.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:33 smithi188 ceph-mon[36117]: pgmap v258: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 179 op/s 2024-04-02T13:43:33.965 INFO:tasks.workunit.client.0.smithi122.stdout: 1 274710 3.20 MB/sec execute 294 sec latency 1545.291 ms 2024-04-02T13:43:34.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 276969 3.24 MB/sec execute 295 sec latency 14.264 ms 2024-04-02T13:43:35.569 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:35 smithi188 ceph-mon[36117]: pgmap v259: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T13:43:35.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:35 smithi122 ceph-mon[32114]: pgmap v259: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T13:43:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:35 smithi187 ceph-mon[36064]: pgmap v259: 129 pgs: 129 active+clean; 194 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-04-02T13:43:35.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277416 3.23 MB/sec execute 296 sec latency 514.527 ms 2024-04-02T13:43:36.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277416 3.22 MB/sec execute 297 sec latency 1514.585 ms 2024-04-02T13:43:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: pgmap v260: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-04-02T13:43:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:37.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:37.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: pgmap v260: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:43:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: pgmap v260: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-04-02T13:43:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:37.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:43:37.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277416 3.21 MB/sec execute 298 sec latency 2514.657 ms 2024-04-02T13:43:38.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277484 3.20 MB/sec execute 299 sec latency 2956.496 ms 2024-04-02T13:43:39.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:39 smithi122 ceph-mon[32114]: pgmap v261: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:39 smithi187 ceph-mon[36064]: pgmap v261: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:39 smithi188 ceph-mon[36117]: pgmap v261: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:39.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277484 3.19 MB/sec execute 300 sec latency 1504.183 ms 2024-04-02T13:43:40.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277484 3.18 MB/sec execute 301 sec latency 2504.324 ms 2024-04-02T13:43:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:41 smithi122 ceph-mon[32114]: pgmap v262: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:41 smithi187 ceph-mon[36064]: pgmap v262: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:41 smithi188 ceph-mon[36117]: pgmap v262: 129 pgs: 129 active+clean; 198 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-04-02T13:43:41.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277484 3.17 MB/sec execute 302 sec latency 3504.450 ms 2024-04-02T13:43:42.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 277484 3.16 MB/sec execute 303 sec latency 4504.505 ms 2024-04-02T13:43:43.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:43 smithi187 ceph-mon[36064]: pgmap v263: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:43:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:43 smithi188 ceph-mon[36117]: pgmap v263: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:43:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:43 smithi122 ceph-mon[32114]: pgmap v263: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:43:43.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 278519 3.16 MB/sec execute 304 sec latency 4946.093 ms 2024-04-02T13:43:44.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 280630 3.20 MB/sec execute 305 sec latency 13.717 ms 2024-04-02T13:43:45.622 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:45 smithi122 ceph-mon[32114]: pgmap v264: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-04-02T13:43:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:45 smithi187 ceph-mon[36064]: pgmap v264: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-04-02T13:43:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:45 smithi188 ceph-mon[36117]: pgmap v264: 129 pgs: 129 active+clean; 199 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 636 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-04-02T13:43:45.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 280909 3.19 MB/sec execute 306 sec latency 726.967 ms 2024-04-02T13:43:46.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 280909 3.18 MB/sec execute 307 sec latency 1727.015 ms 2024-04-02T13:43:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:47 smithi187 ceph-mon[36064]: pgmap v265: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:43:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:47 smithi188 ceph-mon[36117]: pgmap v265: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:43:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:47 smithi122 ceph-mon[32114]: pgmap v265: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:43:47.966 INFO:tasks.workunit.client.0.smithi122.stdout: 1 280909 3.17 MB/sec execute 308 sec latency 2727.069 ms 2024-04-02T13:43:48.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281033 3.16 MB/sec execute 309 sec latency 3174.806 ms 2024-04-02T13:43:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:49 smithi187 ceph-mon[36064]: pgmap v266: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:49 smithi188 ceph-mon[36117]: pgmap v266: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:49 smithi122 ceph-mon[32114]: pgmap v266: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:49.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281033 3.15 MB/sec execute 310 sec latency 1440.933 ms 2024-04-02T13:43:50.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281033 3.14 MB/sec execute 311 sec latency 2440.998 ms 2024-04-02T13:43:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:51 smithi187 ceph-mon[36064]: pgmap v267: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:51 smithi188 ceph-mon[36117]: pgmap v267: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:51 smithi122 ceph-mon[32114]: pgmap v267: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 848 KiB/s wr, 72 op/s 2024-04-02T13:43:51.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281033 3.13 MB/sec execute 312 sec latency 3441.058 ms 2024-04-02T13:43:52.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 281033 3.12 MB/sec execute 313 sec latency 4441.092 ms 2024-04-02T13:43:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:53 smithi187 ceph-mon[36064]: pgmap v268: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T13:43:53.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:53 smithi188 ceph-mon[36117]: pgmap v268: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T13:43:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:53 smithi122 ceph-mon[32114]: pgmap v268: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-04-02T13:43:53.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 282068 3.12 MB/sec execute 314 sec latency 4881.202 ms 2024-04-02T13:43:54.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 284135 3.16 MB/sec execute 315 sec latency 14.989 ms 2024-04-02T13:43:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:55 smithi187 ceph-mon[36064]: pgmap v269: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-02T13:43:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:55 smithi188 ceph-mon[36117]: pgmap v269: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-02T13:43:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:55 smithi122 ceph-mon[32114]: pgmap v269: 129 pgs: 129 active+clean; 201 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-04-02T13:43:55.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 284575 3.15 MB/sec execute 316 sec latency 525.620 ms 2024-04-02T13:43:56.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 284575 3.14 MB/sec execute 317 sec latency 1525.656 ms 2024-04-02T13:43:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:57 smithi187 ceph-mon[36064]: pgmap v270: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-02T13:43:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:57 smithi188 ceph-mon[36117]: pgmap v270: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-02T13:43:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:57 smithi122 ceph-mon[32114]: pgmap v270: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-04-02T13:43:57.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 284575 3.13 MB/sec execute 318 sec latency 2525.701 ms 2024-04-02T13:43:58.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:43:58.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 -- 172.21.15.122:0/3153660284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8102450 msgr2=0x7f87d810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 --2- 172.21.15.122:0/3153660284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8102450 0x7f87d810e960 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f87d0009cd0 tx=0x7f87d002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 -- 172.21.15.122:0/3153660284 shutdown_connections 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 --2- 172.21.15.122:0/3153660284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8102450 0x7f87d810e960 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 --2- 172.21.15.122:0/3153660284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8101a90 0x7f87d8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 --2- 172.21.15.122:0/3153660284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8107aa0 0x7f87d8107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 -- 172.21.15.122:0/3153660284 >> 172.21.15.122:0/3153660284 conn(0x7f87d8078f00 msgr2=0x7f87d8079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 -- 172.21.15.122:0/3153660284 shutdown_connections 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.185+0000 7f87ddf38640 1 -- 172.21.15.122:0/3153660284 wait complete. 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 Processor -- start 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 -- start start 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8101a90 0x7f87d81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8102450 0x7f87d8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8111eb0 con 0x7f87d8107aa0 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8111d30 con 0x7f87d8101a90 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87ddf38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f87d8112030 con 0x7f87d8102450 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40216/0 (socket says 172.21.15.122:40216) 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87d7fff640 1 -- 172.21.15.122:0/1543785646 learned_addr learned my addr 172.21.15.122:0/1543785646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87d77fe640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8101a90 0x7f87d81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.186+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8102450 0x7f87d8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 -- 172.21.15.122:0/1543785646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8102450 msgr2=0x7f87d8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8102450 0x7f87d8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 -- 172.21.15.122:0/1543785646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8101a90 msgr2=0x7f87d81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8101a90 0x7f87d81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 -- 172.21.15.122:0/1543785646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d819d3c0 con 0x7f87d8107aa0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d7fff640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f87d00348f0 tx=0x7f87d0032ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d4ff9640 1 -- 172.21.15.122:0/1543785646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0041020 con 0x7f87d8107aa0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d4ff9640 1 -- 172.21.15.122:0/1543785646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d0034cd0 con 0x7f87d8107aa0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d819d650 con 0x7f87d8107aa0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87d4ff9640 1 -- 172.21.15.122:0/1543785646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d0039040 con 0x7f87d8107aa0 2024-04-02T13:43:58.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.187+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d81a4f40 con 0x7f87d8107aa0 2024-04-02T13:43:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.189+0000 7f87d4ff9640 1 -- 172.21.15.122:0/1543785646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f87d0044020 con 0x7f87d8107aa0 2024-04-02T13:43:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.189+0000 7f87e0bf1740 1 Processor -- start 2024-04-02T13:43:58.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.189+0000 7f87e0bf1740 1 -- start start 2024-04-02T13:43:58.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cb4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb51f0 0x564294cbd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564294c18830 con 0x564294cbddd0 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564294bc0820 con 0x564294cb48d0 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564294c147e0 con 0x564294cb51f0 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d4ff9640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 0x7f87c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d7fff640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 0x7f87c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40222/0 (socket says 172.21.15.122:40222) 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 -- 172.21.15.122:0/2104983533 learned_addr learned my addr 172.21.15.122:0/2104983533 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cb4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d4ff9640 1 -- 172.21.15.122:0/1543785646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f87d00f2930 con 0x7f87d8107aa0 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d77fe640 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb51f0 0x564294cbd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 -- 172.21.15.122:0/2104983533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb51f0 msgr2=0x564294cbd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb51f0 0x564294cbd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 -- 172.21.15.122:0/2104983533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 msgr2=0x564294cb4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cb4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 -- 172.21.15.122:0/2104983533 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642949f67d0 con 0x564294cbddd0 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d7fff640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 0x7f87c00a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87d00efe50 tx=0x7f87d0090030 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.190+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f87c8006aa0 tx=0x7f87c8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d2b80981197e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87bdffb640 1 -- 172.21.15.122:0/2104983533 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8015760 con 0x564294cbddd0 2024-04-02T13:43:58.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87bdffb640 1 -- 172.21.15.122:0/2104983533 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c801d070 con 0x564294cbddd0 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87bdffb640 1 -- 172.21.15.122:0/2104983533 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8018c10 con 0x564294cbddd0 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 -- 172.21.15.122:0/2104983533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 msgr2=0x564294cc01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f87c8006aa0 tx=0x7f87c8009510 comp rx=0 tx=0).stop 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 -- 172.21.15.122:0/2104983533 shutdown_connections 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cc01c0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb51f0 0x564294cbd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/2104983533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cb4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 -- 172.21.15.122:0/2104983533 >> 172.21.15.122:0/2104983533 conn(0x564294caf000 msgr2=0x564294caf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 -- 172.21.15.122:0/2104983533 shutdown_connections 2024-04-02T13:43:58.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.191+0000 7f87e0bf1740 1 -- 172.21.15.122:0/2104983533 wait complete. 2024-04-02T13:43:58.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 -- start start 2024-04-02T13:43:58.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 Processor -- start 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cbd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 0x564294cbafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564294cc0b60 con 0x564294cb51f0 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564294bd3d30 con 0x564294cbddd0 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564294cc0ce0 con 0x564294cb48d0 2024-04-02T13:43:58.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87e0bf1740 10 client.0 _ll_register_callbacks cb 0x564294bd3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40236/0 (socket says 172.21.15.122:40236) 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d77fe640 1 -- 172.21.15.122:0/764602564 learned_addr learned my addr 172.21.15.122:0/764602564 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cbd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 0x564294cbafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d77fe640 1 -- 172.21.15.122:0/764602564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 msgr2=0x564294cbd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.192+0000 7f87d77fe640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cbd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87d77fe640 1 -- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 msgr2=0x564294cbafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87d77fe640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 0x564294cbafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87d77fe640 1 -- 172.21.15.122:0/764602564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56429492c3f0 con 0x564294cb51f0 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 0x564294cbafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87d77fe640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00ba20 tx=0x7f87cc00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ad76c94feddf26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01b640 con 0x564294cb51f0 2024-04-02T13:43:58.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f87cc01b7e0 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c801d070 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f87cc01b9e0 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 10 client.18177 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5642948ec0f0 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c801d070 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f87cc019020 con 0x564294cb51f0 2024-04-02T13:43:58.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 10 client.18177 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 10 client.18177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 20 client.18177 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294ccde70 0x564294cd0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 4 client.18177 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.193+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564294c25d70 con 0x564294ccde70 2024-04-02T13:43:58.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.194+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294ccde70 0x564294cd0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.195+0000 7f87b6ffd640 10 client.18177 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.195+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294ccde70 0x564294cd0290 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de7e2942dfb97f server_cookie=ea427394385de735 in_seq=0 out_seq=0 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87b6ffd640 1 -- 172.21.15.122:0/764602564 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564294c25d70 con 0x564294ccde70 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87b6ffd640 10 client.18177 handle_command_reply: tid=0 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 client.18177 shutdown 2024-04-02T13:43:58.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 msgr2=0x564294cbaa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00ba20 tx=0x7f87cc00bee0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 shutdown_connections 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294ccde70 0x564294cd0290 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cbddd0 0x564294cbafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cbaa60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/764602564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cbd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 >> 172.21.15.122:0/764602564 conn(0x564294caf000 msgr2=0x564294cb0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87b6ffd640 0 client.18177 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:58.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 shutdown_connections 2024-04-02T13:43:58.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 1 -- 172.21.15.122:0/764602564 wait complete. 2024-04-02T13:43:58.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.196+0000 7f87e0bf1740 20 client.18177 trim_cache size 0 max 16384 2024-04-02T13:43:58.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.213+0000 7f87e0bf1740 1 Processor -- start 2024-04-02T13:43:58.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.213+0000 7f87e0bf1740 1 -- start start 2024-04-02T13:43:58.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cc9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 0x564294cae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81af0f0 con 0x564294cbddd0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b0c30 con 0x564294cb51f0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81af7c0 con 0x564294cb48d0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40246/0 (socket says 172.21.15.122:40246) 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 -- 172.21.15.122:0/453458081 learned_addr learned my addr 172.21.15.122:0/453458081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d77fe640 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 0x564294cae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 -- 172.21.15.122:0/453458081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 msgr2=0x564294cc9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cc9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 -- 172.21.15.122:0/453458081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 msgr2=0x564294cae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 0x564294cae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 -- 172.21.15.122:0/453458081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564294caee30 con 0x564294cbddd0 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d77fe640 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 0x564294cae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:58.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f87c80187c0 tx=0x7f87c80187f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5287aefec9c66df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87bd7fa640 1 -- 172.21.15.122:0/453458081 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c802e070 con 0x564294cbddd0 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87bd7fa640 1 -- 172.21.15.122:0/453458081 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c801d070 con 0x564294cbddd0 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.214+0000 7f87bd7fa640 1 -- 172.21.15.122:0/453458081 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8029460 con 0x564294cbddd0 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- 172.21.15.122:0/453458081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 msgr2=0x564294cbd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f87c80187c0 tx=0x7f87c80187f0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- 172.21.15.122:0/453458081 shutdown_connections 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cbddd0 0x564294cbd190 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb51f0 0x564294cae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/453458081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cb48d0 0x564294cc9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- 172.21.15.122:0/453458081 >> 172.21.15.122:0/453458081 conn(0x564294caf000 msgr2=0x564294cad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- 172.21.15.122:0/453458081 shutdown_connections 2024-04-02T13:43:58.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- 172.21.15.122:0/453458081 wait complete. 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.215+0000 7f87e0bf1740 1 -- start start 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 Processor -- start 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cc5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 0x564294cb6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81b0560 con 0x564294cb51f0 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81afe90 con 0x564294cb48d0 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f87d81ad0f0 con 0x564294cbddd0 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 10 client.0 _ll_register_callbacks cb 0x7f87cc004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40260/0 (socket says 172.21.15.122:40260) 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 -- 172.21.15.122:0/1803657037 learned_addr learned my addr 172.21.15.122:0/1803657037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cc5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 0x564294cb6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 -- 172.21.15.122:0/1803657037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 msgr2=0x564294cb6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 0x564294cb6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 -- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 msgr2=0x564294cc5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cc5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 -- 172.21.15.122:0/1803657037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564294cb71f0 con 0x564294cb51f0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 0x564294cb6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87d77fe640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x564294cbc030 tx=0x7f87cc003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4615cc3d6d7783 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc01bd80 con 0x564294cb51f0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f87cc01b440 con 0x564294cb51f0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.216+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87c801d070 con 0x564294cb51f0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f87cc037050 con 0x564294cb51f0 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 10 client.18183 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564294d33a60 con 0x564294cb51f0 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87c801d070 con 0x564294cb51f0 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f87cc019070 con 0x564294cb51f0 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 10 client.18183 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 20 client.18183 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294d34260 0x564294d36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 4 client.18183 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564294c200c0 con 0x564294d34260 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.217+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294d34260 0x564294d36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87d6ffd640 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294d34260 0x564294d36680 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ab3485a95f203c server_cookie=dbbb9d2305a7c6ff in_seq=0 out_seq=0 2024-04-02T13:43:58.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87b6ffd640 10 client.18183 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87b6ffd640 1 -- 172.21.15.122:0/1803657037 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564294c200c0 con 0x564294d34260 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87b6ffd640 10 client.18183 handle_command_reply: tid=0 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 client.18183 shutdown 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 msgr2=0x564294cc64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x564294cbc030 tx=0x7f87cc003f60 comp rx=0 tx=0).stop 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 shutdown_connections 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564294d34260 0x564294d36680 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564294cbddd0 0x564294cb6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564294cb51f0 0x564294cc64d0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 --2- 172.21.15.122:0/1803657037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564294cb48d0 0x564294cc5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87b6ffd640 0 client.18183 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 >> 172.21.15.122:0/1803657037 conn(0x564294caf000 msgr2=0x564294cae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 shutdown_connections 2024-04-02T13:43:58.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 1 -- 172.21.15.122:0/1803657037 wait complete. 2024-04-02T13:43:58.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.218+0000 7f87e0bf1740 20 client.18183 trim_cache size 0 max 16384 2024-04-02T13:43:58.234 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:43:58.234 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:43:58.234 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:43:58.234 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 msgr2=0x7f87c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 0x7f87c00a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f87d00efe50 tx=0x7f87d0090030 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 msgr2=0x7f87d819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f87d00348f0 tx=0x7f87d0032ec0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 shutdown_connections 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f87c00a3d20 0x7f87c00a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87d8107aa0 0x7f87d819cc90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d8102450 0x7f87d8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 --2- 172.21.15.122:0/1543785646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f87d8101a90 0x7f87d81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 >> 172.21.15.122:0/1543785646 conn(0x7f87d8078f00 msgr2=0x7f87d81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 shutdown_connections 2024-04-02T13:43:58.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.219+0000 7f87ddf38640 1 -- 172.21.15.122:0/1543785646 wait complete. 2024-04-02T13:43:58.237 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:43:58.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:43:58.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 -- 172.21.15.122:0/4008126986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 msgr2=0x7fbfac10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/4008126986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac10d8d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009cd0 tx=0x7fbfa402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 -- 172.21.15.122:0/4008126986 shutdown_connections 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/4008126986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac10d8d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/4008126986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfac100a00 0x7fbfac100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/4008126986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 0x7fbfac106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.334+0000 7fbfb11e2640 1 -- 172.21.15.122:0/4008126986 >> 172.21.15.122:0/4008126986 conn(0x7fbfac077b00 msgr2=0x7fbfac077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.335+0000 7fbfb11e2640 1 -- 172.21.15.122:0/4008126986 shutdown_connections 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.335+0000 7fbfb11e2640 1 -- 172.21.15.122:0/4008126986 wait complete. 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.335+0000 7fbfb11e2640 1 Processor -- start 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.335+0000 7fbfb11e2640 1 -- start start 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfac100a00 0x7fbfac1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 0x7fbfac19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac110e20 con 0x7fbfac1013c0 2024-04-02T13:43:58.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac110ca0 con 0x7fbfac100a00 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac110fa0 con 0x7fbfac106a10 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40280/0 (socket says 172.21.15.122:40280) 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 -- 172.21.15.122:0/2836949599 learned_addr learned my addr 172.21.15.122:0/2836949599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfab59e640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 0x7fbfac19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 -- 172.21.15.122:0/2836949599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 msgr2=0x7fbfac19bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 0x7fbfac19bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 -- 172.21.15.122:0/2836949599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfac100a00 msgr2=0x7fbfac1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfac100a00 0x7fbfac1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 -- 172.21.15.122:0/2836949599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac19c2b0 con 0x7fbfac1013c0 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfaa59c640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00d950 tx=0x7fbf9c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c017070 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac19c5a0 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf9c009c30 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00f6b0 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.336+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac1a3e30 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.337+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac0688f0 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.338+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbf9c0109d0 con 0x7fbfac1013c0 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.339+0000 7fbf93fff640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 0x7fbf940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.339+0000 7fbfaad9d640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 0x7fbf940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.339+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7fbf9c0d08d0 con 0x7fbfac1013c0 2024-04-02T13:43:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.339+0000 7fbfaad9d640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 0x7fbf940a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000a860 tx=0x7fbfa0005b60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.346+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbf9c009de0 con 0x7fbfac1013c0 2024-04-02T13:43:58.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.580+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfac198610 con 0x7fbfac1013c0 2024-04-02T13:43:58.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.581+0000 7fbf93fff640 1 -- 172.21.15.122:0/2836949599 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7fbf9c09cb80 con 0x7fbfac1013c0 2024-04-02T13:43:58.584 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:43:58.584 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:43:58.585 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 msgr2=0x7fbf940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 0x7fbf940a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000a860 tx=0x7fbfa0005b60 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 msgr2=0x7fbfac197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00d950 tx=0x7fbf9c00de10 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 shutdown_connections 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf940a3d20 0x7fbf940a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfac106a10 0x7fbfac19bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfac1013c0 0x7fbfac197800 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 --2- 172.21.15.122:0/2836949599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfac100a00 0x7fbfac1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.584+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 >> 172.21.15.122:0/2836949599 conn(0x7fbfac077b00 msgr2=0x7fbfac1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.585+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 shutdown_connections 2024-04-02T13:43:58.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.585+0000 7fbfb11e2640 1 -- 172.21.15.122:0/2836949599 wait complete. 2024-04-02T13:43:58.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:43:58.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.707+0000 7f27f1915640 1 -- 172.21.15.122:0/2746862709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec101330 msgr2=0x7f27ec10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.707+0000 7f27f1915640 1 --2- 172.21.15.122:0/2746862709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec101330 0x7f27ec10d840 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009cd0 tx=0x7f27e402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.707+0000 7f27f1915640 1 -- 172.21.15.122:0/2746862709 shutdown_connections 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- 172.21.15.122:0/2746862709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec101330 0x7f27ec10d840 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- 172.21.15.122:0/2746862709 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec100970 0x7f27ec100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- 172.21.15.122:0/2746862709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec106980 0x7f27ec106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- 172.21.15.122:0/2746862709 >> 172.21.15.122:0/2746862709 conn(0x7f27ec077b00 msgr2=0x7f27ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- 172.21.15.122:0/2746862709 shutdown_connections 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- 172.21.15.122:0/2746862709 wait complete. 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 Processor -- start 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- start start 2024-04-02T13:43:58.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101330 0x7f27ec1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec106980 0x7f27ec197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec110d60 con 0x7f27ec100970 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec110be0 con 0x7f27ec106980 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.708+0000 7f27f1915640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec110ee0 con 0x7f27ec101330 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33380/0 (socket says 172.21.15.122:33380) 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 -- 172.21.15.122:0/3754351513 learned_addr learned my addr 172.21.15.122:0/3754351513 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 -- 172.21.15.122:0/3754351513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101330 msgr2=0x7f27ec1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101330 0x7f27ec1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 -- 172.21.15.122:0/3754351513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec106980 msgr2=0x7f27ec197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec106980 0x7f27ec197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 -- 172.21.15.122:0/3754351513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec1a6800 con 0x7f27ec100970 2024-04-02T13:43:58.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27eaffd640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00d950 tx=0x7f27dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27f0913640 1 -- 172.21.15.122:0/3754351513 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc017070 con 0x7f27ec100970 2024-04-02T13:43:58.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec1a6af0 con 0x7f27ec100970 2024-04-02T13:43:58.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27f0913640 1 -- 172.21.15.122:0/3754351513 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc009c30 con 0x7f27ec100970 2024-04-02T13:43:58.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27f0913640 1 -- 172.21.15.122:0/3754351513 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00f6b0 con 0x7f27ec100970 2024-04-02T13:43:58.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.709+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec1a7040 con 0x7f27ec100970 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.711+0000 7f27f0913640 1 -- 172.21.15.122:0/3754351513 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f27dc0109d0 con 0x7f27ec100970 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 Processor -- start 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 -- start start 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f0913640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d57ca0 0x55ff02d5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ff02cb27b0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ff02c51570 con 0x55ff02d57ca0 2024-04-02T13:43:58.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f45dc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ff02cae760 con 0x55ff02d4f0c0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33394/0 (socket says 172.21.15.122:33394) 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27eaffd640 1 -- 172.21.15.122:0/4134422762 learned_addr learned my addr 172.21.15.122:0/4134422762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27f0913640 1 -- 172.21.15.122:0/3754351513 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f27dc0d0820 con 0x7f27ec100970 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d57ca0 0x55ff02d5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.712+0000 7f27e27fc640 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 -- 172.21.15.122:0/4134422762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 msgr2=0x55ff02d57760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d57760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 -- 172.21.15.122:0/4134422762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d57ca0 msgr2=0x55ff02d5a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d57ca0 0x55ff02d5a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 -- 172.21.15.122:0/4134422762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff029c63f0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 0x7f27d80a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27e4037040 tx=0x7f27e4032c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27eaffd640 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ce180 tx=0x7f27dc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23bb76b3c1eed256 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27e1ffb640 1 -- 172.21.15.122:0/4134422762 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc020070 con 0x55ff02d4e7a0 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27e1ffb640 1 -- 172.21.15.122:0/4134422762 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc09cad0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.713+0000 7f27e1ffb640 1 -- 172.21.15.122:0/4134422762 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0cde00 con 0x55ff02d4e7a0 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- 172.21.15.122:0/4134422762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 msgr2=0x55ff02d4eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ce180 tx=0x7f27dc010d70 comp rx=0 tx=0).stop 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- 172.21.15.122:0/4134422762 shutdown_connections 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d57ca0 0x55ff02d5a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d57760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- 172.21.15.122:0/4134422762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d4eb80 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- 172.21.15.122:0/4134422762 >> 172.21.15.122:0/4134422762 conn(0x55ff02d48ed0 msgr2=0x55ff02d492b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- 172.21.15.122:0/4134422762 shutdown_connections 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- 172.21.15.122:0/4134422762 wait complete. 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 -- start start 2024-04-02T13:43:58.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 Processor -- start 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.714+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d54ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ff02d5aa90 con 0x55ff02d4e7a0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ff02c6de80 con 0x55ff02d4f0c0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ff02d5ac10 con 0x55ff02d57ca0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 10 client.0 _ll_register_callbacks cb 0x55ff02ca9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33400/0 (socket says 172.21.15.122:33400) 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 -- 172.21.15.122:0/1846472226 learned_addr learned my addr 172.21.15.122:0/1846472226 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27e27fc640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d54ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 -- 172.21.15.122:0/1846472226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 msgr2=0x55ff02d551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 -- 172.21.15.122:0/1846472226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 msgr2=0x55ff02d54ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d54ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 -- 172.21.15.122:0/1846472226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff02a907d0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27eaffd640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0058c0 tx=0x7f27dc09c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b985dd37210c4c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27caffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc020070 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f27dc0bb7f0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc09cad0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f27dc0a0780 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 10 client.18201 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.715+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ff02d66b20 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27dc09cad0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc09b890 con 0x55ff02d4e7a0 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 10 client.18201 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 10 client.18201 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 20 client.18201 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02d67e70 0x55ff02d6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 4 client.18201 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff02cbfcf0 con 0x55ff02d67e70 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27e27fc640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02d67e70 0x55ff02d6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27caffd640 10 client.18201 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.716+0000 7f27e27fc640 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02d67e70 0x55ff02d6a260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6298a27f1b47a317 server_cookie=a46e8411158995c5 in_seq=0 out_seq=0 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27caffd640 1 -- 172.21.15.122:0/1846472226 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff02cbfcf0 con 0x55ff02d67e70 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27caffd640 10 client.18201 handle_command_reply: tid=0 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 client.18201 shutdown 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 msgr2=0x55ff02d57730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0058c0 tx=0x7f27dc09c3d0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 shutdown_connections 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02d67e70 0x55ff02d6a260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27caffd640 0 client.18201 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d54ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 --2- 172.21.15.122:0/1846472226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d57730 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 >> 172.21.15.122:0/1846472226 conn(0x55ff02d48ed0 msgr2=0x55ff02d4ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 shutdown_connections 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 1 -- 172.21.15.122:0/1846472226 wait complete. 2024-04-02T13:43:58.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.717+0000 7f27f45dc740 20 client.18201 trim_cache size 0 max 16384 2024-04-02T13:43:58.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 Processor -- start 2024-04-02T13:43:58.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 -- start start 2024-04-02T13:43:58.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d6f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 0x55ff02d6fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b1350 con 0x55ff02d57ca0 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b2e90 con 0x55ff02d4e7a0 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.731+0000 7f27f45dc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b1a20 con 0x55ff02d4f0c0 2024-04-02T13:43:58.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 0x55ff02d6fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d6f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36506/0 (socket says 172.21.15.122:36506) 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 0x55ff02d6fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33416/0 (socket says 172.21.15.122:33416) 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eaffd640 1 -- 172.21.15.122:0/3211142795 learned_addr learned my addr 172.21.15.122:0/3211142795 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 -- 172.21.15.122:0/3211142795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 msgr2=0x55ff02d63950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 -- 172.21.15.122:0/3211142795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 msgr2=0x55ff02d6fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 0x55ff02d6fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 -- 172.21.15.122:0/3211142795 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff02d5abc0 con 0x55ff02d4f0c0 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27eaffd640 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:58.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e27fc640 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d6f880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55ff02d6a820 tx=0x7f27d4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9040e761692c2632 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e17fa640 1 -- 172.21.15.122:0/3211142795 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4023070 con 0x55ff02d4f0c0 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e17fa640 1 -- 172.21.15.122:0/3211142795 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d401d430 con 0x55ff02d4f0c0 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.732+0000 7f27e17fa640 1 -- 172.21.15.122:0/3211142795 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4003a50 con 0x55ff02d4f0c0 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- 172.21.15.122:0/3211142795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 msgr2=0x55ff02d6f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d6f880 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55ff02d6a820 tx=0x7f27d4010800 comp rx=0 tx=0).stop 2024-04-02T13:43:58.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- 172.21.15.122:0/3211142795 shutdown_connections 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d57ca0 0x55ff02d6fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d4f0c0 0x55ff02d6f880 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 --2- 172.21.15.122:0/3211142795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4e7a0 0x55ff02d63950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- 172.21.15.122:0/3211142795 >> 172.21.15.122:0/3211142795 conn(0x55ff02d48ed0 msgr2=0x55ff02d4fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- 172.21.15.122:0/3211142795 shutdown_connections 2024-04-02T13:43:58.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- 172.21.15.122:0/3211142795 wait complete. 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.733+0000 7f27f45dc740 1 -- start start 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 Processor -- start 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d58dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b27c0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b20f0 con 0x55ff02d4f0c0 2024-04-02T13:43:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1af350 con 0x55ff02d57ca0 2024-04-02T13:43:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27f45dc740 10 client.0 _ll_register_callbacks cb 0x7f27dc0a5040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33428/0 (socket says 172.21.15.122:33428) 2024-04-02T13:43:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 -- 172.21.15.122:0/2641061139 learned_addr learned my addr 172.21.15.122:0/2641061139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27e27fc640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d58dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 -- 172.21.15.122:0/2641061139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 msgr2=0x55ff02d58dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d58dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 -- 172.21.15.122:0/2641061139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 msgr2=0x55ff02d52240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d52240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 -- 172.21.15.122:0/2641061139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff02d59310 con 0x55ff02d4e7a0 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eb7fe640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27eaffd640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f27dc098da0 tx=0x7f27dc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d552b3fefc1966c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27caffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc020070 con 0x55ff02d4e7a0 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.734+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f27dc0d05e0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d401d430 con 0x55ff02d4e7a0 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f27dc0106c0 con 0x55ff02d4e7a0 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 10 client.18207 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55ff02d59310 con 0x55ff02d4e7a0 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d401d430 con 0x55ff02d4e7a0 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f27dc09b820 con 0x55ff02d4e7a0 2024-04-02T13:43:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 10 client.18207 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 10 client.18207 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 20 client.18207 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02dcead0 0x55ff02dd0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 4 client.18207 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ff02cba040 con 0x55ff02dcead0 2024-04-02T13:43:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27e27fc640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02dcead0 0x55ff02dd0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.735+0000 7f27caffd640 10 client.18207 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27e27fc640 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02dcead0 0x55ff02dd0ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fb4b51e82ec4eac server_cookie=a9ade8e48a74faca in_seq=0 out_seq=0 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27caffd640 1 -- 172.21.15.122:0/2641061139 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ff02cba040 con 0x55ff02dcead0 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27caffd640 10 client.18207 handle_command_reply: tid=0 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 client.18207 shutdown 2024-04-02T13:43:58.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 msgr2=0x55ff02d60380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f27dc098da0 tx=0x7f27dc010d70 comp rx=0 tx=0).stop 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 shutdown_connections 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55ff02dcead0 0x55ff02dd0ef0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ff02d57ca0 0x55ff02d58dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ff02d4f0c0 0x55ff02d52240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27caffd640 0 client.18207 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:43:58.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 --2- 172.21.15.122:0/2641061139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ff02d4e7a0 0x55ff02d60380 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 >> 172.21.15.122:0/2641061139 conn(0x55ff02d48ed0 msgr2=0x55ff02d476a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 shutdown_connections 2024-04-02T13:43:58.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 1 -- 172.21.15.122:0/2641061139 wait complete. 2024-04-02T13:43:58.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.736+0000 7f27f45dc740 20 client.18207 trim_cache size 0 max 16384 2024-04-02T13:43:58.760 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:58.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 msgr2=0x7f27d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 0x7f27d80a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f27e4037040 tx=0x7f27e4032c90 comp rx=0 tx=0).stop 2024-04-02T13:43:58.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 msgr2=0x7f27ec196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00d950 tx=0x7f27dc00de10 comp rx=0 tx=0).stop 2024-04-02T13:43:58.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 shutdown_connections 2024-04-02T13:43:58.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f27d80a3d20 0x7f27d80a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27ec106980 0x7f27ec197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27ec101330 0x7f27ec1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 --2- 172.21.15.122:0/3754351513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f27ec100970 0x7f27ec196f90 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 >> 172.21.15.122:0/3754351513 conn(0x7f27ec077b00 msgr2=0x7f27ec103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 shutdown_connections 2024-04-02T13:43:58.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.737+0000 7f27f1915640 1 -- 172.21.15.122:0/3754351513 wait complete. 2024-04-02T13:43:58.766 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:43:58.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 -- 172.21.15.122:0/2477544361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308108fe0 msgr2=0x7f0308110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 --2- 172.21.15.122:0/2477544361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308108fe0 0x7f0308110bb0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f0300009ca0 tx=0x7f030002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 -- 172.21.15.122:0/2477544361 shutdown_connections 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 --2- 172.21.15.122:0/2477544361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308108fe0 0x7f0308110bb0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 --2- 172.21.15.122:0/2477544361 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03081048c0 0x7f0308108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 --2- 172.21.15.122:0/2477544361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0308103f10 0x7f03081042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 -- 172.21.15.122:0/2477544361 >> 172.21.15.122:0/2477544361 conn(0x7f03080ff7b0 msgr2=0x7f0308101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 -- 172.21.15.122:0/2477544361 shutdown_connections 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.872+0000 7f030e41b640 1 -- 172.21.15.122:0/2477544361 wait complete. 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 Processor -- start 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 -- start start 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0308108fe0 0x7f030819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f030819ee70 0x7f030819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0308114090 con 0x7f0308103f10 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0308113f10 con 0x7f0308108fe0 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030e41b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0308114210 con 0x7f030819ee70 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f0307fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f0307fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33448/0 (socket says 172.21.15.122:33448) 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f0307fff640 1 -- 172.21.15.122:0/3890838550 learned_addr learned my addr 172.21.15.122:0/3890838550 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f03077fe640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0308108fe0 0x7f030819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.873+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f030819ee70 0x7f030819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 -- 172.21.15.122:0/3890838550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f030819ee70 msgr2=0x7f030819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f030819ee70 0x7f030819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 -- 172.21.15.122:0/3890838550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0308108fe0 msgr2=0x7f030819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0308108fe0 0x7f030819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 -- 172.21.15.122:0/3890838550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030819f960 con 0x7f0308103f10 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f0307fff640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e9f0 tx=0x7f02f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f03057fa640 1 -- 172.21.15.122:0/3890838550 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f800cd80 con 0x7f0308103f10 2024-04-02T13:43:58.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f03057fa640 1 -- 172.21.15.122:0/3890838550 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02f8005500 con 0x7f0308103f10 2024-04-02T13:43:58.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030819fc50 con 0x7f0308103f10 2024-04-02T13:43:58.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f03057fa640 1 -- 172.21.15.122:0/3890838550 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8010690 con 0x7f0308103f10 2024-04-02T13:43:58.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.874+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03081a73f0 con 0x7f0308103f10 2024-04-02T13:43:58.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f03057fa640 1 -- 172.21.15.122:0/3890838550 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f02f80046c0 con 0x7f0308103f10 2024-04-02T13:43:58.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 Processor -- start 2024-04-02T13:43:58.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 -- start start 2024-04-02T13:43:58.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x565345733760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345736090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56534568e7b0 con 0x56534572a7a0 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56534562d570 con 0x565345733ca0 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0311004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56534568a760 con 0x56534572b0c0 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f030c9b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f030c9b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33460/0 (socket says 172.21.15.122:33460) 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f030c9b8640 1 -- 172.21.15.122:0/3341887360 learned_addr learned my addr 172.21.15.122:0/3341887360 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f0307fff640 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x565345733760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.877+0000 7f03077fe640 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345736090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f03057fa640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 0x7f02f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 -- 172.21.15.122:0/3341887360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 msgr2=0x565345733760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x565345733760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 -- 172.21.15.122:0/3341887360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 msgr2=0x565345736090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345736090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 -- 172.21.15.122:0/3341887360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653453a23f0 con 0x56534572a7a0 2024-04-02T13:43:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 0x7f02f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f03057fa640 1 -- 172.21.15.122:0/3890838550 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f02f80d1000 con 0x7f0308103f10 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f0300037040 tx=0x7f0300009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfaa28feba4149f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f02ee7fc640 1 -- 172.21.15.122:0/3341887360 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300039040 con 0x56534572a7a0 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f030c9b8640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 0x7f02f00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f030004c000 tx=0x7f0300045000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f02ee7fc640 1 -- 172.21.15.122:0/3341887360 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300003e50 con 0x56534572a7a0 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.878+0000 7f02ee7fc640 1 -- 172.21.15.122:0/3341887360 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300034440 con 0x56534572a7a0 2024-04-02T13:43:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- 172.21.15.122:0/3341887360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 msgr2=0x56534572ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f0300037040 tx=0x7f0300009980 comp rx=0 tx=0).stop 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- 172.21.15.122:0/3341887360 shutdown_connections 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345736090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x565345733760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 --2- 172.21.15.122:0/3341887360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572a7a0 0x56534572ab80 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- 172.21.15.122:0/3341887360 >> 172.21.15.122:0/3341887360 conn(0x565345724ed0 msgr2=0x5653457252b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- 172.21.15.122:0/3341887360 shutdown_connections 2024-04-02T13:43:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- 172.21.15.122:0/3341887360 wait complete. 2024-04-02T13:43:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 -- start start 2024-04-02T13:43:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.879+0000 7f0311004740 1 Processor -- start 2024-04-02T13:43:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x565345733750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345731220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565345736a90 con 0x56534572b0c0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x565345649e80 con 0x565345733ca0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565345736c10 con 0x56534572a7a0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 10 client.0 _ll_register_callbacks cb 0x565345685aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33464/0 (socket says 172.21.15.122:33464) 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 -- 172.21.15.122:0/269494236 learned_addr learned my addr 172.21.15.122:0/269494236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f030c9b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345731220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0307fff640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x565345733750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 -- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 msgr2=0x565345733750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x565345733750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 -- 172.21.15.122:0/269494236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 msgr2=0x565345731220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345731220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 -- 172.21.15.122:0/269494236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534546c7d0 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f03077fe640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00bed0 tx=0x7f02fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e037b99d576edd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f02e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc004020 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f02fc0041c0 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.880+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0300003e50 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f02fc024050 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 10 client.18219 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565345742b90 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300003e50 con 0x56534572b0c0 2024-04-02T13:43:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f02fc0037d0 con 0x56534572b0c0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 10 client.18219 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 10 client.18219 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 20 client.18219 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457440f0 0x565345746530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 4 client.18219 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534569bcf0 con 0x5653457440f0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0307fff640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457440f0 0x565345746530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f02e77fe640 10 client.18219 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.881+0000 7f0307fff640 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457440f0 0x565345746530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b793b957a8a1e24 server_cookie=cf27025dd7802e26 in_seq=0 out_seq=0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.882+0000 7f02e77fe640 1 -- 172.21.15.122:0/269494236 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56534569bcf0 con 0x5653457440f0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.882+0000 7f02e77fe640 10 client.18219 handle_command_reply: tid=0 2024-04-02T13:43:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.882+0000 7f0311004740 1 client.18219 shutdown 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.882+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 msgr2=0x565345730ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.882+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00bed0 tx=0x7f02fc011810 comp rx=0 tx=0).stop 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 shutdown_connections 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345731220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457440f0 0x565345746530 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345730ae0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 --2- 172.21.15.122:0/269494236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x565345733750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f02e77fe640 0 client.18219 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 >> 172.21.15.122:0/269494236 conn(0x565345724ed0 msgr2=0x565345726c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 shutdown_connections 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 1 -- 172.21.15.122:0/269494236 wait complete. 2024-04-02T13:43:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.883+0000 7f0311004740 20 client.18219 trim_cache size 0 max 16384 2024-04-02T13:43:58.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 Processor -- start 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 -- start start 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345733750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03081b11f0 con 0x56534572b0c0 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03081b2d30 con 0x565345733ca0 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0311004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03081b18c0 con 0x56534572a7a0 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0307fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0307fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36788/0 (socket says 172.21.15.122:36788) 2024-04-02T13:43:58.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33480/0 (socket says 172.21.15.122:33480) 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f0307fff640 1 -- 172.21.15.122:0/1049802513 learned_addr learned my addr 172.21.15.122:0/1049802513 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f030c9b8640 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345733750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 -- 172.21.15.122:0/1049802513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 msgr2=0x5653457269f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 -- 172.21.15.122:0/1049802513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 msgr2=0x565345733750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345733750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.896+0000 7f03077fe640 1 -- 172.21.15.122:0/1049802513 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565345731590 con 0x56534572b0c0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0307fff640 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f03077fe640 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f02fc003c70 tx=0x7f02fc011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d70189a3f18a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f02edffb640 1 -- 172.21.15.122:0/1049802513 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc02c030 con 0x56534572b0c0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f02edffb640 1 -- 172.21.15.122:0/1049802513 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc005040 con 0x56534572b0c0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f02edffb640 1 -- 172.21.15.122:0/1049802513 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02fc003f50 con 0x56534572b0c0 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 -- 172.21.15.122:0/1049802513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 msgr2=0x565345733210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f02fc003c70 tx=0x7f02fc011d90 comp rx=0 tx=0).stop 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 -- 172.21.15.122:0/1049802513 shutdown_connections 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565345733ca0 0x565345733750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56534572b0c0 0x565345733210 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 --2- 172.21.15.122:0/1049802513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572a7a0 0x5653457269f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.897+0000 7f0311004740 1 -- 172.21.15.122:0/1049802513 >> 172.21.15.122:0/1049802513 conn(0x565345724ed0 msgr2=0x5653457262e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- 172.21.15.122:0/1049802513 shutdown_connections 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- 172.21.15.122:0/1049802513 wait complete. 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- start start 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 Processor -- start 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 0x565345738e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03081b2660 con 0x565345733ca0 2024-04-02T13:43:58.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03081b1f90 con 0x56534572a7a0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03081af1f0 con 0x56534572b0c0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0311004740 10 client.0 _ll_register_callbacks cb 0x7f02fc00bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f03077fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.898+0000 7f0307fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36562/0 (socket says 172.21.15.122:36562) 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f03077fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36794/0 (socket says 172.21.15.122:36794) 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 -- 172.21.15.122:0/2836643879 learned_addr learned my addr 172.21.15.122:0/2836643879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f030c9b8640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 0x565345738e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 -- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 msgr2=0x56534573c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 -- 172.21.15.122:0/2836643879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 msgr2=0x565345738e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 0x565345738e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 -- 172.21.15.122:0/2836643879 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565345739470 con 0x56534572a7a0 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f030c9b8640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 0x565345738e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:58.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f03077fe640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0307fff640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02f8099580 tx=0x7f02f80d4b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf6e4a4965f9a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f02e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f809c980 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f02f80ce5e0 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02fc005040 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f02f809c510 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0311004740 10 client.27305 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.899+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5653457ab020 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02fc005040 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f02f80a5390 con 0x56534572a7a0 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 10 client.27305 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 10 client.27305 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 20 client.27305 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:58.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457ab770 0x5653457adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 4 client.27305 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565345696040 con 0x5653457ab770 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f03077fe640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457ab770 0x5653457adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f02e77fe640 10 client.27305 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.900+0000 7f03077fe640 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457ab770 0x5653457adb60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b37d366af529769 server_cookie=bef70d319f2d3787 in_seq=0 out_seq=0 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f02e77fe640 1 -- 172.21.15.122:0/2836643879 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565345696040 con 0x5653457ab770 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f02e77fe640 10 client.27305 handle_command_reply: tid=0 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 client.27305 shutdown 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 msgr2=0x56534573be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f02f8099580 tx=0x7f02f80d4b50 comp rx=0 tx=0).stop 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 shutdown_connections 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5653457ab770 0x5653457adb60 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565345733ca0 0x565345738e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f02e77fe640 0 client.27305 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:43:58.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56534572b0c0 0x56534573c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 --2- 172.21.15.122:0/2836643879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56534572a7a0 0x56534573be60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 >> 172.21.15.122:0/2836643879 conn(0x565345724ed0 msgr2=0x565345726b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 shutdown_connections 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 1 -- 172.21.15.122:0/2836643879 wait complete. 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f0311004740 20 client.27305 trim_cache size 0 max 16384 2024-04-02T13:43:58.914 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:58.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 msgr2=0x7f02f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 0x7f02f00a61e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f030004c000 tx=0x7f0300045000 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 msgr2=0x7f030819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.901+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f02f800e9f0 tx=0x7f02f800edf0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 shutdown_connections 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f02f00a3d20 0x7f02f00a61e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f030819ee70 0x7f030819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0308108fe0 0x7f030819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 --2- 172.21.15.122:0/3890838550 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0308103f10 0x7f030819a5b0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 >> 172.21.15.122:0/3890838550 conn(0x7f03080ff7b0 msgr2=0x7f0308101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 shutdown_connections 2024-04-02T13:43:58.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:58.902+0000 7f030e41b640 1 -- 172.21.15.122:0/3890838550 wait complete. 2024-04-02T13:43:58.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:43:58.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 285610 3.13 MB/sec execute 319 sec latency 2967.095 ms 2024-04-02T13:43:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 -- 172.21.15.122:0/4165387248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14100970 msgr2=0x7f4a14100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 --2- 172.21.15.122:0/4165387248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14100970 0x7f4a14100df0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009930 tx=0x7f4a0802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 -- 172.21.15.122:0/4165387248 shutdown_connections 2024-04-02T13:43:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 --2- 172.21.15.122:0/4165387248 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a1410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 --2- 172.21.15.122:0/4165387248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14100970 0x7f4a14100df0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 --2- 172.21.15.122:0/4165387248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14106980 0x7f4a14106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.023+0000 7f4a1a806640 1 -- 172.21.15.122:0/4165387248 >> 172.21.15.122:0/4165387248 conn(0x7f4a14077b00 msgr2=0x7f4a14077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- 172.21.15.122:0/4165387248 shutdown_connections 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- 172.21.15.122:0/4165387248 wait complete. 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 Processor -- start 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- start start 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 0x7f4a14198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a141988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14110d90 con 0x7f4a14106980 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14110c10 con 0x7f4a14101330 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.024+0000 7f4a1a806640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14110f10 con 0x7f4a14100970 2024-04-02T13:43:59.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a141988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33506/0 (socket says 172.21.15.122:33506) 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 -- 172.21.15.122:0/2442896290 learned_addr learned my addr 172.21.15.122:0/2442896290 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a13fff640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 0x7f4a14198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 -- 172.21.15.122:0/2442896290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 msgr2=0x7f4a14198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 0x7f4a14198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 -- 172.21.15.122:0/2442896290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 msgr2=0x7f4a141988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a141988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 -- 172.21.15.122:0/2442896290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1419d360 con 0x7f4a14106980 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a137fe640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a141988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a13fff640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 0x7f4a14198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a18da3640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a117fa640 1 -- 172.21.15.122:0/2442896290 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004b20 con 0x7f4a14106980 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a117fa640 1 -- 172.21.15.122:0/2442896290 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c004cc0 con 0x7f4a14106980 2024-04-02T13:43:59.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1419d650 con 0x7f4a14106980 2024-04-02T13:43:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a117fa640 1 -- 172.21.15.122:0/2442896290 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010760 con 0x7f4a14106980 2024-04-02T13:43:59.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.025+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141a4ee0 con 0x7f4a14106980 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 Processor -- start 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a117fa640 1 -- 172.21.15.122:0/2442896290 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4a0c005020 con 0x7f4a14106980 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 -- start start 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5551560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5651e54b57b0 con 0x5651e555f0b0 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5651e5454570 con 0x5651e5551b30 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.028+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651e54b1760 con 0x5651e5551180 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33522/0 (socket says 172.21.15.122:33522) 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 -- 172.21.15.122:0/282547053 learned_addr learned my addr 172.21.15.122:0/282547053 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5551560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a117fa640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 0x7f49fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 -- 172.21.15.122:0/282547053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 msgr2=0x5651e5551560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5551560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 -- 172.21.15.122:0/282547053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 msgr2=0x5651e555eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 -- 172.21.15.122:0/282547053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e51c93f0 con 0x5651e555f0b0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a18da3640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 0x7f49fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a117fa640 1 -- 172.21.15.122:0/2442896290 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4a0c0d1220 con 0x7f4a14106980 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f4a137fe640 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005c40 tx=0x7f4a08005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b91caf253008898 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f49fa7fc640 1 -- 172.21.15.122:0/282547053 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0802ca80 con 0x5651e555f0b0 2024-04-02T13:43:59.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f49fa7fc640 1 -- 172.21.15.122:0/282547053 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0802cc20 con 0x5651e555f0b0 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.029+0000 7f49fa7fc640 1 -- 172.21.15.122:0/282547053 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080055a0 con 0x5651e555f0b0 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/282547053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 msgr2=0x5651e55614a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a18da3640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 0x7f49fc0a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c099230 tx=0x7f4a0c005b40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f4a08005c40 tx=0x7f4a08005b70 comp rx=0 tx=0).stop 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/282547053 shutdown_connections 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e55614a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/282547053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5551560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/282547053 >> 172.21.15.122:0/282547053 conn(0x5651e554bed0 msgr2=0x5651e554c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/282547053 shutdown_connections 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/282547053 wait complete. 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.030+0000 7f4a1d3ef740 1 -- start start 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 Processor -- start 2024-04-02T13:43:59.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e555bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5651e5561e10 con 0x5651e555f0b0 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5651e5470e80 con 0x5651e5551b30 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5651e5561f90 con 0x5651e5551180 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a1d3ef740 10 client.0 _ll_register_callbacks cb 0x5651e54acaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33528/0 (socket says 172.21.15.122:33528) 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a137fe640 1 -- 172.21.15.122:0/417584516 learned_addr learned my addr 172.21.15.122:0/417584516 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a137fe640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e555bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 -- 172.21.15.122:0/417584516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 msgr2=0x5651e555bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e555bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 -- 172.21.15.122:0/417584516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 msgr2=0x5651e555c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 -- 172.21.15.122:0/417584516 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e52937d0 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a137fe640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f4a13fff640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555c450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012700 tx=0x7f4a04012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57f86f3f8ef4a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.031+0000 7f49f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04011c70 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4a04011e10 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0802cc20 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4a04005020 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 10 client.27317 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5651e556dfb0 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0802cc20 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4a0401c470 con 0x5651e5551b30 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 10 client.27317 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 10 client.27317 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 20 client.27317 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e556f4c0 0x5651e55718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 4 client.27317 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.032+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e54c2cf0 con 0x5651e556f4c0 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.033+0000 7f4a137fe640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e556f4c0 0x5651e55718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.033+0000 7f4a137fe640 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e556f4c0 0x5651e55718b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3ead1dc8b483b29 server_cookie=89851136d84835db in_seq=0 out_seq=0 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.033+0000 7f49f37fe640 10 client.27317 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f49f37fe640 1 -- 172.21.15.122:0/417584516 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651e54c2cf0 con 0x5651e556f4c0 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f49f37fe640 10 client.27317 handle_command_reply: tid=0 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 client.27317 shutdown 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 msgr2=0x5651e555c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555c450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012700 tx=0x7f4a04012bc0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 shutdown_connections 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e555f0b0 0x5651e555c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e556f4c0 0x5651e55718b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551b30 0x5651e555c450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/417584516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e555bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 >> 172.21.15.122:0/417584516 conn(0x5651e554bed0 msgr2=0x5651e554dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f49f37fe640 0 client.27317 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:59.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 shutdown_connections 2024-04-02T13:43:59.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/417584516 wait complete. 2024-04-02T13:43:59.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.034+0000 7f4a1d3ef740 20 client.27317 trim_cache size 0 max 16384 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 Processor -- start 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 -- start start 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551180 0x5651e555e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 0x5651e555aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141acde0 con 0x5651e5551b30 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141ae920 con 0x5651e5551180 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141ad4b0 con 0x5651e555f0b0 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33540/0 (socket says 172.21.15.122:33540) 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 -- 172.21.15.122:0/2099831839 learned_addr learned my addr 172.21.15.122:0/2099831839 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a137fe640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551180 0x5651e555e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a137fe640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 0x5651e555aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 -- 172.21.15.122:0/2099831839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 msgr2=0x5651e555aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 0x5651e555aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 -- 172.21.15.122:0/2099831839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551180 msgr2=0x5651e555e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551180 0x5651e555e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a13fff640 1 -- 172.21.15.122:0/2099831839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e55518d0 con 0x5651e5551b30 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.048+0000 7f4a137fe640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 0x5651e555aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:59.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a13fff640 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000c00 tx=0x7f4a0401dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f3618c2f47295c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f49f9ffb640 1 -- 172.21.15.122:0/2099831839 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0402d030 con 0x5651e5551b30 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f49f9ffb640 1 -- 172.21.15.122:0/2099831839 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04009e20 con 0x5651e5551b30 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f49f9ffb640 1 -- 172.21.15.122:0/2099831839 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0401c470 con 0x5651e5551b30 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/2099831839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 msgr2=0x5651e556ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f4a04000c00 tx=0x7f4a0401dee0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/2099831839 shutdown_connections 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e555f0b0 0x5651e555aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e556ad40 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/2099831839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e5551180 0x5651e555e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.049+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/2099831839 >> 172.21.15.122:0/2099831839 conn(0x5651e554bed0 msgr2=0x5651e555a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/2099831839 shutdown_connections 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/2099831839 wait complete. 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- start start 2024-04-02T13:43:59.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 Processor -- start 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5560f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e555f0b0 0x5651e555fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141ae250 con 0x5651e5551b30 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141adb80 con 0x5651e555f0b0 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141aade0 con 0x5651e5551180 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.050+0000 7f4a1d3ef740 10 client.0 _ll_register_callbacks cb 0x7f4a040097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a137fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5560f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33546/0 (socket says 172.21.15.122:33546) 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 -- 172.21.15.122:0/3761904942 learned_addr learned my addr 172.21.15.122:0/3761904942 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a137fe640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e555f0b0 0x5651e555fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 -- 172.21.15.122:0/3761904942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 msgr2=0x5651e5560f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5560f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 -- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e555f0b0 msgr2=0x5651e555fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e555f0b0 0x5651e555fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 -- 172.21.15.122:0/3761904942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e5566250 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a137fe640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5560f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a13fff640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012700 tx=0x7f4a0401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd9f9230b814a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f49f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0402d040 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a04009e20 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f4a040051c0 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4a04009600 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a1d3ef740 10 client.18243 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.051+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5651e5567550 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a04009e20 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f4a04032460 con 0x5651e5551b30 2024-04-02T13:43:59.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 10 client.18243 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 10 client.18243 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 20 client.18243 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e55d5a30 0x5651e55d7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 4 client.18243 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651e54bd040 con 0x5651e55d5a30 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a137fe640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e55d5a30 0x5651e55d7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f4a137fe640 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e55d5a30 0x5651e55d7e20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9ec1605cd86dfc2 server_cookie=b24150ae732d61fa in_seq=0 out_seq=0 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.052+0000 7f49f37fe640 10 client.18243 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f49f37fe640 1 -- 172.21.15.122:0/3761904942 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651e54bd040 con 0x5651e55d5a30 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f49f37fe640 10 client.18243 handle_command_reply: tid=0 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 client.18243 shutdown 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 msgr2=0x5651e555f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f4a04012700 tx=0x7f4a0401c720 comp rx=0 tx=0).stop 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 shutdown_connections 2024-04-02T13:43:59.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5651e55d5a30 0x5651e55d7e20 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5651e555f0b0 0x5651e555fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5651e5551b30 0x5651e555f490 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 --2- 172.21.15.122:0/3761904942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5651e5551180 0x5651e5560f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f49f37fe640 0 client.18243 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 >> 172.21.15.122:0/3761904942 conn(0x5651e554bed0 msgr2=0x5651e555b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 shutdown_connections 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 1 -- 172.21.15.122:0/3761904942 wait complete. 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1d3ef740 20 client.18243 trim_cache size 0 max 16384 2024-04-02T13:43:59.066 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 msgr2=0x7f49fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 0x7f49fc0a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c099230 tx=0x7f4a0c005b40 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 msgr2=0x7f4a1419cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00eb10 tx=0x7f4a0c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 shutdown_connections 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49fc0a3d20 0x7f49fc0a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.053+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a14106980 0x7f4a1419cc30 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.054+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a14101330 0x7f4a141988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.054+0000 7f4a1a806640 1 --2- 172.21.15.122:0/2442896290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a14100970 0x7f4a14198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.054+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 >> 172.21.15.122:0/2442896290 conn(0x7f4a14077b00 msgr2=0x7f4a140ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.054+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 shutdown_connections 2024-04-02T13:43:59.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.054+0000 7f4a1a806640 1 -- 172.21.15.122:0/2442896290 wait complete. 2024-04-02T13:43:59.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:43:59.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- 172.21.15.122:0/3653416582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 msgr2=0x7f80d4100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3653416582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4100e00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009930 tx=0x7f80c802bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:59.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- 172.21.15.122:0/3653416582 shutdown_connections 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3653416582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 0x7f80d410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3653416582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4100e00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3653416582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 0x7f80d4106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- 172.21.15.122:0/3653416582 >> 172.21.15.122:0/3653416582 conn(0x7f80d4076d80 msgr2=0x7f80d4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- 172.21.15.122:0/3653416582 shutdown_connections 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- 172.21.15.122:0/3653416582 wait complete. 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 Processor -- start 2024-04-02T13:43:59.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.177+0000 7f80da6b7640 1 -- start start 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 0x7f80d4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 0x7f80d419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4110da0 con 0x7f80d4100980 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4110c20 con 0x7f80d4101340 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80da6b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4110f20 con 0x7f80d4106990 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33572/0 (socket says 172.21.15.122:33572) 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 -- 172.21.15.122:0/3242581023 learned_addr learned my addr 172.21.15.122:0/3242581023 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 0x7f80d419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d37fe640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 0x7f80d4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 -- 172.21.15.122:0/3242581023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 msgr2=0x7f80d419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 0x7f80d419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 -- 172.21.15.122:0/3242581023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 msgr2=0x7f80d4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 0x7f80d4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 -- 172.21.15.122:0/3242581023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d419d400 con 0x7f80d4100980 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.178+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9f0 tx=0x7f80c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.179+0000 7f80d17fa640 1 -- 172.21.15.122:0/3242581023 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c400ccc0 con 0x7f80d4100980 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.179+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d419d6f0 con 0x7f80d4100980 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.179+0000 7f80d17fa640 1 -- 172.21.15.122:0/3242581023 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c400ce60 con 0x7f80d4100980 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.179+0000 7f80d17fa640 1 -- 172.21.15.122:0/3242581023 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c4010690 con 0x7f80d4100980 2024-04-02T13:43:59.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.179+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d41a4f80 con 0x7f80d4100980 2024-04-02T13:43:59.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.181+0000 7f80d17fa640 1 -- 172.21.15.122:0/3242581023 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f80c40046c0 con 0x7f80d4100980 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.181+0000 7f80dd379740 1 Processor -- start 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.181+0000 7f80dd379740 1 -- start start 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142260c0 0x555b1422e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 0x555b14231090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b141897b0 con 0x555b1422eca0 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555b14128570 con 0x555b142257a0 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80dd379740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b14185760 con 0x555b142260c0 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36634/0 (socket says 172.21.15.122:36634) 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 0x555b14231090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 -- 172.21.15.122:0/3773280632 learned_addr learned my addr 172.21.15.122:0/3773280632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142260c0 0x555b1422e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d17fa640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 0x7f80bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 -- 172.21.15.122:0/3773280632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142260c0 msgr2=0x555b1422e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142260c0 0x555b1422e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 -- 172.21.15.122:0/3773280632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 msgr2=0x555b14231090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 0x555b14231090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 -- 172.21.15.122:0/3773280632 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b13e9d3f0 con 0x555b142257a0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d37fe640 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 0x555b14231090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d17fa640 1 -- 172.21.15.122:0/3242581023 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f80c40d0f50 con 0x7f80d4100980 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.182+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 0x7f80bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011570 tx=0x7f80cc011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7ef604609785faf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80ba7fc640 1 -- 172.21.15.122:0/3773280632 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc017070 con 0x555b142257a0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80ba7fc640 1 -- 172.21.15.122:0/3773280632 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80cc00fd20 con 0x555b142257a0 2024-04-02T13:43:59.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80ba7fc640 1 -- 172.21.15.122:0/3773280632 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc01cb30 con 0x555b142257a0 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80dd379740 1 -- 172.21.15.122:0/3773280632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 msgr2=0x555b14225b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80d8c54640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 0x7f80bc0a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80cc023000 tx=0x7f80cc024000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.183+0000 7f80dd379740 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f80cc011570 tx=0x7f80cc011a30 comp rx=0 tx=0).stop 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- 172.21.15.122:0/3773280632 shutdown_connections 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b1422eca0 0x555b14231090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142260c0 0x555b1422e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- 172.21.15.122:0/3773280632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14225b80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- 172.21.15.122:0/3773280632 >> 172.21.15.122:0/3773280632 conn(0x555b1421fed0 msgr2=0x555b142202b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- 172.21.15.122:0/3773280632 shutdown_connections 2024-04-02T13:43:59.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- 172.21.15.122:0/3773280632 wait complete. 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- start start 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 Processor -- start 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 0x555b1422c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b14231a90 con 0x555b142260c0 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555b14144e80 con 0x555b1422eca0 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b14231c10 con 0x555b142257a0 2024-04-02T13:43:59.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.184+0000 7f80dd379740 10 client.0 _ll_register_callbacks cb 0x555b14180aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36878/0 (socket says 172.21.15.122:36878) 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33600/0 (socket says 172.21.15.122:33600) 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 -- 172.21.15.122:0/2878828052 learned_addr learned my addr 172.21.15.122:0/2878828052 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d8c54640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 0x555b1422c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 -- 172.21.15.122:0/2878828052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 msgr2=0x555b1422c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 0x555b1422c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 -- 172.21.15.122:0/2878828052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 msgr2=0x555b1422c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 -- 172.21.15.122:0/2878828052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b13f677d0 con 0x555b142257a0 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d8c54640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 0x555b1422c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d37fe640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80d3fff640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80c40994d0 tx=0x7f80c40d4690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=655d10936c63f55b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c409c460 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f80c4099170 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80cc00fd20 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f80c409d200 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80dd379740 10 client.27151 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.185+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555b1423dc10 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80cc00fd20 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f80c4098850 con 0x555b142257a0 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 10 client.27151 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 10 client.27151 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 20 client.27151 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b1423f0d0 0x555b142414c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 4 client.27151 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:59.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b14196cf0 con 0x555b1423f0d0 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80d37fe640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b1423f0d0 0x555b142414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80d37fe640 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b1423f0d0 0x555b142414c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25bbce3182cd98d0 server_cookie=82d928047241eeea in_seq=0 out_seq=0 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.186+0000 7f80b37fe640 10 client.27151 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80b37fe640 1 -- 172.21.15.122:0/2878828052 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b14196cf0 con 0x555b1423f0d0 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80b37fe640 10 client.27151 handle_command_reply: tid=0 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 client.27151 shutdown 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 msgr2=0x555b1422bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f80c40994d0 tx=0x7f80c40d4690 comp rx=0 tx=0).stop 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 shutdown_connections 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b1423f0d0 0x555b142414c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b1422eca0 0x555b1422c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80b37fe640 0 client.27151 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b1422c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 --2- 172.21.15.122:0/2878828052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b142257a0 0x555b1422bb00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 >> 172.21.15.122:0/2878828052 conn(0x555b1421fed0 msgr2=0x555b14221c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 shutdown_connections 2024-04-02T13:43:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 1 -- 172.21.15.122:0/2878828052 wait complete. 2024-04-02T13:43:59.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.187+0000 7f80dd379740 20 client.27151 trim_cache size 0 max 16384 2024-04-02T13:43:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 Processor -- start 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 -- start start 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 0x555b1423a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b1421edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41af0f0 con 0x555b142257a0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b0c30 con 0x555b142260c0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.202+0000 7f80dd379740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41af7c0 con 0x555b1422eca0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 0x555b1423a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d8c54640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b1421edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33612/0 (socket says 172.21.15.122:33612) 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 -- 172.21.15.122:0/3620388388 learned_addr learned my addr 172.21.15.122:0/3620388388 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 0x555b1423a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36638/0 (socket says 172.21.15.122:36638) 2024-04-02T13:43:59.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 -- 172.21.15.122:0/3620388388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 msgr2=0x555b1421edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b1421edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 -- 172.21.15.122:0/3620388388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 msgr2=0x555b1423a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 0x555b1423a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 -- 172.21.15.122:0/3620388388 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b14241b10 con 0x555b142257a0 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80d3fff640 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f80c409adb0 tx=0x7f80c409b010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e9a261ada16f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80b9ffb640 1 -- 172.21.15.122:0/3620388388 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c409c460 con 0x555b142257a0 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80b9ffb640 1 -- 172.21.15.122:0/3620388388 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c409cb00 con 0x555b142257a0 2024-04-02T13:43:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.203+0000 7f80b9ffb640 1 -- 172.21.15.122:0/3620388388 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c40a0eb0 con 0x555b142257a0 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- 172.21.15.122:0/3620388388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 msgr2=0x555b1422e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f80c409adb0 tx=0x7f80c409b010 comp rx=0 tx=0).stop 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- 172.21.15.122:0/3620388388 shutdown_connections 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b1421edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142260c0 0x555b1423a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 --2- 172.21.15.122:0/3620388388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142257a0 0x555b1422e230 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- 172.21.15.122:0/3620388388 >> 172.21.15.122:0/3620388388 conn(0x555b1421fed0 msgr2=0x555b1421e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- 172.21.15.122:0/3620388388 shutdown_connections 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- 172.21.15.122:0/3620388388 wait complete. 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.204+0000 7f80dd379740 1 -- start start 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 Processor -- start 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b14232450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41b0560 con 0x555b142260c0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41afe90 con 0x555b142257a0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41ad0f0 con 0x555b1422eca0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80dd379740 10 client.0 _ll_register_callbacks cb 0x7f80c4004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33620/0 (socket says 172.21.15.122:33620) 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 -- 172.21.15.122:0/732408734 learned_addr learned my addr 172.21.15.122:0/732408734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d3fff640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36644/0 (socket says 172.21.15.122:36644) 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d8c54640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b14232450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 -- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 msgr2=0x555b14232450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b14232450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 -- 172.21.15.122:0/732408734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 msgr2=0x555b14236e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 -- 172.21.15.122:0/732408734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b14232c80 con 0x555b142260c0 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d8c54640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b14232450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d3fff640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80d37fe640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f80c802cfd0 tx=0x7f80c802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65939bb6e55b16b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.205+0000 7f80b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c804c070 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f80c80483d0 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c409cb00 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f80c805a050 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 10 client.18261 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x555b142a52a0 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c409cb00 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f80c8047040 con 0x555b142260c0 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 10 client.18261 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 10 client.18261 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 20 client.18261 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b142a5a00 0x555b142a7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 4 client.18261 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:59.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b14191040 con 0x555b142a5a00 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.206+0000 7f80d3fff640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b142a5a00 0x555b142a7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80b37fe640 10 client.18261 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80d3fff640 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b142a5a00 0x555b142a7e20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f422138f81dfe289 server_cookie=a1331985bed763a1 in_seq=0 out_seq=0 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80b37fe640 1 -- 172.21.15.122:0/732408734 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b14191040 con 0x555b142a5a00 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80b37fe640 10 client.18261 handle_command_reply: tid=0 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 client.18261 shutdown 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 msgr2=0x555b14237360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f80c802cfd0 tx=0x7f80c802cdf0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 shutdown_connections 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555b142a5a00 0x555b142a7e20 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b1422eca0 0x555b14232450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80b37fe640 0 client.18261 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b142260c0 0x555b14237360 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 --2- 172.21.15.122:0/732408734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b142257a0 0x555b14236e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 >> 172.21.15.122:0/732408734 conn(0x555b1421fed0 msgr2=0x555b1421ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 shutdown_connections 2024-04-02T13:43:59.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 1 -- 172.21.15.122:0/732408734 wait complete. 2024-04-02T13:43:59.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.207+0000 7f80dd379740 20 client.18261 trim_cache size 0 max 16384 2024-04-02T13:43:59.218 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 msgr2=0x7f80bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 0x7f80bc0a61e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80cc023000 tx=0x7f80cc024000 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 msgr2=0x7f80d4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f80c400e9f0 tx=0x7f80c400edf0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 shutdown_connections 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f80bc0a3d20 0x7f80bc0a61e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80d4106990 0x7f80d419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d4101340 0x7f80d4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 --2- 172.21.15.122:0/3242581023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80d4100980 0x7f80d4198410 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 >> 172.21.15.122:0/3242581023 conn(0x7f80d4076d80 msgr2=0x7f80d406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 shutdown_connections 2024-04-02T13:43:59.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.208+0000 7f80da6b7640 1 -- 172.21.15.122:0/3242581023 wait complete. 2024-04-02T13:43:59.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:43:59.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 -- 172.21.15.122:0/2524792262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764106980 msgr2=0x7f9764106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2524792262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764106980 0x7f9764106d60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f9760009930 tx=0x7f976002bc30 comp rx=0 tx=0).stop 2024-04-02T13:43:59.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 -- 172.21.15.122:0/2524792262 shutdown_connections 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2524792262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764101330 0x7f976410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2524792262 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764100970 0x7f9764100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2524792262 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764106980 0x7f9764106d60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 -- 172.21.15.122:0/2524792262 >> 172.21.15.122:0/2524792262 conn(0x7f9764077b00 msgr2=0x7f9764077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 -- 172.21.15.122:0/2524792262 shutdown_connections 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.331+0000 7f976c6e4640 1 -- 172.21.15.122:0/2524792262 wait complete. 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 Processor -- start 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 -- start start 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764100970 0x7f97641991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764106980 0x7f97641a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9764110d90 con 0x7f9764101330 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9764110c10 con 0x7f9764106980 2024-04-02T13:43:59.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976c6e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9764110f10 con 0x7f9764100970 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33634/0 (socket says 172.21.15.122:33634) 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 -- 172.21.15.122:0/2298676847 learned_addr learned my addr 172.21.15.122:0/2298676847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976a480640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764100970 0x7f97641991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f976ac81640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764106980 0x7f97641a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 -- 172.21.15.122:0/2298676847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764100970 msgr2=0x7f97641991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764100970 0x7f97641991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 -- 172.21.15.122:0/2298676847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764106980 msgr2=0x7f97641a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764106980 0x7f97641a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.332+0000 7f9769c7f640 1 -- 172.21.15.122:0/2298676847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97641a4790 con 0x7f9764101330 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9f0 tx=0x7f975400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f975b7fe640 1 -- 172.21.15.122:0/2298676847 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975400cd80 con 0x7f9764101330 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97641a4960 con 0x7f9764101330 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f975b7fe640 1 -- 172.21.15.122:0/2298676847 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9754010910 con 0x7f9764101330 2024-04-02T13:43:59.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f975b7fe640 1 -- 172.21.15.122:0/2298676847 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754018990 con 0x7f9764101330 2024-04-02T13:43:59.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.333+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97641a4eb0 con 0x7f9764101330 2024-04-02T13:43:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.335+0000 7f975b7fe640 1 -- 172.21.15.122:0/2298676847 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9754010ac0 con 0x7f9764101330 2024-04-02T13:43:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.335+0000 7f975b7fe640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 0x7f97500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f975b7fe640 1 -- 172.21.15.122:0/2298676847 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f97540d0f20 con 0x7f9764101330 2024-04-02T13:43:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976a480640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 0x7f97500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 Processor -- start 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 -- start start 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976a480640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 0x7f97500a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9760005ab0 tx=0x7f9760005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556746816df0 0x556746823b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55674677a7b0 con 0x5567468240b0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556746719570 con 0x5567468164d0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976f3a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556746776760 con 0x556746816df0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976ac81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976a480640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976ac81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33648/0 (socket says 172.21.15.122:33648) 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.336+0000 7f976ac81640 1 -- 172.21.15.122:0/1466028108 learned_addr learned my addr 172.21.15.122:0/1466028108 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f9769c7f640 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556746816df0 0x556746823b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 -- 172.21.15.122:0/1466028108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556746816df0 msgr2=0x556746823b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556746816df0 0x556746823b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 -- 172.21.15.122:0/1466028108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 msgr2=0x5567468264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 -- 172.21.15.122:0/1466028108 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55674648e3f0 con 0x5567468164d0 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976ac81640 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f976a480640 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468168b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9760009230 tx=0x7f976003f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb86dd28382b7b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f9758ff9640 1 -- 172.21.15.122:0/1466028108 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004180 con 0x5567468164d0 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f9758ff9640 1 -- 172.21.15.122:0/1466028108 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760004ba0 con 0x5567468164d0 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.337+0000 7f9758ff9640 1 -- 172.21.15.122:0/1466028108 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97600035f0 con 0x5567468164d0 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 -- 172.21.15.122:0/1466028108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 msgr2=0x5567468168b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468168b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9760009230 tx=0x7f976003f000 comp rx=0 tx=0).stop 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 -- 172.21.15.122:0/1466028108 shutdown_connections 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567468240b0 0x5567468264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556746816df0 0x556746823b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 --2- 172.21.15.122:0/1466028108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468168b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 -- 172.21.15.122:0/1466028108 >> 172.21.15.122:0/1466028108 conn(0x556746810ed0 msgr2=0x5567468112b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 -- 172.21.15.122:0/1466028108 shutdown_connections 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.338+0000 7f976f3a7740 1 -- 172.21.15.122:0/1466028108 wait complete. 2024-04-02T13:43:59.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 -- start start 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 Processor -- start 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468240b0 0x556746821a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556746826e10 con 0x556746816df0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556746735e80 con 0x5567468240b0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556746826f90 con 0x5567468164d0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976f3a7740 10 client.0 _ll_register_callbacks cb 0x556746771aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36928/0 (socket says 172.21.15.122:36928) 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33656/0 (socket says 172.21.15.122:33656) 2024-04-02T13:43:59.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 -- 172.21.15.122:0/3442583189 learned_addr learned my addr 172.21.15.122:0/3442583189 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976ac81640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468240b0 0x556746821a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 -- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468240b0 msgr2=0x556746821a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468240b0 0x556746821a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 -- 172.21.15.122:0/3442583189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 msgr2=0x5567468214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f976a480640 1 -- 172.21.15.122:0/3442583189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567465587d0 con 0x5567468164d0 2024-04-02T13:43:59.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.339+0000 7f9769c7f640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:43:59.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.340+0000 7f976a480640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f976002c170 tx=0x7f97600056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c196edb131459f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.340+0000 7f974dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.342+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760004180 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.342+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f9760004320 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.342+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9760004ba0 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f97600035f0 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 10 client.27166 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556746826e60 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9760004ba0 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9760034880 con 0x5567468164d0 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 10 client.27166 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 10 client.27166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 20 client.27166 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556746834100 0x5567468364f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 4 client.27166 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.343+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556746787cf0 con 0x556746834100 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.344+0000 7f9769c7f640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556746834100 0x5567468364f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.344+0000 7f9769c7f640 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556746834100 0x5567468364f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5822cd82c93ad1df server_cookie=4d3f1bb81f6e8105 in_seq=0 out_seq=0 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.344+0000 7f974dffb640 10 client.27166 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f974dffb640 1 -- 172.21.15.122:0/3442583189 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556746787cf0 con 0x556746834100 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f974dffb640 10 client.27166 handle_command_reply: tid=0 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 client.27166 shutdown 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 msgr2=0x556746820f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f976002c170 tx=0x7f97600056e0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 shutdown_connections 2024-04-02T13:43:59.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556746834100 0x5567468364f0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468240b0 0x556746821a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f974dffb640 0 client.27166 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x5567468214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 --2- 172.21.15.122:0/3442583189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468164d0 0x556746820f80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 >> 172.21.15.122:0/3442583189 conn(0x556746810ed0 msgr2=0x556746812c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 shutdown_connections 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 1 -- 172.21.15.122:0/3442583189 wait complete. 2024-04-02T13:43:59.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.345+0000 7f976f3a7740 20 client.27166 trim_cache size 0 max 16384 2024-04-02T13:43:59.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 Processor -- start 2024-04-02T13:43:59.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 -- start start 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674683bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ace90 con 0x556746816df0 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ae9d0 con 0x5567468164d0 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.358+0000 7f976f3a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ad560 con 0x5567468240b0 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33662/0 (socket says 172.21.15.122:33662) 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 -- 172.21.15.122:0/2424671089 learned_addr learned my addr 172.21.15.122:0/2424671089 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f976a480640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f976ac81640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674683bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 -- 172.21.15.122:0/2424671089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 msgr2=0x55674683bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674683bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 -- 172.21.15.122:0/2424671089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 msgr2=0x5567468236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468236b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 -- 172.21.15.122:0/2424671089 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556746816c20 con 0x556746816df0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f976ac81640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674683bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f9769c7f640 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9c0 tx=0x7f9754018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8fae2db1ab5a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f974ffff640 1 -- 172.21.15.122:0/2424671089 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975409ae80 con 0x556746816df0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f974ffff640 1 -- 172.21.15.122:0/2424671089 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97540991f0 con 0x556746816df0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.359+0000 7f974ffff640 1 -- 172.21.15.122:0/2424671089 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975409c2c0 con 0x556746816df0 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- 172.21.15.122:0/2424671089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 msgr2=0x55674682fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9c0 tx=0x7f9754018420 comp rx=0 tx=0).stop 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- 172.21.15.122:0/2424671089 shutdown_connections 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674683bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682fd10 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 --2- 172.21.15.122:0/2424671089 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x5567468236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- 172.21.15.122:0/2424671089 >> 172.21.15.122:0/2424671089 conn(0x556746810ed0 msgr2=0x55674680ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- 172.21.15.122:0/2424671089 shutdown_connections 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- 172.21.15.122:0/2424671089 wait complete. 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.360+0000 7f976f3a7740 1 -- start start 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 Processor -- start 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x55674682c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674681d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97641ae300 con 0x556746816df0 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f97641adc30 con 0x5567468164d0 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97641aae90 con 0x5567468240b0 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976f3a7740 10 client.0 _ll_register_callbacks cb 0x7f975c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:43:59.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33666/0 (socket says 172.21.15.122:33666) 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 -- 172.21.15.122:0/4004571392 learned_addr learned my addr 172.21.15.122:0/4004571392 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976a480640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x55674682c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976ac81640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674681d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 -- 172.21.15.122:0/4004571392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 msgr2=0x55674681d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674681d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 -- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 msgr2=0x55674682c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x55674682c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 -- 172.21.15.122:0/4004571392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55674681d760 con 0x556746816df0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f976ac81640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674681d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f9769c7f640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f975409cd40 tx=0x7f97540ce290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb24193e11926ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.361+0000 7f974dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97540a0e80 con 0x556746816df0 2024-04-02T13:43:59.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f9754018d40 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97540991f0 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f975409bfd0 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 10 client.18279 fetch_fsmap learned FSMap version 40 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556746899d20 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97540991f0 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f97540b8030 con 0x556746816df0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 10 client.18279 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 20 client.18279 populate_metadata read hostname 'smithi122' 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55674689a4a0 0x55674689c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 4 client.18279 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556746782040 con 0x55674689a4a0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.362+0000 7f976a480640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55674689a4a0 0x55674689c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:43:59.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f974dffb640 10 client.18279 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976a480640 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55674689a4a0 0x55674689c8c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb67d4542dae1fe2 server_cookie=ad73a5f9d15fa760 in_seq=0 out_seq=0 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f974dffb640 1 -- 172.21.15.122:0/4004571392 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556746782040 con 0x55674689a4a0 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f974dffb640 10 client.18279 handle_command_reply: tid=0 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 client.18279 shutdown 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 msgr2=0x55674682c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f975409cd40 tx=0x7f97540ce290 comp rx=0 tx=0).stop 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 shutdown_connections 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55674689a4a0 0x55674689c8c0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567468240b0 0x55674681d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556746816df0 0x55674682c760 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 --2- 172.21.15.122:0/4004571392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567468164d0 0x55674682c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 >> 172.21.15.122:0/4004571392 conn(0x556746810ed0 msgr2=0x5567468122b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f974dffb640 0 client.18279 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:43:59.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 shutdown_connections 2024-04-02T13:43:59.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 1 -- 172.21.15.122:0/4004571392 wait complete. 2024-04-02T13:43:59.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.363+0000 7f976f3a7740 20 client.18279 trim_cache size 0 max 16384 2024-04-02T13:43:59.376 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 msgr2=0x7f97500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 0x7f97500a61e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9760005ab0 tx=0x7f9760005fd0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 msgr2=0x7f9764199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f975400e9f0 tx=0x7f975400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 shutdown_connections 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f97500a3d20 0x7f97500a61e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9764106980 0x7f97641a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9764101330 0x7f9764199710 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 --2- 172.21.15.122:0/2298676847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9764100970 0x7f97641991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:43:59.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 >> 172.21.15.122:0/2298676847 conn(0x7f9764077b00 msgr2=0x7f9764103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:43:59.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 shutdown_connections 2024-04-02T13:43:59.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:43:59.364+0000 7f976c6e4640 1 -- 172.21.15.122:0/2298676847 wait complete. 2024-04-02T13:43:59.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:59 smithi122 ceph-mon[32114]: pgmap v271: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:43:59.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:43:59 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2836949599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:59 smithi187 ceph-mon[36064]: pgmap v271: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:43:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:43:59 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2836949599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:59 smithi188 ceph-mon[36117]: pgmap v271: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:43:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:43:59 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2836949599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:43:59.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 287647 3.17 MB/sec execute 320 sec latency 13.193 ms 2024-04-02T13:44:00.381 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:44:00.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:44:00.381 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:44:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- 172.21.15.122:0/392815229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 msgr2=0x7efc40100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 --2- 172.21.15.122:0/392815229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40100e20 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7efc30009900 tx=0x7efc3002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- 172.21.15.122:0/392815229 shutdown_connections 2024-04-02T13:44:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 --2- 172.21.15.122:0/392815229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc4010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 --2- 172.21.15.122:0/392815229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40100e20 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 --2- 172.21.15.122:0/392815229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc40105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- 172.21.15.122:0/392815229 >> 172.21.15.122:0/392815229 conn(0x7efc40077a90 msgr2=0x7efc40077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- 172.21.15.122:0/392815229 shutdown_connections 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- 172.21.15.122:0/392815229 wait complete. 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 Processor -- start 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.499+0000 7efc478eb640 1 -- start start 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc40198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc4019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc40110d60 con 0x7efc401009c0 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc40110be0 con 0x7efc401058a0 2024-04-02T13:44:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc40110ee0 con 0x7efc40101360 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33690/0 (socket says 172.21.15.122:33690) 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 -- 172.21.15.122:0/1967214074 learned_addr learned my addr 172.21.15.122:0/1967214074 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc44e86640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc40198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45e88640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc4019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 -- 172.21.15.122:0/1967214074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 msgr2=0x7efc40198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc40198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 -- 172.21.15.122:0/1967214074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 msgr2=0x7efc4019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc4019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 -- 172.21.15.122:0/1967214074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4019d400 con 0x7efc401009c0 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc44e86640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc40198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45e88640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc4019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc45687640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e9f0 tx=0x7efc3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc367fc640 1 -- 172.21.15.122:0/1967214074 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c00cd80 con 0x7efc401009c0 2024-04-02T13:44:00.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.500+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4019d6f0 con 0x7efc401009c0 2024-04-02T13:44:00.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.501+0000 7efc367fc640 1 -- 172.21.15.122:0/1967214074 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c010910 con 0x7efc401009c0 2024-04-02T13:44:00.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.501+0000 7efc367fc640 1 -- 172.21.15.122:0/1967214074 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0188f0 con 0x7efc401009c0 2024-04-02T13:44:00.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.501+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401a4f80 con 0x7efc401009c0 2024-04-02T13:44:00.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.503+0000 7efc367fc640 1 -- 172.21.15.122:0/1967214074 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7efc3c010ac0 con 0x7efc401009c0 2024-04-02T13:44:00.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 Processor -- start 2024-04-02T13:44:00.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 -- start start 2024-04-02T13:44:00.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df257123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25721fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df25676770 con 0x55df25712910 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df2561e960 con 0x55df2571fbd0 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55df256728c0 con 0x55df25711ff0 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc367fc640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 0x7efc2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33698/0 (socket says 172.21.15.122:33698) 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45687640 1 -- 172.21.15.122:0/1151877574 learned_addr learned my addr 172.21.15.122:0/1151877574 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc44e86640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25721fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df257123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 0x7efc2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 -- 172.21.15.122:0/1151877574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 msgr2=0x55df25721fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25721fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 -- 172.21.15.122:0/1151877574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 msgr2=0x55df2571f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45e88640 1 -- 172.21.15.122:0/1151877574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df254527e0 con 0x55df25711ff0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc45687640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.504+0000 7efc367fc640 1 -- 172.21.15.122:0/1967214074 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7efc3c0d0f20 con 0x7efc401009c0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.505+0000 7efc45e88640 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df257123d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc3800f510 tx=0x7efc3800f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49796124d9cd1403 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.505+0000 7efc45e88640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 0x7efc2c0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc380099e0 tx=0x7efc3801e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.505+0000 7efc2b7fe640 1 -- 172.21.15.122:0/1151877574 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc38004b20 con 0x55df25711ff0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.505+0000 7efc2b7fe640 1 -- 172.21.15.122:0/1151877574 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc38003e50 con 0x55df25711ff0 2024-04-02T13:44:00.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.505+0000 7efc2b7fe640 1 -- 172.21.15.122:0/1151877574 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc380045b0 con 0x55df25711ff0 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- 172.21.15.122:0/1151877574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 msgr2=0x55df257123d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df257123d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc3800f510 tx=0x7efc3800f9d0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- 172.21.15.122:0/1151877574 shutdown_connections 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25721fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/1151877574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df257123d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- 172.21.15.122:0/1151877574 >> 172.21.15.122:0/1151877574 conn(0x55df2570c9f0 msgr2=0x55df2570cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- 172.21.15.122:0/1151877574 shutdown_connections 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- 172.21.15.122:0/1151877574 wait complete. 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 -- start start 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.506+0000 7efc4a5ab740 1 Processor -- start 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df2571fbd0 0x55df2571d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df257228d0 con 0x55df25712910 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df25631d20 con 0x55df25711ff0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55df25722a50 con 0x55df2571fbd0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc4a5ab740 10 client.0 _ll_register_callbacks cb 0x55df255fea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc44e86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36746/0 (socket says 172.21.15.122:36746) 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 -- 172.21.15.122:0/3737497932 learned_addr learned my addr 172.21.15.122:0/3737497932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45e88640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df2571fbd0 0x55df2571d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 -- 172.21.15.122:0/3737497932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df2571fbd0 msgr2=0x55df2571d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df2571fbd0 0x55df2571d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 -- 172.21.15.122:0/3737497932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 msgr2=0x55df2571d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 -- 172.21.15.122:0/3737497932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df254b2f70 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc45687640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce330 tx=0x7efc3c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34060f47a392b9d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.507+0000 7efc28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0a0e80 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc38003e50 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7efc3c09bfd0 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efc3c09c1f0 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 10 client.27371 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df2534a170 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc38003e50 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7efc3c09ba40 con 0x55df25711ff0 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 10 client.27371 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 20 client.27371 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2572fe70 0x55df25732260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 4 client.27371 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.508+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df25683cb0 con 0x55df2572fe70 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.509+0000 7efc44e86640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2572fe70 0x55df25732260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.509+0000 7efc28ff9640 10 client.27371 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.509+0000 7efc44e86640 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2572fe70 0x55df25732260 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91424866e83ad58 server_cookie=5a69ac19b59e70db in_seq=0 out_seq=0 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc28ff9640 1 -- 172.21.15.122:0/3737497932 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df25683cb0 con 0x55df2572fe70 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc28ff9640 10 client.27371 handle_command_reply: tid=0 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 client.27371 shutdown 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 msgr2=0x55df2571cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce330 tx=0x7efc3c009310 comp rx=0 tx=0).stop 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 shutdown_connections 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2572fe70 0x55df25732260 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df2571fbd0 0x55df2571d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25712910 0x55df2571d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/3737497932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25711ff0 0x55df2571cc30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc28ff9640 0 client.27371 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 >> 172.21.15.122:0/3737497932 conn(0x55df2570c9f0 msgr2=0x55df2570e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 shutdown_connections 2024-04-02T13:44:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 1 -- 172.21.15.122:0/3737497932 wait complete. 2024-04-02T13:44:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.510+0000 7efc4a5ab740 20 client.27371 trim_cache size 0 max 16384 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 Processor -- start 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 -- start start 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 0x55df2572b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25737b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc401acec0 con 0x55df25711ff0 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc401aea00 con 0x55df2571fbd0 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc401ad590 con 0x55df25712910 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc45e88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25737b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc45687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33714/0 (socket says 172.21.15.122:33714) 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc44e86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 0x55df2572b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.531+0000 7efc45687640 1 -- 172.21.15.122:0/144499480 learned_addr learned my addr 172.21.15.122:0/144499480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 -- 172.21.15.122:0/144499480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 msgr2=0x55df25737b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25737b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 -- 172.21.15.122:0/144499480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 msgr2=0x55df2571f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 -- 172.21.15.122:0/144499480 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df2571bbe0 con 0x55df25712910 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc45687640 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc44e86640 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 0x55df2572b7d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc30000c00 tx=0x7efc3002cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ffe63e6ecaa856e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc2affd640 1 -- 172.21.15.122:0/144499480 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30004c70 con 0x55df25712910 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc2affd640 1 -- 172.21.15.122:0/144499480 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc30050070 con 0x55df25712910 2024-04-02T13:44:00.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.532+0000 7efc2affd640 1 -- 172.21.15.122:0/144499480 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc300051b0 con 0x55df25712910 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- 172.21.15.122:0/144499480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 msgr2=0x55df2572b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 0x55df2572b7d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc30000c00 tx=0x7efc3002cf90 comp rx=0 tx=0).stop 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- 172.21.15.122:0/144499480 shutdown_connections 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df2571fbd0 0x55df25737b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25712910 0x55df2572b7d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/144499480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df25711ff0 0x55df2571f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- 172.21.15.122:0/144499480 >> 172.21.15.122:0/144499480 conn(0x55df2570c9f0 msgr2=0x55df2570e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- 172.21.15.122:0/144499480 shutdown_connections 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- 172.21.15.122:0/144499480 wait complete. 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 -- start start 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.533+0000 7efc4a5ab740 1 Processor -- start 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df25716f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25712910 0x55df25727f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 0x55df257132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc401ae330 con 0x55df2571fbd0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc401adc60 con 0x55df25712910 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc401aaec0 con 0x55df25711ff0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc4a5ab740 10 client.0 _ll_register_callbacks cb 0x7efc3c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45e88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 0x55df257132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df25716f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45e88640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 0x55df257132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33724/0 (socket says 172.21.15.122:33724) 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45e88640 1 -- 172.21.15.122:0/734368246 learned_addr learned my addr 172.21.15.122:0/734368246 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc44e86640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25712910 0x55df25727f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 -- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25712910 msgr2=0x55df25727f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25712910 0x55df25727f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 -- 172.21.15.122:0/734368246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 msgr2=0x55df257132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 0x55df257132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.534+0000 7efc45687640 1 -- 172.21.15.122:0/734368246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df25713820 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc45687640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df25716f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce880 tx=0x7efc3c0bd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15a9fef440cdee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c010430 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7efc3c0105d0 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc30050070 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efc3c0b1080 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 10 client.27187 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55df2579b390 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c0b1220 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7efc3c087160 con 0x55df25711ff0 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 10 client.27187 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 10 client.27187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:00.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 20 client.27187 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2579bae0 0x55df2579ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 4 client.27187 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.535+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7efc30050070 con 0x55df2579bae0 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.536+0000 7efc44e86640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2579bae0 0x55df2579ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.536+0000 7efc28ff9640 10 client.27187 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.536+0000 7efc44e86640 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2579bae0 0x55df2579ded0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=405af77126eba246 server_cookie=2f6da0b64eee8c34 in_seq=0 out_seq=0 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc28ff9640 1 -- 172.21.15.122:0/734368246 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7efc30050070 con 0x55df2579bae0 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc28ff9640 10 client.27187 handle_command_reply: tid=0 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 client.27187 shutdown 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 msgr2=0x55df25716f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df25716f60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ce880 tx=0x7efc3c0bd040 comp rx=0 tx=0).stop 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 shutdown_connections 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55df2579bae0 0x55df2579ded0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df2571fbd0 0x55df257132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc28ff9640 0 client.27187 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df25712910 0x55df25727f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 --2- 172.21.15.122:0/734368246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df25711ff0 0x55df25716f60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 >> 172.21.15.122:0/734368246 conn(0x55df2570c9f0 msgr2=0x55df2570de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 shutdown_connections 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 1 -- 172.21.15.122:0/734368246 wait complete. 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.537+0000 7efc4a5ab740 20 client.27187 trim_cache size 0 max 16384 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "d1cea3c1-1a92-43dd-9afb-b4785fb065aa", 2024-04-02T13:44:00.547 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:44:00.548 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:44:00.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 msgr2=0x7efc2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 0x7efc2c0a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efc380099e0 tx=0x7efc3801e000 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 msgr2=0x7efc40198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e9f0 tx=0x7efc3c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 shutdown_connections 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efc2c0a3d20 0x7efc2c0a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc401058a0 0x7efc4019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc40101360 0x7efc40198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 --2- 172.21.15.122:0/1967214074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc401009c0 0x7efc40198410 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 >> 172.21.15.122:0/1967214074 conn(0x7efc40077a90 msgr2=0x7efc4006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 shutdown_connections 2024-04-02T13:44:00.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.538+0000 7efc478eb640 1 -- 172.21.15.122:0/1967214074 wait complete. 2024-04-02T13:44:00.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:00.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.672+0000 7f260aaf2640 1 -- 172.21.15.122:0/119635564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604101a90 msgr2=0x7f2604101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.672+0000 7f260aaf2640 1 --2- 172.21.15.122:0/119635564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604101a90 0x7f2604101f10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009930 tx=0x7f25f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.672+0000 7f260aaf2640 1 -- 172.21.15.122:0/119635564 shutdown_connections 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 --2- 172.21.15.122:0/119635564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604102450 0x7f260410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 --2- 172.21.15.122:0/119635564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604101a90 0x7f2604101f10 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 --2- 172.21.15.122:0/119635564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604107aa0 0x7f2604107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 -- 172.21.15.122:0/119635564 >> 172.21.15.122:0/119635564 conn(0x7f2604078ec0 msgr2=0x7f26040792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 -- 172.21.15.122:0/119635564 shutdown_connections 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 -- 172.21.15.122:0/119635564 wait complete. 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 Processor -- start 2024-04-02T13:44:00.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 -- start start 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 0x7f2604198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.673+0000 7f260aaf2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260aaf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604107aa0 0x7f260419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260aaf2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2604111eb0 con 0x7f2604107aa0 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260aaf2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2604111d30 con 0x7f2604102450 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260aaf2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2604112030 con 0x7f2604101a90 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260888e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 0x7f2604198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36780/0 (socket says 172.21.15.122:36780) 2024-04-02T13:44:00.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 -- 172.21.15.122:0/2274905311 learned_addr learned my addr 172.21.15.122:0/2274905311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260908f640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604107aa0 0x7f260419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 -- 172.21.15.122:0/2274905311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 msgr2=0x7f2604198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 0x7f2604198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 -- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604107aa0 msgr2=0x7f260419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604107aa0 0x7f260419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 -- 172.21.15.122:0/2274905311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260419d400 con 0x7f2604102450 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f260888e640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 0x7f2604198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.674+0000 7f25fbfff640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009230 tx=0x7f25f402cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.675+0000 7f25f9ffb640 1 -- 172.21.15.122:0/2274905311 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4041070 con 0x7f2604102450 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.675+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f260419d690 con 0x7f2604102450 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.675+0000 7f25f9ffb640 1 -- 172.21.15.122:0/2274905311 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f4004450 con 0x7f2604102450 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.675+0000 7f25f9ffb640 1 -- 172.21.15.122:0/2274905311 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4034440 con 0x7f2604102450 2024-04-02T13:44:00.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.675+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041a4f80 con 0x7f2604102450 2024-04-02T13:44:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.677+0000 7f260d7bb740 1 Processor -- start 2024-04-02T13:44:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.677+0000 7f260d7bb740 1 -- start start 2024-04-02T13:44:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d49f1c830 con 0x561d49fc61e0 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561d49ec4820 con 0x561d49fb82b0 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260d7bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561d49f187e0 con 0x561d49fb8c60 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f25f9ffb640 1 -- 172.21.15.122:0/2274905311 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f25f4044070 con 0x7f2604102450 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36782/0 (socket says 172.21.15.122:36782) 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 -- 172.21.15.122:0/1434651421 learned_addr learned my addr 172.21.15.122:0/1434651421 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f25fbfff640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260888e640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 -- 172.21.15.122:0/1434651421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 msgr2=0x561d49fc5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 -- 172.21.15.122:0/1434651421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 msgr2=0x561d49fc85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260908f640 1 -- 172.21.15.122:0/1434651421 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d49cfa7d0 con 0x561d49fb82b0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f25fbfff640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.678+0000 7f260888e640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f260908f640 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ee0 tx=0x7f25fc010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03ea12d7e9b38c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f25f9ffb640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 0x7f25ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f25eaffd640 1 -- 172.21.15.122:0/1434651421 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc016070 con 0x561d49fb82b0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f25eaffd640 1 -- 172.21.15.122:0/1434651421 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc004d20 con 0x561d49fb82b0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f25eaffd640 1 -- 172.21.15.122:0/1434651421 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc012810 con 0x561d49fb82b0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f260888e640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 0x7f25ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.679+0000 7f25f9ffb640 1 -- 172.21.15.122:0/2274905311 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f25f40f2520 con 0x7f2604102450 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- 172.21.15.122:0/1434651421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 msgr2=0x561d49fb8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009ee0 tx=0x7f25fc010740 comp rx=0 tx=0).stop 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260888e640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 0x7f25ec0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2600012640 tx=0x7f2600005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- 172.21.15.122:0/1434651421 shutdown_connections 2024-04-02T13:44:00.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1434651421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fb8690 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- 172.21.15.122:0/1434651421 >> 172.21.15.122:0/1434651421 conn(0x561d49fb3000 msgr2=0x561d49fb33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- 172.21.15.122:0/1434651421 shutdown_connections 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- 172.21.15.122:0/1434651421 wait complete. 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 -- start start 2024-04-02T13:44:00.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.680+0000 7f260d7bb740 1 Processor -- start 2024-04-02T13:44:00.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fc5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d49fc8fa0 con 0x561d49fc61e0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561d49ed7d30 con 0x561d49fb82b0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561d49fc9120 con 0x561d49fb8c60 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260d7bb740 10 client.0 _ll_register_callbacks cb 0x561d49ed7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260888e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fc5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33742/0 (socket says 172.21.15.122:33742) 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 -- 172.21.15.122:0/1435883428 learned_addr learned my addr 172.21.15.122:0/1435883428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 -- 172.21.15.122:0/1435883428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 msgr2=0x561d49fc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 -- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 msgr2=0x561d49fc5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fc5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 -- 172.21.15.122:0/1435883428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d49c303f0 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f260908f640 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a010 tx=0x7f25fc009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2222341f5ecc195 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.681+0000 7f25e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc028060 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f25fc0050d0 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc004d20 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f25fc02f050 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 10 client.18309 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d49bf00f0 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc004d20 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f25fc022030 con 0x561d49fc61e0 2024-04-02T13:44:00.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 10 client.18309 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 20 client.18309 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d49fd6090 0x561d49fd8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 4 client.18309 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d49f29d70 con 0x561d49fd6090 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.682+0000 7f260888e640 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d49fd6090 0x561d49fd8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.683+0000 7f25e3fff640 10 client.18309 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.683+0000 7f260888e640 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d49fd6090 0x561d49fd8480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b01c3e6b5c23321 server_cookie=72eb4cd9a059417e in_seq=0 out_seq=0 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.683+0000 7f25e3fff640 1 -- 172.21.15.122:0/1435883428 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d49f29d70 con 0x561d49fd6090 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.683+0000 7f25e3fff640 10 client.18309 handle_command_reply: tid=0 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.683+0000 7f260d7bb740 1 client.18309 shutdown 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 msgr2=0x561d49fc3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00a010 tx=0x7f25fc009c50 comp rx=0 tx=0).stop 2024-04-02T13:44:00.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 shutdown_connections 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d49fd6090 0x561d49fd8480 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fc3340 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f25e3fff640 0 client.18309 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 --2- 172.21.15.122:0/1435883428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fc5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 >> 172.21.15.122:0/1435883428 conn(0x561d49fb3000 msgr2=0x561d49fb4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 shutdown_connections 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 1 -- 172.21.15.122:0/1435883428 wait complete. 2024-04-02T13:44:00.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.684+0000 7f260d7bb740 20 client.18309 trim_cache size 0 max 16384 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.704+0000 7f260d7bb740 1 Processor -- start 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.704+0000 7f260d7bb740 1 -- start start 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fbe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26041acfa0 con 0x561d49fc61e0 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26041aeae0 con 0x561d49fb82b0 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260d7bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26041ad670 con 0x561d49fb8c60 2024-04-02T13:44:00.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33754/0 (socket says 172.21.15.122:33754) 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 -- 172.21.15.122:0/3205856952 learned_addr learned my addr 172.21.15.122:0/3205856952 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f25fbfff640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260888e640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fbe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 -- 172.21.15.122:0/3205856952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 msgr2=0x561d49fc1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 -- 172.21.15.122:0/3205856952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 msgr2=0x561d49fbe4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fbe4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 -- 172.21.15.122:0/3205856952 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d49fb8a00 con 0x561d49fc61e0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260888e640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fbe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f260908f640 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f25fc012fd0 tx=0x7f25fc005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9767e2056cb28cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f25ea7fc640 1 -- 172.21.15.122:0/3205856952 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc028030 con 0x561d49fc61e0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f25ea7fc640 1 -- 172.21.15.122:0/3205856952 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc01a7d0 con 0x561d49fc61e0 2024-04-02T13:44:00.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.705+0000 7f25ea7fc640 1 -- 172.21.15.122:0/3205856952 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc016040 con 0x561d49fc61e0 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 -- 172.21.15.122:0/3205856952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 msgr2=0x561d49fdb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f25fc012fd0 tx=0x7f25fc005a80 comp rx=0 tx=0).stop 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 -- 172.21.15.122:0/3205856952 shutdown_connections 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fdb4a0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fc1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3205856952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fbe4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 -- 172.21.15.122:0/3205856952 >> 172.21.15.122:0/3205856952 conn(0x561d49fb3000 msgr2=0x561d49fc16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 -- 172.21.15.122:0/3205856952 shutdown_connections 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.706+0000 7f260d7bb740 1 -- 172.21.15.122:0/3205856952 wait complete. 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 -- start start 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 Processor -- start 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fdb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fd8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f26041ae410 con 0x561d49fc61e0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f26041add40 con 0x561d49fb82b0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26041aafa0 con 0x561d49fb8c60 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260d7bb740 10 client.0 _ll_register_callbacks cb 0x7f25fc010f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f25fbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fd8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260908f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33760/0 (socket says 172.21.15.122:33760) 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260908f640 1 -- 172.21.15.122:0/3946881493 learned_addr learned my addr 172.21.15.122:0/3946881493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.707+0000 7f260888e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fdb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 -- 172.21.15.122:0/3946881493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 msgr2=0x561d49fd8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fd8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 -- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 msgr2=0x561d49fdb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fdb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 -- 172.21.15.122:0/3946881493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d49fd93c0 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25fbfff640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260908f640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005ca0 tx=0x7f25fc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a901161ce8a3f5ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc041060 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9525+0+0 (secure 0 0 0) 0x7f25fc01a5d0 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25fc01a7d0 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f25fc048050 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260d7bb740 10 client.18315 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561d4a03bfb0 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.708+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc01a7d0 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f25fc022070 con 0x561d49fc61e0 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 10 client.18315 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:00.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 20 client.18315 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d4a03c650 0x561d4a03ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 4 client.18315 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d49f240c0 con 0x561d4a03c650 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260888e640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d4a03c650 0x561d4a03ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f260888e640 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d4a03c650 0x561d4a03ea70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b04f465b8d79e server_cookie=493157b5e826ce49 in_seq=0 out_seq=0 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.709+0000 7f25e3fff640 10 client.18315 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f25e3fff640 1 -- 172.21.15.122:0/3946881493 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561d49f240c0 con 0x561d4a03c650 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f25e3fff640 10 client.18315 handle_command_reply: tid=0 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 client.18315 shutdown 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 msgr2=0x561d49fd8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005ca0 tx=0x7f25fc010740 comp rx=0 tx=0).stop 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 shutdown_connections 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d4a03c650 0x561d4a03ea70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d49fc61e0 0x561d49fd8e80 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d49fb8c60 0x561d49fd8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 --2- 172.21.15.122:0/3946881493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d49fb82b0 0x561d49fdb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 >> 172.21.15.122:0/3946881493 conn(0x561d49fb3000 msgr2=0x561d49fbe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f25e3fff640 0 client.18315 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 shutdown_connections 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 1 -- 172.21.15.122:0/3946881493 wait complete. 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.710+0000 7f260d7bb740 20 client.18315 trim_cache size 0 max 16384 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:44:00.719 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: "d1cea3c1-1a92-43dd-9afb-b4785fb065aa": { 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "d1cea3c1-1a92-43dd-9afb-b4785fb065aa", 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:44:00.720 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 msgr2=0x7f25ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 0x7f25ec0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2600012640 tx=0x7f2600005f90 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 msgr2=0x7f2604198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009230 tx=0x7f25f402cba0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 shutdown_connections 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f25ec0a3d20 0x7f25ec0a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2604107aa0 0x7f260419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2604102450 0x7f2604198950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 --2- 172.21.15.122:0/2274905311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2604101a90 0x7f2604198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:00.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 >> 172.21.15.122:0/2274905311 conn(0x7f2604078ec0 msgr2=0x7f26041045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:00.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 shutdown_connections 2024-04-02T13:44:00.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:00.711+0000 7f260aaf2640 1 -- 172.21.15.122:0/2274905311 wait complete. 2024-04-02T13:44:00.724 INFO:tasks.cephfs.filesystem:scrub status for tag:d1cea3c1-1a92-43dd-9afb-b4785fb065aa - {'path': '/', 'tag': 'd1cea3c1-1a92-43dd-9afb-b4785fb065aa', 'options': 'recursive,force'} 2024-04-02T13:44:00.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 288349 3.16 MB/sec execute 321 sec latency 89.911 ms 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: pgmap v272: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:01.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:01 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: pgmap v272: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:01 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: pgmap v272: 129 pgs: 129 active+clean; 205 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-04-02T13:44:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:01 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:01.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 288945 3.16 MB/sec execute 322 sec latency 64.059 ms 2024-04-02T13:44:02.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:44:02.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:02.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:02.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:02.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:02 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:02 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:02 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:02.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 290673 3.18 MB/sec execute 323 sec latency 49.409 ms 2024-04-02T13:44:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:03 smithi187 ceph-mon[36064]: pgmap v273: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-02T13:44:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:03 smithi187 ceph-mon[36064]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:44:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:03 smithi188 ceph-mon[36117]: pgmap v273: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-02T13:44:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:03 smithi188 ceph-mon[36117]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:44:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:03 smithi122 ceph-mon[32114]: pgmap v273: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 778 KiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-02T13:44:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:03 smithi122 ceph-mon[32114]: osdmap e100: 12 total, 12 up, 12 in 2024-04-02T13:44:03.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 291542 3.19 MB/sec execute 324 sec latency 149.746 ms 2024-04-02T13:44:04.967 INFO:tasks.workunit.client.0.smithi122.stdout: 1 291542 3.18 MB/sec execute 325 sec latency 1149.837 ms 2024-04-02T13:44:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:05 smithi187 ceph-mon[36064]: pgmap v275: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T13:44:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:05 smithi188 ceph-mon[36117]: pgmap v275: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T13:44:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:05 smithi122 ceph-mon[32114]: pgmap v275: 129 pgs: 129 active+clean; 209 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-02T13:44:05.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 291542 3.17 MB/sec execute 326 sec latency 2149.950 ms 2024-04-02T13:44:06.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 291542 3.16 MB/sec execute 327 sec latency 3150.027 ms 2024-04-02T13:44:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: pgmap v276: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:07 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: pgmap v276: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:07 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: pgmap v276: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:07.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:07 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:07.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 291542 3.15 MB/sec execute 328 sec latency 4150.103 ms 2024-04-02T13:44:08.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 292405 3.15 MB/sec execute 329 sec latency 4596.690 ms 2024-04-02T13:44:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:09 smithi187 ceph-mon[36064]: pgmap v277: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:09 smithi188 ceph-mon[36117]: pgmap v277: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:09 smithi122 ceph-mon[32114]: pgmap v277: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:09.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 294655 3.19 MB/sec execute 330 sec latency 13.701 ms 2024-04-02T13:44:10.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295090 3.18 MB/sec execute 331 sec latency 462.967 ms 2024-04-02T13:44:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:11 smithi187 ceph-mon[36064]: pgmap v278: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:11 smithi187 ceph-mon[36064]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:44:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:11 smithi188 ceph-mon[36117]: pgmap v278: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:11 smithi188 ceph-mon[36117]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:44:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:11 smithi122 ceph-mon[32114]: pgmap v278: 129 pgs: 129 active+clean; 211 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.6 MiB/s wr, 173 op/s 2024-04-02T13:44:11.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:11 smithi122 ceph-mon[32114]: osdmap e101: 12 total, 12 up, 12 in 2024-04-02T13:44:11.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295090 3.17 MB/sec execute 332 sec latency 1463.023 ms 2024-04-02T13:44:12.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295090 3.16 MB/sec execute 333 sec latency 2463.140 ms 2024-04-02T13:44:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:13 smithi187 ceph-mon[36064]: pgmap v280: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-04-02T13:44:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:13 smithi188 ceph-mon[36117]: pgmap v280: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-04-02T13:44:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:13 smithi122 ceph-mon[32114]: pgmap v280: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-04-02T13:44:13.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295214 3.15 MB/sec execute 334 sec latency 2911.346 ms 2024-04-02T13:44:14.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295214 3.14 MB/sec execute 335 sec latency 1345.802 ms 2024-04-02T13:44:15.622 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:15 smithi122 ceph-mon[32114]: pgmap v281: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-02T13:44:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:15 smithi187 ceph-mon[36064]: pgmap v281: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-02T13:44:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:15 smithi188 ceph-mon[36117]: pgmap v281: 129 pgs: 129 active+clean; 214 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 154 op/s 2024-04-02T13:44:15.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295214 3.13 MB/sec execute 336 sec latency 2345.914 ms 2024-04-02T13:44:16.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295214 3.12 MB/sec execute 337 sec latency 3345.958 ms 2024-04-02T13:44:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:17 smithi187 ceph-mon[36064]: pgmap v282: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:17 smithi188 ceph-mon[36117]: pgmap v282: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:17 smithi122 ceph-mon[32114]: pgmap v282: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:17.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 295214 3.11 MB/sec execute 338 sec latency 4346.020 ms 2024-04-02T13:44:18.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 296201 3.12 MB/sec execute 339 sec latency 4789.143 ms 2024-04-02T13:44:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:19 smithi187 ceph-mon[36064]: pgmap v283: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:19.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:19 smithi188 ceph-mon[36117]: pgmap v283: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:19 smithi122 ceph-mon[32114]: pgmap v283: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:19.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 298269 3.15 MB/sec execute 340 sec latency 15.268 ms 2024-04-02T13:44:20.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 299467 3.15 MB/sec execute 341 sec latency 15.659 ms 2024-04-02T13:44:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:21 smithi187 ceph-mon[36064]: pgmap v284: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:21.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:21 smithi188 ceph-mon[36117]: pgmap v284: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:21 smithi122 ceph-mon[32114]: pgmap v284: 129 pgs: 129 active+clean; 215 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 921 KiB/s wr, 86 op/s 2024-04-02T13:44:21.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 300954 3.17 MB/sec execute 342 sec latency 30.386 ms 2024-04-02T13:44:22.968 INFO:tasks.workunit.client.0.smithi122.stdout: 1 302211 3.18 MB/sec execute 343 sec latency 11.993 ms 2024-04-02T13:44:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:23 smithi187 ceph-mon[36064]: pgmap v285: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 927 KiB/s wr, 85 op/s 2024-04-02T13:44:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:44:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:23 smithi188 ceph-mon[36117]: pgmap v285: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 927 KiB/s wr, 85 op/s 2024-04-02T13:44:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:44:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:23 smithi122 ceph-mon[32114]: pgmap v285: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 927 KiB/s wr, 85 op/s 2024-04-02T13:44:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:44:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:44:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:44:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:44:23.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 303113 3.18 MB/sec execute 344 sec latency 342.797 ms 2024-04-02T13:44:24.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 305224 3.21 MB/sec execute 345 sec latency 14.301 ms 2024-04-02T13:44:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:25 smithi187 ceph-mon[36064]: pgmap v286: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 826 KiB/s wr, 76 op/s 2024-04-02T13:44:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:25 smithi188 ceph-mon[36117]: pgmap v286: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 826 KiB/s wr, 76 op/s 2024-04-02T13:44:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:25 smithi122 ceph-mon[32114]: pgmap v286: 129 pgs: 129 active+clean; 219 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 826 KiB/s wr, 76 op/s 2024-04-02T13:44:25.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 306324 3.21 MB/sec execute 346 sec latency 19.168 ms 2024-04-02T13:44:26.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 308698 3.25 MB/sec execute 347 sec latency 15.577 ms 2024-04-02T13:44:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:27 smithi187 ceph-mon[36064]: pgmap v287: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-02T13:44:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:27 smithi188 ceph-mon[36117]: pgmap v287: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-02T13:44:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:27 smithi122 ceph-mon[32114]: pgmap v287: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-02T13:44:27.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 309706 3.25 MB/sec execute 348 sec latency 21.082 ms 2024-04-02T13:44:28.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 311787 3.27 MB/sec execute 349 sec latency 16.002 ms 2024-04-02T13:44:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:29 smithi187 ceph-mon[36064]: pgmap v288: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:29 smithi188 ceph-mon[36117]: pgmap v288: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:29 smithi122 ceph-mon[32114]: pgmap v288: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:29.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 312982 3.28 MB/sec execute 350 sec latency 12.377 ms 2024-04-02T13:44:30.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:30.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- 172.21.15.122:0/52773459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 msgr2=0x7f732410e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 --2- 172.21.15.122:0/52773459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f732410e970 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f731c009cd0 tx=0x7f731c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- 172.21.15.122:0/52773459 shutdown_connections 2024-04-02T13:44:30.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 --2- 172.21.15.122:0/52773459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f732410e970 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 --2- 172.21.15.122:0/52773459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f7324101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 --2- 172.21.15.122:0/52773459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 0x7f7324107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- 172.21.15.122:0/52773459 >> 172.21.15.122:0/52773459 conn(0x7f73240fd530 msgr2=0x7f73240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- 172.21.15.122:0/52773459 shutdown_connections 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- 172.21.15.122:0/52773459 wait complete. 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 Processor -- start 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 -- start start 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.836+0000 7f732b660640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f73241983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 0x7f732419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7324111ec0 con 0x7f7324102460 2024-04-02T13:44:30.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7324111d40 con 0x7f7324101aa0 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7324112040 con 0x7f7324107ab0 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f73293fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f73241983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f73293fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f73241983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43012/0 (socket says 172.21.15.122:43012) 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46322/0 (socket says 172.21.15.122:46322) 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f73293fc640 1 -- 172.21.15.122:0/2007871774 learned_addr learned my addr 172.21.15.122:0/2007871774 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7329bfd640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 0x7f732419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 -- 172.21.15.122:0/2007871774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 msgr2=0x7f732419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 0x7f732419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 -- 172.21.15.122:0/2007871774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 msgr2=0x7f73241983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f73241983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 -- 172.21.15.122:0/2007871774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419d390 con 0x7f7324102460 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f7328bfb640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f731400d6d0 tx=0x7f731400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f731a7fc640 1 -- 172.21.15.122:0/2007871774 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314017070 con 0x7f7324102460 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f731a7fc640 1 -- 172.21.15.122:0/2007871774 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314009820 con 0x7f7324102460 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419d680 con 0x7f7324102460 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f731a7fc640 1 -- 172.21.15.122:0/2007871774 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314004d00 con 0x7f7324102460 2024-04-02T13:44:30.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.837+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a4f10 con 0x7f7324102460 2024-04-02T13:44:30.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.840+0000 7f731a7fc640 1 -- 172.21.15.122:0/2007871774 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f73140099d0 con 0x7f7324102460 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 Processor -- start 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 -- start start 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f731a7fc640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 0x7f73100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ec091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e0eb61830 con 0x561e0ec06dd0 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561e0eb09820 con 0x561e0ebfe1f0 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f732e246740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561e0eb5d7e0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f73293fc640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 0x7f73100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f7329bfd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f731a7fc640 1 -- 172.21.15.122:0/2007871774 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f73140d06b0 con 0x7f7324102460 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f7329bfd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:50040/0 (socket says 172.21.15.122:50040) 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f7329bfd640 1 -- 172.21.15.122:0/388742862 learned_addr learned my addr 172.21.15.122:0/388742862 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f7329bfd640 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ec091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.841+0000 7f7328bfb640 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 -- 172.21.15.122:0/388742862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 msgr2=0x561e0ec06890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec06890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 -- 172.21.15.122:0/388742862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 msgr2=0x561e0ec091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ec091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 -- 172.21.15.122:0/388742862 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0e93f7d0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f73293fc640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 0x7f73100a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f732000bc20 tx=0x7f732000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f7329bfd640 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f731c004af0 tx=0x7f731c02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a303fbec29186ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f730f7fe640 1 -- 172.21.15.122:0/388742862 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c039040 con 0x561e0ebfd8d0 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f730f7fe640 1 -- 172.21.15.122:0/388742862 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c003970 con 0x561e0ebfd8d0 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.842+0000 7f730f7fe640 1 -- 172.21.15.122:0/388742862 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c004230 con 0x561e0ebfd8d0 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- 172.21.15.122:0/388742862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 msgr2=0x561e0ebfdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f731c004af0 tx=0x7f731c02e040 comp rx=0 tx=0).stop 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- 172.21.15.122:0/388742862 shutdown_connections 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ec091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 --2- 172.21.15.122:0/388742862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ebfdcb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- 172.21.15.122:0/388742862 >> 172.21.15.122:0/388742862 conn(0x561e0ebf8000 msgr2=0x561e0ebf83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- 172.21.15.122:0/388742862 shutdown_connections 2024-04-02T13:44:30.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- 172.21.15.122:0/388742862 wait complete. 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 -- start start 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.843+0000 7f732e246740 1 Processor -- start 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfe1f0 0x561e0ec039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ec06dd0 0x561e0ec03ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561e0ec09b00 con 0x561e0ebfd8d0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561e0eb1cd30 con 0x561e0ec06dd0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561e0ec09c80 con 0x561e0ebfe1f0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 10 client.0 _ll_register_callbacks cb 0x561e0eb1c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46344/0 (socket says 172.21.15.122:46344) 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 -- 172.21.15.122:0/1438084225 learned_addr learned my addr 172.21.15.122:0/1438084225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f7329bfd640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ec06dd0 0x561e0ec03ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f7328bfb640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfe1f0 0x561e0ec039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 -- 172.21.15.122:0/1438084225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfe1f0 msgr2=0x561e0ec039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfe1f0 0x561e0ec039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 -- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ec06dd0 msgr2=0x561e0ec03ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ec06dd0 0x561e0ec03ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 -- 172.21.15.122:0/1438084225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0e8753f0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f73293fc640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f73200116c0 tx=0x7f7320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507276e153ed1acd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f730cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732001e8f0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f732001ea90 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.844+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f731c003970 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7320030050 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 10 client.18327 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561e0e8350f0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f731c003970 con 0x561e0ebfd8d0 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f732001d3b0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 10 client.18327 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 10 client.18327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 20 client.18327 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec16c80 0x561e0ec190d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 4 client.18327 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e0eb6ed70 con 0x561e0ec16c80 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.845+0000 7f7328bfb640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec16c80 0x561e0ec190d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.846+0000 7f7328bfb640 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec16c80 0x561e0ec190d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b82bd439abd9ea server_cookie=dd99041d5b59bcd2 in_seq=0 out_seq=0 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.846+0000 7f730cff9640 10 client.18327 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.846+0000 7f730cff9640 1 -- 172.21.15.122:0/1438084225 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e0eb6ed70 con 0x561e0ec16c80 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.846+0000 7f730cff9640 10 client.18327 handle_command_reply: tid=0 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.846+0000 7f732e246740 1 client.18327 shutdown 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 msgr2=0x561e0ec06610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f73200116c0 tx=0x7f7320004730 comp rx=0 tx=0).stop 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 shutdown_connections 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec16c80 0x561e0ec190d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ec06dd0 0x561e0ec03ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfe1f0 0x561e0ec039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f730cff9640 0 client.18327 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 --2- 172.21.15.122:0/1438084225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec06610 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 >> 172.21.15.122:0/1438084225 conn(0x561e0ebf8000 msgr2=0x561e0ebf9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 shutdown_connections 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 1 -- 172.21.15.122:0/1438084225 wait complete. 2024-04-02T13:44:30.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.847+0000 7f732e246740 20 client.18327 trim_cache size 0 max 16384 2024-04-02T13:44:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 Processor -- start 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 -- start start 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec1e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 0x561e0ec1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73241af180 con 0x561e0ebfd8d0 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b0cc0 con 0x561e0ebfe1f0 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f732e246740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73241af850 con 0x561e0ec06dd0 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f73293fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f73293fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46360/0 (socket says 172.21.15.122:46360) 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f73293fc640 1 -- 172.21.15.122:0/4097191988 learned_addr learned my addr 172.21.15.122:0/4097191988 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.866+0000 7f7328bfb640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec1e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f7329bfd640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 0x561e0ec1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 -- 172.21.15.122:0/4097191988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 msgr2=0x561e0ec1eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 0x561e0ec1eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 -- 172.21.15.122:0/4097191988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 msgr2=0x561e0ec1e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec1e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 -- 172.21.15.122:0/4097191988 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e0ebf7e30 con 0x561e0ebfd8d0 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f7329bfd640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 0x561e0ec1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f7328bfb640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec1e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f73293fc640 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7320011fd0 tx=0x7f7320010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf59b550de31e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f730effd640 1 -- 172.21.15.122:0/4097191988 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320038070 con 0x561e0ebfd8d0 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f730effd640 1 -- 172.21.15.122:0/4097191988 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732001e6f0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.867+0000 7f730effd640 1 -- 172.21.15.122:0/4097191988 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73200276c0 con 0x561e0ebfd8d0 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 -- 172.21.15.122:0/4097191988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 msgr2=0x561e0ec129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f7320011fd0 tx=0x7f7320010d10 comp rx=0 tx=0).stop 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 -- 172.21.15.122:0/4097191988 shutdown_connections 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ec06dd0 0x561e0ec1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec1e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 --2- 172.21.15.122:0/4097191988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ebfd8d0 0x561e0ec129c0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.868+0000 7f732e246740 1 -- 172.21.15.122:0/4097191988 >> 172.21.15.122:0/4097191988 conn(0x561e0ebf8000 msgr2=0x561e0ec02b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- 172.21.15.122:0/4097191988 shutdown_connections 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- 172.21.15.122:0/4097191988 wait complete. 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- start start 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 Processor -- start 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ec0eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec0f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b05f0 con 0x561e0ec06dd0 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f73241aff20 con 0x561e0ebfe1f0 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73241ad180 con 0x561e0ebfd8d0 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f732e246740 10 client.0 _ll_register_callbacks cb 0x7f732000bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f7329bfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f7329bfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46370/0 (socket says 172.21.15.122:46370) 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f73293fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ec0eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.869+0000 7f7329bfd640 1 -- 172.21.15.122:0/431272132 learned_addr learned my addr 172.21.15.122:0/431272132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7328bfb640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec0f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 -- 172.21.15.122:0/431272132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 msgr2=0x561e0ec0eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ec0eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 -- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 msgr2=0x561e0ec0f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec0f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 -- 172.21.15.122:0/431272132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e0ec007f0 con 0x561e0ec06dd0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f73293fc640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ec0eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f7329bfd640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f80 tx=0x7f731c032a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e525885471c449f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f730cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731c052060 con 0x561e0ec06dd0 2024-04-02T13:44:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f731c003b70 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732001e6f0 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f731c059050 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f732e246740 10 client.18333 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561e0ec7cd20 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.870+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732001e6f0 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f731c03f070 con 0x561e0ec06dd0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 10 client.18333 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 20 client.18333 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec7d000 0x561e0ec7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 4 client.18333 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e0eb690c0 con 0x561e0ec7d000 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f73293fc640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec7d000 0x561e0ec7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f730cff9640 10 client.18333 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:30.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f73293fc640 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec7d000 0x561e0ec7f3f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d9e5a01f8002d8 server_cookie=3195ee598e4f6888 in_seq=0 out_seq=0 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f730cff9640 1 -- 172.21.15.122:0/431272132 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561e0eb690c0 con 0x561e0ec7d000 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f730cff9640 10 client.18333 handle_command_reply: tid=0 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.871+0000 7f732e246740 1 client.18333 shutdown 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 msgr2=0x561e0ebfffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f731c000f80 tx=0x7f731c032a60 comp rx=0 tx=0).stop 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 shutdown_connections 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561e0ec7d000 0x561e0ec7f3f0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561e0ec06dd0 0x561e0ebfffc0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561e0ebfe1f0 0x561e0ec0f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f730cff9640 0 client.18333 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 --2- 172.21.15.122:0/431272132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561e0ebfd8d0 0x561e0ec0eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 >> 172.21.15.122:0/431272132 conn(0x561e0ebf8000 msgr2=0x561e0ebf69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 shutdown_connections 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 1 -- 172.21.15.122:0/431272132 wait complete. 2024-04-02T13:44:30.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732e246740 20 client.18333 trim_cache size 0 max 16384 2024-04-02T13:44:30.881 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:44:30.881 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:44:30.881 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:44:30.881 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 msgr2=0x7f73100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 0x7f73100a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f732000bc20 tx=0x7f732000b410 comp rx=0 tx=0).stop 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 msgr2=0x7f73241988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f731400d6d0 tx=0x7f731400db90 comp rx=0 tx=0).stop 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 shutdown_connections 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f73100a3d20 0x7f73100a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7324107ab0 0x7f732419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7324102460 0x7f73241988e0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 --2- 172.21.15.122:0/2007871774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7324101aa0 0x7f73241983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:30.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.872+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 >> 172.21.15.122:0/2007871774 conn(0x7f73240fd530 msgr2=0x7f73240fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:30.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.873+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 shutdown_connections 2024-04-02T13:44:30.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.873+0000 7f732b660640 1 -- 172.21.15.122:0/2007871774 wait complete. 2024-04-02T13:44:30.887 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:44:30.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:44:30.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 314751 3.29 MB/sec execute 351 sec latency 22.586 ms 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 -- 172.21.15.122:0/3606766798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0106a10 msgr2=0x7fefe0106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 --2- 172.21.15.122:0/3606766798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0106a10 0x7fefe0106df0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009930 tx=0x7fefdc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 -- 172.21.15.122:0/3606766798 shutdown_connections 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 --2- 172.21.15.122:0/3606766798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 --2- 172.21.15.122:0/3606766798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0100a00 0x7fefe0100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 --2- 172.21.15.122:0/3606766798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0106a10 0x7fefe0106df0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.998+0000 7fefe7904640 1 -- 172.21.15.122:0/3606766798 >> 172.21.15.122:0/3606766798 conn(0x7fefe0077b00 msgr2=0x7fefe0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- 172.21.15.122:0/3606766798 shutdown_connections 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- 172.21.15.122:0/3606766798 wait complete. 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 Processor -- start 2024-04-02T13:44:31.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- start start 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 0x7fefe019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0110e20 con 0x7fefe0100a00 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0110ca0 con 0x7fefe01013c0 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe7904640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fefe0110fa0 con 0x7fefe0106a10 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:30.999+0000 7fefe56a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46382/0 (socket says 172.21.15.122:46382) 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 -- 172.21.15.122:0/324395653 learned_addr learned my addr 172.21.15.122:0/324395653 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe5ea1640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 0x7fefe019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe4e9f640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 -- 172.21.15.122:0/324395653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 msgr2=0x7fefe019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 0x7fefe019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 -- 172.21.15.122:0/324395653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 msgr2=0x7fefe0198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe0198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 -- 172.21.15.122:0/324395653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe019d3e0 con 0x7fefe0100a00 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe4e9f640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe5ea1640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 0x7fefe019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe56a0640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009230 tx=0x7fefdc02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc041070 con 0x7fefe0100a00 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefdc004450 con 0x7fefe0100a00 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe019d670 con 0x7fefe0100a00 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc003e50 con 0x7fefe0100a00 2024-04-02T13:44:31.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.000+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe01a4f60 con 0x7fefe0100a00 2024-04-02T13:44:31.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.002+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefe0102140 con 0x7fefe0100a00 2024-04-02T13:44:31.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.003+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fefdc044070 con 0x7fefe0100a00 2024-04-02T13:44:31.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.003+0000 7fefd67fc640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 0x7fefcc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.004+0000 7fefe4e9f640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 0x7fefcc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.004+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fefdc0f24e0 con 0x7fefe0100a00 2024-04-02T13:44:31.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.004+0000 7fefe4e9f640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 0x7fefcc0a6380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefe0069d40 tx=0x7fefd0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.008+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fefdc0be8a0 con 0x7fefe0100a00 2024-04-02T13:44:31.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.308+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefe01a5130 con 0x7fefe0100a00 2024-04-02T13:44:31.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.309+0000 7fefd67fc640 1 -- 172.21.15.122:0/324395653 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7fefdc0be1a0 con 0x7fefe0100a00 2024-04-02T13:44:31.311 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:44:31.311 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:44:31.312 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:44:31.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 msgr2=0x7fefcc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 0x7fefcc0a6380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fefe0069d40 tx=0x7fefd0005f90 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 msgr2=0x7fefe01983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009230 tx=0x7fefdc02cb30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 shutdown_connections 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fefcc0a3ec0 0x7fefcc0a6380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe0106a10 0x7fefe019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fefe01013c0 0x7fefe0198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 --2- 172.21.15.122:0/324395653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefe0100a00 0x7fefe01983f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 >> 172.21.15.122:0/324395653 conn(0x7fefe0077b00 msgr2=0x7fefe00ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 shutdown_connections 2024-04-02T13:44:31.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.311+0000 7fefe7904640 1 -- 172.21.15.122:0/324395653 wait complete. 2024-04-02T13:44:31.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 -- 172.21.15.122:0/1268911020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c101350 msgr2=0x7f487c10d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 --2- 172.21.15.122:0/1268911020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c101350 0x7f487c10d860 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4874009cd0 tx=0x7f487402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 -- 172.21.15.122:0/1268911020 shutdown_connections 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 --2- 172.21.15.122:0/1268911020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c101350 0x7f487c10d860 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 --2- 172.21.15.122:0/1268911020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 0x7f487c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 --2- 172.21.15.122:0/1268911020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c1069a0 0x7f487c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 -- 172.21.15.122:0/1268911020 >> 172.21.15.122:0/1268911020 conn(0x7f487c077ad0 msgr2=0x7f487c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 -- 172.21.15.122:0/1268911020 shutdown_connections 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.437+0000 7f4880daf640 1 -- 172.21.15.122:0/1268911020 wait complete. 2024-04-02T13:44:31.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 Processor -- start 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 -- start start 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 0x7f487c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c101350 0x7f487c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f487c110db0 con 0x7f487c1069a0 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f487c110c30 con 0x7f487c100990 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4880daf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f487c110f30 con 0x7f487c101350 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46396/0 (socket says 172.21.15.122:46396) 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 0x7f487c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 -- 172.21.15.122:0/699868453 learned_addr learned my addr 172.21.15.122:0/699868453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f4873fff640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c101350 0x7f487c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 -- 172.21.15.122:0/699868453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c101350 msgr2=0x7f487c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c101350 0x7f487c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 -- 172.21.15.122:0/699868453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 msgr2=0x7f487c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 0x7f487c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.438+0000 7f487ad9d640 1 -- 172.21.15.122:0/699868453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c19c1b0 con 0x7f487c1069a0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f487ad9d640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4874004af0 tx=0x7f4874032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f48737fe640 1 -- 172.21.15.122:0/699868453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874034740 con 0x7f487c1069a0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c19c440 con 0x7f487c1069a0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f48737fe640 1 -- 172.21.15.122:0/699868453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4874032590 con 0x7f487c1069a0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f48737fe640 1 -- 172.21.15.122:0/699868453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874004040 con 0x7f487c1069a0 2024-04-02T13:44:31.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.439+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c06a110 con 0x7f487c1069a0 2024-04-02T13:44:31.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f48737fe640 1 -- 172.21.15.122:0/699868453 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4874032740 con 0x7f487c1069a0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 Processor -- start 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 -- start start 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320315760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320318090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3202707b0 con 0x55b32030c7a0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b32020f570 con 0x55b32030d0c0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4883999740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b32026c760 con 0x55b320315ca0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f48737fe640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 0x7f48680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f487a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320315760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46412/0 (socket says 172.21.15.122:46412) 2024-04-02T13:44:31.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f487a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320315760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43072/0 (socket says 172.21.15.122:43072) 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4873fff640 1 -- 172.21.15.122:0/1671725101 learned_addr learned my addr 172.21.15.122:0/1671725101 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f4873fff640 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320318090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.442+0000 7f487ad9d640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 0x7f48680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f48737fe640 1 -- 172.21.15.122:0/699868453 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f48740f4670 con 0x7f487c1069a0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 -- 172.21.15.122:0/1671725101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 msgr2=0x55b320318090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320318090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 -- 172.21.15.122:0/1671725101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 msgr2=0x55b320315760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320315760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 -- 172.21.15.122:0/1671725101 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31ff843f0 con 0x55b32030c7a0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f487ad9d640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 0x7f48680a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48740f1b90 tx=0x7f48740bc060 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4873fff640 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f486400af30 tx=0x7f486400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b0a3b9f4d8cdcc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4870ff9640 1 -- 172.21.15.122:0/1671725101 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486400f830 con 0x55b32030c7a0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4870ff9640 1 -- 172.21.15.122:0/1671725101 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486401b400 con 0x55b32030c7a0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.443+0000 7f4870ff9640 1 -- 172.21.15.122:0/1671725101 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486400e610 con 0x55b32030c7a0 2024-04-02T13:44:31.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 -- 172.21.15.122:0/1671725101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 msgr2=0x55b32030cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f486400af30 tx=0x7f486400a710 comp rx=0 tx=0).stop 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 -- 172.21.15.122:0/1671725101 shutdown_connections 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320318090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320315760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 --2- 172.21.15.122:0/1671725101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32030cb80 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 -- 172.21.15.122:0/1671725101 >> 172.21.15.122:0/1671725101 conn(0x55b320306ed0 msgr2=0x55b3203072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 -- 172.21.15.122:0/1671725101 shutdown_connections 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.444+0000 7f4883999740 1 -- 172.21.15.122:0/1671725101 wait complete. 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 -- start start 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 Processor -- start 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320313040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320313580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b320318a90 con 0x55b32030c7a0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b32022be80 con 0x55b32030d0c0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b320318c10 con 0x55b320315ca0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4883999740 10 client.0 _ll_register_callbacks cb 0x55b320267aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46422/0 (socket says 172.21.15.122:46422) 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f487a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320313040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 -- 172.21.15.122:0/1733071148 learned_addr learned my addr 172.21.15.122:0/1733071148 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 -- 172.21.15.122:0/1733071148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 msgr2=0x55b320313580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320313580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 -- 172.21.15.122:0/1733071148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 msgr2=0x55b320313040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320313040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 -- 172.21.15.122:0/1733071148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32004e7d0 con 0x55b32030c7a0 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4873fff640 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f486401be10 tx=0x7f486400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a846a960ce32d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.445+0000 7f4861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486402a060 con 0x55b32030c7a0 2024-04-02T13:44:31.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f486400f830 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486401b400 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f486400fa30 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 10 client.18351 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b320324b60 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486401b400 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f486400e3b0 con 0x55b32030c7a0 2024-04-02T13:44:31.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 10 client.18351 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 10 client.18351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 20 client.18351 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b320325eb0 0x55b3203282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 4 client.18351 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b32027dcf0 con 0x55b320325eb0 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.446+0000 7f487a59c640 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b320325eb0 0x55b3203282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.447+0000 7f4861ffb640 10 client.18351 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.447+0000 7f487a59c640 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b320325eb0 0x55b3203282a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5a1fa23dbd2c02f7 server_cookie=acb5451e2ec71304 in_seq=0 out_seq=0 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.447+0000 7f4861ffb640 1 -- 172.21.15.122:0/1733071148 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b32027dcf0 con 0x55b320325eb0 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.447+0000 7f4861ffb640 10 client.18351 handle_command_reply: tid=0 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.447+0000 7f4883999740 1 client.18351 shutdown 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 msgr2=0x55b320312b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f486401be10 tx=0x7f486400acf0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 shutdown_connections 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b320325eb0 0x55b3203282a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4861ffb640 0 client.18351 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b320313580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b320313040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 --2- 172.21.15.122:0/1733071148 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320312b00 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 >> 172.21.15.122:0/1733071148 conn(0x55b320306ed0 msgr2=0x55b320308c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 shutdown_connections 2024-04-02T13:44:31.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 1 -- 172.21.15.122:0/1733071148 wait complete. 2024-04-02T13:44:31.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.448+0000 7f4883999740 20 client.18351 trim_cache size 0 max 16384 2024-04-02T13:44:31.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 Processor -- start 2024-04-02T13:44:31.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 -- start start 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b32030d0c0 0x55b320321950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b320315ca0 0x55b3203119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1ace00 con 0x55b32030c7a0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1ae940 con 0x55b320315ca0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4883999740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1ad4d0 con 0x55b32030d0c0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46432/0 (socket says 172.21.15.122:46432) 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 -- 172.21.15.122:0/3565350004 learned_addr learned my addr 172.21.15.122:0/3565350004 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f487a59c640 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b32030d0c0 0x55b320321950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b320315ca0 0x55b3203119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 -- 172.21.15.122:0/3565350004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b32030d0c0 msgr2=0x55b320321950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b32030d0c0 0x55b320321950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 -- 172.21.15.122:0/3565350004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b320315ca0 msgr2=0x55b3203119b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b320315ca0 0x55b3203119b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 -- 172.21.15.122:0/3565350004 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b320318970 con 0x55b32030c7a0 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.463+0000 7f4873fff640 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x55b320321ed0 tx=0x7f4864000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e91c8f90ffc360b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4863fff640 1 -- 172.21.15.122:0/3565350004 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486402a070 con 0x55b32030c7a0 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4863fff640 1 -- 172.21.15.122:0/3565350004 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486400a7e0 con 0x55b32030c7a0 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4863fff640 1 -- 172.21.15.122:0/3565350004 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486400f630 con 0x55b32030c7a0 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 -- 172.21.15.122:0/3565350004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 msgr2=0x55b320315230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x55b320321ed0 tx=0x7f4864000bc0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 -- 172.21.15.122:0/3565350004 shutdown_connections 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b320315ca0 0x55b3203119b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b32030d0c0 0x55b320321950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 --2- 172.21.15.122:0/3565350004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b320315230 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 -- 172.21.15.122:0/3565350004 >> 172.21.15.122:0/3565350004 conn(0x55b320306ed0 msgr2=0x55b3203112a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 -- 172.21.15.122:0/3565350004 shutdown_connections 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.464+0000 7f4883999740 1 -- 172.21.15.122:0/3565350004 wait complete. 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 -- start start 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 Processor -- start 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b32031e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b32030f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1ae270 con 0x55b32030c7a0 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1adba0 con 0x55b32030d0c0 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1aae00 con 0x55b320315ca0 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4883999740 10 client.0 _ll_register_callbacks cb 0x7f486c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46440/0 (socket says 172.21.15.122:46440) 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 -- 172.21.15.122:0/4245255357 learned_addr learned my addr 172.21.15.122:0/4245255357 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f487a59c640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b32031e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b32030f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 -- 172.21.15.122:0/4245255357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 msgr2=0x55b32030f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b32030f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 -- 172.21.15.122:0/4245255357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 msgr2=0x55b32031e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b32031e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f4873fff640 1 -- 172.21.15.122:0/4245255357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32030f9c0 con 0x55b32030c7a0 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.465+0000 7f487a59c640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b32031e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4873fff640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f486401bac0 tx=0x7f486401bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73bcbeb70a633e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486403c070 con 0x55b32030c7a0 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f486400fdd0 con 0x55b32030c7a0 2024-04-02T13:44:31.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f486400a7e0 con 0x55b32030c7a0 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f486402a070 con 0x55b32030c7a0 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 10 client.18357 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b32038bd30 con 0x55b32030c7a0 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f486400a7e0 con 0x55b32030c7a0 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f48640389f0 con 0x55b32030c7a0 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 10 client.18357 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 10 client.18357 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 20 client.18357 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b32038c330 0x55b32038e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 4 client.18357 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.466+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b320278040 con 0x55b32038c330 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f487a59c640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b32038c330 0x55b32038e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4861ffb640 10 client.18357 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:44:31.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f487a59c640 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b32038c330 0x55b32038e720 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e509b1112a6c9efc server_cookie=dde83a06fad5cf22 in_seq=0 out_seq=0 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4861ffb640 1 -- 172.21.15.122:0/4245255357 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b320278040 con 0x55b32038c330 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4861ffb640 10 client.18357 handle_command_reply: tid=0 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 client.18357 shutdown 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 msgr2=0x55b32031de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f486401bac0 tx=0x7f486401bd90 comp rx=0 tx=0).stop 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 shutdown_connections 2024-04-02T13:44:31.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b32038c330 0x55b32038e720 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b320315ca0 0x55b32030f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b32030d0c0 0x55b32031e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4861ffb640 0 client.18357 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 --2- 172.21.15.122:0/4245255357 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b32030c7a0 0x55b32031de60 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 >> 172.21.15.122:0/4245255357 conn(0x55b320306ed0 msgr2=0x55b320313350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 shutdown_connections 2024-04-02T13:44:31.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.467+0000 7f4883999740 1 -- 172.21.15.122:0/4245255357 wait complete. 2024-04-02T13:44:31.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4883999740 20 client.18357 trim_cache size 0 max 16384 2024-04-02T13:44:31.484 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:44:31.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 msgr2=0x7f48680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 0x7f48680a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f48740f1b90 tx=0x7f48740bc060 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 msgr2=0x7f487c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f4874004af0 tx=0x7f4874032f30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 shutdown_connections 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48680a3d20 0x7f48680a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f487c1069a0 0x7f487c19ba80 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f487c101350 0x7f487c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 --2- 172.21.15.122:0/699868453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f487c100990 0x7f487c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 >> 172.21.15.122:0/699868453 conn(0x7f487c077ad0 msgr2=0x7f487c102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 shutdown_connections 2024-04-02T13:44:31.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.468+0000 7f4880daf640 1 -- 172.21.15.122:0/699868453 wait complete. 2024-04-02T13:44:31.487 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 -- 172.21.15.122:0/1109419007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0101330 msgr2=0x7f20f010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 --2- 172.21.15.122:0/1109419007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0101330 0x7f20f010d840 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f20e8009cd0 tx=0x7f20e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 -- 172.21.15.122:0/1109419007 shutdown_connections 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 --2- 172.21.15.122:0/1109419007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0101330 0x7f20f010d840 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 --2- 172.21.15.122:0/1109419007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0100970 0x7f20f0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 --2- 172.21.15.122:0/1109419007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0106980 0x7f20f0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.596+0000 7f20f72b8640 1 -- 172.21.15.122:0/1109419007 >> 172.21.15.122:0/1109419007 conn(0x7f20f0077b00 msgr2=0x7f20f0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.597+0000 7f20f72b8640 1 -- 172.21.15.122:0/1109419007 shutdown_connections 2024-04-02T13:44:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.597+0000 7f20f72b8640 1 -- 172.21.15.122:0/1109419007 wait complete. 2024-04-02T13:44:31.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.597+0000 7f20f72b8640 1 Processor -- start 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.597+0000 7f20f72b8640 1 -- start start 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0100970 0x7f20f0102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0101330 0x7f20f0103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0110d90 con 0x7f20f0106980 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0110c10 con 0x7f20f0101330 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0110f10 con 0x7f20f0100970 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46460/0 (socket says 172.21.15.122:46460) 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 -- 172.21.15.122:0/881884865 learned_addr learned my addr 172.21.15.122:0/881884865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f4853640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0101330 0x7f20f0103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5054640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0100970 0x7f20f0102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 -- 172.21.15.122:0/881884865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0100970 msgr2=0x7f20f0102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0100970 0x7f20f0102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 -- 172.21.15.122:0/881884865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0101330 msgr2=0x7f20f0103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0101330 0x7f20f0103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 -- 172.21.15.122:0/881884865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f01a68c0 con 0x7f20f0106980 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f5855640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f20e80348f0 tx=0x7f20e8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20e67fc640 1 -- 172.21.15.122:0/881884865 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8041020 con 0x7f20f0106980 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01a6b50 con 0x7f20f0106980 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20e67fc640 1 -- 172.21.15.122:0/881884865 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8032e80 con 0x7f20f0106980 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.598+0000 7f20e67fc640 1 -- 172.21.15.122:0/881884865 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8039040 con 0x7f20f0106980 2024-04-02T13:44:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.599+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a7090 con 0x7f20f0106980 2024-04-02T13:44:31.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20e67fc640 1 -- 172.21.15.122:0/881884865 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f20e8044020 con 0x7f20f0106980 2024-04-02T13:44:31.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 Processor -- start 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 -- start start 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20e67fc640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 0x7f20dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b19749f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b1973fa7b0 con 0x55b1974967a0 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b197399570 con 0x55b1974970c0 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.601+0000 7f20f9f78740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b1973f6760 con 0x55b19749fca0 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f4853640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 0x7f20dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20e67fc640 1 -- 172.21.15.122:0/881884865 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f20e80f2690 con 0x7f20f0106980 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46476/0 (socket says 172.21.15.122:46476) 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 -- 172.21.15.122:0/2669302345 learned_addr learned my addr 172.21.15.122:0/2669302345 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f4853640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b19749f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5855640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 -- 172.21.15.122:0/2669302345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 msgr2=0x55b1974a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 -- 172.21.15.122:0/2669302345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 msgr2=0x55b19749f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b19749f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 -- 172.21.15.122:0/2669302345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19710e3f0 con 0x55b1974967a0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5855640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f4853640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 0x7f20dc0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20f0104980 tx=0x7f20e0009b70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20f5054640 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009c70 tx=0x7f20ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cb416cfb3d6f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20db7fe640 1 -- 172.21.15.122:0/2669302345 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec01e070 con 0x55b1974967a0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20db7fe640 1 -- 172.21.15.122:0/2669302345 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ec009e20 con 0x55b1974967a0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.602+0000 7f20db7fe640 1 -- 172.21.15.122:0/2669302345 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec019630 con 0x55b1974967a0 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- 172.21.15.122:0/2669302345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 msgr2=0x55b197496b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009c70 tx=0x7f20ec011810 comp rx=0 tx=0).stop 2024-04-02T13:44:31.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- 172.21.15.122:0/2669302345 shutdown_connections 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b19749f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 --2- 172.21.15.122:0/2669302345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b197496b80 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- 172.21.15.122:0/2669302345 >> 172.21.15.122:0/2669302345 conn(0x55b197490ed0 msgr2=0x55b1974912b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- 172.21.15.122:0/2669302345 shutdown_connections 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- 172.21.15.122:0/2669302345 wait complete. 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 -- start start 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.603+0000 7f20f9f78740 1 Processor -- start 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b19749f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b19749d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b1974a2a90 con 0x55b1974970c0 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b1973b5e80 con 0x55b19749fca0 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b1974a2c10 con 0x55b1974967a0 2024-04-02T13:44:31.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f9f78740 10 client.0 _ll_register_callbacks cb 0x55b1973f1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46492/0 (socket says 172.21.15.122:46492) 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f5054640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b19749f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 -- 172.21.15.122:0/349753747 learned_addr learned my addr 172.21.15.122:0/349753747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f5855640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b19749d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 -- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 msgr2=0x55b19749f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b19749f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 -- 172.21.15.122:0/349753747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 msgr2=0x55b19749d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b19749d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 -- 172.21.15.122:0/349753747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1971d87d0 con 0x55b1974970c0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f5855640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b19749d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20f4853640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009db0 tx=0x7f20e001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4251df7b115755 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.604+0000 7f20d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0003800 con 0x55b1974970c0 2024-04-02T13:44:31.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20ec009e20 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f20e00039a0 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f20e0003ba0 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20f9f78740 10 client.18369 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b1974aeb40 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.605+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ec009e20 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f20e0010430 con 0x55b1974970c0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 10 client.18369 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 10 client.18369 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 20 client.18369 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1974afff0 0x55b1974b2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 4 client.18369 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b197407cf0 con 0x55b1974afff0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f5054640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1974afff0 0x55b1974b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20f5054640 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1974afff0 0x55b1974b2410 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95511ac12a24e37e server_cookie=1e2fcbc32f1651a1 in_seq=0 out_seq=0 2024-04-02T13:44:31.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.606+0000 7f20d8ff9640 10 client.18369 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20d8ff9640 1 -- 172.21.15.122:0/349753747 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b197407cf0 con 0x55b1974afff0 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20d8ff9640 10 client.18369 handle_command_reply: tid=0 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 client.18369 shutdown 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 msgr2=0x55b19749caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009db0 tx=0x7f20e001d000 comp rx=0 tx=0).stop 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 shutdown_connections 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b19749d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1974afff0 0x55b1974b2410 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b19749caa0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 --2- 172.21.15.122:0/349753747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b19749f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20d8ff9640 0 client.18369 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 >> 172.21.15.122:0/349753747 conn(0x55b197490ed0 msgr2=0x55b197492c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 shutdown_connections 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 1 -- 172.21.15.122:0/349753747 wait complete. 2024-04-02T13:44:31.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.607+0000 7f20f9f78740 20 client.18369 trim_cache size 0 max 16384 2024-04-02T13:44:31.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 Processor -- start 2024-04-02T13:44:31.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 -- start start 2024-04-02T13:44:31.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b1974ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b1974b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b1270 con 0x55b1974970c0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b2db0 con 0x55b19749fca0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f9f78740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b1940 con 0x55b1974967a0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f4853640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f4853640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46494/0 (socket says 172.21.15.122:46494) 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b1974b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f4853640 1 -- 172.21.15.122:0/1863985639 learned_addr learned my addr 172.21.15.122:0/1863985639 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5054640 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b1974ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 -- 172.21.15.122:0/1863985639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 msgr2=0x55b1974ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b1974ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 -- 172.21.15.122:0/1863985639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 msgr2=0x55b197497de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 -- 172.21.15.122:0/1863985639 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b19748c7a0 con 0x55b19749fca0 2024-04-02T13:44:31.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f4853640 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.622+0000 7f20f5855640 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b1974b7ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20e80efb00 tx=0x7f20e802c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258266ce924e0cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20daffd640 1 -- 172.21.15.122:0/1863985639 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80bde90 con 0x55b19749fca0 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20daffd640 1 -- 172.21.15.122:0/1863985639 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8034920 con 0x55b19749fca0 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20daffd640 1 -- 172.21.15.122:0/1863985639 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8001da0 con 0x55b19749fca0 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 -- 172.21.15.122:0/1863985639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 msgr2=0x55b1974b7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b1974b7ce0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f20e80efb00 tx=0x7f20e802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 -- 172.21.15.122:0/1863985639 shutdown_connections 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b19749fca0 0x55b1974b7ce0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974970c0 0x55b197497de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 --2- 172.21.15.122:0/1863985639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b1974967a0 0x55b1974ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 -- 172.21.15.122:0/1863985639 >> 172.21.15.122:0/1863985639 conn(0x55b197490ed0 msgr2=0x55b19749bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 -- 172.21.15.122:0/1863985639 shutdown_connections 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.623+0000 7f20f9f78740 1 -- 172.21.15.122:0/1863985639 wait complete. 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 -- start start 2024-04-02T13:44:31.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 Processor -- start 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b1974a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b26e0 con 0x55b1974967a0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b2010 con 0x55b1974970c0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01af270 con 0x55b19749fca0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f9f78740 10 client.0 _ll_register_callbacks cb 0x7f20e0011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f5054640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f5054640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46496/0 (socket says 172.21.15.122:46496) 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f4853640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b1974a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f5054640 1 -- 172.21.15.122:0/25450586 learned_addr learned my addr 172.21.15.122:0/25450586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.624+0000 7f20f5855640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 -- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 msgr2=0x55b1974a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 -- 172.21.15.122:0/25450586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 msgr2=0x55b1974a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 -- 172.21.15.122:0/25450586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1974a73a0 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f5054640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f5855640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f4853640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b1974a0e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004700 tx=0x7f20e000ff80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2257670fa7015f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e003a020 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f20e0010430 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e8034920 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f20e0004400 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f9f78740 10 client.27452 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b1974a86a0 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20e8034920 con 0x55b1974970c0 2024-04-02T13:44:31.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.625+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f20e0036a80 con 0x55b1974970c0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 10 client.27452 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 10 client.27452 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 20 client.27452 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1975168c0 0x55b197518ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 4 client.27452 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b197402040 con 0x55b1975168c0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f5054640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1975168c0 0x55b197518ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20d8ff9640 10 client.27452 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f5054640 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1975168c0 0x55b197518ce0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8917c200c926adcf server_cookie=7356930ca447a5aa in_seq=0 out_seq=0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20d8ff9640 1 -- 172.21.15.122:0/25450586 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b197402040 con 0x55b1975168c0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20d8ff9640 10 client.27452 handle_command_reply: tid=0 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 client.27452 shutdown 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 msgr2=0x55b1974a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b1974a0e00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f20e0004700 tx=0x7f20e000ff80 comp rx=0 tx=0).stop 2024-04-02T13:44:31.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 shutdown_connections 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b19749fca0 0x55b1974a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b1974970c0 0x55b1974a0e00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55b1975168c0 0x55b197518ce0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 --2- 172.21.15.122:0/25450586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b1974967a0 0x55b1974a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20d8ff9640 0 client.27452 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.626+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 >> 172.21.15.122:0/25450586 conn(0x55b197490ed0 msgr2=0x55b197497de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 shutdown_connections 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f9f78740 1 -- 172.21.15.122:0/25450586 wait complete. 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f9f78740 20 client.27452 trim_cache size 0 max 16384 2024-04-02T13:44:31.635 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:44:31.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 msgr2=0x7f20dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 0x7f20dc0a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f20f0104980 tx=0x7f20e0009b70 comp rx=0 tx=0).stop 2024-04-02T13:44:31.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 msgr2=0x7f20f0103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f20e80348f0 tx=0x7f20e8032c90 comp rx=0 tx=0).stop 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 shutdown_connections 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20dc0a3d20 0x7f20dc0a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20f0106980 0x7f20f0103970 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f20f0101330 0x7f20f0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 --2- 172.21.15.122:0/881884865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20f0100970 0x7f20f0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 >> 172.21.15.122:0/881884865 conn(0x7f20f0077b00 msgr2=0x7f20f0068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 shutdown_connections 2024-04-02T13:44:31.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.627+0000 7f20f72b8640 1 -- 172.21.15.122:0/881884865 wait complete. 2024-04-02T13:44:31.640 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:31 smithi122 ceph-mon[32114]: pgmap v289: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:31.640 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:31 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/324395653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:31.641 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:44:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:31 smithi187 ceph-mon[36064]: pgmap v289: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:31 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/324395653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- 172.21.15.122:0/2242546172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 msgr2=0x7f8930100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 --2- 172.21.15.122:0/2242546172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f8930100df0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8924009930 tx=0x7f892402bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- 172.21.15.122:0/2242546172 shutdown_connections 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 --2- 172.21.15.122:0/2242546172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f893010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 --2- 172.21.15.122:0/2242546172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f8930100df0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 --2- 172.21.15.122:0/2242546172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f8930106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- 172.21.15.122:0/2242546172 >> 172.21.15.122:0/2242546172 conn(0x7f8930077b00 msgr2=0x7f8930077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- 172.21.15.122:0/2242546172 shutdown_connections 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- 172.21.15.122:0/2242546172 wait complete. 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 Processor -- start 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.750+0000 7f89362f2640 1 -- start start 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f8930197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f893019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110d60 con 0x7f8930100970 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110be0 con 0x7f8930106980 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8930110ee0 con 0x7f8930101330 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46526/0 (socket says 172.21.15.122:46526) 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 -- 172.21.15.122:0/1067768746 learned_addr learned my addr 172.21.15.122:0/1067768746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f893488f640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f893019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892f7fe640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f8930197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 -- 172.21.15.122:0/1067768746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 msgr2=0x7f8930197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f8930197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 -- 172.21.15.122:0/1067768746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 msgr2=0x7f893019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f893019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 -- 172.21.15.122:0/1067768746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893019c2b0 con 0x7f8930100970 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892f7fe640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f8930197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f893488f640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f893019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892ffff640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb00 tx=0x7f8920009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892d7fa640 1 -- 172.21.15.122:0/1067768746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892001c070 con 0x7f8930100970 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892d7fa640 1 -- 172.21.15.122:0/1067768746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892000bc30 con 0x7f8930100970 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893019c5a0 con 0x7f8930100970 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f892d7fa640 1 -- 172.21.15.122:0/1067768746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89200166a0 con 0x7f8930100970 2024-04-02T13:44:31.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.751+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8930068c60 con 0x7f8930100970 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892d7fa640 1 -- 172.21.15.122:0/1067768746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f892001f070 con 0x7f8930100970 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 Processor -- start 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 -- start start 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732a97a0 0x55cb732a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 0x55cb732b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cb7320d7b0 con 0x55cb732aa0c0 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cb731ac570 con 0x55cb732b2ca0 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f8938fbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cb73209760 con 0x55cb732a97a0 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 0x55cb732b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43196/0 (socket says 172.21.15.122:43196) 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 0x55cb732b2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46538/0 (socket says 172.21.15.122:46538) 2024-04-02T13:44:31.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f892f7fe640 1 -- 172.21.15.122:0/2724411026 learned_addr learned my addr 172.21.15.122:0/2724411026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.754+0000 7f893488f640 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732a97a0 0x55cb732a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 -- 172.21.15.122:0/2724411026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732a97a0 msgr2=0x55cb732a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732a97a0 0x55cb732a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 -- 172.21.15.122:0/2724411026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 msgr2=0x55cb732b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 0x55cb732b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 -- 172.21.15.122:0/2724411026 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb72f213f0 con 0x55cb732b2ca0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892d7fa640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 0x7f89180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f893488f640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 0x7f89180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892f7fe640 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f8924005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135063583813568c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f892d7fa640 1 -- 172.21.15.122:0/1067768746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f89200d2130 con 0x7f8930100970 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f89167fc640 1 -- 172.21.15.122:0/2724411026 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892402ca80 con 0x55cb732b2ca0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f89167fc640 1 -- 172.21.15.122:0/2724411026 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892402cc20 con 0x55cb732b2ca0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.755+0000 7f89167fc640 1 -- 172.21.15.122:0/2724411026 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8924004920 con 0x55cb732b2ca0 2024-04-02T13:44:31.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- 172.21.15.122:0/2724411026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 msgr2=0x55cb732b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f893488f640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 0x7f89180a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8928009fd0 tx=0x7f8928009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8924006010 tx=0x7f8924005e60 comp rx=0 tx=0).stop 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- 172.21.15.122:0/2724411026 shutdown_connections 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732b2ca0 0x55cb732b5090 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732aa0c0 0x55cb732b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2724411026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732a97a0 0x55cb732a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- 172.21.15.122:0/2724411026 >> 172.21.15.122:0/2724411026 conn(0x55cb732a3ed0 msgr2=0x55cb732a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- 172.21.15.122:0/2724411026 shutdown_connections 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- 172.21.15.122:0/2724411026 wait complete. 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.756+0000 7f8938fbb740 1 -- start start 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 Processor -- start 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732b0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cb732b5a30 con 0x55cb732b2ca0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cb731c8e80 con 0x55cb732a97a0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cb732b5bb0 con 0x55cb732aa0c0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f8938fbb740 10 client.0 _ll_register_callbacks cb 0x55cb73204aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f893488f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43202/0 (socket says 172.21.15.122:43202) 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 -- 172.21.15.122:0/3872329187 learned_addr learned my addr 172.21.15.122:0/3872329187 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892f7fe640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 -- 172.21.15.122:0/3872329187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 msgr2=0x55cb732afaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732afaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 -- 172.21.15.122:0/3872329187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 msgr2=0x55cb732b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f892ffff640 1 -- 172.21.15.122:0/3872329187 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb72feb7d0 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.757+0000 7f893488f640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f892ffff640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f892000ead0 tx=0x7f89200097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2228e2a94bcea630 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892009d8d0 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892402cc20 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f892009d1e0 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f892009d6f0 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 10 client.27464 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb732c5f50 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892402cc20 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f8920099ae0 con 0x55cb732a97a0 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 10 client.27464 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 10 client.27464 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:44:31.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 20 client.27464 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb732c7360 0x55cb732c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 4 client.27464 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.758+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb7321acf0 con 0x55cb732c7360 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.759+0000 7f892f7fe640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb732c7360 0x55cb732c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.759+0000 7f892f7fe640 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb732c7360 0x55cb732c9800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=151b03f5ecd65e69 server_cookie=61df9be9b76d3aa2 in_seq=0 out_seq=0 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.759+0000 7f890f7fe640 10 client.27464 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f890f7fe640 1 -- 172.21.15.122:0/3872329187 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb7321acf0 con 0x55cb732c7360 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f890f7fe640 10 client.27464 handle_command_reply: tid=0 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 client.27464 shutdown 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 msgr2=0x55cb732b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f892000ead0 tx=0x7f89200097d0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 shutdown_connections 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732afaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb732c7360 0x55cb732c9800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 --2- 172.21.15.122:0/3872329187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732b26a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f890f7fe640 0 client.27464 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:44:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 >> 172.21.15.122:0/3872329187 conn(0x55cb732a3ed0 msgr2=0x55cb732a5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 shutdown_connections 2024-04-02T13:44:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 1 -- 172.21.15.122:0/3872329187 wait complete. 2024-04-02T13:44:31.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.760+0000 7f8938fbb740 20 client.27464 trim_cache size 0 max 16384 2024-04-02T13:44:31.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:31 smithi188 ceph-mon[36117]: pgmap v289: 129 pgs: 129 active+clean; 226 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-04-02T13:44:31.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:31 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/324395653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:44:31.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 Processor -- start 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 -- start start 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89301acf20 con 0x55cb732b2ca0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89301aea60 con 0x55cb732a97a0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.774+0000 7f8938fbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89301ad5f0 con 0x55cb732aa0c0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f892ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f892f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f892ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43206/0 (socket says 172.21.15.122:43206) 2024-04-02T13:44:31.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f892ffff640 1 -- 172.21.15.122:0/2026201290 learned_addr learned my addr 172.21.15.122:0/2026201290 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 -- 172.21.15.122:0/2026201290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 msgr2=0x55cb732b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 -- 172.21.15.122:0/2026201290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 msgr2=0x55cb732af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 -- 172.21.15.122:0/2026201290 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb732b09c0 con 0x55cb732b2ca0 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f892ffff640 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f893488f640 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732cf080 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f89280127c0 tx=0x7f8928012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b6c63f4c4a4575 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f8915ffb640 1 -- 172.21.15.122:0/2026201290 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892802e070 con 0x55cb732b2ca0 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f8915ffb640 1 -- 172.21.15.122:0/2026201290 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892800fda0 con 0x55cb732b2ca0 2024-04-02T13:44:31.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.775+0000 7f8915ffb640 1 -- 172.21.15.122:0/2026201290 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928029e20 con 0x55cb732b2ca0 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- 172.21.15.122:0/2026201290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 msgr2=0x55cb732cf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732cf080 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f89280127c0 tx=0x7f8928012000 comp rx=0 tx=0).stop 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- 172.21.15.122:0/2026201290 shutdown_connections 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732b2ca0 0x55cb732cf080 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732aa0c0 0x55cb732b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 --2- 172.21.15.122:0/2026201290 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732a97a0 0x55cb732af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- 172.21.15.122:0/2026201290 >> 172.21.15.122:0/2026201290 conn(0x55cb732a3ed0 msgr2=0x55cb732a26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- 172.21.15.122:0/2026201290 shutdown_connections 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- 172.21.15.122:0/2026201290 wait complete. 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.776+0000 7f8938fbb740 1 -- start start 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 Processor -- start 2024-04-02T13:44:31.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732aa0c0 0x55cb732bb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732b2ca0 0x55cb732ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89301ae390 con 0x55cb732a97a0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f89301adcc0 con 0x55cb732aa0c0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89301aaf20 con 0x55cb732b2ca0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f8938fbb740 10 client.0 _ll_register_callbacks cb 0x7f89200d5970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46564/0 (socket says 172.21.15.122:46564) 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 -- 172.21.15.122:0/490403927 learned_addr learned my addr 172.21.15.122:0/490403927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892f7fe640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732aa0c0 0x55cb732bb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f893488f640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732b2ca0 0x55cb732ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 -- 172.21.15.122:0/490403927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732b2ca0 msgr2=0x55cb732ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732b2ca0 0x55cb732ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 -- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732aa0c0 msgr2=0x55cb732bb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732aa0c0 0x55cb732bb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 -- 172.21.15.122:0/490403927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb732ad640 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f892ffff640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f892009c040 tx=0x7f892000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2e702d5cd9ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.777+0000 7f890f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892009d8d0 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f892009d1e0 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892800fda0 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f892000bde0 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 10 client.18393 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cb732b4050 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892800fda0 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f89200a3220 con 0x55cb732a97a0 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 10 client.18393 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 10 client.18393 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:44:31.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 20 client.18393 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb7332da00 0x55cb7332fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 4 client.18393 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.778+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cb73215040 con 0x55cb7332da00 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f892f7fe640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb7332da00 0x55cb7332fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f892f7fe640 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb7332da00 0x55cb7332fdf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8423b7c1795d7585 server_cookie=f2f5e1d18da54669 in_seq=0 out_seq=0 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f890f7fe640 10 client.18393 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f890f7fe640 1 -- 172.21.15.122:0/490403927 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cb73215040 con 0x55cb7332da00 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f890f7fe640 10 client.18393 handle_command_reply: tid=0 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 client.18393 shutdown 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 msgr2=0x55cb732bae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f892009c040 tx=0x7f892000b8f0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 shutdown_connections 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cb732b2ca0 0x55cb732ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cb7332da00 0x55cb7332fdf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cb732aa0c0 0x55cb732bb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 --2- 172.21.15.122:0/490403927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cb732a97a0 0x55cb732bae00 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f890f7fe640 0 client.18393 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 >> 172.21.15.122:0/490403927 conn(0x55cb732a3ed0 msgr2=0x55cb732a2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 shutdown_connections 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.779+0000 7f8938fbb740 1 -- 172.21.15.122:0/490403927 wait complete. 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f8938fbb740 20 client.18393 trim_cache size 0 max 16384 2024-04-02T13:44:31.789 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 msgr2=0x7f89180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 0x7f89180a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8928009fd0 tx=0x7f8928009500 comp rx=0 tx=0).stop 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 msgr2=0x7f89301972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f892000eb00 tx=0x7f8920009f20 comp rx=0 tx=0).stop 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 shutdown_connections 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f89180a3d20 0x7f89180a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8930106980 0x7f893019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8930101330 0x7f8930197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 --2- 172.21.15.122:0/1067768746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8930100970 0x7f89301972c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 >> 172.21.15.122:0/1067768746 conn(0x7f8930077b00 msgr2=0x7f89301031a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 shutdown_connections 2024-04-02T13:44:31.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.780+0000 7f89362f2640 1 -- 172.21.15.122:0/1067768746 wait complete. 2024-04-02T13:44:31.794 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:44:31.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 -- 172.21.15.122:0/1852131848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc20107b50 msgr2=0x7fdc20107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1852131848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc20107b50 0x7fdc20107f30 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009930 tx=0x7fdc1c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 -- 172.21.15.122:0/1852131848 shutdown_connections 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1852131848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc200696c0 0x7fdc2010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1852131848 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20068d00 0x7fdc20069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1852131848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc20107b50 0x7fdc20107f30 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.900+0000 7fdc2823c640 1 -- 172.21.15.122:0/1852131848 >> 172.21.15.122:0/1852131848 conn(0x7fdc20077b00 msgr2=0x7fdc20077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- 172.21.15.122:0/1852131848 shutdown_connections 2024-04-02T13:44:31.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- 172.21.15.122:0/1852131848 wait complete. 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 Processor -- start 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- start start 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 0x7fdc20198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20107b50 0x7fdc2019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20111ea0 con 0x7fdc200696c0 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20111d20 con 0x7fdc20107b50 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc2823c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc20112020 con 0x7fdc20068d00 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46574/0 (socket says 172.21.15.122:46574) 2024-04-02T13:44:31.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc257d7640 1 -- 172.21.15.122:0/1405277532 learned_addr learned my addr 172.21.15.122:0/1405277532 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 0x7fdc20198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.901+0000 7fdc267d9640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20107b50 0x7fdc2019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 -- 172.21.15.122:0/1405277532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20107b50 msgr2=0x7fdc2019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20107b50 0x7fdc2019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 -- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 msgr2=0x7fdc20198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 -- 172.21.15.122:0/1405277532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2019d330 con 0x7fdc20068d00 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc257d7640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 0x7fdc20198340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009230 tx=0x7fdc1c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc16ffd640 1 -- 172.21.15.122:0/1405277532 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c041070 con 0x7fdc20068d00 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2019d5c0 con 0x7fdc20068d00 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc16ffd640 1 -- 172.21.15.122:0/1405277532 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc1c0054e0 con 0x7fdc20068d00 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc16ffd640 1 -- 172.21.15.122:0/1405277532 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c003bf0 con 0x7fdc20068d00 2024-04-02T13:44:31.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.902+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc201a4eb0 con 0x7fdc20068d00 2024-04-02T13:44:31.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.904+0000 7fdc16ffd640 1 -- 172.21.15.122:0/1405277532 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdc1c044070 con 0x7fdc20068d00 2024-04-02T13:44:31.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.904+0000 7fdc2ae28740 1 Processor -- start 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 -- start start 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eeac8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc16ffd640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 0x7fdc0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5581eea237b0 con 0x5581eeabf7a0 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5581ee9c2570 con 0x5581eeac8ca0 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5581eea1f760 con 0x5581eeac00c0 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46584/0 (socket says 172.21.15.122:46584) 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 -- 172.21.15.122:0/4078019282 learned_addr learned my addr 172.21.15.122:0/4078019282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 0x7fdc0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc257d7640 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eeac8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc16ffd640 1 -- 172.21.15.122:0/1405277532 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fdc1c0f2560 con 0x7fdc20068d00 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 -- 172.21.15.122:0/4078019282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 msgr2=0x5581eeac8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eeac8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 -- 172.21.15.122:0/4078019282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 msgr2=0x5581eeacb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.905+0000 7fdc267d9640 1 -- 172.21.15.122:0/4078019282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581ee7373f0 con 0x5581eeabf7a0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.906+0000 7fdc25fd8640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 0x7fdc0c0a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0efaa0 tx=0x7fdc1c08cca0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.906+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fdc18013940 tx=0x7fdc18009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27885329aff6f258 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.906+0000 7fdc0bfff640 1 -- 172.21.15.122:0/4078019282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18003970 con 0x5581eeabf7a0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.906+0000 7fdc0bfff640 1 -- 172.21.15.122:0/4078019282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc1800f8c0 con 0x5581eeabf7a0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.906+0000 7fdc0bfff640 1 -- 172.21.15.122:0/4078019282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180043c0 con 0x5581eeabf7a0 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4078019282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 msgr2=0x5581eeabfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fdc18013940 tx=0x7fdc18009b30 comp rx=0 tx=0).stop 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4078019282 shutdown_connections 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eeac8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4078019282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeabf7a0 0x5581eeabfb80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4078019282 >> 172.21.15.122:0/4078019282 conn(0x5581eeab9ed0 msgr2=0x5581eeaba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4078019282 shutdown_connections 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4078019282 wait complete. 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 -- start start 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.907+0000 7fdc2ae28740 1 Processor -- start 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 0x5581eeac6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5581eeacba90 con 0x5581eeac00c0 2024-04-02T13:44:31.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5581ee9dee80 con 0x5581eeabf7a0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5581eeacbc10 con 0x5581eeac8ca0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 10 client.0 _ll_register_callbacks cb 0x5581eea1aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46588/0 (socket says 172.21.15.122:46588) 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 -- 172.21.15.122:0/4159922451 learned_addr learned my addr 172.21.15.122:0/4159922451 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 0x5581eeac6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 -- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 msgr2=0x5581eeac6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 0x5581eeac6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 -- 172.21.15.122:0/4159922451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 msgr2=0x5581eeac8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 -- 172.21.15.122:0/4159922451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581ee8017d0 con 0x5581eeac00c0 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 0x5581eeac6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:31.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc257d7640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdc10012d20 tx=0x7fdc1000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43b3f8b0aa7d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc097fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10012110 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fdc10004050 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1800f8c0 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdc100122b0 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 10 client.18405 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.908+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5581eead7b50 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc1800f8c0 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdc10010500 con 0x5581eeac00c0 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 10 client.18405 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 10 client.18405 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 20 client.18405 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eead9010 0x5581eeadb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 4 client.18405 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:31.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581eea30cf0 con 0x5581eead9010 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eead9010 0x5581eeadb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc267d9640 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eead9010 0x5581eeadb440 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5afa1a5847c84b00 server_cookie=84912b6d9b31e9ad in_seq=0 out_seq=0 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.909+0000 7fdc097fa640 10 client.18405 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc097fa640 1 -- 172.21.15.122:0/4159922451 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581eea30cf0 con 0x5581eead9010 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc097fa640 10 client.18405 handle_command_reply: tid=0 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 client.18405 shutdown 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 msgr2=0x5581eeac5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fdc10012d20 tx=0x7fdc1000f790 comp rx=0 tx=0).stop 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 shutdown_connections 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eead9010 0x5581eeadb440 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac8ca0 0x5581eeac6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac5a90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc097fa640 0 client.18405 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:44:31.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/4159922451 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 >> 172.21.15.122:0/4159922451 conn(0x5581eeab9ed0 msgr2=0x5581eeabbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.910+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 shutdown_connections 2024-04-02T13:44:31.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.911+0000 7fdc2ae28740 1 -- 172.21.15.122:0/4159922451 wait complete. 2024-04-02T13:44:31.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.911+0000 7fdc2ae28740 20 client.18405 trim_cache size 0 max 16384 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.924+0000 7fdc2ae28740 1 Processor -- start 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.924+0000 7fdc2ae28740 1 -- start start 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eead4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201af020 con 0x5581eeac8ca0 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b0b60 con 0x5581eeabf7a0 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201af6f0 con 0x5581eeac00c0 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46594/0 (socket says 172.21.15.122:46594) 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 -- 172.21.15.122:0/3494796340 learned_addr learned my addr 172.21.15.122:0/3494796340 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc257d7640 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eead4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 -- 172.21.15.122:0/3494796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 msgr2=0x5581eead4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eead4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 -- 172.21.15.122:0/3494796340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 msgr2=0x5581eeac04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 -- 172.21.15.122:0/3494796340 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581eeac5860 con 0x5581eeac8ca0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdc18013910 tx=0x7fdc18004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ffe549104d494d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc0b7fe640 1 -- 172.21.15.122:0/3494796340 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800fa30 con 0x5581eeac8ca0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc0b7fe640 1 -- 172.21.15.122:0/3494796340 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc18003e50 con 0x5581eeac8ca0 2024-04-02T13:44:31.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.925+0000 7fdc0b7fe640 1 -- 172.21.15.122:0/3494796340 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1802d760 con 0x5581eeac8ca0 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3494796340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 msgr2=0x5581eeae0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdc18013910 tx=0x7fdc18004060 comp rx=0 tx=0).stop 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3494796340 shutdown_connections 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac8ca0 0x5581eeae0d50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeac00c0 0x5581eead4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3494796340 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeabf7a0 0x5581eeac04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3494796340 >> 172.21.15.122:0/3494796340 conn(0x5581eeab9ed0 msgr2=0x5581eeaba2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3494796340 shutdown_connections 2024-04-02T13:44:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3494796340 wait complete. 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.926+0000 7fdc2ae28740 1 -- start start 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 Processor -- start 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeabf7a0 0x5581eeac9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201b0490 con 0x5581eeac00c0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201afdc0 con 0x5581eeac8ca0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc201ad020 con 0x5581eeabf7a0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc2ae28740 10 client.0 _ll_register_callbacks cb 0x7fdc10009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46608/0 (socket says 172.21.15.122:46608) 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 -- 172.21.15.122:0/3010853116 learned_addr learned my addr 172.21.15.122:0/3010853116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeabf7a0 0x5581eeac9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 -- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeabf7a0 msgr2=0x5581eeac9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeabf7a0 0x5581eeac9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 -- 172.21.15.122:0/3010853116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 msgr2=0x5581eeacfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 -- 172.21.15.122:0/3010853116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581eead0420 con 0x5581eeac00c0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc257d7640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x5581eeabf130 tx=0x7fdc10012200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3890c10ceca23214 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.927+0000 7fdc097fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc10005110 con 0x5581eeac00c0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fdc100052b0 con 0x5581eeac00c0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc18003e50 con 0x5581eeac00c0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdc10037050 con 0x5581eeac00c0 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 10 client.18411 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:31.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5581eead1720 con 0x5581eeac00c0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc18003e50 con 0x5581eeac00c0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fdc10029450 con 0x5581eeac00c0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 10 client.18411 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 10 client.18411 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 20 client.18411 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eeb3fb90 0x5581eeb41fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 4 client.18411 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581eea2b040 con 0x5581eeb3fb90 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.928+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eeb3fb90 0x5581eeb41fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc097fa640 10 client.18411 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc267d9640 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eeb3fb90 0x5581eeb41fb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0a9df7fbe7c8187 server_cookie=ba2fd9e8ce5a0486 in_seq=0 out_seq=0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc097fa640 1 -- 172.21.15.122:0/3010853116 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581eea2b040 con 0x5581eeb3fb90 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc097fa640 10 client.18411 handle_command_reply: tid=0 2024-04-02T13:44:31.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 client.18411 shutdown 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 msgr2=0x5581eeac9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x5581eeabf130 tx=0x7fdc10012200 comp rx=0 tx=0).stop 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 shutdown_connections 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5581eeb3fb90 0x5581eeb41fb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5581eeac8ca0 0x5581eeacfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5581eeac00c0 0x5581eeac9db0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 --2- 172.21.15.122:0/3010853116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5581eeabf7a0 0x5581eeac9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc097fa640 0 client.18411 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 >> 172.21.15.122:0/3010853116 conn(0x5581eeab9ed0 msgr2=0x5581eeac0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 shutdown_connections 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 1 -- 172.21.15.122:0/3010853116 wait complete. 2024-04-02T13:44:31.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.929+0000 7fdc2ae28740 20 client.18411 trim_cache size 0 max 16384 2024-04-02T13:44:31.940 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 msgr2=0x7fdc0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 0x7fdc0c0a61e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0efaa0 tx=0x7fdc1c08cca0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 msgr2=0x7fdc20198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 0x7fdc20198340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009230 tx=0x7fdc1c004270 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 shutdown_connections 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdc0c0a3d20 0x7fdc0c0a61e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc20107b50 0x7fdc2019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc200696c0 0x7fdc20198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 --2- 172.21.15.122:0/1405277532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc20068d00 0x7fdc20198340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 >> 172.21.15.122:0/1405277532 conn(0x7fdc20077b00 msgr2=0x7fdc200fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 shutdown_connections 2024-04-02T13:44:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:31.930+0000 7fdc2823c640 1 -- 172.21.15.122:0/1405277532 wait complete. 2024-04-02T13:44:31.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:44:31.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 316256 3.31 MB/sec execute 352 sec latency 11.741 ms 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 -- 172.21.15.122:0/644426781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 msgr2=0x7f676010e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 --2- 172.21.15.122:0/644426781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f676010e940 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ca0 tx=0x7f675802c180 comp rx=0 tx=0).stop 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 -- 172.21.15.122:0/644426781 shutdown_connections 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 --2- 172.21.15.122:0/644426781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f676010e940 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 --2- 172.21.15.122:0/644426781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760069e80 0x7f676006a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 --2- 172.21.15.122:0/644426781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760107b80 0x7f6760107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.055+0000 7f6767ad3640 1 -- 172.21.15.122:0/644426781 >> 172.21.15.122:0/644426781 conn(0x7f6760077a90 msgr2=0x7f6760077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- 172.21.15.122:0/644426781 shutdown_connections 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- 172.21.15.122:0/644426781 wait complete. 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 Processor -- start 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- start start 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f6760100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760107b80 0x7f67601014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6760111eb0 con 0x7f676006a820 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6760111d30 con 0x7f6760069e80 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.056+0000 7f6767ad3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112030 con 0x7f6760107b80 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676506e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f6760100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43276/0 (socket says 172.21.15.122:43276) 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 -- 172.21.15.122:0/2002657138 learned_addr learned my addr 172.21.15.122:0/2002657138 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676506e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f6760100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46616/0 (socket says 172.21.15.122:46616) 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6766070640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760107b80 0x7f67601014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 -- 172.21.15.122:0/2002657138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760107b80 msgr2=0x7f67601014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760107b80 0x7f67601014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 -- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 msgr2=0x7f6760100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f6760100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 -- 172.21.15.122:0/2002657138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67601a67f0 con 0x7f6760069e80 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f676586f640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f675c00ea60 tx=0x7f675c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6756ffd640 1 -- 172.21.15.122:0/2002657138 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c00cde0 con 0x7f6760069e80 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6756ffd640 1 -- 172.21.15.122:0/2002657138 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c0035f0 con 0x7f6760069e80 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67601a6ae0 con 0x7f6760069e80 2024-04-02T13:44:32.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6756ffd640 1 -- 172.21.15.122:0/2002657138 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c010670 con 0x7f6760069e80 2024-04-02T13:44:32.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.057+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a7030 con 0x7f6760069e80 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 Processor -- start 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f6756ffd640 1 -- 172.21.15.122:0/2002657138 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f675c0037a0 con 0x7f6760069e80 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 -- start start 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 0x565527ec6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565527e2a7b0 con 0x565527ecfca0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x565527dc9570 con 0x565527ec70c0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.060+0000 7f676a793740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565527e26760 con 0x565527ec67a0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676586f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676506e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 0x565527ec6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676586f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43290/0 (socket says 172.21.15.122:43290) 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676506e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46626/0 (socket says 172.21.15.122:46626) 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676586f640 1 -- 172.21.15.122:0/2806710461 learned_addr learned my addr 172.21.15.122:0/2806710461 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 -- 172.21.15.122:0/2806710461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 msgr2=0x565527ecf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6756ffd640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 0x7f674c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 -- 172.21.15.122:0/2806710461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 msgr2=0x565527ed2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 -- 172.21.15.122:0/2806710461 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565527b3e3f0 con 0x565527ec67a0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f676506e640 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6756ffd640 1 -- 172.21.15.122:0/2002657138 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f675c0d0e80 con 0x7f6760069e80 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 0x7f674c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f6766070640 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 0x565527ec6b80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f675802c790 tx=0x7f6758009900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a06447cbb14cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.061+0000 7f674bfff640 1 -- 172.21.15.122:0/2806710461 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758039040 con 0x565527ec67a0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f674bfff640 1 -- 172.21.15.122:0/2806710461 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6758043030 con 0x565527ec67a0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f674bfff640 1 -- 172.21.15.122:0/2806710461 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6758032b10 con 0x565527ec67a0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 -- 172.21.15.122:0/2806710461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 msgr2=0x565527ec6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f6766070640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 0x7f674c0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6758005cc0 tx=0x7f6758032e70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 0x565527ec6b80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f675802c790 tx=0x7f6758009900 comp rx=0 tx=0).stop 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 -- 172.21.15.122:0/2806710461 shutdown_connections 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 --2- 172.21.15.122:0/2806710461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec67a0 0x565527ec6b80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 -- 172.21.15.122:0/2806710461 >> 172.21.15.122:0/2806710461 conn(0x565527ec0ed0 msgr2=0x565527ec12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 -- 172.21.15.122:0/2806710461 shutdown_connections 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.062+0000 7f676a793740 1 -- 172.21.15.122:0/2806710461 wait complete. 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 -- start start 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 Processor -- start 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 0x565527eccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565527ed2a90 con 0x565527ec67a0 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x565527de5e80 con 0x565527ec70c0 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565527ed2c10 con 0x565527ecfca0 2024-04-02T13:44:32.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676586f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676586f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46642/0 (socket says 172.21.15.122:46642) 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676506e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676586f640 1 -- 172.21.15.122:0/1365846891 learned_addr learned my addr 172.21.15.122:0/1365846891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f6766070640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 0x565527eccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.063+0000 7f676a793740 10 client.0 _ll_register_callbacks cb 0x565527e21aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 -- 172.21.15.122:0/1365846891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 msgr2=0x565527eccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 0x565527eccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 -- 172.21.15.122:0/1365846891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 msgr2=0x565527ecf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 -- 172.21.15.122:0/1365846891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565527c087d0 con 0x565527ec70c0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f6766070640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 0x565527eccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676586f640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676506e640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecc980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6760102470 tx=0x7f6750012b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800dce7514a5dbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f67497fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750004050 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f67500041f0 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6758043030 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f6750025050 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676a793740 10 client.27500 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565527edea60 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.064+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6758043030 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f6750003b10 con 0x565527ec70c0 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 10 client.27500 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 10 client.27500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 20 client.27500 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527edff20 0x565527ee2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 4 client.27500 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565527e37cf0 con 0x565527edff20 2024-04-02T13:44:32.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676586f640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527edff20 0x565527ee2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f67497fa640 10 client.27500 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.065+0000 7f676586f640 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527edff20 0x565527ee2330 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b59794d5106b82 server_cookie=7c64f2c71d6c0c03 in_seq=0 out_seq=0 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f67497fa640 1 -- 172.21.15.122:0/1365846891 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565527e37cf0 con 0x565527edff20 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f67497fa640 10 client.27500 handle_command_reply: tid=0 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 client.27500 shutdown 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 msgr2=0x565527ecc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecc980 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6760102470 tx=0x7f6750012b40 comp rx=0 tx=0).stop 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 shutdown_connections 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527edff20 0x565527ee2330 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ecfca0 0x565527eccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec70c0 0x565527ecc980 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f67497fa640 0 client.27500 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 --2- 172.21.15.122:0/1365846891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ecf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 >> 172.21.15.122:0/1365846891 conn(0x565527ec0ed0 msgr2=0x565527ec2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 shutdown_connections 2024-04-02T13:44:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 1 -- 172.21.15.122:0/1365846891 wait complete. 2024-04-02T13:44:32.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.066+0000 7f676a793740 20 client.27500 trim_cache size 0 max 16384 2024-04-02T13:44:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 Processor -- start 2024-04-02T13:44:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 -- start start 2024-04-02T13:44:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527edb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ecfca0 0x565527ecf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b12b0 con 0x565527ec67a0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b2df0 con 0x565527ecfca0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676a793740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b1980 con 0x565527ec70c0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46646/0 (socket says 172.21.15.122:46646) 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 -- 172.21.15.122:0/1308907387 learned_addr learned my addr 172.21.15.122:0/1308907387 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f6766070640 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ecfca0 0x565527ecf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676506e640 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527edb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 -- 172.21.15.122:0/1308907387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 msgr2=0x565527edb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527edb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 -- 172.21.15.122:0/1308907387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ecfca0 msgr2=0x565527ecf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ecfca0 0x565527ecf0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.085+0000 7f676586f640 1 -- 172.21.15.122:0/1308907387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565527ec6ef0 con 0x565527ec67a0 2024-04-02T13:44:32.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.086+0000 7f676586f640 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f675c01f000 tx=0x7f675c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb9b8002ad83549 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.086+0000 7f674b7fe640 1 -- 172.21.15.122:0/1308907387 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c09c800 con 0x565527ec67a0 2024-04-02T13:44:32.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.086+0000 7f674b7fe640 1 -- 172.21.15.122:0/1308907387 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c09d130 con 0x565527ec67a0 2024-04-02T13:44:32.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.086+0000 7f674b7fe640 1 -- 172.21.15.122:0/1308907387 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c09c390 con 0x565527ec67a0 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 -- 172.21.15.122:0/1308907387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 msgr2=0x565527ee2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f675c01f000 tx=0x7f675c087040 comp rx=0 tx=0).stop 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 -- 172.21.15.122:0/1308907387 shutdown_connections 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ecfca0 0x565527ecf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527edb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 --2- 172.21.15.122:0/1308907387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ec67a0 0x565527ee2310 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 -- 172.21.15.122:0/1308907387 >> 172.21.15.122:0/1308907387 conn(0x565527ec0ed0 msgr2=0x565527ec12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 -- 172.21.15.122:0/1308907387 shutdown_connections 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.087+0000 7f676a793740 1 -- 172.21.15.122:0/1308907387 wait complete. 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 -- start start 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 Processor -- start 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 0x565527ed1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527ed0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b2720 con 0x565527ecfca0 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f67601b2050 con 0x565527ec67a0 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67601af2b0 con 0x565527ec70c0 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676a793740 10 client.0 _ll_register_callbacks cb 0x7f6750009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f6766070640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676506e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527ed0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676586f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 0x565527ed1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676506e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527ed0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:50392/0 (socket says 172.21.15.122:50392) 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f6766070640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46656/0 (socket says 172.21.15.122:46656) 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676506e640 1 -- 172.21.15.122:0/1263486589 learned_addr learned my addr 172.21.15.122:0/1263486589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.088+0000 7f676586f640 1 -- 172.21.15.122:0/1263486589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 msgr2=0x565527ed0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676586f640 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527ed0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676586f640 1 -- 172.21.15.122:0/1263486589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 msgr2=0x565527ed06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676586f640 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676586f640 1 -- 172.21.15.122:0/1263486589 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565527ed0eb0 con 0x565527ec67a0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676586f640 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 0x565527ed1b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f675c01f000 tx=0x7f675c09af40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8e0451eb45f6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f67497fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c09c570 con 0x565527ec67a0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c09d130 con 0x565527ec67a0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f675c0990a0 con 0x565527ec67a0 2024-04-02T13:44:32.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f675c09c390 con 0x565527ec67a0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676a793740 10 client.27506 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565527ed7f90 con 0x565527ec67a0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.089+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c09d130 con 0x565527ec67a0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f675c09be80 con 0x565527ec67a0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 10 client.27506 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 10 client.27506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 20 client.27506 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527f464b0 0x565527f488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 4 client.27506 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565527e32040 con 0x565527f464b0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676506e640 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527f464b0 0x565527f488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f67497fa640 10 client.27506 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:32.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676506e640 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527f464b0 0x565527f488a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ae3393e8bfaeb64 server_cookie=b115bb9a4f4b0cff in_seq=0 out_seq=0 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f67497fa640 1 -- 172.21.15.122:0/1263486589 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565527e32040 con 0x565527f464b0 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f67497fa640 10 client.27506 handle_command_reply: tid=0 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.090+0000 7f676a793740 1 client.27506 shutdown 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 msgr2=0x565527ed1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 0x565527ed1b00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f675c01f000 tx=0x7f675c09af40 comp rx=0 tx=0).stop 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 shutdown_connections 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x565527f464b0 0x565527f488a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565527ecfca0 0x565527ed06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565527ec70c0 0x565527ed0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f67497fa640 0 client.27506 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 --2- 172.21.15.122:0/1263486589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565527ec67a0 0x565527ed1b00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 >> 172.21.15.122:0/1263486589 conn(0x565527ec0ed0 msgr2=0x565527ecb210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 shutdown_connections 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 1 -- 172.21.15.122:0/1263486589 wait complete. 2024-04-02T13:44:32.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.091+0000 7f676a793740 20 client.27506 trim_cache size 0 max 16384 2024-04-02T13:44:32.100 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 msgr2=0x7f674c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 0x7f674c0a6210 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6758005cc0 tx=0x7f6758032e70 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 msgr2=0x7f6760100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f675c00ea60 tx=0x7f675c00ef20 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 shutdown_connections 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f674c0a3d50 0x7f674c0a6210 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6760107b80 0x7f67601014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f676006a820 0x7f6760100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 --2- 172.21.15.122:0/2002657138 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6760069e80 0x7f6760100a30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 >> 172.21.15.122:0/2002657138 conn(0x7f6760077a90 msgr2=0x7f67600ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:32.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 shutdown_connections 2024-04-02T13:44:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:32.092+0000 7f6767ad3640 1 -- 172.21.15.122:0/2002657138 wait complete. 2024-04-02T13:44:32.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 316495 3.30 MB/sec execute 353 sec latency 794.513 ms 2024-04-02T13:44:33.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:44:33.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:44:33.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 -- 172.21.15.122:0/3882695278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c106e30 msgr2=0x7f8b8c10ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 --2- 172.21.15.122:0/3882695278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c106e30 0x7f8b8c10ea40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009ca0 tx=0x7f8b8402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 -- 172.21.15.122:0/3882695278 shutdown_connections 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 --2- 172.21.15.122:0/3882695278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c106e30 0x7f8b8c10ea40 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 --2- 172.21.15.122:0/3882695278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c101020 0x7f8b8c106750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 --2- 172.21.15.122:0/3882695278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 0x7f8b8c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.218+0000 7f8b93538640 1 -- 172.21.15.122:0/3882695278 >> 172.21.15.122:0/3882695278 conn(0x7f8b8c075c60 msgr2=0x7f8b8c076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- 172.21.15.122:0/3882695278 shutdown_connections 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- 172.21.15.122:0/3882695278 wait complete. 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 Processor -- start 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- start start 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 0x7f8b8c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 0x7f8b8c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c111fa0 con 0x7f8b8c19cd20 2024-04-02T13:44:33.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c111e20 con 0x7f8b8c106e30 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b93538640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c112120 con 0x7f8b8c100670 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b91ad5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b91ad5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46662/0 (socket says 172.21.15.122:46662) 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b91ad5640 1 -- 172.21.15.122:0/2591294573 learned_addr learned my addr 172.21.15.122:0/2591294573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.219+0000 7f8b912d4640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 0x7f8b8c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 0x7f8b8c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 -- 172.21.15.122:0/2591294573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 msgr2=0x7f8b8c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 0x7f8b8c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 -- 172.21.15.122:0/2591294573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 msgr2=0x7f8b8c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 0x7f8b8c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 -- 172.21.15.122:0/2591294573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c19d810 con 0x7f8b8c19cd20 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 0x7f8b8c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b91ad5640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8b8402c700 tx=0x7f8b84035b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b827fc640 1 -- 172.21.15.122:0/2591294573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84039040 con 0x7f8b8c19cd20 2024-04-02T13:44:33.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b827fc640 1 -- 172.21.15.122:0/2591294573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b84044070 con 0x7f8b8c19cd20 2024-04-02T13:44:33.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c19daa0 con 0x7f8b8c19cd20 2024-04-02T13:44:33.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b827fc640 1 -- 172.21.15.122:0/2591294573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b840043a0 con 0x7f8b8c19cd20 2024-04-02T13:44:33.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.220+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c1a52c0 con 0x7f8b8c19cd20 2024-04-02T13:44:33.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 Processor -- start 2024-04-02T13:44:33.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b827fc640 1 -- 172.21.15.122:0/2591294573 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8b84004540 con 0x7f8b8c19cd20 2024-04-02T13:44:33.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 -- start start 2024-04-02T13:44:33.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b433ff0 0x55684b4343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b441bd0 0x55684b443fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55684b398770 con 0x55684b434910 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55684b340960 con 0x55684b433ff0 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b961f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55684b3948c0 con 0x55684b441bd0 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46674/0 (socket says 172.21.15.122:46674) 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 -- 172.21.15.122:0/3995330516 learned_addr learned my addr 172.21.15.122:0/3995330516 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b441bd0 0x55684b443fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b827fc640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 0x7f8b780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b433ff0 0x55684b4343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 -- 172.21.15.122:0/3995330516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b441bd0 msgr2=0x55684b443fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b441bd0 0x55684b443fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 -- 172.21.15.122:0/3995330516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b433ff0 msgr2=0x55684b4343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b433ff0 0x55684b4343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b912d4640 1 -- 172.21.15.122:0/3995330516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684b1747e0 con 0x55684b434910 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.223+0000 7f8b91ad5640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 0x7f8b780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b827fc640 1 -- 172.21.15.122:0/2591294573 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f8b840f2f90 con 0x7f8b8c19cd20 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800ac10 tx=0x7f8b8800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec41570fbaadd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b777fe640 1 -- 172.21.15.122:0/3995330516 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8801d070 con 0x55684b434910 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b777fe640 1 -- 172.21.15.122:0/3995330516 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b88018b20 con 0x55684b434910 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b777fe640 1 -- 172.21.15.122:0/3995330516 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88010b50 con 0x55684b434910 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b961f0740 1 -- 172.21.15.122:0/3995330516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 msgr2=0x55684b441690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b91ad5640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 0x7f8b780a6260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b840bb0b0 tx=0x7f8b84090690 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.224+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800ac10 tx=0x7f8b8800f790 comp rx=0 tx=0).stop 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- 172.21.15.122:0/3995330516 shutdown_connections 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b441bd0 0x55684b443fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b441690 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3995330516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b433ff0 0x55684b4343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- 172.21.15.122:0/3995330516 >> 172.21.15.122:0/3995330516 conn(0x55684b42e9f0 msgr2=0x55684b42edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- 172.21.15.122:0/3995330516 shutdown_connections 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- 172.21.15.122:0/3995330516 wait complete. 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- start start 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 Processor -- start 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b43ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b43f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55684b444990 con 0x55684b434910 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55684b353d20 con 0x55684b441bd0 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55684b444b10 con 0x55684b433ff0 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b961f0740 10 client.0 _ll_register_callbacks cb 0x55684b320a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:33.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.225+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46688/0 (socket says 172.21.15.122:46688) 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 -- 172.21.15.122:0/3064003065 learned_addr learned my addr 172.21.15.122:0/3064003065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b43f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b43ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 -- 172.21.15.122:0/3064003065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 msgr2=0x55684b43ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b43ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 -- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 msgr2=0x55684b43f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b43f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 -- 172.21.15.122:0/3064003065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684b1d4f70 con 0x55684b434910 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b43ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8b88010470 tx=0x7f8b88005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db0e63dfb00e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8801d050 con 0x55684b434910 2024-04-02T13:44:33.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f8b88018810 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b88018b20 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8b88031050 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b961f0740 10 client.18441 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55684b06c170 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b88018b20 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f8b880046c0 con 0x55684b434910 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b961f0740 10 client.18441 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.226+0000 7f8b961f0740 10 client.18441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b961f0740 20 client.18441 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b451fc0 0x55684b4543b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b961f0740 4 client.18441 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55684b3a5cb0 con 0x55684b451fc0 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b451fc0 0x55684b4543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b74ff9640 10 client.18441 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.227+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b451fc0 0x55684b4543b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994023b31ed19521 server_cookie=24e0beaeaa8b1311 in_seq=0 out_seq=0 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3064003065 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55684b3a5cb0 con 0x55684b451fc0 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b74ff9640 10 client.18441 handle_command_reply: tid=0 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 client.18441 shutdown 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 msgr2=0x55684b43f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f8b88010470 tx=0x7f8b88005e30 comp rx=0 tx=0).stop 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 shutdown_connections 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b451fc0 0x55684b4543b0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b43f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b43f130 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3064003065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b43ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b74ff9640 0 client.18441 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 >> 172.21.15.122:0/3064003065 conn(0x55684b42e9f0 msgr2=0x55684b430740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 shutdown_connections 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 1 -- 172.21.15.122:0/3064003065 wait complete. 2024-04-02T13:44:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.228+0000 7f8b961f0740 20 client.18441 trim_cache size 0 max 16384 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 Processor -- start 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 -- start start 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b441320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b459a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1acd60 con 0x55684b434910 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1ae8a0 con 0x55684b441bd0 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b961f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1ad430 con 0x55684b433ff0 2024-04-02T13:44:33.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46694/0 (socket says 172.21.15.122:46694) 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 -- 172.21.15.122:0/3917516535 learned_addr learned my addr 172.21.15.122:0/3917516535 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b441320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b459a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 -- 172.21.15.122:0/3917516535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 msgr2=0x55684b441320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b441320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 -- 172.21.15.122:0/3917516535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 msgr2=0x55684b459a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b459a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 -- 172.21.15.122:0/3917516535 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55684b43dba0 con 0x55684b434910 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b459a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.246+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800fc90 tx=0x7f8b88005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835600a5f933743f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b76ffd640 1 -- 172.21.15.122:0/3917516535 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8801d040 con 0x55684b434910 2024-04-02T13:44:33.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b76ffd640 1 -- 172.21.15.122:0/3917516535 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8801e040 con 0x55684b434910 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b76ffd640 1 -- 172.21.15.122:0/3917516535 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b88018a00 con 0x55684b434910 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 -- 172.21.15.122:0/3917516535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 msgr2=0x55684b44d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800fc90 tx=0x7f8b88005700 comp rx=0 tx=0).stop 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 -- 172.21.15.122:0/3917516535 shutdown_connections 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b441bd0 0x55684b459a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b434910 0x55684b44d920 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3917516535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b441320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 -- 172.21.15.122:0/3917516535 >> 172.21.15.122:0/3917516535 conn(0x55684b42e9f0 msgr2=0x55684b430610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 -- 172.21.15.122:0/3917516535 shutdown_connections 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.247+0000 7f8b961f0740 1 -- 172.21.15.122:0/3917516535 wait complete. 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 -- start start 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 Processor -- start 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b4b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 0x55684b4b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1ae1d0 con 0x55684b441bd0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1adb00 con 0x55684b434910 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c1aad60 con 0x55684b433ff0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b961f0740 10 client.0 _ll_register_callbacks cb 0x7f8b8801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46700/0 (socket says 172.21.15.122:46700) 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 -- 172.21.15.122:0/3458788434 learned_addr learned my addr 172.21.15.122:0/3458788434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b4b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 0x55684b4b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 -- 172.21.15.122:0/3458788434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 msgr2=0x55684b4b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b4b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 -- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 msgr2=0x55684b4b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 0x55684b4b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b90ad3640 1 -- 172.21.15.122:0/3458788434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55684b434740 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.248+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 0x55684b4b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b90ad3640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b8800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab273239080a6c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7f8b7c01dd00 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8801e040 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8b7c031050 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 10 client.18447 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55684b4b8c20 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b8801e040 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f8b88019ce0 con 0x55684b441bd0 2024-04-02T13:44:33.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 10 client.18447 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 10 client.18447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 20 client.18447 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b4b9320 0x55684b4bb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 4 client.18447 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55684b3a0000 con 0x55684b4b9320 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.249+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b4b9320 0x55684b4bb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.250+0000 7f8b74ff9640 10 client.18447 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.250+0000 7f8b912d4640 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b4b9320 0x55684b4bb740 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48bfe87a3b70f27c server_cookie=3df186e2ffd7e974 in_seq=0 out_seq=0 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b74ff9640 1 -- 172.21.15.122:0/3458788434 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55684b3a0000 con 0x55684b4b9320 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b74ff9640 10 client.18447 handle_command_reply: tid=0 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 client.18447 shutdown 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 msgr2=0x55684b446190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c000c00 tx=0x7f8b8800a470 comp rx=0 tx=0).stop 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 shutdown_connections 2024-04-02T13:44:33.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55684b4b9320 0x55684b4bb740 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55684b441bd0 0x55684b446190 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b74ff9640 0 client.18447 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55684b434910 0x55684b4b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 --2- 172.21.15.122:0/3458788434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55684b433ff0 0x55684b4b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 >> 172.21.15.122:0/3458788434 conn(0x55684b42e9f0 msgr2=0x55684b42fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 shutdown_connections 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 1 -- 172.21.15.122:0/3458788434 wait complete. 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.251+0000 7f8b961f0740 20 client.18447 trim_cache size 0 max 16384 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:44:33.261 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:44:33.262 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "417b39cc-d812-460e-9f1f-248b36f7aa96", 2024-04-02T13:44:33.262 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:44:33.262 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:44:33.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 msgr2=0x7f8b780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 0x7f8b780a6260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b840bb0b0 tx=0x7f8b84090690 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 msgr2=0x7f8b8c19d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f8b8402c700 tx=0x7f8b84035b60 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 shutdown_connections 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8b780a3da0 0x7f8b780a6260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b8c19cd20 0x7f8b8c19d1d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b8c106e30 0x7f8b8c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 --2- 172.21.15.122:0/2591294573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b8c100670 0x7f8b8c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 >> 172.21.15.122:0/2591294573 conn(0x7f8b8c075c60 msgr2=0x7f8b8c0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 shutdown_connections 2024-04-02T13:44:33.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.252+0000 7f8b93538640 1 -- 172.21.15.122:0/2591294573 wait complete. 2024-04-02T13:44:33.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:44:33.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 -- 172.21.15.122:0/4059820061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90101a90 msgr2=0x7fbe90101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 --2- 172.21.15.122:0/4059820061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90101a90 0x7fbe90101f10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009930 tx=0x7fbe8002bc30 comp rx=0 tx=0).stop 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 -- 172.21.15.122:0/4059820061 shutdown_connections 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 --2- 172.21.15.122:0/4059820061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90102450 0x7fbe9010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 --2- 172.21.15.122:0/4059820061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90101a90 0x7fbe90101f10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 --2- 172.21.15.122:0/4059820061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 0x7fbe90107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 -- 172.21.15.122:0/4059820061 >> 172.21.15.122:0/4059820061 conn(0x7fbe90078f00 msgr2=0x7fbe90079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.394+0000 7fbe974fb640 1 -- 172.21.15.122:0/4059820061 shutdown_connections 2024-04-02T13:44:33.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 -- 172.21.15.122:0/4059820061 wait complete. 2024-04-02T13:44:33.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 Processor -- start 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 -- start start 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 0x7fbe9019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90111eb0 con 0x7fbe90102450 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90111d30 con 0x7fbe90101a90 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.395+0000 7fbe974fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe90112030 con 0x7fbe90107aa0 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43380/0 (socket says 172.21.15.122:43380) 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46720/0 (socket says 172.21.15.122:46720) 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 -- 172.21.15.122:0/3790734977 learned_addr learned my addr 172.21.15.122:0/3790734977 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 0x7fbe9019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 -- 172.21.15.122:0/3790734977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 msgr2=0x7fbe9019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 0x7fbe9019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 -- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 msgr2=0x7fbe90198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 -- 172.21.15.122:0/3790734977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9019d400 con 0x7fbe90101a90 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe94a96640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe95297640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00e9f0 tx=0x7fbe8c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe867fc640 1 -- 172.21.15.122:0/3790734977 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00ccc0 con 0x7fbe90101a90 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe867fc640 1 -- 172.21.15.122:0/3790734977 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c00ce60 con 0x7fbe90101a90 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9019d6f0 con 0x7fbe90101a90 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe867fc640 1 -- 172.21.15.122:0/3790734977 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c010690 con 0x7fbe90101a90 2024-04-02T13:44:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.396+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe90073d30 con 0x7fbe90101a90 2024-04-02T13:44:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe867fc640 1 -- 172.21.15.122:0/3790734977 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbe8c004ea0 con 0x7fbe90101a90 2024-04-02T13:44:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 Processor -- start 2024-04-02T13:44:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 -- start start 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558cf0051830 con 0x558cf00f6dd0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cefff9820 con 0x558cf00ee1f0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.399+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cf004d7e0 con 0x558cf00ed8d0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe867fc640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 0x7fbe7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe95297640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe95a98640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46726/0 (socket says 172.21.15.122:46726) 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 -- 172.21.15.122:0/1090728891 learned_addr learned my addr 172.21.15.122:0/1090728891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 0x7fbe7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe867fc640 1 -- 172.21.15.122:0/3790734977 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fbe8c0d0f70 con 0x7fbe90101a90 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 -- 172.21.15.122:0/1090728891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 msgr2=0x558cf00edcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00edcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 -- 172.21.15.122:0/1090728891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 msgr2=0x558cf00f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 -- 172.21.15.122:0/1090728891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cefe2f7d0 con 0x558cf00f6dd0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870f1507c22140bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 0x7fbe7c0a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009b80 tx=0x7fbe88009bf0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe7b7fe640 1 -- 172.21.15.122:0/1090728891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80042070 con 0x558cf00f6dd0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe7b7fe640 1 -- 172.21.15.122:0/1090728891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80004a70 con 0x558cf00f6dd0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.400+0000 7fbe7b7fe640 1 -- 172.21.15.122:0/1090728891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80005190 con 0x558cf00f6dd0 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1090728891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 msgr2=0x558cf00f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fbe80006010 tx=0x7fbe80009630 comp rx=0 tx=0).stop 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1090728891 shutdown_connections 2024-04-02T13:44:33.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f91c0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1090728891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00edcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1090728891 >> 172.21.15.122:0/1090728891 conn(0x558cf00e8000 msgr2=0x558cf00e83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1090728891 shutdown_connections 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.401+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1090728891 wait complete. 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 -- start start 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 Processor -- start 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558cf00f9b60 con 0x558cf00f6dd0 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558cf000cd30 con 0x558cf00ee1f0 2024-04-02T13:44:33.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cf00f9ce0 con 0x558cf00ed8d0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.402+0000 7fbe9a1b7740 10 client.0 _ll_register_callbacks cb 0x558cf000c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46730/0 (socket says 172.21.15.122:46730) 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 -- 172.21.15.122:0/3292117707 learned_addr learned my addr 172.21.15.122:0/3292117707 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe94a96640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95297640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 -- 172.21.15.122:0/3292117707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 msgr2=0x558cf00f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 -- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 msgr2=0x558cf00f3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 -- 172.21.15.122:0/3292117707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cefd653f0 con 0x558cf00f6dd0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe94a96640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbe88003e40 tx=0x7fbe88009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3764847c731369f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:33.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88004d20 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fbe88020670 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe80004a70 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe88020870 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe9a1b7740 10 client.18459 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558cefd250f0 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.403+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe80004a70 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fbe88027070 con 0x558cf00f6dd0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 10 client.18459 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 10 client.18459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 20 client.18459 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf0109110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 4 client.18459 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cf005ed70 con 0x558cf0106cc0 2024-04-02T13:44:33.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe95297640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf0109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe78ff9640 10 client.18459 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.404+0000 7fbe95297640 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf0109110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2fd11811d51cec server_cookie=64268f60ae85825e in_seq=0 out_seq=0 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe78ff9640 1 -- 172.21.15.122:0/3292117707 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558cf005ed70 con 0x558cf0106cc0 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe78ff9640 10 client.18459 handle_command_reply: tid=0 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 client.18459 shutdown 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 msgr2=0x558cf00f3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fbe88003e40 tx=0x7fbe88009b10 comp rx=0 tx=0).stop 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 shutdown_connections 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf0109110 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00f3ea0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ee1f0 0x558cf00f3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe78ff9640 0 client.18459 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3292117707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.405+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 >> 172.21.15.122:0/3292117707 conn(0x558cf00e8000 msgr2=0x558cf00e9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.406+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 shutdown_connections 2024-04-02T13:44:33.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.406+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3292117707 wait complete. 2024-04-02T13:44:33.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.406+0000 7fbe9a1b7740 20 client.18459 trim_cache size 0 max 16384 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 Processor -- start 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 -- start start 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ed8d0 0x558cf00f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 0x558cf00ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901b1260 con 0x558cf00f6dd0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901b2da0 con 0x558cf00ed8d0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.427+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901b1930 con 0x558cf00ee1f0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 0x558cf00ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46734/0 (socket says 172.21.15.122:46734) 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 -- 172.21.15.122:0/3033259973 learned_addr learned my addr 172.21.15.122:0/3033259973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95297640 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ed8d0 0x558cf00f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 -- 172.21.15.122:0/3033259973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 msgr2=0x558cf00ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 0x558cf00ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 -- 172.21.15.122:0/3033259973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ed8d0 msgr2=0x558cf00f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ed8d0 0x558cf00f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 -- 172.21.15.122:0/3033259973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf00e7e30 con 0x558cf00f6dd0 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe94a96640 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 0x558cf00ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:44:33.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe95a98640 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbe880055c0 tx=0x7fbe8800fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d961c499ddfd475 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe7affd640 1 -- 172.21.15.122:0/3033259973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8803b070 con 0x558cf00f6dd0 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe7affd640 1 -- 172.21.15.122:0/3033259973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe880047d0 con 0x558cf00f6dd0 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.428+0000 7fbe7affd640 1 -- 172.21.15.122:0/3033259973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88004d10 con 0x558cf00f6dd0 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3033259973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 msgr2=0x558cf00e70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fbe880055c0 tx=0x7fbe8800fd40 comp rx=0 tx=0).stop 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3033259973 shutdown_connections 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00f6dd0 0x558cf00e70c0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ee1f0 0x558cf00ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/3033259973 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00ed8d0 0x558cf00f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3033259973 >> 172.21.15.122:0/3033259973 conn(0x558cf00e8000 msgr2=0x558cf00e69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3033259973 shutdown_connections 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/3033259973 wait complete. 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 -- start start 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.429+0000 7fbe9a1b7740 1 Processor -- start 2024-04-02T13:44:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00f6dd0 0x558cf00f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901b26d0 con 0x558cf00ee1f0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901b2000 con 0x558cf00f6dd0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe901af260 con 0x558cf00ed8d0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe9a1b7740 10 client.0 _ll_register_callbacks cb 0x7fbe88003d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46744/0 (socket says 172.21.15.122:46744) 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 -- 172.21.15.122:0/1907618838 learned_addr learned my addr 172.21.15.122:0/1907618838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe95297640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe95a98640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00f6dd0 0x558cf00f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 -- 172.21.15.122:0/1907618838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 msgr2=0x558cf00f0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 -- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00f6dd0 msgr2=0x558cf00f13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00f6dd0 0x558cf00f13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.430+0000 7fbe94a96640 1 -- 172.21.15.122:0/1907618838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cf016c540 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe94a96640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbe800096c0 tx=0x7fbe8002c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6b6dc4e43a3cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe80042040 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9525+0+0 (secure 0 0 0) 0x7fbe80004f30 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe880047d0 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbe80005130 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 10 client.18465 fetch_fsmap learned FSMap version 40 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558cf016d850 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe880047d0 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fbe80034440 con 0x558cf00ee1f0 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 10 client.18465 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 10 client.18465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:44:33.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 20 client.18465 populate_metadata read hostname 'smithi122' 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf016fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 4 client.18465 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.431+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cf00590c0 con 0x558cf0106cc0 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe95297640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf016fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe78ff9640 10 client.18465 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe95297640 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf016fa30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41910f622f2c3b server_cookie=b877b0cf1215d33e in_seq=0 out_seq=0 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe78ff9640 1 -- 172.21.15.122:0/1907618838 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558cf00590c0 con 0x558cf0106cc0 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe78ff9640 10 client.18465 handle_command_reply: tid=0 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 client.18465 shutdown 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 msgr2=0x558cf00f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fbe800096c0 tx=0x7fbe8002c3c0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 shutdown_connections 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558cf0106cc0 0x558cf016fa30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558cf00f6dd0 0x558cf00f13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558cf00ee1f0 0x558cf00f0e50 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 --2- 172.21.15.122:0/1907618838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cf00ed8d0 0x558cf00f0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 >> 172.21.15.122:0/1907618838 conn(0x558cf00e8000 msgr2=0x558cf00e7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.432+0000 7fbe78ff9640 0 client.18465 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 shutdown_connections 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe9a1b7740 1 -- 172.21.15.122:0/1907618838 wait complete. 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe9a1b7740 20 client.18465 trim_cache size 0 max 16384 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:44:33.442 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: "417b39cc-d812-460e-9f1f-248b36f7aa96": { 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "417b39cc-d812-460e-9f1f-248b36f7aa96", 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:44:33.443 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 msgr2=0x7fbe7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 0x7fbe7c0a61e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009b80 tx=0x7fbe88009bf0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 msgr2=0x7fbe90198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c00e9f0 tx=0x7fbe8c00edf0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 shutdown_connections 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe7c0a3d20 0x7fbe7c0a61e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe90107aa0 0x7fbe9019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe90102450 0x7fbe90198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 --2- 172.21.15.122:0/3790734977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe90101a90 0x7fbe90198410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 >> 172.21.15.122:0/3790734977 conn(0x7fbe90078f00 msgr2=0x7fbe90104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 shutdown_connections 2024-04-02T13:44:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:44:33.433+0000 7fbe974fb640 1 -- 172.21.15.122:0/3790734977 wait complete. 2024-04-02T13:44:33.447 INFO:tasks.cephfs.filesystem:scrub status for tag:417b39cc-d812-460e-9f1f-248b36f7aa96 - {'path': '/', 'tag': '417b39cc-d812-460e-9f1f-248b36f7aa96', 'options': 'recursive,force'} 2024-04-02T13:44:33.743 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:33 smithi187 ceph-mon[36064]: pgmap v290: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-04-02T13:44:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:33 smithi188 ceph-mon[36117]: pgmap v290: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-04-02T13:44:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:33 smithi122 ceph-mon[32114]: pgmap v290: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-04-02T13:44:33.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 318040 3.31 MB/sec execute 354 sec latency 1104.294 ms 2024-04-02T13:44:34.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:44:34.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:34 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:34 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:44:34.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:44:34.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:34 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:44:34.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 319669 3.34 MB/sec execute 355 sec latency 19.369 ms 2024-04-02T13:44:35.668 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:35 smithi188 ceph-mon[36117]: pgmap v291: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-04-02T13:44:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:35 smithi187 ceph-mon[36064]: pgmap v291: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-04-02T13:44:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:35 smithi122 ceph-mon[32114]: pgmap v291: 129 pgs: 129 active+clean; 234 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 183 op/s 2024-04-02T13:44:35.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 320037 3.33 MB/sec execute 356 sec latency 622.977 ms 2024-04-02T13:44:36.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 320037 3.32 MB/sec execute 357 sec latency 1623.079 ms 2024-04-02T13:44:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:37 smithi187 ceph-mon[36064]: pgmap v292: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-02T13:44:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:37 smithi188 ceph-mon[36117]: pgmap v292: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-02T13:44:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:37 smithi122 ceph-mon[32114]: pgmap v292: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-04-02T13:44:37.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 320037 3.31 MB/sec execute 358 sec latency 2623.114 ms 2024-04-02T13:44:38.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 321107 3.32 MB/sec execute 359 sec latency 3068.077 ms 2024-04-02T13:44:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:39 smithi187 ceph-mon[36064]: pgmap v293: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:39.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:39 smithi188 ceph-mon[36117]: pgmap v293: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:39 smithi122 ceph-mon[32114]: pgmap v293: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:39.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 323198 3.35 MB/sec execute 360 sec latency 11.947 ms 2024-04-02T13:44:40.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 323585 3.34 MB/sec execute 361 sec latency 660.136 ms 2024-04-02T13:44:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:41 smithi187 ceph-mon[36064]: pgmap v294: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:41 smithi188 ceph-mon[36117]: pgmap v294: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:41 smithi122 ceph-mon[32114]: pgmap v294: 129 pgs: 129 active+clean; 240 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 189 op/s 2024-04-02T13:44:41.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 323585 3.33 MB/sec execute 362 sec latency 1660.230 ms 2024-04-02T13:44:42.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:42.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:42.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:42.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:42.756 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:44:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:44:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:44:42.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 323585 3.32 MB/sec execute 363 sec latency 2660.333 ms 2024-04-02T13:44:43.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:43 smithi187 ceph-mon[36064]: pgmap v295: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-04-02T13:44:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:43 smithi188 ceph-mon[36117]: pgmap v295: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-04-02T13:44:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:43 smithi122 ceph-mon[32114]: pgmap v295: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 238 op/s 2024-04-02T13:44:43.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 324710 3.32 MB/sec execute 364 sec latency 3098.133 ms 2024-04-02T13:44:44.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 326658 3.35 MB/sec execute 365 sec latency 14.652 ms 2024-04-02T13:44:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:45 smithi187 ceph-mon[36064]: pgmap v296: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-02T13:44:45.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:45 smithi188 ceph-mon[36117]: pgmap v296: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-02T13:44:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:45 smithi122 ceph-mon[32114]: pgmap v296: 129 pgs: 129 active+clean; 244 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-02T13:44:45.969 INFO:tasks.workunit.client.0.smithi122.stdout: 1 327845 3.36 MB/sec execute 366 sec latency 14.489 ms 2024-04-02T13:44:46.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 329510 3.37 MB/sec execute 367 sec latency 19.516 ms 2024-04-02T13:44:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:47 smithi187 ceph-mon[36064]: pgmap v297: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-04-02T13:44:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:47 smithi188 ceph-mon[36117]: pgmap v297: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-04-02T13:44:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:47 smithi122 ceph-mon[32114]: pgmap v297: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 200 op/s 2024-04-02T13:44:47.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 330552 3.38 MB/sec execute 368 sec latency 93.158 ms 2024-04-02T13:44:48.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 331494 3.38 MB/sec execute 369 sec latency 532.871 ms 2024-04-02T13:44:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:49 smithi187 ceph-mon[36064]: pgmap v298: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:49.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:49 smithi188 ceph-mon[36117]: pgmap v298: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:49 smithi122 ceph-mon[32114]: pgmap v298: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:49.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 333678 3.41 MB/sec execute 370 sec latency 14.669 ms 2024-04-02T13:44:50.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 334151 3.40 MB/sec execute 371 sec latency 476.996 ms 2024-04-02T13:44:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:51 smithi187 ceph-mon[36064]: pgmap v299: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:51 smithi188 ceph-mon[36117]: pgmap v299: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:51 smithi122 ceph-mon[32114]: pgmap v299: 129 pgs: 129 active+clean; 248 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-04-02T13:44:51.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 334151 3.39 MB/sec execute 372 sec latency 1477.111 ms 2024-04-02T13:44:52.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 334151 3.38 MB/sec execute 373 sec latency 2477.232 ms 2024-04-02T13:44:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:53 smithi187 ceph-mon[36064]: pgmap v300: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-02T13:44:53.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:53 smithi188 ceph-mon[36117]: pgmap v300: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-02T13:44:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:53 smithi122 ceph-mon[32114]: pgmap v300: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-02T13:44:53.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 334986 3.39 MB/sec execute 374 sec latency 2915.449 ms 2024-04-02T13:44:54.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 337176 3.41 MB/sec execute 375 sec latency 19.362 ms 2024-04-02T13:44:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:55 smithi187 ceph-mon[36064]: pgmap v301: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-02T13:44:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:55 smithi188 ceph-mon[36117]: pgmap v301: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-02T13:44:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:55 smithi122 ceph-mon[32114]: pgmap v301: 129 pgs: 129 active+clean; 255 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-02T13:44:55.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 338380 3.42 MB/sec execute 376 sec latency 14.685 ms 2024-04-02T13:44:56.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 339905 3.43 MB/sec execute 377 sec latency 22.460 ms 2024-04-02T13:44:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:57 smithi187 ceph-mon[36064]: pgmap v302: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-04-02T13:44:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:57 smithi188 ceph-mon[36117]: pgmap v302: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-04-02T13:44:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:57 smithi122 ceph-mon[32114]: pgmap v302: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 214 op/s 2024-04-02T13:44:57.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 341185 3.44 MB/sec execute 378 sec latency 108.297 ms 2024-04-02T13:44:58.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 341241 3.43 MB/sec execute 379 sec latency 547.394 ms 2024-04-02T13:44:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:44:59 smithi187 ceph-mon[36064]: pgmap v303: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:44:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:44:59 smithi188 ceph-mon[36117]: pgmap v303: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:44:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:44:59 smithi122 ceph-mon[32114]: pgmap v303: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:44:59.970 INFO:tasks.workunit.client.0.smithi122.stdout: 1 341241 3.42 MB/sec execute 380 sec latency 1515.481 ms 2024-04-02T13:45:00.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 341534 3.41 MB/sec execute 381 sec latency 2131.867 ms 2024-04-02T13:45:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:01 smithi187 ceph-mon[36064]: pgmap v304: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:45:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:01 smithi188 ceph-mon[36117]: pgmap v304: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:45:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:01 smithi122 ceph-mon[32114]: pgmap v304: 129 pgs: 129 active+clean; 258 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2024-04-02T13:45:01.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 342344 3.42 MB/sec execute 382 sec latency 73.656 ms 2024-04-02T13:45:02.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 344376 3.44 MB/sec execute 383 sec latency 40.104 ms 2024-04-02T13:45:03.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:03.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.564+0000 7f233af82640 1 -- 172.21.15.122:0/546342783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334100970 msgr2=0x7f2334100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.564+0000 7f233af82640 1 --2- 172.21.15.122:0/546342783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334100970 0x7f2334100df0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f2320009930 tx=0x7f232002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- 172.21.15.122:0/546342783 shutdown_connections 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- 172.21.15.122:0/546342783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334101330 0x7f233410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- 172.21.15.122:0/546342783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334100970 0x7f2334100df0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- 172.21.15.122:0/546342783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 0x7f2334106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- 172.21.15.122:0/546342783 >> 172.21.15.122:0/546342783 conn(0x7f2334077b00 msgr2=0x7f2334077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- 172.21.15.122:0/546342783 shutdown_connections 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- 172.21.15.122:0/546342783 wait complete. 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 Processor -- start 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- start start 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 0x7f2334199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 0x7f23341a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2334110d60 con 0x7f2334101330 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2334110be0 con 0x7f2334106980 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.565+0000 7f233af82640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2334110ee0 con 0x7f2334100970 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 0x7f2334199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60934/0 (socket says 172.21.15.122:60934) 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 0x7f2334199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38800/0 (socket says 172.21.15.122:38800) 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 -- 172.21.15.122:0/4293188347 learned_addr learned my addr 172.21.15.122:0/4293188347 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f233951f640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 0x7f23341a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 -- 172.21.15.122:0/4293188347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 msgr2=0x7f2334199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 0x7f2334199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 -- 172.21.15.122:0/4293188347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 msgr2=0x7f23341a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 0x7f23341a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 -- 172.21.15.122:0/4293188347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23341a47c0 con 0x7f2334101330 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2333fff640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2320009230 tx=0x7f232002cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2331ffb640 1 -- 172.21.15.122:0/4293188347 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320041070 con 0x7f2334101330 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2331ffb640 1 -- 172.21.15.122:0/4293188347 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004450 con 0x7f2334101330 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23341a4990 con 0x7f2334101330 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f2331ffb640 1 -- 172.21.15.122:0/4293188347 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2320034440 con 0x7f2334101330 2024-04-02T13:45:03.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.566+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23341a4e00 con 0x7f2334101330 2024-04-02T13:45:03.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:03 smithi122 ceph-mon[32114]: pgmap v305: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-04-02T13:45:03.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:03 smithi122 ceph-mon[32114]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.569+0000 7f2331ffb640 1 -- 172.21.15.122:0/4293188347 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2320044070 con 0x7f2334101330 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2331ffb640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 0x7f23240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2338d1e640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 0x7f23240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 Processor -- start 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2331ffb640 1 -- 172.21.15.122:0/4293188347 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f23200f24d0 con 0x7f2334101330 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 -- start start 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2712340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd2719e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fd2669830 con 0x563fd2700470 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fd2611820 con 0x563fd26ffab0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233db6c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fd26657e0 con 0x563fd2711f60 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2338d1e640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 0x7f23240a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2328009b50 tx=0x7f2328005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2712340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2333fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f233951f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd2719e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2333fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37926/0 (socket says 172.21.15.122:37926) 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.570+0000 7f2333fff640 1 -- 172.21.15.122:0/329393310 learned_addr learned my addr 172.21.15.122:0/329393310 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 -- 172.21.15.122:0/329393310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 msgr2=0x563fd2712340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2712340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 -- 172.21.15.122:0/329393310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 msgr2=0x563fd2719e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd2719e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 -- 172.21.15.122:0/329393310 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd24477d0 con 0x563fd26ffab0 2024-04-02T13:45:03.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f233951f640 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd2719e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f2333fff640 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f23200c6880 tx=0x7f2320009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767159803d926262 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f231effd640 1 -- 172.21.15.122:0/329393310 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232004b040 con 0x563fd26ffab0 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f231effd640 1 -- 172.21.15.122:0/329393310 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004c80 con 0x563fd26ffab0 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.571+0000 7f231effd640 1 -- 172.21.15.122:0/329393310 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23200efbd0 con 0x563fd26ffab0 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- 172.21.15.122:0/329393310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 msgr2=0x563fd26fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f23200c6880 tx=0x7f2320009260 comp rx=0 tx=0).stop 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- 172.21.15.122:0/329393310 shutdown_connections 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd2719e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd26ffab0 0x563fd26fff30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 --2- 172.21.15.122:0/329393310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2712340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- 172.21.15.122:0/329393310 >> 172.21.15.122:0/329393310 conn(0x563fd2702570 msgr2=0x563fd2702950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- 172.21.15.122:0/329393310 shutdown_connections 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- 172.21.15.122:0/329393310 wait complete. 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.572+0000 7f233db6c740 1 -- start start 2024-04-02T13:45:03.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 Processor -- start 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 0x563fd2716ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2717400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563fd2611820 con 0x563fd26ffab0 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563fd2624d30 con 0x563fd2700470 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563fd271a370 con 0x563fd2711f60 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233db6c740 10 client.0 _ll_register_callbacks cb 0x563fd2624720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60948/0 (socket says 172.21.15.122:60948) 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 -- 172.21.15.122:0/2858280473 learned_addr learned my addr 172.21.15.122:0/2858280473 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2333fff640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 0x563fd2716ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f233951f640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2717400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 -- 172.21.15.122:0/2858280473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 msgr2=0x563fd2717400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2717400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.573+0000 7f2338d1e640 1 -- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 msgr2=0x563fd2716ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 0x563fd2716ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2338d1e640 1 -- 172.21.15.122:0/2858280473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd237d3f0 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2333fff640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 0x563fd2716ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2328003990 tx=0x7f2328004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2db6ee4dd2fafca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232801fdc0 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2320004c80 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f2328005020 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2328005220 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f233db6c740 10 client.18477 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563fd233d0f0 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2320004c80 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.574+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f2328026070 con 0x563fd26ffab0 2024-04-02T13:45:03.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 10 client.18477 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 10 client.18477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 20 client.18477 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd27273b0 0x563fd27297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 4 client.18477 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fd2676d70 con 0x563fd27273b0 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f2333fff640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd27273b0 0x563fd27297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f2313fff640 10 client.18477 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.575+0000 7f2333fff640 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd27273b0 0x563fd27297e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c89faa75b4cbf8 server_cookie=9a7a1fada5859c4c in_seq=0 out_seq=0 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f2313fff640 1 -- 172.21.15.122:0/2858280473 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fd2676d70 con 0x563fd27273b0 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f2313fff640 10 client.18477 handle_command_reply: tid=0 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 client.18477 shutdown 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 msgr2=0x563fd2719b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2328003990 tx=0x7f2328004730 comp rx=0 tx=0).stop 2024-04-02T13:45:03.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 shutdown_connections 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd27273b0 0x563fd27297e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2711f60 0x563fd2717400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2700470 0x563fd2716ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f2313fff640 0 client.18477 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 --2- 172.21.15.122:0/2858280473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd26ffab0 0x563fd2719b30 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.576+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 >> 172.21.15.122:0/2858280473 conn(0x563fd2702570 msgr2=0x563fd2701440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.577+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 shutdown_connections 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.577+0000 7f233db6c740 1 -- 172.21.15.122:0/2858280473 wait complete. 2024-04-02T13:45:03.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.577+0000 7f233db6c740 20 client.18477 trim_cache size 0 max 16384 2024-04-02T13:45:03.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 Processor -- start 2024-04-02T13:45:03.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 -- start start 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2711f60 0x563fd272f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2717590 0x563fd271f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23341acf90 con 0x563fd2700470 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23341aead0 con 0x563fd2711f60 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233db6c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f23341ad660 con 0x563fd2717590 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60958/0 (socket says 172.21.15.122:60958) 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 -- 172.21.15.122:0/2510158746 learned_addr learned my addr 172.21.15.122:0/2510158746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2333fff640 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2711f60 0x563fd272f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f233951f640 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2717590 0x563fd271f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 -- 172.21.15.122:0/2510158746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2717590 msgr2=0x563fd271f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2717590 0x563fd271f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 -- 172.21.15.122:0/2510158746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2711f60 msgr2=0x563fd272f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2711f60 0x563fd272f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.605+0000 7f2338d1e640 1 -- 172.21.15.122:0/2510158746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd26fb8d0 con 0x563fd2700470 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.606+0000 7f2338d1e640 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2328027640 tx=0x7f23280046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fc5b87ffcb5f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.606+0000 7f231e7fc640 1 -- 172.21.15.122:0/2510158746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232801f800 con 0x563fd2700470 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.606+0000 7f231e7fc640 1 -- 172.21.15.122:0/2510158746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23280273f0 con 0x563fd2700470 2024-04-02T13:45:03.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.606+0000 7f231e7fc640 1 -- 172.21.15.122:0/2510158746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328004a00 con 0x563fd2700470 2024-04-02T13:45:03.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- 172.21.15.122:0/2510158746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 msgr2=0x563fd272f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f2328027640 tx=0x7f23280046a0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- 172.21.15.122:0/2510158746 shutdown_connections 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2717590 0x563fd271f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2711f60 0x563fd272f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 --2- 172.21.15.122:0/2510158746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2700470 0x563fd272f0b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- 172.21.15.122:0/2510158746 >> 172.21.15.122:0/2510158746 conn(0x563fd2702570 msgr2=0x563fd2716e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- 172.21.15.122:0/2510158746 shutdown_connections 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- 172.21.15.122:0/2510158746 wait complete. 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 -- start start 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.607+0000 7f233db6c740 1 Processor -- start 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 0x563fd271d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2717590 0x563fd271dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23341ae400 con 0x563fd2711f60 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23341add30 con 0x563fd2717590 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f23341aaf90 con 0x563fd2700470 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233db6c740 10 client.0 _ll_register_callbacks cb 0x7f2328003610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2333fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2333fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60974/0 (socket says 172.21.15.122:60974) 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2333fff640 1 -- 172.21.15.122:0/1022110637 learned_addr learned my addr 172.21.15.122:0/1022110637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 0x563fd271d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f233951f640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2717590 0x563fd271dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 -- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2717590 msgr2=0x563fd271dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2717590 0x563fd271dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 -- 172.21.15.122:0/1022110637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 msgr2=0x563fd271d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:03.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 -- 172.21.15.122:0/1022110637 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd278cc50 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2333fff640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2338d1e640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 0x563fd271d2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2328009a10 tx=0x7f232801f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0f87b7acf1272c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.608+0000 7f2313fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23280055c0 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f2328004710 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23280273f0 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2328004910 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 10 client.27352 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563fd278df00 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23280273f0 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f232803f070 con 0x563fd2700470 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 10 client.27352 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:03.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 10 client.27352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 20 client.27352 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd278e510 0x563fd2790900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 4 client.27352 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563fd26710c0 con 0x563fd278e510 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.609+0000 7f2333fff640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd278e510 0x563fd2790900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f2313fff640 10 client.27352 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f2333fff640 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd278e510 0x563fd2790900 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4bc19fb7fadfbc7 server_cookie=2b6be26dc20e2bfd in_seq=0 out_seq=0 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f2313fff640 1 -- 172.21.15.122:0/1022110637 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563fd26710c0 con 0x563fd278e510 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f2313fff640 10 client.27352 handle_command_reply: tid=0 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 client.27352 shutdown 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 msgr2=0x563fd271d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 0x563fd271d2b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2328009a10 tx=0x7f232801f870 comp rx=0 tx=0).stop 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 shutdown_connections 2024-04-02T13:45:03.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x563fd278e510 0x563fd2790900 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563fd2717590 0x563fd271dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563fd2711f60 0x563fd271d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 --2- 172.21.15.122:0/1022110637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563fd2700470 0x563fd271d2b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 >> 172.21.15.122:0/1022110637 conn(0x563fd2702570 msgr2=0x563fd2710f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f2313fff640 0 client.27352 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 shutdown_connections 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 1 -- 172.21.15.122:0/1022110637 wait complete. 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.610+0000 7f233db6c740 20 client.27352 trim_cache size 0 max 16384 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:45:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:45:03.620 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:03.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 msgr2=0x7f23240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 0x7f23240a61e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2328009b50 tx=0x7f2328005f90 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 msgr2=0x7f2334199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2320009230 tx=0x7f232002cba0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 shutdown_connections 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f23240a3d20 0x7f23240a61e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2334106980 0x7f23341a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2334101330 0x7f2334199740 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 --2- 172.21.15.122:0/4293188347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2334100970 0x7f2334199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 >> 172.21.15.122:0/4293188347 conn(0x7f2334077b00 msgr2=0x7f2334103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 shutdown_connections 2024-04-02T13:45:03.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.611+0000 7f233af82640 1 -- 172.21.15.122:0/4293188347 wait complete. 2024-04-02T13:45:03.625 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:45:03.626 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:45:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:03 smithi187 ceph-mon[36064]: pgmap v305: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-04-02T13:45:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:03 smithi187 ceph-mon[36064]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:45:03.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 -- 172.21.15.122:0/1488933225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c107ad0 msgr2=0x7f3b0c107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 --2- 172.21.15.122:0/1488933225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c107eb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009cd0 tx=0x7f3b0802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 -- 172.21.15.122:0/1488933225 shutdown_connections 2024-04-02T13:45:03.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 --2- 172.21.15.122:0/1488933225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c102480 0x7f3b0c10e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 --2- 172.21.15.122:0/1488933225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 --2- 172.21.15.122:0/1488933225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c107eb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 -- 172.21.15.122:0/1488933225 >> 172.21.15.122:0/1488933225 conn(0x7f3b0c0fd5b0 msgr2=0x7f3b0c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 -- 172.21.15.122:0/1488933225 shutdown_connections 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.733+0000 7f3b13e60640 1 -- 172.21.15.122:0/1488933225 wait complete. 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 Processor -- start 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 -- start start 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 0x7f3b0c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c111ee0 con 0x7f3b0c102480 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c111d60 con 0x7f3b0c101ac0 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b13e60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c112060 con 0x7f3b0c107ad0 2024-04-02T13:45:03.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b113fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 0x7f3b0c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b11bfc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b11bfc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37976/0 (socket says 172.21.15.122:37976) 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b11bfc640 1 -- 172.21.15.122:0/3536823282 learned_addr learned my addr 172.21.15.122:0/3536823282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.734+0000 7f3b123fd640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 -- 172.21.15.122:0/3536823282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c107ad0 msgr2=0x7f3b0c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 -- 172.21.15.122:0/3536823282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 msgr2=0x7f3b0c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:03.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 -- 172.21.15.122:0/3536823282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c19d3e0 con 0x7f3b0c102480 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b113fb640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 0x7f3b0c198980 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e9f0 tx=0x7f3afc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc00ccc0 con 0x7f3b0c102480 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c19d6d0 con 0x7f3b0c102480 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3afc005020 con 0x7f3b0c102480 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc0049e0 con 0x7f3b0c102480 2024-04-02T13:45:03.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.735+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1a4f60 con 0x7f3b0c102480 2024-04-02T13:45:03.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.736+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c0688f0 con 0x7f3b0c102480 2024-04-02T13:45:03.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.737+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3afc005560 con 0x7f3b0c102480 2024-04-02T13:45:03.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.738+0000 7f3b02ffd640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 0x7f3af80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:03.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.738+0000 7f3b11bfc640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 0x7f3af80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:03.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.738+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f3afc0d0bc0 con 0x7f3b0c102480 2024-04-02T13:45:03.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.738+0000 7f3b11bfc640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 0x7f3af80a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000f80 tx=0x7f3b080098e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:03.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:03.743+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3afc0051d0 con 0x7f3b0c102480 2024-04-02T13:45:03.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:03 smithi188 ceph-mon[36117]: pgmap v305: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 233 op/s 2024-04-02T13:45:03.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:03 smithi188 ceph-mon[36117]: osdmap e102: 12 total, 12 up, 12 in 2024-04-02T13:45:03.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 345586 3.45 MB/sec execute 384 sec latency 52.263 ms 2024-04-02T13:45:04.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.014+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b0c109aa0 con 0x7f3b0c102480 2024-04-02T13:45:04.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.015+0000 7f3b02ffd640 1 -- 172.21.15.122:0/3536823282 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7f3afc09ce20 con 0x7f3b0c102480 2024-04-02T13:45:04.018 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:45:04.018 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:45:04.018 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 msgr2=0x7f3af80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 0x7f3af80a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000f80 tx=0x7f3b080098e0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 msgr2=0x7f3b0c198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 0x7f3b0c198980 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00e9f0 tx=0x7f3afc00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 shutdown_connections 2024-04-02T13:45:04.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3af80a3d20 0x7f3af80a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b0c107ad0 0x7f3b0c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b0c102480 0x7f3b0c198980 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 --2- 172.21.15.122:0/3536823282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b0c101ac0 0x7f3b0c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 >> 172.21.15.122:0/3536823282 conn(0x7f3b0c0fd5b0 msgr2=0x7f3b0c0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 shutdown_connections 2024-04-02T13:45:04.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.018+0000 7f3b13e60640 1 -- 172.21.15.122:0/3536823282 wait complete. 2024-04-02T13:45:04.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:45:04.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.158+0000 7f751dd3d640 1 -- 172.21.15.122:0/1028185762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518107b50 msgr2=0x7f7518107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.158+0000 7f751dd3d640 1 --2- 172.21.15.122:0/1028185762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518107b50 0x7f7518107f30 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f7508009cd0 tx=0x7f750802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 -- 172.21.15.122:0/1028185762 shutdown_connections 2024-04-02T13:45:04.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 --2- 172.21.15.122:0/1028185762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75180696c0 0x7f751810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 --2- 172.21.15.122:0/1028185762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7518068d00 0x7f7518069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 --2- 172.21.15.122:0/1028185762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518107b50 0x7f7518107f30 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 -- 172.21.15.122:0/1028185762 >> 172.21.15.122:0/1028185762 conn(0x7f7518077b00 msgr2=0x7f7518077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 -- 172.21.15.122:0/1028185762 shutdown_connections 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 -- 172.21.15.122:0/1028185762 wait complete. 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 Processor -- start 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.159+0000 7f751dd3d640 1 -- start start 2024-04-02T13:45:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75180696c0 0x7f7518198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7518107b50 0x7f751819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111ea0 con 0x7f7518068d00 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7518111d20 con 0x7f75180696c0 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7518112020 con 0x7f7518107b50 2024-04-02T13:45:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32784/0 (socket says 172.21.15.122:32784) 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 -- 172.21.15.122:0/845853537 learned_addr learned my addr 172.21.15.122:0/845853537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f7516ffd640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75180696c0 0x7f7518198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f7517fff640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7518107b50 0x7f751819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 -- 172.21.15.122:0/845853537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7518107b50 msgr2=0x7f751819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7518107b50 0x7f751819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 -- 172.21.15.122:0/845853537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75180696c0 msgr2=0x7f7518198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75180696c0 0x7f7518198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 -- 172.21.15.122:0/845853537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751819d360 con 0x7f7518068d00 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f75177fe640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f750802c700 tx=0x7f75080324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f7514ff9640 1 -- 172.21.15.122:0/845853537 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508042020 con 0x7f7518068d00 2024-04-02T13:45:04.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751819d5f0 con 0x7f7518068d00 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f7514ff9640 1 -- 172.21.15.122:0/845853537 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080346c0 con 0x7f7518068d00 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f7514ff9640 1 -- 172.21.15.122:0/845853537 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7508039040 con 0x7f7518068d00 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.160+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75181a4ee0 con 0x7f7518068d00 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.162+0000 7f7514ff9640 1 -- 172.21.15.122:0/845853537 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7508045020 con 0x7f7518068d00 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7514ff9640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 0x7f75040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7520a02740 1 Processor -- start 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7516ffd640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 0x7f75040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7514ff9640 1 -- 172.21.15.122:0/845853537 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f75080f2b40 con 0x7f7518068d00 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7520a02740 1 -- start start 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.163+0000 7f7520a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7520a02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7520a02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x56069396f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7520a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5606938c37b0 con 0x560693963bb0 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7520a02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560693862570 con 0x5606939644d0 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7520a02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5606938bf760 con 0x56069396d0b0 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32792/0 (socket says 172.21.15.122:32792) 2024-04-02T13:45:04.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 -- 172.21.15.122:0/2984323584 learned_addr learned my addr 172.21.15.122:0/2984323584 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7516ffd640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 0x7f75040a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f75180fdc30 tx=0x7f750000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f7517fff640 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x56069396f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 -- 172.21.15.122:0/2984323584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 msgr2=0x56069396f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x56069396f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 -- 172.21.15.122:0/2984323584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 msgr2=0x56069396cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 -- 172.21.15.122:0/2984323584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606935d73f0 con 0x560693963bb0 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f75177fe640 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75080c6ef0 tx=0x7f7508032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb2cd00329690ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f750dffb640 1 -- 172.21.15.122:0/2984323584 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080beeb0 con 0x560693963bb0 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f750dffb640 1 -- 172.21.15.122:0/2984323584 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080be7b0 con 0x560693963bb0 2024-04-02T13:45:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.164+0000 7f750dffb640 1 -- 172.21.15.122:0/2984323584 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080bae20 con 0x560693963bb0 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- 172.21.15.122:0/2984323584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 msgr2=0x560693963f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f75080c6ef0 tx=0x7f7508032fc0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- 172.21.15.122:0/2984323584 shutdown_connections 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x56069396f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 --2- 172.21.15.122:0/2984323584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x560693963f90 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- 172.21.15.122:0/2984323584 >> 172.21.15.122:0/2984323584 conn(0x56069395e2e0 msgr2=0x56069395e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- 172.21.15.122:0/2984323584 shutdown_connections 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- 172.21.15.122:0/2984323584 wait complete. 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.165+0000 7f7520a02740 1 -- start start 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 Processor -- start 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560693963bb0 0x56069396cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56069396d0b0 0x56069396a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56069396fc60 con 0x5606939644d0 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56069387ee80 con 0x56069396d0b0 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56069396fde0 con 0x560693963bb0 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7520a02740 10 client.0 _ll_register_callbacks cb 0x5606938baaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32808/0 (socket says 172.21.15.122:32808) 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 -- 172.21.15.122:0/2004287405 learned_addr learned my addr 172.21.15.122:0/2004287405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7517fff640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56069396d0b0 0x56069396a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f75177fe640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560693963bb0 0x56069396cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 -- 172.21.15.122:0/2004287405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560693963bb0 msgr2=0x56069396cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560693963bb0 0x56069396cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 -- 172.21.15.122:0/2004287405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56069396d0b0 msgr2=0x56069396a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56069396d0b0 0x56069396a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 -- 172.21.15.122:0/2004287405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606936a17d0 con 0x5606939644d0 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7500011860 tx=0x7f7500004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f543b47a8eadb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.166+0000 7f74f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7500011e60 con 0x5606939644d0 2024-04-02T13:45:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f750001e940 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75080be7b0 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f750001eb80 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 10 client.18501 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56069397bcc0 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080be7b0 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f750001f430 con 0x5606939644d0 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 10 client.18501 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 10 client.18501 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:45:04.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 20 client.18501 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56069397d1d0 0x56069397f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 4 client.18501 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.167+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606938d0cf0 con 0x56069397d1d0 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.168+0000 7f75177fe640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56069397d1d0 0x56069397f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.168+0000 7f75177fe640 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56069397d1d0 0x56069397f650 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50d6051ee27db17a server_cookie=c719dd5b49c2ea05 in_seq=0 out_seq=0 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.168+0000 7f74f6ffd640 10 client.18501 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:04.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.168+0000 7f74f6ffd640 1 -- 172.21.15.122:0/2004287405 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606938d0cf0 con 0x56069397d1d0 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.168+0000 7f74f6ffd640 10 client.18501 handle_command_reply: tid=0 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 client.18501 shutdown 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 msgr2=0x560693969ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7500011860 tx=0x7f7500004730 comp rx=0 tx=0).stop 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 shutdown_connections 2024-04-02T13:45:04.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x56069397d1d0 0x56069397f650 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56069396d0b0 0x56069396a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5606939644d0 0x560693969ed0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f74f6ffd640 0 client.18501 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 --2- 172.21.15.122:0/2004287405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560693963bb0 0x56069396cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 >> 172.21.15.122:0/2004287405 conn(0x56069395e2e0 msgr2=0x560693960030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 shutdown_connections 2024-04-02T13:45:04.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 1 -- 172.21.15.122:0/2004287405 wait complete. 2024-04-02T13:45:04.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.169+0000 7f7520a02740 20 client.18501 trim_cache size 0 max 16384 2024-04-02T13:45:04.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 Processor -- start 2024-04-02T13:45:04.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 -- start start 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x5606939788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693968eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ad020 con 0x560693963bb0 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75181aeb60 con 0x5606939644d0 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7520a02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ad6f0 con 0x56069396d0b0 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32810/0 (socket says 172.21.15.122:32810) 2024-04-02T13:45:04.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f75177fe640 1 -- 172.21.15.122:0/2490342483 learned_addr learned my addr 172.21.15.122:0/2490342483 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.183+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x5606939788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f7517fff640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693968eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 -- 172.21.15.122:0/2490342483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 msgr2=0x560693968eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693968eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 -- 172.21.15.122:0/2490342483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 msgr2=0x5606939788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x5606939788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 -- 172.21.15.122:0/2490342483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560693964300 con 0x560693963bb0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f7516ffd640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x5606939788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f75177fe640 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75080be740 tx=0x7f7508034870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5decb06244aecaea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f750d7fa640 1 -- 172.21.15.122:0/2490342483 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080c2b60 con 0x560693963bb0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f750d7fa640 1 -- 172.21.15.122:0/2490342483 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080beeb0 con 0x560693963bb0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.184+0000 7f750d7fa640 1 -- 172.21.15.122:0/2490342483 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080bae20 con 0x560693963bb0 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- 172.21.15.122:0/2490342483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 msgr2=0x5606939793d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f75080be740 tx=0x7f7508034870 comp rx=0 tx=0).stop 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- 172.21.15.122:0/2490342483 shutdown_connections 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693968eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x5606939788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 --2- 172.21.15.122:0/2490342483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x5606939793d0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- 172.21.15.122:0/2490342483 >> 172.21.15.122:0/2490342483 conn(0x56069395e2e0 msgr2=0x5606939687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- 172.21.15.122:0/2490342483 shutdown_connections 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- 172.21.15.122:0/2490342483 wait complete. 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 -- start start 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.185+0000 7f7520a02740 1 Processor -- start 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693973df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ae490 con 0x560693963bb0 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f75181addc0 con 0x5606939644d0 2024-04-02T13:45:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75181ab020 con 0x56069396d0b0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 10 client.0 _ll_register_callbacks cb 0x7f750000bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32824/0 (socket says 172.21.15.122:32824) 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 -- 172.21.15.122:0/1463442613 learned_addr learned my addr 172.21.15.122:0/1463442613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7516ffd640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7517fff640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693973df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 -- 172.21.15.122:0/1463442613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 msgr2=0x560693973df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693973df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 -- 172.21.15.122:0/1463442613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 msgr2=0x56069396de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 -- 172.21.15.122:0/1463442613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56069396e6e0 con 0x560693963bb0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f75177fe640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75080f0450 tx=0x7f75080be580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240b31bb25afb953 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f74f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75080a2060 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f75080bae20 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75080beeb0 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f75080be7b0 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 10 client.18507 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.186+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560693975440 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75080c2b60 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f75080c6f90 con 0x560693963bb0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 10 client.18507 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 10 client.18507 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 20 client.18507 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5606939e37d0 0x5606939e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 4 client.18507 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f75080beeb0 con 0x5606939e37d0 2024-04-02T13:45:04.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7516ffd640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5606939e37d0 0x5606939e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f74f6ffd640 10 client.18507 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7516ffd640 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5606939e37d0 0x5606939e5bc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ee0322a267a4f0c0 server_cookie=21d51db2d5cd9319 in_seq=0 out_seq=0 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f74f6ffd640 1 -- 172.21.15.122:0/1463442613 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f75080beeb0 con 0x5606939e37d0 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f74f6ffd640 10 client.18507 handle_command_reply: tid=0 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.187+0000 7f7520a02740 1 client.18507 shutdown 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 msgr2=0x56069396d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f75080f0450 tx=0x7f75080be580 comp rx=0 tx=0).stop 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 shutdown_connections 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5606939e37d0 0x5606939e5bc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56069396d0b0 0x560693973df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5606939644d0 0x56069396de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 --2- 172.21.15.122:0/1463442613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560693963bb0 0x56069396d8c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 >> 172.21.15.122:0/1463442613 conn(0x56069395e2e0 msgr2=0x5606939586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f74f6ffd640 0 client.18507 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:04.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 shutdown_connections 2024-04-02T13:45:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 1 -- 172.21.15.122:0/1463442613 wait complete. 2024-04-02T13:45:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f7520a02740 20 client.18507 trim_cache size 0 max 16384 2024-04-02T13:45:04.214 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:04.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 msgr2=0x7f75040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 0x7f75040a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f75180fdc30 tx=0x7f750000b410 comp rx=0 tx=0).stop 2024-04-02T13:45:04.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 msgr2=0x7f75181983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.188+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f750802c700 tx=0x7f75080324f0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 shutdown_connections 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f75040a3d20 0x7f75040a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7518107b50 0x7f751819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f75180696c0 0x7f7518198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 --2- 172.21.15.122:0/845853537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7518068d00 0x7f75181983c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 >> 172.21.15.122:0/845853537 conn(0x7f7518077b00 msgr2=0x7f75180fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 shutdown_connections 2024-04-02T13:45:04.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.189+0000 7f751dd3d640 1 -- 172.21.15.122:0/845853537 wait complete. 2024-04-02T13:45:04.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:45:04.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 -- 172.21.15.122:0/1952481718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c101330 msgr2=0x7f926c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 --2- 172.21.15.122:0/1952481718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c101330 0x7f926c10d840 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f9264009cd0 tx=0x7f926402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 -- 172.21.15.122:0/1952481718 shutdown_connections 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 --2- 172.21.15.122:0/1952481718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c101330 0x7f926c10d840 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 --2- 172.21.15.122:0/1952481718 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c100970 0x7f926c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 --2- 172.21.15.122:0/1952481718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 0x7f926c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 -- 172.21.15.122:0/1952481718 >> 172.21.15.122:0/1952481718 conn(0x7f926c077b00 msgr2=0x7f926c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 -- 172.21.15.122:0/1952481718 shutdown_connections 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.327+0000 7f9271655640 1 -- 172.21.15.122:0/1952481718 wait complete. 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 Processor -- start 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 -- start start 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 0x7f926c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 0x7f926c19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f926c110d90 con 0x7f926c100970 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f926c110c10 con 0x7f926c101330 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9271655640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f926c110f10 con 0x7f926c106980 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f926affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 0x7f926c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f926affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32842/0 (socket says 172.21.15.122:32842) 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f926affd640 1 -- 172.21.15.122:0/639184477 learned_addr learned my addr 172.21.15.122:0/639184477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f926b7fe640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 0x7f926c19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 -- 172.21.15.122:0/639184477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 msgr2=0x7f926c19edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 0x7f926c19edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 -- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 msgr2=0x7f926c19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f9263fff640 1 -- 172.21.15.122:0/639184477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926c19f520 con 0x7f926c101330 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.328+0000 7f926affd640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9263fff640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 0x7f926c19aa70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f925400e9f0 tx=0x7f925400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9268ff9640 1 -- 172.21.15.122:0/639184477 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925400ccc0 con 0x7f926c101330 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9268ff9640 1 -- 172.21.15.122:0/639184477 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9254010910 con 0x7f926c101330 2024-04-02T13:45:04.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926c19f810 con 0x7f926c101330 2024-04-02T13:45:04.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9268ff9640 1 -- 172.21.15.122:0/639184477 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254018960 con 0x7f926c101330 2024-04-02T13:45:04.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.329+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f926c1a70a0 con 0x7f926c101330 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.331+0000 7f9268ff9640 1 -- 172.21.15.122:0/639184477 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9254010ac0 con 0x7f926c101330 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 Processor -- start 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 -- start start 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9268ff9640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 0x7f92580a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926b7fe640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 0x7f92580a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9268ff9640 1 -- 172.21.15.122:0/639184477 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f92540d0f20 con 0x7f926c101330 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 0x55fa5023bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 0x55fa50244760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa50244ca0 0x55fa50247090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa5019f7b0 con 0x55fa50244ca0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fa5013e570 con 0x55fa5023c0c0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9274240740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa5019b760 con 0x55fa5023b7a0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 0x55fa5023bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 0x55fa50244760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 0x55fa5023bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38926/0 (socket says 172.21.15.122:38926) 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926affd640 1 -- 172.21.15.122:0/3556835683 learned_addr learned my addr 172.21.15.122:0/3556835683 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926b7fe640 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa50244ca0 0x55fa50247090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 -- 172.21.15.122:0/3556835683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 msgr2=0x55fa5023bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f926b7fe640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 0x7f92580a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9264005cc0 tx=0x7f9264032340 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 0x55fa5023bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 -- 172.21.15.122:0/3556835683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa50244ca0 msgr2=0x55fa50247090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa50244ca0 0x55fa50247090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.332+0000 7f9263fff640 1 -- 172.21.15.122:0/3556835683 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa4feb33f0 con 0x55fa5023c0c0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f9263fff640 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 0x55fa50244760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92540ce230 tx=0x7f925409cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5871635a169f2755 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f92617fa640 1 -- 172.21.15.122:0/3556835683 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540a56f0 con 0x55fa5023c0c0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f92617fa640 1 -- 172.21.15.122:0/3556835683 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92540ce4b0 con 0x55fa5023c0c0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f92617fa640 1 -- 172.21.15.122:0/3556835683 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254018d60 con 0x55fa5023c0c0 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f9274240740 1 -- 172.21.15.122:0/3556835683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 msgr2=0x55fa50244760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.333+0000 7f9274240740 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 0x55fa50244760 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f92540ce230 tx=0x7f925409cb30 comp rx=0 tx=0).stop 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- 172.21.15.122:0/3556835683 shutdown_connections 2024-04-02T13:45:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa50244ca0 0x55fa50247090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023c0c0 0x55fa50244760 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- 172.21.15.122:0/3556835683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa5023b7a0 0x55fa5023bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- 172.21.15.122:0/3556835683 >> 172.21.15.122:0/3556835683 conn(0x55fa50235ed0 msgr2=0x55fa502362b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- 172.21.15.122:0/3556835683 shutdown_connections 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- 172.21.15.122:0/3556835683 wait complete. 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- start start 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 Processor -- start 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50241b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa50247a30 con 0x55fa5023c0c0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fa5015ae80 con 0x55fa5023b7a0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fa50247bb0 con 0x55fa50244ca0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9274240740 10 client.0 _ll_register_callbacks cb 0x55fa50196aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.334+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32868/0 (socket says 172.21.15.122:32868) 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 -- 172.21.15.122:0/508224059 learned_addr learned my addr 172.21.15.122:0/508224059 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f926b7fe640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f926affd640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50241b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 -- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 msgr2=0x55fa502425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502425b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 -- 172.21.15.122:0/508224059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 msgr2=0x55fa50241b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50241b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 -- 172.21.15.122:0/508224059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa4ff7d7d0 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9263fff640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f925409c830 tx=0x7f9254098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb78213489e027c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f92527fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540b5070 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f9254010640 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92540ce4b0 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f92540a56f0 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9274240740 10 client.18519 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fa50253b30 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.335+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92540ce4b0 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f925409bbb0 con 0x55fa5023c0c0 2024-04-02T13:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 10 client.18519 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 10 client.18519 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 20 client.18519 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa50257270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 4 client.18519 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa501accf0 con 0x55fa50254e80 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f926affd640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa50257270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f92527fc640 10 client.18519 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.336+0000 7f926affd640 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa50257270 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11bfac8e36b4dee7 server_cookie=86102b8486f3b6dc in_seq=0 out_seq=0 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f92527fc640 1 -- 172.21.15.122:0/508224059 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa501accf0 con 0x55fa50254e80 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f92527fc640 10 client.18519 handle_command_reply: tid=0 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 client.18519 shutdown 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 msgr2=0x55fa50242070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f925409c830 tx=0x7f9254098760 comp rx=0 tx=0).stop 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 shutdown_connections 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502425b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa50257270 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f92527fc640 0 client.18519 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50242070 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 --2- 172.21.15.122:0/508224059 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50241b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 >> 172.21.15.122:0/508224059 conn(0x55fa50235ed0 msgr2=0x55fa50237c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 shutdown_connections 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 1 -- 172.21.15.122:0/508224059 wait complete. 2024-04-02T13:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.337+0000 7f9274240740 20 client.18519 trim_cache size 0 max 16384 2024-04-02T13:45:04.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 Processor -- start 2024-04-02T13:45:04.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 -- start start 2024-04-02T13:45:04.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa5023ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa5023f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1b1380 con 0x55fa5023c0c0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1b2ec0 con 0x55fa5023b7a0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.351+0000 7f9274240740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1b1a50 con 0x55fa50244ca0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa5023ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38060/0 (socket says 172.21.15.122:38060) 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa5023ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32882/0 (socket says 172.21.15.122:32882) 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 -- 172.21.15.122:0/2201371297 learned_addr learned my addr 172.21.15.122:0/2201371297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926b7fe640 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa5023f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 -- 172.21.15.122:0/2201371297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 msgr2=0x55fa5023f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa5023f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 -- 172.21.15.122:0/2201371297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 msgr2=0x55fa5023ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa5023ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 -- 172.21.15.122:0/2201371297 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa5023bef0 con 0x55fa5023b7a0 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f926affd640 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f925c012440 tx=0x7f925c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbea095d9475a3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f9260ff9640 1 -- 172.21.15.122:0/2201371297 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c010b90 con 0x55fa5023b7a0 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f9260ff9640 1 -- 172.21.15.122:0/2201371297 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c025070 con 0x55fa5023b7a0 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.352+0000 7f9260ff9640 1 -- 172.21.15.122:0/2201371297 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c01f660 con 0x55fa5023b7a0 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- 172.21.15.122:0/2201371297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 msgr2=0x55fa50244260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f925c012440 tx=0x7f925c005f90 comp rx=0 tx=0).stop 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- 172.21.15.122:0/2201371297 shutdown_connections 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa5023f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa5023ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 --2- 172.21.15.122:0/2201371297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50244260 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- 172.21.15.122:0/2201371297 >> 172.21.15.122:0/2201371297 conn(0x55fa50235ed0 msgr2=0x55fa50250690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- 172.21.15.122:0/2201371297 shutdown_connections 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- 172.21.15.122:0/2201371297 wait complete. 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.353+0000 7f9274240740 1 -- start start 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 Processor -- start 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50255d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1b27f0 con 0x55fa5023c0c0 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1b2120 con 0x55fa5023b7a0 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f926c1af380 con 0x55fa50244ca0 2024-04-02T13:45:04.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9274240740 10 client.0 _ll_register_callbacks cb 0x7f9264045030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32886/0 (socket says 172.21.15.122:32886) 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 -- 172.21.15.122:0/200707058 learned_addr learned my addr 172.21.15.122:0/200707058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f926b7fe640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f926affd640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50255d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 -- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 msgr2=0x55fa502567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 -- 172.21.15.122:0/200707058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 msgr2=0x55fa50255d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50255d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 -- 172.21.15.122:0/200707058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa502ba0a0 con 0x55fa5023c0c0 2024-04-02T13:45:04.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f9263fff640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9254098f40 tx=0x7f92540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8f1b61fc64830e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f92527fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92540b5070 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.354+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f9254018b00 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925c025070 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f92540ce4b0 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 10 client.18525 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fa502bb3a0 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925c025070 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f92540870e0 con 0x55fa5023c0c0 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 10 client.18525 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 10 client.18525 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 20 client.18525 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa502bd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 4 client.18525 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa501a7040 con 0x55fa50254e80 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.355+0000 7f926affd640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa502bd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f926affd640 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa502bd970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ea8f52e124dadac6 server_cookie=4478c71398c9d2f8 in_seq=0 out_seq=0 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f92527fc640 10 client.18525 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f92527fc640 1 -- 172.21.15.122:0/200707058 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa501a7040 con 0x55fa50254e80 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f92527fc640 10 client.18525 handle_command_reply: tid=0 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 client.18525 shutdown 2024-04-02T13:45:04.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 msgr2=0x55fa50256260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f9254098f40 tx=0x7f92540057f0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 shutdown_connections 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fa50244ca0 0x55fa502567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55fa50254e80 0x55fa502bd970 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa5023c0c0 0x55fa50256260 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 --2- 172.21.15.122:0/200707058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fa5023b7a0 0x55fa50255d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 >> 172.21.15.122:0/200707058 conn(0x55fa50235ed0 msgr2=0x55fa50241ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f92527fc640 0 client.18525 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 shutdown_connections 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 1 -- 172.21.15.122:0/200707058 wait complete. 2024-04-02T13:45:04.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.356+0000 7f9274240740 20 client.18525 trim_cache size 0 max 16384 2024-04-02T13:45:04.375 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:04.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 msgr2=0x7f92580a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 0x7f92580a6260 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9264005cc0 tx=0x7f9264032340 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 msgr2=0x7f926c19aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 0x7f926c19aa70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f925400e9f0 tx=0x7f925400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 shutdown_connections 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92580a3da0 0x7f92580a6260 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f926c106980 0x7f926c19edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f926c101330 0x7f926c19aa70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 --2- 172.21.15.122:0/639184477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f926c100970 0x7f926c19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 >> 172.21.15.122:0/639184477 conn(0x7f926c077b00 msgr2=0x7f926c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 shutdown_connections 2024-04-02T13:45:04.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.357+0000 7f9271655640 1 -- 172.21.15.122:0/639184477 wait complete. 2024-04-02T13:45:04.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 -- 172.21.15.122:0/807865786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 msgr2=0x7f4678106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 --2- 172.21.15.122:0/807865786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f4678106d60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f4674009930 tx=0x7f467402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 -- 172.21.15.122:0/807865786 shutdown_connections 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 --2- 172.21.15.122:0/807865786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f467810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 --2- 172.21.15.122:0/807865786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 0x7f4678100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 --2- 172.21.15.122:0/807865786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f4678106d60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 -- 172.21.15.122:0/807865786 >> 172.21.15.122:0/807865786 conn(0x7f4678077b00 msgr2=0x7f4678077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.485+0000 7f467f3a8640 1 -- 172.21.15.122:0/807865786 shutdown_connections 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 -- 172.21.15.122:0/807865786 wait complete. 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 Processor -- start 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 -- start start 2024-04-02T13:45:04.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 0x7f46781991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f4678199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4678110d90 con 0x7f4678106980 2024-04-02T13:45:04.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4678110c10 con 0x7f4678101330 2024-04-02T13:45:04.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467f3a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4678110f10 con 0x7f4678100970 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467d945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467d945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32908/0 (socket says 172.21.15.122:32908) 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467d945640 1 -- 172.21.15.122:0/1595252123 learned_addr learned my addr 172.21.15.122:0/1595252123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467c943640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f4678199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.486+0000 7f467d144640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 0x7f46781991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 -- 172.21.15.122:0/1595252123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 msgr2=0x7f46781991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 0x7f46781991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 -- 172.21.15.122:0/1595252123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 msgr2=0x7f4678199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f4678199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 -- 172.21.15.122:0/1595252123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46781a47b0 con 0x7f4678106980 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467c943640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f4678199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467d945640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f467000eb10 tx=0x7f467000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f466e7fc640 1 -- 172.21.15.122:0/1595252123 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670004b20 con 0x7f4678106980 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f466e7fc640 1 -- 172.21.15.122:0/1595252123 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4670004cc0 con 0x7f4678106980 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f466e7fc640 1 -- 172.21.15.122:0/1595252123 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670010760 con 0x7f4678106980 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46781a4980 con 0x7f4678106980 2024-04-02T13:45:04.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.487+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46781a4e80 con 0x7f4678106980 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.489+0000 7f466e7fc640 1 -- 172.21.15.122:0/1595252123 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4670005020 con 0x7f4678106980 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 Processor -- start 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 -- start start 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f466e7fc640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 0x7f46640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 0x55601f4fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55601f4537b0 con 0x55601f4f00c0 2024-04-02T13:45:04.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55601f3f2570 con 0x55601f4f8ca0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.490+0000 7f4681f8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55601f44f760 con 0x55601f4ef7a0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467d144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38968/0 (socket says 172.21.15.122:38968) 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467d144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32910/0 (socket says 172.21.15.122:32910) 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 -- 172.21.15.122:0/2451723442 learned_addr learned my addr 172.21.15.122:0/2451723442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 0x7f46640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f466e7fc640 1 -- 172.21.15.122:0/1595252123 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f46700d1190 con 0x7f4678106980 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467d945640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 0x55601f4fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 -- 172.21.15.122:0/2451723442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 msgr2=0x55601f4fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 0x55601f4fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 -- 172.21.15.122:0/2451723442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 msgr2=0x55601f4f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 -- 172.21.15.122:0/2451723442 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601f1673f0 con 0x55601f4ef7a0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467d945640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 0x55601f4fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467d144640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 0x7f46640a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f466800fc20 tx=0x7f4668018000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f467c943640 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4668011ea0 tx=0x7f4668011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8252a8c078419a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.491+0000 7f46637fe640 1 -- 172.21.15.122:0/2451723442 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668022070 con 0x55601f4ef7a0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f46637fe640 1 -- 172.21.15.122:0/2451723442 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466800ed10 con 0x55601f4ef7a0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f46637fe640 1 -- 172.21.15.122:0/2451723442 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466801e660 con 0x55601f4ef7a0 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 -- 172.21.15.122:0/2451723442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 msgr2=0x55601f4efb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4668011ea0 tx=0x7f4668011f90 comp rx=0 tx=0).stop 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 -- 172.21.15.122:0/2451723442 shutdown_connections 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f8ca0 0x55601f4fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 --2- 172.21.15.122:0/2451723442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4ef7a0 0x55601f4efb80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 -- 172.21.15.122:0/2451723442 >> 172.21.15.122:0/2451723442 conn(0x55601f4e9ed0 msgr2=0x55601f4ea2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 -- 172.21.15.122:0/2451723442 shutdown_connections 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.492+0000 7f4681f8d740 1 -- 172.21.15.122:0/2451723442 wait complete. 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 -- start start 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 Processor -- start 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f4f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f00c0 0x55601f4f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55601f4fb9d0 con 0x55601f4f8ca0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55601f40ee80 con 0x55601f4ef7a0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55601f4fbb50 con 0x55601f4f00c0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f4681f8d740 10 client.0 _ll_register_callbacks cb 0x55601f44aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f4f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32912/0 (socket says 172.21.15.122:32912) 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f4f86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38106/0 (socket says 172.21.15.122:38106) 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d945640 1 -- 172.21.15.122:0/1362048747 learned_addr learned my addr 172.21.15.122:0/1362048747 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467c943640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f00c0 0x55601f4f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.493+0000 7f467d945640 1 -- 172.21.15.122:0/1362048747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f00c0 msgr2=0x55601f4f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f467d945640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f00c0 0x55601f4f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f467d945640 1 -- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 msgr2=0x55601f4f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f467d945640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f4f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f467d945640 1 -- 172.21.15.122:0/1362048747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601f2317d0 con 0x55601f4f8ca0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f467d945640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f46700990f0 tx=0x7f467009c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a31b56da26d4b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467007fac0 con 0x55601f4f8ca0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f46700bd0c0 con 0x55601f4f8ca0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f466800ed10 con 0x55601f4f8ca0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f46700ce8d0 con 0x55601f4f8ca0 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 10 client.18537 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55601f507a20 con 0x55601f4f8ca0 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f466800ed10 con 0x55601f4f8ca0 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f46700a5470 con 0x55601f4f8ca0 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 10 client.18537 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 10 client.18537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 20 client.18537 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f508d70 0x55601f50b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 4 client.18537 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.494+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55601f460cf0 con 0x55601f508d70 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.495+0000 7f467d144640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f508d70 0x55601f50b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.495+0000 7f467d144640 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f508d70 0x55601f50b160 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2566d46c6eab63d server_cookie=2e6ef03fb102a3c8 in_seq=0 out_seq=0 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.495+0000 7f4660ff9640 10 client.18537 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4660ff9640 1 -- 172.21.15.122:0/1362048747 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55601f460cf0 con 0x55601f508d70 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4660ff9640 10 client.18537 handle_command_reply: tid=0 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 client.18537 shutdown 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 msgr2=0x55601f4f61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f46700990f0 tx=0x7f467009c890 comp rx=0 tx=0).stop 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 shutdown_connections 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f8ca0 0x55601f4f61c0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f00c0 0x55601f4f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f508d70 0x55601f50b160 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 --2- 172.21.15.122:0/1362048747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f4f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4660ff9640 0 client.18537 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 >> 172.21.15.122:0/1362048747 conn(0x55601f4e9ed0 msgr2=0x55601f4ebc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 shutdown_connections 2024-04-02T13:45:04.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 1 -- 172.21.15.122:0/1362048747 wait complete. 2024-04-02T13:45:04.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.496+0000 7f4681f8d740 20 client.18537 trim_cache size 0 max 16384 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 Processor -- start 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 -- start start 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f00c0 0x55601f4f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46781af190 con 0x55601f4ef7a0 2024-04-02T13:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b0cd0 con 0x55601f4f00c0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f4681f8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46781af860 con 0x55601f4f8ca0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32922/0 (socket says 172.21.15.122:32922) 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 -- 172.21.15.122:0/3841084533 learned_addr learned my addr 172.21.15.122:0/3841084533 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d945640 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467c943640 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f00c0 0x55601f4f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 -- 172.21.15.122:0/3841084533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 msgr2=0x55601f4f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 -- 172.21.15.122:0/3841084533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f00c0 msgr2=0x55601f4f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f00c0 0x55601f4f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.511+0000 7f467d144640 1 -- 172.21.15.122:0/3841084533 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601f50b4d0 con 0x55601f4ef7a0 2024-04-02T13:45:04.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.512+0000 7f467d144640 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f4674005ab0 tx=0x7f4674005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988a555d61aded8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.512+0000 7f4662ffd640 1 -- 172.21.15.122:0/3841084533 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467404c070 con 0x55601f4ef7a0 2024-04-02T13:45:04.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.512+0000 7f4662ffd640 1 -- 172.21.15.122:0/3841084533 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467402cd60 con 0x55601f4ef7a0 2024-04-02T13:45:04.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.512+0000 7f4662ffd640 1 -- 172.21.15.122:0/3841084533 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740046c0 con 0x55601f4ef7a0 2024-04-02T13:45:04.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- 172.21.15.122:0/3841084533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 msgr2=0x55601f504890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f4674005ab0 tx=0x7f4674005fd0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- 172.21.15.122:0/3841084533 shutdown_connections 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4f00c0 0x55601f4f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- 172.21.15.122:0/3841084533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4ef7a0 0x55601f504890 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- 172.21.15.122:0/3841084533 >> 172.21.15.122:0/3841084533 conn(0x55601f4e9ed0 msgr2=0x55601f4f4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- 172.21.15.122:0/3841084533 shutdown_connections 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- 172.21.15.122:0/3841084533 wait complete. 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 -- start start 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 Processor -- start 2024-04-02T13:45:04.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f501080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.513+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46781b0600 con 0x55601f4f00c0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46781aff30 con 0x55601f4ef7a0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46781ad190 con 0x55601f4f8ca0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 10 client.0 _ll_register_callbacks cb 0x7f467009ca60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467c943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f501080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467c943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32936/0 (socket says 172.21.15.122:32936) 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467c943640 1 -- 172.21.15.122:0/296026922 learned_addr learned my addr 172.21.15.122:0/296026922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d945640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 -- 172.21.15.122:0/296026922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 msgr2=0x55601f4f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 -- 172.21.15.122:0/296026922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 msgr2=0x55601f4f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 -- 172.21.15.122:0/296026922 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601f4f3030 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f467d144640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f501080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f467402c170 tx=0x7f467402bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39403fa66d5bf6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674057070 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f46740046c0 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467402cd60 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4674060050 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 10 client.27608 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.514+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55601f56f250 con 0x55601f4ef7a0 2024-04-02T13:45:04.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467402cd60 con 0x55601f4ef7a0 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f467405a020 con 0x55601f4ef7a0 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 10 client.27608 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 10 client.27608 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 20 client.27608 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f56f900 0x55601f571d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 4 client.27608 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55601f45b040 con 0x55601f56f900 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f467c943640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f56f900 0x55601f571d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4660ff9640 10 client.27608 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f467c943640 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f56f900 0x55601f571d20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=efa11593329e3d6d server_cookie=992994e88d3a6d5b in_seq=0 out_seq=0 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4660ff9640 1 -- 172.21.15.122:0/296026922 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55601f45b040 con 0x55601f56f900 2024-04-02T13:45:04.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4660ff9640 10 client.27608 handle_command_reply: tid=0 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.515+0000 7f4681f8d740 1 client.27608 shutdown 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 msgr2=0x55601f501080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f501080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f467402c170 tx=0x7f467402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 shutdown_connections 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55601f4f8ca0 0x55601f4f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55601f4f00c0 0x55601f4f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55601f56f900 0x55601f571d20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 --2- 172.21.15.122:0/296026922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55601f4ef7a0 0x55601f501080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4660ff9640 0 client.27608 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 >> 172.21.15.122:0/296026922 conn(0x55601f4e9ed0 msgr2=0x55601f4e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 shutdown_connections 2024-04-02T13:45:04.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 1 -- 172.21.15.122:0/296026922 wait complete. 2024-04-02T13:45:04.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f4681f8d740 20 client.27608 trim_cache size 0 max 16384 2024-04-02T13:45:04.528 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 msgr2=0x7f46640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 0x7f46640a61e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f466800fc20 tx=0x7f4668018000 comp rx=0 tx=0).stop 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 msgr2=0x7f46781a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f467000eb10 tx=0x7f467000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 shutdown_connections 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f46640a3d20 0x7f46640a61e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4678106980 0x7f46781a4270 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4678101330 0x7f4678199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 --2- 172.21.15.122:0/1595252123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4678100970 0x7f46781991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.516+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 >> 172.21.15.122:0/1595252123 conn(0x7f4678077b00 msgr2=0x7f46781031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.517+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 shutdown_connections 2024-04-02T13:45:04.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.517+0000 7f467f3a8640 1 -- 172.21.15.122:0/1595252123 wait complete. 2024-04-02T13:45:04.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:45:04.555 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3536823282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- 172.21.15.122:0/3402514513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0106980 msgr2=0x7ff7a0106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 --2- 172.21.15.122:0/3402514513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0106980 0x7ff7a0106d60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff790009930 tx=0x7ff79002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- 172.21.15.122:0/3402514513 shutdown_connections 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 --2- 172.21.15.122:0/3402514513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0101330 0x7ff7a010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 --2- 172.21.15.122:0/3402514513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 --2- 172.21.15.122:0/3402514513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0106980 0x7ff7a0106d60 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- 172.21.15.122:0/3402514513 >> 172.21.15.122:0/3402514513 conn(0x7ff7a0077b00 msgr2=0x7ff7a0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- 172.21.15.122:0/3402514513 shutdown_connections 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- 172.21.15.122:0/3402514513 wait complete. 2024-04-02T13:45:04.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 Processor -- start 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.654+0000 7ff7a638d640 1 -- start start 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a01972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106980 0x7ff7a019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0110d90 con 0x7ff7a0101330 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0110c10 con 0x7ff7a0100970 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a638d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a0110f10 con 0x7ff7a0106980 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32960/0 (socket says 172.21.15.122:32960) 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 -- 172.21.15.122:0/2749397261 learned_addr learned my addr 172.21.15.122:0/2749397261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79ffff640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a01972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff7a492a640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106980 0x7ff7a019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 -- 172.21.15.122:0/2749397261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106980 msgr2=0x7ff7a019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106980 0x7ff7a019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 -- 172.21.15.122:0/2749397261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 msgr2=0x7ff7a01972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a01972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 -- 172.21.15.122:0/2749397261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a019c2a0 con 0x7ff7a0101330 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79ffff640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.655+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff79400eb00 tx=0x7ff794009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.656+0000 7ff79d7fa640 1 -- 172.21.15.122:0/2749397261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79401c070 con 0x7ff7a0101330 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.656+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a019c590 con 0x7ff7a0101330 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.656+0000 7ff79d7fa640 1 -- 172.21.15.122:0/2749397261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79400bc30 con 0x7ff7a0101330 2024-04-02T13:45:04.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.656+0000 7ff79d7fa640 1 -- 172.21.15.122:0/2749397261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940166a0 con 0x7ff7a0101330 2024-04-02T13:45:04.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.656+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a0068c60 con 0x7ff7a0101330 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.658+0000 7ff79d7fa640 1 -- 172.21.15.122:0/2749397261 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff79401f070 con 0x7ff7a0101330 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.658+0000 7ff7a8f74740 1 Processor -- start 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79d7fa640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 0x7ff7880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 -- start start 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a492a640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 0x7ff7880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79d7fa640 1 -- 172.21.15.122:0/2749397261 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7ff7940d2280 con 0x7ff7a0101330 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f87e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55592f7d27b0 con 0x55592f86e7a0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55592f771570 con 0x55592f87c0b0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55592f7ce760 con 0x55592f86f0c0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32964/0 (socket says 172.21.15.122:32964) 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 -- 172.21.15.122:0/4211210391 learned_addr learned my addr 172.21.15.122:0/4211210391 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a492a640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f87e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 -- 172.21.15.122:0/4211210391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 msgr2=0x55592f87bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 -- 172.21.15.122:0/4211210391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 msgr2=0x55592f87e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f87e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 -- 172.21.15.122:0/4211210391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592f4e63f0 con 0x55592f86e7a0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff7a492a640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 0x7ff7880a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff798009fd0 tx=0x7ff798014b70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.659+0000 7ff79ffff640 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff790005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29e5422070cd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.660+0000 7ff7867fc640 1 -- 172.21.15.122:0/4211210391 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79002ca80 con 0x55592f86e7a0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.660+0000 7ff7867fc640 1 -- 172.21.15.122:0/4211210391 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79002cc20 con 0x55592f86e7a0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.660+0000 7ff7867fc640 1 -- 172.21.15.122:0/4211210391 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900046c0 con 0x55592f86e7a0 2024-04-02T13:45:04.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- 172.21.15.122:0/4211210391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 msgr2=0x55592f86eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff790005df0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- 172.21.15.122:0/4211210391 shutdown_connections 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f87e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/4211210391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f86eb80 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- 172.21.15.122:0/4211210391 >> 172.21.15.122:0/4211210391 conn(0x55592f868ed0 msgr2=0x55592f8692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- 172.21.15.122:0/4211210391 shutdown_connections 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- 172.21.15.122:0/4211210391 wait complete. 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 -- start start 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 Processor -- start 2024-04-02T13:45:04.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f8724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.661+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55592f87ee40 con 0x55592f87c0b0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55592f78de80 con 0x55592f86e7a0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55592f87efc0 con 0x55592f86f0c0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a8f74740 10 client.0 _ll_register_callbacks cb 0x55592f7c9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a492a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a492a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32966/0 (socket says 172.21.15.122:32966) 2024-04-02T13:45:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38158/0 (socket says 172.21.15.122:38158) 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a492a640 1 -- 172.21.15.122:0/119852983 learned_addr learned my addr 172.21.15.122:0/119852983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f8724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 -- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 msgr2=0x55592f8724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f8724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 -- 172.21.15.122:0/119852983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 msgr2=0x55592f872ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 -- 172.21.15.122:0/119852983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55592f5b07d0 con 0x55592f86e7a0 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a492a640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff79ffff640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff790036040 tx=0x7ff790005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8da7b953c6b5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff77f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff790004040 con 0x55592f86e7a0 2024-04-02T13:45:04.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7ff7900041e0 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.662+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79002cc20 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff790056050 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 10 client.27620 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55592f88ae10 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79002cc20 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff790046630 con 0x55592f86e7a0 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 10 client.27620 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 10 client.27620 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 20 client.27620 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f88e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 4 client.27620 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:04.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55592f7dfcf0 con 0x55592f88c160 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.663+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f88e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.664+0000 7ff77f7fe640 10 client.27620 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.664+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f88e550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d832ed168e0e7a42 server_cookie=ee15052613b9b555 in_seq=0 out_seq=0 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.664+0000 7ff77f7fe640 1 -- 172.21.15.122:0/119852983 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55592f7dfcf0 con 0x55592f88c160 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.664+0000 7ff77f7fe640 10 client.27620 handle_command_reply: tid=0 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 client.27620 shutdown 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 msgr2=0x55592f875110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff790036040 tx=0x7ff790005bd0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 shutdown_connections 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f88e550 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f87c0b0 0x55592f872ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff77f7fe640 0 client.27620 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f8724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/119852983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f86e7a0 0x55592f875110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 >> 172.21.15.122:0/119852983 conn(0x55592f868ed0 msgr2=0x55592f86ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 shutdown_connections 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 1 -- 172.21.15.122:0/119852983 wait complete. 2024-04-02T13:45:04.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.665+0000 7ff7a8f74740 20 client.27620 trim_cache size 0 max 16384 2024-04-02T13:45:04.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.679+0000 7ff7a8f74740 1 Processor -- start 2024-04-02T13:45:04.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.679+0000 7ff7a8f74740 1 -- start start 2024-04-02T13:45:04.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f8714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01ace70 con 0x55592f86e7a0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01ae9b0 con 0x55592f87c0b0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01ad540 con 0x55592f86f0c0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32978/0 (socket says 172.21.15.122:32978) 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 -- 172.21.15.122:0/27663145 learned_addr learned my addr 172.21.15.122:0/27663145 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff7a492a640 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f8714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 -- 172.21.15.122:0/27663145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 msgr2=0x55592f87e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 -- 172.21.15.122:0/27663145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 msgr2=0x55592f8714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f8714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 -- 172.21.15.122:0/27663145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592f86eef0 con 0x55592f86e7a0 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.680+0000 7ff79ffff640 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff79002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f063085ca9eb88f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.681+0000 7ff785ffb640 1 -- 172.21.15.122:0/27663145 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79002c650 con 0x55592f86e7a0 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.681+0000 7ff785ffb640 1 -- 172.21.15.122:0/27663145 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff790046cc0 con 0x55592f86e7a0 2024-04-02T13:45:04.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.681+0000 7ff785ffb640 1 -- 172.21.15.122:0/27663145 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7900035f0 con 0x55592f86e7a0 2024-04-02T13:45:04.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- 172.21.15.122:0/27663145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 msgr2=0x55592f887d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff790000c00 tx=0x7ff79002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:04.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- 172.21.15.122:0/27663145 shutdown_connections 2024-04-02T13:45:04.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f8714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86f0c0 0x55592f87e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/27663145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86e7a0 0x55592f887d00 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- 172.21.15.122:0/27663145 >> 172.21.15.122:0/27663145 conn(0x55592f868ed0 msgr2=0x55592f8692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- 172.21.15.122:0/27663145 shutdown_connections 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- 172.21.15.122:0/27663145 wait complete. 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- start start 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 Processor -- start 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86e7a0 0x55592f8846e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f879b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01ae2e0 con 0x55592f86f0c0 2024-04-02T13:45:04.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01adc10 con 0x55592f87c0b0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.682+0000 7ff7a8f74740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a01aae70 con 0x55592f86e7a0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff7a8f74740 10 client.0 _ll_register_callbacks cb 0x7ff794099960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:32990/0 (socket says 172.21.15.122:32990) 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 -- 172.21.15.122:0/1696952367 learned_addr learned my addr 172.21.15.122:0/1696952367 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79ffff640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86e7a0 0x55592f8846e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff7a492a640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f879b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 -- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86e7a0 msgr2=0x55592f8846e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86e7a0 0x55592f8846e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 -- 172.21.15.122:0/1696952367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 msgr2=0x55592f879b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f879b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 -- 172.21.15.122:0/1696952367 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55592f87b930 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff79f7fe640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b750 tx=0x7ff7940be040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cbc5966b0374e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff77f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940a6a50 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7ff7940cf810 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff790046cc0 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff79409a450 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff7a8f74740 10 client.18561 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.683+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55592f8f1a20 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff790046cc0 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff794083020 con 0x55592f86f0c0 2024-04-02T13:45:04.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 10 client.18561 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 10 client.18561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 20 client.18561 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f8f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 4 client.18561 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55592f7da040 con 0x55592f88c160 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff79ffff640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f8f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff77f7fe640 10 client.18561 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.684+0000 7ff79ffff640 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f8f41e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6870cf87eb18dc server_cookie=6548834be2274675 in_seq=0 out_seq=0 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff77f7fe640 1 -- 172.21.15.122:0/1696952367 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55592f7da040 con 0x55592f88c160 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff77f7fe640 10 client.18561 handle_command_reply: tid=0 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 client.18561 shutdown 2024-04-02T13:45:04.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 msgr2=0x55592f87b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff79400b750 tx=0x7ff7940be040 comp rx=0 tx=0).stop 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 shutdown_connections 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55592f88c160 0x55592f8f41e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55592f87c0b0 0x55592f879b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55592f86f0c0 0x55592f87b060 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff77f7fe640 0 client.18561 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 --2- 172.21.15.122:0/1696952367 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55592f86e7a0 0x55592f8846e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 >> 172.21.15.122:0/1696952367 conn(0x55592f868ed0 msgr2=0x55592f87d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 shutdown_connections 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 1 -- 172.21.15.122:0/1696952367 wait complete. 2024-04-02T13:45:04.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.685+0000 7ff7a8f74740 20 client.18561 trim_cache size 0 max 16384 2024-04-02T13:45:04.696 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:04.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 msgr2=0x7ff7880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 0x7ff7880a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff798009fd0 tx=0x7ff798014b70 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 msgr2=0x7ff7a01977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff79400eb00 tx=0x7ff794009f20 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 shutdown_connections 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7880a3d20 0x7ff7880a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7a0106980 0x7ff7a019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7a0101330 0x7ff7a01977f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 --2- 172.21.15.122:0/2749397261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7a0100970 0x7ff7a01972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 >> 172.21.15.122:0/2749397261 conn(0x7ff7a0077b00 msgr2=0x7ff7a0103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 shutdown_connections 2024-04-02T13:45:04.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.686+0000 7ff7a638d640 1 -- 172.21.15.122:0/2749397261 wait complete. 2024-04-02T13:45:04.699 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:45:04.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3536823282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:04.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3536823282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:04.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- 172.21.15.122:0/3196748782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0101360 msgr2=0x7fc0b010d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 --2- 172.21.15.122:0/3196748782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0101360 0x7fc0b010d820 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009ca0 tx=0x7fc0a802c180 comp rx=0 tx=0).stop 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- 172.21.15.122:0/3196748782 shutdown_connections 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 --2- 172.21.15.122:0/3196748782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b0101360 0x7fc0b010d820 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 --2- 172.21.15.122:0/3196748782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 0x7fc0b0100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 --2- 172.21.15.122:0/3196748782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b01058a0 0x7fc0b0105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- 172.21.15.122:0/3196748782 >> 172.21.15.122:0/3196748782 conn(0x7fc0b0077a90 msgr2=0x7fc0b0077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- 172.21.15.122:0/3196748782 shutdown_connections 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- 172.21.15.122:0/3196748782 wait complete. 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 Processor -- start 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.852+0000 7fc0b5886640 1 -- start start 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 0x7fc0b01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0101360 0x7fc0b0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0110d90 con 0x7fc0b01058a0 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0110c10 con 0x7fc0b01009c0 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0110f10 con 0x7fc0b0101360 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 0x7fc0b01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33020/0 (socket says 172.21.15.122:33020) 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 -- 172.21.15.122:0/140896838 learned_addr learned my addr 172.21.15.122:0/140896838 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0101360 0x7fc0b0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 -- 172.21.15.122:0/140896838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0101360 msgr2=0x7fc0b0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0101360 0x7fc0b0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 -- 172.21.15.122:0/140896838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 msgr2=0x7fc0b01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 0x7fc0b01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 -- 172.21.15.122:0/140896838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b019d2c0 con 0x7fc0b01058a0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0af7fe640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc0a802c790 tx=0x7fc0a8035ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b4884640 1 -- 172.21.15.122:0/140896838 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8039040 con 0x7fc0b01058a0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b019d550 con 0x7fc0b01058a0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b4884640 1 -- 172.21.15.122:0/140896838 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a8044070 con 0x7fc0b01058a0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b4884640 1 -- 172.21.15.122:0/140896838 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80329f0 con 0x7fc0b01058a0 2024-04-02T13:45:04.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.853+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01a4e40 con 0x7fc0b01058a0 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b4884640 1 -- 172.21.15.122:0/140896838 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc0a8032b90 con 0x7fc0b01058a0 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 Processor -- start 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 -- start start 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d2db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d36ca0 0x55bd07d39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bd07c917b0 con 0x55bd07d2e0c0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.856+0000 7fc0b846f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bd07c30570 con 0x55bd07d2d7a0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0b846f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bd07c8d760 con 0x55bd07d36ca0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0b4884640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 0x7fc09c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33030/0 (socket says 172.21.15.122:33030) 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 -- 172.21.15.122:0/677007342 learned_addr learned my addr 172.21.15.122:0/677007342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0af7fe640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 0x7fc09c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d36ca0 0x55bd07d39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0b4884640 1 -- 172.21.15.122:0/140896838 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fc0a80f30a0 con 0x7fc0b01058a0 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 -- 172.21.15.122:0/677007342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d36ca0 msgr2=0x55bd07d39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d36ca0 0x55bd07d39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 -- 172.21.15.122:0/677007342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 msgr2=0x55bd07d2db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d2db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 -- 172.21.15.122:0/677007342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd079a53f0 con 0x55bd07d2e0c0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d2db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0af7fe640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 0x7fc09c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80bb170 tx=0x7fc0a809c040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000bc50 tx=0x7fc0a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b9daae4ca2f881 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a5ffb640 1 -- 172.21.15.122:0/677007342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a001c480 con 0x55bd07d2e0c0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a5ffb640 1 -- 172.21.15.122:0/677007342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a001cb00 con 0x55bd07d2e0c0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.857+0000 7fc0a5ffb640 1 -- 172.21.15.122:0/677007342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0011c70 con 0x55bd07d2e0c0 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 -- 172.21.15.122:0/677007342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 msgr2=0x55bd07d36760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000bc50 tx=0x7fc0a00107a0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 -- 172.21.15.122:0/677007342 shutdown_connections 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d36ca0 0x55bd07d39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36760 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 --2- 172.21.15.122:0/677007342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d2db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 -- 172.21.15.122:0/677007342 >> 172.21.15.122:0/677007342 conn(0x55bd07d27ed0 msgr2=0x55bd07d282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 -- 172.21.15.122:0/677007342 shutdown_connections 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.858+0000 7fc0b846f740 1 -- 172.21.15.122:0/677007342 wait complete. 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 -- start start 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 Processor -- start 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 0x55bd07d33a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bd07d39a90 con 0x55bd07d36ca0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bd07c4ce80 con 0x55bd07d2d7a0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bd07d39c10 con 0x55bd07d2e0c0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0b846f740 10 client.0 _ll_register_callbacks cb 0x55bd07c88aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 0x55bd07d33a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0a7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38202/0 (socket says 172.21.15.122:38202) 2024-04-02T13:45:04.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3202060902 learned_addr learned my addr 172.21.15.122:0/3202060902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.859+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3202060902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 msgr2=0x55bd07d33a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 0x55bd07d33a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3202060902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 msgr2=0x55bd07d34050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d34050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3202060902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd07a6f7d0 con 0x55bd07d2d7a0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 0x55bd07d33a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc09800b510 tx=0x7fc09800bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59647e194394e22a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc08effd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098017be0 con 0x55bd07d2d7a0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fc098017d80 con 0x55bd07d2d7a0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a001cb00 con 0x55bd07d2d7a0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0980299f0 con 0x55bd07d2d7a0 2024-04-02T13:45:04.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0b846f740 10 client.27638 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.860+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd07d45b20 con 0x55bd07d2d7a0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a001cb00 con 0x55bd07d2d7a0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc098004c30 con 0x55bd07d2d7a0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 10 client.27638 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 10 client.27638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 20 client.27638 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07d46fe0 0x55bd07d493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 4 client.27638 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd07c9ecf0 con 0x55bd07d46fe0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07d46fe0 0x55bd07d493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.861+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07d46fe0 0x55bd07d493d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c04967272592c server_cookie=3861e65c652ea95f in_seq=0 out_seq=0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc08effd640 10 client.27638 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc08effd640 1 -- 172.21.15.122:0/3202060902 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd07c9ecf0 con 0x55bd07d46fe0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc08effd640 10 client.27638 handle_command_reply: tid=0 2024-04-02T13:45:04.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 client.27638 shutdown 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 msgr2=0x55bd07d366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc09800b510 tx=0x7fc09800bbb0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 shutdown_connections 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07d46fe0 0x55bd07d493d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d34050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2e0c0 0x55bd07d33a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc08effd640 0 client.27638 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3202060902 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2d7a0 0x55bd07d366b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 >> 172.21.15.122:0/3202060902 conn(0x55bd07d27ed0 msgr2=0x55bd07d29c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 shutdown_connections 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.862+0000 7fc0b846f740 1 -- 172.21.15.122:0/3202060902 wait complete. 2024-04-02T13:45:04.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.863+0000 7fc0b846f740 20 client.27638 trim_cache size 0 max 16384 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 Processor -- start 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 -- start start 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01acfc0 con 0x55bd07d36ca0 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01aeb00 con 0x55bd07d2e0c0 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.877+0000 7fc0b846f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01ad690 con 0x55bd07d2d7a0 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38212/0 (socket says 172.21.15.122:38212) 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 -- 172.21.15.122:0/1701988508 learned_addr learned my addr 172.21.15.122:0/1701988508 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 -- 172.21.15.122:0/1701988508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 msgr2=0x55bd07d429b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d429b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 -- 172.21.15.122:0/1701988508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 msgr2=0x55bd07d366b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d366b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 -- 172.21.15.122:0/1701988508 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd07d343c0 con 0x55bd07d2e0c0 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000bb70 tx=0x7fc0a000bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff04f12abeda8f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0a57fa640 1 -- 172.21.15.122:0/1701988508 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a002f030 con 0x55bd07d2e0c0 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0a57fa640 1 -- 172.21.15.122:0/1701988508 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a0005040 con 0x55bd07d2e0c0 2024-04-02T13:45:04.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.878+0000 7fc0a57fa640 1 -- 172.21.15.122:0/1701988508 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00118f0 con 0x55bd07d2e0c0 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 -- 172.21.15.122:0/1701988508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 msgr2=0x55bd07d36170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000bb70 tx=0x7fc0a000bed0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 -- 172.21.15.122:0/1701988508 shutdown_connections 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d36170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 --2- 172.21.15.122:0/1701988508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d429b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 -- 172.21.15.122:0/1701988508 >> 172.21.15.122:0/1701988508 conn(0x55bd07d27ed0 msgr2=0x55bd07d32350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 -- 172.21.15.122:0/1701988508 shutdown_connections 2024-04-02T13:45:04.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.879+0000 7fc0b846f740 1 -- 172.21.15.122:0/1701988508 wait complete. 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 -- start start 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 Processor -- start 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d3f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01ae430 con 0x55bd07d36ca0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01add60 con 0x55bd07d2e0c0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01aafc0 con 0x55bd07d2d7a0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0b846f740 10 client.0 _ll_register_callbacks cb 0x7fc098019770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33064/0 (socket says 172.21.15.122:33064) 2024-04-02T13:45:04.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0aeffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38228/0 (socket says 172.21.15.122:38228) 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3111457280 learned_addr learned my addr 172.21.15.122:0/3111457280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d3f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3111457280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 msgr2=0x55bd07d3f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d3f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 msgr2=0x55bd07d30d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0a7fff640 1 -- 172.21.15.122:0/3111457280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd07d49740 con 0x55bd07d36ca0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.880+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0a7fff640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc098019880 tx=0x7fc09800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618a989b9e295bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098029d90 con 0x55bd07d36ca0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fc098004020 con 0x55bd07d36ca0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a0005040 con 0x55bd07d36ca0 2024-04-02T13:45:04.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc098004220 con 0x55bd07d36ca0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 10 client.18579 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd07d382b0 con 0x55bd07d36ca0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0a0005040 con 0x55bd07d36ca0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fc0980293e0 con 0x55bd07d36ca0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 10 client.18579 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 10 client.18579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 20 client.18579 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07dad8b0 0x55bd07dafcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 4 client.18579 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.881+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd07c99040 con 0x55bd07dad8b0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07dad8b0 0x55bd07dafcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc08effd640 10 client.18579 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:04.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0aeffd640 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07dad8b0 0x55bd07dafcd0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6be42769702017e9 server_cookie=53dcc56c36d8f37c in_seq=0 out_seq=0 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc08effd640 1 -- 172.21.15.122:0/3111457280 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd07c99040 con 0x55bd07dad8b0 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc08effd640 10 client.18579 handle_command_reply: tid=0 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0b846f740 1 client.18579 shutdown 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 msgr2=0x55bd07d31240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fc098019880 tx=0x7fc09800b480 comp rx=0 tx=0).stop 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.882+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 shutdown_connections 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bd07dad8b0 0x55bd07dafcd0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bd07d36ca0 0x55bd07d31240 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc08effd640 0 client.18579 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bd07d2e0c0 0x55bd07d30d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 --2- 172.21.15.122:0/3111457280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bd07d2d7a0 0x55bd07d3f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 >> 172.21.15.122:0/3111457280 conn(0x55bd07d27ed0 msgr2=0x55bd07d266a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 shutdown_connections 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 1 -- 172.21.15.122:0/3111457280 wait complete. 2024-04-02T13:45:04.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b846f740 20 client.18579 trim_cache size 0 max 16384 2024-04-02T13:45:04.892 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 msgr2=0x7fc09c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 0x7fc09c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80bb170 tx=0x7fc0a809c040 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 msgr2=0x7fc0b019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc0a802c790 tx=0x7fc0a8035ed0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 shutdown_connections 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc09c0a3d20 0x7fc09c0a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0b01058a0 0x7fc0b019cb90 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0b0101360 0x7fc0b0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 --2- 172.21.15.122:0/140896838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc0b01009c0 0x7fc0b01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 >> 172.21.15.122:0/140896838 conn(0x7fc0b0077a90 msgr2=0x7fc0b006a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 shutdown_connections 2024-04-02T13:45:04.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:04.883+0000 7fc0b5886640 1 -- 172.21.15.122:0/140896838 wait complete. 2024-04-02T13:45:04.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 347725 3.47 MB/sec execute 385 sec latency 28.497 ms 2024-04-02T13:45:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:05 smithi122 ceph-mon[32114]: pgmap v307: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-04-02T13:45:05.668 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:05 smithi188 ceph-mon[36117]: pgmap v307: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-04-02T13:45:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:05 smithi187 ceph-mon[36064]: pgmap v307: 129 pgs: 129 active+clean; 264 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.3 MiB/s wr, 190 op/s 2024-04-02T13:45:05.897 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:45:05.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:45:05.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:45:05.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 348333 3.47 MB/sec execute 386 sec latency 321.134 ms 2024-04-02T13:45:06.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 -- 172.21.15.122:0/697256431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa964100670 msgr2=0x7fa964100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 --2- 172.21.15.122:0/697256431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa964100670 0x7fa964100a50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fa960009900 tx=0x7fa96002bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 -- 172.21.15.122:0/697256431 shutdown_connections 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 --2- 172.21.15.122:0/697256431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9641054f0 0x7fa96410d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 --2- 172.21.15.122:0/697256431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964101020 0x7fa964104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 --2- 172.21.15.122:0/697256431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa964100670 0x7fa964100a50 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 -- 172.21.15.122:0/697256431 >> 172.21.15.122:0/697256431 conn(0x7fa964075c60 msgr2=0x7fa964076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 -- 172.21.15.122:0/697256431 shutdown_connections 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.002+0000 7fa96bec8640 1 -- 172.21.15.122:0/697256431 wait complete. 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 Processor -- start 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 -- start start 2024-04-02T13:45:06.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 0x7fa9641972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96419bbb0 0x7fa96419c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa964110dd0 con 0x7fa9641054f0 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa964110c50 con 0x7fa96419bbb0 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96bec8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa964110f50 con 0x7fa964100670 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33080/0 (socket says 172.21.15.122:33080) 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 -- 172.21.15.122:0/3623026771 learned_addr learned my addr 172.21.15.122:0/3623026771 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa96a465640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96419bbb0 0x7fa96419c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969c64640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 0x7fa9641972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 -- 172.21.15.122:0/3623026771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 msgr2=0x7fa9641972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 0x7fa9641972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.003+0000 7fa969463640 1 -- 172.21.15.122:0/3623026771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96419bbb0 msgr2=0x7fa96419c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa969463640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96419bbb0 0x7fa96419c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa969463640 1 -- 172.21.15.122:0/3623026771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96419c6a0 con 0x7fa9641054f0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa969c64640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 0x7fa9641972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa969463640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa954005fd0 tx=0x7fa95400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa95affd640 1 -- 172.21.15.122:0/3623026771 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954017070 con 0x7fa9641054f0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96419c990 con 0x7fa9641054f0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa95affd640 1 -- 172.21.15.122:0/3623026771 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa954009c30 con 0x7fa9641054f0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa95affd640 1 -- 172.21.15.122:0/3623026771 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954003d70 con 0x7fa9641054f0 2024-04-02T13:45:06.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.004+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9641a4180 con 0x7fa9641054f0 2024-04-02T13:45:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.006+0000 7fa95affd640 1 -- 172.21.15.122:0/3623026771 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa954003f10 con 0x7fa9641054f0 2024-04-02T13:45:06.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.006+0000 7fa96eb89740 1 Processor -- start 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa95affd640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 -- start start 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db95f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96a465640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db961bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7db8ba770 con 0x55c7db9562c0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c7db862960 con 0x55c7db95f7c0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96eb89740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c7db8b68c0 con 0x55c7db956be0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa95affd640 1 -- 172.21.15.122:0/3623026771 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fa9540d0550 con 0x7fa9641054f0 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33086/0 (socket says 172.21.15.122:33086) 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 -- 172.21.15.122:0/3186175469 learned_addr learned my addr 172.21.15.122:0/3186175469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969463640 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db95f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96a465640 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db961bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 -- 172.21.15.122:0/3186175469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 msgr2=0x55c7db95f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db95f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 -- 172.21.15.122:0/3186175469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 msgr2=0x55c7db961bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db961bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 -- 172.21.15.122:0/3186175469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7db6967e0 con 0x55c7db9562c0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa96a465640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 0x7fa9500a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa95c005fd0 tx=0x7fa95c010740 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa969c64640 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa960000c00 tx=0x7fa96002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c25a56d668a3f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.007+0000 7fa94ffff640 1 -- 172.21.15.122:0/3186175469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96002cb40 con 0x55c7db9562c0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa94ffff640 1 -- 172.21.15.122:0/3186175469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96002cce0 con 0x55c7db9562c0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa94ffff640 1 -- 172.21.15.122:0/3186175469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600048c0 con 0x55c7db9562c0 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 -- 172.21.15.122:0/3186175469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 msgr2=0x55c7db9566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fa960000c00 tx=0x7fa96002c5d0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 -- 172.21.15.122:0/3186175469 shutdown_connections 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db961bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db95f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3186175469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db9566a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 -- 172.21.15.122:0/3186175469 >> 172.21.15.122:0/3186175469 conn(0x55c7db9509f0 msgr2=0x55c7db950dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 -- 172.21.15.122:0/3186175469 shutdown_connections 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.008+0000 7fa96eb89740 1 -- 172.21.15.122:0/3186175469 wait complete. 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 -- start start 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 Processor -- start 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db95c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db95d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c7db962550 con 0x55c7db956be0 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c7db875d20 con 0x55c7db9562c0 2024-04-02T13:45:06.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c7db9626d0 con 0x55c7db95f7c0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa96eb89740 10 client.0 _ll_register_callbacks cb 0x55c7db842a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33094/0 (socket says 172.21.15.122:33094) 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 -- 172.21.15.122:0/3198840691 learned_addr learned my addr 172.21.15.122:0/3198840691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969c64640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db95c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 -- 172.21.15.122:0/3198840691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 msgr2=0x55c7db95d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db95d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 -- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 msgr2=0x55c7db95c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db95c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969463640 1 -- 172.21.15.122:0/3198840691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7db6f6f70 con 0x55c7db956be0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.009+0000 7fa969c64640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db95c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa969463640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa95409be50 tx=0x7fa95409bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625bd8e86e8d1832 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954020050 con 0x55c7db956be0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fa9540a0120 con 0x55c7db956be0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7db8c7cb0 con 0x55c7db956be0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa9540a0500 con 0x55c7db956be0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 10 client.18591 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c7db58e170 con 0x55c7db956be0 2024-04-02T13:45:06.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c7db8c7cb0 con 0x55c7db956be0 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa954004200 con 0x55c7db956be0 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 10 client.18591 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 20 client.18591 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db96fb40 0x55c7db971fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 4 client.18591 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:06.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7db8c2000 con 0x55c7db96fb40 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.010+0000 7fa969c64640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db96fb40 0x55c7db971fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.011+0000 7fa94d7fa640 10 client.18591 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.011+0000 7fa969c64640 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db96fb40 0x55c7db971fb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f651bd5107a1723 server_cookie=b8129b38bad1d37b in_seq=0 out_seq=0 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa94d7fa640 1 -- 172.21.15.122:0/3198840691 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7db8c2000 con 0x55c7db96fb40 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa94d7fa640 10 client.18591 handle_command_reply: tid=0 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 client.18591 shutdown 2024-04-02T13:45:06.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 msgr2=0x55c7db95cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fa95409be50 tx=0x7fa95409bba0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 shutdown_connections 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db96fb40 0x55c7db971fb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db95d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db95cc80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 --2- 172.21.15.122:0/3198840691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db95c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa94d7fa640 0 client.18591 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 >> 172.21.15.122:0/3198840691 conn(0x55c7db9509f0 msgr2=0x55c7db952740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 shutdown_connections 2024-04-02T13:45:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 1 -- 172.21.15.122:0/3198840691 wait complete. 2024-04-02T13:45:06.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.012+0000 7fa96eb89740 20 client.18591 trim_cache size 0 max 16384 2024-04-02T13:45:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 Processor -- start 2024-04-02T13:45:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 -- start start 2024-04-02T13:45:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db96b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db952500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641ae080 con 0x55c7db956be0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641afbc0 con 0x55c7db9562c0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96eb89740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641ae750 con 0x55c7db95f7c0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa969463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33104/0 (socket says 172.21.15.122:33104) 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa969463640 1 -- 172.21.15.122:0/1946423055 learned_addr learned my addr 172.21.15.122:0/1946423055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.029+0000 7fa96a465640 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db952500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 -- 172.21.15.122:0/1946423055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 msgr2=0x55c7db952500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db952500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 -- 172.21.15.122:0/1946423055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 msgr2=0x55c7db96b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969c64640 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db96b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db96b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 -- 172.21.15.122:0/1946423055 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7db950520 con 0x55c7db956be0 2024-04-02T13:45:06.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa969463640 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa954009610 tx=0x7fa95409c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f81840ee206487e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa94f7fe640 1 -- 172.21.15.122:0/1946423055 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954014030 con 0x55c7db956be0 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa94f7fe640 1 -- 172.21.15.122:0/1946423055 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9540d01b0 con 0x55c7db956be0 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa94f7fe640 1 -- 172.21.15.122:0/1946423055 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa954020070 con 0x55c7db956be0 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa96eb89740 1 -- 172.21.15.122:0/1946423055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 msgr2=0x55c7db967b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.030+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fa954009610 tx=0x7fa95409c2d0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- 172.21.15.122:0/1946423055 shutdown_connections 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db95f7c0 0x55c7db952500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db956be0 0x55c7db967b40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1946423055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db9562c0 0x55c7db96b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- 172.21.15.122:0/1946423055 >> 172.21.15.122:0/1946423055 conn(0x55c7db9509f0 msgr2=0x55c7db951df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- 172.21.15.122:0/1946423055 shutdown_connections 2024-04-02T13:45:06.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- 172.21.15.122:0/1946423055 wait complete. 2024-04-02T13:45:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- start start 2024-04-02T13:45:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 Processor -- start 2024-04-02T13:45:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db958ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db9d7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641af4f0 con 0x55c7db9562c0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641aee20 con 0x55c7db95f7c0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9641ac080 con 0x55c7db956be0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa96eb89740 10 client.0 _ll_register_callbacks cb 0x7fa95400df10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa969c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa969c64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33106/0 (socket says 172.21.15.122:33106) 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.031+0000 7fa969c64640 1 -- 172.21.15.122:0/1838667234 learned_addr learned my addr 172.21.15.122:0/1838667234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 -- 172.21.15.122:0/1838667234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 msgr2=0x55c7db958ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969463640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db958ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db958ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 -- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 msgr2=0x55c7db9d7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db9d7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 -- 172.21.15.122:0/1838667234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7db9d7650 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa969c64640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fa96002c170 tx=0x7fa96002cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a8a14dd94ca216 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96004dd50 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7fa960004040 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9540d01b0 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa960004240 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 10 client.18597 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c7db9d8800 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9540d01b0 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa960003680 con 0x55c7db9562c0 2024-04-02T13:45:06.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 10 client.18597 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 20 client.18597 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db9d8ae0 0x55c7db9daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 4 client.18597 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.032+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7db83fbe0 con 0x55c7db9d8ae0 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.033+0000 7fa969463640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db9d8ae0 0x55c7db9daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.033+0000 7fa94d7fa640 10 client.18597 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.033+0000 7fa969463640 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db9d8ae0 0x55c7db9daed0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b44e74804470d36 server_cookie=ebc3aa47c6b59323 in_seq=0 out_seq=0 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa94d7fa640 1 -- 172.21.15.122:0/1838667234 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7db83fbe0 con 0x55c7db9d8ae0 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa94d7fa640 10 client.18597 handle_command_reply: tid=0 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 client.18597 shutdown 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 msgr2=0x55c7db958990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fa96002c170 tx=0x7fa96002cea0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 shutdown_connections 2024-04-02T13:45:06.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c7db9d8ae0 0x55c7db9daed0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c7db95f7c0 0x55c7db9d7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c7db956be0 0x55c7db958ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa94d7fa640 0 client.18597 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 --2- 172.21.15.122:0/1838667234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c7db9562c0 0x55c7db958990 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 >> 172.21.15.122:0/1838667234 conn(0x55c7db9509f0 msgr2=0x55c7db95cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 shutdown_connections 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 1 -- 172.21.15.122:0/1838667234 wait complete. 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.034+0000 7fa96eb89740 20 client.18597 trim_cache size 0 max 16384 2024-04-02T13:45:06.045 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:45:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "160c030c-ddc4-4e7f-a4e2-967f4df9c16f", 2024-04-02T13:45:06.046 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:45:06.046 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 msgr2=0x7fa9500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 0x7fa9500a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa95c005fd0 tx=0x7fa95c010740 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 msgr2=0x7fa964197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa954005fd0 tx=0x7fa95400dde0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 shutdown_connections 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9500a3d20 0x7fa9500a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa96419bbb0 0x7fa96419c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9641054f0 0x7fa964197830 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 --2- 172.21.15.122:0/3623026771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa964100670 0x7fa9641972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 >> 172.21.15.122:0/3623026771 conn(0x7fa964075c60 msgr2=0x7fa9640fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 shutdown_connections 2024-04-02T13:45:06.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.035+0000 7fa96bec8640 1 -- 172.21.15.122:0/3623026771 wait complete. 2024-04-02T13:45:06.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:06.050 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:06.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 -- 172.21.15.122:0/652683916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc107a80 msgr2=0x7f9bbc107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 --2- 172.21.15.122:0/652683916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc107a80 0x7f9bbc107e60 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009930 tx=0x7f9bb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 -- 172.21.15.122:0/652683916 shutdown_connections 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 --2- 172.21.15.122:0/652683916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc10e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 --2- 172.21.15.122:0/652683916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc101a70 0x7f9bbc101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 --2- 172.21.15.122:0/652683916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc107a80 0x7f9bbc107e60 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 -- 172.21.15.122:0/652683916 >> 172.21.15.122:0/652683916 conn(0x7f9bbc0fd560 msgr2=0x7f9bbc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 -- 172.21.15.122:0/652683916 shutdown_connections 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.169+0000 7f9bc463c640 1 -- 172.21.15.122:0/652683916 wait complete. 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 Processor -- start 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 -- start start 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc107a80 0x7f9bbc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc111e90 con 0x7f9bbc101a70 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc111d10 con 0x7f9bbc102430 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc463c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc112010 con 0x7f9bbc107a80 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33136/0 (socket says 172.21.15.122:33136) 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 -- 172.21.15.122:0/278067691 learned_addr learned my addr 172.21.15.122:0/278067691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc107a80 0x7f9bbc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 -- 172.21.15.122:0/278067691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc107a80 msgr2=0x7f9bbc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc107a80 0x7f9bbc19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 -- 172.21.15.122:0/278067691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 msgr2=0x7f9bbc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 -- 172.21.15.122:0/278067691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc19d3e0 con 0x7f9bbc101a70 2024-04-02T13:45:06.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.170+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009230 tx=0x7f9bb802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.171+0000 7f9bb37fe640 1 -- 172.21.15.122:0/278067691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8041070 con 0x7f9bbc101a70 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.171+0000 7f9bb37fe640 1 -- 172.21.15.122:0/278067691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb8004450 con 0x7f9bbc101a70 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.171+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc19d670 con 0x7f9bbc101a70 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.171+0000 7f9bb37fe640 1 -- 172.21.15.122:0/278067691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8034440 con 0x7f9bbc101a70 2024-04-02T13:45:06.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.171+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc1a4f60 con 0x7f9bbc101a70 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.173+0000 7f9bb37fe640 1 -- 172.21.15.122:0/278067691 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9bb8044070 con 0x7f9bbc101a70 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.173+0000 7f9bc7224740 1 Processor -- start 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.173+0000 7f9bc7224740 1 -- start start 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.173+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657858d0 0x561265785cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657861f0 0x56126578e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc7224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5612656e9830 con 0x56126578edd0 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc7224740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561265691820 con 0x5612657858d0 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc7224740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5612656e57e0 con 0x5612657861f0 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bb37fe640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 0x7f9ba80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc23d8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657861f0 0x56126578e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33150/0 (socket says 172.21.15.122:33150) 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/639351482 learned_addr learned my addr 172.21.15.122:0/639351482 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657858d0 0x561265785cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 0x7f9ba80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bb37fe640 1 -- 172.21.15.122:0/278067691 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f9bb80f2560 con 0x7f9bbc101a70 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/639351482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657861f0 msgr2=0x56126578e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657861f0 0x56126578e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/639351482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657858d0 msgr2=0x561265785cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657858d0 0x561265785cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/639351482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612654c77d0 con 0x56126578edd0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 0x7f9ba80a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005fd0 tx=0x7f9bb4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00cc60 tx=0x7f9bac00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5ef88af6a555bc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bb0bfe640 1 -- 172.21.15.122:0/639351482 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac014070 con 0x56126578edd0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bb0bfe640 1 -- 172.21.15.122:0/639351482 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac0047a0 con 0x56126578edd0 2024-04-02T13:45:06.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.174+0000 7f9bb0bfe640 1 -- 172.21.15.122:0/639351482 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac010720 con 0x56126578edd0 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 -- 172.21.15.122:0/639351482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 msgr2=0x5612657911c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00cc60 tx=0x7f9bac00e720 comp rx=0 tx=0).stop 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 -- 172.21.15.122:0/639351482 shutdown_connections 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x5612657911c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657861f0 0x56126578e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 --2- 172.21.15.122:0/639351482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657858d0 0x561265785cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 -- 172.21.15.122:0/639351482 >> 172.21.15.122:0/639351482 conn(0x561265780000 msgr2=0x5612657803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 -- 172.21.15.122:0/639351482 shutdown_connections 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.175+0000 7f9bc7224740 1 -- 172.21.15.122:0/639351482 wait complete. 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 -- start start 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 Processor -- start 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126578e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657861f0 0x56126578b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561265791bc0 con 0x56126578edd0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5612656a4d30 con 0x5612657861f0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561265791d40 con 0x5612657858d0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc7224740 10 client.0 _ll_register_callbacks cb 0x5612656a4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33158/0 (socket says 172.21.15.122:33158) 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 -- 172.21.15.122:0/1296892898 learned_addr learned my addr 172.21.15.122:0/1296892898 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126578e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657861f0 0x56126578b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 -- 172.21.15.122:0/1296892898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 msgr2=0x56126578e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126578e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 -- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657861f0 msgr2=0x56126578b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657861f0 0x56126578b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 -- 172.21.15.122:0/1296892898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612653fd3f0 con 0x56126578edd0 2024-04-02T13:45:06.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4003c40 tx=0x7f9bb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4030b996fd0ef171 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.176+0000 7f9ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4025070 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f9bb4011ce0 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bac0047a0 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bb4031050 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 10 client.18609 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5612653bd0f0 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac0047a0 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9bb4020040 con 0x56126578edd0 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 10 client.18609 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 20 client.18609 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56126579eda0 0x5612657a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 4 client.18609 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:06.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612656f6d70 con 0x56126579eda0 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.177+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56126579eda0 0x5612657a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.178+0000 7f9ba5ffb640 10 client.18609 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.178+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56126579eda0 0x5612657a1240 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8538e12a9e9710 server_cookie=5e7727194774c15e in_seq=0 out_seq=0 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.178+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1296892898 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612656f6d70 con 0x56126579eda0 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.178+0000 7f9ba5ffb640 10 client.18609 handle_command_reply: tid=0 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.178+0000 7f9bc7224740 1 client.18609 shutdown 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 msgr2=0x56126578bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4003c40 tx=0x7f9bb4004a40 comp rx=0 tx=0).stop 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 shutdown_connections 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56126579eda0 0x5612657a1240 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56126578edd0 0x56126578bf30 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9ba5ffb640 0 client.18609 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5612657861f0 0x56126578b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1296892898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126578e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 >> 172.21.15.122:0/1296892898 conn(0x561265780000 msgr2=0x561265781d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 shutdown_connections 2024-04-02T13:45:06.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 1 -- 172.21.15.122:0/1296892898 wait complete. 2024-04-02T13:45:06.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.179+0000 7f9bc7224740 20 client.18609 trim_cache size 0 max 16384 2024-04-02T13:45:06.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 Processor -- start 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 -- start start 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126579aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x56126578e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1af250 con 0x5612657861f0 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b0d90 con 0x56126578edd0 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc7224740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1af920 con 0x5612657858d0 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33168/0 (socket says 172.21.15.122:33168) 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/3492844692 learned_addr learned my addr 172.21.15.122:0/3492844692 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/3492844692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 msgr2=0x56126579aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126579aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/3492844692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 msgr2=0x56126578e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x56126578e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/3492844692 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56126578c530 con 0x5612657861f0 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.196+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ec20 tx=0x7f9bac005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da26e0c2f44431a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9ba7fff640 1 -- 172.21.15.122:0/3492844692 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac025070 con 0x5612657861f0 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9ba7fff640 1 -- 172.21.15.122:0/3492844692 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac004660 con 0x5612657861f0 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9ba7fff640 1 -- 172.21.15.122:0/3492844692 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac014050 con 0x5612657861f0 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- 172.21.15.122:0/3492844692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 msgr2=0x56126578e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ec20 tx=0x7f9bac005d00 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- 172.21.15.122:0/3492844692 shutdown_connections 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x56126578e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x56126578e120 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 --2- 172.21.15.122:0/3492844692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x56126579aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- 172.21.15.122:0/3492844692 >> 172.21.15.122:0/3492844692 conn(0x561265780000 msgr2=0x5612657870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- 172.21.15.122:0/3492844692 shutdown_connections 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- 172.21.15.122:0/3492844692 wait complete. 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.197+0000 7f9bc7224740 1 -- start start 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 Processor -- start 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x561265797180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x561265788780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1b06c0 con 0x5612657861f0 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1afff0 con 0x56126578edd0 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1ad250 con 0x5612657858d0 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 10 client.0 _ll_register_callbacks cb 0x7f9bb4014350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33182/0 (socket says 172.21.15.122:33182) 2024-04-02T13:45:06.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/1163488961 learned_addr learned my addr 172.21.15.122:0/1163488961 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x561265797180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc2bd9640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x561265788780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/1163488961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 msgr2=0x561265797180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x561265797180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 msgr2=0x561265788780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x561265788780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 -- 172.21.15.122:0/1163488961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56126577fe30 con 0x5612657861f0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc1bd7640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0057c0 tx=0x7f9bac004ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3e1c170657581a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9ba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac02e030 con 0x5612657861f0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9613+0+0 (secure 0 0 0) 0x7f9bac004020 con 0x5612657861f0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.198+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bac004660 con 0x5612657861f0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9bac038050 con 0x5612657861f0 2024-04-02T13:45:06.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 10 client.18615 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x561265804ff0 con 0x5612657861f0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac004660 con 0x5612657861f0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9bac003850 con 0x5612657861f0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 10 client.18615 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 10 client.18615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 20 client.18615 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5612658054f0 0x561265807910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 4 client.18615 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612656f10c0 con 0x5612658054f0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5612658054f0 0x561265807910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9bc23d8640 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5612658054f0 0x561265807910 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a314b410bc2332bd server_cookie=ad500b898f0d7136 in_seq=0 out_seq=0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.199+0000 7f9ba5ffb640 10 client.18615 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9ba5ffb640 1 -- 172.21.15.122:0/1163488961 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5612656f10c0 con 0x5612658054f0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9ba5ffb640 10 client.18615 handle_command_reply: tid=0 2024-04-02T13:45:06.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 client.18615 shutdown 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 msgr2=0x561265788240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0057c0 tx=0x7f9bac004ea0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 shutdown_connections 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5612658054f0 0x561265807910 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56126578edd0 0x561265788780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9ba5ffb640 0 client.18615 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5612657861f0 0x561265788240 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 --2- 172.21.15.122:0/1163488961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5612657858d0 0x561265797180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 >> 172.21.15.122:0/1163488961 conn(0x561265780000 msgr2=0x56126577e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 shutdown_connections 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 1 -- 172.21.15.122:0/1163488961 wait complete. 2024-04-02T13:45:06.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc7224740 20 client.18615 trim_cache size 0 max 16384 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "160c030c-ddc4-4e7f-a4e2-967f4df9c16f": { 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "160c030c-ddc4-4e7f-a4e2-967f4df9c16f", 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:45:06.210 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:06.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 msgr2=0x7f9ba80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 0x7f9ba80a6260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005fd0 tx=0x7f9bb4009500 comp rx=0 tx=0).stop 2024-04-02T13:45:06.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 msgr2=0x7f9bbc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:06.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8009230 tx=0x7f9bb802cba0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 shutdown_connections 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba80a3da0 0x7f9ba80a6260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc107a80 0x7f9bbc19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc102430 0x7f9bbc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 --2- 172.21.15.122:0/278067691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc101a70 0x7f9bbc1983f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 >> 172.21.15.122:0/278067691 conn(0x7f9bbc0fd560 msgr2=0x7f9bbc0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 shutdown_connections 2024-04-02T13:45:06.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:06.200+0000 7f9bc463c640 1 -- 172.21.15.122:0/278067691 wait complete. 2024-04-02T13:45:06.214 INFO:tasks.cephfs.filesystem:scrub status for tag:160c030c-ddc4-4e7f-a4e2-967f4df9c16f - {'path': '/', 'tag': '160c030c-ddc4-4e7f-a4e2-967f4df9c16f', 'options': 'recursive,force'} 2024-04-02T13:45:06.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 349418 3.47 MB/sec execute 387 sec latency 426.376 ms 2024-04-02T13:45:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: pgmap v308: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:45:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:07.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:07 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: pgmap v308: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:45:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:07.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:07 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: pgmap v308: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:07 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:07.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 351399 3.50 MB/sec execute 388 sec latency 13.252 ms 2024-04-02T13:45:08.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 352606 3.50 MB/sec execute 389 sec latency 18.641 ms 2024-04-02T13:45:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:09 smithi122 ceph-mon[32114]: pgmap v309: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:09 smithi187 ceph-mon[36064]: pgmap v309: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:09 smithi188 ceph-mon[36117]: pgmap v309: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:09.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 354780 3.53 MB/sec execute 390 sec latency 22.285 ms 2024-04-02T13:45:10.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 355491 3.52 MB/sec execute 391 sec latency 211.369 ms 2024-04-02T13:45:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:11 smithi122 ceph-mon[32114]: pgmap v310: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:11 smithi122 ceph-mon[32114]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:45:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:11 smithi187 ceph-mon[36064]: pgmap v310: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:11 smithi187 ceph-mon[36064]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:45:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:11 smithi188 ceph-mon[36117]: pgmap v310: 129 pgs: 129 active+clean; 268 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 241 op/s 2024-04-02T13:45:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:11 smithi188 ceph-mon[36117]: osdmap e103: 12 total, 12 up, 12 in 2024-04-02T13:45:11.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 355491 3.51 MB/sec execute 392 sec latency 1211.469 ms 2024-04-02T13:45:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:12.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:12.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:12.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 355491 3.50 MB/sec execute 393 sec latency 2211.566 ms 2024-04-02T13:45:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:13 smithi122 ceph-mon[32114]: pgmap v312: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-04-02T13:45:13.636 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:13 smithi187 ceph-mon[36064]: pgmap v312: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-04-02T13:45:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:13 smithi188 ceph-mon[36117]: pgmap v312: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 261 op/s 2024-04-02T13:45:13.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 356230 3.51 MB/sec execute 394 sec latency 2654.088 ms 2024-04-02T13:45:14.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 358363 3.53 MB/sec execute 395 sec latency 24.861 ms 2024-04-02T13:45:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:15 smithi122 ceph-mon[32114]: pgmap v313: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-04-02T13:45:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:15 smithi187 ceph-mon[36064]: pgmap v313: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-04-02T13:45:15.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:15 smithi188 ceph-mon[36117]: pgmap v313: 129 pgs: 129 active+clean; 276 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 253 op/s 2024-04-02T13:45:15.971 INFO:tasks.workunit.client.0.smithi122.stdout: 1 358972 3.52 MB/sec execute 396 sec latency 298.975 ms 2024-04-02T13:45:16.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 358972 3.52 MB/sec execute 397 sec latency 1299.090 ms 2024-04-02T13:45:17.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:17 smithi122 ceph-mon[32114]: pgmap v314: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:17 smithi187 ceph-mon[36064]: pgmap v314: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:17.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:17 smithi188 ceph-mon[36117]: pgmap v314: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:17.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 358972 3.51 MB/sec execute 398 sec latency 2299.202 ms 2024-04-02T13:45:18.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359040 3.50 MB/sec execute 399 sec latency 2744.819 ms 2024-04-02T13:45:19.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:19 smithi122 ceph-mon[32114]: pgmap v315: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:19 smithi187 ceph-mon[36064]: pgmap v315: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:19.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:19 smithi188 ceph-mon[36117]: pgmap v315: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:19.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359040 3.49 MB/sec execute 400 sec latency 1498.697 ms 2024-04-02T13:45:20.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359040 3.48 MB/sec execute 401 sec latency 2498.798 ms 2024-04-02T13:45:21.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:21 smithi122 ceph-mon[32114]: pgmap v316: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:21 smithi187 ceph-mon[36064]: pgmap v316: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:21 smithi188 ceph-mon[36117]: pgmap v316: 129 pgs: 129 active+clean; 280 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-04-02T13:45:21.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359040 3.47 MB/sec execute 402 sec latency 3498.916 ms 2024-04-02T13:45:22.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359040 3.46 MB/sec execute 403 sec latency 4499.012 ms 2024-04-02T13:45:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:23 smithi187 ceph-mon[36064]: pgmap v317: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2024-04-02T13:45:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:23 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:23 smithi188 ceph-mon[36117]: pgmap v317: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2024-04-02T13:45:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:23 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:23 smithi122 ceph-mon[32114]: pgmap v317: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.3 MiB/s wr, 87 op/s 2024-04-02T13:45:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:23 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:45:23.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 359953 3.47 MB/sec execute 404 sec latency 4937.657 ms 2024-04-02T13:45:24.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 362047 3.49 MB/sec execute 405 sec latency 34.807 ms 2024-04-02T13:45:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: pgmap v318: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-04-02T13:45:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:25 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: pgmap v318: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:25 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: pgmap v318: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.2 MiB/s wr, 77 op/s 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:25 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:25.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 362458 3.48 MB/sec execute 406 sec latency 479.831 ms 2024-04-02T13:45:26.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:26.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:45:26.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:26.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:26.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:45:26.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:45:26.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:45:26.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:45:26.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:45:26.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:45:26.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 362458 3.48 MB/sec execute 407 sec latency 1479.897 ms 2024-04-02T13:45:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:27 smithi187 ceph-mon[36064]: pgmap v319: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-02T13:45:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:27 smithi188 ceph-mon[36117]: pgmap v319: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-02T13:45:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:27 smithi122 ceph-mon[32114]: pgmap v319: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-04-02T13:45:27.972 INFO:tasks.workunit.client.0.smithi122.stdout: 1 362458 3.47 MB/sec execute 408 sec latency 2479.997 ms 2024-04-02T13:45:28.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 363321 3.47 MB/sec execute 409 sec latency 2921.895 ms 2024-04-02T13:45:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:29 smithi187 ceph-mon[36064]: pgmap v320: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:29 smithi188 ceph-mon[36117]: pgmap v320: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:29 smithi122 ceph-mon[32114]: pgmap v320: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:29.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 365560 3.50 MB/sec execute 410 sec latency 31.040 ms 2024-04-02T13:45:30.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 366130 3.49 MB/sec execute 411 sec latency 295.048 ms 2024-04-02T13:45:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:31 smithi187 ceph-mon[36064]: pgmap v321: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:31 smithi188 ceph-mon[36117]: pgmap v321: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:31 smithi122 ceph-mon[32114]: pgmap v321: 129 pgs: 129 active+clean; 283 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-02T13:45:31.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 366130 3.48 MB/sec execute 412 sec latency 1295.179 ms 2024-04-02T13:45:32.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 366130 3.47 MB/sec execute 413 sec latency 2295.251 ms 2024-04-02T13:45:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:33 smithi187 ceph-mon[36064]: pgmap v322: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-04-02T13:45:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:33 smithi188 ceph-mon[36117]: pgmap v322: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-04-02T13:45:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:33 smithi122 ceph-mon[32114]: pgmap v322: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-04-02T13:45:33.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 367166 3.48 MB/sec execute 414 sec latency 2732.926 ms 2024-04-02T13:45:34.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 369178 3.50 MB/sec execute 415 sec latency 22.302 ms 2024-04-02T13:45:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:35 smithi187 ceph-mon[36064]: pgmap v323: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-02T13:45:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:35 smithi188 ceph-mon[36117]: pgmap v323: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-02T13:45:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:35 smithi122 ceph-mon[32114]: pgmap v323: 129 pgs: 129 active+clean; 287 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 1.5 MiB/s wr, 121 op/s 2024-04-02T13:45:35.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 369550 3.49 MB/sec execute 416 sec latency 565.228 ms 2024-04-02T13:45:36.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:36.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:36.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- 172.21.15.122:0/389038325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4105130 msgr2=0x7fadc4105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 --2- 172.21.15.122:0/389038325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4105130 0x7fadc4105510 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009900 tx=0x7fadb802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- 172.21.15.122:0/389038325 shutdown_connections 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 --2- 172.21.15.122:0/389038325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4100d60 0x7fadc410d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 --2- 172.21.15.122:0/389038325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc41003c0 0x7fadc4100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 --2- 172.21.15.122:0/389038325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc4105130 0x7fadc4105510 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- 172.21.15.122:0/389038325 >> 172.21.15.122:0/389038325 conn(0x7fadc4075d40 msgr2=0x7fadc4076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- 172.21.15.122:0/389038325 shutdown_connections 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- 172.21.15.122:0/389038325 wait complete. 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 Processor -- start 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.321+0000 7fadcae93640 1 -- start start 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 0x7fadc41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105130 0x7fadc419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111ed0 con 0x7fadc41003c0 2024-04-02T13:45:36.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4111d50 con 0x7fadc4100d60 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4112050 con 0x7fadc4105130 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50268/0 (socket says 172.21.15.122:50268) 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1571385872 learned_addr learned my addr 172.21.15.122:0/1571385872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc9430640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105130 0x7fadc419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 0x7fadc41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1571385872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105130 msgr2=0x7fadc419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105130 0x7fadc419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1571385872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 msgr2=0x7fadc41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 0x7fadc41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1571385872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc419d350 con 0x7fadc41003c0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 0x7fadc41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005940 tx=0x7fadb80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc1ffb640 1 -- 172.21.15.122:0/1571385872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8041070 con 0x7fadc41003c0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc1ffb640 1 -- 172.21.15.122:0/1571385872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb802ccc0 con 0x7fadc41003c0 2024-04-02T13:45:36.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc419d5e0 con 0x7fadc41003c0 2024-04-02T13:45:36.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.322+0000 7fadc1ffb640 1 -- 172.21.15.122:0/1571385872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8003e20 con 0x7fadc41003c0 2024-04-02T13:45:36.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.323+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a4ed0 con 0x7fadc41003c0 2024-04-02T13:45:36.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc1ffb640 1 -- 172.21.15.122:0/1571385872 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fadb8044070 con 0x7fadc41003c0 2024-04-02T13:45:36.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 Processor -- start 2024-04-02T13:45:36.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 -- start start 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc1ffb640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96b8d0 0x56319d96bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 0x56319d9771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56319d8cf830 con 0x56319d974dd0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56319d877820 con 0x56319d96c1f0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadcda7f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56319d8cb7e0 con 0x56319d96b8d0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc9430640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc1ffb640 1 -- 172.21.15.122:0/1571385872 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fadb80f2730 con 0x7fadc41003c0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:42424/0 (socket says 172.21.15.122:42424) 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc9430640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 0x56319d9771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc3fff640 1 -- 172.21.15.122:0/3238363951 learned_addr learned my addr 172.21.15.122:0/3238363951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.325+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96b8d0 0x56319d96bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 -- 172.21.15.122:0/3238363951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96b8d0 msgr2=0x56319d96bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96b8d0 0x56319d96bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 -- 172.21.15.122:0/3238363951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 msgr2=0x56319d9771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 0x56319d9771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 -- 172.21.15.122:0/3238363951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319d6ad7d0 con 0x56319d96c1f0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc9430640 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 0x56319d9771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc9430640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 0x7fadb40a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadbc013a90 tx=0x7fadbc00bc70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadc3fff640 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009c20 tx=0x7fadb000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae8e804760fc93e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadaeffd640 1 -- 172.21.15.122:0/3238363951 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb0016070 con 0x56319d96c1f0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadaeffd640 1 -- 172.21.15.122:0/3238363951 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb0009e50 con 0x56319d96c1f0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.326+0000 7fadaeffd640 1 -- 172.21.15.122:0/3238363951 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00107f0 con 0x56319d96c1f0 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- 172.21.15.122:0/3238363951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 msgr2=0x56319d974890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009c20 tx=0x7fadb000e6c0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- 172.21.15.122:0/3238363951 shutdown_connections 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d974dd0 0x56319d9771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d96c1f0 0x56319d974890 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 --2- 172.21.15.122:0/3238363951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96b8d0 0x56319d96bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- 172.21.15.122:0/3238363951 >> 172.21.15.122:0/3238363951 conn(0x56319d966000 msgr2=0x56319d9663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- 172.21.15.122:0/3238363951 shutdown_connections 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- 172.21.15.122:0/3238363951 wait complete. 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 -- start start 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.327+0000 7fadcda7f740 1 Processor -- start 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d971970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d971eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56319d977bc0 con 0x56319d96b8d0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56319d88ad30 con 0x56319d974dd0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56319d977d40 con 0x56319d96c1f0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 10 client.0 _ll_register_callbacks cb 0x56319d88a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50282/0 (socket says 172.21.15.122:50282) 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 -- 172.21.15.122:0/2247349938 learned_addr learned my addr 172.21.15.122:0/2247349938 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc9430640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d971eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc3fff640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d971970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 -- 172.21.15.122:0/2247349938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 msgr2=0x56319d971970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d971970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 -- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 msgr2=0x56319d971eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d971eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 -- 172.21.15.122:0/2247349938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319d5e33f0 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc3fff640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d971970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fadb80be380 tx=0x7fadb8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83e0c6b624e59ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fada3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb804b040 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fadb80efd10 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb0009e50 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fadb80be3f0 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 10 client.18627 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.328+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56319d5a30f0 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb0009e50 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fadb80e9360 con 0x56319d96b8d0 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 10 client.18627 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 20 client.18627 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d984d00 0x56319d987190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 4 client.18627 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56319d8dcd70 con 0x56319d984d00 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadc3fff640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d984d00 0x56319d987190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fadc3fff640 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d984d00 0x56319d987190 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1706f57b3a112b server_cookie=85b818125b8a7fe8 in_seq=0 out_seq=0 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.329+0000 7fada3fff640 10 client.18627 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fada3fff640 1 -- 172.21.15.122:0/2247349938 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56319d8dcd70 con 0x56319d984d00 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fada3fff640 10 client.18627 handle_command_reply: tid=0 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 client.18627 shutdown 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 msgr2=0x56319d9745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fadb80be380 tx=0x7fadb8009680 comp rx=0 tx=0).stop 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 shutdown_connections 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d984d00 0x56319d987190 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d971eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fada3fff640 0 client.18627 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:36.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d971970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 --2- 172.21.15.122:0/2247349938 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d9745e0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 >> 172.21.15.122:0/2247349938 conn(0x56319d966000 msgr2=0x56319d967d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 shutdown_connections 2024-04-02T13:45:36.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 1 -- 172.21.15.122:0/2247349938 wait complete. 2024-04-02T13:45:36.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.330+0000 7fadcda7f740 20 client.18627 trim_cache size 0 max 16384 2024-04-02T13:45:36.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 Processor -- start 2024-04-02T13:45:36.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 -- start start 2024-04-02T13:45:36.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d970ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41af0a0 con 0x56319d96b8d0 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41b0be0 con 0x56319d974dd0 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadcda7f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41af770 con 0x56319d96c1f0 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50296/0 (socket says 172.21.15.122:50296) 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 -- 172.21.15.122:0/926477544 learned_addr learned my addr 172.21.15.122:0/926477544 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 -- 172.21.15.122:0/926477544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 msgr2=0x56319d970ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d970ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 -- 172.21.15.122:0/926477544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 msgr2=0x56319d96ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 -- 172.21.15.122:0/926477544 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56319d96c020 con 0x56319d96b8d0 2024-04-02T13:45:36.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.348+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fadb80efc50 tx=0x7fadb8005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e091f4d71a7e83a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadae7fc640 1 -- 172.21.15.122:0/926477544 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80d7070 con 0x56319d96b8d0 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadae7fc640 1 -- 172.21.15.122:0/926477544 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80c27a0 con 0x56319d96b8d0 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadae7fc640 1 -- 172.21.15.122:0/926477544 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb804b040 con 0x56319d96b8d0 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 -- 172.21.15.122:0/926477544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 msgr2=0x56319d980ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fadb80efc50 tx=0x7fadb8005700 comp rx=0 tx=0).stop 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 -- 172.21.15.122:0/926477544 shutdown_connections 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d970ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 --2- 172.21.15.122:0/926477544 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d980ab0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 -- 172.21.15.122:0/926477544 >> 172.21.15.122:0/926477544 conn(0x56319d966000 msgr2=0x56319d96c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 -- 172.21.15.122:0/926477544 shutdown_connections 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.349+0000 7fadcda7f740 1 -- 172.21.15.122:0/926477544 wait complete. 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 -- start start 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 Processor -- start 2024-04-02T13:45:36.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d96eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41b0510 con 0x56319d96b8d0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41afe40 con 0x56319d974dd0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fadc41ad0a0 con 0x56319d96c1f0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadcda7f740 10 client.0 _ll_register_callbacks cb 0x7fadb80bdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50302/0 (socket says 172.21.15.122:50302) 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1236428711 learned_addr learned my addr 172.21.15.122:0/1236428711 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d96eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc9430640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1236428711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 msgr2=0x56319d96eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d96eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 msgr2=0x56319d96f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc8c2f640 1 -- 172.21.15.122:0/1236428711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319d9ea280 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc9430640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.350+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d96eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadc8c2f640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ba850 tx=0x7fadb8009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e205833a27eba1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb80dc060 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fadb80c6dd0 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb80c27a0 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fadb804b070 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 10 client.18633 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56319d9ea280 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadb80c27a0 con 0x56319d96b8d0 2024-04-02T13:45:36.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fadb80bd840 con 0x56319d96b8d0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 10 client.18633 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 10 client.18633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 20 client.18633 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d9eb7d0 0x56319d9edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 4 client.18633 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56319d8d70c0 con 0x56319d9eb7d0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.351+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d9eb7d0 0x56319d9edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadc3fff640 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d9eb7d0 0x56319d9edbc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da754b312b66466 server_cookie=b5a35009932cc6c0 in_seq=0 out_seq=0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fada3fff640 10 client.18633 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fada3fff640 1 -- 172.21.15.122:0/1236428711 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56319d8d70c0 con 0x56319d9eb7d0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fada3fff640 10 client.18633 handle_command_reply: tid=0 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 client.18633 shutdown 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 msgr2=0x56319d96e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fadb80ba850 tx=0x7fadb8009680 comp rx=0 tx=0).stop 2024-04-02T13:45:36.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 shutdown_connections 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56319d9eb7d0 0x56319d9edbc0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56319d974dd0 0x56319d96f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fada3fff640 0 client.18633 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56319d96c1f0 0x56319d96eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 --2- 172.21.15.122:0/1236428711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56319d96b8d0 0x56319d96e5a0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 >> 172.21.15.122:0/1236428711 conn(0x56319d966000 msgr2=0x56319d965240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 shutdown_connections 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 1 -- 172.21.15.122:0/1236428711 wait complete. 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.352+0000 7fadcda7f740 20 client.18633 trim_cache size 0 max 16384 2024-04-02T13:45:36.362 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:36.363 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:45:36.363 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:45:36.363 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:36.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 msgr2=0x7fadb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 0x7fadb40a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadbc013a90 tx=0x7fadbc00bc70 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 msgr2=0x7fadc41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005940 tx=0x7fadb80043b0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 shutdown_connections 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fadb40a3d20 0x7fadb40a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadc4105130 0x7fadc419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadc4100d60 0x7fadc41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 --2- 172.21.15.122:0/1571385872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc41003c0 0x7fadc41983b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 >> 172.21.15.122:0/1571385872 conn(0x7fadc4075d40 msgr2=0x7fadc40fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 shutdown_connections 2024-04-02T13:45:36.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.353+0000 7fadcae93640 1 -- 172.21.15.122:0/1571385872 wait complete. 2024-04-02T13:45:36.366 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:45:36.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:45:36.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.465+0000 7ff71c782640 1 -- 172.21.15.122:0/3476565213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7141013c0 msgr2=0x7ff71410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.465+0000 7ff71c782640 1 --2- 172.21.15.122:0/3476565213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7141013c0 0x7ff71410d8d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009930 tx=0x7ff70c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- 172.21.15.122:0/3476565213 shutdown_connections 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- 172.21.15.122:0/3476565213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7141013c0 0x7ff71410d8d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- 172.21.15.122:0/3476565213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714100a00 0x7ff714100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- 172.21.15.122:0/3476565213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff714106a10 0x7ff714106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- 172.21.15.122:0/3476565213 >> 172.21.15.122:0/3476565213 conn(0x7ff714077b00 msgr2=0x7ff714077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- 172.21.15.122:0/3476565213 shutdown_connections 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- 172.21.15.122:0/3476565213 wait complete. 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 Processor -- start 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- start start 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 0x7ff714198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714106a10 0x7ff71419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff714110e20 con 0x7ff714100a00 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff714110ca0 con 0x7ff7141013c0 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71c782640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff714110fa0 con 0x7ff714106a10 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71a51e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71a51e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50332/0 (socket says 172.21.15.122:50332) 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.466+0000 7ff71a51e640 1 -- 172.21.15.122:0/3364740710 learned_addr learned my addr 172.21.15.122:0/3364740710 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71ad1f640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714106a10 0x7ff71419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff719d1d640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 0x7ff714198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 -- 172.21.15.122:0/3364740710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714106a10 msgr2=0x7ff71419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714106a10 0x7ff71419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 -- 172.21.15.122:0/3364740710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 msgr2=0x7ff714198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 0x7ff714198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 -- 172.21.15.122:0/3364740710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff71419d2e0 con 0x7ff714100a00 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff719d1d640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 0x7ff714198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71a51e640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff71000ebc0 tx=0x7ff71000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710010630 con 0x7ff714100a00 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff710010cb0 con 0x7ff714100a00 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff71419d5d0 con 0x7ff714100a00 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710018bd0 con 0x7ff714100a00 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.467+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7141a4e60 con 0x7ff714100a00 2024-04-02T13:45:36.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.468+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7140688f0 con 0x7ff714100a00 2024-04-02T13:45:36.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.469+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff7100107d0 con 0x7ff714100a00 2024-04-02T13:45:36.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.470+0000 7ff70b7fe640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.470+0000 7ff719d1d640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.470+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7ff7100d14b0 con 0x7ff714100a00 2024-04-02T13:45:36.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.470+0000 7ff719d1d640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 0x7ff7000a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff7141005a0 tx=0x7ff70400a5c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.473+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff71009d760 con 0x7ff714100a00 2024-04-02T13:45:36.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.695+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff714199640 con 0x7ff714100a00 2024-04-02T13:45:36.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.696+0000 7ff70b7fe640 1 -- 172.21.15.122:0/3364740710 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7ff71009d060 con 0x7ff714100a00 2024-04-02T13:45:36.699 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:45:36.699 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:45:36.700 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:45:36.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 msgr2=0x7ff7000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 0x7ff7000a61e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff7141005a0 tx=0x7ff70400a5c0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 msgr2=0x7ff714198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7ff71000ebc0 tx=0x7ff71000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 shutdown_connections 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7000a3d20 0x7ff7000a61e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff714106a10 0x7ff71419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7141013c0 0x7ff714198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 --2- 172.21.15.122:0/3364740710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff714100a00 0x7ff714198340 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 >> 172.21.15.122:0/3364740710 conn(0x7ff714077b00 msgr2=0x7ff714068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 shutdown_connections 2024-04-02T13:45:36.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.698+0000 7ff71c782640 1 -- 172.21.15.122:0/3364740710 wait complete. 2024-04-02T13:45:36.712 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:45:36.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- 172.21.15.122:0/853480549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc107aa0 msgr2=0x7fcafc107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 --2- 172.21.15.122:0/853480549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc107aa0 0x7fcafc107e80 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009930 tx=0x7fcaec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- 172.21.15.122:0/853480549 shutdown_connections 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 --2- 172.21.15.122:0/853480549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 0x7fcafc10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 --2- 172.21.15.122:0/853480549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc101a90 0x7fcafc101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 --2- 172.21.15.122:0/853480549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc107aa0 0x7fcafc107e80 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- 172.21.15.122:0/853480549 >> 172.21.15.122:0/853480549 conn(0x7fcafc078f00 msgr2=0x7fcafc079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- 172.21.15.122:0/853480549 shutdown_connections 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- 172.21.15.122:0/853480549 wait complete. 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 Processor -- start 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.814+0000 7fcb02372640 1 -- start start 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 0x7fcafc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc107aa0 0x7fcafc19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc111e80 con 0x7fcafc101a90 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc111d00 con 0x7fcafc102450 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc112000 con 0x7fcafc107aa0 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50340/0 (socket says 172.21.15.122:50340) 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 -- 172.21.15.122:0/1509956325 learned_addr learned my addr 172.21.15.122:0/1509956325 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc107aa0 0x7fcafc19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 0x7fcafc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 -- 172.21.15.122:0/1509956325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc107aa0 msgr2=0x7fcafc19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc107aa0 0x7fcafc19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 -- 172.21.15.122:0/1509956325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 msgr2=0x7fcafc198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 0x7fcafc198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 -- 172.21.15.122:0/1509956325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcafc19d3b0 con 0x7fcafc101a90 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcafbfff640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009230 tx=0x7fcaec02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcaf97fa640 1 -- 172.21.15.122:0/1509956325 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec041070 con 0x7fcafc101a90 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcaf97fa640 1 -- 172.21.15.122:0/1509956325 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaec004450 con 0x7fcafc101a90 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.815+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcafc19d640 con 0x7fcafc101a90 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.816+0000 7fcaf97fa640 1 -- 172.21.15.122:0/1509956325 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec034440 con 0x7fcafc101a90 2024-04-02T13:45:36.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.816+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcafc1a4f30 con 0x7fcafc101a90 2024-04-02T13:45:36.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcaf97fa640 1 -- 172.21.15.122:0/1509956325 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcaec044070 con 0x7fcafc101a90 2024-04-02T13:45:36.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 Processor -- start 2024-04-02T13:45:36.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 -- start start 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b16901090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b168597b0 con 0x556b168f57a0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b167f8570 con 0x556b168f60c0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb05032740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b16855760 con 0x556b168feca0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcaf97fa640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 0x7fcae80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50354/0 (socket says 172.21.15.122:50354) 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb0090f640 1 -- 172.21.15.122:0/2290897032 learned_addr learned my addr 172.21.15.122:0/2290897032 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 0x7fcae80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcaf97fa640 1 -- 172.21.15.122:0/1509956325 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fcaec0f2650 con 0x7fcafc101a90 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b16901090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.818+0000 7fcafbfff640 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 -- 172.21.15.122:0/2290897032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 msgr2=0x556b16901090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b16901090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 -- 172.21.15.122:0/2290897032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 msgr2=0x556b168fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 -- 172.21.15.122:0/2290897032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1656d3f0 con 0x556b168f57a0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4012c50 tx=0x7fcaf4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd2873bc61136b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 0x7fcae80a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4010c40 tx=0x7fcaf4017000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcaf27fc640 1 -- 172.21.15.122:0/2290897032 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4023030 con 0x556b168f57a0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcaf27fc640 1 -- 172.21.15.122:0/2290897032 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf4009c00 con 0x556b168f57a0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.819+0000 7fcaf27fc640 1 -- 172.21.15.122:0/2290897032 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4005020 con 0x556b168f57a0 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- 172.21.15.122:0/2290897032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 msgr2=0x556b168f5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4012c50 tx=0x7fcaf4010740 comp rx=0 tx=0).stop 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- 172.21.15.122:0/2290897032 shutdown_connections 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b16901090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- 172.21.15.122:0/2290897032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168f5b80 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- 172.21.15.122:0/2290897032 >> 172.21.15.122:0/2290897032 conn(0x556b168efed0 msgr2=0x556b168f02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- 172.21.15.122:0/2290897032 shutdown_connections 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- 172.21.15.122:0/2290897032 wait complete. 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- start start 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 Processor -- start 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b168fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b16901a30 con 0x556b168f57a0 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b16814e80 con 0x556b168f60c0 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.820+0000 7fcb05032740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b16901bb0 con 0x556b168feca0 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb05032740 10 client.0 _ll_register_callbacks cb 0x556b16850aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50370/0 (socket says 172.21.15.122:50370) 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 -- 172.21.15.122:0/3582747484 learned_addr learned my addr 172.21.15.122:0/3582747484 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 -- 172.21.15.122:0/3582747484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 msgr2=0x556b168fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb0090f640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b168fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b168fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 -- 172.21.15.122:0/3582747484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 msgr2=0x556b168fbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 -- 172.21.15.122:0/3582747484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b166377d0 con 0x556b168f57a0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb0090f640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b168fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcafbfff640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009900 tx=0x7fcaec0be1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e037470f196e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0f2040 con 0x556b168f57a0 2024-04-02T13:45:36.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fcaec0c6af0 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b16866cf0 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcaec04b040 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb05032740 10 client.18651 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556b16901a30 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.821+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556b16866cf0 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcaec0dd120 con 0x556b168f57a0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 10 client.18651 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 10 client.18651 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 20 client.18651 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b1690f020 0x556b16911450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 4 client.18651 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b16861040 con 0x556b1690f020 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b1690f020 0x556b16911450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b1690f020 0x556b16911450 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31a77e3990029f5a server_cookie=c27ba9c7ffa3c3cb in_seq=0 out_seq=0 2024-04-02T13:45:36.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.822+0000 7fcadb7fe640 10 client.18651 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcadb7fe640 1 -- 172.21.15.122:0/3582747484 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b16861040 con 0x556b1690f020 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcadb7fe640 10 client.18651 handle_command_reply: tid=0 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 client.18651 shutdown 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 msgr2=0x556b168fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009900 tx=0x7fcaec0be1a0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 shutdown_connections 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b1690f020 0x556b16911450 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168feca0 0x556b168fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f60c0 0x556b168fbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 --2- 172.21.15.122:0/3582747484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168f57a0 0x556b168fe730 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcadb7fe640 0 client.18651 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:36.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 >> 172.21.15.122:0/3582747484 conn(0x556b168efed0 msgr2=0x556b168f1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 shutdown_connections 2024-04-02T13:45:36.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 1 -- 172.21.15.122:0/3582747484 wait complete. 2024-04-02T13:45:36.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.823+0000 7fcb05032740 20 client.18651 trim_cache size 0 max 16384 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 Processor -- start 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 -- start start 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b1690a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b168f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1aefa0 con 0x556b168feca0 2024-04-02T13:45:36.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1b0ae0 con 0x556b168f57a0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.836+0000 7fcb05032740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1af670 con 0x556b168f60c0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50372/0 (socket says 172.21.15.122:50372) 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 -- 172.21.15.122:0/1150278873 learned_addr learned my addr 172.21.15.122:0/1150278873 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcafbfff640 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b1690a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 -- 172.21.15.122:0/1150278873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 msgr2=0x556b168f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b168f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 -- 172.21.15.122:0/1150278873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 msgr2=0x556b1690a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b1690a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 -- 172.21.15.122:0/1150278873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b168fc520 con 0x556b168feca0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcb0090f640 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009b60 tx=0x7fcaf40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634310ce02fdf39c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcaf1ffb640 1 -- 172.21.15.122:0/1150278873 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf402c070 con 0x556b168feca0 2024-04-02T13:45:36.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcaf1ffb640 1 -- 172.21.15.122:0/1150278873 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf4012730 con 0x556b168feca0 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.837+0000 7fcaf1ffb640 1 -- 172.21.15.122:0/1150278873 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf40054f0 con 0x556b168feca0 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- 172.21.15.122:0/1150278873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 msgr2=0x556b168fe1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009b60 tx=0x7fcaf40093c0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- 172.21.15.122:0/1150278873 shutdown_connections 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b168fe1f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b168f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- 172.21.15.122:0/1150278873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b1690a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- 172.21.15.122:0/1150278873 >> 172.21.15.122:0/1150278873 conn(0x556b168efed0 msgr2=0x556b168fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- 172.21.15.122:0/1150278873 shutdown_connections 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- 172.21.15.122:0/1150278873 wait complete. 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- start start 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 Processor -- start 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b16902920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b16902e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1b0410 con 0x556b168feca0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1afd40 con 0x556b168f57a0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.838+0000 7fcb05032740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcafc1acfa0 con 0x556b168f60c0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb05032740 10 client.0 _ll_register_callbacks cb 0x7fcaec009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50388/0 (socket says 172.21.15.122:50388) 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 -- 172.21.15.122:0/4013282297 learned_addr learned my addr 172.21.15.122:0/4013282297 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcafbfff640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b16902920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcafb7fe640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b16902e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 -- 172.21.15.122:0/4013282297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 msgr2=0x556b16902e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b16902e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 -- 172.21.15.122:0/4013282297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 msgr2=0x556b16902920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b16902920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 -- 172.21.15.122:0/4013282297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b16974630 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb0090f640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4012af0 tx=0x7fcaf400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f155ac5899f8e4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcadb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf402c040 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7fcaf4004eb0 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf4012730 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fcaf4005290 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb05032740 10 client.18657 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556b16975910 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.839+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf4012730 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fcaf401f810 con 0x556b168feca0 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 10 client.18657 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 10 client.18657 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 20 client.18657 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b16975de0 0x556b169781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 4 client.18657 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b167de6b0 con 0x556b16975de0 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcafbfff640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b16975de0 0x556b169781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcadb7fe640 10 client.18657 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcafbfff640 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b16975de0 0x556b169781d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a86978b12742301a server_cookie=450801768d115d29 in_seq=0 out_seq=0 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcadb7fe640 1 -- 172.21.15.122:0/4013282297 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b167de6b0 con 0x556b16975de0 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcadb7fe640 10 client.18657 handle_command_reply: tid=0 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 1 client.18657 shutdown 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 msgr2=0x556b169033a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.840+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4012af0 tx=0x7fcaf400cd50 comp rx=0 tx=0).stop 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 shutdown_connections 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x556b16975de0 0x556b169781d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b168feca0 0x556b169033a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcadb7fe640 0 client.18657 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:45:36.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b168f60c0 0x556b16902e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 --2- 172.21.15.122:0/4013282297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b168f57a0 0x556b16902920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 >> 172.21.15.122:0/4013282297 conn(0x556b168efed0 msgr2=0x556b168ee6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 shutdown_connections 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 1 -- 172.21.15.122:0/4013282297 wait complete. 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb05032740 20 client.18657 trim_cache size 0 max 16384 2024-04-02T13:45:36.851 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:36.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 msgr2=0x7fcae80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 0x7fcae80a61e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4010c40 tx=0x7fcaf4017000 comp rx=0 tx=0).stop 2024-04-02T13:45:36.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 msgr2=0x7fcafc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009230 tx=0x7fcaec02cc70 comp rx=0 tx=0).stop 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 shutdown_connections 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcae80a3d20 0x7fcae80a61e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc107aa0 0x7fcafc19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcafc102450 0x7fcafc198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 --2- 172.21.15.122:0/1509956325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcafc101a90 0x7fcafc1983c0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 >> 172.21.15.122:0/1509956325 conn(0x7fcafc078f00 msgr2=0x7fcafc1045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 shutdown_connections 2024-04-02T13:45:36.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.841+0000 7fcb02372640 1 -- 172.21.15.122:0/1509956325 wait complete. 2024-04-02T13:45:36.855 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:45:36.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 -- 172.21.15.122:0/1427115551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08102460 msgr2=0x7f9b0810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1427115551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08102460 0x7f9b0810e970 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009cd0 tx=0x7f9b0002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 -- 172.21.15.122:0/1427115551 shutdown_connections 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1427115551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08102460 0x7f9b0810e970 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1427115551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08101aa0 0x7f9b08101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1427115551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08107ab0 0x7f9b08107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 -- 172.21.15.122:0/1427115551 >> 172.21.15.122:0/1427115551 conn(0x7f9b080fd530 msgr2=0x7f9b080ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 -- 172.21.15.122:0/1427115551 shutdown_connections 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 -- 172.21.15.122:0/1427115551 wait complete. 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.955+0000 7f9b0d086640 1 Processor -- start 2024-04-02T13:45:36.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 -- start start 2024-04-02T13:45:36.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08102460 0x7f9b08198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08107ab0 0x7f9b0819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08111e90 con 0x7f9b08101aa0 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08111d10 con 0x7f9b08107ab0 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0d086640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b08112010 con 0x7f9b08102460 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50400/0 (socket says 172.21.15.122:50400) 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 -- 172.21.15.122:0/1251354047 learned_addr learned my addr 172.21.15.122:0/1251354047 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0759e640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08107ab0 0x7f9b0819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b0659c640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08102460 0x7f9b08198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 -- 172.21.15.122:0/1251354047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08102460 msgr2=0x7f9b08198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08102460 0x7f9b08198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 -- 172.21.15.122:0/1251354047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08107ab0 msgr2=0x7f9b0819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08107ab0 0x7f9b0819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 -- 172.21.15.122:0/1251354047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0819d300 con 0x7f9b08101aa0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c890 tx=0x7f9afc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.956+0000 7f9aeffff640 1 -- 172.21.15.122:0/1251354047 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc004220 con 0x7f9b08101aa0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.957+0000 7f9aeffff640 1 -- 172.21.15.122:0/1251354047 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc0043c0 con 0x7f9b08101aa0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.957+0000 7f9aeffff640 1 -- 172.21.15.122:0/1251354047 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0049e0 con 0x7f9b08101aa0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.957+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0819d5f0 con 0x7f9b08101aa0 2024-04-02T13:45:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.957+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b081a4e80 con 0x7f9b08101aa0 2024-04-02T13:45:36.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9aeffff640 1 -- 172.21.15.122:0/1251354047 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9afc0079b0 con 0x7f9b08101aa0 2024-04-02T13:45:36.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 Processor -- start 2024-04-02T13:45:36.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 -- start start 2024-04-02T13:45:36.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x55832476cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324778090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583246d07b0 con 0x55832476d0c0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55832466f570 con 0x55832476c7a0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5583246cc760 con 0x558324775ca0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9aeffff640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 0x7f9af00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50416/0 (socket says 172.21.15.122:50416) 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b06d9d640 1 -- 172.21.15.122:0/112421814 learned_addr learned my addr 172.21.15.122:0/112421814 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0759e640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 0x7f9af00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0659c640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324778090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.959+0000 7f9b0759e640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x55832476cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9aeffff640 1 -- 172.21.15.122:0/1251354047 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9afc0d0a90 con 0x7f9b08101aa0 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 -- 172.21.15.122:0/112421814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 msgr2=0x558324778090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324778090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 -- 172.21.15.122:0/112421814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 msgr2=0x55832476cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x55832476cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 -- 172.21.15.122:0/112421814 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583243e43f0 con 0x55832476d0c0 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b0759e640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x55832476cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0cdd60 tx=0x7f9afc0054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ea6efe389609d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9aed7fa640 1 -- 172.21.15.122:0/112421814 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc01d070 con 0x55832476d0c0 2024-04-02T13:45:36.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9b0759e640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 0x7f9af00a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b00005cc0 tx=0x7f9b0002e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9aed7fa640 1 -- 172.21.15.122:0/112421814 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc0870e0 con 0x55832476d0c0 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.960+0000 7f9aed7fa640 1 -- 172.21.15.122:0/112421814 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc098d60 con 0x55832476d0c0 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/112421814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 msgr2=0x558324775760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0cdd60 tx=0x7f9afc0054a0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/112421814 shutdown_connections 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324778090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324775760 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/112421814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x55832476cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/112421814 >> 172.21.15.122:0/112421814 conn(0x558324766ed0 msgr2=0x5583247672b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/112421814 shutdown_connections 2024-04-02T13:45:36.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/112421814 wait complete. 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- start start 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 Processor -- start 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 0x558324772a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324773060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558324778a90 con 0x55832476c7a0 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55832468be80 con 0x55832476d0c0 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558324778c10 con 0x558324775ca0 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b0fc6e740 10 client.0 _ll_register_callbacks cb 0x5583246c7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b06d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50432/0 (socket says 172.21.15.122:50432) 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.961+0000 7f9b06d9d640 1 -- 172.21.15.122:0/2504719505 learned_addr learned my addr 172.21.15.122:0/2504719505 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0659c640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 0x558324772a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0759e640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324773060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 -- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 msgr2=0x558324773060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324773060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 -- 172.21.15.122:0/2504719505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 msgr2=0x558324772a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 0x558324772a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 -- 172.21.15.122:0/2504719505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583244ae7d0 con 0x55832476c7a0 2024-04-02T13:45:36.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0659c640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 0x558324772a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0759e640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324773060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0990c0 tx=0x7f9afc098f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada4a2f670fdfe0f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc01d040 con 0x55832476c7a0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9afc09cb50 con 0x55832476c7a0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9afc0870e0 con 0x55832476c7a0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9afc0ba050 con 0x55832476c7a0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 10 client.18669 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558324784ac0 con 0x55832476c7a0 2024-04-02T13:45:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9afc0870e0 con 0x55832476c7a0 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9afc09bb40 con 0x55832476c7a0 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 10 client.18669 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 10 client.18669 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 20 client.18669 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558324785e10 0x558324788200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.962+0000 7f9b0fc6e740 4 client.18669 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.963+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583246ddcf0 con 0x558324785e10 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.963+0000 7f9b0659c640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558324785e10 0x558324788200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.963+0000 7f9aeaff5640 10 client.18669 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.963+0000 7f9b0659c640 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558324785e10 0x558324788200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ce6c91a33a07986 server_cookie=843e89a8ba875766 in_seq=0 out_seq=0 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9aeaff5640 1 -- 172.21.15.122:0/2504719505 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583246ddcf0 con 0x558324785e10 2024-04-02T13:45:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9aeaff5640 10 client.18669 handle_command_reply: tid=0 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 client.18669 shutdown 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 msgr2=0x5583247756c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0990c0 tx=0x7f9afc098f90 comp rx=0 tx=0).stop 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 shutdown_connections 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324773060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476d0c0 0x558324772a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558324785e10 0x558324788200 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/2504719505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476c7a0 0x5583247756c0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9aeaff5640 0 client.18669 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 >> 172.21.15.122:0/2504719505 conn(0x558324766ed0 msgr2=0x558324768c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 shutdown_connections 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/2504719505 wait complete. 2024-04-02T13:45:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.964+0000 7f9b0fc6e740 20 client.18669 trim_cache size 0 max 16384 2024-04-02T13:45:36.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 369550 3.48 MB/sec execute 417 sec latency 1565.257 ms 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 Processor -- start 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 -- start start 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x558324772a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x5583247672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081aeff0 con 0x55832476d0c0 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081b0b30 con 0x55832476c7a0 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.977+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081af6c0 con 0x558324775ca0 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50440/0 (socket says 172.21.15.122:50440) 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 -- 172.21.15.122:0/155689335 learned_addr learned my addr 172.21.15.122:0/155689335 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0759e640 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x5583247672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 -- 172.21.15.122:0/155689335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 msgr2=0x5583247672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x5583247672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 -- 172.21.15.122:0/155689335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 msgr2=0x558324772a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x558324772a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x558324772a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 -- 172.21.15.122:0/155689335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583247733d0 con 0x55832476d0c0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9b0659c640 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x558324768060 tx=0x7f9af800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9996fcef291614d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9aecff9640 1 -- 172.21.15.122:0/155689335 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8005020 con 0x55832476d0c0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9aecff9640 1 -- 172.21.15.122:0/155689335 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8023030 con 0x55832476d0c0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.978+0000 7f9aecff9640 1 -- 172.21.15.122:0/155689335 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af801f110 con 0x55832476d0c0 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/155689335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 msgr2=0x558324771bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x558324768060 tx=0x7f9af800b390 comp rx=0 tx=0).stop 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/155689335 shutdown_connections 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x5583247672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x558324771bb0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/155689335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x558324772a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/155689335 >> 172.21.15.122:0/155689335 conn(0x558324766ed0 msgr2=0x5583247722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/155689335 shutdown_connections 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/155689335 wait complete. 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- start start 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 Processor -- start 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x5583247e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324786620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081b0460 con 0x55832476d0c0 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081afd90 con 0x55832476c7a0 2024-04-02T13:45:36.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b081acff0 con 0x558324775ca0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0fc6e740 10 client.0 _ll_register_callbacks cb 0x7f9af800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50442/0 (socket says 172.21.15.122:50442) 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0659c640 1 -- 172.21.15.122:0/766606250 learned_addr learned my addr 172.21.15.122:0/766606250 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x5583247e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.979+0000 7f9b0759e640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324786620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 -- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 msgr2=0x558324786620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324786620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 -- 172.21.15.122:0/766606250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 msgr2=0x5583247e4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x5583247e4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 -- 172.21.15.122:0/766606250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832476cef0 con 0x55832476d0c0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x5583247e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0659c640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005890 tx=0x7f9af8004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256ca2b968af720 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af802c070 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9af8004930 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af8023030 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9af8038050 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 10 client.18675 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5583247eb940 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af8023030 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9af801f560 con 0x55832476d0c0 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 10 client.18675 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 10 client.18675 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 20 client.18675 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5583247ebfb0 0x5583247ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 4 client.18675 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.980+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583246d8040 con 0x5583247ebfb0 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5583247ebfb0 0x5583247ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9aeaff5640 10 client.18675 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b06d9d640 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5583247ebfb0 0x5583247ee3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2549a6f777bf6c6 server_cookie=6b62ee07aa685b10 in_seq=0 out_seq=0 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9aeaff5640 1 -- 172.21.15.122:0/766606250 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583246d8040 con 0x5583247ebfb0 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9aeaff5640 10 client.18675 handle_command_reply: tid=0 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 client.18675 shutdown 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 msgr2=0x5583247e4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005890 tx=0x7f9af8004ef0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 shutdown_connections 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558324775ca0 0x558324786620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5583247ebfb0 0x5583247ee3d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55832476d0c0 0x5583247e4860 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9aeaff5640 0 client.18675 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:45:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 --2- 172.21.15.122:0/766606250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55832476c7a0 0x5583247e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.981+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 >> 172.21.15.122:0/766606250 conn(0x558324766ed0 msgr2=0x558324771320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 shutdown_connections 2024-04-02T13:45:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0fc6e740 1 -- 172.21.15.122:0/766606250 wait complete. 2024-04-02T13:45:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0fc6e740 20 client.18675 trim_cache size 0 max 16384 2024-04-02T13:45:36.992 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 msgr2=0x7f9af00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 0x7f9af00a6260 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9b00005cc0 tx=0x7f9b0002e040 comp rx=0 tx=0).stop 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 msgr2=0x7f9b08198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00c890 tx=0x7f9afc00cd50 comp rx=0 tx=0).stop 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 shutdown_connections 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9af00a3da0 0x7f9af00a6260 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9b08107ab0 0x7f9b0819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b08102460 0x7f9b08198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 --2- 172.21.15.122:0/1251354047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b08101aa0 0x7f9b08198310 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 >> 172.21.15.122:0/1251354047 conn(0x7f9b080fd530 msgr2=0x7f9b080fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 shutdown_connections 2024-04-02T13:45:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:36.982+0000 7f9b0d086640 1 -- 172.21.15.122:0/1251354047 wait complete. 2024-04-02T13:45:36.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:45:37.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- 172.21.15.122:0/3740595058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc103660 msgr2=0x7f32fc10bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 --2- 172.21.15.122:0/3740595058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc103660 0x7f32fc10bb40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f32ec009900 tx=0x7f32ec02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- 172.21.15.122:0/3740595058 shutdown_connections 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 --2- 172.21.15.122:0/3740595058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc10c110 0x7f32fc10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 --2- 172.21.15.122:0/3740595058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc103660 0x7f32fc10bb40 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 --2- 172.21.15.122:0/3740595058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc102d40 0x7f32fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- 172.21.15.122:0/3740595058 >> 172.21.15.122:0/3740595058 conn(0x7f32fc076d80 msgr2=0x7f32fc077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- 172.21.15.122:0/3740595058 shutdown_connections 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- 172.21.15.122:0/3740595058 wait complete. 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 Processor -- start 2024-04-02T13:45:37.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.100+0000 7f3302311640 1 -- start start 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc102d40 0x7f32fc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc103660 0x7f32fc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc111ea0 con 0x7f32fc10c110 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc111d20 con 0x7f32fc102d40 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f3302311640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc112020 con 0x7f32fc103660 2024-04-02T13:45:37.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50460/0 (socket says 172.21.15.122:50460) 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 -- 172.21.15.122:0/2005696341 learned_addr learned my addr 172.21.15.122:0/2005696341 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc102d40 0x7f32fc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc103660 0x7f32fc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 -- 172.21.15.122:0/2005696341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc103660 msgr2=0x7f32fc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc103660 0x7f32fc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 -- 172.21.15.122:0/2005696341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc102d40 msgr2=0x7f32fc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc102d40 0x7f32fc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 -- 172.21.15.122:0/2005696341 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32fc19d390 con 0x7f32fc10c110 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f33008ae640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f32f400e9d0 tx=0x7f32f400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.101+0000 7f32f97fa640 1 -- 172.21.15.122:0/2005696341 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f400cd80 con 0x7f32fc10c110 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.102+0000 7f32f97fa640 1 -- 172.21.15.122:0/2005696341 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f4005020 con 0x7f32fc10c110 2024-04-02T13:45:37.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.102+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc19d680 con 0x7f32fc10c110 2024-04-02T13:45:37.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.102+0000 7f32f97fa640 1 -- 172.21.15.122:0/2005696341 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f40116d0 con 0x7f32fc10c110 2024-04-02T13:45:37.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.102+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc1a4f10 con 0x7f32fc10c110 2024-04-02T13:45:37.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f32f97fa640 1 -- 172.21.15.122:0/2005696341 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f32f40051d0 con 0x7f32fc10c110 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 Processor -- start 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 -- start start 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5640e98497b0 con 0x5640e98ea4d0 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5640e97e8570 con 0x5640e98f30b0 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.104+0000 7f3304fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640e9845760 con 0x5640e98e9bb0 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50472/0 (socket says 172.21.15.122:50472) 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 -- 172.21.15.122:0/3200387518 learned_addr learned my addr 172.21.15.122:0/3200387518 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32f97fa640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 0x7f32e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f33008ae640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 0x7f32e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 -- 172.21.15.122:0/3200387518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 msgr2=0x5640e98e9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98e9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 -- 172.21.15.122:0/3200387518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 msgr2=0x5640e98f54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 -- 172.21.15.122:0/3200387518 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e955d3f0 con 0x5640e98ea4d0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32f97fa640 1 -- 172.21.15.122:0/2005696341 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f32f40d1140 con 0x7f32fc10c110 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32fbfff640 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b760 tx=0x7f32f000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aa32465aef00f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32e27fc640 1 -- 172.21.15.122:0/3200387518 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0003dd0 con 0x5640e98ea4d0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32e27fc640 1 -- 172.21.15.122:0/3200387518 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f0015070 con 0x5640e98ea4d0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.105+0000 7f32e27fc640 1 -- 172.21.15.122:0/3200387518 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0011440 con 0x5640e98ea4d0 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- 172.21.15.122:0/3200387518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 msgr2=0x5640e98f2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f33008ae640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 0x7f32e40a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32f4099150 tx=0x7f32f4067840 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b760 tx=0x7f32f000bc20 comp rx=0 tx=0).stop 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- 172.21.15.122:0/3200387518 shutdown_connections 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f2b70 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 --2- 172.21.15.122:0/3200387518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98e9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- 172.21.15.122:0/3200387518 >> 172.21.15.122:0/3200387518 conn(0x5640e98e42e0 msgr2=0x5640e98e46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- 172.21.15.122:0/3200387518 shutdown_connections 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- 172.21.15.122:0/3200387518 wait complete. 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 -- start start 2024-04-02T13:45:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.106+0000 7f3304fcf740 1 Processor -- start 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98ea4d0 0x5640e98f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5640e98f5ea0 con 0x5640e98e9bb0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5640e9804e80 con 0x5640e98f30b0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5640e98f6020 con 0x5640e98ea4d0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 10 client.0 _ll_register_callbacks cb 0x5640e9840aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50486/0 (socket says 172.21.15.122:50486) 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 -- 172.21.15.122:0/191068494 learned_addr learned my addr 172.21.15.122:0/191068494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 -- 172.21.15.122:0/191068494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98ea4d0 msgr2=0x5640e98f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98ea4d0 0x5640e98f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 -- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 msgr2=0x5640e98f09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 -- 172.21.15.122:0/191068494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e96277d0 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f32ec005c40 tx=0x7f32ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b1119b8f324e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0035f0 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f32ec003790 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f0015070 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f32ec052050 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 10 client.18687 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.107+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5640e9901f40 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f0015070 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f32ec034a50 con 0x5640e98e9bb0 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 10 client.18687 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 10 client.18687 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:45:37.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 20 client.18687 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e9903400 0x5640e9905880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 4 client.18687 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640e9856cf0 con 0x5640e9903400 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.108+0000 7f32fbfff640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e9903400 0x5640e9905880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f32db7fe640 10 client.18687 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f32fbfff640 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e9903400 0x5640e9905880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db0ac1d9d2520830 server_cookie=ecbcc3fd06dee31a in_seq=0 out_seq=0 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f32db7fe640 1 -- 172.21.15.122:0/191068494 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640e9856cf0 con 0x5640e9903400 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f32db7fe640 10 client.18687 handle_command_reply: tid=0 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f3304fcf740 1 client.18687 shutdown 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 msgr2=0x5640e98eff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.109+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f32ec005c40 tx=0x7f32ec009260 comp rx=0 tx=0).stop 2024-04-02T13:45:37.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 shutdown_connections 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e9903400 0x5640e9905880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98f30b0 0x5640e98f09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98ea4d0 0x5640e98f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f32db7fe640 0 client.18687 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 --2- 172.21.15.122:0/191068494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98e9bb0 0x5640e98eff70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 >> 172.21.15.122:0/191068494 conn(0x5640e98e42e0 msgr2=0x5640e98e6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 shutdown_connections 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 1 -- 172.21.15.122:0/191068494 wait complete. 2024-04-02T13:45:37.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.110+0000 7f3304fcf740 20 client.18687 trim_cache size 0 max 16384 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 Processor -- start 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 -- start start 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 0x5640e98fed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1acef0 con 0x5640e98f30b0 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1aea30 con 0x5640e98ea4d0 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f3304fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1ad5c0 con 0x5640e98e9bb0 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50494/0 (socket says 172.21.15.122:50494) 2024-04-02T13:45:37.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 -- 172.21.15.122:0/2957538661 learned_addr learned my addr 172.21.15.122:0/2957538661 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 0x5640e98fed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 -- 172.21.15.122:0/2957538661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 msgr2=0x5640e98f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 -- 172.21.15.122:0/2957538661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 msgr2=0x5640e98fed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 0x5640e98fed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 -- 172.21.15.122:0/2957538661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e98da310 con 0x5640e98f30b0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 0x5640e98fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f32ec001eb0 tx=0x7f32ec02cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22883449ae5b96b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.123+0000 7f32e1ffb640 1 -- 172.21.15.122:0/2957538661 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec02c670 con 0x5640e98f30b0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f32e1ffb640 1 -- 172.21.15.122:0/2957538661 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec047020 con 0x5640e98f30b0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f32e1ffb640 1 -- 172.21.15.122:0/2957538661 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ec034440 con 0x5640e98f30b0 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- 172.21.15.122:0/2957538661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 msgr2=0x5640e990b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f32ec001eb0 tx=0x7f32ec02cf90 comp rx=0 tx=0).stop 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- 172.21.15.122:0/2957538661 shutdown_connections 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98f30b0 0x5640e990b150 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98ea4d0 0x5640e98fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2957538661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98e9bb0 0x5640e98f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- 172.21.15.122:0/2957538661 >> 172.21.15.122:0/2957538661 conn(0x5640e98e42e0 msgr2=0x5640e98def30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- 172.21.15.122:0/2957538661 shutdown_connections 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- 172.21.15.122:0/2957538661 wait complete. 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.124+0000 7f3304fcf740 1 -- start start 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 Processor -- start 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 0x5640e98f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98f30b0 0x5640e98f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1ae360 con 0x5640e98ea4d0 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1adc90 con 0x5640e98e9bb0 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1aaef0 con 0x5640e98f30b0 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f3304fcf740 10 client.0 _ll_register_callbacks cb 0x7f32ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50500/0 (socket says 172.21.15.122:50500) 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 -- 172.21.15.122:0/2776951771 learned_addr learned my addr 172.21.15.122:0/2776951771 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98f30b0 0x5640e98f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 0x5640e98f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 -- 172.21.15.122:0/2776951771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98f30b0 msgr2=0x5640e98f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98f30b0 0x5640e98f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 -- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 msgr2=0x5640e98f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 0x5640e98f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 -- 172.21.15.122:0/2776951771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e9968bc0 con 0x5640e98ea4d0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 0x5640e98f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32fbfff640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f32f00039c0 tx=0x7f32f00049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ab5eda2314a46b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.125+0000 7f32db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0003dd0 con 0x5640e98ea4d0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f32f0025430 con 0x5640e98ea4d0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32ec047020 con 0x5640e98ea4d0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f32f0033050 con 0x5640e98ea4d0 2024-04-02T13:45:37.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 10 client.18693 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5640e9969e10 con 0x5640e98ea4d0 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ec047020 con 0x5640e98ea4d0 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f32f0011920 con 0x5640e98ea4d0 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 10 client.18693 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 10 client.18693 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 20 client.18693 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e996a360 0x5640e996c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 4 client.18693 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640e9851040 con 0x5640e996a360 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.126+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e996a360 0x5640e996c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f32db7fe640 10 client.18693 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f32fb7fe640 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e996a360 0x5640e996c780 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7155eb813a5ad2c9 server_cookie=5e9cb08587de8d in_seq=0 out_seq=0 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f32db7fe640 1 -- 172.21.15.122:0/2776951771 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640e9851040 con 0x5640e996a360 2024-04-02T13:45:37.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f32db7fe640 10 client.18693 handle_command_reply: tid=0 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 client.18693 shutdown 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 msgr2=0x5640e98f73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f32f00039c0 tx=0x7f32f00049b0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 shutdown_connections 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5640e98f30b0 0x5640e98f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5640e98ea4d0 0x5640e98f73c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5640e996a360 0x5640e996c780 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 --2- 172.21.15.122:0/2776951771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5640e98e9bb0 0x5640e98f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f32db7fe640 0 client.18693 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 >> 172.21.15.122:0/2776951771 conn(0x5640e98e42e0 msgr2=0x5640e98def30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 shutdown_connections 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 1 -- 172.21.15.122:0/2776951771 wait complete. 2024-04-02T13:45:37.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.127+0000 7f3304fcf740 20 client.18693 trim_cache size 0 max 16384 2024-04-02T13:45:37.139 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:37.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 msgr2=0x7f32e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 0x7f32e40a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f32f4099150 tx=0x7f32f4067840 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 msgr2=0x7f32fc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f32f400e9d0 tx=0x7f32f400ee90 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 shutdown_connections 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f32e40a3d20 0x7f32e40a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32fc10c110 0x7f32fc19cc60 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32fc103660 0x7f32fc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 --2- 172.21.15.122:0/2005696341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f32fc102d40 0x7f32fc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 >> 172.21.15.122:0/2005696341 conn(0x7f32fc076d80 msgr2=0x7f32fc0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 shutdown_connections 2024-04-02T13:45:37.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.128+0000 7f3302311640 1 -- 172.21.15.122:0/2005696341 wait complete. 2024-04-02T13:45:37.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:45:37.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 -- 172.21.15.122:0/750394133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc100a20 msgr2=0x7f9edc100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 --2- 172.21.15.122:0/750394133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc100a20 0x7f9edc100ea0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009930 tx=0x7f9ec402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 -- 172.21.15.122:0/750394133 shutdown_connections 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 --2- 172.21.15.122:0/750394133 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc1013e0 0x7f9edc10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 --2- 172.21.15.122:0/750394133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc100a20 0x7f9edc100ea0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 --2- 172.21.15.122:0/750394133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc106a30 0x7f9edc106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 -- 172.21.15.122:0/750394133 >> 172.21.15.122:0/750394133 conn(0x7f9edc077b60 msgr2=0x7f9edc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 -- 172.21.15.122:0/750394133 shutdown_connections 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.248+0000 7f9ee1960640 1 -- 172.21.15.122:0/750394133 wait complete. 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 Processor -- start 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 -- start start 2024-04-02T13:45:37.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc100a20 0x7f9edc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc1013e0 0x7f9edc197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc110e40 con 0x7f9edc106a30 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc110cc0 con 0x7f9edc1013e0 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ee1960640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc110fc0 con 0x7f9edc100a20 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50516/0 (socket says 172.21.15.122:50516) 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 -- 172.21.15.122:0/2421324260 learned_addr learned my addr 172.21.15.122:0/2421324260 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc1013e0 0x7f9edc197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edaffd640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc100a20 0x7f9edc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 -- 172.21.15.122:0/2421324260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc100a20 msgr2=0x7f9edc1972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc100a20 0x7f9edc1972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 -- 172.21.15.122:0/2421324260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc1013e0 msgr2=0x7f9edc197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc1013e0 0x7f9edc197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 -- 172.21.15.122:0/2421324260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9edc19c2b0 con 0x7f9edc106a30 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.249+0000 7f9edb7fe640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400b970 tx=0x7f9ed400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.250+0000 7f9ee095e640 1 -- 172.21.15.122:0/2421324260 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4019070 con 0x7f9edc106a30 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.250+0000 7f9ee095e640 1 -- 172.21.15.122:0/2421324260 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ed400cb40 con 0x7f9edc106a30 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.250+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9edc19c5a0 con 0x7f9edc106a30 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.250+0000 7f9ee095e640 1 -- 172.21.15.122:0/2421324260 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4007650 con 0x7f9edc106a30 2024-04-02T13:45:37.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.250+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9edc1a3e30 con 0x7f9edc106a30 2024-04-02T13:45:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee095e640 1 -- 172.21.15.122:0/2421324260 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9ed401c070 con 0x7f9edc106a30 2024-04-02T13:45:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 Processor -- start 2024-04-02T13:45:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 -- start start 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d49b0 0x5654371d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee095e640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 0x7f9ec80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5654371367b0 con 0x5654371ddeb0 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5654370d5570 con 0x5654371d52d0 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.252+0000 7f9ee4629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565437132760 con 0x5654371d49b0 2024-04-02T13:45:37.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50524/0 (socket says 172.21.15.122:50524) 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 -- 172.21.15.122:0/2469698859 learned_addr learned my addr 172.21.15.122:0/2469698859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ee095e640 1 -- 172.21.15.122:0/2421324260 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9ed40d00f0 con 0x7f9edc106a30 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9edaffd640 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9edb7fe640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 0x7f9ec80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d49b0 0x5654371d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 -- 172.21.15.122:0/2469698859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d49b0 msgr2=0x5654371d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d49b0 0x5654371d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 -- 172.21.15.122:0/2469698859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 msgr2=0x5654371dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 -- 172.21.15.122:0/2469698859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565436e4a3f0 con 0x5654371ddeb0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4036040 tx=0x7f9ec40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2593d6fc03bfc2b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9edb7fe640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 0x7f9ec80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000f80 tx=0x7f9ed400e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed1ffb640 1 -- 172.21.15.122:0/2469698859 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4048070 con 0x5654371ddeb0 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed1ffb640 1 -- 172.21.15.122:0/2469698859 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec40046c0 con 0x5654371ddeb0 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.253+0000 7f9ed1ffb640 1 -- 172.21.15.122:0/2469698859 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4005250 con 0x5654371ddeb0 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- 172.21.15.122:0/2469698859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 msgr2=0x5654371e02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4036040 tx=0x7f9ec40058b0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- 172.21.15.122:0/2469698859 shutdown_connections 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371e02a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 --2- 172.21.15.122:0/2469698859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d49b0 0x5654371d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- 172.21.15.122:0/2469698859 >> 172.21.15.122:0/2469698859 conn(0x5654371cf0e0 msgr2=0x5654371cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- 172.21.15.122:0/2469698859 shutdown_connections 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- 172.21.15.122:0/2469698859 wait complete. 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 -- start start 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.254+0000 7f9ee4629740 1 Processor -- start 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371ddeb0 0x5654371db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5654371e0c40 con 0x5654371d49b0 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5654370f1e80 con 0x5654371d52d0 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5654371e0dc0 con 0x5654371ddeb0 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 10 client.0 _ll_register_callbacks cb 0x56543712daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50528/0 (socket says 172.21.15.122:50528) 2024-04-02T13:45:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 -- 172.21.15.122:0/1648714242 learned_addr learned my addr 172.21.15.122:0/1648714242 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9edaffd640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 -- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371ddeb0 msgr2=0x5654371db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371ddeb0 0x5654371db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 -- 172.21.15.122:0/1648714242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 msgr2=0x5654371dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 -- 172.21.15.122:0/1648714242 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565436f147d0 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005ec0 tx=0x7f9ec402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febbc1d036778f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4053060 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9ec40051f0 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.255+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec40046c0 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ec405a050 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 10 client.18705 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5654371eccc0 con 0x5654371d49b0 2024-04-02T13:45:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec40046c0 con 0x5654371d49b0 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9ec4034440 con 0x5654371d49b0 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 10 client.18705 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 10 client.18705 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 20 client.18705 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654371ee010 0x5654371f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 4 client.18705 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565437143cf0 con 0x5654371ee010 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.256+0000 7f9edaffd640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654371ee010 0x5654371f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.257+0000 7f9ebaffd640 10 client.18705 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.257+0000 7f9edaffd640 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654371ee010 0x5654371f0400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d42f59691073fd7d server_cookie=574dd9d192f88b35 in_seq=0 out_seq=0 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.257+0000 7f9ebaffd640 1 -- 172.21.15.122:0/1648714242 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565437143cf0 con 0x5654371ee010 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.257+0000 7f9ebaffd640 10 client.18705 handle_command_reply: tid=0 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.257+0000 7f9ee4629740 1 client.18705 shutdown 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 msgr2=0x5654371dd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4005ec0 tx=0x7f9ec402bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 shutdown_connections 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654371ee010 0x5654371f0400 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371ddeb0 0x5654371db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d52d0 0x5654371dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ebaffd640 0 client.18705 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 --2- 172.21.15.122:0/1648714242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371dd900 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 >> 172.21.15.122:0/1648714242 conn(0x5654371cf0e0 msgr2=0x5654371d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 shutdown_connections 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 1 -- 172.21.15.122:0/1648714242 wait complete. 2024-04-02T13:45:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.258+0000 7f9ee4629740 20 client.18705 trim_cache size 0 max 16384 2024-04-02T13:45:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 Processor -- start 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 -- start start 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d49b0 0x5654371d9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1ace00 con 0x5654371ddeb0 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1ae940 con 0x5654371d49b0 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ee4629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1ad4d0 con 0x5654371d52d0 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50538/0 (socket says 172.21.15.122:50538) 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 -- 172.21.15.122:0/501441908 learned_addr learned my addr 172.21.15.122:0/501441908 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 -- 172.21.15.122:0/501441908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 msgr2=0x5654371dd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9edaffd640 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371dd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 -- 172.21.15.122:0/501441908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d49b0 msgr2=0x5654371d9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d49b0 0x5654371d9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9ed3fff640 1 -- 172.21.15.122:0/501441908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654371d5100 con 0x5654371ddeb0 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.271+0000 7f9edaffd640 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec4005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108a61c9980ab35d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ed17fa640 1 -- 172.21.15.122:0/501441908 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4053040 con 0x5654371ddeb0 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ed17fa640 1 -- 172.21.15.122:0/501441908 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4044030 con 0x5654371ddeb0 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ed17fa640 1 -- 172.21.15.122:0/501441908 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4048070 con 0x5654371ddeb0 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 -- 172.21.15.122:0/501441908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 msgr2=0x5654371dd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4006010 tx=0x7f9ec4005a60 comp rx=0 tx=0).stop 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 -- 172.21.15.122:0/501441908 shutdown_connections 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371ddeb0 0x5654371dd900 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 --2- 172.21.15.122:0/501441908 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371d49b0 0x5654371d9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 -- 172.21.15.122:0/501441908 >> 172.21.15.122:0/501441908 conn(0x5654371cf0e0 msgr2=0x5654371d9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 -- 172.21.15.122:0/501441908 shutdown_connections 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.272+0000 7f9ee4629740 1 -- 172.21.15.122:0/501441908 wait complete. 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 -- start start 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 Processor -- start 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371ddeb0 0x5654371eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1ae270 con 0x5654371d49b0 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1adba0 con 0x5654371ddeb0 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9edc1aae00 con 0x5654371d52d0 2024-04-02T13:45:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ee4629740 10 client.0 _ll_register_callbacks cb 0x7f9ec4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50542/0 (socket says 172.21.15.122:50542) 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 -- 172.21.15.122:0/4256983989 learned_addr learned my addr 172.21.15.122:0/4256983989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 -- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 msgr2=0x5654371eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 -- 172.21.15.122:0/4256983989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371ddeb0 msgr2=0x5654371eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371ddeb0 0x5654371eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.273+0000 7f9ed3fff640 1 -- 172.21.15.122:0/4256983989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654371ef630 con 0x5654371d49b0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ed3fff640 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40349b0 tx=0x7f9ec4005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a9b2f84da55cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4053030 con 0x5654371d49b0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f9ec4034440 con 0x5654371d49b0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec4044030 con 0x5654371d49b0 2024-04-02T13:45:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9ec4048040 con 0x5654371d49b0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 10 client.18711 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x565437254000 con 0x5654371d49b0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ec4044030 con 0x5654371d49b0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f9ec404e740 con 0x5654371d49b0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 10 client.18711 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 10 client.18711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 20 client.18711 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654372545c0 0x5654372569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 4 client.18711 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56543713e040 con 0x5654372545c0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.274+0000 7f9edaffd640 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654372545c0 0x5654372569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ebaffd640 10 client.18711 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9edaffd640 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654372545c0 0x5654372569e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60129dafcc140e05 server_cookie=d77fe8d1bf20303e in_seq=0 out_seq=0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ebaffd640 1 -- 172.21.15.122:0/4256983989 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56543713e040 con 0x5654372545c0 2024-04-02T13:45:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ebaffd640 10 client.18711 handle_command_reply: tid=0 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 client.18711 shutdown 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 msgr2=0x5654371f36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40349b0 tx=0x7f9ec4005940 comp rx=0 tx=0).stop 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 shutdown_connections 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5654372545c0 0x5654372569e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5654371ddeb0 0x5654371eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ebaffd640 0 client.18711 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5654371d52d0 0x5654371eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 --2- 172.21.15.122:0/4256983989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5654371d49b0 0x5654371f36b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 >> 172.21.15.122:0/4256983989 conn(0x5654371cf0e0 msgr2=0x5654371cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 shutdown_connections 2024-04-02T13:45:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 1 -- 172.21.15.122:0/4256983989 wait complete. 2024-04-02T13:45:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.275+0000 7f9ee4629740 20 client.18711 trim_cache size 0 max 16384 2024-04-02T13:45:37.287 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 msgr2=0x7f9ec80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 0x7f9ec80a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4000f80 tx=0x7f9ed400e040 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 msgr2=0x7f9edc19bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400b970 tx=0x7f9ed400be30 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 shutdown_connections 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ec80a3d20 0x7f9ec80a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9edc106a30 0x7f9edc19bb80 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9edc1013e0 0x7f9edc197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 --2- 172.21.15.122:0/2421324260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9edc100a20 0x7f9edc1972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 >> 172.21.15.122:0/2421324260 conn(0x7f9edc077b60 msgr2=0x7f9edc103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 shutdown_connections 2024-04-02T13:45:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.276+0000 7f9ee1960640 1 -- 172.21.15.122:0/2421324260 wait complete. 2024-04-02T13:45:37.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:45:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 -- 172.21.15.122:0/3679091313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c1009b0 msgr2=0x7f5e0c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 --2- 172.21.15.122:0/3679091313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c100e30 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009930 tx=0x7f5e0802bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 -- 172.21.15.122:0/3679091313 shutdown_connections 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 --2- 172.21.15.122:0/3679091313 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c101370 0x7f5e0c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 --2- 172.21.15.122:0/3679091313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c100e30 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 --2- 172.21.15.122:0/3679091313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1069c0 0x7f5e0c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 -- 172.21.15.122:0/3679091313 >> 172.21.15.122:0/3679091313 conn(0x7f5e0c076d80 msgr2=0x7f5e0c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 -- 172.21.15.122:0/3679091313 shutdown_connections 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.391+0000 7f5e132c5640 1 -- 172.21.15.122:0/3679091313 wait complete. 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 Processor -- start 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 -- start start 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c1069c0 0x7f5e0c1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c110e20 con 0x7f5e0c101370 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c110ca0 con 0x7f5e0c1069c0 2024-04-02T13:45:37.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e132c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c110fa0 con 0x7f5e0c1009b0 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50564/0 (socket says 172.21.15.122:50564) 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 -- 172.21.15.122:0/1454913459 learned_addr learned my addr 172.21.15.122:0/1454913459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e11061640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e11862640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c1069c0 0x7f5e0c1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 -- 172.21.15.122:0/1454913459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1009b0 msgr2=0x7f5e0c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 -- 172.21.15.122:0/1454913459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c1069c0 msgr2=0x7f5e0c1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c1069c0 0x7f5e0c1a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 -- 172.21.15.122:0/1454913459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a47c0 con 0x7f5e0c101370 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.392+0000 7f5e10860640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009900 tx=0x7f5e08004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.393+0000 7f5e027fc640 1 -- 172.21.15.122:0/1454913459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08043020 con 0x7f5e0c101370 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.393+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a4990 con 0x7f5e0c101370 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.393+0000 7f5e027fc640 1 -- 172.21.15.122:0/1454913459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080054e0 con 0x7f5e0c101370 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.393+0000 7f5e027fc640 1 -- 172.21.15.122:0/1454913459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003d00 con 0x7f5e0c101370 2024-04-02T13:45:37.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.393+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a4ed0 con 0x7f5e0c101370 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.395+0000 7f5e027fc640 1 -- 172.21.15.122:0/1454913459 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5e08046020 con 0x7f5e0c101370 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.395+0000 7f5e15f84740 1 Processor -- start 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.395+0000 7f5e15f84740 1 -- start start 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e027fc640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 0x7f5df80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560db69987b0 con 0x560db6a350c0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560db6937570 con 0x560db6a3dca0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e15f84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560db6994760 con 0x560db6a347a0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e11862640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 0x7f5df80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50580/0 (socket says 172.21.15.122:50580) 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 -- 172.21.15.122:0/3398691573 learned_addr learned my addr 172.21.15.122:0/3398691573 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e027fc640 1 -- 172.21.15.122:0/1454913459 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f5e080f2d50 con 0x7f5e0c101370 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e11061640 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 -- 172.21.15.122:0/3398691573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 msgr2=0x560db6a34b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a34b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 -- 172.21.15.122:0/3398691573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 msgr2=0x560db6a40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 -- 172.21.15.122:0/3398691573 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db66ac3f0 con 0x560db6a350c0 2024-04-02T13:45:37.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e10860640 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c70f0 tx=0x7f5e08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511aee4639d3be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5e11862640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 0x7f5df80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005fd0 tx=0x7f5e040097a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5df77fe640 1 -- 172.21.15.122:0/3398691573 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080be310 con 0x560db6a350c0 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5df77fe640 1 -- 172.21.15.122:0/3398691573 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e08004c80 con 0x560db6a350c0 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.396+0000 7f5df77fe640 1 -- 172.21.15.122:0/3398691573 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080431c0 con 0x560db6a350c0 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- 172.21.15.122:0/3398691573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 msgr2=0x560db6a3d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5e080c70f0 tx=0x7f5e08009260 comp rx=0 tx=0).stop 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- 172.21.15.122:0/3398691573 shutdown_connections 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3d760 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 --2- 172.21.15.122:0/3398691573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a34b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- 172.21.15.122:0/3398691573 >> 172.21.15.122:0/3398691573 conn(0x560db6a2eed0 msgr2=0x560db6a2f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- 172.21.15.122:0/3398691573 shutdown_connections 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- 172.21.15.122:0/3398691573 wait complete. 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.397+0000 7f5e15f84740 1 -- start start 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 Processor -- start 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a3ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560db6a40850 con 0x560db6a350c0 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560db6953e80 con 0x560db6a3dca0 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560db6a409d0 con 0x560db6a347a0 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e15f84740 10 client.0 _ll_register_callbacks cb 0x560db698faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50588/0 (socket says 172.21.15.122:50588) 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 -- 172.21.15.122:0/355843149 learned_addr learned my addr 172.21.15.122:0/355843149 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e11061640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a3ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e11862640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 -- 172.21.15.122:0/355843149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 msgr2=0x560db6a3ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a3ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 -- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 msgr2=0x560db6a3b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a3b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 -- 172.21.15.122:0/355843149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db67767d0 con 0x560db6a350c0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e11061640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a3ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e11862640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5e10860640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5e0802c210 tx=0x7f5e080df0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd7da070b34651 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.398+0000 7f5df4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080035f0 con 0x560db6a350c0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f5e080bccb0 con 0x560db6a350c0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e08004c80 con 0x560db6a350c0 2024-04-02T13:45:37.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5e08005690 con 0x560db6a350c0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 10 client.18723 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560db6a40850 con 0x560db6a350c0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e08004c80 con 0x560db6a350c0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f5e080bfce0 con 0x560db6a350c0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 10 client.18723 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 20 client.18723 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6a4db70 0x560db6a4ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 4 client.18723 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560db69a5cf0 con 0x560db6a4db70 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.399+0000 7f5e11061640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6a4db70 0x560db6a4ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.400+0000 7f5e11061640 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6a4db70 0x560db6a4ffc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85256aa6a2a1da3 server_cookie=841d8e3c50562467 in_seq=0 out_seq=0 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.400+0000 7f5df4ff9640 10 client.18723 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5df4ff9640 1 -- 172.21.15.122:0/355843149 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560db69a5cf0 con 0x560db6a4db70 2024-04-02T13:45:37.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5df4ff9640 10 client.18723 handle_command_reply: tid=0 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 client.18723 shutdown 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 msgr2=0x560db6a3b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5e0802c210 tx=0x7f5e080df0f0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 shutdown_connections 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6a4db70 0x560db6a4ffc0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a3dca0 0x560db6a3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a3b060 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 --2- 172.21.15.122:0/355843149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a3ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 >> 172.21.15.122:0/355843149 conn(0x560db6a2eed0 msgr2=0x560db6a30c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5df4ff9640 0 client.18723 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 shutdown_connections 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 1 -- 172.21.15.122:0/355843149 wait complete. 2024-04-02T13:45:37.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.401+0000 7f5e15f84740 20 client.18723 trim_cache size 0 max 16384 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 Processor -- start 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 -- start start 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a347a0 0x560db6a49680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a3dca0 0x560db6a2dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1af100 con 0x560db6a350c0 2024-04-02T13:45:37.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1b0c40 con 0x560db6a347a0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e15f84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1af7d0 con 0x560db6a3dca0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e10860640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50596/0 (socket says 172.21.15.122:50596) 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e10860640 1 -- 172.21.15.122:0/1063938337 learned_addr learned my addr 172.21.15.122:0/1063938337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.414+0000 7f5e11061640 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a347a0 0x560db6a49680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 -- 172.21.15.122:0/1063938337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a3dca0 msgr2=0x560db6a2dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e11862640 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a3dca0 0x560db6a2dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a3dca0 0x560db6a2dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 -- 172.21.15.122:0/1063938337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a347a0 msgr2=0x560db6a49680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a347a0 0x560db6a49680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 -- 172.21.15.122:0/1063938337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560db6a31110 con 0x560db6a350c0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e10860640 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5e080bb330 tx=0x7f5e08009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68ff52b90157d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5df6ffd640 1 -- 172.21.15.122:0/1063938337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080035f0 con 0x560db6a350c0 2024-04-02T13:45:37.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5df6ffd640 1 -- 172.21.15.122:0/1063938337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080bf0b0 con 0x560db6a350c0 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5df6ffd640 1 -- 172.21.15.122:0/1063938337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080bcd80 con 0x560db6a350c0 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e15f84740 1 -- 172.21.15.122:0/1063938337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 msgr2=0x560db6a2d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.415+0000 7f5e15f84740 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5e080bb330 tx=0x7f5e08009260 comp rx=0 tx=0).stop 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- 172.21.15.122:0/1063938337 shutdown_connections 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a3dca0 0x560db6a2dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a350c0 0x560db6a2d6a0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- 172.21.15.122:0/1063938337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a347a0 0x560db6a49680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- 172.21.15.122:0/1063938337 >> 172.21.15.122:0/1063938337 conn(0x560db6a2eed0 msgr2=0x560db6a2f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- 172.21.15.122:0/1063938337 shutdown_connections 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- 172.21.15.122:0/1063938337 wait complete. 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- start start 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 Processor -- start 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a40b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a350c0 0x560db6a41050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1b0570 con 0x560db6a3dca0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1afea0 con 0x560db6a350c0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c1ad100 con 0x560db6a347a0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e15f84740 10 client.0 _ll_register_callbacks cb 0x7f5e08009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e11862640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e11862640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50602/0 (socket says 172.21.15.122:50602) 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e11862640 1 -- 172.21.15.122:0/2462103391 learned_addr learned my addr 172.21.15.122:0/2462103391 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.416+0000 7f5e10860640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a350c0 0x560db6a41050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11061640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a40b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 -- 172.21.15.122:0/2462103391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 msgr2=0x560db6a40b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a40b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 -- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a350c0 msgr2=0x560db6a41050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a350c0 0x560db6a41050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 -- 172.21.15.122:0/2462103391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560db6ab2630 con 0x560db6a3dca0 2024-04-02T13:45:37.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e11862640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5e04012720 tx=0x7f5e04004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eae4a502e6a24e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5df4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0401f8c0 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f5e0401fa60 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e080bf0b0 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5e04031050 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e15f84740 10 client.18729 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560db6ab3830 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.417+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080bf0b0 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f5e0401c3e0 con 0x560db6a3dca0 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 10 client.18729 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 10 client.18729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 20 client.18729 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6ab3fe0 0x560db6ab6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 4 client.18729 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:45:37.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560db69a0040 con 0x560db6ab3fe0 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e11061640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6ab3fe0 0x560db6ab6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5df4ff9640 10 client.18729 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5e11061640 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6ab3fe0 0x560db6ab6400 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad361a4b411ab98a server_cookie=3bd87d2b14a231c3 in_seq=0 out_seq=0 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5df4ff9640 1 -- 172.21.15.122:0/2462103391 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560db69a0040 con 0x560db6ab3fe0 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.418+0000 7f5df4ff9640 10 client.18729 handle_command_reply: tid=0 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 client.18729 shutdown 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 msgr2=0x560db6ab20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5e04012720 tx=0x7f5e04004ab0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 shutdown_connections 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560db6ab3fe0 0x560db6ab6400 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560db6a3dca0 0x560db6ab20f0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560db6a350c0 0x560db6a41050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5df4ff9640 0 client.18729 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:37.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 --2- 172.21.15.122:0/2462103391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560db6a347a0 0x560db6a40b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 >> 172.21.15.122:0/2462103391 conn(0x560db6a2eed0 msgr2=0x560db6a3aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 shutdown_connections 2024-04-02T13:45:37.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 1 -- 172.21.15.122:0/2462103391 wait complete. 2024-04-02T13:45:37.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e15f84740 20 client.18729 trim_cache size 0 max 16384 2024-04-02T13:45:37.429 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:45:37.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 msgr2=0x7f5df80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 0x7f5df80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005fd0 tx=0x7f5e040097a0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 msgr2=0x7f5e0c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:37.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009900 tx=0x7f5e08004430 comp rx=0 tx=0).stop 2024-04-02T13:45:37.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 shutdown_connections 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5df80a3d20 0x7f5df80a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e0c1069c0 0x7f5e0c1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e0c101370 0x7f5e0c199740 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 --2- 172.21.15.122:0/1454913459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e0c1009b0 0x7f5e0c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 >> 172.21.15.122:0/1454913459 conn(0x7f5e0c076d80 msgr2=0x7f5e0c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 shutdown_connections 2024-04-02T13:45:37.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:37.419+0000 7f5e132c5640 1 -- 172.21.15.122:0/1454913459 wait complete. 2024-04-02T13:45:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:37 smithi122 ceph-mon[32114]: pgmap v324: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-02T13:45:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:37 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3364740710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:37 smithi187 ceph-mon[36064]: pgmap v324: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-02T13:45:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:37 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3364740710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:37 smithi188 ceph-mon[36117]: pgmap v324: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 179 op/s 2024-04-02T13:45:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:37 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3364740710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:45:37.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 369550 3.48 MB/sec execute 418 sec latency 2565.295 ms 2024-04-02T13:45:38.433 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:45:38.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:45:38.434 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:45:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 -- 172.21.15.122:0/2071368288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664101360 msgr2=0x7ff66410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 --2- 172.21.15.122:0/2071368288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664101360 0x7ff66410d820 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009be0 tx=0x7ff65c02bf80 comp rx=0 tx=0).stop 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 -- 172.21.15.122:0/2071368288 shutdown_connections 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 --2- 172.21.15.122:0/2071368288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff664101360 0x7ff66410d820 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 --2- 172.21.15.122:0/2071368288 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff6641009c0 0x7ff664100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 --2- 172.21.15.122:0/2071368288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 0x7ff664105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 -- 172.21.15.122:0/2071368288 >> 172.21.15.122:0/2071368288 conn(0x7ff664077a90 msgr2=0x7ff664077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 -- 172.21.15.122:0/2071368288 shutdown_connections 2024-04-02T13:45:38.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.569+0000 7ff66a83c640 1 -- 172.21.15.122:0/2071368288 wait complete. 2024-04-02T13:45:38.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 Processor -- start 2024-04-02T13:45:38.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 -- start start 2024-04-02T13:45:38.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 0x7ff664102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 0x7ff664102fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff664110de0 con 0x7ff6641009c0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff664110c60 con 0x7ff664101360 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff66a83c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff664110f60 con 0x7ff6641058a0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff663fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff663fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49280/0 (socket says 172.21.15.122:49280) 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff663fff640 1 -- 172.21.15.122:0/1571021401 learned_addr learned my addr 172.21.15.122:0/1571021401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.570+0000 7ff6637fe640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 0x7ff664102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff668dd9640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 0x7ff664102fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 -- 172.21.15.122:0/1571021401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 msgr2=0x7ff664102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 0x7ff664102fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 -- 172.21.15.122:0/1571021401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 msgr2=0x7ff664102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 0x7ff664102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 -- 172.21.15.122:0/1571021401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6641a6800 con 0x7ff6641009c0 2024-04-02T13:45:38.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff6637fe640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 0x7ff664102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff663fff640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9c0 tx=0x7ff65400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff6617fa640 1 -- 172.21.15.122:0/1571021401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65400cde0 con 0x7ff6641009c0 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a6af0 con 0x7ff6641009c0 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff6617fa640 1 -- 172.21.15.122:0/1571021401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6540035f0 con 0x7ff6641009c0 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff6617fa640 1 -- 172.21.15.122:0/1571021401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654010690 con 0x7ff6641009c0 2024-04-02T13:45:38.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.571+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a7040 con 0x7ff6641009c0 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.574+0000 7ff6617fa640 1 -- 172.21.15.122:0/1571021401 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff6540037a0 con 0x7ff6641009c0 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.574+0000 7ff66d425740 1 Processor -- start 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 -- start start 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2e996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2ea2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571d2dfd770 con 0x5571d2ea27c0 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5571d2da5960 con 0x5571d2e99be0 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff66d425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5571d2df98c0 con 0x5571d2e992c0 2024-04-02T13:45:38.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2e996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff663fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2ea2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49282/0 (socket says 172.21.15.122:49282) 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2e996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58020/0 (socket says 172.21.15.122:58020) 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 -- 172.21.15.122:0/566612251 learned_addr learned my addr 172.21.15.122:0/566612251 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6617fa640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 0x7ff64c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff668dd9640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 0x7ff64c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 -- 172.21.15.122:0/566612251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 msgr2=0x5571d2e996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2e996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 -- 172.21.15.122:0/566612251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 msgr2=0x5571d2ea2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2ea2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6637fe640 1 -- 172.21.15.122:0/566612251 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d2bd97e0 con 0x5571d2ea27c0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.575+0000 7ff6617fa640 1 -- 172.21.15.122:0/1571021401 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7ff6540d0ea0 con 0x7ff6641009c0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff6637fe640 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff6580114f0 tx=0x7ff6580119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff08f24b4cb6d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff64a7fc640 1 -- 172.21.15.122:0/566612251 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65801e070 con 0x5571d2ea27c0 2024-04-02T13:45:38.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff668dd9640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 0x7ff64c0a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c02bf80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff64a7fc640 1 -- 172.21.15.122:0/566612251 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6580035f0 con 0x5571d2ea27c0 2024-04-02T13:45:38.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff64a7fc640 1 -- 172.21.15.122:0/566612251 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65800f820 con 0x5571d2ea27c0 2024-04-02T13:45:38.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff66d425740 1 -- 172.21.15.122:0/566612251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 msgr2=0x5571d2ea4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.576+0000 7ff66d425740 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff6580114f0 tx=0x7ff6580119b0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 -- 172.21.15.122:0/566612251 shutdown_connections 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea4bb0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2ea2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 --2- 172.21.15.122:0/566612251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2e996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 -- 172.21.15.122:0/566612251 >> 172.21.15.122:0/566612251 conn(0x5571d2e939f0 msgr2=0x5571d2e93dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 -- 172.21.15.122:0/566612251 shutdown_connections 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 -- 172.21.15.122:0/566612251 wait complete. 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 -- start start 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 Processor -- start 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.577+0000 7ff66d425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 0x5571d2e9f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2e9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5571d2ea55b0 con 0x5571d2ea27c0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5571d2db8d20 con 0x5571d2e992c0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5571d2ea5730 con 0x5571d2e99be0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 0x5571d2e9f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49294/0 (socket says 172.21.15.122:49294) 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff668dd9640 1 -- 172.21.15.122:0/971859184 learned_addr learned my addr 172.21.15.122:0/971859184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff6637fe640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2e9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 -- 172.21.15.122:0/971859184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 msgr2=0x5571d2e9fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2e9fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 -- 172.21.15.122:0/971859184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 msgr2=0x5571d2ea01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 -- 172.21.15.122:0/971859184 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d2c39f70 con 0x5571d2e992c0 2024-04-02T13:45:38.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff668dd9640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 10 client.0 _ll_register_callbacks cb 0x5571d2d85a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6580035f0 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff663fff640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 0x5571d2e9f750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6540ce200 tx=0x7ff654009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bbe0fbb86ff45df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.578+0000 7ff6437fe640 10 client.27752 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff6437fe640 1 -- 172.21.15.122:0/971859184 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540a0e00 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff6437fe640 1 -- 172.21.15.122:0/971859184 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7ff6540990c0 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff6437fe640 1 -- 172.21.15.122:0/971859184 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff6540a56c0 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 10 client.27752 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=40}) -- 0x5571d2ad1170 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff6437fe640 1 -- 172.21.15.122:0/971859184 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff6540b9020 con 0x5571d2e992c0 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 10 client.27752 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 10 client.27752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 20 client.27752 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:38.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.579+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2eb2800 0x5571d2eb4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.580+0000 7ff66d425740 4 client.27752 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.580+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d2e0acb0 con 0x5571d2eb2800 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.580+0000 7ff6637fe640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2eb2800 0x5571d2eb4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.580+0000 7ff6437fe640 10 client.27752 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.580+0000 7ff6637fe640 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2eb2800 0x5571d2eb4ca0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84c3bd928ac118c server_cookie=46a3beeb3608bffc in_seq=0 out_seq=0 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff6437fe640 1 -- 172.21.15.122:0/971859184 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5571d2e0acb0 con 0x5571d2eb2800 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff6437fe640 10 client.27752 handle_command_reply: tid=0 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 client.27752 shutdown 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 msgr2=0x5571d2e9f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 0x5571d2e9f750 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff6540ce200 tx=0x7ff654009620 comp rx=0 tx=0).stop 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 shutdown_connections 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2eb2800 0x5571d2eb4ca0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2ea01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff6437fe640 0 client.27752 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2e9fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 --2- 172.21.15.122:0/971859184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e992c0 0x5571d2e9f750 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 >> 172.21.15.122:0/971859184 conn(0x5571d2e939f0 msgr2=0x5571d2e95740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 shutdown_connections 2024-04-02T13:45:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 1 -- 172.21.15.122:0/971859184 wait complete. 2024-04-02T13:45:38.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.581+0000 7ff66d425740 20 client.27752 trim_cache size 0 max 16384 2024-04-02T13:45:38.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 Processor -- start 2024-04-02T13:45:38.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 -- start start 2024-04-02T13:45:38.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2ea1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2eae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b12a0 con 0x5571d2ea27c0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b2de0 con 0x5571d2e99be0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff66d425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b1970 con 0x5571d2e992c0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff668dd9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49300/0 (socket says 172.21.15.122:49300) 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff668dd9640 1 -- 172.21.15.122:0/592807646 learned_addr learned my addr 172.21.15.122:0/592807646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff6637fe640 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2eae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff663fff640 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2ea1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff668dd9640 1 -- 172.21.15.122:0/592807646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 msgr2=0x5571d2ea1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.600+0000 7ff668dd9640 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2ea1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff668dd9640 1 -- 172.21.15.122:0/592807646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 msgr2=0x5571d2eae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff668dd9640 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2eae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff668dd9640 1 -- 172.21.15.122:0/592807646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d2e94c20 con 0x5571d2ea27c0 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff668dd9640 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff65c036040 tx=0x7ff65c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209aa0448cb6060b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff649ffb640 1 -- 172.21.15.122:0/592807646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c049070 con 0x5571d2ea27c0 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff649ffb640 1 -- 172.21.15.122:0/592807646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c054070 con 0x5571d2ea27c0 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.601+0000 7ff649ffb640 1 -- 172.21.15.122:0/592807646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c043480 con 0x5571d2ea27c0 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- 172.21.15.122:0/592807646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 msgr2=0x5571d2eba5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff65c036040 tx=0x7ff65c005e30 comp rx=0 tx=0).stop 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- 172.21.15.122:0/592807646 shutdown_connections 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2ea27c0 0x5571d2eba5b0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2e99be0 0x5571d2eae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 --2- 172.21.15.122:0/592807646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e992c0 0x5571d2ea1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- 172.21.15.122:0/592807646 >> 172.21.15.122:0/592807646 conn(0x5571d2e939f0 msgr2=0x5571d2e956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- 172.21.15.122:0/592807646 shutdown_connections 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- 172.21.15.122:0/592807646 wait complete. 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 -- start start 2024-04-02T13:45:38.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.602+0000 7ff66d425740 1 Processor -- start 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2ea6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2ea27c0 0x5571d2ea7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b2710 con 0x5571d2e992c0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641b2040 con 0x5571d2ea27c0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49312/0 (socket says 172.21.15.122:49312) 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 -- 172.21.15.122:0/2667976803 learned_addr learned my addr 172.21.15.122:0/2667976803 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff6637fe640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2ea6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641af2a0 con 0x5571d2e99be0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 10 client.0 _ll_register_callbacks cb 0x7ff654003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff668dd9640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2ea27c0 0x5571d2ea7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 -- 172.21.15.122:0/2667976803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 msgr2=0x5571d2ea6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2ea6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 -- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2ea27c0 msgr2=0x5571d2ea7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2ea27c0 0x5571d2ea7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 -- 172.21.15.122:0/2667976803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d2ea7ab0 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff663fff640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff654099420 tx=0x7ff6540d33b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10acbc9936abdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff6437fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540a0e00 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7ff654004210 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c054070 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.603+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff654098cf0 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 10 client.18747 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5571d2f19cc0 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff65c054070 con 0x5571d2e992c0 2024-04-02T13:45:38.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff6540b9040 con 0x5571d2e992c0 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 10 client.18747 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 20 client.18747 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2f1a430 0x5571d2f1c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 4 client.18747 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571d2e05000 con 0x5571d2f1a430 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff6637fe640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2f1a430 0x5571d2f1c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff6437fe640 10 client.18747 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.604+0000 7ff6637fe640 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2f1a430 0x5571d2f1c820 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2357a7de923e095e server_cookie=75df3f9250121900 in_seq=0 out_seq=0 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.605+0000 7ff6437fe640 1 -- 172.21.15.122:0/2667976803 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571d2e05000 con 0x5571d2f1a430 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.605+0000 7ff6437fe640 10 client.18747 handle_command_reply: tid=0 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.605+0000 7ff66d425740 1 client.18747 shutdown 2024-04-02T13:45:38.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 msgr2=0x5571d2eaabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7ff654099420 tx=0x7ff6540d33b0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 shutdown_connections 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5571d2f1a430 0x5571d2f1c820 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5571d2ea27c0 0x5571d2ea7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff6437fe640 0 client.18747 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5571d2e99be0 0x5571d2ea6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 --2- 172.21.15.122:0/2667976803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5571d2e992c0 0x5571d2eaabb0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 >> 172.21.15.122:0/2667976803 conn(0x5571d2e939f0 msgr2=0x5571d2e94df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 shutdown_connections 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 1 -- 172.21.15.122:0/2667976803 wait complete. 2024-04-02T13:45:38.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66d425740 20 client.18747 trim_cache size 0 max 16384 2024-04-02T13:45:38.616 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:45:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "cba16428-881a-42bd-9bd2-fdcb79d31953", 2024-04-02T13:45:38.616 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:45:38.616 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 msgr2=0x7ff64c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.606+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 0x7ff64c0a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006010 tx=0x7ff65c02bf80 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 msgr2=0x7ff664102540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7ff65400e9c0 tx=0x7ff65400ee80 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 shutdown_connections 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff64c0a3d20 0x7ff64c0a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6641058a0 0x7ff664102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff664101360 0x7ff664102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 --2- 172.21.15.122:0/1571021401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6641009c0 0x7ff664102540 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 >> 172.21.15.122:0/1571021401 conn(0x7ff664077a90 msgr2=0x7ff664069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 shutdown_connections 2024-04-02T13:45:38.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.607+0000 7ff66a83c640 1 -- 172.21.15.122:0/1571021401 wait complete. 2024-04-02T13:45:38.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:45:38.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:45:38.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 -- 172.21.15.122:0/1383236670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 msgr2=0x7f0130069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 --2- 172.21.15.122:0/1383236670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130069180 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f011c009930 tx=0x7f011c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 -- 172.21.15.122:0/1383236670 shutdown_connections 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 --2- 172.21.15.122:0/1383236670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 0x7f013010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 --2- 172.21.15.122:0/1383236670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130069180 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 --2- 172.21.15.122:0/1383236670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 0x7f0130107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 -- 172.21.15.122:0/1383236670 >> 172.21.15.122:0/1383236670 conn(0x7f0130077b00 msgr2=0x7f0130077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 -- 172.21.15.122:0/1383236670 shutdown_connections 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.739+0000 7f0136e3c640 1 -- 172.21.15.122:0/1383236670 wait complete. 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 Processor -- start 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 -- start start 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 0x7f0130198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 0x7f013019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0130111ea0 con 0x7f0130068d00 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0130111d20 con 0x7f01300696c0 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0136e3c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0130112020 con 0x7f0130107b50 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49324/0 (socket says 172.21.15.122:49324) 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 -- 172.21.15.122:0/421173534 learned_addr learned my addr 172.21.15.122:0/421173534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f012ffff640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 0x7f0130198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f01353d9640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 0x7f013019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 -- 172.21.15.122:0/421173534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 msgr2=0x7f013019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 0x7f013019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 -- 172.21.15.122:0/421173534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 msgr2=0x7f0130198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 0x7f0130198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.740+0000 7f0134bd8640 1 -- 172.21.15.122:0/421173534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013019d3a0 con 0x7f0130068d00 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f0134bd8640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9f0 tx=0x7f012400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f012dffb640 1 -- 172.21.15.122:0/421173534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400cd80 con 0x7f0130068d00 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013019d690 con 0x7f0130068d00 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f012dffb640 1 -- 172.21.15.122:0/421173534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0124005500 con 0x7f0130068d00 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f012dffb640 1 -- 172.21.15.122:0/421173534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124010690 con 0x7f0130068d00 2024-04-02T13:45:38.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.741+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a4f20 con 0x7f0130068d00 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.743+0000 7f012dffb640 1 -- 172.21.15.122:0/421173534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f01240046c0 con 0x7f0130068d00 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.743+0000 7f0139a23740 1 Processor -- start 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 -- start start 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b3c958830 con 0x556b3c9fddd0 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b3c900820 con 0x556b3c9f51f0 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0139a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b3c9547e0 con 0x556b3c9f48d0 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f01353d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f01353d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58086/0 (socket says 172.21.15.122:58086) 2024-04-02T13:45:38.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f01353d9640 1 -- 172.21.15.122:0/1711209118 learned_addr learned my addr 172.21.15.122:0/1711209118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012dffb640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 0x7f01200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f0134bd8640 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f01353d9640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 0x7f01200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 -- 172.21.15.122:0/1711209118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 msgr2=0x556b3c9f4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 -- 172.21.15.122:0/1711209118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 msgr2=0x556b3c9fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 -- 172.21.15.122:0/1711209118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3c7367d0 con 0x556b3c9fddd0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f01353d9640 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012dffb640 1 -- 172.21.15.122:0/421173534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f01240d0fe0 con 0x7f0130068d00 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.744+0000 7f012ffff640 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca001c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f011c009230 tx=0x7f011c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7005a1ae435ec318 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f011affd640 1 -- 172.21.15.122:0/1711209118 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c042070 con 0x556b3c9fddd0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f01353d9640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 0x7f01200a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0128005c40 tx=0x7f0128004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f011affd640 1 -- 172.21.15.122:0/1711209118 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c02c8b0 con 0x556b3c9fddd0 2024-04-02T13:45:38.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f011affd640 1 -- 172.21.15.122:0/1711209118 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c004bf0 con 0x556b3c9fddd0 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f0139a23740 1 -- 172.21.15.122:0/1711209118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 msgr2=0x556b3ca001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.745+0000 7f0139a23740 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca001c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f011c009230 tx=0x7f011c009670 comp rx=0 tx=0).stop 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- 172.21.15.122:0/1711209118 shutdown_connections 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca001c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- 172.21.15.122:0/1711209118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9f4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- 172.21.15.122:0/1711209118 >> 172.21.15.122:0/1711209118 conn(0x556b3c9ef000 msgr2=0x556b3c9ef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- 172.21.15.122:0/1711209118 shutdown_connections 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- 172.21.15.122:0/1711209118 wait complete. 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- start start 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 Processor -- start 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9fddd0 0x556b3c9fae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b3ca00b60 con 0x556b3c9f51f0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b3c913d30 con 0x556b3c9fddd0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b3ca00ce0 con 0x556b3c9f48d0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0139a23740 10 client.0 _ll_register_callbacks cb 0x556b3c913720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f012ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.746+0000 7f012ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49342/0 (socket says 172.21.15.122:49342) 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 -- 172.21.15.122:0/2563801946 learned_addr learned my addr 172.21.15.122:0/2563801946 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f01353d9640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9fddd0 0x556b3c9fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 -- 172.21.15.122:0/2563801946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 msgr2=0x556b3c9fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 -- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9fddd0 msgr2=0x556b3c9fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9fddd0 0x556b3c9fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 -- 172.21.15.122:0/2563801946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3c66c3f0 con 0x556b3c9f51f0 2024-04-02T13:45:38.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f0134bd8640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f012ffff640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f011c02f020 tx=0x7f011c02c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec4475eb5445acd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f010ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c042040 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f011c004f10 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f011c02c8b0 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f011c005110 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f0139a23740 10 client.18759 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.747+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556b3c62c0f0 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c02c8b0 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f011c034440 con 0x556b3c9f51f0 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 10 client.18759 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 10 client.18759 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 20 client.18759 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca0dae0 0x556b3ca0fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 4 client.18759 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:45:38.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3c965d70 con 0x556b3ca0dae0 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0134bd8640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca0dae0 0x556b3ca0fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f010ffff640 10 client.18759 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.748+0000 7f0134bd8640 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca0dae0 0x556b3ca0fed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24cde582fff709eb server_cookie=558895539de642f8 in_seq=0 out_seq=0 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f010ffff640 1 -- 172.21.15.122:0/2563801946 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b3c965d70 con 0x556b3ca0dae0 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f010ffff640 10 client.18759 handle_command_reply: tid=0 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 client.18759 shutdown 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 msgr2=0x556b3c9fa930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f011c02f020 tx=0x7f011c02c800 comp rx=0 tx=0).stop 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 shutdown_connections 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca0dae0 0x556b3ca0fed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9fddd0 0x556b3c9fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f51f0 0x556b3c9fa930 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 --2- 172.21.15.122:0/2563801946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f48d0 0x556b3c9fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f010ffff640 0 client.18759 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.749+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 >> 172.21.15.122:0/2563801946 conn(0x556b3c9ef000 msgr2=0x556b3c9f0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.750+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 shutdown_connections 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.750+0000 7f0139a23740 1 -- 172.21.15.122:0/2563801946 wait complete. 2024-04-02T13:45:38.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.750+0000 7f0139a23740 20 client.18759 trim_cache size 0 max 16384 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 Processor -- start 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 -- start start 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9fddd0 0x556b3c9fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01301aef70 con 0x556b3c9f48d0 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01301b0ab0 con 0x556b3c9f51f0 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0139a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01301af640 con 0x556b3c9fddd0 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49356/0 (socket says 172.21.15.122:49356) 2024-04-02T13:45:38.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 -- 172.21.15.122:0/4070619594 learned_addr learned my addr 172.21.15.122:0/4070619594 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f012ffff640 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f01353d9640 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9fddd0 0x556b3c9fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 -- 172.21.15.122:0/4070619594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9fddd0 msgr2=0x556b3c9fd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9fddd0 0x556b3c9fd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 -- 172.21.15.122:0/4070619594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 msgr2=0x556b3c9fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.768+0000 7f0134bd8640 1 -- 172.21.15.122:0/4070619594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3ca00cc0 con 0x556b3c9f48d0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0134bd8640 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0124099560 tx=0x7f01240d34f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74b9dc72dd56ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f011a7fc640 1 -- 172.21.15.122:0/4070619594 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240c2e60 con 0x556b3c9f48d0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f011a7fc640 1 -- 172.21.15.122:0/4070619594 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012409cb90 con 0x556b3c9f48d0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f011a7fc640 1 -- 172.21.15.122:0/4070619594 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124099200 con 0x556b3c9f48d0 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 -- 172.21.15.122:0/4070619594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 msgr2=0x556b3c9ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0124099560 tx=0x7f01240d34f0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 -- 172.21.15.122:0/4070619594 shutdown_connections 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9fddd0 0x556b3c9fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f51f0 0x556b3c9fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.769+0000 7f0139a23740 1 --2- 172.21.15.122:0/4070619594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9f48d0 0x556b3c9ee0c0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- 172.21.15.122:0/4070619594 >> 172.21.15.122:0/4070619594 conn(0x556b3c9ef000 msgr2=0x556b3c9ed9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- 172.21.15.122:0/4070619594 shutdown_connections 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- 172.21.15.122:0/4070619594 wait complete. 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- start start 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 Processor -- start 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 0x556b3ca06150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 0x556b3ca02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f01301b03e0 con 0x556b3c9fddd0 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f01301afd10 con 0x556b3c9f48d0 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01301acf70 con 0x556b3c9f51f0 2024-04-02T13:45:38.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0139a23740 10 client.0 _ll_register_callbacks cb 0x7f011c02cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f01353d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f0134bd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 0x556b3ca06150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f01353d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49360/0 (socket says 172.21.15.122:49360) 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f012ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 0x556b3ca02f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.770+0000 7f01353d9640 1 -- 172.21.15.122:0/3893264951 learned_addr learned my addr 172.21.15.122:0/3893264951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 -- 172.21.15.122:0/3893264951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 msgr2=0x556b3ca02f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 0x556b3ca02f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 -- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 msgr2=0x556b3ca06150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 0x556b3ca06150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 -- 172.21.15.122:0/3893264951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3c9eee30 con 0x556b3c9fddd0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f0134bd8640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 0x556b3ca06150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f012ffff640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 0x556b3ca02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f01353d9640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0128004410 tx=0x7f0128004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33631b19918499e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f010ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:45:38.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128013d00 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9525+0+0 (secure 0 0 0) 0x7f0128004b20 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f012409cb90 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0128035050 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f0139a23740 10 client.18765 fetch_fsmap learned FSMap version 40 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556b3ca78cf0 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.771+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f012409cb90 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f012801f040 con 0x556b3c9fddd0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 10 client.18765 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 10 client.18765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 20 client.18765 populate_metadata read hostname 'smithi122' 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca794a0 0x556b3ca7b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 4 client.18765 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b3c9600c0 con 0x556b3ca794a0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0134bd8640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca794a0 0x556b3ca7b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f010ffff640 10 client.18765 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0134bd8640 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca794a0 0x556b3ca7b8c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813607f6ffb4027b server_cookie=d5dbb4393ebef0fa in_seq=0 out_seq=0 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f010ffff640 1 -- 172.21.15.122:0/3893264951 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556b3c9600c0 con 0x556b3ca794a0 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f010ffff640 10 client.18765 handle_command_reply: tid=0 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 client.18765 shutdown 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 msgr2=0x556b3ca03470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0128004410 tx=0x7f0128004a40 comp rx=0 tx=0).stop 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 shutdown_connections 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b3ca794a0 0x556b3ca7b8c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b3c9fddd0 0x556b3ca03470 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b3c9f51f0 0x556b3ca02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f010ffff640 0 client.18765 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 --2- 172.21.15.122:0/3893264951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b3c9f48d0 0x556b3ca06150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.772+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 >> 172.21.15.122:0/3893264951 conn(0x556b3c9ef000 msgr2=0x556b3c9f0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 shutdown_connections 2024-04-02T13:45:38.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0139a23740 1 -- 172.21.15.122:0/3893264951 wait complete. 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0139a23740 20 client.18765 trim_cache size 0 max 16384 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "cba16428-881a-42bd-9bd2-fdcb79d31953": { 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "cba16428-881a-42bd-9bd2-fdcb79d31953", 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:45:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:45:38.784 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:45:38.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 msgr2=0x7f01200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 0x7f01200a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0128005c40 tx=0x7f0128004ab0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 msgr2=0x7f0130198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9f0 tx=0x7f012400edf0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 shutdown_connections 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f01200a3d20 0x7f01200a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0130107b50 0x7f013019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f01300696c0 0x7f0130198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 --2- 172.21.15.122:0/421173534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0130068d00 0x7f0130198400 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 >> 172.21.15.122:0/421173534 conn(0x7f0130077b00 msgr2=0x7f01300fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 shutdown_connections 2024-04-02T13:45:38.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:45:38.773+0000 7f0136e3c640 1 -- 172.21.15.122:0/421173534 wait complete. 2024-04-02T13:45:38.787 INFO:tasks.cephfs.filesystem:scrub status for tag:cba16428-881a-42bd-9bd2-fdcb79d31953 - {'path': '/', 'tag': 'cba16428-881a-42bd-9bd2-fdcb79d31953', 'options': 'recursive,force'} 2024-04-02T13:45:38.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 370403 3.48 MB/sec execute 419 sec latency 3008.422 ms 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: pgmap v325: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:39.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:39.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:39 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: pgmap v325: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:39 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: pgmap v325: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:39 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:39.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 372632 3.50 MB/sec execute 420 sec latency 22.398 ms 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:45:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:40 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:45:40.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:40.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:40.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:45:40.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:40 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:45:40.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:45:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:40 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:45:40.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 373154 3.50 MB/sec execute 421 sec latency 330.992 ms 2024-04-02T13:45:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:41 smithi187 ceph-mon[36064]: pgmap v326: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:41 smithi188 ceph-mon[36117]: pgmap v326: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:41 smithi122 ceph-mon[32114]: pgmap v326: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-04-02T13:45:41.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 373154 3.49 MB/sec execute 422 sec latency 1331.106 ms 2024-04-02T13:45:42.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 373154 3.48 MB/sec execute 423 sec latency 2331.204 ms 2024-04-02T13:45:43.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:43 smithi187 ceph-mon[36064]: pgmap v327: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-04-02T13:45:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:43 smithi188 ceph-mon[36117]: pgmap v327: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-04-02T13:45:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:43 smithi122 ceph-mon[32114]: pgmap v327: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-04-02T13:45:43.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 374126 3.48 MB/sec execute 424 sec latency 2772.116 ms 2024-04-02T13:45:44.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 376254 3.51 MB/sec execute 425 sec latency 34.017 ms 2024-04-02T13:45:45.622 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:45 smithi122 ceph-mon[32114]: pgmap v328: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-02T13:45:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:45 smithi187 ceph-mon[36064]: pgmap v328: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-02T13:45:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:45 smithi188 ceph-mon[36117]: pgmap v328: 129 pgs: 129 active+clean; 296 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-04-02T13:45:45.973 INFO:tasks.workunit.client.0.smithi122.stdout: 1 376641 3.50 MB/sec execute 426 sec latency 566.740 ms 2024-04-02T13:45:46.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 376641 3.49 MB/sec execute 427 sec latency 1566.812 ms 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: pgmap v329: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: pgmap v329: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:45:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: pgmap v329: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-04-02T13:45:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:45:47.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 376641 3.48 MB/sec execute 428 sec latency 2567.142 ms 2024-04-02T13:45:48.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 377623 3.49 MB/sec execute 429 sec latency 3003.433 ms 2024-04-02T13:45:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:49 smithi187 ceph-mon[36064]: pgmap v330: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:49 smithi188 ceph-mon[36117]: pgmap v330: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:49 smithi122 ceph-mon[32114]: pgmap v330: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:49.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 379826 3.51 MB/sec execute 430 sec latency 15.206 ms 2024-04-02T13:45:50.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 380353 3.51 MB/sec execute 431 sec latency 401.842 ms 2024-04-02T13:45:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:51 smithi187 ceph-mon[36064]: pgmap v331: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:51 smithi188 ceph-mon[36117]: pgmap v331: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:51 smithi122 ceph-mon[32114]: pgmap v331: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-04-02T13:45:51.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 380353 3.50 MB/sec execute 432 sec latency 1401.963 ms 2024-04-02T13:45:52.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 380353 3.49 MB/sec execute 433 sec latency 2402.079 ms 2024-04-02T13:45:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:53 smithi187 ceph-mon[36064]: pgmap v332: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:45:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:53 smithi188 ceph-mon[36117]: pgmap v332: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:45:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:53 smithi122 ceph-mon[32114]: pgmap v332: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:45:53.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 381339 3.49 MB/sec execute 434 sec latency 2839.916 ms 2024-04-02T13:45:54.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 383428 3.52 MB/sec execute 435 sec latency 12.630 ms 2024-04-02T13:45:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:55 smithi187 ceph-mon[36064]: pgmap v333: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-02T13:45:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:55 smithi188 ceph-mon[36117]: pgmap v333: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-02T13:45:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:55 smithi122 ceph-mon[32114]: pgmap v333: 129 pgs: 129 active+clean; 302 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-04-02T13:45:55.974 INFO:tasks.workunit.client.0.smithi122.stdout: 1 383771 3.51 MB/sec execute 436 sec latency 594.872 ms 2024-04-02T13:45:56.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 383771 3.50 MB/sec execute 437 sec latency 1594.997 ms 2024-04-02T13:45:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:57 smithi187 ceph-mon[36064]: pgmap v334: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-04-02T13:45:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:57 smithi188 ceph-mon[36117]: pgmap v334: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-04-02T13:45:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:57 smithi122 ceph-mon[32114]: pgmap v334: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 161 op/s 2024-04-02T13:45:57.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 383771 3.49 MB/sec execute 438 sec latency 2595.062 ms 2024-04-02T13:45:58.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 384634 3.50 MB/sec execute 439 sec latency 3035.797 ms 2024-04-02T13:45:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:45:59 smithi187 ceph-mon[36064]: pgmap v335: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:45:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:45:59 smithi188 ceph-mon[36117]: pgmap v335: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:45:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:45:59 smithi122 ceph-mon[32114]: pgmap v335: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:45:59.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 386806 3.52 MB/sec execute 440 sec latency 14.521 ms 2024-04-02T13:46:00.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 387467 3.51 MB/sec execute 441 sec latency 168.895 ms 2024-04-02T13:46:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:01 smithi187 ceph-mon[36064]: pgmap v336: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:46:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:01 smithi188 ceph-mon[36117]: pgmap v336: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:46:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:01 smithi122 ceph-mon[32114]: pgmap v336: 129 pgs: 129 active+clean; 307 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-04-02T13:46:01.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 387467 3.50 MB/sec execute 442 sec latency 1169.013 ms 2024-04-02T13:46:02.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 387467 3.50 MB/sec execute 443 sec latency 2169.070 ms 2024-04-02T13:46:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:03 smithi187 ceph-mon[36064]: pgmap v337: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-02T13:46:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:03 smithi187 ceph-mon[36064]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:46:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:03 smithi188 ceph-mon[36117]: pgmap v337: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-02T13:46:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:03 smithi188 ceph-mon[36117]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:46:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:03 smithi122 ceph-mon[32114]: pgmap v337: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 180 op/s 2024-04-02T13:46:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:03 smithi122 ceph-mon[32114]: osdmap e104: 12 total, 12 up, 12 in 2024-04-02T13:46:03.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 387704 3.49 MB/sec execute 444 sec latency 2605.216 ms 2024-04-02T13:46:04.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 388978 3.50 MB/sec execute 445 sec latency 72.760 ms 2024-04-02T13:46:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:05 smithi187 ceph-mon[36064]: pgmap v339: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-04-02T13:46:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:05 smithi188 ceph-mon[36117]: pgmap v339: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-04-02T13:46:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:05 smithi122 ceph-mon[32114]: pgmap v339: 129 pgs: 129 active+clean; 311 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 150 op/s 2024-04-02T13:46:05.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 390582 3.52 MB/sec execute 446 sec latency 40.982 ms 2024-04-02T13:46:06.991 INFO:tasks.workunit.client.0.smithi122.stdout: 1 391868 3.52 MB/sec execute 447 sec latency 15.406 ms 2024-04-02T13:46:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:07 smithi187 ceph-mon[36064]: pgmap v340: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:07 smithi188 ceph-mon[36117]: pgmap v340: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:07 smithi122 ceph-mon[32114]: pgmap v340: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:07.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 394008 3.54 MB/sec execute 448 sec latency 18.019 ms 2024-04-02T13:46:08.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:08.789 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:08.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- 172.21.15.122:0/3155978682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d40696a0 msgr2=0x7f48d410e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 --2- 172.21.15.122:0/3155978682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d40696a0 0x7f48d410e940 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f48d0009930 tx=0x7f48d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:08.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- 172.21.15.122:0/3155978682 shutdown_connections 2024-04-02T13:46:08.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 --2- 172.21.15.122:0/3155978682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d40696a0 0x7f48d410e940 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 --2- 172.21.15.122:0/3155978682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4068ce0 0x7f48d4069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 --2- 172.21.15.122:0/3155978682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d4107a80 0x7f48d4107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- 172.21.15.122:0/3155978682 >> 172.21.15.122:0/3155978682 conn(0x7f48d4077b00 msgr2=0x7f48d4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- 172.21.15.122:0/3155978682 shutdown_connections 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- 172.21.15.122:0/3155978682 wait complete. 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 Processor -- start 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.898+0000 7f48db869640 1 -- start start 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 0x7f48d41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4107a80 0x7f48d419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111e90 con 0x7f48d4068ce0 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4111d10 con 0x7f48d40696a0 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f48d4112010 con 0x7f48d4107a80 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34592/0 (socket says 172.21.15.122:34592) 2024-04-02T13:46:08.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 -- 172.21.15.122:0/2833893261 learned_addr learned my addr 172.21.15.122:0/2833893261 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9e06640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4107a80 0x7f48d419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d8e04640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 0x7f48d41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 -- 172.21.15.122:0/2833893261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4107a80 msgr2=0x7f48d419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4107a80 0x7f48d419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 -- 172.21.15.122:0/2833893261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 msgr2=0x7f48d41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 0x7f48d41988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 -- 172.21.15.122:0/2833893261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d419d360 con 0x7f48d4068ce0 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d8e04640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 0x7f48d41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:08.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48d9605640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00eab0 tx=0x7f48cc00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48ca7fc640 1 -- 172.21.15.122:0/2833893261 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc00ce40 con 0x7f48d4068ce0 2024-04-02T13:46:08.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48ca7fc640 1 -- 172.21.15.122:0/2833893261 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc011950 con 0x7f48d4068ce0 2024-04-02T13:46:08.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d419d650 con 0x7f48d4068ce0 2024-04-02T13:46:08.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48ca7fc640 1 -- 172.21.15.122:0/2833893261 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc010ae0 con 0x7f48d4068ce0 2024-04-02T13:46:08.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.899+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41a4ee0 con 0x7f48d4068ce0 2024-04-02T13:46:08.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48ca7fc640 1 -- 172.21.15.122:0/2833893261 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f48cc011b00 con 0x7f48d4068ce0 2024-04-02T13:46:08.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 Processor -- start 2024-04-02T13:46:08.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 -- start start 2024-04-02T13:46:08.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbe3041830 con 0x55cbe30dd8d0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbe2fe9820 con 0x55cbe30e6dd0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.902+0000 7f48de44b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbe303d7e0 con 0x55cbe30de1f0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48ca7fc640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 0x7f48c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9605640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34596/0 (socket says 172.21.15.122:34596) 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9e06640 1 -- 172.21.15.122:0/2525397596 learned_addr learned my addr 172.21.15.122:0/2525397596 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9e06640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 0x7f48c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48ca7fc640 1 -- 172.21.15.122:0/2833893261 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7f48cc0d1290 con 0x7f48d4068ce0 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 -- 172.21.15.122:0/2525397596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 msgr2=0x55cbe30e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 -- 172.21.15.122:0/2525397596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 msgr2=0x55cbe30ddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ddcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 -- 172.21.15.122:0/2525397596 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe2e1f7d0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9605640 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d9e06640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 0x7f48c00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48d002c820 tx=0x7f48d00058d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48d8e04640 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e91c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f48c40058c0 tx=0x7f48c400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad70832735a59d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.903+0000 7f48bf7fe640 1 -- 172.21.15.122:0/2525397596 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005220 con 0x55cbe30e6dd0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48bf7fe640 1 -- 172.21.15.122:0/2525397596 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c4004c80 con 0x55cbe30e6dd0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48bf7fe640 1 -- 172.21.15.122:0/2525397596 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4010700 con 0x55cbe30e6dd0 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 -- 172.21.15.122:0/2525397596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 msgr2=0x55cbe30e91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e91c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f48c40058c0 tx=0x7f48c400e720 comp rx=0 tx=0).stop 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 -- 172.21.15.122:0/2525397596 shutdown_connections 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e91c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 --2- 172.21.15.122:0/2525397596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 -- 172.21.15.122:0/2525397596 >> 172.21.15.122:0/2525397596 conn(0x55cbe30d8000 msgr2=0x55cbe30d83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 -- 172.21.15.122:0/2525397596 shutdown_connections 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.904+0000 7f48de44b740 1 -- 172.21.15.122:0/2525397596 wait complete. 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 -- start start 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 Processor -- start 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbe30e9bc0 con 0x55cbe30dd8d0 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cbe2ffcd30 con 0x55cbe30e6dd0 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cbe30e9d40 con 0x55cbe30de1f0 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48de44b740 10 client.0 _ll_register_callbacks cb 0x55cbe2ffc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34612/0 (socket says 172.21.15.122:34612) 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48d9605640 1 -- 172.21.15.122:0/3998748826 learned_addr learned my addr 172.21.15.122:0/3998748826 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.905+0000 7f48d8e04640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 -- 172.21.15.122:0/3998748826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 msgr2=0x55cbe30e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 -- 172.21.15.122:0/3998748826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 msgr2=0x55cbe30e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 -- 172.21.15.122:0/3998748826 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe2d553f0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9605640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48d9e06640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e3eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48d002cfd0 tx=0x7f48d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4fefae3ec1a8c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0052020 con 0x55cbe30e6dd0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7f48d0004f10 con 0x55cbe30e6dd0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48c4004c80 con 0x55cbe30e6dd0 2024-04-02T13:46:08.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f48d005d050 con 0x55cbe30e6dd0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48de44b740 10 client.27788 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cbe2d150f0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48c4004c80 con 0x55cbe30e6dd0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.906+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f48d0003bc0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 10 client.27788 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 10 client.27788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 20 client.27788 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe30f6d30 0x55cbe30f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 4 client.27788 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbe304ed70 con 0x55cbe30f6d30 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48d9605640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe30f6d30 0x55cbe30f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48bcff9640 10 client.27788 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.907+0000 7f48d9605640 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe30f6d30 0x55cbe30f9160 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a3c4ab96faae2af server_cookie=e9aef0f811e599d in_seq=0 out_seq=0 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48bcff9640 1 -- 172.21.15.122:0/3998748826 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cbe304ed70 con 0x55cbe30f6d30 2024-04-02T13:46:08.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48bcff9640 10 client.27788 handle_command_reply: tid=0 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 client.27788 shutdown 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 msgr2=0x55cbe30e3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e3eb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48d002cfd0 tx=0x7f48d0009260 comp rx=0 tx=0).stop 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 shutdown_connections 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe30f6d30 0x55cbe30f9160 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e3eb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 --2- 172.21.15.122:0/3998748826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48bcff9640 0 client.27788 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 >> 172.21.15.122:0/3998748826 conn(0x55cbe30d8000 msgr2=0x55cbe30d9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 shutdown_connections 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 1 -- 172.21.15.122:0/3998748826 wait complete. 2024-04-02T13:46:08.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.908+0000 7f48de44b740 20 client.27788 trim_cache size 0 max 16384 2024-04-02T13:46:08.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 Processor -- start 2024-04-02T13:46:08.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 -- start start 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30de1f0 0x55cbe30d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41af290 con 0x55cbe30e6dd0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b0dd0 con 0x55cbe30de1f0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48de44b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41af960 con 0x55cbe30dd8d0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9605640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d8e04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30de1f0 0x55cbe30d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34616/0 (socket says 172.21.15.122:34616) 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9605640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47298/0 (socket says 172.21.15.122:47298) 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 -- 172.21.15.122:0/1053743694 learned_addr learned my addr 172.21.15.122:0/1053743694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 -- 172.21.15.122:0/1053743694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 msgr2=0x55cbe30f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 -- 172.21.15.122:0/1053743694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30de1f0 msgr2=0x55cbe30d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30de1f0 0x55cbe30d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9e06640 1 -- 172.21.15.122:0/1053743694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbe30d7e30 con 0x55cbe30e6dd0 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.927+0000 7f48d9605640 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48d9e06640 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f48d004d530 tx=0x7f48d00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9176c3905269da06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48beffd640 1 -- 172.21.15.122:0/1053743694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d0052040 con 0x55cbe30e6dd0 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48beffd640 1 -- 172.21.15.122:0/1053743694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d002cbc0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48beffd640 1 -- 172.21.15.122:0/1053743694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d00038e0 con 0x55cbe30e6dd0 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48de44b740 1 -- 172.21.15.122:0/1053743694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 msgr2=0x55cbe30e60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.928+0000 7f48de44b740 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f48d004d530 tx=0x7f48d00092d0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- 172.21.15.122:0/1053743694 shutdown_connections 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e60a0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30de1f0 0x55cbe30d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- 172.21.15.122:0/1053743694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30dd8d0 0x55cbe30f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- 172.21.15.122:0/1053743694 >> 172.21.15.122:0/1053743694 conn(0x55cbe30d8000 msgr2=0x55cbe30d69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- 172.21.15.122:0/1053743694 shutdown_connections 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- 172.21.15.122:0/1053743694 wait complete. 2024-04-02T13:46:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- start start 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 Processor -- start 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b0700 con 0x55cbe30dd8d0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41b0030 con 0x55cbe30e6dd0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f48d41ad290 con 0x55cbe30de1f0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48de44b740 10 client.0 _ll_register_callbacks cb 0x7f48d00043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48d8e04640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48d9605640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34620/0 (socket says 172.21.15.122:34620) 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48d9e06640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.929+0000 7f48d9605640 1 -- 172.21.15.122:0/175609773 learned_addr learned my addr 172.21.15.122:0/175609773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 -- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 msgr2=0x55cbe30e0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 -- 172.21.15.122:0/175609773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 msgr2=0x55cbe30ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 -- 172.21.15.122:0/175609773 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbe30e1020 con 0x55cbe30de1f0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d9605640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48d8e04640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e02e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c400c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e03fba756ee276b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c402b030 con 0x55cbe30de1f0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d002cbc0 con 0x55cbe30de1f0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7f48c4004020 con 0x55cbe30de1f0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f48c4035050 con 0x55cbe30de1f0 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48de44b740 10 client.27577 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55cbe315d280 con 0x55cbe30de1f0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.930+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48d002cbc0 con 0x55cbe30de1f0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f48c40035f0 con 0x55cbe30de1f0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 10 client.27577 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 20 client.27577 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe315d720 0x55cbe315fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 4 client.27577 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbe30490c0 con 0x55cbe315d720 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48d9605640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe315d720 0x55cbe315fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48bcff9640 10 client.27577 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48d9605640 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe315d720 0x55cbe315fb40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303684ad55a7d768 server_cookie=7569d15d16badc90 in_seq=0 out_seq=0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48bcff9640 1 -- 172.21.15.122:0/175609773 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cbe30490c0 con 0x55cbe315d720 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48bcff9640 10 client.27577 handle_command_reply: tid=0 2024-04-02T13:46:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 client.27577 shutdown 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 msgr2=0x55cbe30e02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e02e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c400c640 comp rx=0 tx=0).stop 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 shutdown_connections 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cbe315d720 0x55cbe315fb40 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cbe30e6dd0 0x55cbe30e0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48bcff9640 0 client.27577 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cbe30de1f0 0x55cbe30e02e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 --2- 172.21.15.122:0/175609773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbe30dd8d0 0x55cbe30ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.931+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 >> 172.21.15.122:0/175609773 conn(0x55cbe30d8000 msgr2=0x55cbe30defa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 shutdown_connections 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48de44b740 1 -- 172.21.15.122:0/175609773 wait complete. 2024-04-02T13:46:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48de44b740 20 client.27577 trim_cache size 0 max 16384 2024-04-02T13:46:08.943 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:08.943 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:46:08.943 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:46:08.943 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 msgr2=0x7f48c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 0x7f48c00a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f48d002c820 tx=0x7f48d00058d0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 msgr2=0x7f48d4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00eab0 tx=0x7f48cc00ef70 comp rx=0 tx=0).stop 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 shutdown_connections 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f48c00a3d20 0x7f48c00a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48d4107a80 0x7f48d419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f48d40696a0 0x7f48d41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 --2- 172.21.15.122:0/2833893261 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f48d4068ce0 0x7f48d4198370 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:08.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 >> 172.21.15.122:0/2833893261 conn(0x7f48d4077b00 msgr2=0x7f48d40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:08.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 shutdown_connections 2024-04-02T13:46:08.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:08.932+0000 7f48db869640 1 -- 172.21.15.122:0/2833893261 wait complete. 2024-04-02T13:46:08.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:46:08.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:46:08.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 395260 3.55 MB/sec execute 449 sec latency 19.625 ms 2024-04-02T13:46:09.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 -- 172.21.15.122:0/3198108401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1069d0 msgr2=0x7fbccc106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/3198108401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1069d0 0x7fbccc106db0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009930 tx=0x7fbcc802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:09.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 -- 172.21.15.122:0/3198108401 shutdown_connections 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/3198108401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc101380 0x7fbccc10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/3198108401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc1009c0 0x7fbccc100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/3198108401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1069d0 0x7fbccc106db0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 -- 172.21.15.122:0/3198108401 >> 172.21.15.122:0/3198108401 conn(0x7fbccc077aa0 msgr2=0x7fbccc077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 -- 172.21.15.122:0/3198108401 shutdown_connections 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 -- 172.21.15.122:0/3198108401 wait complete. 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.060+0000 7fbcd34e4640 1 Processor -- start 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 -- start start 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc101380 0x7fbccc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc1069d0 0x7fbccc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10fc60 con 0x7fbccc1009c0 2024-04-02T13:46:09.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10fae0 con 0x7fbccc1069d0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd34e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc10fde0 con 0x7fbccc101380 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34632/0 (socket says 172.21.15.122:34632) 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 -- 172.21.15.122:0/2181949103 learned_addr learned my addr 172.21.15.122:0/2181949103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd0a7f640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc101380 0x7fbccc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1a81640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc1069d0 0x7fbccc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 -- 172.21.15.122:0/2181949103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc101380 msgr2=0x7fbccc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc101380 0x7fbccc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 -- 172.21.15.122:0/2181949103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc1069d0 msgr2=0x7fbccc19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc1069d0 0x7fbccc19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 -- 172.21.15.122:0/2181949103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc19f620 con 0x7fbccc1009c0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcd1280640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036040 tx=0x7fbcc8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.061+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8043020 con 0x7fbccc1009c0 2024-04-02T13:46:09.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.062+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc80054e0 con 0x7fbccc1009c0 2024-04-02T13:46:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.062+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803e450 con 0x7fbccc1009c0 2024-04-02T13:46:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.062+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc19f8b0 con 0x7fbccc1009c0 2024-04-02T13:46:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.062+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1a71a0 con 0x7fbccc1009c0 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.063+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc068d40 con 0x7fbccc1009c0 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.064+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbcc8046020 con 0x7fbccc1009c0 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.065+0000 7fbcc27fc640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 0x7fbcb80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.065+0000 7fbcd0a7f640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 0x7fbcb80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.065+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fbcc80f3470 con 0x7fbccc1009c0 2024-04-02T13:46:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.065+0000 7fbcd0a7f640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 0x7fbcb80a6260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0095f0 tx=0x7fbcbc005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.069+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbcc804d020 con 0x7fbccc1009c0 2024-04-02T13:46:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.314+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbccc19b980 con 0x7fbccc1009c0 2024-04-02T13:46:09.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.315+0000 7fbcc27fc640 1 -- 172.21.15.122:0/2181949103 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11942 (secure 0 0 0) 0x7fbcc80bf7e0 con 0x7fbccc1009c0 2024-04-02T13:46:09.318 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:46:09.319 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:42:56.407937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:46:09.319 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 40 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 msgr2=0x7fbcb80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 0x7fbcb80a6260 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0095f0 tx=0x7fbcbc005f90 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 msgr2=0x7fbccc19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036040 tx=0x7fbcc8004040 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 shutdown_connections 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbcb80a3da0 0x7fbcb80a6260 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbccc1069d0 0x7fbccc19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbccc101380 0x7fbccc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 --2- 172.21.15.122:0/2181949103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc1009c0 0x7fbccc19a630 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 >> 172.21.15.122:0/2181949103 conn(0x7fbccc077aa0 msgr2=0x7fbccc0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 shutdown_connections 2024-04-02T13:46:09.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.317+0000 7fbcd34e4640 1 -- 172.21.15.122:0/2181949103 wait complete. 2024-04-02T13:46:09.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:46:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 -- 172.21.15.122:0/1209578728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 msgr2=0x7fac1c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1209578728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c100df0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fac10009930 tx=0x7fac1002bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 -- 172.21.15.122:0/1209578728 shutdown_connections 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1209578728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 0x7fac1c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1209578728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c100df0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1209578728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 0x7fac1c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 -- 172.21.15.122:0/1209578728 >> 172.21.15.122:0/1209578728 conn(0x7fac1c077b00 msgr2=0x7fac1c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 -- 172.21.15.122:0/1209578728 shutdown_connections 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.481+0000 7fac20bcd640 1 -- 172.21.15.122:0/1209578728 wait complete. 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 Processor -- start 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 -- start start 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 0x7fac1c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 0x7fac1c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c110de0 con 0x7fac1c100970 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c110c60 con 0x7fac1c106980 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac20bcd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c110f60 con 0x7fac1c101330 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34654/0 (socket says 172.21.15.122:34654) 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 -- 172.21.15.122:0/1482301790 learned_addr learned my addr 172.21.15.122:0/1482301790 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac0bfff640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 0x7fac1c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 0x7fac1c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 -- 172.21.15.122:0/1482301790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 msgr2=0x7fac1c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 0x7fac1c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 -- 172.21.15.122:0/1482301790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 msgr2=0x7fac1c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 0x7fac1c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.482+0000 7fac1a59c640 1 -- 172.21.15.122:0/1482301790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19d330 con 0x7fac1c100970 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac1a59c640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00ebc0 tx=0x7fac0c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac0b7fe640 1 -- 172.21.15.122:0/1482301790 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c010630 con 0x7fac1c100970 2024-04-02T13:46:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19d620 con 0x7fac1c100970 2024-04-02T13:46:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac0b7fe640 1 -- 172.21.15.122:0/1482301790 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c010cb0 con 0x7fac1c100970 2024-04-02T13:46:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac0b7fe640 1 -- 172.21.15.122:0/1482301790 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c018bd0 con 0x7fac1c100970 2024-04-02T13:46:09.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.483+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a4eb0 con 0x7fac1c100970 2024-04-02T13:46:09.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.485+0000 7fac0b7fe640 1 -- 172.21.15.122:0/1482301790 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fac0c0107d0 con 0x7fac1c100970 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.485+0000 7fac237b1740 1 Processor -- start 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 -- start start 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0b7fe640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628e17a0 0x5618628e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628eaca0 0x5618628ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5618628457b0 con 0x5618628e20c0 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5618627e4570 con 0x5618628e17a0 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac237b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561862841760 con 0x5618628eaca0 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0b7fe640 1 -- 172.21.15.122:0/1482301790 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fac0c0d1220 con 0x7fac1c100970 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34658/0 (socket says 172.21.15.122:34658) 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 -- 172.21.15.122:0/326868535 learned_addr learned my addr 172.21.15.122:0/326868535 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac1a59c640 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628e17a0 0x5618628e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628eaca0 0x5618628ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 -- 172.21.15.122:0/326868535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628eaca0 msgr2=0x5618628ed090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628eaca0 0x5618628ed090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 -- 172.21.15.122:0/326868535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628e17a0 msgr2=0x5618628e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628e17a0 0x5618628e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 -- 172.21.15.122:0/326868535 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618625593f0 con 0x5618628e20c0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 0x7fac040a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fac1400baa0 tx=0x7fac14004230 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.486+0000 7fac0bfff640 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b4d4f66742763a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac08ff9640 1 -- 172.21.15.122:0/326868535 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10045070 con 0x5618628e20c0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac08ff9640 1 -- 172.21.15.122:0/326868535 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1002c880 con 0x5618628e20c0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac08ff9640 1 -- 172.21.15.122:0/326868535 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10004ac0 con 0x5618628e20c0 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 -- 172.21.15.122:0/326868535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 msgr2=0x5618628ea760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fac10000c00 tx=0x7fac100095e0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 -- 172.21.15.122:0/326868535 shutdown_connections 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628eaca0 0x5618628ed090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628ea760 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 --2- 172.21.15.122:0/326868535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628e17a0 0x5618628e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.487+0000 7fac237b1740 1 -- 172.21.15.122:0/326868535 >> 172.21.15.122:0/326868535 conn(0x5618628dbed0 msgr2=0x5618628dc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- 172.21.15.122:0/326868535 shutdown_connections 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- 172.21.15.122:0/326868535 wait complete. 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- start start 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 Processor -- start 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e17a0 0x5618628ea690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5618628eda30 con 0x5618628e20c0 2024-04-02T13:46:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561862800e80 con 0x5618628eaca0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618628edbb0 con 0x5618628e17a0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac237b1740 10 client.0 _ll_register_callbacks cb 0x56186283caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e17a0 0x5618628ea690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34660/0 (socket says 172.21.15.122:34660) 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac1ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.488+0000 7fac0bfff640 1 -- 172.21.15.122:0/4039898443 learned_addr learned my addr 172.21.15.122:0/4039898443 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 -- 172.21.15.122:0/4039898443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e17a0 msgr2=0x5618628ea690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e17a0 0x5618628ea690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 -- 172.21.15.122:0/4039898443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 msgr2=0x5618628e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 -- 172.21.15.122:0/4039898443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618626237d0 con 0x5618628e20c0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac0bfff640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac10036040 tx=0x7fac1002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6bc77ba83f9175 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10045040 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fac100052f0 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1002c880 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fac1005b050 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 10 client.18801 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5618628f9a50 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac1002c880 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fac1004ace0 con 0x5618628e20c0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 10 client.18801 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 10 client.18801 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 20 client.18801 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618628fada0 0x5618628fd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 4 client.18801 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.489+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561862852cf0 con 0x5618628fada0 2024-04-02T13:46:09.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac1a59c640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618628fada0 0x5618628fd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac01ffb640 10 client.18801 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac1a59c640 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618628fada0 0x5618628fd190 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7213bdf993ed53e1 server_cookie=5612c4e67c303d04 in_seq=0 out_seq=0 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac01ffb640 1 -- 172.21.15.122:0/4039898443 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561862852cf0 con 0x5618628fada0 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac01ffb640 10 client.18801 handle_command_reply: tid=0 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.490+0000 7fac237b1740 1 client.18801 shutdown 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 msgr2=0x5618628e7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fac10036040 tx=0x7fac1002c2b0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 shutdown_connections 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618628fada0 0x5618628fd190 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac01ffb640 0 client.18801 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e20c0 0x5618628e7a20 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 --2- 172.21.15.122:0/4039898443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e17a0 0x5618628ea690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 >> 172.21.15.122:0/4039898443 conn(0x5618628dbed0 msgr2=0x5618628ddc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 shutdown_connections 2024-04-02T13:46:09.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 1 -- 172.21.15.122:0/4039898443 wait complete. 2024-04-02T13:46:09.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.491+0000 7fac237b1740 20 client.18801 trim_cache size 0 max 16384 2024-04-02T13:46:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.504+0000 7fac237b1740 1 Processor -- start 2024-04-02T13:46:09.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 -- start start 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1af0f0 con 0x5618628e17a0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1b0c30 con 0x5618628eaca0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac237b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1af7c0 con 0x5618628e20c0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34662/0 (socket says 172.21.15.122:34662) 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 -- 172.21.15.122:0/3464718979 learned_addr learned my addr 172.21.15.122:0/3464718979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac0bfff640 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 -- 172.21.15.122:0/3464718979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 msgr2=0x5618628e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 -- 172.21.15.122:0/3464718979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 msgr2=0x5618628ea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 -- 172.21.15.122:0/3464718979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618628e83a0 con 0x5618628e17a0 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac1a59c640 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac0c09cad0 tx=0x7fac0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4921bfe4e8f4f8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac03fff640 1 -- 172.21.15.122:0/3464718979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0a5990 con 0x5618628e17a0 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.505+0000 7fac03fff640 1 -- 172.21.15.122:0/3464718979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c010ac0 con 0x5618628e17a0 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac03fff640 1 -- 172.21.15.122:0/3464718979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0ce750 con 0x5618628e17a0 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- 172.21.15.122:0/3464718979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 msgr2=0x5618628f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fac0c09cad0 tx=0x7fac0c00cf90 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- 172.21.15.122:0/3464718979 shutdown_connections 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 --2- 172.21.15.122:0/3464718979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628f6920 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- 172.21.15.122:0/3464718979 >> 172.21.15.122:0/3464718979 conn(0x5618628dbed0 msgr2=0x5618628dc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- 172.21.15.122:0/3464718979 shutdown_connections 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- 172.21.15.122:0/3464718979 wait complete. 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.506+0000 7fac237b1740 1 -- start start 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 Processor -- start 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628ebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1b0560 con 0x5618628e17a0 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1afe90 con 0x5618628eaca0 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1ad0f0 con 0x5618628e20c0 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 10 client.0 _ll_register_callbacks cb 0x7fac0c0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34672/0 (socket says 172.21.15.122:34672) 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 -- 172.21.15.122:0/2524851500 learned_addr learned my addr 172.21.15.122:0/2524851500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1ad9d640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac0bfff640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628ebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 -- 172.21.15.122:0/2524851500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 msgr2=0x5618628ebf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628ebf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 -- 172.21.15.122:0/2524851500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 msgr2=0x5618628ec450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ec450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 -- 172.21.15.122:0/2524851500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618628f0f00 con 0x5618628e17a0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac1a59c640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac0c052010 tx=0x7fac0c09c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3335ef05e6e7e190 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0b5020 con 0x5618628e17a0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fac0c09b0d0 con 0x5618628e17a0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.507+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac0c010ac0 con 0x5618628e17a0 2024-04-02T13:46:09.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fac0c010e60 con 0x5618628e17a0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 10 client.18807 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5618629612a0 con 0x5618628e17a0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac0c010ac0 con 0x5618628e17a0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fac0c0bd170 con 0x5618628e17a0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 10 client.18807 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 10 client.18807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 20 client.18807 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618629619c0 0x561862963de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 4 client.18807 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56186284d040 con 0x5618629619c0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac0bfff640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618629619c0 0x561862963de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac0bfff640 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618629619c0 0x561862963de0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=88b1fc28977df4f7 server_cookie=40a618c02eaeb069 in_seq=0 out_seq=0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 10 client.18807 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 1 -- 172.21.15.122:0/2524851500 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56186284d040 con 0x5618629619c0 2024-04-02T13:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac01ffb640 10 client.18807 handle_command_reply: tid=0 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.508+0000 7fac237b1740 1 client.18807 shutdown 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 msgr2=0x5618628eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fac0c052010 tx=0x7fac0c09c990 comp rx=0 tx=0).stop 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 shutdown_connections 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5618629619c0 0x561862963de0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618628eaca0 0x5618628ec450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618628e20c0 0x5618628ebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac01ffb640 0 client.18807 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 --2- 172.21.15.122:0/2524851500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618628e17a0 0x5618628eb9d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 >> 172.21.15.122:0/2524851500 conn(0x5618628dbed0 msgr2=0x5618628e0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 shutdown_connections 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 1 -- 172.21.15.122:0/2524851500 wait complete. 2024-04-02T13:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac237b1740 20 client.18807 trim_cache size 0 max 16384 2024-04-02T13:46:09.520 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:09.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 msgr2=0x7fac040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 0x7fac040a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fac1400baa0 tx=0x7fac14004230 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 msgr2=0x7fac1c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00ebc0 tx=0x7fac0c00c6a0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 shutdown_connections 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fac040a3d20 0x7fac040a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fac1c106980 0x7fac1c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac1c101330 0x7fac1c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 --2- 172.21.15.122:0/1482301790 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c100970 0x7fac1c198340 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 >> 172.21.15.122:0/1482301790 conn(0x7fac1c077b00 msgr2=0x7fac1c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 shutdown_connections 2024-04-02T13:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.509+0000 7fac20bcd640 1 -- 172.21.15.122:0/1482301790 wait complete. 2024-04-02T13:46:09.524 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 -- 172.21.15.122:0/772188952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20100d30 msgr2=0x7fec20104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 --2- 172.21.15.122:0/772188952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20100d30 0x7fec20104d70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fec14009900 tx=0x7fec1402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 -- 172.21.15.122:0/772188952 shutdown_connections 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 --2- 172.21.15.122:0/772188952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20105450 0x7fec2010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 --2- 172.21.15.122:0/772188952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20100d30 0x7fec20104d70 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 --2- 172.21.15.122:0/772188952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec20100380 0x7fec20100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 -- 172.21.15.122:0/772188952 >> 172.21.15.122:0/772188952 conn(0x7fec20076f20 msgr2=0x7fec200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 -- 172.21.15.122:0/772188952 shutdown_connections 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.639+0000 7fec25eba640 1 -- 172.21.15.122:0/772188952 wait complete. 2024-04-02T13:46:09.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 Processor -- start 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 -- start start 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 0x7fec2019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 0x7fec2019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010fc80 con 0x7fec20105450 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010fb00 con 0x7fec2019eeb0 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec25eba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec2010fe00 con 0x7fec20100380 2024-04-02T13:46:09.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 0x7fec2019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47378/0 (socket says 172.21.15.122:47378) 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec1f7fe640 1 -- 172.21.15.122:0/2824696538 learned_addr learned my addr 172.21.15.122:0/2824696538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.640+0000 7fec1ffff640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 0x7fec2019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 -- 172.21.15.122:0/2824696538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 msgr2=0x7fec2019f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 0x7fec2019f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 -- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 msgr2=0x7fec2019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 0x7fec2019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 -- 172.21.15.122:0/2824696538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2019f9a0 con 0x7fec20100380 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1ffff640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 0x7fec2019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1effd640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 0x7fec2019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec1000e9f0 tx=0x7fec1000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1cff9640 1 -- 172.21.15.122:0/2824696538 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1000cd80 con 0x7fec20100380 2024-04-02T13:46:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2019fc90 con 0x7fec20100380 2024-04-02T13:46:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a74a0 con 0x7fec20100380 2024-04-02T13:46:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1cff9640 1 -- 172.21.15.122:0/2824696538 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec10010910 con 0x7fec20100380 2024-04-02T13:46:09.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.641+0000 7fec1cff9640 1 -- 172.21.15.122:0/2824696538 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100188f0 con 0x7fec20100380 2024-04-02T13:46:09.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 Processor -- start 2024-04-02T13:46:09.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 -- start start 2024-04-02T13:46:09.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1cff9640 1 -- 172.21.15.122:0/2824696538 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fec10010430 con 0x7fec20100380 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabadc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbaba347b0 con 0x55bbabad07a0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bbab9d3570 con 0x55bbabad9ca0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec28b76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bbaba30760 con 0x55bbabad10c0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34696/0 (socket says 172.21.15.122:34696) 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 -- 172.21.15.122:0/1439968657 learned_addr learned my addr 172.21.15.122:0/1439968657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1effd640 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabadc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 -- 172.21.15.122:0/1439968657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 msgr2=0x55bbabad9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 -- 172.21.15.122:0/1439968657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 msgr2=0x55bbabadc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.644+0000 7fec1ffff640 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabadc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec1ffff640 1 -- 172.21.15.122:0/1439968657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab7483f0 con 0x55bbabad07a0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec1cff9640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 0x7fec080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 0x7fec080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec1ffff640 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e6e0 tx=0x7fec1800eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47de63c31356efaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec1cff9640 1 -- 172.21.15.122:0/2824696538 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fec100d0eb0 con 0x7fec20100380 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec05ffb640 1 -- 172.21.15.122:0/1439968657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1802d070 con 0x55bbabad07a0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec05ffb640 1 -- 172.21.15.122:0/1439968657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec1800dc20 con 0x55bbabad07a0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.645+0000 7fec05ffb640 1 -- 172.21.15.122:0/1439968657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800c740 con 0x55bbabad07a0 2024-04-02T13:46:09.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.646+0000 7fec28b76740 1 -- 172.21.15.122:0/1439968657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 msgr2=0x55bbabad0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.646+0000 7fec1f7fe640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 0x7fec080a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec100ce270 tx=0x7fec100d4a00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.646+0000 7fec28b76740 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fec1800e6e0 tx=0x7fec1800eba0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 -- 172.21.15.122:0/1439968657 shutdown_connections 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabadc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 --2- 172.21.15.122:0/1439968657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad07a0 0x55bbabad0b80 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 -- 172.21.15.122:0/1439968657 >> 172.21.15.122:0/1439968657 conn(0x55bbabacaed0 msgr2=0x55bbabacb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.647+0000 7fec28b76740 1 -- 172.21.15.122:0/1439968657 shutdown_connections 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.648+0000 7fec28b76740 1 -- 172.21.15.122:0/1439968657 wait complete. 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.648+0000 7fec28b76740 1 -- start start 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 Processor -- start 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad07a0 0x55bbabad9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 0x55bbabad7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bbabadca90 con 0x55bbabad9ca0 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bbab9efe80 con 0x55bbabad07a0 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bbabadcc10 con 0x55bbabad10c0 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.649+0000 7fec28b76740 10 client.0 _ll_register_callbacks cb 0x55bbaba2baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 0x55bbabad7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 0x55bbabad7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34710/0 (socket says 172.21.15.122:34710) 2024-04-02T13:46:09.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1ffff640 1 -- 172.21.15.122:0/3675580928 learned_addr learned my addr 172.21.15.122:0/3675580928 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad07a0 0x55bbabad9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 -- 172.21.15.122:0/3675580928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad07a0 msgr2=0x55bbabad9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad07a0 0x55bbabad9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 -- 172.21.15.122:0/3675580928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 msgr2=0x55bbabad7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 0x55bbabad7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.650+0000 7fec1effd640 1 -- 172.21.15.122:0/3675580928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab8127d0 con 0x55bbabad10c0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.651+0000 7fec1effd640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad6a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec14036040 tx=0x7fec1402cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244dfbeee67686fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.651+0000 7febfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.651+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14042030 con 0x55bbabad10c0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.651+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fec14004240 con 0x55bbabad10c0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.651+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec1800dc20 con 0x55bbabad10c0 2024-04-02T13:46:09.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.652+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fec14004440 con 0x55bbabad10c0 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.652+0000 7fec28b76740 10 client.27613 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.652+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bbabae8ab0 con 0x55bbabad10c0 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.652+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec1800dc20 con 0x55bbabad10c0 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.652+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fec14003950 con 0x55bbabad10c0 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 10 client.27613 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 10 client.27613 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 20 client.27613 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabaec450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 4 client.27613 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:09.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.653+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbaba41cf0 con 0x55bbabaea030 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.654+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabaec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.655+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabaec450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ac6070d92db561c server_cookie=f914c96da319556e in_seq=0 out_seq=0 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.655+0000 7febfeffd640 10 client.27613 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.656+0000 7febfeffd640 1 -- 172.21.15.122:0/3675580928 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bbaba41cf0 con 0x55bbabaea030 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.656+0000 7febfeffd640 10 client.27613 handle_command_reply: tid=0 2024-04-02T13:46:09.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.656+0000 7fec28b76740 1 client.27613 shutdown 2024-04-02T13:46:09.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.657+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 msgr2=0x55bbabad6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.657+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad6a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fec14036040 tx=0x7fec1402cab0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 shutdown_connections 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabaec450 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad9ca0 0x55bbabad7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7febfeffd640 0 client.27613 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad10c0 0x55bbabad6a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 --2- 172.21.15.122:0/3675580928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad07a0 0x55bbabad9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 >> 172.21.15.122:0/3675580928 conn(0x55bbabacaed0 msgr2=0x55bbabaccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 shutdown_connections 2024-04-02T13:46:09.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.658+0000 7fec28b76740 1 -- 172.21.15.122:0/3675580928 wait complete. 2024-04-02T13:46:09.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.659+0000 7fec28b76740 20 client.27613 trim_cache size 0 max 16384 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 Processor -- start 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 -- start start 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabae5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabaf2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c010120 con 0x55bbabad10c0 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c011c60 con 0x55bbabad9ca0 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec28b76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c0107f0 con 0x55bbabad07a0 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34712/0 (socket says 172.21.15.122:34712) 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec1effd640 1 -- 172.21.15.122:0/2289479508 learned_addr learned my addr 172.21.15.122:0/2289479508 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec1ffff640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabae5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.681+0000 7fec1ffff640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabaf2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 -- 172.21.15.122:0/2289479508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 msgr2=0x55bbabae5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabae5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 -- 172.21.15.122:0/2289479508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 msgr2=0x55bbabaf2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabaf2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 -- 172.21.15.122:0/2289479508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbabad0ef0 con 0x55bbabad10c0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1ffff640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabaf2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec1effd640 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fec14003ee0 tx=0x7fec14004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6706c6eabfc13135 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec057fa640 1 -- 172.21.15.122:0/2289479508 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14042070 con 0x55bbabad10c0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec057fa640 1 -- 172.21.15.122:0/2289479508 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec140035f0 con 0x55bbabad10c0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.682+0000 7fec057fa640 1 -- 172.21.15.122:0/2289479508 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14034a70 con 0x55bbabad10c0 2024-04-02T13:46:09.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- 172.21.15.122:0/2289479508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 msgr2=0x55bbabaf1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fec14003ee0 tx=0x7fec14004e30 comp rx=0 tx=0).stop 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- 172.21.15.122:0/2289479508 shutdown_connections 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabaf2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabaf1ce0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 --2- 172.21.15.122:0/2289479508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabae5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- 172.21.15.122:0/2289479508 >> 172.21.15.122:0/2289479508 conn(0x55bbabacaed0 msgr2=0x55bbabad1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- 172.21.15.122:0/2289479508 shutdown_connections 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- 172.21.15.122:0/2289479508 wait complete. 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 -- start start 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.683+0000 7fec28b76740 1 Processor -- start 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabadaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabae0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c011590 con 0x55bbabad10c0 2024-04-02T13:46:09.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c010ec0 con 0x55bbabad9ca0 2024-04-02T13:46:09.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c00e120 con 0x55bbabad07a0 2024-04-02T13:46:09.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 10 client.0 _ll_register_callbacks cb 0x7fec14005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34714/0 (socket says 172.21.15.122:34714) 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 -- 172.21.15.122:0/3747816233 learned_addr learned my addr 172.21.15.122:0/3747816233 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabadaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabae0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 -- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 msgr2=0x55bbabadaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabadaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 -- 172.21.15.122:0/3747816233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 msgr2=0x55bbabae0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabae0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 -- 172.21.15.122:0/3747816233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbabae1400 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabae0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec1effd640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fec14003d70 tx=0x7fec1402cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eece3b0e8fd944c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7febfeffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14042040 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fec140347e0 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.684+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec140035f0 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fec14066050 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 10 client.18825 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bbabae26e0 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec140035f0 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fec140054e0 con 0x55bbabad10c0 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 10 client.18825 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 10 client.18825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:46:09.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 20 client.18825 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabb52980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 4 client.18825 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbaba3c040 con 0x55bbabaea030 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.685+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabb52980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7febfeffd640 10 client.18825 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec1ffff640 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabb52980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d7da7254ada9f979 server_cookie=e7bde00aa68be261 in_seq=0 out_seq=0 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7febfeffd640 1 -- 172.21.15.122:0/3747816233 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbaba3c040 con 0x55bbabaea030 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7febfeffd640 10 client.18825 handle_command_reply: tid=0 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 client.18825 shutdown 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 msgr2=0x55bbabadaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fec14003d70 tx=0x7fec1402cab0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 shutdown_connections 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bbabad9ca0 0x55bbabae0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55bbabaea030 0x55bbabb52980 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bbabad10c0 0x55bbabadaf70 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 --2- 172.21.15.122:0/3747816233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bbabad07a0 0x55bbabadaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 >> 172.21.15.122:0/3747816233 conn(0x55bbabacaed0 msgr2=0x55bbabac96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7febfeffd640 0 client.18825 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.686+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 shutdown_connections 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec28b76740 1 -- 172.21.15.122:0/3747816233 wait complete. 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec28b76740 20 client.18825 trim_cache size 0 max 16384 2024-04-02T13:46:09.696 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 msgr2=0x7fec080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 0x7fec080a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec100ce270 tx=0x7fec100d4a00 comp rx=0 tx=0).stop 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 msgr2=0x7fec2019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec1000e9f0 tx=0x7fec1000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 shutdown_connections 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fec080a3d20 0x7fec080a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fec2019eeb0 0x7fec2019f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec20105450 0x7fec2019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 --2- 172.21.15.122:0/2824696538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec20100380 0x7fec2019a640 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 >> 172.21.15.122:0/2824696538 conn(0x7fec20076f20 msgr2=0x7fec20103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 shutdown_connections 2024-04-02T13:46:09.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.687+0000 7fec25eba640 1 -- 172.21.15.122:0/2824696538 wait complete. 2024-04-02T13:46:09.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:09 smithi122 ceph-mon[32114]: pgmap v341: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:09.701 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:09 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2181949103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:09.702 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:46:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:09 smithi187 ceph-mon[36064]: pgmap v341: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:09 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2181949103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:09 smithi188 ceph-mon[36117]: pgmap v341: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:09 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2181949103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:09.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 -- 172.21.15.122:0/2629652553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764068ce0 msgr2=0x7ff764069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 --2- 172.21.15.122:0/2629652553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764068ce0 0x7ff764069160 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff750009930 tx=0x7ff75002bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 -- 172.21.15.122:0/2629652553 shutdown_connections 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 --2- 172.21.15.122:0/2629652553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff7640696a0 0x7ff76410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 --2- 172.21.15.122:0/2629652553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764068ce0 0x7ff764069160 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 --2- 172.21.15.122:0/2629652553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff764107a80 0x7ff764107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 -- 172.21.15.122:0/2629652553 >> 172.21.15.122:0/2629652553 conn(0x7ff764077b00 msgr2=0x7ff764077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 -- 172.21.15.122:0/2629652553 shutdown_connections 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 -- 172.21.15.122:0/2629652553 wait complete. 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.818+0000 7ff76b196640 1 Processor -- start 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 -- start start 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff764068ce0 0x7ff7641983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 0x7ff764198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff764111e90 con 0x7ff764107a80 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff764111d10 con 0x7ff764068ce0 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff764112010 con 0x7ff7640696a0 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34726/0 (socket says 172.21.15.122:34726) 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 -- 172.21.15.122:0/2480833877 learned_addr learned my addr 172.21.15.122:0/2480833877 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff763fff640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 0x7ff764198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff768f32640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff764068ce0 0x7ff7641983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 -- 172.21.15.122:0/2480833877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 msgr2=0x7ff764198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 0x7ff764198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 -- 172.21.15.122:0/2480833877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff764068ce0 msgr2=0x7ff7641983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff764068ce0 0x7ff7641983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 -- 172.21.15.122:0/2480833877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76419d3e0 con 0x7ff764107a80 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff763fff640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 0x7ff764198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff769733640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00efd0 tx=0x7ff75c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff761ffb640 1 -- 172.21.15.122:0/2480833877 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c003b70 con 0x7ff764107a80 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff761ffb640 1 -- 172.21.15.122:0/2480833877 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75c003d10 con 0x7ff764107a80 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.819+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76419d6d0 con 0x7ff764107a80 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.820+0000 7ff761ffb640 1 -- 172.21.15.122:0/2480833877 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0109f0 con 0x7ff764107a80 2024-04-02T13:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.820+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641a4f60 con 0x7ff764107a80 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.822+0000 7ff761ffb640 1 -- 172.21.15.122:0/2480833877 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff75c004000 con 0x7ff764107a80 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.822+0000 7ff76dd76740 1 Processor -- start 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.822+0000 7ff76dd76740 1 -- start start 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff761ffb640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 0x5649f3907b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f3910ca0 0x5649f3913090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5649f386b7b0 con 0x5649f39080c0 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5649f380a570 con 0x5649f39077a0 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff76dd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649f3867760 con 0x5649f3910ca0 2024-04-02T13:46:09.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff761ffb640 1 -- 172.21.15.122:0/2480833877 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff75c0d0ff0 con 0x7ff764107a80 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 0x5649f3907b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34738/0 (socket says 172.21.15.122:34738) 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 -- 172.21.15.122:0/4061129606 learned_addr learned my addr 172.21.15.122:0/4061129606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff769733640 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f3910ca0 0x5649f3913090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 -- 172.21.15.122:0/4061129606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f3910ca0 msgr2=0x5649f3913090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f3910ca0 0x5649f3913090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 -- 172.21.15.122:0/4061129606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 msgr2=0x5649f3910760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff768f32640 1 -- 172.21.15.122:0/4061129606 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f357f3f0 con 0x5649f39077a0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.823+0000 7ff763fff640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 0x7ff7540a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750005e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff768f32640 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 0x5649f3907b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff758009c00 tx=0x7ff758011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73dfa37aef7c557 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff74effd640 1 -- 172.21.15.122:0/4061129606 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004050 con 0x5649f39077a0 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff74effd640 1 -- 172.21.15.122:0/4061129606 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff758005500 con 0x5649f39077a0 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff74effd640 1 -- 172.21.15.122:0/4061129606 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758004d40 con 0x5649f39077a0 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff76dd76740 1 -- 172.21.15.122:0/4061129606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 msgr2=0x5649f3907b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.824+0000 7ff76dd76740 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 0x5649f3907b80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff758009c00 tx=0x7ff758011810 comp rx=0 tx=0).stop 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 -- 172.21.15.122:0/4061129606 shutdown_connections 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f3910ca0 0x5649f3913090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f3910760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 --2- 172.21.15.122:0/4061129606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39077a0 0x5649f3907b80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 -- 172.21.15.122:0/4061129606 >> 172.21.15.122:0/4061129606 conn(0x5649f3901ed0 msgr2=0x5649f39022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 -- 172.21.15.122:0/4061129606 shutdown_connections 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 -- 172.21.15.122:0/4061129606 wait complete. 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 -- start start 2024-04-02T13:46:09.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.825+0000 7ff76dd76740 1 Processor -- start 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3910640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f390d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5649f3913a90 con 0x5649f3910ca0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5649f3826e80 con 0x5649f39080c0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5649f3913c10 con 0x5649f39077a0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 10 client.0 _ll_register_callbacks cb 0x5649f3862aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34754/0 (socket says 172.21.15.122:34754) 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 -- 172.21.15.122:0/216791987 learned_addr learned my addr 172.21.15.122:0/216791987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff763fff640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f390d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff768f32640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3910640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 -- 172.21.15.122:0/216791987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 msgr2=0x5649f3910640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3910640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 -- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 msgr2=0x5649f390d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f390d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 -- 172.21.15.122:0/216791987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f36497d0 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff768f32640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3910640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff763fff640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f390d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff769733640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e1c827c6b2ca49 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff743fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0b6060 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff75c0991a0 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.826+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff758005500 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff75c09de60 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 10 client.18837 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5649f391fa40 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff758005500 con 0x5649f3910ca0 2024-04-02T13:46:09.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff75c01f020 con 0x5649f3910ca0 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 10 client.18837 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 10 client.18837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 20 client.18837 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3920e50 0x5649f39232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 4 client.18837 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f3878cf0 con 0x5649f3920e50 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.827+0000 7ff768f32640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3920e50 0x5649f39232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.828+0000 7ff743fff640 10 client.18837 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.828+0000 7ff768f32640 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3920e50 0x5649f39232a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6302da9494aeda3b server_cookie=44483b4f333bde79 in_seq=0 out_seq=0 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.828+0000 7ff743fff640 1 -- 172.21.15.122:0/216791987 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649f3878cf0 con 0x5649f3920e50 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.828+0000 7ff743fff640 10 client.18837 handle_command_reply: tid=0 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.828+0000 7ff76dd76740 1 client.18837 shutdown 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 msgr2=0x5649f390df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c004d60 comp rx=0 tx=0).stop 2024-04-02T13:46:09.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 shutdown_connections 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390df70 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3920e50 0x5649f39232a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f390d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 --2- 172.21.15.122:0/216791987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3910640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff743fff640 0 client.18837 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 >> 172.21.15.122:0/216791987 conn(0x5649f3901ed0 msgr2=0x5649f3903c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 shutdown_connections 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 1 -- 172.21.15.122:0/216791987 wait complete. 2024-04-02T13:46:09.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.829+0000 7ff76dd76740 20 client.18837 trim_cache size 0 max 16384 2024-04-02T13:46:09.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 Processor -- start 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 -- start start 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3923240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 0x5649f3910100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641af150 con 0x5649f39080c0 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641b0c90 con 0x5649f3910ca0 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff76dd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641af820 con 0x5649f39077a0 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff763fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff769733640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 0x5649f3910100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff763fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34760/0 (socket says 172.21.15.122:34760) 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff768f32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3923240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.844+0000 7ff763fff640 1 -- 172.21.15.122:0/3597520036 learned_addr learned my addr 172.21.15.122:0/3597520036 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 -- 172.21.15.122:0/3597520036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 msgr2=0x5649f3923240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3923240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 -- 172.21.15.122:0/3597520036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 msgr2=0x5649f391c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 -- 172.21.15.122:0/3597520036 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f39095d0 con 0x5649f3910ca0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff763fff640 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff768f32640 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3923240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff769733640 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 0x5649f3910100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff75c098f90 tx=0x7ff75c0b61e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc68c09290d5b1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff74e7fc640 1 -- 172.21.15.122:0/3597520036 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0b6040 con 0x5649f3910ca0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff74e7fc640 1 -- 172.21.15.122:0/3597520036 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75c009820 con 0x5649f3910ca0 2024-04-02T13:46:09.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.845+0000 7ff74e7fc640 1 -- 172.21.15.122:0/3597520036 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0ce560 con 0x5649f3910ca0 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- 172.21.15.122:0/3597520036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 msgr2=0x5649f3910100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 0x5649f3910100 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff75c098f90 tx=0x7ff75c0b61e0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- 172.21.15.122:0/3597520036 shutdown_connections 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f3910ca0 0x5649f3910100 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f39080c0 0x5649f391c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3597520036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3923240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- 172.21.15.122:0/3597520036 >> 172.21.15.122:0/3597520036 conn(0x5649f3901ed0 msgr2=0x5649f390d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- 172.21.15.122:0/3597520036 shutdown_connections 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- 172.21.15.122:0/3597520036 wait complete. 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 -- start start 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.846+0000 7ff76dd76740 1 Processor -- start 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3918e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f39193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641b05c0 con 0x5649f3910ca0 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641afef0 con 0x5649f39080c0 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641ad150 con 0x5649f39077a0 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff76dd76740 10 client.0 _ll_register_callbacks cb 0x7ff75c098ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34774/0 (socket says 172.21.15.122:34774) 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 -- 172.21.15.122:0/3256327916 learned_addr learned my addr 172.21.15.122:0/3256327916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff763fff640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f39193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff768f32640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3918e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 -- 172.21.15.122:0/3256327916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 msgr2=0x5649f3918e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3918e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 -- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 msgr2=0x5649f39193a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f39193a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 -- 172.21.15.122:0/3256327916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f390b450 con 0x5649f3910ca0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff769733640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a20d0477847d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff743fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c0b6040 con 0x5649f3910ca0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.847+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff75c0ce560 con 0x5649f3910ca0 2024-04-02T13:46:09.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75c009820 con 0x5649f3910ca0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff75c09c670 con 0x5649f3910ca0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 10 client.18843 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5649f390b450 con 0x5649f3910ca0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75c009820 con 0x5649f3910ca0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff75c01f020 con 0x5649f3910ca0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 10 client.18843 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 10 client.18843 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 20 client.18843 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3987430 0x5649f3989820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 4 client.18843 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649f3873040 con 0x5649f3987430 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.848+0000 7ff768f32640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3987430 0x5649f3989820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff768f32640 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3987430 0x5649f3989820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5a0f8402790ad333 server_cookie=6393db9c72203f20 in_seq=0 out_seq=0 2024-04-02T13:46:09.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff743fff640 10 client.18843 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff743fff640 1 -- 172.21.15.122:0/3256327916 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649f3873040 con 0x5649f3987430 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff743fff640 10 client.18843 handle_command_reply: tid=0 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 client.18843 shutdown 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 msgr2=0x5649f390ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005c40 tx=0x7ff75c000f40 comp rx=0 tx=0).stop 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 shutdown_connections 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5649f3910ca0 0x5649f390ac50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5649f3987430 0x5649f3989820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5649f39080c0 0x5649f39193a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff743fff640 0 client.18843 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 --2- 172.21.15.122:0/3256327916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5649f39077a0 0x5649f3918e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 >> 172.21.15.122:0/3256327916 conn(0x5649f3901ed0 msgr2=0x5649f390c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 shutdown_connections 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.849+0000 7ff76dd76740 1 -- 172.21.15.122:0/3256327916 wait complete. 2024-04-02T13:46:09.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76dd76740 20 client.18843 trim_cache size 0 max 16384 2024-04-02T13:46:09.860 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:09.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 msgr2=0x7ff7540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 0x7ff7540a61e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750005e80 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 msgr2=0x7ff76419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00efd0 tx=0x7ff75c00c6a0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 shutdown_connections 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff7540a3d20 0x7ff7540a61e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff764107a80 0x7ff76419ccb0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7640696a0 0x7ff764198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 --2- 172.21.15.122:0/2480833877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff764068ce0 0x7ff7641983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 >> 172.21.15.122:0/2480833877 conn(0x7ff764077b00 msgr2=0x7ff7640fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 shutdown_connections 2024-04-02T13:46:09.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.850+0000 7ff76b196640 1 -- 172.21.15.122:0/2480833877 wait complete. 2024-04-02T13:46:09.864 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:46:09.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 397449 3.57 MB/sec execute 450 sec latency 21.057 ms 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 -- 172.21.15.122:0/3574710944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0108fe0 msgr2=0x7fa8d0110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 --2- 172.21.15.122:0/3574710944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0108fe0 0x7fa8d0110bb0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8009ca0 tx=0x7fa8c802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 -- 172.21.15.122:0/3574710944 shutdown_connections 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 --2- 172.21.15.122:0/3574710944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0108fe0 0x7fa8d0110bb0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 --2- 172.21.15.122:0/3574710944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d01048c0 0x7fa8d0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 --2- 172.21.15.122:0/3574710944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d0103f10 0x7fa8d01042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 -- 172.21.15.122:0/3574710944 >> 172.21.15.122:0/3574710944 conn(0x7fa8d00ff770 msgr2=0x7fa8d0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 -- 172.21.15.122:0/3574710944 shutdown_connections 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 -- 172.21.15.122:0/3574710944 wait complete. 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.978+0000 7fa8d8260640 1 Processor -- start 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 -- start start 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d0108fe0 0x7fa8d019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d019ee00 0x7fa8d019f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0114090 con 0x7fa8d0103f10 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0113f10 con 0x7fa8d0108fe0 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d0114210 con 0x7fa8d019ee00 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34800/0 (socket says 172.21.15.122:34800) 2024-04-02T13:46:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/560818689 learned_addr learned my addr 172.21.15.122:0/560818689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d67fd640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d019ee00 0x7fa8d019f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d0108fe0 0x7fa8d019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/560818689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d019ee00 msgr2=0x7fa8d019f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d019ee00 0x7fa8d019f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/560818689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d0108fe0 msgr2=0x7fa8d019aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d0108fe0 0x7fa8d019aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/560818689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d019f8f0 con 0x7fa8d0103f10 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00ea60 tx=0x7fa8cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8c6ffd640 1 -- 172.21.15.122:0/560818689 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00cde0 con 0x7fa8d0103f10 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8c6ffd640 1 -- 172.21.15.122:0/560818689 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8cc0035f0 con 0x7fa8d0103f10 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d019fbe0 con 0x7fa8d0103f10 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.979+0000 7fa8c6ffd640 1 -- 172.21.15.122:0/560818689 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc010670 con 0x7fa8d0103f10 2024-04-02T13:46:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.980+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d01a73d0 con 0x7fa8d0103f10 2024-04-02T13:46:09.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.982+0000 7fa8c6ffd640 1 -- 172.21.15.122:0/560818689 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa8cc0037a0 con 0x7fa8d0103f10 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.982+0000 7fa8dae4a740 1 Processor -- start 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.982+0000 7fa8dae4a740 1 -- start start 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.982+0000 7fa8c6ffd640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 0x7fa8bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8c6ffd640 1 -- 172.21.15.122:0/560818689 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fa8cc0d0f00 con 0x7fa8d0103f10 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34802/0 (socket says 172.21.15.122:34802) 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 -- 172.21.15.122:0/4219260962 learned_addr learned my addr 172.21.15.122:0/4219260962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47486/0 (socket says 172.21.15.122:47486) 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564f088227b0 con 0x564f088bf0c0 2024-04-02T13:46:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564f087c1570 con 0x564f088c7ca0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f0881e760 con 0x564f088be7a0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d67fd640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 0x7fa8bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 -- 172.21.15.122:0/4219260962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 msgr2=0x564f088beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 -- 172.21.15.122:0/4219260962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 msgr2=0x564f088ca090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088ca090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 -- 172.21.15.122:0/4219260962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f085363f0 con 0x564f088bf0c0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.983+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000bbd0 tx=0x7fa8c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe08e9bb7e5db00 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.984+0000 7fa8bbfff640 1 -- 172.21.15.122:0/4219260962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c001c480 con 0x564f088bf0c0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.984+0000 7fa8d67fd640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 0x7fa8bc0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8004af0 tx=0x7fa8c8005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.984+0000 7fa8bbfff640 1 -- 172.21.15.122:0/4219260962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c001cb00 con 0x564f088bf0c0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.984+0000 7fa8bbfff640 1 -- 172.21.15.122:0/4219260962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0011bc0 con 0x564f088bf0c0 2024-04-02T13:46:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 msgr2=0x564f088c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000bbd0 tx=0x7fa8c00107a0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 shutdown_connections 2024-04-02T13:46:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088c7760 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/4219260962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088beb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 >> 172.21.15.122:0/4219260962 conn(0x564f088b8ed0 msgr2=0x564f088b92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 shutdown_connections 2024-04-02T13:46:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- 172.21.15.122:0/4219260962 wait complete. 2024-04-02T13:46:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 -- start start 2024-04-02T13:46:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.985+0000 7fa8dae4a740 1 Processor -- start 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564f088caa90 con 0x564f088be7a0 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564f087dde80 con 0x564f088bf0c0 2024-04-02T13:46:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564f088cac10 con 0x564f088c7ca0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40650/0 (socket says 172.21.15.122:40650) 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d57fb640 1 -- 172.21.15.122:0/1228560948 learned_addr learned my addr 172.21.15.122:0/1228560948 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34808/0 (socket says 172.21.15.122:34808) 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d67fd640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8dae4a740 10 client.0 _ll_register_callbacks cb 0x564f08819aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 msgr2=0x564f088c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/1228560948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 msgr2=0x564f088c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 -- 172.21.15.122:0/1228560948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f086007d0 con 0x564f088be7a0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0ce210 tx=0x7fa8cc09c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8ef05cd054d7b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.986+0000 7fa8b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc09ca60 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fa8cc0ce490 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c001cb00 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8cc0a0e10 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 10 client.18855 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564f088d6ba0 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c001cb00 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa8cc09beb0 con 0x564f088be7a0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 10 client.18855 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 20 client.18855 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f088d7fb0 0x564f088da420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 4 client.18855 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f0882fcf0 con 0x564f088d7fb0 2024-04-02T13:46:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.987+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f088d7fb0 0x564f088da420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.988+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f088d7fb0 0x564f088da420 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=81c6dbd5c5fc1ce0 server_cookie=1062fcc0e3f42540 in_seq=0 out_seq=0 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.988+0000 7fa8b97fa640 10 client.18855 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.988+0000 7fa8b97fa640 1 -- 172.21.15.122:0/1228560948 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f0882fcf0 con 0x564f088d7fb0 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.988+0000 7fa8b97fa640 10 client.18855 handle_command_reply: tid=0 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.988+0000 7fa8dae4a740 1 client.18855 shutdown 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 msgr2=0x564f088c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0ce210 tx=0x7fa8cc09c4b0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 shutdown_connections 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f088d7fb0 0x564f088da420 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8b97fa640 0 client.18855 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088bf0c0 0x564f088c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/1228560948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088be7a0 0x564f088c4b00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 >> 172.21.15.122:0/1228560948 conn(0x564f088b8ed0 msgr2=0x564f088bac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 shutdown_connections 2024-04-02T13:46:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 1 -- 172.21.15.122:0/1228560948 wait complete. 2024-04-02T13:46:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:09.989+0000 7fa8dae4a740 20 client.18855 trim_cache size 0 max 16384 2024-04-02T13:46:10.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 Processor -- start 2024-04-02T13:46:10.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 -- start start 2024-04-02T13:46:10.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088be7a0 0x564f088c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1400 con 0x564f088bf0c0 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b2f40 con 0x564f088be7a0 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b1ad0 con 0x564f088c7ca0 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d67fd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34816/0 (socket says 172.21.15.122:34816) 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 -- 172.21.15.122:0/2992995731 learned_addr learned my addr 172.21.15.122:0/2992995731 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088be7a0 0x564f088c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 -- 172.21.15.122:0/2992995731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 msgr2=0x564f088dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 -- 172.21.15.122:0/2992995731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088be7a0 msgr2=0x564f088c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088be7a0 0x564f088c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.003+0000 7fa8d57fb640 1 -- 172.21.15.122:0/2992995731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f088beef0 con 0x564f088bf0c0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.004+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000bbd0 tx=0x7fa8c000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a12fec201dba8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.004+0000 7fa8bb7fe640 1 -- 172.21.15.122:0/2992995731 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0004020 con 0x564f088bf0c0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.004+0000 7fa8bb7fe640 1 -- 172.21.15.122:0/2992995731 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0005520 con 0x564f088bf0c0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.004+0000 7fa8bb7fe640 1 -- 172.21.15.122:0/2992995731 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c001c770 con 0x564f088bf0c0 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- 172.21.15.122:0/2992995731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 msgr2=0x564f088d3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fa8c000bbd0 tx=0x7fa8c000b860 comp rx=0 tx=0).stop 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- 172.21.15.122:0/2992995731 shutdown_connections 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088c7ca0 0x564f088dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088d3950 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/2992995731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088be7a0 0x564f088c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- 172.21.15.122:0/2992995731 >> 172.21.15.122:0/2992995731 conn(0x564f088b8ed0 msgr2=0x564f088babb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- 172.21.15.122:0/2992995731 shutdown_connections 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- 172.21.15.122:0/2992995731 wait complete. 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 -- start start 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.005+0000 7fa8dae4a740 1 Processor -- start 2024-04-02T13:46:10.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b2870 con 0x564f088bf0c0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01b21a0 con 0x564f088c7ca0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d01af400 con 0x564f088be7a0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 10 client.0 _ll_register_callbacks cb 0x7fa8cc009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34820/0 (socket says 172.21.15.122:34820) 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d67fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d5ffc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 -- 172.21.15.122:0/727901296 learned_addr learned my addr 172.21.15.122:0/727901296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 -- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 msgr2=0x564f088d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 -- 172.21.15.122:0/727901296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 msgr2=0x564f088cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 -- 172.21.15.122:0/727901296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f088cc020 con 0x564f088bf0c0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8d57fb640 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00107e0 tx=0x7fa8c0010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25125a389b79a728 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0039020 con 0x564f088bf0c0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7fa8c0004220 con 0x564f088bf0c0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8c0005520 con 0x564f088bf0c0 2024-04-02T13:46:10.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.006+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa8c0044050 con 0x564f088bf0c0 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 10 client.18861 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564f0893e3a0 con 0x564f088bf0c0 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8c0005520 con 0x564f088bf0c0 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fa8c00117e0 con 0x564f088bf0c0 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 10 client.18861 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 10 client.18861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 20 client.18861 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:10.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f0893e960 0x564f08940d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 4 client.18861 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f0882a040 con 0x564f0893e960 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f0893e960 0x564f08940d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8b97fa640 10 client.18861 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.007+0000 7fa8d5ffc640 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f0893e960 0x564f08940d80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49aade453b22e11b server_cookie=e4b72065d2870a94 in_seq=0 out_seq=0 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8b97fa640 1 -- 172.21.15.122:0/727901296 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f0882a040 con 0x564f0893e960 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8b97fa640 10 client.18861 handle_command_reply: tid=0 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 client.18861 shutdown 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 msgr2=0x564f088cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00107e0 tx=0x7fa8c0010e40 comp rx=0 tx=0).stop 2024-04-02T13:46:10.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 shutdown_connections 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x564f0893e960 0x564f08940d80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564f088c7ca0 0x564f088cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564f088bf0c0 0x564f088cb4a0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8b97fa640 0 client.18861 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 --2- 172.21.15.122:0/727901296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564f088be7a0 0x564f088d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 >> 172.21.15.122:0/727901296 conn(0x564f088b8ed0 msgr2=0x564f088c4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 shutdown_connections 2024-04-02T13:46:10.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 1 -- 172.21.15.122:0/727901296 wait complete. 2024-04-02T13:46:10.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.008+0000 7fa8dae4a740 20 client.18861 trim_cache size 0 max 16384 2024-04-02T13:46:10.025 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:10.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 msgr2=0x7fa8bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 0x7fa8bc0a61e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa8c8004af0 tx=0x7fa8c8005c50 comp rx=0 tx=0).stop 2024-04-02T13:46:10.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 msgr2=0x7fa8d019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00ea60 tx=0x7fa8cc00ef20 comp rx=0 tx=0).stop 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 shutdown_connections 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8bc0a3d20 0x7fa8bc0a61e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8d019ee00 0x7fa8d019f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8d0108fe0 0x7fa8d019aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 --2- 172.21.15.122:0/560818689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8d0103f10 0x7fa8d019a590 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 >> 172.21.15.122:0/560818689 conn(0x7fa8d00ff770 msgr2=0x7fa8d0101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 shutdown_connections 2024-04-02T13:46:10.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.009+0000 7fa8d8260640 1 -- 172.21.15.122:0/560818689 wait complete. 2024-04-02T13:46:10.029 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:46:10.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 -- 172.21.15.122:0/4253040353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 msgr2=0x7ff040100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 --2- 172.21.15.122:0/4253040353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff040100df0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ff034009930 tx=0x7ff03402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 -- 172.21.15.122:0/4253040353 shutdown_connections 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 --2- 172.21.15.122:0/4253040353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 0x7ff04010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 --2- 172.21.15.122:0/4253040353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff040100df0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 --2- 172.21.15.122:0/4253040353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff040106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.135+0000 7ff04794d640 1 -- 172.21.15.122:0/4253040353 >> 172.21.15.122:0/4253040353 conn(0x7ff040077b00 msgr2=0x7ff040077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- 172.21.15.122:0/4253040353 shutdown_connections 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- 172.21.15.122:0/4253040353 wait complete. 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 Processor -- start 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- start start 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 0x7ff0401988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff04019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff040110d90 con 0x7ff040100970 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff040110c10 con 0x7ff040106980 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.136+0000 7ff04794d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff040110f10 con 0x7ff040101330 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff045eea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff04019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34828/0 (socket says 172.21.15.122:34828) 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff044ee8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 0x7ff0401988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 -- 172.21.15.122:0/1987247674 learned_addr learned my addr 172.21.15.122:0/1987247674 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 -- 172.21.15.122:0/1987247674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 msgr2=0x7ff0401988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 0x7ff0401988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 -- 172.21.15.122:0/1987247674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 msgr2=0x7ff04019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff04019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 -- 172.21.15.122:0/1987247674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04019d390 con 0x7ff040100970 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff045eea640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff04019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff0456e9640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ebc0 tx=0x7ff03000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff03e7fc640 1 -- 172.21.15.122:0/1987247674 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030010630 con 0x7ff040100970 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04019d680 con 0x7ff040100970 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff03e7fc640 1 -- 172.21.15.122:0/1987247674 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff030010cb0 con 0x7ff040100970 2024-04-02T13:46:10.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff03e7fc640 1 -- 172.21.15.122:0/1987247674 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030018bd0 con 0x7ff040100970 2024-04-02T13:46:10.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.137+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401a4f10 con 0x7ff040100970 2024-04-02T13:46:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff03e7fc640 1 -- 172.21.15.122:0/1987247674 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff0300107d0 con 0x7ff040100970 2024-04-02T13:46:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 Processor -- start 2024-04-02T13:46:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 -- start start 2024-04-02T13:46:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab76760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fedaad17b0 con 0x55fedab6d7a0 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fedaa70570 con 0x55fedab6e0c0 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff04a60b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fedaacd760 con 0x55fedab76ca0 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff045eea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff045eea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34838/0 (socket says 172.21.15.122:34838) 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff044ee8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.140+0000 7ff045eea640 1 -- 172.21.15.122:0/3202833782 learned_addr learned my addr 172.21.15.122:0/3202833782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff0456e9640 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab76760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff03e7fc640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 -- 172.21.15.122:0/3202833782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 msgr2=0x55fedab76760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab76760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 -- 172.21.15.122:0/3202833782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 msgr2=0x55fedab6db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 -- 172.21.15.122:0/3202833782 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feda7e53f0 con 0x55fedab76ca0 2024-04-02T13:46:10.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff045eea640 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff045eea640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff03e7fc640 1 -- 172.21.15.122:0/1987247674 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff0300d11c0 con 0x7ff040100970 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff044ee8640 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab79090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff034009230 tx=0x7ff034009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a53b66d328d8348 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff045eea640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 0x7ff0280a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff038004650 tx=0x7ff0380046c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff0277fe640 1 -- 172.21.15.122:0/3202833782 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034042070 con 0x55fedab76ca0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff0277fe640 1 -- 172.21.15.122:0/3202833782 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03402c690 con 0x55fedab76ca0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.141+0000 7ff0277fe640 1 -- 172.21.15.122:0/3202833782 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034004b60 con 0x55fedab76ca0 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- 172.21.15.122:0/3202833782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 msgr2=0x55fedab79090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab79090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff034009230 tx=0x7ff034009610 comp rx=0 tx=0).stop 2024-04-02T13:46:10.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- 172.21.15.122:0/3202833782 shutdown_connections 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab79090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab76760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3202833782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab6db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- 172.21.15.122:0/3202833782 >> 172.21.15.122:0/3202833782 conn(0x55fedab67ed0 msgr2=0x55fedab682b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- 172.21.15.122:0/3202833782 shutdown_connections 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- 172.21.15.122:0/3202833782 wait complete. 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 -- start start 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.142+0000 7ff04a60b740 1 Processor -- start 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6d7a0 0x55fedab73b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fedab79a30 con 0x55fedab76ca0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fedaa8ce80 con 0x55fedab6d7a0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fedab79bb0 con 0x55fedab6e0c0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff04a60b740 10 client.0 _ll_register_callbacks cb 0x55fedaac8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff045eea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6d7a0 0x55fedab73b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff045eea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34852/0 (socket says 172.21.15.122:34852) 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff045eea640 1 -- 172.21.15.122:0/444040151 learned_addr learned my addr 172.21.15.122:0/444040151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 -- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6d7a0 msgr2=0x55fedab73b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6d7a0 0x55fedab73b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 -- 172.21.15.122:0/444040151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 msgr2=0x55fedab745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 -- 172.21.15.122:0/444040151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feda8af7d0 con 0x55fedab6e0c0 2024-04-02T13:46:10.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff045eea640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:10.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff044ee8640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab74090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff034009930 tx=0x7ff034009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=841d0ed60f001eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.143+0000 7ff024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:46:10.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034042040 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff034004240 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fedaadecf0 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff034057050 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 10 client.27667 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fedab79a80 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fedaadecf0 con 0x55fedab6e0c0 2024-04-02T13:46:10.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff0340035f0 con 0x55fedab6e0c0 2024-04-02T13:46:10.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 10 client.27667 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:10.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 10 client.27667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:10.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 20 client.27667 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:10.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedab86d90 0x55fedab89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 4 client.27667 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fedaad9040 con 0x55fedab86d90 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.144+0000 7ff0456e9640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedab86d90 0x55fedab89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.145+0000 7ff0456e9640 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedab86d90 0x55fedab89180 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6357c38aa0c7c8 server_cookie=735afccee7acbf68 in_seq=0 out_seq=0 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.145+0000 7ff024ff9640 10 client.27667 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.145+0000 7ff024ff9640 1 -- 172.21.15.122:0/444040151 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fedaad9040 con 0x55fedab86d90 2024-04-02T13:46:10.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.145+0000 7ff024ff9640 10 client.27667 handle_command_reply: tid=0 2024-04-02T13:46:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.145+0000 7ff04a60b740 1 client.27667 shutdown 2024-04-02T13:46:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 msgr2=0x55fedab74090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab74090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff034009930 tx=0x7ff034009680 comp rx=0 tx=0).stop 2024-04-02T13:46:10.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 shutdown_connections 2024-04-02T13:46:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedab86d90 0x55fedab89180 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab76ca0 0x55fedab745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab74090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 --2- 172.21.15.122:0/444040151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6d7a0 0x55fedab73b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff024ff9640 0 client.27667 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 >> 172.21.15.122:0/444040151 conn(0x55fedab67ed0 msgr2=0x55fedab69c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 shutdown_connections 2024-04-02T13:46:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 1 -- 172.21.15.122:0/444040151 wait complete. 2024-04-02T13:46:10.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.146+0000 7ff04a60b740 20 client.27667 trim_cache size 0 max 16384 2024-04-02T13:46:10.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 Processor -- start 2024-04-02T13:46:10.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 -- start start 2024-04-02T13:46:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab76ca0 0x55fedab72b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401af0d0 con 0x55fedab6d7a0 2024-04-02T13:46:10.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b0c10 con 0x55fedab76ca0 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff04a60b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401af7a0 con 0x55fedab6e0c0 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34864/0 (socket says 172.21.15.122:34864) 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 -- 172.21.15.122:0/3834142422 learned_addr learned my addr 172.21.15.122:0/3834142422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff045eea640 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab76ca0 0x55fedab72b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff044ee8640 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 -- 172.21.15.122:0/3834142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 msgr2=0x55fedab828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 -- 172.21.15.122:0/3834142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab76ca0 msgr2=0x55fedab72b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab76ca0 0x55fedab72b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.160+0000 7ff0456e9640 1 -- 172.21.15.122:0/3834142422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedab6def0 con 0x55fedab6d7a0 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff0456e9640 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff03009cad0 tx=0x7ff0300d36a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b28b97c10b26c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff026ffd640 1 -- 172.21.15.122:0/3834142422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030010e60 con 0x55fedab6d7a0 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff026ffd640 1 -- 172.21.15.122:0/3834142422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03009cdd0 con 0x55fedab6d7a0 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff026ffd640 1 -- 172.21.15.122:0/3834142422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030010ac0 con 0x55fedab6d7a0 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 -- 172.21.15.122:0/3834142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 msgr2=0x55fedab76280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7ff03009cad0 tx=0x7ff0300d36a0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 -- 172.21.15.122:0/3834142422 shutdown_connections 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab76ca0 0x55fedab72b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab6e0c0 0x55fedab828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 --2- 172.21.15.122:0/3834142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab76280 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 -- 172.21.15.122:0/3834142422 >> 172.21.15.122:0/3834142422 conn(0x55fedab67ed0 msgr2=0x55fedab72420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 -- 172.21.15.122:0/3834142422 shutdown_connections 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.161+0000 7ff04a60b740 1 -- 172.21.15.122:0/3834142422 wait complete. 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 -- start start 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 Processor -- start 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab7a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab7a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401b0540 con 0x55fedab6d7a0 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401afe70 con 0x55fedab6e0c0 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401ad0d0 con 0x55fedab76ca0 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff04a60b740 10 client.0 _ll_register_callbacks cb 0x7ff038009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff0456e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34874/0 (socket says 172.21.15.122:34874) 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff0456e9640 1 -- 172.21.15.122:0/2350493369 learned_addr learned my addr 172.21.15.122:0/2350493369 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:10.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff044ee8640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab7a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.162+0000 7ff045eea640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab7a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 -- 172.21.15.122:0/2350493369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 msgr2=0x55fedab7a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab7a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 -- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 msgr2=0x55fedab7a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab7a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 -- 172.21.15.122:0/2350493369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedab894f0 con 0x55fedab6d7a0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff045eea640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff0456e9640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff0300ce580 tx=0x7ff030000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3cdaed0ba1f492 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0300b6060 con 0x55fedab6d7a0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9613+0+0 (secure 0 0 0) 0x7ff03009b0d0 con 0x55fedab6d7a0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03009cdd0 con 0x55fedab6d7a0 2024-04-02T13:46:10.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff030010e60 con 0x55fedab6d7a0 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 10 client.18879 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55fedabec570 con 0x55fedab6d7a0 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03009cdd0 con 0x55fedab6d7a0 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff030099440 con 0x55fedab6d7a0 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 10 client.18879 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 20 client.18879 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedabecc10 0x55fedabef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 4 client.18879 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.163+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fedaa566b0 con 0x55fedabecc10 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff044ee8640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedabecc10 0x55fedabef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:10.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff024ff9640 10 client.18879 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff044ee8640 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedabecc10 0x55fedabef030 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c87d755e276f86e server_cookie=a67cc9637b22d93a in_seq=0 out_seq=0 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff024ff9640 1 -- 172.21.15.122:0/2350493369 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fedaa566b0 con 0x55fedabecc10 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff024ff9640 10 client.18879 handle_command_reply: tid=0 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 client.18879 shutdown 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 msgr2=0x55fedab79dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7ff0300ce580 tx=0x7ff030000f40 comp rx=0 tx=0).stop 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 shutdown_connections 2024-04-02T13:46:10.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55fedabecc10 0x55fedabef030 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fedab76ca0 0x55fedab7a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff024ff9640 0 client.18879 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fedab6e0c0 0x55fedab7a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 --2- 172.21.15.122:0/2350493369 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fedab6d7a0 0x55fedab79dc0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 >> 172.21.15.122:0/2350493369 conn(0x55fedab67ed0 msgr2=0x55fedab73af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 shutdown_connections 2024-04-02T13:46:10.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 1 -- 172.21.15.122:0/2350493369 wait complete. 2024-04-02T13:46:10.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.164+0000 7ff04a60b740 20 client.18879 trim_cache size 0 max 16384 2024-04-02T13:46:10.206 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 msgr2=0x7ff0280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 0x7ff0280a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff038004650 tx=0x7ff0380046c0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 msgr2=0x7ff0401983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff03000ebc0 tx=0x7ff03000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 shutdown_connections 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0280a3d20 0x7ff0280a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff040106980 0x7ff04019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff040101330 0x7ff0401988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 --2- 172.21.15.122:0/1987247674 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff040100970 0x7ff0401983a0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:10.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 >> 172.21.15.122:0/1987247674 conn(0x7ff040077b00 msgr2=0x7ff040068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:10.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 shutdown_connections 2024-04-02T13:46:10.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:10.165+0000 7ff04794d640 1 -- 172.21.15.122:0/1987247674 wait complete. 2024-04-02T13:46:10.975 INFO:tasks.workunit.client.0.smithi122.stdout: 1 397976 3.56 MB/sec execute 451 sec latency 411.838 ms 2024-04-02T13:46:11.212 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:46:11.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:46:11.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 -- 172.21.15.122:0/3636437354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 msgr2=0x7f0080100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 --2- 172.21.15.122:0/3636437354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080100df0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f0068009930 tx=0x7f006802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 -- 172.21.15.122:0/3636437354 shutdown_connections 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 --2- 172.21.15.122:0/3636437354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f008010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 --2- 172.21.15.122:0/3636437354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080100df0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 --2- 172.21.15.122:0/3636437354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 0x7f0080106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 -- 172.21.15.122:0/3636437354 >> 172.21.15.122:0/3636437354 conn(0x7f0080077b00 msgr2=0x7f0080077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 -- 172.21.15.122:0/3636437354 shutdown_connections 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.317+0000 7f0084967640 1 -- 172.21.15.122:0/3636437354 wait complete. 2024-04-02T13:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 Processor -- start 2024-04-02T13:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 -- start start 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f0080103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 0x7f0080103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0080110d90 con 0x7f0080100970 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0080110c10 con 0x7f0080101330 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0084967640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0080110f10 con 0x7f0080106980 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34898/0 (socket says 172.21.15.122:34898) 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f007e59c640 1 -- 172.21.15.122:0/953397358 learned_addr learned my addr 172.21.15.122:0/953397358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f0075d9b640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f0080103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.318+0000 7f007ed9d640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 0x7f0080103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 -- 172.21.15.122:0/953397358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 msgr2=0x7f0080103a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 0x7f0080103a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 -- 172.21.15.122:0/953397358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 msgr2=0x7f0080103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f0080103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 -- 172.21.15.122:0/953397358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00801a68a0 con 0x7f0080100970 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0075d9b640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f0080103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f007e59c640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f007000d770 tx=0x7f007000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0077fff640 1 -- 172.21.15.122:0/953397358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070017070 con 0x7f0080100970 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0077fff640 1 -- 172.21.15.122:0/953397358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0070005500 con 0x7f0080100970 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00801a6b90 con 0x7f0080100970 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0077fff640 1 -- 172.21.15.122:0/953397358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070004e00 con 0x7f0080100970 2024-04-02T13:46:11.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.319+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00801a7060 con 0x7f0080100970 2024-04-02T13:46:11.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.321+0000 7f0077fff640 1 -- 172.21.15.122:0/953397358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0070004fa0 con 0x7f0080100970 2024-04-02T13:46:11.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.321+0000 7f008762b740 1 Processor -- start 2024-04-02T13:46:11.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 -- start start 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0077fff640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 0x7f006c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 0x560da4a4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 0x560da4a57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560da49b2770 con 0x560da4a577c0 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560da495a960 con 0x560da4a4e2c0 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f008762b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560da49ae8c0 con 0x560da4a4ebe0 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007ed9d640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 0x7f006c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34904/0 (socket says 172.21.15.122:34904) 2024-04-02T13:46:11.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007ed9d640 1 -- 172.21.15.122:0/1565609874 learned_addr learned my addr 172.21.15.122:0/1565609874 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0077fff640 1 -- 172.21.15.122:0/953397358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f00700d04f0 con 0x7f0080100970 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 0x560da4a57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007e59c640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 0x560da4a4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 -- 172.21.15.122:0/1565609874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 msgr2=0x560da4a4e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 0x560da4a4e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 -- 172.21.15.122:0/1565609874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 msgr2=0x560da4a59bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f0075d9b640 1 -- 172.21.15.122:0/1565609874 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da478e7e0 con 0x560da4a4ebe0 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007ed9d640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.322+0000 7f007e59c640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 0x560da4a4e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f007ed9d640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 0x7f006c0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0078011d00 tx=0x7f0078011440 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 0x560da4a57280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0068009230 tx=0x7f0068005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4a183403cdc668f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f0074d99640 1 -- 172.21.15.122:0/1565609874 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068042070 con 0x560da4a4ebe0 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f0074d99640 1 -- 172.21.15.122:0/1565609874 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0068004a70 con 0x560da4a4ebe0 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f0074d99640 1 -- 172.21.15.122:0/1565609874 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068005190 con 0x560da4a4ebe0 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f008762b740 1 -- 172.21.15.122:0/1565609874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 msgr2=0x560da4a57280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.323+0000 7f008762b740 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 0x560da4a57280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0068009230 tx=0x7f0068005d50 comp rx=0 tx=0).stop 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- 172.21.15.122:0/1565609874 shutdown_connections 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a577c0 0x560da4a59bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4ebe0 0x560da4a57280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- 172.21.15.122:0/1565609874 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4e2c0 0x560da4a4e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- 172.21.15.122:0/1565609874 >> 172.21.15.122:0/1565609874 conn(0x560da4a489f0 msgr2=0x560da4a48dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- 172.21.15.122:0/1565609874 shutdown_connections 2024-04-02T13:46:11.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- 172.21.15.122:0/1565609874 wait complete. 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- start start 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 Processor -- start 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4e2c0 0x560da4a54800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 0x560da4a55280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560da4a5a550 con 0x560da4a4ebe0 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560da496dd20 con 0x560da4a577c0 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560da4a5a6d0 con 0x560da4a4e2c0 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f008762b740 10 client.0 _ll_register_callbacks cb 0x560da493aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f0075d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f0075d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34908/0 (socket says 172.21.15.122:34908) 2024-04-02T13:46:11.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f0075d9b640 1 -- 172.21.15.122:0/795986000 learned_addr learned my addr 172.21.15.122:0/795986000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f007ed9d640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 0x560da4a55280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.324+0000 7f007e59c640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4e2c0 0x560da4a54800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 -- 172.21.15.122:0/795986000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4e2c0 msgr2=0x560da4a54800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4e2c0 0x560da4a54800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 -- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 msgr2=0x560da4a55280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 0x560da4a55280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 -- 172.21.15.122:0/795986000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da47eef70 con 0x560da4a4ebe0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f007ed9d640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 0x560da4a55280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0075d9b640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0068036040 tx=0x7f0068004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9a9acb1fe7f7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068042070 con 0x560da4a4ebe0 2024-04-02T13:46:11.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0068004a70 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f006802cc90 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0068058050 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 10 client.18891 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560da4686170 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0068004a70 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f006804e030 con 0x560da4a4ebe0 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 10 client.18891 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:11.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 20 client.18891 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4a67990 0x560da4a69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 4 client.18891 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.325+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560da49bfcb0 con 0x560da4a67990 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.326+0000 7f007e59c640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4a67990 0x560da4a69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.326+0000 7f0065ffb640 10 client.18891 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.326+0000 7f007e59c640 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4a67990 0x560da4a69d80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70ce6a6c3c58e5a server_cookie=fde140fc2761d50b in_seq=0 out_seq=0 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f0065ffb640 1 -- 172.21.15.122:0/795986000 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560da49bfcb0 con 0x560da4a67990 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f0065ffb640 10 client.18891 handle_command_reply: tid=0 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 client.18891 shutdown 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 msgr2=0x560da4a54d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0068036040 tx=0x7f0068004460 comp rx=0 tx=0).stop 2024-04-02T13:46:11.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 shutdown_connections 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4a67990 0x560da4a69d80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a577c0 0x560da4a55280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4ebe0 0x560da4a54d40 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 --2- 172.21.15.122:0/795986000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a4e2c0 0x560da4a54800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f0065ffb640 0 client.18891 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 >> 172.21.15.122:0/795986000 conn(0x560da4a489f0 msgr2=0x560da4a4a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 shutdown_connections 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 1 -- 172.21.15.122:0/795986000 wait complete. 2024-04-02T13:46:11.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.327+0000 7f008762b740 20 client.18891 trim_cache size 0 max 16384 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 Processor -- start 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 -- start start 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a6f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00801b1170 con 0x560da4a4e2c0 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00801b2cb0 con 0x560da4a4ebe0 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.344+0000 7f008762b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00801b1840 con 0x560da4a577c0 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34914/0 (socket says 172.21.15.122:34914) 2024-04-02T13:46:11.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 -- 172.21.15.122:0/1034067225 learned_addr learned my addr 172.21.15.122:0/1034067225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007ed9d640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a6f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 -- 172.21.15.122:0/1034067225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 msgr2=0x560da4a6f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a6f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 -- 172.21.15.122:0/1034067225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 msgr2=0x560da4a54ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a54ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 -- 172.21.15.122:0/1034067225 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560da4a59bb0 con 0x560da4a4e2c0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007ed9d640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a6f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f0075d9b640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f007e59c640 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f007009c0e0 tx=0x7f00700048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c2637cd91cd1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f0067fff640 1 -- 172.21.15.122:0/1034067225 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070020040 con 0x560da4a4e2c0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f0067fff640 1 -- 172.21.15.122:0/1034067225 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007009c150 con 0x560da4a4e2c0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.345+0000 7f0067fff640 1 -- 172.21.15.122:0/1034067225 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070009cc0 con 0x560da4a4e2c0 2024-04-02T13:46:11.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- 172.21.15.122:0/1034067225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 msgr2=0x560da4a63410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f007009c0e0 tx=0x7f00700048a0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- 172.21.15.122:0/1034067225 shutdown_connections 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a6f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- 172.21.15.122:0/1034067225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a63410 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- 172.21.15.122:0/1034067225 >> 172.21.15.122:0/1034067225 conn(0x560da4a489f0 msgr2=0x560da4a547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- 172.21.15.122:0/1034067225 shutdown_connections 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- 172.21.15.122:0/1034067225 wait complete. 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- start start 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 Processor -- start 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a5c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a5c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00801b25e0 con 0x560da4a4e2c0 2024-04-02T13:46:11.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00801b1f10 con 0x560da4a4ebe0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00801af170 con 0x560da4a577c0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f008762b740 10 client.0 _ll_register_callbacks cb 0x7f0078011850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f007e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34922/0 (socket says 172.21.15.122:34922) 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f007e59c640 1 -- 172.21.15.122:0/3115210551 learned_addr learned my addr 172.21.15.122:0/3115210551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f0075d9b640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a5c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.346+0000 7f007ed9d640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a5c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 -- 172.21.15.122:0/3115210551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 msgr2=0x560da4a5c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a5c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 -- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 msgr2=0x560da4a5c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a5c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 -- 172.21.15.122:0/3115210551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560da4a5cdc0 con 0x560da4a4e2c0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f007e59c640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00700ac3c0 tx=0x7f00700cd830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c875f67451a3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00700a9060 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f00700a00d0 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f007009c150 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0070020050 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 10 client.18897 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x560da4acec10 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f007009c150 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f00700a9ce0 con 0x560da4a4e2c0 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 10 client.18897 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 20 client.18897 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4acf280 0x560da4ad16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 4 client.18897 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.347+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560da49ba000 con 0x560da4acf280 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.348+0000 7f0075d9b640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4acf280 0x560da4ad16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.348+0000 7f0065ffb640 10 client.18897 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.348+0000 7f0075d9b640 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4acf280 0x560da4ad16a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38421d3698101242 server_cookie=83e0b2509a350f96 in_seq=0 out_seq=0 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f0065ffb640 1 -- 172.21.15.122:0/3115210551 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560da49ba000 con 0x560da4acf280 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f0065ffb640 10 client.18897 handle_command_reply: tid=0 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 client.18897 shutdown 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 msgr2=0x560da4a5fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f00700ac3c0 tx=0x7f00700cd830 comp rx=0 tx=0).stop 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 shutdown_connections 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560da4acf280 0x560da4ad16a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560da4a577c0 0x560da4a5c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560da4a4ebe0 0x560da4a5c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 --2- 172.21.15.122:0/3115210551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560da4a4e2c0 0x560da4a5fd00 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 >> 172.21.15.122:0/3115210551 conn(0x560da4a489f0 msgr2=0x560da4a4a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f0065ffb640 0 client.18897 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 shutdown_connections 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 1 -- 172.21.15.122:0/3115210551 wait complete. 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.349+0000 7f008762b740 20 client.18897 trim_cache size 0 max 16384 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:11.361 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:46:11.362 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "5197713d-eab1-423d-aadd-e982c6b0392b", 2024-04-02T13:46:11.362 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:46:11.362 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:11.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 msgr2=0x7f006c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 0x7f006c0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0078011d00 tx=0x7f0078011440 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 msgr2=0x7f0080102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f007000d770 tx=0x7f007000dc30 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 shutdown_connections 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f006c0a3d20 0x7f006c0a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0080106980 0x7f0080103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0080101330 0x7f0080103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 --2- 172.21.15.122:0/953397358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0080100970 0x7f0080102fe0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 >> 172.21.15.122:0/953397358 conn(0x7f0080077b00 msgr2=0x7f0080068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 shutdown_connections 2024-04-02T13:46:11.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.350+0000 7f0084967640 1 -- 172.21.15.122:0/953397358 wait complete. 2024-04-02T13:46:11.369 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:11.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:11.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2757182431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a41054f0 msgr2=0x7f17a410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2757182431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a41054f0 0x7f17a410d070 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f179c009ca0 tx=0x7f179c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2757182431 shutdown_connections 2024-04-02T13:46:11.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2757182431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a41054f0 0x7f17a410d070 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2757182431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4101020 0x7f17a4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2757182431 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a4100670 0x7f17a4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2757182431 >> 172.21.15.122:0/2757182431 conn(0x7f17a4075c60 msgr2=0x7f17a4076070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2757182431 shutdown_connections 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.484+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2757182431 wait complete. 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 Processor -- start 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 -- start start 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a41054f0 0x7f17a41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4199c80 0x7f17a41a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4110e30 con 0x7f17a4100670 2024-04-02T13:46:11.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4110cb0 con 0x7f17a41054f0 2024-04-02T13:46:11.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a9c0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a4110fb0 con 0x7f17a4199c80 2024-04-02T13:46:11.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34940/0 (socket says 172.21.15.122:34940) 2024-04-02T13:46:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 -- 172.21.15.122:0/2225984516 learned_addr learned my addr 172.21.15.122:0/2225984516 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a41054f0 0x7f17a41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4199c80 0x7f17a41a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 -- 172.21.15.122:0/2225984516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4199c80 msgr2=0x7f17a41a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4199c80 0x7f17a41a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 -- 172.21.15.122:0/2225984516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a41054f0 msgr2=0x7f17a41996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a41054f0 0x7f17a41996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.485+0000 7f17a37fe640 1 -- 172.21.15.122:0/2225984516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a41a4a00 con 0x7f17a4100670 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a37fe640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f179400c970 tx=0x7f179400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a0ff9640 1 -- 172.21.15.122:0/2225984516 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794004220 con 0x7f17a4100670 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a0ff9640 1 -- 172.21.15.122:0/2225984516 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940043c0 con 0x7f17a4100670 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a41a4cf0 con 0x7f17a4100670 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a0ff9640 1 -- 172.21.15.122:0/2225984516 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179400f450 con 0x7f17a4100670 2024-04-02T13:46:11.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.486+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a41a51e0 con 0x7f17a4100670 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.488+0000 7f17a0ff9640 1 -- 172.21.15.122:0/2225984516 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f17940079b0 con 0x7f17a4100670 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.488+0000 7f17ac7f7740 1 Processor -- start 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.488+0000 7f17ac7f7740 1 -- start start 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.488+0000 7f17a0ff9640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 0x7f178c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 0x7f178c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57ead890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 0x557b57eb01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a0ff9640 1 -- 172.21.15.122:0/2225984516 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f17940d09b0 con 0x7f17a4100670 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b57e08830 con 0x557b57ea48d0 2024-04-02T13:46:11.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557b57db0820 con 0x557b57eaddd0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557b57e047e0 con 0x557b57ea51f0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34950/0 (socket says 172.21.15.122:34950) 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 -- 172.21.15.122:0/4263253276 learned_addr learned my addr 172.21.15.122:0/4263253276 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a3fff640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 0x557b57eb01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57ead890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 -- 172.21.15.122:0/4263253276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 msgr2=0x557b57ead890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57ead890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 -- 172.21.15.122:0/4263253276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 msgr2=0x557b57eb01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 0x557b57eb01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 -- 172.21.15.122:0/4263253276 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b57be67d0 con 0x557b57ea48d0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a3fff640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 0x557b57eb01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 0x7f178c0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f179c005cc0 tx=0x7f179c032490 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f179409c4f0 tx=0x7f179409bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398ac280a3a4fa9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.489+0000 7f1789ffb640 1 -- 172.21.15.122:0/4263253276 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179401d040 con 0x557b57ea48d0 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f1789ffb640 1 -- 172.21.15.122:0/4263253276 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940a0910 con 0x557b57ea48d0 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f1789ffb640 1 -- 172.21.15.122:0/4263253276 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1794007ca0 con 0x557b57ea48d0 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4263253276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 msgr2=0x557b57ea4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f179409c4f0 tx=0x7f179409bec0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4263253276 shutdown_connections 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57eaddd0 0x557b57eb01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57ead890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4263253276 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea48d0 0x557b57ea4cb0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4263253276 >> 172.21.15.122:0/4263253276 conn(0x557b57e9f000 msgr2=0x557b57e9f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4263253276 shutdown_connections 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.490+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4263253276 wait complete. 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 -- start start 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 Processor -- start 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57eaaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557b57eb0bc0 con 0x557b57eaddd0 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557b57dc3d30 con 0x557b57ea48d0 2024-04-02T13:46:11.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557b57eb0d40 con 0x557b57ea51f0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17ac7f7740 10 client.0 _ll_register_callbacks cb 0x557b57dc3720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40790/0 (socket says 172.21.15.122:40790) 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34966/0 (socket says 172.21.15.122:34966) 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a37fe640 1 -- 172.21.15.122:0/4078828175 learned_addr learned my addr 172.21.15.122:0/4078828175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.491+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57eaaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 -- 172.21.15.122:0/4078828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 msgr2=0x557b57eaaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57eaaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 -- 172.21.15.122:0/4078828175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 msgr2=0x557b57eaaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 -- 172.21.15.122:0/4078828175 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b57b1c3f0 con 0x557b57ea48d0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a3fff640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17940ce290 tx=0x7f1794004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b018f679ae1105 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17940b9070 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f17940bb2c0 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17940a0910 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f179401d050 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 10 client.27911 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557b57adc0f0 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17940a0910 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f17940b5ce0 con 0x557b57ea48d0 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 10 client.27911 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 10 client.27911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 20 client.27911 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57ebdc50 0x557b57ec0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 4 client.27911 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.492+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b57e15d70 con 0x557b57ebdc50 2024-04-02T13:46:11.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.493+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57ebdc50 0x557b57ec0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.493+0000 7f1782ffd640 10 client.27911 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.493+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57ebdc50 0x557b57ec0040 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c264f10dd9e041 server_cookie=def95c410e4da4a4 in_seq=0 out_seq=0 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f1782ffd640 1 -- 172.21.15.122:0/4078828175 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557b57e15d70 con 0x557b57ebdc50 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f1782ffd640 10 client.27911 handle_command_reply: tid=0 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 client.27911 shutdown 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 msgr2=0x557b57ead6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f17940ce290 tx=0x7f1794004570 comp rx=0 tx=0).stop 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 shutdown_connections 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57ebdc50 0x557b57ec0040 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57eaaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea51f0 0x557b57eaaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4078828175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ead6c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f1782ffd640 0 client.27911 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 >> 172.21.15.122:0/4078828175 conn(0x557b57e9f000 msgr2=0x557b57ea0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 shutdown_connections 2024-04-02T13:46:11.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4078828175 wait complete. 2024-04-02T13:46:11.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.494+0000 7f17ac7f7740 20 client.27911 trim_cache size 0 max 16384 2024-04-02T13:46:11.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 Processor -- start 2024-04-02T13:46:11.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 -- start start 2024-04-02T13:46:11.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea48d0 0x557b57eb9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea51f0 0x557b57ea5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41af170 con 0x557b57eaddd0 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41b0cb0 con 0x557b57ea51f0 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41af840 con 0x557b57ea48d0 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34982/0 (socket says 172.21.15.122:34982) 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17a3fff640 1 -- 172.21.15.122:0/2179641236 learned_addr learned my addr 172.21.15.122:0/2179641236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.511+0000 7f17a37fe640 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea48d0 0x557b57eb9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea51f0 0x557b57ea5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 -- 172.21.15.122:0/2179641236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea48d0 msgr2=0x557b57eb9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea48d0 0x557b57eb9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 -- 172.21.15.122:0/2179641236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea51f0 msgr2=0x557b57ea5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea51f0 0x557b57ea5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 -- 172.21.15.122:0/2179641236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b57ea8fc0 con 0x557b57eaddd0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17a3fff640 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f179c009ca0 tx=0x7f179c045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea05b0150ec36741 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17897fa640 1 -- 172.21.15.122:0/2179641236 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c047030 con 0x557b57eaddd0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17897fa640 1 -- 172.21.15.122:0/2179641236 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c051070 con 0x557b57eaddd0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.512+0000 7f17897fa640 1 -- 172.21.15.122:0/2179641236 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c039040 con 0x557b57eaddd0 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- 172.21.15.122:0/2179641236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 msgr2=0x557b57ead180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f179c009ca0 tx=0x7f179c045000 comp rx=0 tx=0).stop 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- 172.21.15.122:0/2179641236 shutdown_connections 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57eaddd0 0x557b57ead180 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea51f0 0x557b57ea5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/2179641236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57ea48d0 0x557b57eb9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- 172.21.15.122:0/2179641236 >> 172.21.15.122:0/2179641236 conn(0x557b57e9f000 msgr2=0x557b57ea5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- 172.21.15.122:0/2179641236 shutdown_connections 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- 172.21.15.122:0/2179641236 wait complete. 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- start start 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 Processor -- start 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ec2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57eaddd0 0x557b57ec0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41b05e0 con 0x557b57ea51f0 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41aff10 con 0x557b57ea48d0 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41ad170 con 0x557b57eaddd0 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.513+0000 7f17ac7f7740 10 client.0 _ll_register_callbacks cb 0x7f17940cdc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34998/0 (socket says 172.21.15.122:34998) 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 -- 172.21.15.122:0/4279776749 learned_addr learned my addr 172.21.15.122:0/4279776749 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a3fff640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57eaddd0 0x557b57ec0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ec2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 -- 172.21.15.122:0/4279776749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57eaddd0 msgr2=0x557b57ec0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57eaddd0 0x557b57ec0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 -- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 msgr2=0x557b57ec2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ec2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 -- 172.21.15.122:0/4279776749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b57ec0a40 con 0x557b57ea51f0 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ec2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17a2ffd640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1798004700 tx=0x7f1798012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5686f7521393d30e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:11.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f1782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798011b70 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f1798011d10 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f179c051070 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f1798030050 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17ac7f7740 10 client.18915 fetch_fsmap learned FSMap version 40 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557b57f23950 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.514+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c051070 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7f1798027990 con 0x557b57ea51f0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 10 client.18915 fetch_fsmap finished waiting for FSMap version 40 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 10 client.18915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 20 client.18915 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57f240b0 0x557b57f264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 4 client.18915 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b57e100c0 con 0x557b57f240b0 2024-04-02T13:46:11.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57f240b0 0x557b57f264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f1782ffd640 10 client.18915 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17a37fe640 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57f240b0 0x557b57f264d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9ba32be37cd9664 server_cookie=5a3f1090dbb66015 in_seq=0 out_seq=0 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f1782ffd640 1 -- 172.21.15.122:0/4279776749 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557b57e100c0 con 0x557b57f240b0 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f1782ffd640 10 client.18915 handle_command_reply: tid=0 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.515+0000 7f17ac7f7740 1 client.18915 shutdown 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 msgr2=0x557b57ec3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f1798004700 tx=0x7f1798012b10 comp rx=0 tx=0).stop 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 shutdown_connections 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557b57f240b0 0x557b57f264d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557b57eaddd0 0x557b57ec0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557b57ea51f0 0x557b57ec3360 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f1782ffd640 0 client.18915 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 --2- 172.21.15.122:0/4279776749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557b57ea48d0 0x557b57ec2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 >> 172.21.15.122:0/4279776749 conn(0x557b57e9f000 msgr2=0x557b57ea0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 shutdown_connections 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 1 -- 172.21.15.122:0/4279776749 wait complete. 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17ac7f7740 20 client.18915 trim_cache size 0 max 16384 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:46:11.527 INFO:teuthology.orchestra.run.smithi122.stdout: "5197713d-eab1-423d-aadd-e982c6b0392b": { 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "5197713d-eab1-423d-aadd-e982c6b0392b", 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:46:11.528 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 msgr2=0x7f178c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 0x7f178c0a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f179c005cc0 tx=0x7f179c032490 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 msgr2=0x7f17a41991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f179400c970 tx=0x7f179400ce30 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 shutdown_connections 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f178c0a3d20 0x7f178c0a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f17a4199c80 0x7f17a41a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17a41054f0 0x7f17a41996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 --2- 172.21.15.122:0/2225984516 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17a4100670 0x7f17a41991b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 >> 172.21.15.122:0/2225984516 conn(0x7f17a4075c60 msgr2=0x7f17a40ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:11.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 shutdown_connections 2024-04-02T13:46:11.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:11.516+0000 7f17a9c0d640 1 -- 172.21.15.122:0/2225984516 wait complete. 2024-04-02T13:46:11.530 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:11 smithi122 ceph-mon[32114]: pgmap v342: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:11.530 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:11 smithi122 ceph-mon[32114]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:46:11.531 INFO:tasks.cephfs.filesystem:scrub status for tag:5197713d-eab1-423d-aadd-e982c6b0392b - {'path': '/', 'tag': '5197713d-eab1-423d-aadd-e982c6b0392b', 'options': 'recursive,force'} 2024-04-02T13:46:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:11 smithi187 ceph-mon[36064]: pgmap v342: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:11 smithi187 ceph-mon[36064]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:46:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:11 smithi188 ceph-mon[36117]: pgmap v342: 129 pgs: 129 active+clean; 315 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-02T13:46:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:11 smithi188 ceph-mon[36117]: osdmap e105: 12 total, 12 up, 12 in 2024-04-02T13:46:11.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 398032 3.55 MB/sec execute 452 sec latency 830.817 ms 2024-04-02T13:46:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:46:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:46:12.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:12.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 398032 3.55 MB/sec execute 453 sec latency 1530.758 ms 2024-04-02T13:46:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:13 smithi187 ceph-mon[36064]: pgmap v344: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-04-02T13:46:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:13 smithi188 ceph-mon[36117]: pgmap v344: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-04-02T13:46:13.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:13 smithi122 ceph-mon[32114]: pgmap v344: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 208 op/s 2024-04-02T13:46:13.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 398995 3.55 MB/sec execute 454 sec latency 1968.233 ms 2024-04-02T13:46:14.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 401151 3.57 MB/sec execute 455 sec latency 14.247 ms 2024-04-02T13:46:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:15 smithi187 ceph-mon[36064]: pgmap v345: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-04-02T13:46:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:15 smithi188 ceph-mon[36117]: pgmap v345: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-04-02T13:46:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:15 smithi122 ceph-mon[32114]: pgmap v345: 129 pgs: 129 active+clean; 324 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 200 op/s 2024-04-02T13:46:15.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 401581 3.56 MB/sec execute 456 sec latency 519.144 ms 2024-04-02T13:46:16.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 401581 3.56 MB/sec execute 457 sec latency 1519.196 ms 2024-04-02T13:46:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: pgmap v346: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: pgmap v346: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: pgmap v346: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:17.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 401581 3.55 MB/sec execute 458 sec latency 2519.301 ms 2024-04-02T13:46:18.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 402508 3.55 MB/sec execute 459 sec latency 2961.208 ms 2024-04-02T13:46:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:19 smithi187 ceph-mon[36064]: pgmap v347: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:19 smithi188 ceph-mon[36117]: pgmap v347: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:19 smithi122 ceph-mon[32114]: pgmap v347: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:19.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 404645 3.57 MB/sec execute 460 sec latency 15.755 ms 2024-04-02T13:46:20.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 405651 3.57 MB/sec execute 461 sec latency 22.180 ms 2024-04-02T13:46:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:21 smithi187 ceph-mon[36064]: pgmap v348: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:21 smithi188 ceph-mon[36117]: pgmap v348: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:21 smithi122 ceph-mon[32114]: pgmap v348: 129 pgs: 129 active+clean; 330 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 192 op/s 2024-04-02T13:46:21.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 407913 3.60 MB/sec execute 462 sec latency 20.870 ms 2024-04-02T13:46:22.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 408615 3.59 MB/sec execute 463 sec latency 305.717 ms 2024-04-02T13:46:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:23 smithi187 ceph-mon[36064]: pgmap v349: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-02T13:46:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:23 smithi188 ceph-mon[36117]: pgmap v349: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-02T13:46:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:23 smithi122 ceph-mon[32114]: pgmap v349: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 139 op/s 2024-04-02T13:46:23.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 409443 3.60 MB/sec execute 464 sec latency 743.181 ms 2024-04-02T13:46:24.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 410959 3.60 MB/sec execute 465 sec latency 59.813 ms 2024-04-02T13:46:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:25 smithi187 ceph-mon[36064]: pgmap v350: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-02T13:46:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:25 smithi188 ceph-mon[36117]: pgmap v350: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-02T13:46:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:25 smithi122 ceph-mon[32114]: pgmap v350: 129 pgs: 129 active+clean; 333 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-04-02T13:46:25.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 412272 3.61 MB/sec execute 466 sec latency 29.431 ms 2024-04-02T13:46:26.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:26 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:26.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:26 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:26.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:26 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:46:26.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 412282 3.60 MB/sec execute 467 sec latency 981.663 ms 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: pgmap v351: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: pgmap v351: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2024-04-02T13:46:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 412282 3.60 MB/sec execute 468 sec latency 1981.745 ms 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: pgmap v351: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 208 op/s 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:46:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:46:28.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 413315 3.60 MB/sec execute 469 sec latency 2419.362 ms 2024-04-02T13:46:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:29 smithi188 ceph-mon[36117]: pgmap v352: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:29 smithi122 ceph-mon[32114]: pgmap v352: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:29.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 415349 3.62 MB/sec execute 470 sec latency 12.520 ms 2024-04-02T13:46:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:29 smithi187 ceph-mon[36064]: pgmap v352: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:30.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 416559 3.62 MB/sec execute 471 sec latency 25.458 ms 2024-04-02T13:46:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:31 smithi188 ceph-mon[36117]: pgmap v353: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:31 smithi122 ceph-mon[32114]: pgmap v353: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:31.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 418673 3.64 MB/sec execute 472 sec latency 18.233 ms 2024-04-02T13:46:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:31 smithi187 ceph-mon[36064]: pgmap v353: 129 pgs: 129 active+clean; 343 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-04-02T13:46:32.976 INFO:tasks.workunit.client.0.smithi122.stdout: 1 419772 3.64 MB/sec execute 473 sec latency 15.554 ms 2024-04-02T13:46:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:33 smithi188 ceph-mon[36117]: pgmap v354: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2024-04-02T13:46:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:33 smithi122 ceph-mon[32114]: pgmap v354: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2024-04-02T13:46:33.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 421178 3.65 MB/sec execute 474 sec latency 16.968 ms 2024-04-02T13:46:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:33 smithi187 ceph-mon[36064]: pgmap v354: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 213 op/s 2024-04-02T13:46:34.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 422689 3.66 MB/sec execute 475 sec latency 11.421 ms 2024-04-02T13:46:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:35 smithi188 ceph-mon[36117]: pgmap v355: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-02T13:46:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:35 smithi122 ceph-mon[32114]: pgmap v355: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-02T13:46:35.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 423959 3.67 MB/sec execute 476 sec latency 33.026 ms 2024-04-02T13:46:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:35 smithi187 ceph-mon[36064]: pgmap v355: 129 pgs: 129 active+clean; 348 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-02T13:46:36.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 425969 3.69 MB/sec execute 477 sec latency 38.459 ms 2024-04-02T13:46:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:37 smithi188 ceph-mon[36117]: pgmap v356: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-04-02T13:46:37.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:37 smithi122 ceph-mon[32114]: pgmap v356: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-04-02T13:46:37.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 427365 3.69 MB/sec execute 478 sec latency 18.171 ms 2024-04-02T13:46:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:37 smithi187 ceph-mon[36064]: pgmap v356: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-04-02T13:46:38.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:38 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:38 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:38.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 429483 3.71 MB/sec execute 479 sec latency 14.137 ms 2024-04-02T13:46:38.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:38 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:39 smithi188 ceph-mon[36117]: pgmap v357: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:39 smithi122 ceph-mon[32114]: pgmap v357: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:39.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 430690 3.71 MB/sec execute 480 sec latency 17.853 ms 2024-04-02T13:46:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:39 smithi187 ceph-mon[36064]: pgmap v357: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:40.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 431829 3.71 MB/sec execute 481 sec latency 127.333 ms 2024-04-02T13:46:41.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:41.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 -- 172.21.15.122:0/4236165106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8100970 msgr2=0x7fb1b8100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 --2- 172.21.15.122:0/4236165106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8100970 0x7fb1b8100df0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009930 tx=0x7fb1a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 -- 172.21.15.122:0/4236165106 shutdown_connections 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 --2- 172.21.15.122:0/4236165106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8101330 0x7fb1b810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 --2- 172.21.15.122:0/4236165106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8100970 0x7fb1b8100df0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 --2- 172.21.15.122:0/4236165106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8106980 0x7fb1b8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 -- 172.21.15.122:0/4236165106 >> 172.21.15.122:0/4236165106 conn(0x7fb1b8077b00 msgr2=0x7fb1b8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 -- 172.21.15.122:0/4236165106 shutdown_connections 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.640+0000 7fb1bec55640 1 -- 172.21.15.122:0/4236165106 wait complete. 2024-04-02T13:46:41.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 Processor -- start 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 -- start start 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8100970 0x7fb1b8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8106980 0x7fb1b81a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110d90 con 0x7fb1b8101330 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110c10 con 0x7fb1b8100970 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bec55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b8110f10 con 0x7fb1b8106980 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58340/0 (socket says 172.21.15.122:58340) 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3127610757 learned_addr learned my addr 172.21.15.122:0/3127610757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8100970 0x7fb1b8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.641+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8106980 0x7fb1b81a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3127610757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8106980 msgr2=0x7fb1b81a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8106980 0x7fb1b81a41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3127610757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8100970 msgr2=0x7fb1b8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8100970 0x7fb1b8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3127610757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b81a46f0 con 0x7fb1b8101330 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009230 tx=0x7fb1a402cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b5ffb640 1 -- 172.21.15.122:0/3127610757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4041070 con 0x7fb1b8101330 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b81a48c0 con 0x7fb1b8101330 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b5ffb640 1 -- 172.21.15.122:0/3127610757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a4004450 con 0x7fb1b8101330 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1b5ffb640 1 -- 172.21.15.122:0/3127610757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4034440 con 0x7fb1b8101330 2024-04-02T13:46:41.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.642+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b81a4dd0 con 0x7fb1b8101330 2024-04-02T13:46:41.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.644+0000 7fb1b5ffb640 1 -- 172.21.15.122:0/3127610757 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb1a4044070 con 0x7fb1b8101330 2024-04-02T13:46:41.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1b5ffb640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 0x7fb1a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 Processor -- start 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1b5ffb640 1 -- 172.21.15.122:0/3127610757 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fb1a40f24f0 con 0x7fb1b8101330 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 0x7fb1a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 -- start start 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7ddf6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddffdd0 0x558e7de021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e7dd5a830 con 0x558e7ddffdd0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558e7dd02820 con 0x558e7ddf68d0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1c1838740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e7dd567e0 con 0x558e7ddf71f0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 0x7fb1a80a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00bb60 tx=0x7fb1ac00b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34360/0 (socket says 172.21.15.122:34360) 2024-04-02T13:46:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1bc9f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7ddf6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1b7fff640 1 -- 172.21.15.122:0/23343870 learned_addr learned my addr 172.21.15.122:0/23343870 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.645+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddffdd0 0x558e7de021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 -- 172.21.15.122:0/23343870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 msgr2=0x558e7ddf6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7ddf6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 -- 172.21.15.122:0/23343870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddffdd0 msgr2=0x558e7de021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddffdd0 0x558e7de021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 -- 172.21.15.122:0/23343870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7db387d0 con 0x558e7ddf71f0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40bac30 tx=0x7fb1a4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9088e1a8530cd402 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1a2ffd640 1 -- 172.21.15.122:0/23343870 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a404b040 con 0x558e7ddf71f0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1a2ffd640 1 -- 172.21.15.122:0/23343870 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a40c2560 con 0x558e7ddf71f0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.646+0000 7fb1a2ffd640 1 -- 172.21.15.122:0/23343870 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40baa40 con 0x558e7ddf71f0 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- 172.21.15.122:0/23343870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 msgr2=0x558e7ddff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40bac30 tx=0x7fb1a4009260 comp rx=0 tx=0).stop 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- 172.21.15.122:0/23343870 shutdown_connections 2024-04-02T13:46:41.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddffdd0 0x558e7de021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf71f0 0x558e7ddff890 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 --2- 172.21.15.122:0/23343870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7ddf6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- 172.21.15.122:0/23343870 >> 172.21.15.122:0/23343870 conn(0x558e7ddf1000 msgr2=0x558e7ddf13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- 172.21.15.122:0/23343870 shutdown_connections 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- 172.21.15.122:0/23343870 wait complete. 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 -- start start 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.647+0000 7fb1c1838740 1 Processor -- start 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddfc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddfceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558e7de02980 con 0x558e7ddf71f0 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558e7dd15d30 con 0x558e7ddffdd0 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558e7de02b00 con 0x558e7ddf68d0 2024-04-02T13:46:41.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 10 client.0 _ll_register_callbacks cb 0x558e7dd15720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddfc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bc9f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bc9f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34368/0 (socket says 172.21.15.122:34368) 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bc9f1640 1 -- 172.21.15.122:0/3589878604 learned_addr learned my addr 172.21.15.122:0/3589878604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddfceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3589878604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 msgr2=0x558e7ddff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 msgr2=0x558e7ddfceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddfceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 -- 172.21.15.122:0/3589878604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7da6e3f0 con 0x558e7ddf71f0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddfceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddfc970 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x558e7ddf63a0 tx=0x7fb1a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca752bdc7f2ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb197fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:41.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a404b070 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fb1a40c2120 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.648+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1a40c2560 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb1a40c2360 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 10 client.18927 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558e7da2e0f0 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a40c2560 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fb1a40a7250 con 0x558e7ddf71f0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 10 client.18927 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 10 client.18927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 20 client.18927 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de0f8a0 0x558e7de11c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 4 client.18927 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e7dd67d70 con 0x558e7de0f8a0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de0f8a0 0x558e7de11c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb197fff640 10 client.18927 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.649+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de0f8a0 0x558e7de11c90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab100957276cf4ce server_cookie=5bdd9d0e756f3685 in_seq=0 out_seq=0 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb197fff640 1 -- 172.21.15.122:0/3589878604 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e7dd67d70 con 0x558e7de0f8a0 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb197fff640 10 client.18927 handle_command_reply: tid=0 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 client.18927 shutdown 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 msgr2=0x558e7ddfc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddfc970 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x558e7ddf63a0 tx=0x7fb1a4000bc0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 shutdown_connections 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de0f8a0 0x558e7de11c90 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddfceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddfc970 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb197fff640 0 client.18927 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 --2- 172.21.15.122:0/3589878604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7ddff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 >> 172.21.15.122:0/3589878604 conn(0x558e7ddf1000 msgr2=0x558e7ddf2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 shutdown_connections 2024-04-02T13:46:41.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.650+0000 7fb1c1838740 1 -- 172.21.15.122:0/3589878604 wait complete. 2024-04-02T13:46:41.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.651+0000 7fb1c1838740 20 client.18927 trim_cache size 0 max 16384 2024-04-02T13:46:41.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 Processor -- start 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 -- start start 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7de0b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddf00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 0x558e7ddff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81aef30 con 0x558e7ddf71f0 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b0a70 con 0x558e7ddf68d0 2024-04-02T13:46:41.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.670+0000 7fb1c1838740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81af600 con 0x558e7ddffdd0 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddf00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddf00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58360/0 (socket says 172.21.15.122:58360) 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1b7fff640 1 -- 172.21.15.122:0/2935751162 learned_addr learned my addr 172.21.15.122:0/2935751162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 0x558e7ddff0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7de0b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/2935751162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 msgr2=0x558e7de0b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7de0b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/2935751162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 msgr2=0x558e7ddf00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddf00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.671+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/2935751162 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7ddf0e30 con 0x558e7ddffdd0 2024-04-02T13:46:41.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.672+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 0x558e7ddff0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0010750 tx=0x7fb1b0007520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=411457f1b8e349a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.672+0000 7fb1a27fc640 1 -- 172.21.15.122:0/2935751162 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0004b20 con 0x558e7ddffdd0 2024-04-02T13:46:41.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.672+0000 7fb1a27fc640 1 -- 172.21.15.122:0/2935751162 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b000fe60 con 0x558e7ddffdd0 2024-04-02T13:46:41.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.672+0000 7fb1a27fc640 1 -- 172.21.15.122:0/2935751162 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0005840 con 0x558e7ddffdd0 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.673+0000 7fb1c1838740 1 -- 172.21.15.122:0/2935751162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 msgr2=0x558e7ddff0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.673+0000 7fb1c1838740 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 0x558e7ddff0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0010750 tx=0x7fb1b0007520 comp rx=0 tx=0).stop 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.674+0000 7fb1c1838740 1 -- 172.21.15.122:0/2935751162 shutdown_connections 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.674+0000 7fb1c1838740 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddffdd0 0x558e7ddff0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.674+0000 7fb1c1838740 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7ddf00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.674+0000 7fb1c1838740 1 --2- 172.21.15.122:0/2935751162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddf68d0 0x558e7de0b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.674+0000 7fb1c1838740 1 -- 172.21.15.122:0/2935751162 >> 172.21.15.122:0/2935751162 conn(0x558e7ddf1000 msgr2=0x558e7ddef9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.676+0000 7fb1c1838740 1 -- 172.21.15.122:0/2935751162 shutdown_connections 2024-04-02T13:46:41.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.677+0000 7fb1c1838740 1 -- 172.21.15.122:0/2935751162 wait complete. 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.677+0000 7fb1c1838740 1 -- start start 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.677+0000 7fb1c1838740 1 Processor -- start 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7de07c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7de08160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81b03a0 con 0x558e7ddf71f0 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81afcd0 con 0x558e7ddffdd0 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b81acf30 con 0x558e7ddf68d0 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1c1838740 10 client.0 _ll_register_callbacks cb 0x7fb1ac00b7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40010/0 (socket says 172.21.15.122:40010) 2024-04-02T13:46:41.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/817412221 learned_addr learned my addr 172.21.15.122:0/817412221 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1b7fff640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7de08160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/817412221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 msgr2=0x558e7de07c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7de07c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/817412221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 msgr2=0x558e7de08160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.678+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7de08160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb1bd1f2640 1 -- 172.21.15.122:0/817412221 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7ddf9330 con 0x558e7ddffdd0 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb1bd1f2640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005460 tx=0x7fb1b000fbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af4671a5b6874b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb197fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b003a070 con 0x558e7ddffdd0 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fb1b0004c50 con 0x558e7ddffdd0 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b000fe60 con 0x558e7ddffdd0 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb1b0046050 con 0x558e7ddffdd0 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb1c1838740 10 client.27932 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:41.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.679+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558e7de75560 con 0x558e7ddffdd0 2024-04-02T13:46:41.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b000fe60 con 0x558e7ddffdd0 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fb1b0004070 con 0x558e7ddffdd0 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 10 client.27932 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 20 client.27932 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de75bf0 0x558e7de80800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 4 client.27932 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.680+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e7dd620c0 con 0x558e7de75bf0 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.681+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de75bf0 0x558e7de80800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb1bc9f1640 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de75bf0 0x558e7de80800 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1578422ec1386c server_cookie=4096c1ab50afc547 in_seq=0 out_seq=0 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb197fff640 10 client.27932 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb197fff640 1 -- 172.21.15.122:0/817412221 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558e7dd620c0 con 0x558e7de75bf0 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb197fff640 10 client.27932 handle_command_reply: tid=0 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb1c1838740 1 client.27932 shutdown 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 msgr2=0x558e7ddf8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.682+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0005460 tx=0x7fb1b000fbb0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 shutdown_connections 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x558e7de75bf0 0x558e7de80800 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558e7ddffdd0 0x558e7ddf8cc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558e7ddf71f0 0x558e7de08160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 --2- 172.21.15.122:0/817412221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558e7ddf68d0 0x558e7de07c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 >> 172.21.15.122:0/817412221 conn(0x558e7ddf1000 msgr2=0x558e7ddf2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb197fff640 0 client.27932 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 shutdown_connections 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 1 -- 172.21.15.122:0/817412221 wait complete. 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.683+0000 7fb1c1838740 20 client.27932 trim_cache size 0 max 16384 2024-04-02T13:46:41.691 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:46:41.692 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:46:41.692 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 msgr2=0x7fb1a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 0x7fb1a80a61e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00bb60 tx=0x7fb1ac00b390 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 msgr2=0x7fb1b8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009230 tx=0x7fb1a402cc70 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 shutdown_connections 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb1a80a3d20 0x7fb1a80a61e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1b8106980 0x7fb1b81a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8101330 0x7fb1b8199670 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.684+0000 7fb1bec55640 1 --2- 172.21.15.122:0/3127610757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1b8100970 0x7fb1b8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.685+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 >> 172.21.15.122:0/3127610757 conn(0x7fb1b8077b00 msgr2=0x7fb1b81030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.685+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 shutdown_connections 2024-04-02T13:46:41.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.685+0000 7fb1bec55640 1 -- 172.21.15.122:0/3127610757 wait complete. 2024-04-02T13:46:41.705 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:46:41.705 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:46:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:41 smithi188 ceph-mon[36117]: pgmap v358: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 -- 172.21.15.122:0/4016377458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250100a00 msgr2=0x7f2250100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 --2- 172.21.15.122:0/4016377458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250100a00 0x7f2250100e80 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f2240009930 tx=0x7f224002bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 -- 172.21.15.122:0/4016377458 shutdown_connections 2024-04-02T13:46:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 --2- 172.21.15.122:0/4016377458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22501013c0 0x7f225010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 --2- 172.21.15.122:0/4016377458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250100a00 0x7f2250100e80 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 --2- 172.21.15.122:0/4016377458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2250106a10 0x7f2250106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 -- 172.21.15.122:0/4016377458 >> 172.21.15.122:0/4016377458 conn(0x7f2250077b00 msgr2=0x7f2250077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 -- 172.21.15.122:0/4016377458 shutdown_connections 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.814+0000 7f2258738640 1 -- 172.21.15.122:0/4016377458 wait complete. 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 Processor -- start 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 -- start start 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2250100a00 0x7f22501972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22501013c0 0x7f2250197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2250110e20 con 0x7f2250106a10 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2250110ca0 con 0x7f22501013c0 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2258738640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2250110fa0 con 0x7f2250100a00 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2256cd5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2256cd5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58388/0 (socket says 172.21.15.122:58388) 2024-04-02T13:46:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2256cd5640 1 -- 172.21.15.122:0/256137643 learned_addr learned my addr 172.21.15.122:0/256137643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f22564d4640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2250100a00 0x7f22501972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.815+0000 7f2255cd3640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22501013c0 0x7f2250197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 -- 172.21.15.122:0/256137643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2250100a00 msgr2=0x7f22501972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2250100a00 0x7f22501972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 -- 172.21.15.122:0/256137643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22501013c0 msgr2=0x7f2250197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22501013c0 0x7f2250197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 -- 172.21.15.122:0/256137643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225019c2b0 con 0x7f2250106a10 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2256cd5640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f224800eca0 tx=0x7f224800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248004b20 con 0x7f2250106a10 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225019c5a0 con 0x7f2250106a10 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2248004cc0 con 0x7f2250106a10 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2250068c60 con 0x7f2250106a10 2024-04-02T13:46:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.816+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248017610 con 0x7f2250106a10 2024-04-02T13:46:41.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:41 smithi122 ceph-mon[32114]: pgmap v358: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:41.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.817+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22500688f0 con 0x7f2250106a10 2024-04-02T13:46:41.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.818+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2248004e70 con 0x7f2250106a10 2024-04-02T13:46:41.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.819+0000 7f22477fe640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 0x7f223c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:41.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.819+0000 7f22564d4640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 0x7f223c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:41.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.819+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f22480d2aa0 con 0x7f2250106a10 2024-04-02T13:46:41.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.819+0000 7f22564d4640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 0x7f223c0a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f224c009b50 tx=0x7f224c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:41.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:41.822+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f224809e700 con 0x7f2250106a10 2024-04-02T13:46:41.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 433236 3.73 MB/sec execute 482 sec latency 32.953 ms 2024-04-02T13:46:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:41 smithi187 ceph-mon[36064]: pgmap v358: 129 pgs: 129 active+clean; 354 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-04-02T13:46:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.063+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2250069680 con 0x7f2250106a10 2024-04-02T13:46:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.063+0000 7f22477fe640 1 -- 172.21.15.122:0/256137643 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11943 (secure 0 0 0) 0x7f22480890c0 con 0x7f2250106a10 2024-04-02T13:46:42.067 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:46:42.068 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:46:36.707583+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:46:42.068 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 41 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.065+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 msgr2=0x7f223c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.065+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 0x7f223c0a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f224c009b50 tx=0x7f224c005f90 comp rx=0 tx=0).stop 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 msgr2=0x7f225019bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f224800eca0 tx=0x7f224800b690 comp rx=0 tx=0).stop 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 shutdown_connections 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f223c0a3d20 0x7f223c0a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2250106a10 0x7f225019bb80 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f22501013c0 0x7f2250197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 --2- 172.21.15.122:0/256137643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2250100a00 0x7f22501972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 >> 172.21.15.122:0/256137643 conn(0x7f2250077b00 msgr2=0x7f2250102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 shutdown_connections 2024-04-02T13:46:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.066+0000 7f2258738640 1 -- 172.21.15.122:0/256137643 wait complete. 2024-04-02T13:46:42.084 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:46:42.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 -- 172.21.15.122:0/1169990882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c84101360 msgr2=0x7f3c8410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 --2- 172.21.15.122:0/1169990882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c84101360 0x7f3c8410d820 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009900 tx=0x7f3c7402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 -- 172.21.15.122:0/1169990882 shutdown_connections 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 --2- 172.21.15.122:0/1169990882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c84101360 0x7f3c8410d820 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 --2- 172.21.15.122:0/1169990882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c841009c0 0x7f3c84100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 --2- 172.21.15.122:0/1169990882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 0x7f3c84105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.217+0000 7f3c889f6640 1 -- 172.21.15.122:0/1169990882 >> 172.21.15.122:0/1169990882 conn(0x7f3c84077a90 msgr2=0x7f3c84077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- 172.21.15.122:0/1169990882 shutdown_connections 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- 172.21.15.122:0/1169990882 wait complete. 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 Processor -- start 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- start start 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c84101360 0x7f3c841996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 0x7f3c841a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84110d90 con 0x7f3c841009c0 2024-04-02T13:46:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84110c10 con 0x7f3c84101360 2024-04-02T13:46:42.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.218+0000 7f3c889f6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84110f10 con 0x7f3c841058a0 2024-04-02T13:46:42.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58412/0 (socket says 172.21.15.122:58412) 2024-04-02T13:46:42.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 -- 172.21.15.122:0/3919649799 learned_addr learned my addr 172.21.15.122:0/3919649799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c84101360 0x7f3c841996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 0x7f3c841a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 -- 172.21.15.122:0/3919649799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 msgr2=0x7f3c841a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 0x7f3c841a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 -- 172.21.15.122:0/3919649799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c84101360 msgr2=0x7f3c841996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c84101360 0x7f3c841996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 -- 172.21.15.122:0/3919649799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c841a4750 con 0x7f3c841009c0 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c8259c640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c7b7fe640 1 -- 172.21.15.122:0/3919649799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00cd80 con 0x7f3c841009c0 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c7b7fe640 1 -- 172.21.15.122:0/3919649799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c7c005500 con 0x7f3c841009c0 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c841a4920 con 0x7f3c841009c0 2024-04-02T13:46:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c7b7fe640 1 -- 172.21.15.122:0/3919649799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c010690 con 0x7f3c841009c0 2024-04-02T13:46:42.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.219+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c841a4df0 con 0x7f3c841009c0 2024-04-02T13:46:42.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 Processor -- start 2024-04-02T13:46:42.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 -- start start 2024-04-02T13:46:42.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c7b7fe640 1 -- 172.21.15.122:0/3919649799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3c7c0046c0 con 0x7f3c841009c0 2024-04-02T13:46:42.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a1b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a2b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555f0197f7b0 con 0x555f01a290b0 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f0191e570 con 0x555f01a1bd70 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555f0197b760 con 0x555f01a1b450 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a2b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a2b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58422/0 (socket says 172.21.15.122:58422) 2024-04-02T13:46:42.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c81d9b640 1 -- 172.21.15.122:0/1372690752 learned_addr learned my addr 172.21.15.122:0/1372690752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c8259c640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.222+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a1b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1372690752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 msgr2=0x555f01a28b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1372690752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 msgr2=0x555f01a2b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a2b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1372690752 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f016933f0 con 0x555f01a1b450 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c8259c640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c7b7fe640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 0x7f3c700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a1b830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009230 tx=0x7f3c740095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8aa75041b2ab05c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c8259c640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 0x7f3c700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c78ff9640 1 -- 172.21.15.122:0/1372690752 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74042070 con 0x555f01a1b450 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c7b7fe640 1 -- 172.21.15.122:0/3919649799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f3c7c0d0f70 con 0x7f3c841009c0 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c78ff9640 1 -- 172.21.15.122:0/1372690752 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74004040 con 0x555f01a1b450 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.223+0000 7f3c78ff9640 1 -- 172.21.15.122:0/1372690752 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74004c30 con 0x555f01a1b450 2024-04-02T13:46:42.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1372690752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 msgr2=0x555f01a1b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8259c640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 0x7f3c700a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0ce380 tx=0x7f3c7c0d4ac0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a1b830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009230 tx=0x7f3c740095f0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1372690752 shutdown_connections 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a2b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1372690752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a1b830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1372690752 >> 172.21.15.122:0/1372690752 conn(0x555f01a15ed0 msgr2=0x555f01a162b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1372690752 shutdown_connections 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1372690752 wait complete. 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.224+0000 7f3c8b5db740 1 -- start start 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 Processor -- start 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1b450 0x555f01a28aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1bd70 0x555f01a25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555f01a2bf30 con 0x555f01a290b0 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555f0193ae80 con 0x555f01a1b450 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555f01a2c0b0 con 0x555f01a1bd70 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 10 client.0 _ll_register_callbacks cb 0x555f01976aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58434/0 (socket says 172.21.15.122:58434) 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 -- 172.21.15.122:0/918890669 learned_addr learned my addr 172.21.15.122:0/918890669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1b450 0x555f01a28aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1bd70 0x555f01a25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 -- 172.21.15.122:0/918890669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1bd70 msgr2=0x555f01a25e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1bd70 0x555f01a25e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 -- 172.21.15.122:0/918890669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1b450 msgr2=0x555f01a28aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1b450 0x555f01a28aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 -- 172.21.15.122:0/918890669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f0175d7d0 con 0x555f01a290b0 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009850 tx=0x7f3c74005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1208d60a3c1592b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74042040 con 0x555f01a290b0 2024-04-02T13:46:42.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f3c74004c00 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.225+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c74004040 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.226+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3c7405c050 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.226+0000 7f3c8b5db740 10 client.18951 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.226+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f01a37f90 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.226+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74004040 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f3c740035f0 con 0x555f01a290b0 2024-04-02T13:46:42.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 10 client.18951 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 10 client.18951 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 20 client.18951 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a392e0 0x555f01a3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 4 client.18951 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f0198ccf0 con 0x555f01a392e0 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a392e0 0x555f01a3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c69ffb640 10 client.18951 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.227+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a392e0 0x555f01a3b6d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dc2e21334056242 server_cookie=48f4abc5caa10905 in_seq=0 out_seq=0 2024-04-02T13:46:42.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c69ffb640 1 -- 172.21.15.122:0/918890669 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f0198ccf0 con 0x555f01a392e0 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c69ffb640 10 client.18951 handle_command_reply: tid=0 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 client.18951 shutdown 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 msgr2=0x555f01a26440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009850 tx=0x7f3c74005e50 comp rx=0 tx=0).stop 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 shutdown_connections 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a392e0 0x555f01a3b6d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a26440 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1bd70 0x555f01a25e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c69ffb640 0 client.18951 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/918890669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1b450 0x555f01a28aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 >> 172.21.15.122:0/918890669 conn(0x555f01a15ed0 msgr2=0x555f01a17c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 shutdown_connections 2024-04-02T13:46:42.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 1 -- 172.21.15.122:0/918890669 wait complete. 2024-04-02T13:46:42.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.228+0000 7f3c8b5db740 20 client.18951 trim_cache size 0 max 16384 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 Processor -- start 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 -- start start 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a28aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841acd40 con 0x555f01a290b0 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ae880 con 0x555f01a1bd70 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ad410 con 0x555f01a1b450 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40072/0 (socket says 172.21.15.122:40072) 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34450/0 (socket says 172.21.15.122:34450) 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 -- 172.21.15.122:0/3650928551 learned_addr learned my addr 172.21.15.122:0/3650928551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a28aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 -- 172.21.15.122:0/3650928551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 msgr2=0x555f01a24e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 -- 172.21.15.122:0/3650928551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 msgr2=0x555f01a28aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a28aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c81d9b640 1 -- 172.21.15.122:0/3650928551 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f01a41510 con 0x555f01a1bd70 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.245+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.246+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x555f01a10370 tx=0x7f3c6c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=263d985712c338ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.246+0000 7f3c6bfff640 1 -- 172.21.15.122:0/3650928551 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c003dc0 con 0x555f01a1bd70 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.246+0000 7f3c6bfff640 1 -- 172.21.15.122:0/3650928551 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c004040 con 0x555f01a1bd70 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.246+0000 7f3c6bfff640 1 -- 172.21.15.122:0/3650928551 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c01e270 con 0x555f01a1bd70 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- 172.21.15.122:0/3650928551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 msgr2=0x555f01a28560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x555f01a10370 tx=0x7f3c6c004a40 comp rx=0 tx=0).stop 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- 172.21.15.122:0/3650928551 shutdown_connections 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a290b0 0x555f01a28aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a28560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/3650928551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a1b450 0x555f01a24e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- 172.21.15.122:0/3650928551 >> 172.21.15.122:0/3650928551 conn(0x555f01a15ed0 msgr2=0x555f01a24700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- 172.21.15.122:0/3650928551 shutdown_connections 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- 172.21.15.122:0/3650928551 wait complete. 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- start start 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 Processor -- start 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a39df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a290b0 0x555f01a9d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841ae1b0 con 0x555f01a1b450 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841adae0 con 0x555f01a1bd70 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841aad40 con 0x555f01a290b0 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c8b5db740 10 client.0 _ll_register_callbacks cb 0x7f3c6c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c81d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a39df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c82d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58444/0 (socket says 172.21.15.122:58444) 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1718338136 learned_addr learned my addr 172.21.15.122:0/1718338136 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.247+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a290b0 0x555f01a9d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1718338136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a290b0 msgr2=0x555f01a9d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a290b0 0x555f01a9d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1718338136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 msgr2=0x555f01a39df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a39df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 -- 172.21.15.122:0/1718338136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f01a9da30 con 0x555f01a1b450 2024-04-02T13:46:42.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a39df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c82d9d640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3c740092a0 tx=0x7f3c74005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b504a251326966ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74042040 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f3c740035f0 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c004040 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f3c740037f0 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c8b5db740 10 client.18957 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.248+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f01a9ed30 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c6c004040 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f3c7402cc10 con 0x555f01a1b450 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 10 client.18957 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 10 client.18957 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:46:42.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 20 client.18957 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a9f340 0x555f01aa1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 4 client.18957 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a9f340 0x555f01aa1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.249+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f01987040 con 0x555f01a9f340 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c69ffb640 10 client.18957 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c81d9b640 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a9f340 0x555f01aa1730 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4e497a50bad5e3ab server_cookie=110dbc59f55ad329 in_seq=0 out_seq=0 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c69ffb640 1 -- 172.21.15.122:0/1718338136 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f01987040 con 0x555f01a9f340 2024-04-02T13:46:42.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c69ffb640 10 client.18957 handle_command_reply: tid=0 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c8b5db740 1 client.18957 shutdown 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 msgr2=0x555f01a398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3c740092a0 tx=0x7f3c74005fd0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.250+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 shutdown_connections 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555f01a9f340 0x555f01aa1730 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555f01a290b0 0x555f01a9d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555f01a1bd70 0x555f01a39df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c69ffb640 0 client.18957 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 --2- 172.21.15.122:0/1718338136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555f01a1b450 0x555f01a398b0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 >> 172.21.15.122:0/1718338136 conn(0x555f01a15ed0 msgr2=0x555f01a24f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 shutdown_connections 2024-04-02T13:46:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 1 -- 172.21.15.122:0/1718338136 wait complete. 2024-04-02T13:46:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c8b5db740 20 client.18957 trim_cache size 0 max 16384 2024-04-02T13:46:42.281 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 msgr2=0x7f3c700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 0x7f3c700a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c0ce380 tx=0x7f3c7c0d4ac0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 msgr2=0x7f3c84199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c00e9f0 tx=0x7f3c7c00edf0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 shutdown_connections 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3c700a3d20 0x7f3c700a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c841058a0 0x7f3c841a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c84101360 0x7f3c841996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 --2- 172.21.15.122:0/3919649799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3c841009c0 0x7f3c84199190 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 >> 172.21.15.122:0/3919649799 conn(0x7f3c84077a90 msgr2=0x7f3c840fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.251+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 shutdown_connections 2024-04-02T13:46:42.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.252+0000 7f3c889f6640 1 -- 172.21.15.122:0/3919649799 wait complete. 2024-04-02T13:46:42.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:46:42.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 -- 172.21.15.122:0/3298278637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884107b50 msgr2=0x7fc884107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 --2- 172.21.15.122:0/3298278637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884107b50 0x7fc884107f30 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc874009930 tx=0x7fc87402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 -- 172.21.15.122:0/3298278637 shutdown_connections 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 --2- 172.21.15.122:0/3298278637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 0x7fc88410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 --2- 172.21.15.122:0/3298278637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884068d00 0x7fc884069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 --2- 172.21.15.122:0/3298278637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884107b50 0x7fc884107f30 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 -- 172.21.15.122:0/3298278637 >> 172.21.15.122:0/3298278637 conn(0x7fc884077b00 msgr2=0x7fc884077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 -- 172.21.15.122:0/3298278637 shutdown_connections 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.398+0000 7fc88919d640 1 -- 172.21.15.122:0/3298278637 wait complete. 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 Processor -- start 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 -- start start 2024-04-02T13:46:42.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 0x7fc884198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884107b50 0x7fc88419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc884111ea0 con 0x7fc884068d00 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc884111d20 con 0x7fc8840696c0 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88919d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc884112020 con 0x7fc884107b50 2024-04-02T13:46:42.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58454/0 (socket says 172.21.15.122:58454) 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 -- 172.21.15.122:0/331866680 learned_addr learned my addr 172.21.15.122:0/331866680 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc87bfff640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 0x7fc884198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc88359e640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884107b50 0x7fc88419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 -- 172.21.15.122:0/331866680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884107b50 msgr2=0x7fc88419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884107b50 0x7fc88419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 -- 172.21.15.122:0/331866680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 msgr2=0x7fc884198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 0x7fc884198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 -- 172.21.15.122:0/331866680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88419d3a0 con 0x7fc884068d00 2024-04-02T13:46:42.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.399+0000 7fc882d9d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc874009230 tx=0x7fc874004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.400+0000 7fc880d99640 1 -- 172.21.15.122:0/331866680 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874041070 con 0x7fc884068d00 2024-04-02T13:46:42.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.400+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88419d630 con 0x7fc884068d00 2024-04-02T13:46:42.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.400+0000 7fc880d99640 1 -- 172.21.15.122:0/331866680 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8740054e0 con 0x7fc884068d00 2024-04-02T13:46:42.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.400+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8841a4f20 con 0x7fc884068d00 2024-04-02T13:46:42.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.400+0000 7fc880d99640 1 -- 172.21.15.122:0/331866680 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874003c90 con 0x7fc884068d00 2024-04-02T13:46:42.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc880d99640 1 -- 172.21.15.122:0/331866680 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc874044070 con 0x7fc884068d00 2024-04-02T13:46:42.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 Processor -- start 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 -- start start 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bb7a0 0x557f843bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 0x557f843c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f8431f7b0 con 0x557f843bb7a0 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f842be570 con 0x557f843c4ca0 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.402+0000 7fc88bd87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557f8431b760 con 0x557f843bc0c0 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc88359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bb7a0 0x557f843bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40090/0 (socket says 172.21.15.122:40090) 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 -- 172.21.15.122:0/3083695063 learned_addr learned my addr 172.21.15.122:0/3083695063 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc880d99640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc88359e640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 -- 172.21.15.122:0/3083695063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 msgr2=0x557f843c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc882d9d640 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 0x557f843c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 0x557f843c4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc880d99640 1 -- 172.21.15.122:0/331866680 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fc8740f2890 con 0x7fc884068d00 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 -- 172.21.15.122:0/3083695063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bb7a0 msgr2=0x557f843bbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bb7a0 0x557f843bbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 -- 172.21.15.122:0/3083695063 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f840333f0 con 0x557f843c4ca0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc882d9d640 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 0x557f843c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ca0 tx=0x7fc86c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4611640f273f7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc88359e640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 0x7fc8700a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009fd0 tx=0x7fc87c004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc8797fa640 1 -- 172.21.15.122:0/3083695063 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c01e070 con 0x557f843c4ca0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc8797fa640 1 -- 172.21.15.122:0/3083695063 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c019b10 con 0x557f843c4ca0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.403+0000 7fc8797fa640 1 -- 172.21.15.122:0/3083695063 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00ea30 con 0x557f843c4ca0 2024-04-02T13:46:42.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 -- 172.21.15.122:0/3083695063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 msgr2=0x557f843c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ca0 tx=0x7fc86c011810 comp rx=0 tx=0).stop 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 -- 172.21.15.122:0/3083695063 shutdown_connections 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843c4ca0 0x557f843c7090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bc0c0 0x557f843c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3083695063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bb7a0 0x557f843bbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 -- 172.21.15.122:0/3083695063 >> 172.21.15.122:0/3083695063 conn(0x557f843b5ed0 msgr2=0x557f843b62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 -- 172.21.15.122:0/3083695063 shutdown_connections 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.404+0000 7fc88bd87740 1 -- 172.21.15.122:0/3083695063 wait complete. 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 -- start start 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 Processor -- start 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bb7a0 0x557f843c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bc0c0 0x557f843c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557f843cbea0 con 0x557f843c4ca0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557f842dae80 con 0x557f843bc0c0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557f843cc020 con 0x557f843bb7a0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88bd87740 10 client.0 _ll_register_callbacks cb 0x557f84316aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bc0c0 0x557f843c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58484/0 (socket says 172.21.15.122:58484) 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 -- 172.21.15.122:0/1207067113 learned_addr learned my addr 172.21.15.122:0/1207067113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc882d9d640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bb7a0 0x557f843c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 -- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bb7a0 msgr2=0x557f843c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bb7a0 0x557f843c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 -- 172.21.15.122:0/1207067113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bc0c0 msgr2=0x557f843c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bc0c0 0x557f843c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 -- 172.21.15.122:0/1207067113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f840fd7d0 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc88359e640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00ba30 tx=0x7fc87c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330e535796ca9576 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.405+0000 7fc86a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c027060 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fc87c0045d0 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c019b10 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc87c02e050 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 10 client.18969 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f843d7ed0 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c019b10 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc87c010ad0 con 0x557f843c4ca0 2024-04-02T13:46:42.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 10 client.18969 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 10 client.18969 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 20 client.18969 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f843db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 4 client.18969 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.406+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f8432ccf0 con 0x557f843d9390 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.407+0000 7fc882d9d640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f843db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.407+0000 7fc86a7fc640 10 client.18969 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.407+0000 7fc882d9d640 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f843db7c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=de0969a67fcc31df server_cookie=9d089ffece602955 in_seq=0 out_seq=0 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.407+0000 7fc86a7fc640 1 -- 172.21.15.122:0/1207067113 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f8432ccf0 con 0x557f843d9390 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.407+0000 7fc86a7fc640 10 client.18969 handle_command_reply: tid=0 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 client.18969 shutdown 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 msgr2=0x557f843c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc87c00ba30 tx=0x7fc87c004a40 comp rx=0 tx=0).stop 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 shutdown_connections 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f843db7c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843c4ca0 0x557f843c2050 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc86a7fc640 0 client.18969 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bc0c0 0x557f843c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 --2- 172.21.15.122:0/1207067113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843bb7a0 0x557f843c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 >> 172.21.15.122:0/1207067113 conn(0x557f843b5ed0 msgr2=0x557f843b7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 shutdown_connections 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 1 -- 172.21.15.122:0/1207067113 wait complete. 2024-04-02T13:46:42.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.408+0000 7fc88bd87740 20 client.18969 trim_cache size 0 max 16384 2024-04-02T13:46:42.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 Processor -- start 2024-04-02T13:46:42.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 -- start start 2024-04-02T13:46:42.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841acee0 con 0x557f843bc0c0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841aea20 con 0x557f843bb7a0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88bd87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841ad5b0 con 0x557f843c4ca0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58486/0 (socket says 172.21.15.122:58486) 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc882d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 -- 172.21.15.122:0/3662214184 learned_addr learned my addr 172.21.15.122:0/3662214184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 -- 172.21.15.122:0/3662214184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 msgr2=0x557f843c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88359e640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 -- 172.21.15.122:0/3662214184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 msgr2=0x557f843d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc87bfff640 1 -- 172.21.15.122:0/3662214184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f843bff90 con 0x557f843bc0c0 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc88359e640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.422+0000 7fc882d9d640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc87bfff640 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ca0 tx=0x7fc86c00e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca4e67fa9a193d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc878ff9640 1 -- 172.21.15.122:0/3662214184 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c01e040 con 0x557f843bc0c0 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc878ff9640 1 -- 172.21.15.122:0/3662214184 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c004640 con 0x557f843bc0c0 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc878ff9640 1 -- 172.21.15.122:0/3662214184 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00e8d0 con 0x557f843bc0c0 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc88bd87740 1 -- 172.21.15.122:0/3662214184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 msgr2=0x557f843c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.423+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009ca0 tx=0x7fc86c00e5d0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- 172.21.15.122:0/3662214184 shutdown_connections 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843c4170 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- 172.21.15.122:0/3662214184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- 172.21.15.122:0/3662214184 >> 172.21.15.122:0/3662214184 conn(0x557f843b5ed0 msgr2=0x557f843b62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- 172.21.15.122:0/3662214184 shutdown_connections 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- 172.21.15.122:0/3662214184 wait complete. 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- start start 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 Processor -- start 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841ae350 con 0x557f843bc0c0 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841adc80 con 0x557f843bb7a0 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8841aaee0 con 0x557f843c4ca0 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88bd87740 10 client.0 _ll_register_callbacks cb 0x7fc87c009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58500/0 (socket says 172.21.15.122:58500) 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc882d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc87bfff640 1 -- 172.21.15.122:0/345079689 learned_addr learned my addr 172.21.15.122:0/345079689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.424+0000 7fc88359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 -- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 msgr2=0x557f843bf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843bf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 -- 172.21.15.122:0/345079689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 msgr2=0x557f843d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 -- 172.21.15.122:0/345079689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f843c5870 con 0x557f843bc0c0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc87bfff640 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009c70 tx=0x7fc86c004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307a0014f14f7a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc86a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c03a060 con 0x557f843bc0c0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7fc86c005450 con 0x557f843bc0c0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc86c004640 con 0x557f843bc0c0 2024-04-02T13:46:42.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc86c01e040 con 0x557f843bc0c0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc88bd87740 10 client.18975 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557f8443f990 con 0x557f843bc0c0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.425+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc86c004640 con 0x557f843bc0c0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7fc86c034030 con 0x557f843bc0c0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 10 client.18975 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 10 client.18975 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 20 client.18975 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f84441ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 4 client.18975 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f84327040 con 0x557f843d9390 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc882d9d640 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f84441ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc86a7fc640 10 client.18975 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc882d9d640 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f84441ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e9b85e91e7b3ce14 server_cookie=30429e828a5f4e06 in_seq=0 out_seq=0 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc86a7fc640 1 -- 172.21.15.122:0/345079689 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f84327040 con 0x557f843d9390 2024-04-02T13:46:42.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc86a7fc640 10 client.18975 handle_command_reply: tid=0 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 1 client.18975 shutdown 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 msgr2=0x557f843becc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.426+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009c70 tx=0x7fc86c004b80 comp rx=0 tx=0).stop 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 shutdown_connections 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557f843c4ca0 0x557f843bf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x557f843d9390 0x557f84441ed0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc86a7fc640 0 client.18975 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557f843bc0c0 0x557f843becc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 --2- 172.21.15.122:0/345079689 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557f843bb7a0 0x557f843d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 >> 172.21.15.122:0/345079689 conn(0x557f843b5ed0 msgr2=0x557f843c0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 shutdown_connections 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 1 -- 172.21.15.122:0/345079689 wait complete. 2024-04-02T13:46:42.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88bd87740 20 client.18975 trim_cache size 0 max 16384 2024-04-02T13:46:42.437 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:42.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 msgr2=0x7fc8700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 0x7fc8700a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc87c009fd0 tx=0x7fc87c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 msgr2=0x7fc884198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc874009230 tx=0x7fc874004270 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 shutdown_connections 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc8700a3d20 0x7fc8700a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc884107b50 0x7fc88419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc8840696c0 0x7fc884198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 --2- 172.21.15.122:0/331866680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc884068d00 0x7fc884198400 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 >> 172.21.15.122:0/331866680 conn(0x7fc884077b00 msgr2=0x7fc8840fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 shutdown_connections 2024-04-02T13:46:42.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.427+0000 7fc88919d640 1 -- 172.21.15.122:0/331866680 wait complete. 2024-04-02T13:46:42.441 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:46:42.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 -- 172.21.15.122:0/1926863618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4100970 msgr2=0x7f00f4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 --2- 172.21.15.122:0/1926863618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4100970 0x7f00f4100df0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009930 tx=0x7f00e802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 -- 172.21.15.122:0/1926863618 shutdown_connections 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 --2- 172.21.15.122:0/1926863618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 0x7f00f410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 --2- 172.21.15.122:0/1926863618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4100970 0x7f00f4100df0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 --2- 172.21.15.122:0/1926863618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4106980 0x7f00f4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 -- 172.21.15.122:0/1926863618 >> 172.21.15.122:0/1926863618 conn(0x7f00f4077b00 msgr2=0x7f00f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 -- 172.21.15.122:0/1926863618 shutdown_connections 2024-04-02T13:46:42.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.548+0000 7f00f8c93640 1 -- 172.21.15.122:0/1926863618 wait complete. 2024-04-02T13:46:42.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.549+0000 7f00f8c93640 1 Processor -- start 2024-04-02T13:46:42.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 -- start start 2024-04-02T13:46:42.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 0x7f00f41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4106980 0x7f00f419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111e90 con 0x7f00f4106980 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111d10 con 0x7f00f4101330 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f8c93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4112010 con 0x7f00f4100970 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34522/0 (socket says 172.21.15.122:34522) 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f259c640 1 -- 172.21.15.122:0/2323380253 learned_addr learned my addr 172.21.15.122:0/2323380253 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.550+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 0x7f00f41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4106980 0x7f00f419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 -- 172.21.15.122:0/2323380253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 msgr2=0x7f00f41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 0x7f00f41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 -- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4106980 msgr2=0x7f00f419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4106980 0x7f00f419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 -- 172.21.15.122:0/2323380253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f419d3a0 con 0x7f00f4100970 2024-04-02T13:46:42.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f259c640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009f40 tx=0x7f00e400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00e37fe640 1 -- 172.21.15.122:0/2323380253 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e400cd80 con 0x7f00f4100970 2024-04-02T13:46:42.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f419d690 con 0x7f00f4100970 2024-04-02T13:46:42.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00e37fe640 1 -- 172.21.15.122:0/2323380253 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e4005500 con 0x7f00f4100970 2024-04-02T13:46:42.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00e37fe640 1 -- 172.21.15.122:0/2323380253 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4004a00 con 0x7f00f4100970 2024-04-02T13:46:42.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.551+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f410e200 con 0x7f00f4100970 2024-04-02T13:46:42.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.554+0000 7f00e37fe640 1 -- 172.21.15.122:0/2323380253 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f00e4004ba0 con 0x7f00f4100970 2024-04-02T13:46:42.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.554+0000 7f00fb522740 1 Processor -- start 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.554+0000 7f00fb522740 1 -- start start 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00e37fe640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 0x7f00dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531627a0 0x561453162b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5614530c67b0 con 0x56145316bca0 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561453065570 con 0x5614531627a0 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00fb522740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5614530c2760 con 0x5614531630c0 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 0x7f00dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531627a0 0x561453162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00e37fe640 1 -- 172.21.15.122:0/2323380253 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f00e40d4e40 con 0x7f00f4100970 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34528/0 (socket says 172.21.15.122:34528) 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 -- 172.21.15.122:0/2111592502 learned_addr learned my addr 172.21.15.122:0/2111592502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 -- 172.21.15.122:0/2111592502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531627a0 msgr2=0x561453162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531627a0 0x561453162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 -- 172.21.15.122:0/2111592502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 msgr2=0x56145316e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.555+0000 7f00f1d9b640 1 -- 172.21.15.122:0/2111592502 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561452dda3f0 con 0x5614531630c0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 0x7f00dc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009270 tx=0x7f00ec009ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f00e802cfd0 tx=0x7f00e80095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=406b8c94a807e91a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00e0ff9640 1 -- 172.21.15.122:0/2111592502 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8042070 con 0x5614531630c0 2024-04-02T13:46:42.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00e0ff9640 1 -- 172.21.15.122:0/2111592502 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8004030 con 0x5614531630c0 2024-04-02T13:46:42.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00e0ff9640 1 -- 172.21.15.122:0/2111592502 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e8004bc0 con 0x5614531630c0 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 -- 172.21.15.122:0/2111592502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 msgr2=0x56145316b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f00e802cfd0 tx=0x7f00e80095f0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 -- 172.21.15.122:0/2111592502 shutdown_connections 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531630c0 0x56145316b760 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 --2- 172.21.15.122:0/2111592502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531627a0 0x561453162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.556+0000 7f00fb522740 1 -- 172.21.15.122:0/2111592502 >> 172.21.15.122:0/2111592502 conn(0x56145315ced0 msgr2=0x56145315d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- 172.21.15.122:0/2111592502 shutdown_connections 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- 172.21.15.122:0/2111592502 wait complete. 2024-04-02T13:46:42.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- start start 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 Processor -- start 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x56145316b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x561453168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56145316ea30 con 0x56145316bca0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561453081e80 con 0x5614531630c0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56145316ebb0 con 0x5614531627a0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00fb522740 10 client.0 _ll_register_callbacks cb 0x5614530bdaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x56145316b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58530/0 (socket says 172.21.15.122:58530) 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f2d9d640 1 -- 172.21.15.122:0/563227254 learned_addr learned my addr 172.21.15.122:0/563227254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x561453168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f2d9d640 1 -- 172.21.15.122:0/563227254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 msgr2=0x56145316b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.557+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x56145316b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00f2d9d640 1 -- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 msgr2=0x561453168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x561453168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00f2d9d640 1 -- 172.21.15.122:0/563227254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561452ea47d0 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f00ec010fd0 tx=0x7f00ec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedb4ac862ab086b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0123f0 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f00ec011d10 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e8004030 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f00ec029430 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 10 client.18987 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56145317aac0 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e8004030 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f00ec01f050 con 0x56145316bca0 2024-04-02T13:46:42.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 10 client.18987 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 10 client.18987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 20 client.18987 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56145317bf80 0x56145317e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 4 client.18987 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.558+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614530d3cf0 con 0x56145317bf80 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.559+0000 7f00f259c640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56145317bf80 0x56145317e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.559+0000 7f00d9ffb640 10 client.18987 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.559+0000 7f00f259c640 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56145317bf80 0x56145317e400 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9196c1203aa44da server_cookie=69cac657581d8ee8 in_seq=0 out_seq=0 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00d9ffb640 1 -- 172.21.15.122:0/563227254 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614530d3cf0 con 0x56145317bf80 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00d9ffb640 10 client.18987 handle_command_reply: tid=0 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 client.18987 shutdown 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 msgr2=0x5614531691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f00ec010fd0 tx=0x7f00ec009500 comp rx=0 tx=0).stop 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 shutdown_connections 2024-04-02T13:46:42.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x5614531691b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56145317bf80 0x56145317e400 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x561453168ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 --2- 172.21.15.122:0/563227254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x56145316b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00d9ffb640 0 client.18987 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 >> 172.21.15.122:0/563227254 conn(0x56145315ced0 msgr2=0x56145315ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 shutdown_connections 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 1 -- 172.21.15.122:0/563227254 wait complete. 2024-04-02T13:46:42.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.560+0000 7f00fb522740 20 client.18987 trim_cache size 0 max 16384 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 Processor -- start 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 -- start start 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x5614531778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ace60 con 0x56145316bca0 2024-04-02T13:46:42.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ae9a0 con 0x5614531630c0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00fb522740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ad530 con 0x5614531627a0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58536/0 (socket says 172.21.15.122:58536) 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 -- 172.21.15.122:0/2626046205 learned_addr learned my addr 172.21.15.122:0/2626046205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f259c640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x5614531778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 -- 172.21.15.122:0/2626046205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 msgr2=0x5614531778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x5614531778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 -- 172.21.15.122:0/2626046205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 msgr2=0x56145316b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 -- 172.21.15.122:0/2626046205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561453162ef0 con 0x56145316bca0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.574+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009240 tx=0x7f00ec0092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20720693ba870dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00dbfff640 1 -- 172.21.15.122:0/2626046205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec03c070 con 0x56145316bca0 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00dbfff640 1 -- 172.21.15.122:0/2626046205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec0123f0 con 0x56145316bca0 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00dbfff640 1 -- 172.21.15.122:0/2626046205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0295d0 con 0x56145316bca0 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 -- 172.21.15.122:0/2626046205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 msgr2=0x56145316b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009240 tx=0x7f00ec0092a0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 -- 172.21.15.122:0/2626046205 shutdown_connections 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56145316bca0 0x56145316b730 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 --2- 172.21.15.122:0/2626046205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5614531627a0 0x5614531778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 -- 172.21.15.122:0/2626046205 >> 172.21.15.122:0/2626046205 conn(0x56145315ced0 msgr2=0x561453167390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 -- 172.21.15.122:0/2626046205 shutdown_connections 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.575+0000 7f00fb522740 1 -- 172.21.15.122:0/2626046205 wait complete. 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 -- start start 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 Processor -- start 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145316bca0 0x56145316cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41ae2d0 con 0x5614531627a0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41adc00 con 0x5614531630c0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41aae60 con 0x56145316bca0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00fb522740 10 client.0 _ll_register_callbacks cb 0x7f00ec009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58550/0 (socket says 172.21.15.122:58550) 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 -- 172.21.15.122:0/3208317901 learned_addr learned my addr 172.21.15.122:0/3208317901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f2d9d640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145316bca0 0x56145316cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 -- 172.21.15.122:0/3208317901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145316bca0 msgr2=0x56145316cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145316bca0 0x56145316cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 -- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 msgr2=0x56145316c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.576+0000 7f00f259c640 1 -- 172.21.15.122:0/3208317901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145316d2a0 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00f259c640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00e409cf60 tx=0x7f00e408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520871ed08be9040 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40a0b00 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f00e409ee00 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ec0123f0 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f00e40a4eb0 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 10 client.18993 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5614531e2590 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ec0123f0 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f00e40a92e0 con 0x5614531627a0 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 10 client.18993 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 10 client.18993 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:46:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 20 client.18993 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5614531e2af0 0x5614531e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 4 client.18993 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.577+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614530ce040 con 0x5614531e2af0 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5614531e2af0 0x5614531e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00d9ffb640 10 client.18993 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00f1d9b640 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5614531e2af0 0x5614531e4f10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38771983447c67bb server_cookie=6241049b2c2852d5 in_seq=0 out_seq=0 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00d9ffb640 1 -- 172.21.15.122:0/3208317901 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614530ce040 con 0x5614531e2af0 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00d9ffb640 10 client.18993 handle_command_reply: tid=0 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 client.18993 shutdown 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 msgr2=0x561453174070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f00e409cf60 tx=0x7f00e408b040 comp rx=0 tx=0).stop 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 shutdown_connections 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56145316bca0 0x56145316cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5614531e2af0 0x5614531e4f10 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5614531630c0 0x56145316c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 --2- 172.21.15.122:0/3208317901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5614531627a0 0x561453174070 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00d9ffb640 0 client.18993 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 >> 172.21.15.122:0/3208317901 conn(0x56145315ced0 msgr2=0x5614531677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 shutdown_connections 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 1 -- 172.21.15.122:0/3208317901 wait complete. 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.578+0000 7f00fb522740 20 client.18993 trim_cache size 0 max 16384 2024-04-02T13:46:42.588 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 msgr2=0x7f00dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 0x7f00dc0a61e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009270 tx=0x7f00ec009ef0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 msgr2=0x7f00f41983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009f40 tx=0x7f00e400ee80 comp rx=0 tx=0).stop 2024-04-02T13:46:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 shutdown_connections 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00dc0a3d20 0x7f00dc0a61e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00f4106980 0x7f00f419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00f4101330 0x7f00f41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 --2- 172.21.15.122:0/2323380253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f4100970 0x7f00f41983b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 >> 172.21.15.122:0/2323380253 conn(0x7f00f4077b00 msgr2=0x7f00f41031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 shutdown_connections 2024-04-02T13:46:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.579+0000 7f00f8c93640 1 -- 172.21.15.122:0/2323380253 wait complete. 2024-04-02T13:46:42.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 -- 172.21.15.122:0/2003091044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48106980 msgr2=0x7f6d48106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2003091044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48106980 0x7f6d48106d60 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009930 tx=0x7f6d3802bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 -- 172.21.15.122:0/2003091044 shutdown_connections 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2003091044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48101330 0x7f6d4810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2003091044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48100970 0x7f6d48100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2003091044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48106980 0x7f6d48106d60 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 -- 172.21.15.122:0/2003091044 >> 172.21.15.122:0/2003091044 conn(0x7f6d48077b00 msgr2=0x7f6d48077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 -- 172.21.15.122:0/2003091044 shutdown_connections 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.706+0000 7f6d4d367640 1 -- 172.21.15.122:0/2003091044 wait complete. 2024-04-02T13:46:42.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 Processor -- start 2024-04-02T13:46:42.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 -- start start 2024-04-02T13:46:42.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 0x7f6d48198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48106980 0x7f6d4819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48110d60 con 0x7f6d48100970 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48110be0 con 0x7f6d48101330 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d4d367640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d48110ee0 con 0x7f6d48106980 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58562/0 (socket says 172.21.15.122:58562) 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2429989979 learned_addr learned my addr 172.21.15.122:0/2429989979 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48106980 0x7f6d4819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 0x7f6d48198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2429989979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48106980 msgr2=0x7f6d4819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48106980 0x7f6d4819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2429989979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 msgr2=0x7f6d48198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 0x7f6d48198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2429989979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d4819d420 con 0x7f6d48100970 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.707+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 0x7f6d48198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009230 tx=0x7f6d3802cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d44ff9640 1 -- 172.21.15.122:0/2429989979 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38041070 con 0x7f6d48100970 2024-04-02T13:46:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d44ff9640 1 -- 172.21.15.122:0/2429989979 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d38004450 con 0x7f6d48100970 2024-04-02T13:46:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d4819d6b0 con 0x7f6d48100970 2024-04-02T13:46:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d44ff9640 1 -- 172.21.15.122:0/2429989979 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d38003e50 con 0x7f6d48100970 2024-04-02T13:46:42.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.708+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d481a4fa0 con 0x7f6d48100970 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.710+0000 7f6d44ff9640 1 -- 172.21.15.122:0/2429989979 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6d38044070 con 0x7f6d48100970 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.710+0000 7f6d5002b740 1 Processor -- start 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.710+0000 7f6d5002b740 1 -- start start 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690450 0x56000e690830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56000e5f47b0 con 0x56000e690d70 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56000e593570 con 0x56000e69e0b0 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d5002b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56000e5f0760 con 0x56000e690450 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d44ff9640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 0x7f6d340a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58566/0 (socket says 172.21.15.122:58566) 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40204/0 (socket says 172.21.15.122:40204) 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2337404165 learned_addr learned my addr 172.21.15.122:0/2337404165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d477fe640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 0x7f6d340a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d477fe640 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690450 0x56000e690830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d44ff9640 1 -- 172.21.15.122:0/2429989979 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f6d380f2380 con 0x7f6d48100970 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/2337404165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690450 msgr2=0x56000e690830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690450 0x56000e690830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/2337404165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 msgr2=0x56000e69db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/2337404165 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000e3083f0 con 0x56000e69e0b0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d477fe640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 0x7f6d340a6380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40009e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.711+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fa0 tx=0x7f6d300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc777b1edea78b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d3d7fa640 1 -- 172.21.15.122:0/2337404165 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30004030 con 0x56000e69e0b0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d3d7fa640 1 -- 172.21.15.122:0/2337404165 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30005500 con 0x56000e69e0b0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d3d7fa640 1 -- 172.21.15.122:0/2337404165 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300035f0 con 0x56000e69e0b0 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 -- 172.21.15.122:0/2337404165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 msgr2=0x56000e6a04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009fa0 tx=0x7f6d300118d0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 -- 172.21.15.122:0/2337404165 shutdown_connections 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e69e0b0 0x56000e6a04a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2337404165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690450 0x56000e690830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 -- 172.21.15.122:0/2337404165 >> 172.21.15.122:0/2337404165 conn(0x56000e68aed0 msgr2=0x56000e68b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 -- 172.21.15.122:0/2337404165 shutdown_connections 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.712+0000 7f6d5002b740 1 -- 172.21.15.122:0/2337404165 wait complete. 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 -- start start 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 Processor -- start 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56000e6a0ef0 con 0x56000e69e0b0 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56000e5afe80 con 0x56000e690450 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56000e6a1070 con 0x56000e690d70 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d5002b740 10 client.0 _ll_register_callbacks cb 0x56000e5ebaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58582/0 (socket says 172.21.15.122:58582) 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d477fe640 1 -- 172.21.15.122:0/4140394961 learned_addr learned my addr 172.21.15.122:0/4140394961 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 -- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 msgr2=0x56000e69ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 -- 172.21.15.122:0/4140394961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 msgr2=0x56000e69b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d46ffd640 1 -- 172.21.15.122:0/4140394961 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000e3d27d0 con 0x56000e690450 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.713+0000 7f6d477fe640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69dad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009900 tx=0x7f6d380057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8495c86cf91a4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380d7060 con 0x56000e690450 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f6d380ba760 con 0x56000e690450 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30005500 con 0x56000e690450 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d3804b040 con 0x56000e690450 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 10 client.28001 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56000e6acff0 con 0x56000e690450 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30005500 con 0x56000e690450 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f6d380bd540 con 0x56000e690450 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 10 client.28001 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 10 client.28001 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 20 client.28001 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e6ae450 0x56000e6b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 4 client.28001 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.714+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56000e601cf0 con 0x56000e6ae450 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.715+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e6ae450 0x56000e6b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.715+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e6ae450 0x56000e6b0880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=478a98871a1e7c01 server_cookie=e7801288f4853633 in_seq=0 out_seq=0 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.715+0000 7f6d2e7fc640 10 client.28001 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.715+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/4140394961 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56000e601cf0 con 0x56000e6ae450 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.715+0000 7f6d2e7fc640 10 client.28001 handle_command_reply: tid=0 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 client.28001 shutdown 2024-04-02T13:46:42.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 msgr2=0x56000e69dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69dad0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009900 tx=0x7f6d380057c0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 shutdown_connections 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e6ae450 0x56000e6b0880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e69b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d2e7fc640 0 client.28001 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 --2- 172.21.15.122:0/4140394961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69dad0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 >> 172.21.15.122:0/4140394961 conn(0x56000e68aed0 msgr2=0x56000e68cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 shutdown_connections 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 1 -- 172.21.15.122:0/4140394961 wait complete. 2024-04-02T13:46:42.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.716+0000 7f6d5002b740 20 client.28001 trim_cache size 0 max 16384 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.730+0000 7f6d5002b740 1 Processor -- start 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.730+0000 7f6d5002b740 1 -- start start 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e6a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e69e0b0 0x56000e69dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481acef0 con 0x56000e690d70 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481aea30 con 0x56000e690450 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d5002b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481ad5c0 con 0x56000e69e0b0 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58598/0 (socket says 172.21.15.122:58598) 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e6a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/926449955 learned_addr learned my addr 172.21.15.122:0/926449955 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d477fe640 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e69e0b0 0x56000e69dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/926449955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e69e0b0 msgr2=0x56000e69dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e69e0b0 0x56000e69dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/926449955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 msgr2=0x56000e6a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e6a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 -- 172.21.15.122:0/926449955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56000e69b7e0 con 0x56000e690d70 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e6a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000a010 tx=0x7f6d300049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890bfb28befeab53 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3cff9640 1 -- 172.21.15.122:0/926449955 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d300042e0 con 0x56000e690d70 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3cff9640 1 -- 172.21.15.122:0/926449955 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30004480 con 0x56000e690d70 2024-04-02T13:46:42.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.731+0000 7f6d3cff9640 1 -- 172.21.15.122:0/926449955 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d30003950 con 0x56000e690d70 2024-04-02T13:46:42.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- 172.21.15.122:0/926449955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 msgr2=0x56000e69d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6d3000a010 tx=0x7f6d300049e0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- 172.21.15.122:0/926449955 shutdown_connections 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e69e0b0 0x56000e69dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e690d70 0x56000e69d590 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 --2- 172.21.15.122:0/926449955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e6a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- 172.21.15.122:0/926449955 >> 172.21.15.122:0/926449955 conn(0x56000e68aed0 msgr2=0x56000e696150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- 172.21.15.122:0/926449955 shutdown_connections 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- 172.21.15.122:0/926449955 wait complete. 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.732+0000 7f6d5002b740 1 -- start start 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 Processor -- start 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481ae360 con 0x56000e69e0b0 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481adc90 con 0x56000e690450 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d481aaef0 con 0x56000e690d70 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d5002b740 10 client.0 _ll_register_callbacks cb 0x7f6d380ba270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40232/0 (socket says 172.21.15.122:40232) 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58604/0 (socket says 172.21.15.122:58604) 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2277286190 learned_addr learned my addr 172.21.15.122:0/2277286190 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 msgr2=0x56000e69f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2277286190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 msgr2=0x56000e6a5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 -- 172.21.15.122:0/2277286190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000e6a5980 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d477fe640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d46ffd640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d380baac0 tx=0x7f6d380ba440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683785648950afb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.733+0000 7f6d2e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d380bda50 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d30004480 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f6d380bd540 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6d380efaa0 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 10 client.28007 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56000e6a5980 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d30004480 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f6d380a71a0 con 0x56000e690450 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 10 client.28007 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 10 client.28007 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:46:42.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 20 client.28007 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e714ba0 0x56000e716fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 4 client.28007 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56000e5fc040 con 0x56000e714ba0 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.734+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e714ba0 0x56000e716fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d2e7fc640 10 client.28007 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d3e7fc640 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e714ba0 0x56000e716fc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84cffbb051ec24a6 server_cookie=7ee9bea14c1f899c in_seq=0 out_seq=0 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d2e7fc640 1 -- 172.21.15.122:0/2277286190 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56000e5fc040 con 0x56000e714ba0 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d2e7fc640 10 client.28007 handle_command_reply: tid=0 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 client.28007 shutdown 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 msgr2=0x56000e69ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6d380baac0 tx=0x7f6d380ba440 comp rx=0 tx=0).stop 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 shutdown_connections 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56000e714ba0 0x56000e716fc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56000e69e0b0 0x56000e6a5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d2e7fc640 0 client.28007 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56000e690d70 0x56000e69f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 --2- 172.21.15.122:0/2277286190 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56000e690450 0x56000e69ec60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 >> 172.21.15.122:0/2277286190 conn(0x56000e68aed0 msgr2=0x56000e68c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 shutdown_connections 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 1 -- 172.21.15.122:0/2277286190 wait complete. 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.735+0000 7f6d5002b740 20 client.28007 trim_cache size 0 max 16384 2024-04-02T13:46:42.745 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 msgr2=0x7f6d340a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 0x7f6d340a6380 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6d40005fd0 tx=0x7f6d40009e80 comp rx=0 tx=0).stop 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 msgr2=0x7f6d48198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009230 tx=0x7f6d3802cb30 comp rx=0 tx=0).stop 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 shutdown_connections 2024-04-02T13:46:42.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d340a3ec0 0x7f6d340a6380 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d48106980 0x7f6d4819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d48101330 0x7f6d48198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 --2- 172.21.15.122:0/2429989979 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d48100970 0x7f6d48198430 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 >> 172.21.15.122:0/2429989979 conn(0x7f6d48077b00 msgr2=0x7f6d48069040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 shutdown_connections 2024-04-02T13:46:42.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.736+0000 7f6d4d367640 1 -- 172.21.15.122:0/2429989979 wait complete. 2024-04-02T13:46:42.749 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:42 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/256137643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:42.751 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:46:42.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:42 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/256137643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 -- 172.21.15.122:0/1527566588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 msgr2=0x7f5334107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 --2- 172.21.15.122:0/1527566588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f5334107e70 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f5324009930 tx=0x7f532402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 -- 172.21.15.122:0/1527566588 shutdown_connections 2024-04-02T13:46:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 --2- 172.21.15.122:0/1527566588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334102440 0x7f533410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 --2- 172.21.15.122:0/1527566588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334101a80 0x7f5334101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 --2- 172.21.15.122:0/1527566588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f5334107e70 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 -- 172.21.15.122:0/1527566588 >> 172.21.15.122:0/1527566588 conn(0x7f53340fd590 msgr2=0x7f53340ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 -- 172.21.15.122:0/1527566588 shutdown_connections 2024-04-02T13:46:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.857+0000 7f533bf52640 1 -- 172.21.15.122:0/1527566588 wait complete. 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 Processor -- start 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 -- start start 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334101a80 0x7f5334198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334102440 0x7f5334198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5334111ea0 con 0x7f5334107a90 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5334111d20 con 0x7f5334102440 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533bf52640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5334112020 con 0x7f5334101a80 2024-04-02T13:46:42.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f5339cee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334101a80 0x7f5334198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58632/0 (socket says 172.21.15.122:58632) 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 -- 172.21.15.122:0/1836721006 learned_addr learned my addr 172.21.15.122:0/1836721006 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f53394ed640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334102440 0x7f5334198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 -- 172.21.15.122:0/1836721006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334101a80 msgr2=0x7f5334198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334101a80 0x7f5334198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 -- 172.21.15.122:0/1836721006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334102440 msgr2=0x7f5334198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334102440 0x7f5334198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 -- 172.21.15.122:0/1836721006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533419d3f0 con 0x7f5334107a90 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.858+0000 7f533a4ef640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eb10 tx=0x7f532c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.859+0000 7f532affd640 1 -- 172.21.15.122:0/1836721006 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c004b20 con 0x7f5334107a90 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.859+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533419d6e0 con 0x7f5334107a90 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.859+0000 7f532affd640 1 -- 172.21.15.122:0/1836721006 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f532c004cc0 con 0x7f5334107a90 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.859+0000 7f532affd640 1 -- 172.21.15.122:0/1836721006 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f532c010760 con 0x7f5334107a90 2024-04-02T13:46:42.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.859+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53341a4f70 con 0x7f5334107a90 2024-04-02T13:46:42.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.861+0000 7f532affd640 1 -- 172.21.15.122:0/1836721006 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f532c005020 con 0x7f5334107a90 2024-04-02T13:46:42.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.861+0000 7f533ec18740 1 Processor -- start 2024-04-02T13:46:42.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 -- start start 2024-04-02T13:46:42.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 0x559c0b521830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b52f0b0 0x559c0b5314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c0b4857b0 con 0x559c0b52f0b0 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559c0b424570 con 0x559c0b521450 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533ec18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c0b481760 con 0x559c0b521d70 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f53394ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 0x559c0b521830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f532affd640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34644/0 (socket says 172.21.15.122:34644) 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 -- 172.21.15.122:0/4109571627 learned_addr learned my addr 172.21.15.122:0/4109571627 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f53394ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 0x559c0b521830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:40254/0 (socket says 172.21.15.122:40254) 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f53394ed640 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b52f0b0 0x559c0b5314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f533a4ef640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 -- 172.21.15.122:0/4109571627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 msgr2=0x559c0b521830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 0x559c0b521830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 -- 172.21.15.122:0/4109571627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b52f0b0 msgr2=0x559c0b5314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b52f0b0 0x559c0b5314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 -- 172.21.15.122:0/4109571627 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0b1993f0 con 0x559c0b521d70 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f532affd640 1 -- 172.21.15.122:0/1836721006 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f532c0d1280 con 0x7f5334107a90 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.862+0000 7f5339cee640 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f532402cfd0 tx=0x7f53240095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edfa50fe61d16b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f531ffff640 1 -- 172.21.15.122:0/4109571627 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324042070 con 0x559c0b521d70 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533a4ef640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 0x7f53200a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f532c0997a0 tx=0x7f532c00efd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f531ffff640 1 -- 172.21.15.122:0/4109571627 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5324004030 con 0x559c0b521d70 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f531ffff640 1 -- 172.21.15.122:0/4109571627 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324004b50 con 0x559c0b521d70 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 -- 172.21.15.122:0/4109571627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 msgr2=0x559c0b52eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f532402cfd0 tx=0x7f53240095f0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 -- 172.21.15.122:0/4109571627 shutdown_connections 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b52f0b0 0x559c0b5314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521d70 0x559c0b52eb70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 --2- 172.21.15.122:0/4109571627 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521450 0x559c0b521830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.863+0000 7f533ec18740 1 -- 172.21.15.122:0/4109571627 >> 172.21.15.122:0/4109571627 conn(0x559c0b51bed0 msgr2=0x559c0b51c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- 172.21.15.122:0/4109571627 shutdown_connections 2024-04-02T13:46:42.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- 172.21.15.122:0/4109571627 wait complete. 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- start start 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 Processor -- start 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b52f0b0 0x559c0b52c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559c0b531e90 con 0x559c0b521d70 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559c0b440e80 con 0x559c0b52f0b0 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c0b532010 con 0x559c0b521450 2024-04-02T13:46:42.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f533ec18740 10 client.0 _ll_register_callbacks cb 0x559c0b47caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f5339cee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f53394ed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f53394ed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34646/0 (socket says 172.21.15.122:34646) 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f5339cee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58648/0 (socket says 172.21.15.122:58648) 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f53394ed640 1 -- 172.21.15.122:0/2701661847 learned_addr learned my addr 172.21.15.122:0/2701661847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.864+0000 7f53394ed640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b52f0b0 0x559c0b52c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 -- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b52f0b0 msgr2=0x559c0b52c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b52f0b0 0x559c0b52c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 -- 172.21.15.122:0/2701661847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 msgr2=0x559c0b52bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 -- 172.21.15.122:0/2701661847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0b2637d0 con 0x559c0b521450 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f5339cee640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f53394ed640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5330011d20 tx=0x7f5330004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defb0db6ec9a111c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f531d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533000fe60 con 0x559c0b521450 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5324004030 con 0x559c0b521450 2024-04-02T13:46:42.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f5330022470 con 0x559c0b521450 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f533002d050 con 0x559c0b521450 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f533ec18740 10 client.27796 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c0b53dff0 con 0x559c0b521450 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.865+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5324004030 con 0x559c0b521450 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f533000e6c0 con 0x559c0b521450 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 10 client.27796 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 10 client.27796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 20 client.27796 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b53f400 0x559c0b541850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 4 client.27796 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c0b492cf0 con 0x559c0b53f400 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f5339cee640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b53f400 0x559c0b541850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f5339cee640 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b53f400 0x559c0b541850 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ec033543dd47bb7 server_cookie=f9873205cfb159d3 in_seq=0 out_seq=0 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.866+0000 7f531d7fa640 10 client.27796 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.867+0000 7f531d7fa640 1 -- 172.21.15.122:0/2701661847 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c0b492cf0 con 0x559c0b53f400 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.867+0000 7f531d7fa640 10 client.27796 handle_command_reply: tid=0 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.867+0000 7f533ec18740 1 client.27796 shutdown 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 msgr2=0x559c0b52eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5330011d20 tx=0x7f5330004730 comp rx=0 tx=0).stop 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 shutdown_connections 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b53f400 0x559c0b541850 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b52f0b0 0x559c0b52c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521d70 0x559c0b52bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 --2- 172.21.15.122:0/2701661847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b521450 0x559c0b52eb10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f531d7fa640 0 client.27796 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:42.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 >> 172.21.15.122:0/2701661847 conn(0x559c0b51bed0 msgr2=0x559c0b51dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 shutdown_connections 2024-04-02T13:46:42.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 1 -- 172.21.15.122:0/2701661847 wait complete. 2024-04-02T13:46:42.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.868+0000 7f533ec18740 20 client.27796 trim_cache size 0 max 16384 2024-04-02T13:46:42.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 Processor -- start 2024-04-02T13:46:42.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 -- start start 2024-04-02T13:46:42.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b53adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b52ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53341acf70 con 0x559c0b521450 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53341aeab0 con 0x559c0b521d70 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f533ec18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53341ad640 con 0x559c0b52f0b0 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58650/0 (socket says 172.21.15.122:58650) 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 -- 172.21.15.122:0/1832346704 learned_addr learned my addr 172.21.15.122:0/1832346704 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f5339cee640 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b53adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b52ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 -- 172.21.15.122:0/1832346704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 msgr2=0x559c0b52ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b52ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 -- 172.21.15.122:0/1832346704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 msgr2=0x559c0b53adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b53adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.882+0000 7f53394ed640 1 -- 172.21.15.122:0/1832346704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c0b51d090 con 0x559c0b521450 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f53394ed640 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5330011a80 tx=0x7f5330004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991ee84d93d13782 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f531f7fe640 1 -- 172.21.15.122:0/1832346704 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533000fe60 con 0x559c0b521450 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f531f7fe640 1 -- 172.21.15.122:0/1832346704 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533000ed00 con 0x559c0b521450 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f531f7fe640 1 -- 172.21.15.122:0/1832346704 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53300055f0 con 0x559c0b521450 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f533ec18740 1 -- 172.21.15.122:0/1832346704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 msgr2=0x559c0b531490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.883+0000 7f533ec18740 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f5330011a80 tx=0x7f5330004680 comp rx=0 tx=0).stop 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- 172.21.15.122:0/1832346704 shutdown_connections 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b52ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b53adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- 172.21.15.122:0/1832346704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b531490 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- 172.21.15.122:0/1832346704 >> 172.21.15.122:0/1832346704 conn(0x559c0b51bed0 msgr2=0x559c0b52a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- 172.21.15.122:0/1832346704 shutdown_connections 2024-04-02T13:46:42.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- 172.21.15.122:0/1832346704 wait complete. 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- start start 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 Processor -- start 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b5300f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b530630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53341ae3e0 con 0x559c0b521450 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53341add10 con 0x559c0b521d70 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53341aaf70 con 0x559c0b52f0b0 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f533ec18740 10 client.0 _ll_register_callbacks cb 0x7f5330011550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f53394ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f53394ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58656/0 (socket says 172.21.15.122:58656) 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f53394ed640 1 -- 172.21.15.122:0/676129540 learned_addr learned my addr 172.21.15.122:0/676129540 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.884+0000 7f5339cee640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b5300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b530630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 -- 172.21.15.122:0/676129540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 msgr2=0x559c0b530630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b530630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 -- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 msgr2=0x559c0b5300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b5300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 -- 172.21.15.122:0/676129540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c0b5365a0 con 0x559c0b521450 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f5339cee640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b5300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f53394ed640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f533000f010 tx=0x7f5330011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93f015139e31756d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f531d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5330011ba0 con 0x559c0b521450 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f533000e800 con 0x559c0b521450 2024-04-02T13:46:42.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533000ed00 con 0x559c0b521450 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f5330004020 con 0x559c0b521450 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f533ec18740 10 client.19029 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559c0b5378a0 con 0x559c0b521450 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.885+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f533000ed00 con 0x559c0b521450 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f5330004780 con 0x559c0b521450 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 10 client.19029 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 20 client.19029 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b5a5af0 0x559c0b5a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 4 client.19029 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c0b48d040 con 0x559c0b5a5af0 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f5339cee640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b5a5af0 0x559c0b5a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f531d7fa640 10 client.19029 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f5339cee640 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b5a5af0 0x559c0b5a7f10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2032f77ee361ea64 server_cookie=a7fd9de9d720db80 in_seq=0 out_seq=0 2024-04-02T13:46:42.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f531d7fa640 1 -- 172.21.15.122:0/676129540 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c0b48d040 con 0x559c0b5a5af0 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f531d7fa640 10 client.19029 handle_command_reply: tid=0 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.886+0000 7f533ec18740 1 client.19029 shutdown 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 msgr2=0x559c0b52fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f533000f010 tx=0x7f5330011440 comp rx=0 tx=0).stop 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 shutdown_connections 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559c0b5a5af0 0x559c0b5a7f10 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c0b52f0b0 0x559c0b530630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559c0b521d70 0x559c0b5300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 --2- 172.21.15.122:0/676129540 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559c0b521450 0x559c0b52fbb0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f531d7fa640 0 client.19029 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 >> 172.21.15.122:0/676129540 conn(0x559c0b51bed0 msgr2=0x559c0b51dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 shutdown_connections 2024-04-02T13:46:42.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 1 -- 172.21.15.122:0/676129540 wait complete. 2024-04-02T13:46:42.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533ec18740 20 client.19029 trim_cache size 0 max 16384 2024-04-02T13:46:42.899 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 msgr2=0x7f53200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 0x7f53200a61e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f532c0997a0 tx=0x7f532c00efd0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 msgr2=0x7f533419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f532c00eb10 tx=0x7f532c00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 shutdown_connections 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f53200a3d20 0x7f53200a61e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5334107a90 0x7f533419ccc0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5334102440 0x7f5334198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 --2- 172.21.15.122:0/1836721006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5334101a80 0x7f5334198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 >> 172.21.15.122:0/1836721006 conn(0x7f53340fd590 msgr2=0x7f53340fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 shutdown_connections 2024-04-02T13:46:42.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:42.887+0000 7f533bf52640 1 -- 172.21.15.122:0/1836721006 wait complete. 2024-04-02T13:46:42.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 434966 3.73 MB/sec execute 483 sec latency 16.927 ms 2024-04-02T13:46:42.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:42 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/256137643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:46:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:43 smithi188 ceph-mon[36117]: pgmap v359: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-02T13:46:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:43 smithi122 ceph-mon[32114]: pgmap v359: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-02T13:46:43.886 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:43 smithi187 ceph-mon[36064]: pgmap v359: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 251 op/s 2024-04-02T13:46:43.902 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:46:43.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:46:43.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:46:43.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 436695 3.75 MB/sec execute 484 sec latency 14.536 ms 2024-04-02T13:46:44.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 -- 172.21.15.122:0/78959863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 msgr2=0x7f8f8c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 --2- 172.21.15.122:0/78959863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c10d840 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8f84009cd0 tx=0x7f8f8402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 -- 172.21.15.122:0/78959863 shutdown_connections 2024-04-02T13:46:44.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 --2- 172.21.15.122:0/78959863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c10d840 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 --2- 172.21.15.122:0/78959863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c100970 0x7f8f8c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 --2- 172.21.15.122:0/78959863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c106980 0x7f8f8c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 -- 172.21.15.122:0/78959863 >> 172.21.15.122:0/78959863 conn(0x7f8f8c077b00 msgr2=0x7f8f8c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 -- 172.21.15.122:0/78959863 shutdown_connections 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 -- 172.21.15.122:0/78959863 wait complete. 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.010+0000 7f8f93738640 1 Processor -- start 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 -- start start 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c100970 0x7f8f8c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c106980 0x7f8f8c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110d90 con 0x7f8f8c101330 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110c10 con 0x7f8f8c106980 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c110f10 con 0x7f8f8c100970 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58674/0 (socket says 172.21.15.122:58674) 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2753177592 learned_addr learned my addr 172.21.15.122:0/2753177592 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c106980 0x7f8f8c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f914d4640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c100970 0x7f8f8c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2753177592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c100970 msgr2=0x7f8f8c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c100970 0x7f8f8c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2753177592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c106980 msgr2=0x7f8f8c1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c106980 0x7f8f8c1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2753177592 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c1a47d0 con 0x7f8f8c101330 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00d950 tx=0x7f8f7c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f827fc640 1 -- 172.21.15.122:0/2753177592 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c017030 con 0x7f8f8c101330 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c1a49a0 con 0x7f8f8c101330 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f827fc640 1 -- 172.21.15.122:0/2753177592 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c004020 con 0x7f8f8c101330 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.011+0000 7f8f827fc640 1 -- 172.21.15.122:0/2753177592 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0047a0 con 0x7f8f8c101330 2024-04-02T13:46:44.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.012+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a4e60 con 0x7f8f8c101330 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.014+0000 7f8f827fc640 1 -- 172.21.15.122:0/2753177592 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8f7c0041d0 con 0x7f8f8c101330 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.014+0000 7f8f963f1740 1 Processor -- start 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.014+0000 7f8f963f1740 1 -- start start 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247cf2c0 0x560d247cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560d24733770 con 0x560d247d87c0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560d246db960 con 0x560d247cfbe0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f963f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560d2472f8c0 con 0x560d247cf2c0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f827fc640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 0x7f8f780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f914d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58676/0 (socket says 172.21.15.122:58676) 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 -- 172.21.15.122:0/3830215098 learned_addr learned my addr 172.21.15.122:0/3830215098 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 0x7f8f780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247cf2c0 0x560d247cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f827fc640 1 -- 172.21.15.122:0/2753177592 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f8f7c0d04b0 con 0x7f8f8c101330 2024-04-02T13:46:44.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 -- 172.21.15.122:0/3830215098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247cf2c0 msgr2=0x560d247cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247cf2c0 0x560d247cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 -- 172.21.15.122:0/3830215098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 msgr2=0x560d247d8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 -- 172.21.15.122:0/3830215098 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d2450f7e0 con 0x560d247d87c0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f914d4640 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 0x7f8f780a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005cc0 tx=0x7f8f8402ce20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cd910 tx=0x7f8f7c09c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe35169825ec6879 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.015+0000 7f8f777fe640 1 -- 172.21.15.122:0/3830215098 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c09c800 con 0x560d247d87c0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f777fe640 1 -- 172.21.15.122:0/3830215098 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c014070 con 0x560d247d87c0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f777fe640 1 -- 172.21.15.122:0/3830215098 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c098770 con 0x560d247d87c0 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 -- 172.21.15.122:0/3830215098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 msgr2=0x560d247dabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0cd910 tx=0x7f8f7c09c000 comp rx=0 tx=0).stop 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 -- 172.21.15.122:0/3830215098 shutdown_connections 2024-04-02T13:46:44.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247d87c0 0x560d247dabb0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3830215098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247cf2c0 0x560d247cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 -- 172.21.15.122:0/3830215098 >> 172.21.15.122:0/3830215098 conn(0x560d247c99f0 msgr2=0x560d247c9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 -- 172.21.15.122:0/3830215098 shutdown_connections 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.016+0000 7f8f963f1740 1 -- 172.21.15.122:0/3830215098 wait complete. 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 -- start start 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 Processor -- start 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247d6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560d247db550 con 0x560d247cf2c0 2024-04-02T13:46:44.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560d246eed20 con 0x560d247cfbe0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560d247db6d0 con 0x560d247d87c0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f963f1740 10 client.0 _ll_register_callbacks cb 0x560d246bba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f914d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f914d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58688/0 (socket says 172.21.15.122:58688) 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f914d4640 1 -- 172.21.15.122:0/1860775119 learned_addr learned my addr 172.21.15.122:0/1860775119 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.017+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247d6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 -- 172.21.15.122:0/1860775119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 msgr2=0x560d247d6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247d6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 -- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 msgr2=0x560d247d5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 -- 172.21.15.122:0/1860775119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d2456ff70 con 0x560d247cf2c0 2024-04-02T13:46:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f914d4640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b990 tx=0x7f8f8800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa6f73b96ea2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88003e00 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f8f88011440 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c014070 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8f88005020 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 10 client.19041 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560d24407170 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c014070 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f8f88011640 con 0x560d247cf2c0 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 10 client.19041 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 20 client.19041 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d247e8af0 0x560d247eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 4 client.19041 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.018+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d24740cb0 con 0x560d247e8af0 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.019+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d247e8af0 0x560d247eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.019+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d247e8af0 0x560d247eaf40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9619715ae85ebad0 server_cookie=7fcc363270c54d6e in_seq=0 out_seq=0 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.019+0000 7f8f74ff9640 10 client.19041 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f74ff9640 1 -- 172.21.15.122:0/1860775119 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d24740cb0 con 0x560d247e8af0 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f74ff9640 10 client.19041 handle_command_reply: tid=0 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 client.19041 shutdown 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 msgr2=0x560d247d57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b990 tx=0x7f8f8800be50 comp rx=0 tx=0).stop 2024-04-02T13:46:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 shutdown_connections 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d247e8af0 0x560d247eaf40 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247d6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f74ff9640 0 client.19041 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247d5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 --2- 172.21.15.122:0/1860775119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247d57b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 >> 172.21.15.122:0/1860775119 conn(0x560d247c99f0 msgr2=0x560d247cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 shutdown_connections 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 1 -- 172.21.15.122:0/1860775119 wait complete. 2024-04-02T13:46:44.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.020+0000 7f8f963f1740 20 client.19041 trim_cache size 0 max 16384 2024-04-02T13:46:44.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 Processor -- start 2024-04-02T13:46:44.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 -- start start 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 0x560d247d7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1af100 con 0x560d247cfbe0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b0c40 con 0x560d247cf2c0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f963f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1af7d0 con 0x560d247d87c0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58700/0 (socket says 172.21.15.122:58700) 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2223201872 learned_addr learned my addr 172.21.15.122:0/2223201872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f914d4640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 0x560d247d7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2223201872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 msgr2=0x560d247f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2223201872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 msgr2=0x560d247d7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 0x560d247d7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 -- 172.21.15.122:0/2223201872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d247d4760 con 0x560d247cfbe0 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f914d4640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 0x560d247d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.038+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c000c00 tx=0x7f8f7c0d0820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fa55fb885fb9a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f76ffd640 1 -- 172.21.15.122:0/2223201872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0a04b0 con 0x560d247cfbe0 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f76ffd640 1 -- 172.21.15.122:0/2223201872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c014040 con 0x560d247cfbe0 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f76ffd640 1 -- 172.21.15.122:0/2223201872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0cdb30 con 0x560d247cfbe0 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- 172.21.15.122:0/2223201872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 msgr2=0x560d247e4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c000c00 tx=0x7f8f7c0d0820 comp rx=0 tx=0).stop 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- 172.21.15.122:0/2223201872 shutdown_connections 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cfbe0 0x560d247e4410 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 --2- 172.21.15.122:0/2223201872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cf2c0 0x560d247d7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- 172.21.15.122:0/2223201872 >> 172.21.15.122:0/2223201872 conn(0x560d247c99f0 msgr2=0x560d247cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- 172.21.15.122:0/2223201872 shutdown_connections 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- 172.21.15.122:0/2223201872 wait complete. 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.039+0000 7f8f963f1740 1 -- start start 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 Processor -- start 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1b0570 con 0x560d247cf2c0 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1afea0 con 0x560d247cfbe0 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1ad100 con 0x560d247d87c0 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f963f1740 10 client.0 _ll_register_callbacks cb 0x7f8f880041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58714/0 (socket says 172.21.15.122:58714) 2024-04-02T13:46:44.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 -- 172.21.15.122:0/3141942925 learned_addr learned my addr 172.21.15.122:0/3141942925 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f91cd5640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 -- 172.21.15.122:0/3141942925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 msgr2=0x560d247dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247dd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 -- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 msgr2=0x560d247dcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247dcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.040+0000 7f8f914d4640 1 -- 172.21.15.122:0/3141942925 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d247ddb30 con 0x560d247cf2c0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f914d4640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011ca0 tx=0x7f8f88011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bd228fcf5dc3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8802e070 con 0x560d247cf2c0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f8f88011840 con 0x560d247cf2c0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f7c014040 con 0x560d247cf2c0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8f880051c0 con 0x560d247cf2c0 2024-04-02T13:46:44.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 10 client.19047 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x560d24854370 con 0x560d247cf2c0 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c014040 con 0x560d247cf2c0 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f8f88004470 con 0x560d247cf2c0 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 10 client.19047 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 10 client.19047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 20 client.19047 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d24854af0 0x560d24856f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 4 client.19047 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560d2473b000 con 0x560d24854af0 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.041+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d24854af0 0x560d24856f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.042+0000 7f8f74ff9640 10 client.19047 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.042+0000 7f8f90cd3640 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d24854af0 0x560d24856f10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21c1983a52dc54b0 server_cookie=8f91a1d905eefe2 in_seq=0 out_seq=0 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f74ff9640 1 -- 172.21.15.122:0/3141942925 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560d2473b000 con 0x560d24854af0 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f74ff9640 10 client.19047 handle_command_reply: tid=0 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 client.19047 shutdown 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 msgr2=0x560d247e0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011ca0 tx=0x7f8f88011440 comp rx=0 tx=0).stop 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 shutdown_connections 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560d24854af0 0x560d24856f10 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560d247d87c0 0x560d247dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560d247cfbe0 0x560d247dcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 --2- 172.21.15.122:0/3141942925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560d247cf2c0 0x560d247e0b80 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f74ff9640 0 client.19047 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 >> 172.21.15.122:0/3141942925 conn(0x560d247c99f0 msgr2=0x560d247c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 shutdown_connections 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 1 -- 172.21.15.122:0/3141942925 wait complete. 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.043+0000 7f8f963f1740 20 client.19047 trim_cache size 0 max 16384 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "fc4fa322-0b5a-4035-8ede-8743b0a7efaa", 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:46:44.053 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:44.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 msgr2=0x7f8f780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 0x7f8f780a61e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8f84005cc0 tx=0x7f8f8402ce20 comp rx=0 tx=0).stop 2024-04-02T13:46:44.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 msgr2=0x7f8f8c199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00d950 tx=0x7f8f7c00de10 comp rx=0 tx=0).stop 2024-04-02T13:46:44.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 shutdown_connections 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8f780a3d20 0x7f8f780a61e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8f8c106980 0x7f8f8c1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f8c101330 0x7f8f8c199750 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 --2- 172.21.15.122:0/2753177592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f8c100970 0x7f8f8c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 >> 172.21.15.122:0/2753177592 conn(0x7f8f8c077b00 msgr2=0x7f8f8c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 shutdown_connections 2024-04-02T13:46:44.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.044+0000 7f8f93738640 1 -- 172.21.15.122:0/2753177592 wait complete. 2024-04-02T13:46:44.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:46:44.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:46:44.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 -- 172.21.15.122:0/2562904687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af8068d00 msgr2=0x7f7af8069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/2562904687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af8068d00 0x7f7af8069180 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009930 tx=0x7f7ae402bc30 comp rx=0 tx=0).stop 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 -- 172.21.15.122:0/2562904687 shutdown_connections 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/2562904687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af80696c0 0x7f7af810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/2562904687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af8068d00 0x7f7af8069180 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/2562904687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 0x7f7af8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.165+0000 7f7aff1e9640 1 -- 172.21.15.122:0/2562904687 >> 172.21.15.122:0/2562904687 conn(0x7f7af8077b00 msgr2=0x7f7af8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- 172.21.15.122:0/2562904687 shutdown_connections 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- 172.21.15.122:0/2562904687 wait complete. 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 Processor -- start 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- start start 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af8068d00 0x7f7af8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 0x7f7af819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8111e70 con 0x7f7af80696c0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8111cf0 con 0x7f7af8107b50 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7aff1e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8111ff0 con 0x7f7af8068d00 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af8068d00 0x7f7af8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58736/0 (socket says 172.21.15.122:58736) 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.166+0000 7f7af7fff640 1 -- 172.21.15.122:0/996727139 learned_addr learned my addr 172.21.15.122:0/996727139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7afd786640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 0x7f7af819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 -- 172.21.15.122:0/996727139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af8068d00 msgr2=0x7f7af8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af8068d00 0x7f7af8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 -- 172.21.15.122:0/996727139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 msgr2=0x7f7af819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 0x7f7af819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 -- 172.21.15.122:0/996727139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af819d340 con 0x7f7af80696c0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af7fff640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009230 tx=0x7f7ae40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af5ffb640 1 -- 172.21.15.122:0/996727139 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4041070 con 0x7f7af80696c0 2024-04-02T13:46:44.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af5ffb640 1 -- 172.21.15.122:0/996727139 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae402ccc0 con 0x7f7af80696c0 2024-04-02T13:46:44.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7af5ffb640 1 -- 172.21.15.122:0/996727139 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4003de0 con 0x7f7af80696c0 2024-04-02T13:46:44.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af819d5d0 con 0x7f7af80696c0 2024-04-02T13:46:44.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.167+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af81a4ec0 con 0x7f7af80696c0 2024-04-02T13:46:44.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.169+0000 7f7af5ffb640 1 -- 172.21.15.122:0/996727139 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7ae4044070 con 0x7f7af80696c0 2024-04-02T13:46:44.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 Processor -- start 2024-04-02T13:46:44.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 -- start start 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x561384021cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x56138402a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561383f85830 con 0x56138402add0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561383f2d820 con 0x5613840221f0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561383f817e0 con 0x5613840218d0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af5ffb640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 0x7f7ae80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7afd786640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x561384021cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58750/0 (socket says 172.21.15.122:58750) 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x56138402a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 -- 172.21.15.122:0/863680334 learned_addr learned my addr 172.21.15.122:0/863680334 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7afd786640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x561384021cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34726/0 (socket says 172.21.15.122:34726) 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7afd786640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 0x7f7ae80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af5ffb640 1 -- 172.21.15.122:0/996727139 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f7ae40f2760 con 0x7f7af80696c0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 -- 172.21.15.122:0/863680334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 msgr2=0x561384021cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x561384021cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 -- 172.21.15.122:0/863680334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 msgr2=0x56138402a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x56138402a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7af7fff640 1 -- 172.21.15.122:0/863680334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561383d637d0 con 0x56138402add0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.170+0000 7f7afcf85640 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x56138402a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7af7fff640 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40be1d0 tx=0x7f7ae4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20d636971c0cee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7afd786640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 0x7f7ae80a6260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005c40 tx=0x7f7af0004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7ae2ffd640 1 -- 172.21.15.122:0/863680334 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae404b070 con 0x56138402add0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7ae2ffd640 1 -- 172.21.15.122:0/863680334 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae40c10e0 con 0x56138402add0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7ae2ffd640 1 -- 172.21.15.122:0/863680334 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40efd40 con 0x56138402add0 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 -- 172.21.15.122:0/863680334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 msgr2=0x56138402d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40be1d0 tx=0x7f7ae4009260 comp rx=0 tx=0).stop 2024-04-02T13:46:44.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 -- 172.21.15.122:0/863680334 shutdown_connections 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x56138402d1c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x56138402a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/863680334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x561384021cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 -- 172.21.15.122:0/863680334 >> 172.21.15.122:0/863680334 conn(0x56138401c000 msgr2=0x56138401c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 -- 172.21.15.122:0/863680334 shutdown_connections 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.171+0000 7f7b01dd4740 1 -- 172.21.15.122:0/863680334 wait complete. 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 -- start start 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 Processor -- start 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384027a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x561384027f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56138402d980 con 0x56138402add0 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561383f40d30 con 0x5613840221f0 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.172+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56138402db00 con 0x5613840218d0 2024-04-02T13:46:44.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7b01dd4740 10 client.0 _ll_register_callbacks cb 0x561383f40720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afd786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x561384027f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34742/0 (socket says 172.21.15.122:34742) 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 -- 172.21.15.122:0/2722058583 learned_addr learned my addr 172.21.15.122:0/2722058583 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7af7fff640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384027a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 -- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 msgr2=0x561384027a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384027a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 -- 172.21.15.122:0/2722058583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 msgr2=0x561384027f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x561384027f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 -- 172.21.15.122:0/2722058583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561383c993f0 con 0x5613840218d0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afd786640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x561384027f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7afcf85640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00baa0 tx=0x7f7aec00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3d75aff2fd496f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:46:44.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003df0 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f7aec011440 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.173+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae40c10e0 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7aec011640 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 10 client.27832 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x561383c590f0 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae40c10e0 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f7aec01b980 con 0x5613840218d0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 10 client.27832 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 10 client.27832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 20 client.27832 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56138403aac0 0x56138403cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 4 client.27832 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561383f92d70 con 0x56138403aac0 2024-04-02T13:46:44.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.174+0000 7f7af7fff640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56138403aac0 0x56138403cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7ad7fff640 10 client.27832 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7af7fff640 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56138403aac0 0x56138403cf10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832abfecca93d20 server_cookie=6aecff42af2286a5 in_seq=0 out_seq=0 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7ad7fff640 1 -- 172.21.15.122:0/2722058583 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561383f92d70 con 0x56138403aac0 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7ad7fff640 10 client.27832 handle_command_reply: tid=0 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 client.27832 shutdown 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 msgr2=0x56138402a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00baa0 tx=0x7f7aec00bea0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 shutdown_connections 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56138403aac0 0x56138403cf10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56138402add0 0x561384027f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384027a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7ad7fff640 0 client.27832 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/2722058583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840218d0 0x56138402a680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.175+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 >> 172.21.15.122:0/2722058583 conn(0x56138401c000 msgr2=0x56138401dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.176+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 shutdown_connections 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.176+0000 7f7b01dd4740 1 -- 172.21.15.122:0/2722058583 wait complete. 2024-04-02T13:46:44.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.176+0000 7f7b01dd4740 20 client.27832 trim_cache size 0 max 16384 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 Processor -- start 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 -- start start 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384022e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56138402add0 0x56138402a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81af0d0 con 0x5613840218d0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81b0c10 con 0x5613840221f0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81af7a0 con 0x56138402add0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58764/0 (socket says 172.21.15.122:58764) 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7afcf85640 1 -- 172.21.15.122:0/183432428 learned_addr learned my addr 172.21.15.122:0/183432428 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384022e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7afd786640 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56138402add0 0x56138402a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 -- 172.21.15.122:0/183432428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56138402add0 msgr2=0x56138402a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56138402add0 0x56138402a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 -- 172.21.15.122:0/183432428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 msgr2=0x56138401b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7af7fff640 1 -- 172.21.15.122:0/183432428 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138401be30 con 0x5613840221f0 2024-04-02T13:46:44.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.194+0000 7f7afcf85640 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.195+0000 7f7af7fff640 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384022e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005940 tx=0x7f7ae4009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd30b3749c06bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.195+0000 7f7ae27fc640 1 -- 172.21.15.122:0/183432428 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae404b040 con 0x5613840221f0 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.195+0000 7f7ae27fc640 1 -- 172.21.15.122:0/183432428 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae40be4d0 con 0x5613840221f0 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.195+0000 7f7ae27fc640 1 -- 172.21.15.122:0/183432428 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40efd40 con 0x5613840221f0 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- 172.21.15.122:0/183432428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 msgr2=0x561384022e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384022e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005940 tx=0x7f7ae4009680 comp rx=0 tx=0).stop 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- 172.21.15.122:0/183432428 shutdown_connections 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56138402add0 0x56138402a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613840221f0 0x561384022e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/183432428 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x56138401b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- 172.21.15.122:0/183432428 >> 172.21.15.122:0/183432428 conn(0x56138401c000 msgr2=0x561384022710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- 172.21.15.122:0/183432428 shutdown_connections 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- 172.21.15.122:0/183432428 wait complete. 2024-04-02T13:46:44.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 -- start start 2024-04-02T13:46:44.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.196+0000 7f7b01dd4740 1 Processor -- start 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840221f0 0x561384024b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 0x5613840250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81b0540 con 0x5613840218d0 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81afe70 con 0x56138402add0 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7af81ad0d0 con 0x5613840221f0 2024-04-02T13:46:44.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7b01dd4740 10 client.0 _ll_register_callbacks cb 0x7f7aec0044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58768/0 (socket says 172.21.15.122:58768) 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 -- 172.21.15.122:0/604479559 learned_addr learned my addr 172.21.15.122:0/604479559 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afd786640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 0x5613840250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7af7fff640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840221f0 0x561384024b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 -- 172.21.15.122:0/604479559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840221f0 msgr2=0x561384024b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840221f0 0x561384024b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 -- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 msgr2=0x5613840250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 0x5613840250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 -- 172.21.15.122:0/604479559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561384021070 con 0x5613840218d0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afd786640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 0x5613840250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7afcf85640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7aec01be70 tx=0x7f7aec010750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fc1bb25b0dcb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.197+0000 7f7ad7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:46:44.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec02e070 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9525+0+0 (secure 0 0 0) 0x7f7aec005250 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae40be4d0 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7aec003ef0 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 10 client.19065 fetch_fsmap learned FSMap version 41 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5613840a1090 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ae40be4d0 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f7aec010a30 con 0x5613840218d0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 10 client.19065 fetch_fsmap finished waiting for FSMap version 41 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 10 client.19065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 20 client.19065 populate_metadata read hostname 'smithi122' 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613840a17b0 0x5613840a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 4 client.19065 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561383f8d0c0 con 0x5613840a17b0 2024-04-02T13:46:44.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.198+0000 7f7af7fff640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613840a17b0 0x5613840a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7ad7fff640 10 client.19065 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7af7fff640 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613840a17b0 0x5613840a3ba0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e445fba9cc06bb2 server_cookie=8984ec25c545cbf1 in_seq=0 out_seq=0 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7ad7fff640 1 -- 172.21.15.122:0/604479559 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561383f8d0c0 con 0x5613840a17b0 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7ad7fff640 10 client.19065 handle_command_reply: tid=0 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 client.19065 shutdown 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 msgr2=0x561384024620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7aec01be70 tx=0x7f7aec010750 comp rx=0 tx=0).stop 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 shutdown_connections 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613840a17b0 0x5613840a3ba0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56138402add0 0x5613840250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613840221f0 0x561384024b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7ad7fff640 0 client.19065 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 --2- 172.21.15.122:0/604479559 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613840218d0 0x561384024620 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 >> 172.21.15.122:0/604479559 conn(0x56138401c000 msgr2=0x56138401b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.199+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 shutdown_connections 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7b01dd4740 1 -- 172.21.15.122:0/604479559 wait complete. 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7b01dd4740 20 client.19065 trim_cache size 0 max 16384 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "fc4fa322-0b5a-4035-8ede-8743b0a7efaa": { 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "fc4fa322-0b5a-4035-8ede-8743b0a7efaa", 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:46:44.209 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:46:44.210 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:46:44.210 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:46:44.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 msgr2=0x7f7ae80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 0x7f7ae80a6260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7af0005c40 tx=0x7f7af0004ab0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 msgr2=0x7f7af8198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009230 tx=0x7f7ae40043b0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 shutdown_connections 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7ae80a3da0 0x7f7ae80a6260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7af8107b50 0x7f7af819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7af80696c0 0x7f7af8198890 secure :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009230 tx=0x7f7ae40043b0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 --2- 172.21.15.122:0/996727139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7af8068d00 0x7f7af8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.200+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 >> 172.21.15.122:0/996727139 conn(0x7f7af8077b00 msgr2=0x7f7af80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.201+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 shutdown_connections 2024-04-02T13:46:44.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:46:44.203+0000 7f7aff1e9640 1 -- 172.21.15.122:0/996727139 wait complete. 2024-04-02T13:46:44.220 INFO:tasks.cephfs.filesystem:scrub status for tag:fc4fa322-0b5a-4035-8ede-8743b0a7efaa - {'path': '/', 'tag': 'fc4fa322-0b5a-4035-8ede-8743b0a7efaa', 'options': 'recursive,force'} 2024-04-02T13:46:44.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:46:44.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:44 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 437922 3.76 MB/sec execute 485 sec latency 15.618 ms 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:44.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:44.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:44 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:46:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:44 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: pgmap v360: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-02T13:46:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:46:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 439839 3.77 MB/sec execute 486 sec latency 16.123 ms 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: pgmap v360: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:46:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: pgmap v360: 129 pgs: 129 active+clean; 363 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:46:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:46:46.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 440834 3.77 MB/sec execute 487 sec latency 22.198 ms 2024-04-02T13:46:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:47 smithi122 ceph-mon[32114]: pgmap v361: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-04-02T13:46:47.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 442876 3.79 MB/sec execute 488 sec latency 15.981 ms 2024-04-02T13:46:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:47 smithi187 ceph-mon[36064]: pgmap v361: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-04-02T13:46:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:47 smithi188 ceph-mon[36117]: pgmap v361: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-04-02T13:46:48.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:48 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:48.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 444146 3.79 MB/sec execute 489 sec latency 38.115 ms 2024-04-02T13:46:48.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:48 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:49.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:48 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:46:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:49 smithi122 ceph-mon[32114]: pgmap v362: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:49.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 446172 3.81 MB/sec execute 490 sec latency 17.989 ms 2024-04-02T13:46:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:49 smithi187 ceph-mon[36064]: pgmap v362: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:49 smithi188 ceph-mon[36117]: pgmap v362: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:50.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 447566 3.82 MB/sec execute 491 sec latency 14.798 ms 2024-04-02T13:46:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:51 smithi122 ceph-mon[32114]: pgmap v363: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:51 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:51 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:51.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 449371 3.82 MB/sec execute 492 sec latency 30.485 ms 2024-04-02T13:46:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:51 smithi187 ceph-mon[36064]: pgmap v363: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:51 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:51 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:51 smithi188 ceph-mon[36117]: pgmap v363: 129 pgs: 129 active+clean; 372 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 206 op/s 2024-04-02T13:46:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:51 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:51 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:46:52.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 450874 3.84 MB/sec execute 493 sec latency 24.242 ms 2024-04-02T13:46:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:52.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:46:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:53.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:46:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:53 smithi122 ceph-mon[32114]: pgmap v364: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-04-02T13:46:53.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 452144 3.84 MB/sec execute 494 sec latency 15.909 ms 2024-04-02T13:46:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:53 smithi187 ceph-mon[36064]: pgmap v364: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-04-02T13:46:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:53 smithi188 ceph-mon[36117]: pgmap v364: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 313 op/s 2024-04-02T13:46:54.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 454259 3.86 MB/sec execute 495 sec latency 17.919 ms 2024-04-02T13:46:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:55 smithi122 ceph-mon[32114]: pgmap v365: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-02T13:46:55.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 455422 3.86 MB/sec execute 496 sec latency 14.476 ms 2024-04-02T13:46:56.099 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:55 smithi188 ceph-mon[36117]: pgmap v365: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-02T13:46:56.100 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:55 smithi187 ceph-mon[36064]: pgmap v365: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 216 op/s 2024-04-02T13:46:56.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 456966 3.87 MB/sec execute 497 sec latency 46.738 ms 2024-04-02T13:46:57.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:57 smithi122 ceph-mon[32114]: pgmap v366: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-04-02T13:46:57.977 INFO:tasks.workunit.client.0.smithi122.stdout: 1 458196 3.88 MB/sec execute 498 sec latency 19.659 ms 2024-04-02T13:46:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:57 smithi187 ceph-mon[36064]: pgmap v366: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-04-02T13:46:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:57 smithi188 ceph-mon[36117]: pgmap v366: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-04-02T13:46:58.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 459158 3.91 MB/sec execute 499 sec latency 52.946 ms 2024-04-02T13:46:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:46:59 smithi122 ceph-mon[32114]: pgmap v367: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:46:59.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 461247 3.93 MB/sec execute 500 sec latency 22.966 ms 2024-04-02T13:46:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:46:59 smithi187 ceph-mon[36064]: pgmap v367: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:47:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:46:59 smithi188 ceph-mon[36117]: pgmap v367: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:47:00.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462437 3.93 MB/sec execute 501 sec latency 164.402 ms 2024-04-02T13:47:01.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:01 smithi122 ceph-mon[32114]: pgmap v368: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:47:01.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462437 3.93 MB/sec execute 502 sec latency 1164.521 ms 2024-04-02T13:47:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:01 smithi187 ceph-mon[36064]: pgmap v368: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:47:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:01 smithi188 ceph-mon[36117]: pgmap v368: 129 pgs: 129 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-04-02T13:47:02.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462437 3.92 MB/sec execute 503 sec latency 2164.609 ms 2024-04-02T13:47:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:03 smithi122 ceph-mon[32114]: pgmap v369: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-02T13:47:03.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462437 3.91 MB/sec execute 504 sec latency 3164.669 ms 2024-04-02T13:47:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:03 smithi187 ceph-mon[36064]: pgmap v369: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-02T13:47:04.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:03 smithi188 ceph-mon[36117]: pgmap v369: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-04-02T13:47:04.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:04 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:47:04.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462509 3.90 MB/sec execute 505 sec latency 3773.086 ms 2024-04-02T13:47:04.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:04 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:47:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:04 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:47:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:05 smithi122 ceph-mon[32114]: pgmap v370: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-04-02T13:47:05.918 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:05 smithi188 ceph-mon[36117]: pgmap v370: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-04-02T13:47:05.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462509 3.90 MB/sec execute 506 sec latency 1292.437 ms 2024-04-02T13:47:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:05 smithi187 ceph-mon[36064]: pgmap v370: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-04-02T13:47:06.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462509 3.89 MB/sec execute 507 sec latency 2292.470 ms 2024-04-02T13:47:07.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462509 3.88 MB/sec execute 508 sec latency 3292.573 ms 2024-04-02T13:47:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:07 smithi187 ceph-mon[36064]: pgmap v371: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-04-02T13:47:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:07 smithi187 ceph-mon[36064]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:47:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:07 smithi188 ceph-mon[36117]: pgmap v371: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-04-02T13:47:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:07 smithi188 ceph-mon[36117]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:47:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:07 smithi122 ceph-mon[32114]: pgmap v371: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 257 op/s 2024-04-02T13:47:08.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:07 smithi122 ceph-mon[32114]: osdmap e106: 12 total, 12 up, 12 in 2024-04-02T13:47:08.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 462921 3.88 MB/sec execute 509 sec latency 3734.288 ms 2024-04-02T13:47:09.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 464231 3.88 MB/sec execute 510 sec latency 58.110 ms 2024-04-02T13:47:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:09 smithi187 ceph-mon[36064]: pgmap v373: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:09 smithi188 ceph-mon[36117]: pgmap v373: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:09 smithi122 ceph-mon[32114]: pgmap v373: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:10.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 465863 3.90 MB/sec execute 511 sec latency 14.719 ms 2024-04-02T13:47:11.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 467425 3.90 MB/sec execute 512 sec latency 34.805 ms 2024-04-02T13:47:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:11 smithi187 ceph-mon[36064]: pgmap v374: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:11 smithi188 ceph-mon[36117]: pgmap v374: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:11 smithi122 ceph-mon[32114]: pgmap v374: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-04-02T13:47:12.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 469293 3.92 MB/sec execute 513 sec latency 16.885 ms 2024-04-02T13:47:13.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 470697 3.92 MB/sec execute 514 sec latency 15.608 ms 2024-04-02T13:47:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:13 smithi187 ceph-mon[36064]: pgmap v375: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:13 smithi188 ceph-mon[36117]: pgmap v375: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:13 smithi122 ceph-mon[32114]: pgmap v375: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:14.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:14.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- 172.21.15.122:0/4104643910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc068d00 msgr2=0x7f95cc0690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 --2- 172.21.15.122:0/4104643910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc068d00 0x7f95cc0690e0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009900 tx=0x7f95bc02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- 172.21.15.122:0/4104643910 shutdown_connections 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 --2- 172.21.15.122:0/4104643910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc10e4a0 0x7f95cc110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 --2- 172.21.15.122:0/4104643910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc10df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 --2- 172.21.15.122:0/4104643910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc068d00 0x7f95cc0690e0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- 172.21.15.122:0/4104643910 >> 172.21.15.122:0/4104643910 conn(0x7f95cc077b00 msgr2=0x7f95cc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- 172.21.15.122:0/4104643910 shutdown_connections 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- 172.21.15.122:0/4104643910 wait complete. 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 Processor -- start 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.333+0000 7f95d33a8640 1 -- start start 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc068d00 0x7f95cc19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc114070 con 0x7f95cc10e4a0 2024-04-02T13:47:14.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc113ef0 con 0x7f95cc069620 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1141f0 con 0x7f95cc068d00 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d0943640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57212/0 (socket says 172.21.15.122:57212) 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 -- 172.21.15.122:0/1765647560 learned_addr learned my addr 172.21.15.122:0/1765647560 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1144640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc068d00 0x7f95cc19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 -- 172.21.15.122:0/1765647560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc068d00 msgr2=0x7f95cc19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc068d00 0x7f95cc19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 -- 172.21.15.122:0/1765647560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 msgr2=0x7f95cc19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 -- 172.21.15.122:0/1765647560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc19f4c0 con 0x7f95cc10e4a0 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d0943640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d1945640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f95c400eb10 tx=0x7f95c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95c27fc640 1 -- 172.21.15.122:0/1765647560 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4004b20 con 0x7f95cc10e4a0 2024-04-02T13:47:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95c27fc640 1 -- 172.21.15.122:0/1765647560 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c4004cc0 con 0x7f95cc10e4a0 2024-04-02T13:47:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc19f7b0 con 0x7f95cc10e4a0 2024-04-02T13:47:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95c27fc640 1 -- 172.21.15.122:0/1765647560 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c4010760 con 0x7f95cc10e4a0 2024-04-02T13:47:14.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.334+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc1a7040 con 0x7f95cc10e4a0 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.337+0000 7f95c27fc640 1 -- 172.21.15.122:0/1765647560 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f95c4005020 con 0x7f95cc10e4a0 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.337+0000 7f95d5f90740 1 Processor -- start 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.337+0000 7f95d5f90740 1 -- start start 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd87e9f830 con 0x55cd87f3c1f0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd87e47820 con 0x55cd87f3b8d0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d5f90740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd87e9b7e0 con 0x55cd87f44dd0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57226/0 (socket says 172.21.15.122:57226) 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1144640 1 -- 172.21.15.122:0/3635106948 learned_addr learned my addr 172.21.15.122:0/3635106948 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95c27fc640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 0x7f95b80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1945640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 0x7f95b80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 -- 172.21.15.122:0/3635106948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 msgr2=0x55cd87f471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 -- 172.21.15.122:0/3635106948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 msgr2=0x55cd87f44890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 -- 172.21.15.122:0/3635106948 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd87c7d7d0 con 0x55cd87f3b8d0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1144640 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95c27fc640 1 -- 172.21.15.122:0/1765647560 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f95c40d1270 con 0x7f95cc10e4a0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d0943640 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3bcb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95cc1003a0 tx=0x7f95c8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a2a746b0a02705 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95d1945640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 0x7f95b80a6260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95c409cd60 tx=0x7f95c40bbf90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.338+0000 7f95b77fe640 1 -- 172.21.15.122:0/3635106948 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c800f910 con 0x55cd87f3b8d0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95b77fe640 1 -- 172.21.15.122:0/3635106948 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8005020 con 0x55cd87f3b8d0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95b77fe640 1 -- 172.21.15.122:0/3635106948 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8004b20 con 0x55cd87f3b8d0 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 -- 172.21.15.122:0/3635106948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 msgr2=0x55cd87f3bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3bcb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f95cc1003a0 tx=0x7f95c8005850 comp rx=0 tx=0).stop 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 -- 172.21.15.122:0/3635106948 shutdown_connections 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f44890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3635106948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3bcb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 -- 172.21.15.122:0/3635106948 >> 172.21.15.122:0/3635106948 conn(0x55cd87f36000 msgr2=0x55cd87f363e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 -- 172.21.15.122:0/3635106948 shutdown_connections 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.339+0000 7f95d5f90740 1 -- 172.21.15.122:0/3635106948 wait complete. 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 -- start start 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 Processor -- start 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f3c1f0 0x55cd87f419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f44dd0 0x55cd87f41f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd87f47bc0 con 0x55cd87f3b8d0 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd87e5ad30 con 0x55cd87f44dd0 2024-04-02T13:47:14.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd87f47d40 con 0x55cd87f3c1f0 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d5f90740 10 client.0 _ll_register_callbacks cb 0x55cd87e5a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57240/0 (socket says 172.21.15.122:57240) 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 -- 172.21.15.122:0/3413811881 learned_addr learned my addr 172.21.15.122:0/3413811881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d1144640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f3c1f0 0x55cd87f419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f44dd0 0x55cd87f41f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 -- 172.21.15.122:0/3413811881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f3c1f0 msgr2=0x55cd87f419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f3c1f0 0x55cd87f419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 -- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f44dd0 msgr2=0x55cd87f41f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.340+0000 7f95d0943640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f44dd0 0x55cd87f41f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d0943640 1 -- 172.21.15.122:0/3413811881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd87bb33f0 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d0943640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f95c800fb00 tx=0x7f95c80114d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffbb1f02ad0fa7c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c802c060 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f95c800fc60 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c8005020 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f95c8022070 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 10 client.19077 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cd87b730f0 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8005020 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f95c801d030 con 0x55cd87f3b8d0 2024-04-02T13:47:14.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 10 client.19077 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 10 client.19077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 20 client.19077 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87f54c40 0x55cd87f57030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 4 client.19077 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.341+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd87eacd70 con 0x55cd87f54c40 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.342+0000 7f95d1144640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87f54c40 0x55cd87f57030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.342+0000 7f95b4ff9640 10 client.19077 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.342+0000 7f95d1144640 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87f54c40 0x55cd87f57030 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36641aacf296f819 server_cookie=b98fc19833a1cb1d in_seq=0 out_seq=0 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95b4ff9640 1 -- 172.21.15.122:0/3413811881 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd87eacd70 con 0x55cd87f54c40 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95b4ff9640 10 client.19077 handle_command_reply: tid=0 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 client.19077 shutdown 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 msgr2=0x55cd87f44640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f95c800fb00 tx=0x7f95c80114d0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 shutdown_connections 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87f54c40 0x55cd87f57030 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f44dd0 0x55cd87f41f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f3c1f0 0x55cd87f419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95b4ff9640 0 client.19077 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 --2- 172.21.15.122:0/3413811881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f44640 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 >> 172.21.15.122:0/3413811881 conn(0x55cd87f36000 msgr2=0x55cd87f37d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 shutdown_connections 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 1 -- 172.21.15.122:0/3413811881 wait complete. 2024-04-02T13:47:14.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.343+0000 7f95d5f90740 20 client.19077 trim_cache size 0 max 16384 2024-04-02T13:47:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 Processor -- start 2024-04-02T13:47:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 -- start start 2024-04-02T13:47:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3c1f0 0x55cd87f40880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f44100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1b1480 con 0x55cd87f3b8d0 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1b2fc0 con 0x55cd87f3c1f0 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d5f90740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1b1b50 con 0x55cd87f44dd0 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57244/0 (socket says 172.21.15.122:57244) 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 -- 172.21.15.122:0/1980318362 learned_addr learned my addr 172.21.15.122:0/1980318362 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d1144640 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3c1f0 0x55cd87f40880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 -- 172.21.15.122:0/1980318362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 msgr2=0x55cd87f44100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f44100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 -- 172.21.15.122:0/1980318362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3c1f0 msgr2=0x55cd87f40880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3c1f0 0x55cd87f40880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 -- 172.21.15.122:0/1980318362 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd87f35e30 con 0x55cd87f3b8d0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.363+0000 7f95d0943640 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f95c80059f0 tx=0x7f95c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ee11b3c9e0a297 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95b6ffd640 1 -- 172.21.15.122:0/1980318362 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c802c040 con 0x55cd87f3b8d0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95b6ffd640 1 -- 172.21.15.122:0/1980318362 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8011030 con 0x55cd87f3b8d0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95b6ffd640 1 -- 172.21.15.122:0/1980318362 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8004cc0 con 0x55cd87f3b8d0 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 -- 172.21.15.122:0/1980318362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 msgr2=0x55cd87f3cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f95c80059f0 tx=0x7f95c8005f90 comp rx=0 tx=0).stop 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 -- 172.21.15.122:0/1980318362 shutdown_connections 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f44100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3c1f0 0x55cd87f40880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 --2- 172.21.15.122:0/1980318362 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3b8d0 0x55cd87f3cfa0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 -- 172.21.15.122:0/1980318362 >> 172.21.15.122:0/1980318362 conn(0x55cd87f36000 msgr2=0x55cd87f40170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 -- 172.21.15.122:0/1980318362 shutdown_connections 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.364+0000 7f95d5f90740 1 -- 172.21.15.122:0/1980318362 wait complete. 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 -- start start 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 Processor -- start 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f3e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1b28f0 con 0x55cd87f3c1f0 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1b2220 con 0x55cd87f3b8d0 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc1af480 con 0x55cd87f44dd0 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d5f90740 10 client.0 _ll_register_callbacks cb 0x7f95c8011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57250/0 (socket says 172.21.15.122:57250) 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 -- 172.21.15.122:0/980411757 learned_addr learned my addr 172.21.15.122:0/980411757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d0943640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d0943640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f3e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 -- 172.21.15.122:0/980411757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 msgr2=0x55cd87f3e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f3e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 -- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 msgr2=0x55cd87f4cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f4cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d1144640 1 -- 172.21.15.122:0/980411757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd87f3e840 con 0x55cd87f3c1f0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.365+0000 7f95d0943640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f3e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d1144640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009610 tx=0x7f95bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93585908cd4ec26 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc02c920 con 0x55cd87f3c1f0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c8011030 con 0x55cd87f3c1f0 2024-04-02T13:47:14.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f95bc02cac0 con 0x55cd87f3c1f0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f95bc05b050 con 0x55cd87f3c1f0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 10 client.19083 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cd87fba990 con 0x55cd87f3c1f0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8011030 con 0x55cd87f3c1f0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f95bc04a6c0 con 0x55cd87f3c1f0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 10 client.19083 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 10 client.19083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 20 client.19083 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87fbb0b0 0x55cd87fbd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 4 client.19083 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.366+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd87ea70c0 con 0x55cd87fbb0b0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d0943640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87fbb0b0 0x55cd87fbd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d0943640 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87fbb0b0 0x55cd87fbd4d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=940b3c598b23a1c5 server_cookie=823822e0de154782 in_seq=0 out_seq=0 2024-04-02T13:47:14.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95b4ff9640 10 client.19083 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95b4ff9640 1 -- 172.21.15.122:0/980411757 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd87ea70c0 con 0x55cd87fbb0b0 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95b4ff9640 10 client.19083 handle_command_reply: tid=0 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 client.19083 shutdown 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 msgr2=0x55cd87f4d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009610 tx=0x7f95bc004270 comp rx=0 tx=0).stop 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 shutdown_connections 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cd87fbb0b0 0x55cd87fbd4d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd87f44dd0 0x55cd87f3e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd87f3c1f0 0x55cd87f4d500 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95b4ff9640 0 client.19083 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 --2- 172.21.15.122:0/980411757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd87f3b8d0 0x55cd87f4cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 >> 172.21.15.122:0/980411757 conn(0x55cd87f36000 msgr2=0x55cd87f40a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 shutdown_connections 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.367+0000 7f95d5f90740 1 -- 172.21.15.122:0/980411757 wait complete. 2024-04-02T13:47:14.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d5f90740 20 client.19083 trim_cache size 0 max 16384 2024-04-02T13:47:14.381 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:47:14.381 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:47:14.381 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 msgr2=0x7f95b80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 0x7f95b80a6260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95c409cd60 tx=0x7f95c40bbf90 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 msgr2=0x7f95cc19ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f95c400eb10 tx=0x7f95c400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 shutdown_connections 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f95b80a3da0 0x7f95b80a6260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95cc10e4a0 0x7f95cc19ed90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95cc069620 0x7f95cc19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 --2- 172.21.15.122:0/1765647560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95cc068d00 0x7f95cc19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 >> 172.21.15.122:0/1765647560 conn(0x7f95cc077b00 msgr2=0x7f95cc10f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 shutdown_connections 2024-04-02T13:47:14.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.368+0000 7f95d33a8640 1 -- 172.21.15.122:0/1765647560 wait complete. 2024-04-02T13:47:14.383 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:47:14.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:47:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 -- 172.21.15.122:0/2669872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01012a0 msgr2=0x7f9af010df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 --2- 172.21.15.122:0/2669872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01012a0 0x7f9af010df60 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009900 tx=0x7f9ad802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 -- 172.21.15.122:0/2669872175 shutdown_connections 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 --2- 172.21.15.122:0/2669872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af0110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 --2- 172.21.15.122:0/2669872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01012a0 0x7f9af010df60 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 --2- 172.21.15.122:0/2669872175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01008f0 0x7f9af0100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 -- 172.21.15.122:0/2669872175 >> 172.21.15.122:0/2669872175 conn(0x7f9af0077b00 msgr2=0x7f9af0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 -- 172.21.15.122:0/2669872175 shutdown_connections 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.489+0000 7f9af49d4640 1 -- 172.21.15.122:0/2669872175 wait complete. 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 Processor -- start 2024-04-02T13:47:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 -- start start 2024-04-02T13:47:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01012a0 0x7f9af019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9af01140d0 con 0x7f9af01008f0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0113f50 con 0x7f9af010e4a0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9af49d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9af0114250 con 0x7f9af01012a0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aee59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57268/0 (socket says 172.21.15.122:57268) 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aee59c640 1 -- 172.21.15.122:0/1549981082 learned_addr learned my addr 172.21.15.122:0/1549981082 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aedd9b640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01012a0 0x7f9af019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 -- 172.21.15.122:0/1549981082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01012a0 msgr2=0x7f9af019abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01012a0 0x7f9af019abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 -- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 msgr2=0x7f9af019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aeed9d640 1 -- 172.21.15.122:0/1549981082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af019f630 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.490+0000 7f9aee59c640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9aeed9d640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af019ef00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0005fd0 tx=0x7f9ae000dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0009a00 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ae0009ba0 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af019f920 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0010460 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.491+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af01a71b0 con 0x7f9af010e4a0 2024-04-02T13:47:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.492+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af00688f0 con 0x7f9af010e4a0 2024-04-02T13:47:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.493+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9ae0003e70 con 0x7f9af010e4a0 2024-04-02T13:47:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.494+0000 7f9ae77fe640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 0x7f9adc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.494+0000 7f9aee59c640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 0x7f9adc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.494+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f9ae00d0510 con 0x7f9af010e4a0 2024-04-02T13:47:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.494+0000 7f9aee59c640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 0x7f9adc0a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009fd0 tx=0x7f9ae8009720 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.498+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9ae0017020 con 0x7f9af010e4a0 2024-04-02T13:47:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.748+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9af0109630 con 0x7f9af010e4a0 2024-04-02T13:47:14.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.749+0000 7f9ae77fe640 1 -- 172.21.15.122:0/1549981082 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11946 (secure 0 0 0) 0x7f9ae009c120 con 0x7f9af010e4a0 2024-04-02T13:47:14.752 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:47:14.752 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:47:03.399575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:47:14.752 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 43 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 msgr2=0x7f9adc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 0x7f9adc0a61e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009fd0 tx=0x7f9ae8009720 comp rx=0 tx=0).stop 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 msgr2=0x7f9af019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af019ef00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0005fd0 tx=0x7f9ae000dbb0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 shutdown_connections 2024-04-02T13:47:14.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9adc0a3d20 0x7f9adc0a61e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9af010e4a0 0x7f9af019ef00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af01012a0 0x7f9af019abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 --2- 172.21.15.122:0/1549981082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9af01008f0 0x7f9af019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 >> 172.21.15.122:0/1549981082 conn(0x7f9af0077b00 msgr2=0x7f9af010fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 shutdown_connections 2024-04-02T13:47:14.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.751+0000 7f9af49d4640 1 -- 172.21.15.122:0/1549981082 wait complete. 2024-04-02T13:47:14.765 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:47:14.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- 172.21.15.122:0/1849614603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4106980 msgr2=0x7f3ab4106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 --2- 172.21.15.122:0/1849614603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4106980 0x7f3ab4106d60 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009930 tx=0x7f3a9c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:14.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- 172.21.15.122:0/1849614603 shutdown_connections 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 --2- 172.21.15.122:0/1849614603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4101330 0x7f3ab410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 --2- 172.21.15.122:0/1849614603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4100970 0x7f3ab4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 --2- 172.21.15.122:0/1849614603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4106980 0x7f3ab4106d60 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- 172.21.15.122:0/1849614603 >> 172.21.15.122:0/1849614603 conn(0x7f3ab4077b00 msgr2=0x7f3ab4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- 172.21.15.122:0/1849614603 shutdown_connections 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- 172.21.15.122:0/1849614603 wait complete. 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 Processor -- start 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.875+0000 7f3aba30e640 1 -- start start 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4101330 0x7f3ab4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4106980 0x7f3ab41a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4110d90 con 0x7f3ab4100970 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4110c10 con 0x7f3ab4101330 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4110f10 con 0x7f3ab4106980 2024-04-02T13:47:14.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57280/0 (socket says 172.21.15.122:57280) 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 -- 172.21.15.122:0/3703968915 learned_addr learned my addr 172.21.15.122:0/3703968915 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4101330 0x7f3ab4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4106980 0x7f3ab41a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 -- 172.21.15.122:0/3703968915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4106980 msgr2=0x7f3ab41a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4106980 0x7f3ab41a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 -- 172.21.15.122:0/3703968915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4101330 msgr2=0x7f3ab4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4101330 0x7f3ab4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 -- 172.21.15.122:0/3703968915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a4790 con 0x7f3ab4100970 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009230 tx=0x7f3a9c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab17fa640 1 -- 172.21.15.122:0/3703968915 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c041070 con 0x7f3ab4100970 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a4960 con 0x7f3ab4100970 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab17fa640 1 -- 172.21.15.122:0/3703968915 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a9c004450 con 0x7f3ab4100970 2024-04-02T13:47:14.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3ab17fa640 1 -- 172.21.15.122:0/3703968915 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c003e50 con 0x7f3ab4100970 2024-04-02T13:47:14.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.876+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a4dd0 con 0x7f3ab4100970 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.878+0000 7f3ab17fa640 1 -- 172.21.15.122:0/3703968915 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3a9c044070 con 0x7f3ab4100970 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 Processor -- start 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 -- start start 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab17fa640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 0x7f3aa00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a30920b7b0 con 0x55a3092a80c0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a3091aa570 con 0x55a3092a77a0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a309207760 con 0x55a3092b0ca0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 0x7f3aa00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57282/0 (socket says 172.21.15.122:57282) 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 -- 172.21.15.122:0/3989800640 learned_addr learned my addr 172.21.15.122:0/3989800640 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab17fa640 1 -- 172.21.15.122:0/3703968915 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3a9c0f2420 con 0x7f3ab4100970 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 -- 172.21.15.122:0/3989800640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 msgr2=0x55a3092b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 -- 172.21.15.122:0/3989800640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 msgr2=0x55a3092a7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab37fe640 1 -- 172.21.15.122:0/3989800640 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a308f1f3f0 con 0x55a3092a80c0 2024-04-02T13:47:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.879+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 0x7f3aa00a6380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005fd0 tx=0x7f3aac009e90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009c90 tx=0x7f3aa4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcba58ef296ba9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3aaa7fc640 1 -- 172.21.15.122:0/3989800640 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4019620 con 0x55a3092a80c0 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3aaa7fc640 1 -- 172.21.15.122:0/3989800640 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa4009e40 con 0x55a3092a80c0 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3aaa7fc640 1 -- 172.21.15.122:0/3989800640 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4018bc0 con 0x55a3092a80c0 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3abcfcd740 1 -- 172.21.15.122:0/3989800640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 msgr2=0x55a3092b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.880+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4009c90 tx=0x7f3aa4011810 comp rx=0 tx=0).stop 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- 172.21.15.122:0/3989800640 shutdown_connections 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092b0760 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/3989800640 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- 172.21.15.122:0/3989800640 >> 172.21.15.122:0/3989800640 conn(0x55a3092a1ed0 msgr2=0x55a3092a22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- 172.21.15.122:0/3989800640 shutdown_connections 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- 172.21.15.122:0/3989800640 wait complete. 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- start start 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 Processor -- start 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 0x55a3092b0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092b0ca0 0x55a3092ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a3092b3a90 con 0x55a3092a80c0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a3091c6e80 con 0x55a3092b0ca0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a3092b3c10 con 0x55a3092a77a0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3abcfcd740 10 client.0 _ll_register_callbacks cb 0x55a309202aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.881+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57288/0 (socket says 172.21.15.122:57288) 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab37fe640 1 -- 172.21.15.122:0/1652469845 learned_addr learned my addr 172.21.15.122:0/1652469845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 0x55a3092b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092b0ca0 0x55a3092ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 -- 172.21.15.122:0/1652469845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092b0ca0 msgr2=0x55a3092ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092b0ca0 0x55a3092ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 -- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 msgr2=0x55a3092adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 -- 172.21.15.122:0/1652469845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a308fe97d0 con 0x55a3092a77a0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 0x55a3092b0720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ef840 tx=0x7f3a9c04d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b78dc6679b8519e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c04b040 con 0x55a3092a77a0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3a9c0ef9b0 con 0x55a3092a77a0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a309218cf0 con 0x55a3092a77a0 2024-04-02T13:47:14.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3a9c004c80 con 0x55a3092a77a0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3abcfcd740 10 client.27874 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.882+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a3092b3af0 con 0x55a3092a77a0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a309218cf0 con 0x55a3092a77a0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f3a9c0ba7b0 con 0x55a3092a77a0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 10 client.27874 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 10 client.27874 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 20 client.27874 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a3092c0f90 0x55a3092c3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 4 client.27874 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a309213040 con 0x55a3092c0f90 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a3092c0f90 0x55a3092c3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3a937fe640 10 client.27874 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.883+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a3092c0f90 0x55a3092c3450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d7cf884f94e5288 server_cookie=10c5bca488b107e8 in_seq=0 out_seq=0 2024-04-02T13:47:14.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3a937fe640 1 -- 172.21.15.122:0/1652469845 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a309213040 con 0x55a3092c0f90 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3a937fe640 10 client.27874 handle_command_reply: tid=0 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 client.27874 shutdown 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 msgr2=0x55a3092b0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 0x55a3092b0720 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ef840 tx=0x7f3a9c04d000 comp rx=0 tx=0).stop 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 shutdown_connections 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a3092c0f90 0x55a3092c3450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092b0ca0 0x55a3092ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/1652469845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a77a0 0x55a3092b0720 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3a937fe640 0 client.27874 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 >> 172.21.15.122:0/1652469845 conn(0x55a3092a1ed0 msgr2=0x55a3092a3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 shutdown_connections 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 1 -- 172.21.15.122:0/1652469845 wait complete. 2024-04-02T13:47:14.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.884+0000 7f3abcfcd740 20 client.27874 trim_cache size 0 max 16384 2024-04-02T13:47:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 Processor -- start 2024-04-02T13:47:14.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 -- start start 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 0x55a3092c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41af070 con 0x55a3092b0ca0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b0bb0 con 0x55a3092a77a0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41af740 con 0x55a3092a80c0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57304/0 (socket says 172.21.15.122:57304) 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 -- 172.21.15.122:0/2871492889 learned_addr learned my addr 172.21.15.122:0/2871492889 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 0x55a3092c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 -- 172.21.15.122:0/2871492889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 msgr2=0x55a3092c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 0x55a3092c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 -- 172.21.15.122:0/2871492889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 msgr2=0x55a3092bc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092bc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab88ab640 1 -- 172.21.15.122:0/2871492889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3092ae560 con 0x55a3092b0ca0 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.898+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 0x55a3092c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009770 tx=0x7f3aac021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0a1b75c6fab71e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3aa9ffb640 1 -- 172.21.15.122:0/2871492889 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac004b20 con 0x55a3092b0ca0 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3aa9ffb640 1 -- 172.21.15.122:0/2871492889 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac004cc0 con 0x55a3092b0ca0 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3aa9ffb640 1 -- 172.21.15.122:0/2871492889 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac005440 con 0x55a3092b0ca0 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2871492889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 msgr2=0x55a3092aca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009770 tx=0x7f3aac021000 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2871492889 shutdown_connections 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092b0ca0 0x55a3092aca90 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092a80c0 0x55a3092c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2871492889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092bc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2871492889 >> 172.21.15.122:0/2871492889 conn(0x55a3092a1ed0 msgr2=0x55a3092ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2871492889 shutdown_connections 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.899+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2871492889 wait complete. 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 -- start start 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 Processor -- start 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b04e0 con 0x55a3092a80c0 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41afe10 con 0x55a3092a77a0 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41ad070 con 0x55a3092b0ca0 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3abcfcd740 10 client.0 _ll_register_callbacks cb 0x7f3a9c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57312/0 (socket says 172.21.15.122:57312) 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 -- 172.21.15.122:0/2758130360 learned_addr learned my addr 172.21.15.122:0/2758130360 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:14.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab88ab640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 -- 172.21.15.122:0/2758130360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 msgr2=0x55a3092aa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092aa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 -- 172.21.15.122:0/2758130360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 msgr2=0x55a3092a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.900+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3ab37fe640 1 -- 172.21.15.122:0/2758130360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3092b7e60 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3ab37fe640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4018720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e654167b9be97e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40054c0 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3aa40035f0 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac004cc0 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3aa4039050 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 10 client.19107 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a3092b9190 con 0x55a3092a80c0 2024-04-02T13:47:14.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac004cc0 con 0x55a3092a80c0 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f3aa4034030 con 0x55a3092a80c0 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 10 client.19107 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 10 client.19107 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 20 client.19107 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a309327580 0x55a309329970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 4 client.19107 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.901+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3091906b0 con 0x55a309327580 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a309327580 0x55a309329970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3ab3fff640 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a309327580 0x55a309329970 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d11f50f3797d67e2 server_cookie=98fb22adbb5978c3 in_seq=0 out_seq=0 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3a937fe640 10 client.19107 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3a937fe640 1 -- 172.21.15.122:0/2758130360 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3091906b0 con 0x55a309327580 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3a937fe640 10 client.19107 handle_command_reply: tid=0 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 client.19107 shutdown 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 msgr2=0x55a3092aa260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4005fd0 tx=0x7f3aa4018720 comp rx=0 tx=0).stop 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 shutdown_connections 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55a309327580 0x55a309329970 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a3092b0ca0 0x55a3092aa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a3092a80c0 0x55a3092aa260 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3a937fe640 0 client.19107 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 --2- 172.21.15.122:0/2758130360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a3092a77a0 0x55a3092a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 >> 172.21.15.122:0/2758130360 conn(0x55a3092a1ed0 msgr2=0x55a3092a3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 shutdown_connections 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 1 -- 172.21.15.122:0/2758130360 wait complete. 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.902+0000 7f3abcfcd740 20 client.19107 trim_cache size 0 max 16384 2024-04-02T13:47:14.911 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:14.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 msgr2=0x7f3aa00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 0x7f3aa00a6380 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005fd0 tx=0x7f3aac009e90 comp rx=0 tx=0).stop 2024-04-02T13:47:14.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 msgr2=0x7f3ab41991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:14.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009230 tx=0x7f3a9c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 shutdown_connections 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3aa00a3ec0 0x7f3aa00a6380 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ab4106980 0x7f3ab41a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3ab4101330 0x7f3ab4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 --2- 172.21.15.122:0/3703968915 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ab4100970 0x7f3ab41991d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 >> 172.21.15.122:0/3703968915 conn(0x7f3ab4077b00 msgr2=0x7f3ab4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 shutdown_connections 2024-04-02T13:47:14.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:14.903+0000 7f3aba30e640 1 -- 172.21.15.122:0/3703968915 wait complete. 2024-04-02T13:47:14.916 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:47:14.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 472827 3.94 MB/sec execute 515 sec latency 14.792 ms 2024-04-02T13:47:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 -- 172.21.15.122:0/3466551219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c106980 msgr2=0x7f3e2c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 --2- 172.21.15.122:0/3466551219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c106980 0x7f3e2c106d60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009930 tx=0x7f3e2802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 -- 172.21.15.122:0/3466551219 shutdown_connections 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 --2- 172.21.15.122:0/3466551219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c101330 0x7f3e2c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 --2- 172.21.15.122:0/3466551219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c100970 0x7f3e2c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 --2- 172.21.15.122:0/3466551219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c106980 0x7f3e2c106d60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 -- 172.21.15.122:0/3466551219 >> 172.21.15.122:0/3466551219 conn(0x7f3e2c077b00 msgr2=0x7f3e2c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 -- 172.21.15.122:0/3466551219 shutdown_connections 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.019+0000 7f3e34006640 1 -- 172.21.15.122:0/3466551219 wait complete. 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 Processor -- start 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 -- start start 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c101330 0x7f3e2c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c106980 0x7f3e2c19bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c110d60 con 0x7f3e2c100970 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c110be0 con 0x7f3e2c101330 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e34006640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c110ee0 con 0x7f3e2c106980 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57322/0 (socket says 172.21.15.122:57322) 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 -- 172.21.15.122:0/867584172 learned_addr learned my addr 172.21.15.122:0/867584172 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e315a1640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c101330 0x7f3e2c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e325a3640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c106980 0x7f3e2c19bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 -- 172.21.15.122:0/867584172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c106980 msgr2=0x7f3e2c19bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c106980 0x7f3e2c19bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 -- 172.21.15.122:0/867584172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c101330 msgr2=0x7f3e2c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c101330 0x7f3e2c1977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.020+0000 7f3e31da2640 1 -- 172.21.15.122:0/867584172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c19c290 con 0x7f3e2c100970 2024-04-02T13:47:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e31da2640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009230 tx=0x7f3e2802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e22ffd640 1 -- 172.21.15.122:0/867584172 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28041070 con 0x7f3e2c100970 2024-04-02T13:47:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e22ffd640 1 -- 172.21.15.122:0/867584172 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e28004450 con 0x7f3e2c100970 2024-04-02T13:47:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c19c520 con 0x7f3e2c100970 2024-04-02T13:47:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e22ffd640 1 -- 172.21.15.122:0/867584172 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e28034440 con 0x7f3e2c100970 2024-04-02T13:47:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.021+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c1a3e10 con 0x7f3e2c100970 2024-04-02T13:47:15.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.023+0000 7f3e22ffd640 1 -- 172.21.15.122:0/867584172 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3e28044070 con 0x7f3e2c100970 2024-04-02T13:47:15.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.023+0000 7f3e36bef740 1 Processor -- start 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.023+0000 7f3e36bef740 1 -- start start 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860214fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x55860215b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5586020b37b0 con 0x5586021500c0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558602052570 con 0x55860214f7a0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e36bef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586020af760 con 0x558602158ca0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e325a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860214fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57332/0 (socket says 172.21.15.122:57332) 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 -- 172.21.15.122:0/3662555953 learned_addr learned my addr 172.21.15.122:0/3662555953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e22ffd640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 0x7f3e180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e325a3640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 0x7f3e180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e315a1640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x55860215b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e22ffd640 1 -- 172.21.15.122:0/867584172 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3e280f25a0 con 0x7f3e2c100970 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 -- 172.21.15.122:0/3662555953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 msgr2=0x55860215b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x55860215b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 -- 172.21.15.122:0/3662555953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 msgr2=0x55860214fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860214fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 -- 172.21.15.122:0/3662555953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558601dc73f0 con 0x5586021500c0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e325a3640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860214fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e315a1640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x55860215b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e31da2640 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f3e280c68f0 tx=0x7f3e28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921cc05ff2c31d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e17fff640 1 -- 172.21.15.122:0/3662555953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2804b070 con 0x5586021500c0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.024+0000 7f3e325a3640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 0x7f3e180a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005fd0 tx=0x7f3e24004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e17fff640 1 -- 172.21.15.122:0/3662555953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280be260 con 0x5586021500c0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e17fff640 1 -- 172.21.15.122:0/3662555953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280c80e0 con 0x5586021500c0 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 -- 172.21.15.122:0/3662555953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 msgr2=0x558602158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f3e280c68f0 tx=0x7f3e28000bc0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 -- 172.21.15.122:0/3662555953 shutdown_connections 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x55860215b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586021500c0 0x558602158760 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 --2- 172.21.15.122:0/3662555953 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860214fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 -- 172.21.15.122:0/3662555953 >> 172.21.15.122:0/3662555953 conn(0x558602149ed0 msgr2=0x55860214a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 -- 172.21.15.122:0/3662555953 shutdown_connections 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.025+0000 7f3e36bef740 1 -- 172.21.15.122:0/3662555953 wait complete. 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 -- start start 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 Processor -- start 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55860214f7a0 0x5586021586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602155a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5586020b37b0 con 0x558602158ca0 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55860206ee80 con 0x5586021500c0 2024-04-02T13:47:15.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55860215b850 con 0x55860214f7a0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e36bef740 10 client.0 _ll_register_callbacks cb 0x5586020aaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57340/0 (socket says 172.21.15.122:57340) 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 -- 172.21.15.122:0/1411081459 learned_addr learned my addr 172.21.15.122:0/1411081459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55860214f7a0 0x5586021586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e315a1640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 -- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55860214f7a0 msgr2=0x5586021586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55860214f7a0 0x5586021586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 -- 172.21.15.122:0/1411081459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 msgr2=0x558602155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e325a3640 1 -- 172.21.15.122:0/1411081459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558601e917d0 con 0x558602158ca0 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.026+0000 7f3e315a1640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e325a3640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3e24012c80 tx=0x7f3e24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c615c68d66e8e76e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24025070 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3e24014230 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e280be260 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3e2401a070 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 10 client.19119 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558602167790 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280be260 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f3e2400f040 con 0x558602158ca0 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 10 client.19119 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 10 client.19119 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 20 client.19119 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558602168ce0 0x55860216b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 4 client.19119 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:15.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.027+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586020c0cf0 con 0x558602168ce0 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.028+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558602168ce0 0x55860216b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.028+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558602168ce0 0x55860216b120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=669e25e5171cd4b3 server_cookie=2f706ec33f2e8153 in_seq=0 out_seq=0 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.028+0000 7f3e157fa640 10 client.19119 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e157fa640 1 -- 172.21.15.122:0/1411081459 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586020c0cf0 con 0x558602168ce0 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e157fa640 10 client.19119 handle_command_reply: tid=0 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 client.19119 shutdown 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 msgr2=0x558602156060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f3e24012c80 tx=0x7f3e24004a40 comp rx=0 tx=0).stop 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 shutdown_connections 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558602168ce0 0x55860216b120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602156060 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e157fa640 0 client.19119 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1411081459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55860214f7a0 0x5586021586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 >> 172.21.15.122:0/1411081459 conn(0x558602149ed0 msgr2=0x55860214bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 shutdown_connections 2024-04-02T13:47:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 1 -- 172.21.15.122:0/1411081459 wait complete. 2024-04-02T13:47:15.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.029+0000 7f3e36bef740 20 client.19119 trim_cache size 0 max 16384 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 Processor -- start 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 -- start start 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602158180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x5586021586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ae140 con 0x55860214f7a0 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1afc80 con 0x5586021500c0 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e36bef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ae810 con 0x558602158ca0 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57356/0 (socket says 172.21.15.122:57356) 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 -- 172.21.15.122:0/1488433717 learned_addr learned my addr 172.21.15.122:0/1488433717 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e315a1640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e325a3640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x5586021586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 -- 172.21.15.122:0/1488433717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 msgr2=0x5586021586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x5586021586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 -- 172.21.15.122:0/1488433717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 msgr2=0x558602158180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602158180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e31da2640 1 -- 172.21.15.122:0/1488433717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586021457a0 con 0x55860214f7a0 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.043+0000 7f3e315a1640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e31da2640 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3e280be1f0 tx=0x7f3e28003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b9d6f16cb829cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e177fe640 1 -- 172.21.15.122:0/1488433717 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2804b040 con 0x55860214f7a0 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e177fe640 1 -- 172.21.15.122:0/1488433717 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280c80e0 con 0x55860214f7a0 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e177fe640 1 -- 172.21.15.122:0/1488433717 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e280036f0 con 0x55860214f7a0 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e36bef740 1 -- 172.21.15.122:0/1488433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 msgr2=0x5586021645c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.044+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f3e280be1f0 tx=0x7f3e28003a90 comp rx=0 tx=0).stop 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- 172.21.15.122:0/1488433717 shutdown_connections 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558602158ca0 0x5586021586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586021500c0 0x558602158180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- 172.21.15.122:0/1488433717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55860214f7a0 0x5586021645c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- 172.21.15.122:0/1488433717 >> 172.21.15.122:0/1488433717 conn(0x558602149ed0 msgr2=0x558602155a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- 172.21.15.122:0/1488433717 shutdown_connections 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- 172.21.15.122:0/1488433717 wait complete. 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- start start 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 Processor -- start 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860215a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586021500c0 0x558602152d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1af5b0 con 0x558602158ca0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1aeee0 con 0x55860214f7a0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c1ac140 con 0x5586021500c0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e36bef740 10 client.0 _ll_register_callbacks cb 0x7f3e240145e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e325a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e325a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57368/0 (socket says 172.21.15.122:57368) 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e325a3640 1 -- 172.21.15.122:0/2170029328 learned_addr learned my addr 172.21.15.122:0/2170029328 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e315a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586021500c0 0x558602152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.045+0000 7f3e31da2640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860215a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 -- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586021500c0 msgr2=0x558602152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586021500c0 0x558602152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 -- 172.21.15.122:0/2170029328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 msgr2=0x55860215a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860215a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 -- 172.21.15.122:0/2170029328 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558602153a50 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e325a3640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d00 tx=0x7f3e24009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e6d5133dc4f878 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24025040 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f3e24021a60 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e280c80e0 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3e24043050 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 10 client.19125 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5586021d1280 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e280c80e0 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f3e24035c30 con 0x558602158ca0 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 10 client.19125 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 10 client.19125 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 20 client.19125 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5586021d18d0 0x5586021d3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 4 client.19125 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.046+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586020bb040 con 0x5586021d18d0 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e31da2640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5586021d18d0 0x5586021d3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e157fa640 10 client.19125 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e31da2640 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5586021d18d0 0x5586021d3cc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=63d3fcf955bc399e server_cookie=12adfa046d3ae79d in_seq=0 out_seq=0 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e157fa640 1 -- 172.21.15.122:0/2170029328 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586020bb040 con 0x5586021d18d0 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e157fa640 10 client.19125 handle_command_reply: tid=0 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 client.19125 shutdown 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 msgr2=0x558602153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d00 tx=0x7f3e24009e60 comp rx=0 tx=0).stop 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 shutdown_connections 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5586021d18d0 0x5586021d3cc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558602158ca0 0x558602153270 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e157fa640 0 client.19125 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586021500c0 0x558602152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 --2- 172.21.15.122:0/2170029328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55860214f7a0 0x55860215a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 >> 172.21.15.122:0/2170029328 conn(0x558602149ed0 msgr2=0x558602154340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 shutdown_connections 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.047+0000 7f3e36bef740 1 -- 172.21.15.122:0/2170029328 wait complete. 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e36bef740 20 client.19125 trim_cache size 0 max 16384 2024-04-02T13:47:15.058 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 msgr2=0x7f3e180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 0x7f3e180a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e24005fd0 tx=0x7f3e24004ab0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 msgr2=0x7f3e2c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009230 tx=0x7f3e2802cc70 comp rx=0 tx=0).stop 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 shutdown_connections 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e180a3d20 0x7f3e180a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e2c106980 0x7f3e2c19bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e2c101330 0x7f3e2c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 --2- 172.21.15.122:0/867584172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e2c100970 0x7f3e2c1972a0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 >> 172.21.15.122:0/867584172 conn(0x7f3e2c077b00 msgr2=0x7f3e2c1031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 shutdown_connections 2024-04-02T13:47:15.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.048+0000 7f3e34006640 1 -- 172.21.15.122:0/867584172 wait complete. 2024-04-02T13:47:15.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:47:15.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 -- 172.21.15.122:0/4040529624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 msgr2=0x7f0030106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 --2- 172.21.15.122:0/4040529624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f0030106d60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0028009930 tx=0x7f002802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:15.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 -- 172.21.15.122:0/4040529624 shutdown_connections 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 --2- 172.21.15.122:0/4040529624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030101330 0x7f003010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 --2- 172.21.15.122:0/4040529624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030100970 0x7f0030100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 --2- 172.21.15.122:0/4040529624 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f0030106d60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 -- 172.21.15.122:0/4040529624 >> 172.21.15.122:0/4040529624 conn(0x7f0030077b00 msgr2=0x7f0030077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 -- 172.21.15.122:0/4040529624 shutdown_connections 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 -- 172.21.15.122:0/4040529624 wait complete. 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.164+0000 7f0037d0e640 1 Processor -- start 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 -- start start 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030100970 0x7f0030199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030101330 0x7f0030199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0030110d90 con 0x7f0030106980 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0030110c10 con 0x7f0030101330 2024-04-02T13:47:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0030110f10 con 0x7f0030100970 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57382/0 (socket says 172.21.15.122:57382) 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 -- 172.21.15.122:0/245415500 learned_addr learned my addr 172.21.15.122:0/245415500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00352a9640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030101330 0x7f0030199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0035aaa640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030100970 0x7f0030199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 -- 172.21.15.122:0/245415500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030100970 msgr2=0x7f0030199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030100970 0x7f0030199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 -- 172.21.15.122:0/245415500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030101330 msgr2=0x7f0030199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030101330 0x7f0030199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 -- 172.21.15.122:0/245415500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00301a47d0 con 0x7f0030106980 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f00362ab640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f002000eb10 tx=0x7f002000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f002effd640 1 -- 172.21.15.122:0/245415500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020004b20 con 0x7f0030106980 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f002effd640 1 -- 172.21.15.122:0/245415500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0020004cc0 con 0x7f0030106980 2024-04-02T13:47:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f002effd640 1 -- 172.21.15.122:0/245415500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0020010760 con 0x7f0030106980 2024-04-02T13:47:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.165+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00301a49a0 con 0x7f0030106980 2024-04-02T13:47:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.166+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00301a4f10 con 0x7f0030106980 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.168+0000 7f002effd640 1 -- 172.21.15.122:0/245415500 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0020005020 con 0x7f0030106980 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.168+0000 7f003a9d2740 1 Processor -- start 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 -- start start 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd583b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd5864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55effd4da7b0 con 0x55effd57b4d0 2024-04-02T13:47:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55effd479570 con 0x55effd5840b0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f003a9d2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55effd4d6760 con 0x55effd57abb0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f002effd640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 0x7f001c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f0035aaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd583b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00362ab640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 0x7f001c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53344/0 (socket says 172.21.15.122:53344) 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f0035aaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd583b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57392/0 (socket says 172.21.15.122:57392) 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 -- 172.21.15.122:0/1341798321 learned_addr learned my addr 172.21.15.122:0/1341798321 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd5864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f002effd640 1 -- 172.21.15.122:0/245415500 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f00200d11f0 con 0x7f0030106980 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 -- 172.21.15.122:0/1341798321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 msgr2=0x55effd5864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd5864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 -- 172.21.15.122:0/1341798321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 msgr2=0x55effd583b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd583b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.169+0000 7f00352a9640 1 -- 172.21.15.122:0/1341798321 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effd1ee3f0 con 0x55effd57abb0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f00362ab640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 0x7f001c0a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00200991b0 tx=0x7f00200bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f00352a9640 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f002400b6c0 tx=0x7f002400b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4447f944918f697 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f001bfff640 1 -- 172.21.15.122:0/1341798321 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024012d00 con 0x55effd57abb0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f001bfff640 1 -- 172.21.15.122:0/1341798321 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024010770 con 0x55effd57abb0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f001bfff640 1 -- 172.21.15.122:0/1341798321 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003850 con 0x55effd57abb0 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f003a9d2740 1 -- 172.21.15.122:0/1341798321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 msgr2=0x55effd57af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.170+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f002400b6c0 tx=0x7f002400b650 comp rx=0 tx=0).stop 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- 172.21.15.122:0/1341798321 shutdown_connections 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd5864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd583b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1341798321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd57af90 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- 172.21.15.122:0/1341798321 >> 172.21.15.122:0/1341798321 conn(0x55effd5752e0 msgr2=0x55effd5756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- 172.21.15.122:0/1341798321 shutdown_connections 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- 172.21.15.122:0/1341798321 wait complete. 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- start start 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 Processor -- start 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 0x55effd5814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd5840b0 0x55effd581a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55effd586ea0 con 0x55effd57abb0 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55effd495e80 con 0x55effd57b4d0 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55effd587020 con 0x55effd5840b0 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.171+0000 7f003a9d2740 10 client.0 _ll_register_callbacks cb 0x55effd4d1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f0035aaa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 0x55effd5814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57406/0 (socket says 172.21.15.122:57406) 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 -- 172.21.15.122:0/1942086070 learned_addr learned my addr 172.21.15.122:0/1942086070 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd5840b0 0x55effd581a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 -- 172.21.15.122:0/1942086070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd5840b0 msgr2=0x55effd581a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd5840b0 0x55effd581a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 -- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 msgr2=0x55effd5814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 0x55effd5814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 -- 172.21.15.122:0/1942086070 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effd2b87d0 con 0x55effd57abb0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f0035aaa640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 0x55effd5814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00352a9640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0024005bc0 tx=0x7f0024004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd97969be022f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00197fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024005040 con 0x55effd57abb0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f00240051e0 con 0x55effd57abb0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0024010770 con 0x55effd57abb0 2024-04-02T13:47:15.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0024034050 con 0x55effd57abb0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f003a9d2740 10 client.19137 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55effd592fd0 con 0x55effd57abb0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.172+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0024010770 con 0x55effd57abb0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f00240035f0 con 0x55effd57abb0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 10 client.19137 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 10 client.19137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 20 client.19137 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd594320 0x55effd596710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 4 client.19137 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55effd4e7cf0 con 0x55effd594320 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f0035aaa640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd594320 0x55effd596710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f0035aaa640 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd594320 0x55effd596710 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89015b20cc0850f8 server_cookie=bcf22aee89fe15a in_seq=0 out_seq=0 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.173+0000 7f00197fa640 10 client.19137 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f00197fa640 1 -- 172.21.15.122:0/1942086070 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55effd4e7cf0 con 0x55effd594320 2024-04-02T13:47:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f00197fa640 10 client.19137 handle_command_reply: tid=0 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 client.19137 shutdown 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 msgr2=0x55effd580f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0024005bc0 tx=0x7f0024004590 comp rx=0 tx=0).stop 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 shutdown_connections 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd5840b0 0x55effd581a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd594320 0x55effd596710 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57b4d0 0x55effd5814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f00197fa640 0 client.19137 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 --2- 172.21.15.122:0/1942086070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57abb0 0x55effd580f80 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 >> 172.21.15.122:0/1942086070 conn(0x55effd5752e0 msgr2=0x55effd577030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 shutdown_connections 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 1 -- 172.21.15.122:0/1942086070 wait complete. 2024-04-02T13:47:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.174+0000 7f003a9d2740 20 client.19137 trim_cache size 0 max 16384 2024-04-02T13:47:15.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.187+0000 7f003a9d2740 1 Processor -- start 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 -- start start 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd58fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd580730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ad0c0 con 0x55effd57b4d0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00301aec00 con 0x55effd5840b0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f003a9d2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ad790 con 0x55effd57abb0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f00352a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd58fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57408/0 (socket says 172.21.15.122:57408) 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 -- 172.21.15.122:0/2928305693 learned_addr learned my addr 172.21.15.122:0/2928305693 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f00352a9640 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd580730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 -- 172.21.15.122:0/2928305693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 msgr2=0x55effd58fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd58fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 -- 172.21.15.122:0/2928305693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 msgr2=0x55effd580730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd580730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 -- 172.21.15.122:0/2928305693 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55effd581ee0 con 0x55effd57b4d0 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f00352a9640 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd580730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f0035aaa640 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0028005ab0 tx=0x7f002802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61837e49f9a3ca03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f001b7fe640 1 -- 172.21.15.122:0/2928305693 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002804d070 con 0x55effd57b4d0 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f001b7fe640 1 -- 172.21.15.122:0/2928305693 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00280493b0 con 0x55effd57b4d0 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.188+0000 7f001b7fe640 1 -- 172.21.15.122:0/2928305693 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028004b80 con 0x55effd57b4d0 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- 172.21.15.122:0/2928305693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 msgr2=0x55effd570170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0028005ab0 tx=0x7f002802e040 comp rx=0 tx=0).stop 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- 172.21.15.122:0/2928305693 shutdown_connections 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd5840b0 0x55effd580730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd57b4d0 0x55effd570170 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 --2- 172.21.15.122:0/2928305693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57abb0 0x55effd58fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- 172.21.15.122:0/2928305693 >> 172.21.15.122:0/2928305693 conn(0x55effd5752e0 msgr2=0x55effd5817d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- 172.21.15.122:0/2928305693 shutdown_connections 2024-04-02T13:47:15.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- 172.21.15.122:0/2928305693 wait complete. 2024-04-02T13:47:15.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.189+0000 7f003a9d2740 1 -- start start 2024-04-02T13:47:15.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 Processor -- start 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57abb0 0x55effd58c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 0x55effd57d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ae530 con 0x55effd5840b0 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ade60 con 0x55effd57abb0 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00301ab0c0 con 0x55effd57b4d0 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f003a9d2740 10 client.0 _ll_register_callbacks cb 0x7f0028009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57420/0 (socket says 172.21.15.122:57420) 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 -- 172.21.15.122:0/286913293 learned_addr learned my addr 172.21.15.122:0/286913293 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f0035aaa640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 0x55effd57d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57abb0 0x55effd58c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 -- 172.21.15.122:0/286913293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 msgr2=0x55effd57d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 0x55effd57d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 -- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57abb0 msgr2=0x55effd58c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57abb0 0x55effd58c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f00352a9640 1 -- 172.21.15.122:0/286913293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55effd57e430 con 0x55effd5840b0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.190+0000 7f0035aaa640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 0x55effd57d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00352a9640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f00240125f0 tx=0x7f0024012540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b728c2c2d003d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024003c30 con 0x55effd5840b0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f002402a040 con 0x55effd5840b0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00280493b0 con 0x55effd5840b0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0024003dd0 con 0x55effd5840b0 2024-04-02T13:47:15.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 10 client.19143 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55effd5fa5e0 con 0x55effd5840b0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00280493b0 con 0x55effd5840b0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f0024004660 con 0x55effd5840b0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 10 client.19143 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 10 client.19143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 20 client.19143 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd5fab90 0x55effd5fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 4 client.19143 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55effd4e2040 con 0x55effd5fab90 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.191+0000 7f0035aaa640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd5fab90 0x55effd5fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f00197fa640 10 client.19143 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f0035aaa640 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd5fab90 0x55effd5fcfb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a5b6004208dd48e server_cookie=12431cdc9f2b819f in_seq=0 out_seq=0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f00197fa640 1 -- 172.21.15.122:0/286913293 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55effd4e2040 con 0x55effd5fab90 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f00197fa640 10 client.19143 handle_command_reply: tid=0 2024-04-02T13:47:15.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 client.19143 shutdown 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 msgr2=0x55effd57dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f00240125f0 tx=0x7f0024012540 comp rx=0 tx=0).stop 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 shutdown_connections 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55effd5840b0 0x55effd57dd70 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55effd57b4d0 0x55effd57d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55effd5fab90 0x55effd5fcfb0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 --2- 172.21.15.122:0/286913293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55effd57abb0 0x55effd58c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f00197fa640 0 client.19143 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 >> 172.21.15.122:0/286913293 conn(0x55effd5752e0 msgr2=0x55effd576550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 shutdown_connections 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 1 -- 172.21.15.122:0/286913293 wait complete. 2024-04-02T13:47:15.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.192+0000 7f003a9d2740 20 client.19143 trim_cache size 0 max 16384 2024-04-02T13:47:15.202 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:15.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 msgr2=0x7f001c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 0x7f001c0a6260 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00200991b0 tx=0x7f00200bd040 comp rx=0 tx=0).stop 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 msgr2=0x7f00301a4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f002000eb10 tx=0x7f002000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 shutdown_connections 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f001c0a3da0 0x7f001c0a6260 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0030106980 0x7f00301a4290 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0030101330 0x7f0030199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 --2- 172.21.15.122:0/245415500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0030100970 0x7f0030199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 >> 172.21.15.122:0/245415500 conn(0x7f0030077b00 msgr2=0x7f0030103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 shutdown_connections 2024-04-02T13:47:15.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.193+0000 7f0037d0e640 1 -- 172.21.15.122:0/245415500 wait complete. 2024-04-02T13:47:15.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:47:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- 172.21.15.122:0/1071233283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c102460 msgr2=0x7f476c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 --2- 172.21.15.122:0/1071233283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c102460 0x7f476c10e970 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f4764009cd0 tx=0x7f476402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- 172.21.15.122:0/1071233283 shutdown_connections 2024-04-02T13:47:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 --2- 172.21.15.122:0/1071233283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c102460 0x7f476c10e970 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 --2- 172.21.15.122:0/1071233283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c101aa0 0x7f476c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 --2- 172.21.15.122:0/1071233283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 0x7f476c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- 172.21.15.122:0/1071233283 >> 172.21.15.122:0/1071233283 conn(0x7f476c0fd530 msgr2=0x7f476c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- 172.21.15.122:0/1071233283 shutdown_connections 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- 172.21.15.122:0/1071233283 wait complete. 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 Processor -- start 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.324+0000 7f4774655640 1 -- start start 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c102460 0x7f476c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 0x7f476c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f476c111ec0 con 0x7f476c101aa0 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f476c111d40 con 0x7f476c102460 2024-04-02T13:47:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f476c112040 con 0x7f476c107ab0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57438/0 (socket says 172.21.15.122:57438) 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 -- 172.21.15.122:0/840106330 learned_addr learned my addr 172.21.15.122:0/840106330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4771bf0640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c102460 0x7f476c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4772bf2640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 0x7f476c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 -- 172.21.15.122:0/840106330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 msgr2=0x7f476c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 0x7f476c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 -- 172.21.15.122:0/840106330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c102460 msgr2=0x7f476c074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c102460 0x7f476c074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 -- 172.21.15.122:0/840106330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a68a0 con 0x7f476c101aa0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47723f1640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f476800d770 tx=0x7f476800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47637fe640 1 -- 172.21.15.122:0/840106330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768017070 con 0x7f476c101aa0 2024-04-02T13:47:15.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47637fe640 1 -- 172.21.15.122:0/840106330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768005500 con 0x7f476c101aa0 2024-04-02T13:47:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a6b90 con 0x7f476c101aa0 2024-04-02T13:47:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f47637fe640 1 -- 172.21.15.122:0/840106330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768004e00 con 0x7f476c101aa0 2024-04-02T13:47:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.325+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a7090 con 0x7f476c101aa0 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.328+0000 7f47637fe640 1 -- 172.21.15.122:0/840106330 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4768004fa0 con 0x7f476c101aa0 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.328+0000 7f4777241740 1 Processor -- start 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 -- start start 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3370357b0 con 0x55b3370daca0 2024-04-02T13:47:15.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b336fd4570 con 0x55b3370d20c0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4777241740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b337031760 con 0x55b3370d17a0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f47723f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57444/0 (socket says 172.21.15.122:57444) 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 -- 172.21.15.122:0/765755914 learned_addr learned my addr 172.21.15.122:0/765755914 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4772bf2640 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f47637fe640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 0x7f47580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4772bf2640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 0x7f47580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 -- 172.21.15.122:0/765755914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 msgr2=0x55b3370d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 -- 172.21.15.122:0/765755914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 msgr2=0x55b3370da760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370da760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f4771bf0640 1 -- 172.21.15.122:0/765755914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b336d493f0 con 0x55b3370daca0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.329+0000 7f47637fe640 1 -- 172.21.15.122:0/840106330 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f47680d0560 con 0x7f476c101aa0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.330+0000 7f4771bf0640 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f475c00bc90 tx=0x7f475c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ae40c531863c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.330+0000 7f4760bfe640 1 -- 172.21.15.122:0/765755914 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c005220 con 0x55b3370daca0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.330+0000 7f4772bf2640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 0x7f47580a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4764005cc0 tx=0x7f476402c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.330+0000 7f4760bfe640 1 -- 172.21.15.122:0/765755914 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f475c00be40 con 0x55b3370daca0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.330+0000 7f4760bfe640 1 -- 172.21.15.122:0/765755914 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c011790 con 0x55b3370daca0 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- 172.21.15.122:0/765755914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 msgr2=0x55b3370dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f475c00bc90 tx=0x7f475c0107a0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- 172.21.15.122:0/765755914 shutdown_connections 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370dd090 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370da760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 --2- 172.21.15.122:0/765755914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- 172.21.15.122:0/765755914 >> 172.21.15.122:0/765755914 conn(0x55b3370cbed0 msgr2=0x55b3370cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- 172.21.15.122:0/765755914 shutdown_connections 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- 172.21.15.122:0/765755914 wait complete. 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 -- start start 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.331+0000 7f4777241740 1 Processor -- start 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370da700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370daca0 0x55b3370d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3370dda90 con 0x55b3370d20c0 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b336ff0e80 con 0x55b3370daca0 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b3370ddc10 con 0x55b3370d17a0 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57448/0 (socket says 172.21.15.122:57448) 2024-04-02T13:47:15.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 -- 172.21.15.122:0/2931566077 learned_addr learned my addr 172.21.15.122:0/2931566077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f47723f1640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370da700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 10 client.0 _ll_register_callbacks cb 0x55b33702caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4772bf2640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370daca0 0x55b3370d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 -- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 msgr2=0x55b3370da700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370da700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 -- 172.21.15.122:0/2931566077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370daca0 msgr2=0x55b3370d81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370daca0 0x55b3370d81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 -- 172.21.15.122:0/2931566077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b336e137d0 con 0x55b3370d20c0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4771bf0640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f475c011010 tx=0x7f475c0051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa29b6b5a8a5845 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c0048d0 con 0x55b3370d20c0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f475c004a70 con 0x55b3370d20c0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.332+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b337042cf0 con 0x55b3370d20c0 2024-04-02T13:47:15.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f475c004c70 con 0x55b3370d20c0 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 10 client.19155 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b3370e9b30 con 0x55b3370d20c0 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b337042cf0 con 0x55b3370d20c0 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f475c01b4c0 con 0x55b3370d20c0 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 10 client.19155 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 10 client.19155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 20 client.19155 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b3370eaf90 0x55b3370ed380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 4 client.19155 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33703d040 con 0x55b3370eaf90 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.333+0000 7f47723f1640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b3370eaf90 0x55b3370ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4755ffb640 10 client.19155 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:15.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f47723f1640 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b3370eaf90 0x55b3370ed380 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbd4e991ff14ac2f server_cookie=875a358b497bf18f in_seq=0 out_seq=0 2024-04-02T13:47:15.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4755ffb640 1 -- 172.21.15.122:0/2931566077 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b33703d040 con 0x55b3370eaf90 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4755ffb640 10 client.19155 handle_command_reply: tid=0 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4777241740 1 client.19155 shutdown 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 msgr2=0x55b3370d7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.334+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f475c011010 tx=0x7f475c0051e0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 shutdown_connections 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b3370eaf90 0x55b3370ed380 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370daca0 0x55b3370d81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4755ffb640 0 client.19155 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d20c0 0x55b3370d7a90 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 --2- 172.21.15.122:0/2931566077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d17a0 0x55b3370da700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 >> 172.21.15.122:0/2931566077 conn(0x55b3370cbed0 msgr2=0x55b3370cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 shutdown_connections 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 1 -- 172.21.15.122:0/2931566077 wait complete. 2024-04-02T13:47:15.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.335+0000 7f4777241740 20 client.19155 trim_cache size 0 max 16384 2024-04-02T13:47:15.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 Processor -- start 2024-04-02T13:47:15.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 -- start start 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d20c0 0x55b3370d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b13b0 con 0x55b3370daca0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b2ef0 con 0x55b3370d17a0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.348+0000 7f4777241740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b1a80 con 0x55b3370d20c0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f4772bf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59262/0 (socket says 172.21.15.122:59262) 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 -- 172.21.15.122:0/1366667061 learned_addr learned my addr 172.21.15.122:0/1366667061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f4771bf0640 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d20c0 0x55b3370d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 -- 172.21.15.122:0/1366667061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d20c0 msgr2=0x55b3370d6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d20c0 0x55b3370d6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 -- 172.21.15.122:0/1366667061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 msgr2=0x55b3370f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 -- 172.21.15.122:0/1366667061 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3370d84e0 con 0x55b3370d17a0 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f47723f1640 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4768005c20 tx=0x7f47680d2ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baed6cc5074d0ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f4757fff640 1 -- 172.21.15.122:0/1366667061 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768020040 con 0x55b3370d17a0 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f4757fff640 1 -- 172.21.15.122:0/1366667061 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47680046c0 con 0x55b3370d17a0 2024-04-02T13:47:15.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.349+0000 7f4757fff640 1 -- 172.21.15.122:0/1366667061 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476809a470 con 0x55b3370d17a0 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- 172.21.15.122:0/1366667061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 msgr2=0x55b3370e6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4768005c20 tx=0x7f47680d2ae0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- 172.21.15.122:0/1366667061 shutdown_connections 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370daca0 0x55b3370f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370d20c0 0x55b3370d6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 --2- 172.21.15.122:0/1366667061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d17a0 0x55b3370e6950 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- 172.21.15.122:0/1366667061 >> 172.21.15.122:0/1366667061 conn(0x55b3370cbed0 msgr2=0x55b3370d6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- 172.21.15.122:0/1366667061 shutdown_connections 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- 172.21.15.122:0/1366667061 wait complete. 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 -- start start 2024-04-02T13:47:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.350+0000 7f4777241740 1 Processor -- start 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370db9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 0x55b3370e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b2820 con 0x55b3370d17a0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b2150 con 0x55b3370d20c0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1af3b0 con 0x55b3370daca0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 10 client.0 _ll_register_callbacks cb 0x7f4768098410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57468/0 (socket says 172.21.15.122:57468) 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 -- 172.21.15.122:0/3111775716 learned_addr learned my addr 172.21.15.122:0/3111775716 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4771bf0640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370db9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4772bf2640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 0x55b3370e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 -- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 msgr2=0x55b3370e1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 0x55b3370e1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 -- 172.21.15.122:0/3111775716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 msgr2=0x55b3370db9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370db9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 -- 172.21.15.122:0/3111775716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3370e23a0 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4772bf2640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 0x55b3370e1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f47723f1640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f476809be70 tx=0x7f47680d2a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251800e101640d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768020050 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7f47680a0160 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b336fba6b0 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f47680a0470 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 10 client.19161 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b3370e36a0 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.351+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b336fba6b0 con 0x55b3370d17a0 2024-04-02T13:47:15.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f4768096ce0 con 0x55b3370d17a0 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 10 client.19161 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 10 client.19161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 20 client.19161 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b337151700 0x55b337153af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 4 client.19161 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b337006e40 con 0x55b337151700 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4771bf0640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b337151700 0x55b337153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4755ffb640 10 client.19161 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4771bf0640 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b337151700 0x55b337153af0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4ff3a91ecc371a5 server_cookie=b69685afcdee7d5b in_seq=0 out_seq=0 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4755ffb640 1 -- 172.21.15.122:0/3111775716 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b337006e40 con 0x55b337151700 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4755ffb640 10 client.19161 handle_command_reply: tid=0 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.352+0000 7f4777241740 1 client.19161 shutdown 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 msgr2=0x55b3370db470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f476809be70 tx=0x7f47680d2a70 comp rx=0 tx=0).stop 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 shutdown_connections 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b337151700 0x55b337153af0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3370daca0 0x55b3370e1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3370d20c0 0x55b3370db9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4755ffb640 0 client.19161 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 --2- 172.21.15.122:0/3111775716 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3370d17a0 0x55b3370db470 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 >> 172.21.15.122:0/3111775716 conn(0x55b3370cbed0 msgr2=0x55b3370ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 shutdown_connections 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 1 -- 172.21.15.122:0/3111775716 wait complete. 2024-04-02T13:47:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4777241740 20 client.19161 trim_cache size 0 max 16384 2024-04-02T13:47:15.363 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 msgr2=0x7f47580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 0x7f47580a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4764005cc0 tx=0x7f476402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 msgr2=0x7f476c073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f476800d770 tx=0x7f476800dc30 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 shutdown_connections 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f47580a3d20 0x7f47580a61e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f476c107ab0 0x7f476c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f476c102460 0x7f476c074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 --2- 172.21.15.122:0/840106330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f476c101aa0 0x7f476c073e00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.353+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 >> 172.21.15.122:0/840106330 conn(0x7f476c0fd530 msgr2=0x7f476c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.354+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 shutdown_connections 2024-04-02T13:47:15.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.354+0000 7f4774655640 1 -- 172.21.15.122:0/840106330 wait complete. 2024-04-02T13:47:15.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:47:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.482+0000 7fbc96ed5640 1 -- 172.21.15.122:0/867649797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90106980 msgr2=0x7fbc90106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.482+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/867649797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90106980 0x7fbc90106d60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009930 tx=0x7fbc8402bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.482+0000 7fbc96ed5640 1 -- 172.21.15.122:0/867649797 shutdown_connections 2024-04-02T13:47:15.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.482+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/867649797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90101330 0x7fbc9010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.482+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/867649797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/867649797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90106980 0x7fbc90106d60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- 172.21.15.122:0/867649797 >> 172.21.15.122:0/867649797 conn(0x7fbc90077b00 msgr2=0x7fbc90077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- 172.21.15.122:0/867649797 shutdown_connections 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- 172.21.15.122:0/867649797 wait complete. 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 Processor -- start 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- start start 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 0x7fbc90197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90110d90 con 0x7fbc90101330 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90110c10 con 0x7fbc90106980 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc96ed5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90110f10 con 0x7fbc90100970 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.483+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57482/0 (socket says 172.21.15.122:57482) 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 -- 172.21.15.122:0/4039434453 learned_addr learned my addr 172.21.15.122:0/4039434453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc95472640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 0x7fbc90197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc94c71640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 -- 172.21.15.122:0/4039434453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 msgr2=0x7fbc90197000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90197000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 -- 172.21.15.122:0/4039434453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 msgr2=0x7fbc90197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 0x7fbc90197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 -- 172.21.15.122:0/4039434453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc901a6870 con 0x7fbc90101330 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc95472640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 0x7fbc90197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc94c71640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e9f0 tx=0x7fbc7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8dffb640 1 -- 172.21.15.122:0/4039434453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00cd80 con 0x7fbc90101330 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc901a6b60 con 0x7fbc90101330 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8dffb640 1 -- 172.21.15.122:0/4039434453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc7c005500 con 0x7fbc90101330 2024-04-02T13:47:15.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc8dffb640 1 -- 172.21.15.122:0/4039434453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c010690 con 0x7fbc90101330 2024-04-02T13:47:15.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.484+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901a70b0 con 0x7fbc90101330 2024-04-02T13:47:15.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.486+0000 7fbc8dffb640 1 -- 172.21.15.122:0/4039434453 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbc7c0046c0 con 0x7fbc90101330 2024-04-02T13:47:15.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 Processor -- start 2024-04-02T13:47:15.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 -- start start 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc8dffb640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0552507b0 con 0x55f0552ed0c0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f0551ef570 con 0x55f0552f5ca0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f05524c760 con 0x55f0552ec7a0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc94c71640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc95472640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc95472640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552ecb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53448/0 (socket says 172.21.15.122:53448) 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc95472640 1 -- 172.21.15.122:0/4035685812 learned_addr learned my addr 172.21.15.122:0/4035685812 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc8dffb640 1 -- 172.21.15.122:0/4039434453 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7fbc7c0d10e0 con 0x7fbc90101330 2024-04-02T13:47:15.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.487+0000 7fbc95472640 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 -- 172.21.15.122:0/4035685812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 msgr2=0x55f0552ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 -- 172.21.15.122:0/4035685812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 msgr2=0x55f0552f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 -- 172.21.15.122:0/4035685812 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f054f643f0 con 0x55f0552f5ca0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc94c71640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 0x7fbc800a61e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc8402c6f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc95472640 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f8090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc88011e80 tx=0x7fbc88009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6865bdae3d7a91a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc7affd640 1 -- 172.21.15.122:0/4035685812 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800fd10 con 0x55f0552f5ca0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc7affd640 1 -- 172.21.15.122:0/4035685812 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc88003970 con 0x55f0552f5ca0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.488+0000 7fbc7affd640 1 -- 172.21.15.122:0/4035685812 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc880043b0 con 0x55f0552f5ca0 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- 172.21.15.122:0/4035685812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 msgr2=0x55f0552f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f8090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbc88011e80 tx=0x7fbc88009500 comp rx=0 tx=0).stop 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- 172.21.15.122:0/4035685812 shutdown_connections 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f8090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/4035685812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- 172.21.15.122:0/4035685812 >> 172.21.15.122:0/4035685812 conn(0x55f0552e6ed0 msgr2=0x55f0552e72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- 172.21.15.122:0/4035685812 shutdown_connections 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- 172.21.15.122:0/4035685812 wait complete. 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 -- start start 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.489+0000 7fbc99b8b740 1 Processor -- start 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 0x55f0552f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552f5ca0 0x55f0552f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0552f8a90 con 0x55f0552ed0c0 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f05520be80 con 0x55f0552ec7a0 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0552f8c10 con 0x55f0552f5ca0 2024-04-02T13:47:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 10 client.0 _ll_register_callbacks cb 0x55f055247aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc94c71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 0x55f0552f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57500/0 (socket says 172.21.15.122:57500) 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc94c71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 0x55f0552f5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59302/0 (socket says 172.21.15.122:59302) 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3273678218 learned_addr learned my addr 172.21.15.122:0/3273678218 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc95472640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552f5ca0 0x55f0552f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3273678218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552f5ca0 msgr2=0x55f0552f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552f5ca0 0x55f0552f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 msgr2=0x55f0552f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 0x55f0552f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3273678218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f05502e7d0 con 0x55f0552ed0c0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ce450 tx=0x7fbc7c099560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3519b757826c52e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c09b200 con 0x55f0552ed0c0 2024-04-02T13:47:15.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7fbc7c09afa0 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.490+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc88003970 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.491+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc7c09a290 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.491+0000 7fbc99b8b740 10 client.19173 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.491+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f055304b80 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.491+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc88003970 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fbc7c0bd0d0 con 0x55f0552ed0c0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 10 client.19173 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 10 client.19173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 20 client.19173 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f055306030 0x55f055308440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 4 client.19173 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f05525dcf0 con 0x55f055306030 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc94c71640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f055306030 0x55f055308440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc6ffff640 10 client.19173 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.492+0000 7fbc94c71640 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f055306030 0x55f055308440 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a33bd22b4bd9760e server_cookie=2c67b98869ed2bce in_seq=0 out_seq=0 2024-04-02T13:47:15.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3273678218 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f05525dcf0 con 0x55f055306030 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc6ffff640 10 client.19173 handle_command_reply: tid=0 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 client.19173 shutdown 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 msgr2=0x55f0552f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c0ce450 tx=0x7fbc7c099560 comp rx=0 tx=0).stop 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 shutdown_connections 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f055306030 0x55f055308440 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552f5ca0 0x55f0552f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f0552f2ac0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3273678218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ec7a0 0x55f0552f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc6ffff640 0 client.19173 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 >> 172.21.15.122:0/3273678218 conn(0x55f0552e6ed0 msgr2=0x55f0552e8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 shutdown_connections 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3273678218 wait complete. 2024-04-02T13:47:15.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.493+0000 7fbc99b8b740 20 client.19173 trim_cache size 0 max 16384 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 Processor -- start 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 -- start start 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f055308420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901b1420 con 0x55f0552ed0c0 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901b2f60 con 0x55f0552f5ca0 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901b1af0 con 0x55f0552ec7a0 2024-04-02T13:47:15.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57514/0 (socket says 172.21.15.122:57514) 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc8ffff640 1 -- 172.21.15.122:0/206754823 learned_addr learned my addr 172.21.15.122:0/206754823 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.508+0000 7fbc94c71640 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f055308420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc95472640 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 -- 172.21.15.122:0/206754823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 msgr2=0x55f055308420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f055308420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 -- 172.21.15.122:0/206754823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 msgr2=0x55f0552f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 -- 172.21.15.122:0/206754823 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0552e1310 con 0x55f0552ed0c0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c09b6f0 tx=0x7fbc7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c286e35343e0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc7a7fc640 1 -- 172.21.15.122:0/206754823 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c09a470 con 0x55f0552ed0c0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc7a7fc640 1 -- 172.21.15.122:0/206754823 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc7c087040 con 0x55f0552ed0c0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.509+0000 7fbc7a7fc640 1 -- 172.21.15.122:0/206754823 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c09ab20 con 0x55f0552ed0c0 2024-04-02T13:47:15.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 -- 172.21.15.122:0/206754823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 msgr2=0x55f055301950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c09b6f0 tx=0x7fbc7c004a40 comp rx=0 tx=0).stop 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 -- 172.21.15.122:0/206754823 shutdown_connections 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552f5ca0 0x55f0552f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552ed0c0 0x55f055301950 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/206754823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f055308420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 -- 172.21.15.122:0/206754823 >> 172.21.15.122:0/206754823 conn(0x55f0552e6ed0 msgr2=0x55f0552f1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 -- 172.21.15.122:0/206754823 shutdown_connections 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.510+0000 7fbc99b8b740 1 -- 172.21.15.122:0/206754823 wait complete. 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 -- start start 2024-04-02T13:47:15.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 Processor -- start 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 0x55f0552f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 0x55f0552fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901b2890 con 0x55f0552f5ca0 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901b21c0 con 0x55f0552ed0c0 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901af420 con 0x55f0552ec7a0 2024-04-02T13:47:15.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc99b8b740 10 client.0 _ll_register_callbacks cb 0x7fbc7c004ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc95472640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 0x55f0552fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc94c71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc8ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 0x55f0552f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc95472640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 0x55f0552fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57524/0 (socket says 172.21.15.122:57524) 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.511+0000 7fbc95472640 1 -- 172.21.15.122:0/3968215137 learned_addr learned my addr 172.21.15.122:0/3968215137 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3968215137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 msgr2=0x55f0552f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3968215137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 msgr2=0x55f0552fce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 0x55f0552fce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 -- 172.21.15.122:0/3968215137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0552fd3a0 con 0x55f0552ed0c0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc8ffff640 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 0x55f0552f6db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55f0552d8820 tx=0x7fbc7c099560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80f3a403f35d8353 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c065070 con 0x55f0552ed0c0 2024-04-02T13:47:15.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9613+0+0 (secure 0 0 0) 0x7fbc7c0a5630 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc7c087040 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc7c09ad80 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc99b8b740 10 client.28175 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.512+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f0552fe640 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc7c087040 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fbc7c09cea0 con 0x55f0552ed0c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 10 client.28175 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 10 client.28175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 20 client.28175 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f05536c6c0 0x55f05536eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 4 client.28175 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f055258040 con 0x55f05536c6c0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc94c71640 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f05536c6c0 0x55f05536eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:15.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc94c71640 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f05536c6c0 0x55f05536eae0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf46b39fb887ccd server_cookie=276caaf580e5b734 in_seq=0 out_seq=0 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.513+0000 7fbc6ffff640 10 client.28175 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc6ffff640 1 -- 172.21.15.122:0/3968215137 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f055258040 con 0x55f05536c6c0 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc6ffff640 10 client.28175 handle_command_reply: tid=0 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 client.28175 shutdown 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 msgr2=0x55f0552f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 0x55f0552f6db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55f0552d8820 tx=0x7fbc7c099560 comp rx=0 tx=0).stop 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 shutdown_connections 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f05536c6c0 0x55f05536eae0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc6ffff640 0 client.28175 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0552f5ca0 0x55f0552fce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0552ed0c0 0x55f0552f6db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 --2- 172.21.15.122:0/3968215137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0552ec7a0 0x55f0552f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 >> 172.21.15.122:0/3968215137 conn(0x55f0552e6ed0 msgr2=0x55f0552e8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 shutdown_connections 2024-04-02T13:47:15.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 1 -- 172.21.15.122:0/3968215137 wait complete. 2024-04-02T13:47:15.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.514+0000 7fbc99b8b740 20 client.28175 trim_cache size 0 max 16384 2024-04-02T13:47:15.523 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:15.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 msgr2=0x7fbc800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 0x7fbc800a61e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc84006010 tx=0x7fbc8402c6f0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 msgr2=0x7fbc90197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:15.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00e9f0 tx=0x7fbc7c00edf0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 shutdown_connections 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc800a3d20 0x7fbc800a61e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbc90106980 0x7fbc90197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc90101330 0x7fbc90197540 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 --2- 172.21.15.122:0/4039434453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc90100970 0x7fbc90197000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 >> 172.21.15.122:0/4039434453 conn(0x7fbc90077b00 msgr2=0x7fbc901031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 shutdown_connections 2024-04-02T13:47:15.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:15.515+0000 7fbc96ed5640 1 -- 172.21.15.122:0/4039434453 wait complete. 2024-04-02T13:47:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:15 smithi122 ceph-mon[32114]: pgmap v376: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:15 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1549981082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:15.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 474198 3.94 MB/sec execute 516 sec latency 17.460 ms 2024-04-02T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:15 smithi187 ceph-mon[36064]: pgmap v376: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:15 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1549981082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:15 smithi188 ceph-mon[36117]: pgmap v376: 129 pgs: 129 active+clean; 403 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 134 op/s 2024-04-02T13:47:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:15 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1549981082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:16.529 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:47:16.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:47:16.530 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 -- 172.21.15.122:0/1753221063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 msgr2=0x7f8a7c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1753221063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c106d60 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009930 tx=0x7f8a7802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 -- 172.21.15.122:0/1753221063 shutdown_connections 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1753221063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c101330 0x7f8a7c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1753221063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c100970 0x7f8a7c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1753221063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c106d60 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 -- 172.21.15.122:0/1753221063 >> 172.21.15.122:0/1753221063 conn(0x7f8a7c077b00 msgr2=0x7f8a7c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 -- 172.21.15.122:0/1753221063 shutdown_connections 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.643+0000 7f8a82fff640 1 -- 172.21.15.122:0/1753221063 wait complete. 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 Processor -- start 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 -- start start 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c101330 0x7f8a7c103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c110d60 con 0x7f8a7c106980 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c110be0 con 0x7f8a7c100970 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a82fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c110ee0 con 0x7f8a7c101330 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a8159c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a8159c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57538/0 (socket says 172.21.15.122:57538) 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c101330 0x7f8a7c103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a8159c640 1 -- 172.21.15.122:0/1832133877 learned_addr learned my addr 172.21.15.122:0/1832133877 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.644+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59338/0 (socket says 172.21.15.122:59338) 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 -- 172.21.15.122:0/1832133877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c101330 msgr2=0x7f8a7c103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c101330 0x7f8a7c103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 -- 172.21.15.122:0/1832133877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 msgr2=0x7f8a7c102fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 -- 172.21.15.122:0/1832133877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7c1a6890 con 0x7f8a7c106980 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a8159c640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400eb10 tx=0x7f8a7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a71ffb640 1 -- 172.21.15.122:0/1832133877 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74004b20 con 0x7f8a7c106980 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7c1a6b80 con 0x7f8a7c106980 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a71ffb640 1 -- 172.21.15.122:0/1832133877 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a74004cc0 con 0x7f8a7c106980 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a71ffb640 1 -- 172.21.15.122:0/1832133877 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a74010760 con 0x7f8a7c106980 2024-04-02T13:47:16.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.645+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c1a70d0 con 0x7f8a7c106980 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.648+0000 7f8a85be3740 1 Processor -- start 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.648+0000 7f8a71ffb640 1 -- 172.21.15.122:0/1832133877 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8a74005020 con 0x7f8a7c106980 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.648+0000 7f8a85be3740 1 -- start start 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389ff280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56213895a770 con 0x5621389ff7c0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562138902960 con 0x5621389f6be0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a85be3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621389568c0 con 0x5621389f62c0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389ff280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57546/0 (socket says 172.21.15.122:57546) 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a73fff640 1 -- 172.21.15.122:0/2443799429 learned_addr learned my addr 172.21.15.122:0/2443799429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a71ffb640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 0x7f8a640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a73fff640 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a8159c640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 0x7f8a640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2443799429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 msgr2=0x5621389f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2443799429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 msgr2=0x562138a01bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2443799429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621387367e0 con 0x5621389f6be0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a71ffb640 1 -- 172.21.15.122:0/1832133877 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f8a740d10f0 con 0x7f8a7c106980 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.649+0000 7f8a73fff640 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389ff280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009230 tx=0x7f8a78005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a5434d9d3634216 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a8159c640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 0x7f8a640a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a7409c9a0 tx=0x7f8a7406e6e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a62ffd640 1 -- 172.21.15.122:0/2443799429 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78042070 con 0x5621389f6be0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a62ffd640 1 -- 172.21.15.122:0/2443799429 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a62ffd640 1 -- 172.21.15.122:0/2443799429 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78005190 con 0x5621389f6be0 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a85be3740 1 -- 172.21.15.122:0/2443799429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 msgr2=0x5621389ff280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.650+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389ff280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009230 tx=0x7f8a78005d50 comp rx=0 tx=0).stop 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- 172.21.15.122:0/2443799429 shutdown_connections 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389ff280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2443799429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- 172.21.15.122:0/2443799429 >> 172.21.15.122:0/2443799429 conn(0x5621389f09f0 msgr2=0x5621389f0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- 172.21.15.122:0/2443799429 shutdown_connections 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- 172.21.15.122:0/2443799429 wait complete. 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- start start 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 Processor -- start 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562138a02550 con 0x5621389ff7c0 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562138915d20 con 0x5621389f6be0 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562138a026d0 con 0x5621389f62c0 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a85be3740 10 client.0 _ll_register_callbacks cb 0x5621388e2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.651+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57558/0 (socket says 172.21.15.122:57558) 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a73fff640 1 -- 172.21.15.122:0/3474466810 learned_addr learned my addr 172.21.15.122:0/3474466810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3474466810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 msgr2=0x5621389fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3474466810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 msgr2=0x5621389fd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3474466810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562138796f70 con 0x5621389f6be0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389fcd20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802c170 tx=0x7f8a7802c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f32f0b885e83fa8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78042050 con 0x5621389f6be0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f8a7802cc90 con 0x5621389f6be0 2024-04-02T13:47:16.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8a78058050 con 0x5621389f6be0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a85be3740 10 client.28187 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.652+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56213862e170 con 0x5621389f6be0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f8a78003970 con 0x5621389f6be0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 10 client.28187 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 10 client.28187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 20 client.28187 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a0f9b0 0x562138a11da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 4 client.28187 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562138967cb0 con 0x562138a0f9b0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a0f9b0 0x562138a11da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a0f9b0 0x562138a11da0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82330771bfca7a6 server_cookie=71fa1bc16e81c797 in_seq=0 out_seq=0 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.653+0000 7f8a5bfff640 10 client.28187 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3474466810 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562138967cb0 con 0x562138a0f9b0 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a5bfff640 10 client.28187 handle_command_reply: tid=0 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 client.28187 shutdown 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 msgr2=0x5621389fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389fcd20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802c170 tx=0x7f8a7802c3b0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 shutdown_connections 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a0f9b0 0x562138a11da0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x5621389fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x5621389fcd20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a5bfff640 0 client.28187 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3474466810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x5621389fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 >> 172.21.15.122:0/3474466810 conn(0x5621389f09f0 msgr2=0x5621389f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 shutdown_connections 2024-04-02T13:47:16.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.654+0000 7f8a85be3740 1 -- 172.21.15.122:0/3474466810 wait complete. 2024-04-02T13:47:16.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.655+0000 7f8a85be3740 20 client.28187 trim_cache size 0 max 16384 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.674+0000 7f8a85be3740 1 Processor -- start 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.674+0000 7f8a85be3740 1 -- start start 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f62c0 0x562138a0b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389ff7c0 0x562138a17340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1b1290 con 0x5621389f6be0 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1b2dd0 con 0x5621389f62c0 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a85be3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1b1960 con 0x5621389ff7c0 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57566/0 (socket says 172.21.15.122:57566) 2024-04-02T13:47:16.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2112120254 learned_addr learned my addr 172.21.15.122:0/2112120254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a73fff640 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f62c0 0x562138a0b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2112120254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389ff7c0 msgr2=0x562138a17340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389ff7c0 0x562138a17340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2112120254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f62c0 msgr2=0x562138a0b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f62c0 0x562138a0b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 -- 172.21.15.122:0/2112120254 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621389f5c90 con 0x5621389f6be0 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802bc70 tx=0x7f8a78005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2b435185f581dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a627fc640 1 -- 172.21.15.122:0/2112120254 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78042070 con 0x5621389f6be0 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a627fc640 1 -- 172.21.15.122:0/2112120254 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.675+0000 7f8a627fc640 1 -- 172.21.15.122:0/2112120254 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7802ca90 con 0x5621389f6be0 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 -- 172.21.15.122:0/2112120254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 msgr2=0x5621389feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8a7802bc70 tx=0x7f8a78005e40 comp rx=0 tx=0).stop 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 -- 172.21.15.122:0/2112120254 shutdown_connections 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389ff7c0 0x562138a17340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389f6be0 0x5621389feea0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 --2- 172.21.15.122:0/2112120254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f62c0 0x562138a0b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 -- 172.21.15.122:0/2112120254 >> 172.21.15.122:0/2112120254 conn(0x5621389f09f0 msgr2=0x5621389efab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 -- 172.21.15.122:0/2112120254 shutdown_connections 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.676+0000 7f8a85be3740 1 -- 172.21.15.122:0/2112120254 wait complete. 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 -- start start 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 Processor -- start 2024-04-02T13:47:16.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x562138a07980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1b2700 con 0x5621389ff7c0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1b2030 con 0x5621389f6be0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c1af290 con 0x5621389f62c0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a85be3740 10 client.0 _ll_register_callbacks cb 0x7f8a6c00b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x562138a07980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a80d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59382/0 (socket says 172.21.15.122:59382) 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3804287579 learned_addr learned my addr 172.21.15.122:0/3804287579 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.677+0000 7f8a73fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57572/0 (socket says 172.21.15.122:57572) 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3804287579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 msgr2=0x562138a07980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x562138a07980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3804287579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 msgr2=0x562138a04990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 -- 172.21.15.122:0/3804287579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562138a051c0 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a80d9b640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a78041040 tx=0x7f8a78003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b86c684948b68c1c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a78061060 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f8a78003800 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8a78042040 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a85be3740 10 client.28193 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562138a76760 con 0x5621389f6be0 2024-04-02T13:47:16.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.678+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a78004a70 con 0x5621389f6be0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f8a7804f4f0 con 0x5621389f6be0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 10 client.28193 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 10 client.28193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 20 client.28193 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a76be0 0x562138a78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 4 client.28193 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562138962000 con 0x562138a76be0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a76be0 0x562138a78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a5bfff640 10 client.28193 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.679+0000 7f8a73fff640 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a76be0 0x562138a78fd0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60edbbd62bd87987 server_cookie=c81859684ccd6dcc in_seq=0 out_seq=0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.680+0000 7f8a5bfff640 1 -- 172.21.15.122:0/3804287579 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562138962000 con 0x562138a76be0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.680+0000 7f8a5bfff640 10 client.28193 handle_command_reply: tid=0 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.680+0000 7f8a85be3740 1 client.28193 shutdown 2024-04-02T13:47:16.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.680+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 msgr2=0x562138a07ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8a78041040 tx=0x7f8a78003be0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 shutdown_connections 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562138a76be0 0x562138a78fd0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621389ff7c0 0x562138a04990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621389f6be0 0x562138a07ec0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 --2- 172.21.15.122:0/3804287579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621389f62c0 0x562138a07980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 >> 172.21.15.122:0/3804287579 conn(0x5621389f09f0 msgr2=0x5621389ef220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a5bfff640 0 client.28193 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 shutdown_connections 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 1 -- 172.21.15.122:0/3804287579 wait complete. 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a85be3740 20 client.28193 trim_cache size 0 max 16384 2024-04-02T13:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "e34283ad-844f-45da-8384-fddc1f7b94ab", 2024-04-02T13:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 msgr2=0x7f8a640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 0x7f8a640a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a7409c9a0 tx=0x7f8a7406e6e0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 msgr2=0x7f8a7c103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400eb10 tx=0x7f8a7400c2d0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 shutdown_connections 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a640a3d20 0x7f8a640a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a7c106980 0x7f8a7c103a60 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a7c101330 0x7f8a7c103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 --2- 172.21.15.122:0/1832133877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a7c100970 0x7f8a7c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 >> 172.21.15.122:0/1832133877 conn(0x7f8a7c077b00 msgr2=0x7f8a7c068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.681+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 shutdown_connections 2024-04-02T13:47:16.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.682+0000 7f8a82fff640 1 -- 172.21.15.122:0/1832133877 wait complete. 2024-04-02T13:47:16.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:16.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 -- 172.21.15.122:0/1460752400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380101aa0 msgr2=0x7f3380101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 --2- 172.21.15.122:0/1460752400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380101aa0 0x7f3380101f20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f3370009930 tx=0x7f337002bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 -- 172.21.15.122:0/1460752400 shutdown_connections 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 --2- 172.21.15.122:0/1460752400 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380102460 0x7f338010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 --2- 172.21.15.122:0/1460752400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380101aa0 0x7f3380101f20 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 --2- 172.21.15.122:0/1460752400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380107ab0 0x7f3380107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 -- 172.21.15.122:0/1460752400 >> 172.21.15.122:0/1460752400 conn(0x7f33800fd530 msgr2=0x7f33800ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 -- 172.21.15.122:0/1460752400 shutdown_connections 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.824+0000 7f3388086640 1 -- 172.21.15.122:0/1460752400 wait complete. 2024-04-02T13:47:16.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 Processor -- start 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 -- start start 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380101aa0 0x7f3380073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380102460 0x7f3380074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3380111ec0 con 0x7f3380107ab0 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3380111d40 con 0x7f3380101aa0 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3388086640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3380112040 con 0x7f3380102460 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3386623640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3386623640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57588/0 (socket says 172.21.15.122:57588) 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3386623640 1 -- 172.21.15.122:0/725758022 learned_addr learned my addr 172.21.15.122:0/725758022 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3385e22640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380101aa0 0x7f3380073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3385621640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380102460 0x7f3380074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3386623640 1 -- 172.21.15.122:0/725758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380102460 msgr2=0x7f3380074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.825+0000 7f3386623640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380102460 0x7f3380074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3386623640 1 -- 172.21.15.122:0/725758022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380101aa0 msgr2=0x7f3380073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3386623640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380101aa0 0x7f3380073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3386623640 1 -- 172.21.15.122:0/725758022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33801a6900 con 0x7f3380107ab0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3386623640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f337800eb10 tx=0x7f337800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3376ffd640 1 -- 172.21.15.122:0/725758022 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378004b20 con 0x7f3380107ab0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3376ffd640 1 -- 172.21.15.122:0/725758022 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378004cc0 con 0x7f3380107ab0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.826+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33801a6bf0 con 0x7f3380107ab0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.827+0000 7f3376ffd640 1 -- 172.21.15.122:0/725758022 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378010760 con 0x7f3380107ab0 2024-04-02T13:47:16.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.827+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33801a7110 con 0x7f3380107ab0 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.828+0000 7f3376ffd640 1 -- 172.21.15.122:0/725758022 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3378005000 con 0x7f3380107ab0 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.829+0000 7f3376ffd640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 0x7f336c0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.829+0000 7f3385e22640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 0x7f336c0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.829+0000 7f338ac6f740 1 Processor -- start 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.829+0000 7f3376ffd640 1 -- 172.21.15.122:0/725758022 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f33780d10f0 con 0x7f3380107ab0 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 -- start start 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267483cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385e22640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 0x7f336c0a6140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f337c009b30 tx=0x7f337c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x55926748f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592673e7830 con 0x5592674841f0 2024-04-02T13:47:16.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55926738f820 con 0x5592674838d0 2024-04-02T13:47:16.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f338ac6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592673e37e0 con 0x55926748cdd0 2024-04-02T13:47:16.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267483cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57596/0 (socket says 172.21.15.122:57596) 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 -- 172.21.15.122:0/3777912715 learned_addr learned my addr 172.21.15.122:0/3777912715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3386623640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x55926748f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 -- 172.21.15.122:0/3777912715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 msgr2=0x55926748f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x55926748f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 -- 172.21.15.122:0/3777912715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 msgr2=0x559267483cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267483cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385621640 1 -- 172.21.15.122:0/3777912715 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592671c57d0 con 0x5592674841f0 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3386623640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x55926748f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.830+0000 7f3385e22640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267483cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.831+0000 7f3385621640 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f337002cfd0 tx=0x7f33700095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157d3a4bcb0815aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.831+0000 7f336bfff640 1 -- 172.21.15.122:0/3777912715 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370042070 con 0x5592674841f0 2024-04-02T13:47:16.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.831+0000 7f336bfff640 1 -- 172.21.15.122:0/3777912715 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3370004030 con 0x5592674841f0 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.831+0000 7f338ac6f740 1 -- 172.21.15.122:0/3777912715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 msgr2=0x55926748c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.831+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f337002cfd0 tx=0x7f33700095f0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 -- 172.21.15.122:0/3777912715 shutdown_connections 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x55926748f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x55926748c890 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3777912715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267483cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 -- 172.21.15.122:0/3777912715 >> 172.21.15.122:0/3777912715 conn(0x55926747e000 msgr2=0x55926747e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 -- 172.21.15.122:0/3777912715 shutdown_connections 2024-04-02T13:47:16.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 -- 172.21.15.122:0/3777912715 wait complete. 2024-04-02T13:47:16.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 -- start start 2024-04-02T13:47:16.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.832+0000 7f338ac6f740 1 Processor -- start 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x559267489970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x559267489eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385621640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x559267489970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59422/0 (socket says 172.21.15.122:59422) 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 -- 172.21.15.122:0/4097845281 learned_addr learned my addr 172.21.15.122:0/4097845281 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385621640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x559267489970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57608/0 (socket says 172.21.15.122:57608) 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55926748fbb0 con 0x5592674841f0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592673a2d30 con 0x5592674838d0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55926748fd30 con 0x55926748cdd0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3386623640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x559267489eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f338ac6f740 10 client.0 _ll_register_callbacks cb 0x5592673a2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 -- 172.21.15.122:0/4097845281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 msgr2=0x559267489eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x559267489eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 -- 172.21.15.122:0/4097845281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 msgr2=0x559267489970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x559267489970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592670fb3f0 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f3385e22640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f337c0038a0 tx=0x7f337c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7655214fc4ac24f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f33697fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.833+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c01fbb0 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f337c01fd50 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3370004030 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f337c005020 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 10 client.28205 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5592670bb0f0 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3370004030 con 0x5592674838d0 2024-04-02T13:47:16.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f337c026070 con 0x5592674838d0 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 10 client.28205 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 10 client.28205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 20 client.28205 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55926749cd40 0x55926749f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 4 client.28205 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592673f4d70 con 0x55926749cd40 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.834+0000 7f3385621640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55926749cd40 0x55926749f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f3385621640 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55926749cd40 0x55926749f200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21519f49661b0b93 server_cookie=5cd56d788cb7f5cd in_seq=0 out_seq=0 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f33697fa640 10 client.28205 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f33697fa640 1 -- 172.21.15.122:0/4097845281 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592673f4d70 con 0x55926749cd40 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f33697fa640 10 client.28205 handle_command_reply: tid=0 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f338ac6f740 1 client.28205 shutdown 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 msgr2=0x55926748c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.835+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f337c0038a0 tx=0x7f337c004730 comp rx=0 tx=0).stop 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 shutdown_connections 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55926749cd40 0x55926749f200 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55926748cdd0 0x559267489eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f33697fa640 0 client.28205 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592674841f0 0x559267489970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 --2- 172.21.15.122:0/4097845281 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x55926748c5e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 >> 172.21.15.122:0/4097845281 conn(0x55926747e000 msgr2=0x55926747fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 shutdown_connections 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 1 -- 172.21.15.122:0/4097845281 wait complete. 2024-04-02T13:47:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.836+0000 7f338ac6f740 20 client.28205 trim_cache size 0 max 16384 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.853+0000 7f338ac6f740 1 Processor -- start 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.853+0000 7f338ac6f740 1 -- start start 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267498ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267484710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b0fb0 con 0x55926748cdd0 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b2af0 con 0x5592674838d0 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f338ac6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b1680 con 0x5592674841f0 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3386623640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385621640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267484710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267498ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3386623640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57616/0 (socket says 172.21.15.122:57616) 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3386623640 1 -- 172.21.15.122:0/3888145628 learned_addr learned my addr 172.21.15.122:0/3888145628 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 -- 172.21.15.122:0/3888145628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 msgr2=0x559267484710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267484710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 -- 172.21.15.122:0/3888145628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 msgr2=0x55926748c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 -- 172.21.15.122:0/3888145628 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926747de30 con 0x5592674838d0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3386623640 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.854+0000 7f3385e22640 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267498ac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f337c004010 tx=0x7f337c027760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972dff1fbcdb85de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f336b7fe640 1 -- 172.21.15.122:0/3888145628 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c0047a0 con 0x5592674838d0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f336b7fe640 1 -- 172.21.15.122:0/3888145628 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c0051c0 con 0x5592674838d0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f336b7fe640 1 -- 172.21.15.122:0/3888145628 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c027c70 con 0x5592674838d0 2024-04-02T13:47:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 -- 172.21.15.122:0/3888145628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 msgr2=0x559267498ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267498ac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f337c004010 tx=0x7f337c027760 comp rx=0 tx=0).stop 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 -- 172.21.15.122:0/3888145628 shutdown_connections 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x55926748c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267484710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3888145628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267498ac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 -- 172.21.15.122:0/3888145628 >> 172.21.15.122:0/3888145628 conn(0x55926747e000 msgr2=0x55926747d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 -- 172.21.15.122:0/3888145628 shutdown_connections 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.855+0000 7f338ac6f740 1 -- 172.21.15.122:0/3888145628 wait complete. 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 -- start start 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 Processor -- start 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267490850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x559267490e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b2420 con 0x55926748cdd0 2024-04-02T13:47:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33801b1d50 con 0x5592674838d0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33801aefb0 con 0x5592674841f0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f338ac6f740 10 client.0 _ll_register_callbacks cb 0x7f337c0044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f3386623640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x559267490e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f3385e22640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:59430/0 (socket says 172.21.15.122:59430) 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f3385621640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267490850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.856+0000 7f3385e22640 1 -- 172.21.15.122:0/3595942617 learned_addr learned my addr 172.21.15.122:0/3595942617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 -- 172.21.15.122:0/3595942617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 msgr2=0x559267490850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267490850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 -- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 msgr2=0x559267495490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 -- 172.21.15.122:0/3595942617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559267491660 con 0x55926748cdd0 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3385e22640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f3386623640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x559267490e60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3378099670 tx=0x7f337809d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa9c454c68e17f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780b5020 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f33780a5480 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f337c0051c0 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f33780bbe60 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 10 client.19215 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559267503500 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f337c0051c0 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f33780cecc0 con 0x55926748cdd0 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 10 client.19215 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 10 client.19215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 20 client.19215 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559267503c20 0x559267506010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 4 client.19215 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.857+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592673ef0c0 con 0x559267503c20 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f3385e22640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559267503c20 0x559267506010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f3385e22640 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559267503c20 0x559267506010 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebfe5035da9295d1 server_cookie=3f8ab9dcdb36955c in_seq=0 out_seq=0 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f33697fa640 10 client.19215 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f33697fa640 1 -- 172.21.15.122:0/3595942617 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5592673ef0c0 con 0x559267503c20 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f33697fa640 10 client.19215 handle_command_reply: tid=0 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 client.19215 shutdown 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 msgr2=0x559267490e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x559267490e60 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3378099670 tx=0x7f337809d3a0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 shutdown_connections 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559267503c20 0x559267506010 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55926748cdd0 0x559267490e60 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f33697fa640 0 client.19215 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592674841f0 0x559267490850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 --2- 172.21.15.122:0/3595942617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592674838d0 0x559267495490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 >> 172.21.15.122:0/3595942617 conn(0x55926747e000 msgr2=0x55926747c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 shutdown_connections 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 1 -- 172.21.15.122:0/3595942617 wait complete. 2024-04-02T13:47:16.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.858+0000 7f338ac6f740 20 client.19215 trim_cache size 0 max 16384 2024-04-02T13:47:16.869 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:16.869 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:47:16.869 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:47:16.869 INFO:teuthology.orchestra.run.smithi122.stdout: "e34283ad-844f-45da-8384-fddc1f7b94ab": { 2024-04-02T13:47:16.869 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:47:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "e34283ad-844f-45da-8384-fddc1f7b94ab", 2024-04-02T13:47:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:47:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:47:16.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:47:16.870 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 msgr2=0x7f336c0a6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 0x7f336c0a6140 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f337c009b30 tx=0x7f337c005f90 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 msgr2=0x7f3380074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f337800eb10 tx=0x7f337800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 shutdown_connections 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f336c0a3c80 0x7f336c0a6140 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3380107ab0 0x7f3380074880 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3380102460 0x7f3380074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 --2- 172.21.15.122:0/725758022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3380101aa0 0x7f3380073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 >> 172.21.15.122:0/725758022 conn(0x7f33800fd530 msgr2=0x7f33800fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 shutdown_connections 2024-04-02T13:47:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:16.859+0000 7f3388086640 1 -- 172.21.15.122:0/725758022 wait complete. 2024-04-02T13:47:16.873 INFO:tasks.cephfs.filesystem:scrub status for tag:e34283ad-844f-45da-8384-fddc1f7b94ab - {'path': '/', 'tag': 'e34283ad-844f-45da-8384-fddc1f7b94ab', 'options': 'recursive,force'} 2024-04-02T13:47:16.978 INFO:tasks.workunit.client.0.smithi122.stdout: 1 476191 3.96 MB/sec execute 517 sec latency 41.610 ms 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: pgmap v377: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:17 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: pgmap v377: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:17 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: pgmap v377: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: osdmap e107: 12 total, 12 up, 12 in 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:17 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:17.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 477309 3.96 MB/sec execute 518 sec latency 15.128 ms 2024-04-02T13:47:18.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:47:18.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:18.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:18 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:18 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:18 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:18.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 479488 3.97 MB/sec execute 519 sec latency 27.896 ms 2024-04-02T13:47:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:19 smithi187 ceph-mon[36064]: pgmap v379: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:19.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:19 smithi188 ceph-mon[36117]: pgmap v379: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:19 smithi122 ceph-mon[32114]: pgmap v379: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:19.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 480648 3.98 MB/sec execute 520 sec latency 14.533 ms 2024-04-02T13:47:20.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 482660 3.99 MB/sec execute 521 sec latency 19.953 ms 2024-04-02T13:47:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:21 smithi187 ceph-mon[36064]: pgmap v380: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:21 smithi188 ceph-mon[36117]: pgmap v380: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:21 smithi122 ceph-mon[32114]: pgmap v380: 129 pgs: 129 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 232 op/s 2024-04-02T13:47:21.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 483999 4.00 MB/sec execute 522 sec latency 26.121 ms 2024-04-02T13:47:22.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 485789 4.00 MB/sec execute 523 sec latency 39.680 ms 2024-04-02T13:47:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:23 smithi187 ceph-mon[36064]: pgmap v381: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:23 smithi188 ceph-mon[36117]: pgmap v381: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:23 smithi122 ceph-mon[32114]: pgmap v381: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:23.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 487294 4.02 MB/sec execute 524 sec latency 20.186 ms 2024-04-02T13:47:24.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 489139 4.02 MB/sec execute 525 sec latency 18.531 ms 2024-04-02T13:47:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:25 smithi187 ceph-mon[36064]: pgmap v382: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:25 smithi188 ceph-mon[36117]: pgmap v382: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:25 smithi122 ceph-mon[32114]: pgmap v382: 129 pgs: 129 active+clean; 427 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 274 op/s 2024-04-02T13:47:25.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 490717 4.04 MB/sec execute 526 sec latency 11.037 ms 2024-04-02T13:47:26.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 492399 4.04 MB/sec execute 527 sec latency 18.280 ms 2024-04-02T13:47:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: pgmap v383: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-04-02T13:47:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:27.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:27 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: pgmap v383: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:27 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: pgmap v383: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.9 MiB/s wr, 282 op/s 2024-04-02T13:47:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:27.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:27.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:27 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:47:27.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 494100 4.06 MB/sec execute 528 sec latency 57.712 ms 2024-04-02T13:47:28.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 495348 4.06 MB/sec execute 529 sec latency 24.778 ms 2024-04-02T13:47:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: pgmap v384: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-02T13:47:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:28 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: pgmap v384: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-02T13:47:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:28 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: pgmap v384: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:47:29.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:28 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:47:29.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 497610 4.08 MB/sec execute 530 sec latency 14.800 ms 2024-04-02T13:47:30.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 498909 4.08 MB/sec execute 531 sec latency 16.440 ms 2024-04-02T13:47:31.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:31 smithi187 ceph-mon[36064]: pgmap v385: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-04-02T13:47:31.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:31 smithi188 ceph-mon[36117]: pgmap v385: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-04-02T13:47:31.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:31 smithi122 ceph-mon[32114]: pgmap v385: 129 pgs: 129 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-04-02T13:47:31.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 501162 4.10 MB/sec execute 532 sec latency 14.780 ms 2024-04-02T13:47:32.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 502111 4.10 MB/sec execute 533 sec latency 100.887 ms 2024-04-02T13:47:33.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:33 smithi187 ceph-mon[36064]: pgmap v386: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-04-02T13:47:33.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:33 smithi188 ceph-mon[36117]: pgmap v386: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-04-02T13:47:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:33 smithi122 ceph-mon[32114]: pgmap v386: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 350 op/s 2024-04-02T13:47:33.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 504305 4.11 MB/sec execute 534 sec latency 20.620 ms 2024-04-02T13:47:34.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 505529 4.12 MB/sec execute 535 sec latency 14.302 ms 2024-04-02T13:47:35.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:35 smithi187 ceph-mon[36064]: pgmap v387: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-02T13:47:35.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:35 smithi188 ceph-mon[36117]: pgmap v387: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-02T13:47:35.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:35 smithi122 ceph-mon[32114]: pgmap v387: 129 pgs: 129 active+clean; 448 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 234 op/s 2024-04-02T13:47:35.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 507696 4.13 MB/sec execute 536 sec latency 16.836 ms 2024-04-02T13:47:36.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 508947 4.14 MB/sec execute 537 sec latency 12.523 ms 2024-04-02T13:47:37.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:37 smithi187 ceph-mon[36064]: pgmap v388: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 341 op/s 2024-04-02T13:47:37.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:37 smithi188 ceph-mon[36117]: pgmap v388: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 341 op/s 2024-04-02T13:47:37.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:37 smithi122 ceph-mon[32114]: pgmap v388: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 341 op/s 2024-04-02T13:47:37.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 511054 4.15 MB/sec execute 538 sec latency 17.586 ms 2024-04-02T13:47:38.979 INFO:tasks.workunit.client.0.smithi122.stdout: 1 512322 4.15 MB/sec execute 539 sec latency 23.152 ms 2024-04-02T13:47:39.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:39 smithi187 ceph-mon[36064]: pgmap v389: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:39.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:39 smithi188 ceph-mon[36117]: pgmap v389: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:39.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:39 smithi122 ceph-mon[32114]: pgmap v389: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:39.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 514189 4.16 MB/sec execute 540 sec latency 17.997 ms 2024-04-02T13:47:40.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 515718 4.17 MB/sec execute 541 sec latency 12.610 ms 2024-04-02T13:47:41.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:41 smithi187 ceph-mon[36064]: pgmap v390: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:41.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:41 smithi188 ceph-mon[36117]: pgmap v390: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:41.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:41 smithi122 ceph-mon[32114]: pgmap v390: 129 pgs: 129 active+clean; 478 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 221 op/s 2024-04-02T13:47:41.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 517562 4.18 MB/sec execute 542 sec latency 16.270 ms 2024-04-02T13:47:42.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 519103 4.19 MB/sec execute 543 sec latency 43.190 ms 2024-04-02T13:47:43.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:43 smithi187 ceph-mon[36064]: pgmap v391: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-04-02T13:47:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:43 smithi188 ceph-mon[36117]: pgmap v391: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-04-02T13:47:43.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:43 smithi122 ceph-mon[32114]: pgmap v391: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.4 MiB/s wr, 315 op/s 2024-04-02T13:47:43.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 520588 4.20 MB/sec execute 544 sec latency 30.474 ms 2024-04-02T13:47:44.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 522501 4.21 MB/sec execute 545 sec latency 80.169 ms 2024-04-02T13:47:45.372 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:45 smithi122 ceph-mon[32114]: pgmap v392: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2024-04-02T13:47:45.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:45 smithi187 ceph-mon[36064]: pgmap v392: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2024-04-02T13:47:45.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:45 smithi188 ceph-mon[36117]: pgmap v392: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.7 MiB/s wr, 200 op/s 2024-04-02T13:47:45.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 523900 4.22 MB/sec execute 546 sec latency 16.528 ms 2024-04-02T13:47:46.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:46.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:46.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 526025 4.23 MB/sec execute 547 sec latency 16.274 ms 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.986+0000 7fc5aa152640 1 -- 172.21.15.122:0/1324947130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 msgr2=0x7fc5a4101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.986+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1324947130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4101160 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc598009900 tx=0x7fc59802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.986+0000 7fc5aa152640 1 -- 172.21.15.122:0/1324947130 shutdown_connections 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.986+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1324947130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 0x7fc5a410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.986+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1324947130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4101160 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:46.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1324947130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 -- 172.21.15.122:0/1324947130 >> 172.21.15.122:0/1324947130 conn(0x7fc5a40779e0 msgr2=0x7fc5a4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 -- 172.21.15.122:0/1324947130 shutdown_connections 2024-04-02T13:47:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 -- 172.21.15.122:0/1324947130 wait complete. 2024-04-02T13:47:46.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 Processor -- start 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 -- start start 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.987+0000 7fc5aa152640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 0x7fc5a4196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410fcc0 con 0x7fc5a4100ce0 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410fb40 con 0x7fc5a41016a0 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410fe40 con 0x7fc5a4106a30 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:46.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39648/0 (socket says 172.21.15.122:39648) 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 -- 172.21.15.122:0/1023404951 learned_addr learned my addr 172.21.15.122:0/1023404951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 0x7fc5a4196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 -- 172.21.15.122:0/1023404951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 msgr2=0x7fc5a419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 -- 172.21.15.122:0/1023404951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 msgr2=0x7fc5a4196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 0x7fc5a4196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 -- 172.21.15.122:0/1023404951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a419b110 con 0x7fc5a4100ce0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc594010ad0 tx=0x7fc594010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a0ff9640 1 -- 172.21.15.122:0/1023404951 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59401a020 con 0x7fc5a4100ce0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a0ff9640 1 -- 172.21.15.122:0/1023404951 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59400ec30 con 0x7fc5a4100ce0 2024-04-02T13:47:46.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a419b400 con 0x7fc5a4100ce0 2024-04-02T13:47:46.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5a0ff9640 1 -- 172.21.15.122:0/1023404951 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc594015760 con 0x7fc5a4100ce0 2024-04-02T13:47:46.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.988+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a2c90 con 0x7fc5a4100ce0 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5a0ff9640 1 -- 172.21.15.122:0/1023404951 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc59401d020 con 0x7fc5a4100ce0 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 Processor -- start 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 -- start start 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996f20 0x55d7069a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 0x55d7069a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7068fa830 con 0x55d706996f20 2024-04-02T13:47:47.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d7068a2820 con 0x55d7069a41e0 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.991+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7068f67e0 con 0x55d706996600 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996f20 0x55d7069a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 0x55d7069a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069969e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:56934/0 (socket says 172.21.15.122:56934) 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a3fff640 1 -- 172.21.15.122:0/2483883469 learned_addr learned my addr 172.21.15.122:0/2483883469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a0ff9640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 0x7fc58c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 0x7fc58c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/2483883469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 msgr2=0x55d7069969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/2483883469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996f20 msgr2=0x55d7069a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996f20 0x55d7069a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 -- 172.21.15.122:0/2483883469 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7066d87d0 con 0x55d7069a41e0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a0ff9640 1 -- 172.21.15.122:0/1023404951 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fc5940d3130 con 0x7fc5a4100ce0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 0x55d7069a65d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc59802c170 tx=0x7fc598005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d0d37cd89889a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 0x7fc58c0a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0099a0 tx=0x7fc59c004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc589ffb640 1 -- 172.21.15.122:0/2483883469 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598045070 con 0x55d7069a41e0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc589ffb640 1 -- 172.21.15.122:0/2483883469 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59802c880 con 0x55d7069a41e0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.992+0000 7fc589ffb640 1 -- 172.21.15.122:0/2483883469 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc598004bc0 con 0x55d7069a41e0 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2483883469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 msgr2=0x55d7069a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 0x55d7069a65d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc59802c170 tx=0x7fc598005be0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2483883469 shutdown_connections 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d7069a41e0 0x55d7069a65d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996f20 0x55d7069a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2483883469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2483883469 >> 172.21.15.122:0/2483883469 conn(0x55d706991000 msgr2=0x55d7069913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2483883469 shutdown_connections 2024-04-02T13:47:47.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2483883469 wait complete. 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.993+0000 7fc5ace1b740 1 -- start start 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 Processor -- start 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d7069a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7069a41e0 0x55d7069a1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d7069a6f40 con 0x55d706996600 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d7068b5d30 con 0x55d706996f20 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7069a70c0 con 0x55d7069a41e0 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5ace1b740 10 client.0 _ll_register_callbacks cb 0x55d7068b5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39666/0 (socket says 172.21.15.122:39666) 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 -- 172.21.15.122:0/2958053891 learned_addr learned my addr 172.21.15.122:0/2958053891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d7069a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7069a41e0 0x55d7069a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 -- 172.21.15.122:0/2958053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7069a41e0 msgr2=0x55d7069a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7069a41e0 0x55d7069a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 -- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 msgr2=0x55d7069a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d7069a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 -- 172.21.15.122:0/2958053891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d70660e3f0 con 0x55d706996600 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc59409b140 tx=0x7fc59400b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161964e4e1f8b184 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.994+0000 7fc582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59409f3e0 con 0x55d706996600 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fc5940d0be0 con 0x55d706996600 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59802c880 con 0x55d706996600 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc59400ede0 con 0x55d706996600 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 10 client.19227 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d7065ce0f0 con 0x55d706996600 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59802c880 con 0x55d706996600 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fc5940bf0b0 con 0x55d706996600 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 10 client.19227 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 10 client.19227 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 20 client.19227 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7069b4130 0x55d7069b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 4 client.19227 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d706907d70 con 0x55d7069b4130 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.995+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7069b4130 0x55d7069b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7069b4130 0x55d7069b65c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d79f44c6d4630b0 server_cookie=1f2deeb7846f8ad3 in_seq=0 out_seq=0 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc582ffd640 10 client.19227 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc582ffd640 1 -- 172.21.15.122:0/2958053891 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d706907d70 con 0x55d7069b4130 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc582ffd640 10 client.19227 handle_command_reply: tid=0 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc5ace1b740 1 client.19227 shutdown 2024-04-02T13:47:47.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 msgr2=0x55d7069a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.996+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fc59409b140 tx=0x7fc59400b9e0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 shutdown_connections 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d7069b4130 0x55d7069b65c0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7069a41e0 0x55d7069a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d7069a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/2958053891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d706996600 0x55d7069a3a40 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 >> 172.21.15.122:0/2958053891 conn(0x55d706991000 msgr2=0x55d706992d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc582ffd640 0 client.19227 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 shutdown_connections 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 1 -- 172.21.15.122:0/2958053891 wait complete. 2024-04-02T13:47:47.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:46.997+0000 7fc5ace1b740 20 client.19227 trim_cache size 0 max 16384 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.016+0000 7fc5ace1b740 1 Processor -- start 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 -- start start 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069afe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d70699c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41ace50 con 0x55d7069a41e0 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41ae990 con 0x55d706996f20 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41ad520 con 0x55d706996600 2024-04-02T13:47:47.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39678/0 (socket says 172.21.15.122:39678) 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 -- 172.21.15.122:0/1864265044 learned_addr learned my addr 172.21.15.122:0/1864265044 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d70699c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 -- 172.21.15.122:0/1864265044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 msgr2=0x55d7069afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 -- 172.21.15.122:0/1864265044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 msgr2=0x55d70699c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d70699c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 -- 172.21.15.122:0/1864265044 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d706996d50 con 0x55d7069a41e0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d70699c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0095b0 tx=0x7fc59c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553b39aff1a51ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.017+0000 7fc5897fa640 1 -- 172.21.15.122:0/1864265044 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c012110 con 0x55d7069a41e0 2024-04-02T13:47:47.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5897fa640 1 -- 172.21.15.122:0/1864265044 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c025070 con 0x55d7069a41e0 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5897fa640 1 -- 172.21.15.122:0/1864265044 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c003d30 con 0x55d7069a41e0 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 -- 172.21.15.122:0/1864265044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 msgr2=0x55d70699fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0095b0 tx=0x7fc59c009bf0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 -- 172.21.15.122:0/1864265044 shutdown_connections 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d70699fdb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996f20 0x55d70699c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/1864265044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996600 0x55d7069afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 -- 172.21.15.122:0/1864265044 >> 172.21.15.122:0/1864265044 conn(0x55d706991000 msgr2=0x55d70699f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 -- 172.21.15.122:0/1864265044 shutdown_connections 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.018+0000 7fc5ace1b740 1 -- 172.21.15.122:0/1864265044 wait complete. 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 -- start start 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 Processor -- start 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996f20 0x55d7069a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d706a18710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41ae2c0 con 0x55d7069a41e0 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41adbf0 con 0x55d706996600 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a41aae50 con 0x55d706996f20 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5ace1b740 10 client.0 _ll_register_callbacks cb 0x7fc59400b910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5a37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35756/0 (socket says 172.21.15.122:35756) 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5a37fe640 1 -- 172.21.15.122:0/3543526882 learned_addr learned my addr 172.21.15.122:0/3543526882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5a3fff640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d706a18710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.019+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996f20 0x55d7069a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 -- 172.21.15.122:0/3543526882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996f20 msgr2=0x55d7069a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996f20 0x55d7069a7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 -- 172.21.15.122:0/3543526882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 msgr2=0x55d706a18710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d706a18710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 -- 172.21.15.122:0/3543526882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d706a18c50 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5a37fe640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc594010aa0 tx=0x7fc594016c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4aecb92ecb67c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59409f3e0 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fc59409b350 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c025070 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc5940a3090 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5ace1b740 10 client.28229 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.020+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d706a19f30 con 0x55d706996600 2024-04-02T13:47:47.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c025070 con 0x55d706996600 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fc59409e130 con 0x55d706996600 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 10 client.28229 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 10 client.28229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 20 client.28229 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d706a1a6b0 0x55d706a1cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 4 client.28229 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.021+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7069020c0 con 0x55d706a1a6b0 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.022+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d706a1a6b0 0x55d706a1cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.022+0000 7fc5a2ffd640 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d706a1a6b0 0x55d706a1cad0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34bd0bb77ba502f0 server_cookie=b7c6012f71da6fed in_seq=0 out_seq=0 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.022+0000 7fc582ffd640 10 client.28229 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc582ffd640 1 -- 172.21.15.122:0/3543526882 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7069020c0 con 0x55d706a1a6b0 2024-04-02T13:47:47.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc582ffd640 10 client.28229 handle_command_reply: tid=0 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 client.28229 shutdown 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 msgr2=0x55d7069a7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc594010aa0 tx=0x7fc594016c00 comp rx=0 tx=0).stop 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 shutdown_connections 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d706a1a6b0 0x55d706a1cad0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d7069a41e0 0x55d706a18710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc582ffd640 0 client.28229 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d706996f20 0x55d7069a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 --2- 172.21.15.122:0/3543526882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d706996600 0x55d7069a7500 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 >> 172.21.15.122:0/3543526882 conn(0x55d706991000 msgr2=0x55d70699ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 shutdown_connections 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 1 -- 172.21.15.122:0/3543526882 wait complete. 2024-04-02T13:47:47.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.023+0000 7fc5ace1b740 20 client.28229 trim_cache size 0 max 16384 2024-04-02T13:47:47.033 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:47.033 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:47:47.033 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:47:47.033 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 msgr2=0x7fc58c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 0x7fc58c0a61e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0099a0 tx=0x7fc59c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 msgr2=0x7fc5a4196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fc594010ad0 tx=0x7fc594010f90 comp rx=0 tx=0).stop 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 shutdown_connections 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc58c0a3d20 0x7fc58c0a61e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5a4106a30 0x7fc5a419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5a41016a0 0x7fc5a4196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 --2- 172.21.15.122:0/1023404951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5a4100ce0 0x7fc5a4196120 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 >> 172.21.15.122:0/1023404951 conn(0x7fc5a40779e0 msgr2=0x7fc5a40ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 shutdown_connections 2024-04-02T13:47:47.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.024+0000 7fc5aa152640 1 -- 172.21.15.122:0/1023404951 wait complete. 2024-04-02T13:47:47.039 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:47:47.039 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 -- 172.21.15.122:0/3730037841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 msgr2=0x7f93ec10e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 --2- 172.21.15.122:0/3730037841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec10e5e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009ca0 tx=0x7f93e402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 -- 172.21.15.122:0/3730037841 shutdown_connections 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 --2- 172.21.15.122:0/3730037841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec10e5e0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 --2- 172.21.15.122:0/3730037841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 --2- 172.21.15.122:0/3730037841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 -- 172.21.15.122:0/3730037841 >> 172.21.15.122:0/3730037841 conn(0x7f93ec077b00 msgr2=0x7f93ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 -- 172.21.15.122:0/3730037841 shutdown_connections 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.150+0000 7f93f378f640 1 -- 172.21.15.122:0/3730037841 wait complete. 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 Processor -- start 2024-04-02T13:47:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 -- start start 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec111f40 con 0x7f93ec10c140 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec111dc0 con 0x7f93ec102d70 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f378f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec1120c0 con 0x7f93ec103690 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f1d2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f0d2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35776/0 (socket says 172.21.15.122:35776) 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 -- 172.21.15.122:0/2967081854 learned_addr learned my addr 172.21.15.122:0/2967081854 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f1d2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39694/0 (socket says 172.21.15.122:39694) 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 -- 172.21.15.122:0/2967081854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 msgr2=0x7f93ec198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 -- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 msgr2=0x7f93ec19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f152b640 1 -- 172.21.15.122:0/2967081854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec19d3b0 con 0x7f93ec102d70 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f0d2a640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.151+0000 7f93f1d2c640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93f152b640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009ca0 tx=0x7f93e800d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e80183f0 con 0x7f93ec102d70 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e8018a70 con 0x7f93ec102d70 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec19d6a0 con 0x7f93ec102d70 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e800fa60 con 0x7f93ec102d70 2024-04-02T13:47:47.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.152+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec1008f0 con 0x7f93ec102d70 2024-04-02T13:47:47.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.154+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec0688f0 con 0x7f93ec102d70 2024-04-02T13:47:47.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.155+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f93e800fc00 con 0x7f93ec102d70 2024-04-02T13:47:47.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.156+0000 7f93e27fc640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 0x7f93d80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.156+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f93e80d1030 con 0x7f93ec102d70 2024-04-02T13:47:47.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.156+0000 7f93f0d2a640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 0x7f93d80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.156+0000 7f93f0d2a640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 0x7f93d80a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f93ec100370 tx=0x7f93dc005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.160+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f93e809cc90 con 0x7f93ec102d70 2024-04-02T13:47:47.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:47 smithi122 ceph-mon[32114]: pgmap v393: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-04-02T13:47:47.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.410+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93ec199710 con 0x7f93ec102d70 2024-04-02T13:47:47.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.410+0000 7f93e27fc640 1 -- 172.21.15.122:0/2967081854 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11946 (secure 0 0 0) 0x7f93e80a0f90 con 0x7f93ec102d70 2024-04-02T13:47:47.414 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:47:47.414 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:47:03.399575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:47:47.414 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 43 2024-04-02T13:47:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 msgr2=0x7f93d80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 0x7f93d80a61e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f93ec100370 tx=0x7f93dc005f90 comp rx=0 tx=0).stop 2024-04-02T13:47:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 msgr2=0x7f93ec1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009ca0 tx=0x7f93e800d6b0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 shutdown_connections 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a3d20 0x7f93d80a61e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93ec10c140 0x7f93ec19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93ec103690 0x7f93ec198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 --2- 172.21.15.122:0/2967081854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93ec102d70 0x7f93ec1983c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 >> 172.21.15.122:0/2967081854 conn(0x7f93ec077b00 msgr2=0x7f93ec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 shutdown_connections 2024-04-02T13:47:47.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.413+0000 7f93f378f640 1 -- 172.21.15.122:0/2967081854 wait complete. 2024-04-02T13:47:47.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:47:47.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:47 smithi187 ceph-mon[36064]: pgmap v393: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-04-02T13:47:47.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:47 smithi188 ceph-mon[36117]: pgmap v393: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 310 op/s 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.543+0000 7f777a03c640 1 -- 172.21.15.122:0/457690747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 msgr2=0x7f7774069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.543+0000 7f777a03c640 1 --2- 172.21.15.122:0/457690747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f7774069180 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f7768009930 tx=0x7f776802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 -- 172.21.15.122:0/457690747 shutdown_connections 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- 172.21.15.122:0/457690747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77740696c0 0x7f777410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- 172.21.15.122:0/457690747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f7774069180 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- 172.21.15.122:0/457690747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774107b50 0x7f7774107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 -- 172.21.15.122:0/457690747 >> 172.21.15.122:0/457690747 conn(0x7f7774077b00 msgr2=0x7f7774077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 -- 172.21.15.122:0/457690747 shutdown_connections 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 -- 172.21.15.122:0/457690747 wait complete. 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 Processor -- start 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 -- start start 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 0x7f7774198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.544+0000 7f777a03c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 0x7f777419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f777a03c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111e70 con 0x7f7774068d00 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f777a03c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111cf0 con 0x7f7774107b50 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f777a03c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7774111ff0 con 0x7f77740696c0 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39708/0 (socket says 172.21.15.122:39708) 2024-04-02T13:47:47.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 -- 172.21.15.122:0/1806913785 learned_addr learned my addr 172.21.15.122:0/1806913785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7773fff640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 0x7f777419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7772ffd640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 0x7f7774198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 -- 172.21.15.122:0/1806913785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 msgr2=0x7f7774198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 0x7f7774198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 -- 172.21.15.122:0/1806913785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 msgr2=0x7f777419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 0x7f777419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 -- 172.21.15.122:0/1806913785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419d370 con 0x7f7774068d00 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7773fff640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 0x7f777419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7772ffd640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 0x7f7774198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f77737fe640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f776400e9f0 tx=0x7f776400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7770ff9640 1 -- 172.21.15.122:0/1806913785 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400cd80 con 0x7f7774068d00 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419d660 con 0x7f7774068d00 2024-04-02T13:47:47.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7770ff9640 1 -- 172.21.15.122:0/1806913785 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7764010910 con 0x7f7774068d00 2024-04-02T13:47:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.545+0000 7f7770ff9640 1 -- 172.21.15.122:0/1806913785 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640188f0 con 0x7f7774068d00 2024-04-02T13:47:47.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.546+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741a4ef0 con 0x7f7774068d00 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7770ff9640 1 -- 172.21.15.122:0/1806913785 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7764010ac0 con 0x7f7774068d00 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 Processor -- start 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 -- start start 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5607282ff7b0 con 0x56072839fbb0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56072829e570 con 0x5607283a04d0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f777cc29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607282fb760 con 0x5607283a90b0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7773fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7773fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39714/0 (socket says 172.21.15.122:39714) 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7773fff640 1 -- 172.21.15.122:0/3575706199 learned_addr learned my addr 172.21.15.122:0/3575706199 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f77737fe640 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7772ffd640 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.549+0000 7f7770ff9640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 0x7f775c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 -- 172.21.15.122:0/3575706199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 msgr2=0x5607283ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 -- 172.21.15.122:0/3575706199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 msgr2=0x5607283a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283a8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 -- 172.21.15.122:0/3575706199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607280133f0 con 0x56072839fbb0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7770ff9640 1 -- 172.21.15.122:0/1806913785 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f77640d0f90 con 0x7f7774068d00 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 0x7f775c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f776c011940 tx=0x7f776c011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c25868cb6b9c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7773fff640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 0x7f775c0a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f776c023000 tx=0x7f776c024000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7759ffb640 1 -- 172.21.15.122:0/3575706199 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c017070 con 0x56072839fbb0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7759ffb640 1 -- 172.21.15.122:0/3575706199 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0046a0 con 0x56072839fbb0 2024-04-02T13:47:47.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.550+0000 7f7759ffb640 1 -- 172.21.15.122:0/3575706199 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c00e5f0 con 0x56072839fbb0 2024-04-02T13:47:47.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 -- 172.21.15.122:0/3575706199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 msgr2=0x56072839ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f776c011940 tx=0x7f776c011e00 comp rx=0 tx=0).stop 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 -- 172.21.15.122:0/3575706199 shutdown_connections 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 --2- 172.21.15.122:0/3575706199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x56072839ff90 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 -- 172.21.15.122:0/3575706199 >> 172.21.15.122:0/3575706199 conn(0x56072839a2e0 msgr2=0x56072839a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 -- 172.21.15.122:0/3575706199 shutdown_connections 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.551+0000 7f777cc29740 1 -- 172.21.15.122:0/3575706199 wait complete. 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 -- start start 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 Processor -- start 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 0x5607283a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5607283abea0 con 0x5607283a04d0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5607282bae80 con 0x56072839fbb0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5607283ac020 con 0x5607283a90b0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f777cc29740 10 client.0 _ll_register_callbacks cb 0x5607282f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39716/0 (socket says 172.21.15.122:39716) 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f77737fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 0x5607283a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.552+0000 7f7772ffd640 1 -- 172.21.15.122:0/4277139083 learned_addr learned my addr 172.21.15.122:0/4277139083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7773fff640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 -- 172.21.15.122:0/4277139083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 msgr2=0x5607283a69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283a69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 -- 172.21.15.122:0/4277139083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 msgr2=0x5607283a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 0x5607283a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 -- 172.21.15.122:0/4277139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607280dd7d0 con 0x5607283a04d0 2024-04-02T13:47:47.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f77737fe640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 0x5607283a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7772ffd640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7768009900 tx=0x7f7768004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161f49cc67cf52f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776802cab0 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c0046a0 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f776802cc50 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7768004ef0 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f777cc29740 10 client.19251 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5607283b7f90 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.553+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c0046a0 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f7768046070 con 0x5607283a04d0 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 10 client.19251 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 10 client.19251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 20 client.19251 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5607283b93f0 0x5607283bb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 4 client.19251 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56072830ccf0 con 0x5607283b93f0 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f77737fe640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5607283b93f0 0x5607283bb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f7752ffd640 10 client.19251 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.554+0000 7f77737fe640 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5607283b93f0 0x5607283bb870 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40a4bca27c26b5bb server_cookie=78aa0a1864533899 in_seq=0 out_seq=0 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f7752ffd640 1 -- 172.21.15.122:0/4277139083 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56072830ccf0 con 0x5607283b93f0 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f7752ffd640 10 client.19251 handle_command_reply: tid=0 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 client.19251 shutdown 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 msgr2=0x5607283a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f7768009900 tx=0x7f7768004270 comp rx=0 tx=0).stop 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 shutdown_connections 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5607283b93f0 0x5607283bb870 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283a69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f7752ffd640 0 client.19251 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283a6470 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 --2- 172.21.15.122:0/4277139083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56072839fbb0 0x5607283a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 >> 172.21.15.122:0/4277139083 conn(0x56072839a2e0 msgr2=0x56072839c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 shutdown_connections 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 1 -- 172.21.15.122:0/4277139083 wait complete. 2024-04-02T13:47:47.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.555+0000 7f777cc29740 20 client.19251 trim_cache size 0 max 16384 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 Processor -- start 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 -- start start 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ace90 con 0x56072839fbb0 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ae9d0 con 0x5607283a04d0 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f777cc29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ad560 con 0x5607283a90b0 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39722/0 (socket says 172.21.15.122:39722) 2024-04-02T13:47:47.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 -- 172.21.15.122:0/3899658723 learned_addr learned my addr 172.21.15.122:0/3899658723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f7772ffd640 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f7773fff640 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 -- 172.21.15.122:0/3899658723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 msgr2=0x5607283c1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 -- 172.21.15.122:0/3899658723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 msgr2=0x5607283b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.569+0000 7f77737fe640 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.570+0000 7f77737fe640 1 -- 172.21.15.122:0/3899658723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607283a0300 con 0x56072839fbb0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.570+0000 7f77737fe640 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f77640995c0 tx=0x7f77640d34a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35687951e673537 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.570+0000 7f77597fa640 1 -- 172.21.15.122:0/3899658723 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776409cbf0 con 0x56072839fbb0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.570+0000 7f77597fa640 1 -- 172.21.15.122:0/3899658723 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77640a0ef0 con 0x56072839fbb0 2024-04-02T13:47:47.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.570+0000 7f77597fa640 1 -- 172.21.15.122:0/3899658723 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640d4c90 con 0x56072839fbb0 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- 172.21.15.122:0/3899658723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 msgr2=0x5607283a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f77640995c0 tx=0x7f77640d34a0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- 172.21.15.122:0/3899658723 shutdown_connections 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5607283a90b0 0x5607283c1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a04d0 0x5607283b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 --2- 172.21.15.122:0/3899658723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56072839fbb0 0x5607283a8660 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- 172.21.15.122:0/3899658723 >> 172.21.15.122:0/3899658723 conn(0x56072839a2e0 msgr2=0x56072839bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- 172.21.15.122:0/3899658723 shutdown_connections 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- 172.21.15.122:0/3899658723 wait complete. 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 -- start start 2024-04-02T13:47:47.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.571+0000 7f777cc29740 1 Processor -- start 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56072839fbb0 0x5607283acf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 0x5607283ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77741ae300 con 0x5607283a04d0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77741adc30 con 0x5607283a90b0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77741aae90 con 0x56072839fbb0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 10 client.0 _ll_register_callbacks cb 0x7f7768009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7773fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 0x5607283ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39730/0 (socket says 172.21.15.122:39730) 2024-04-02T13:47:47.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 -- 172.21.15.122:0/1319953258 learned_addr learned my addr 172.21.15.122:0/1319953258 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f77737fe640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56072839fbb0 0x5607283acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 -- 172.21.15.122:0/1319953258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56072839fbb0 msgr2=0x5607283acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56072839fbb0 0x5607283acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 -- 172.21.15.122:0/1319953258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 msgr2=0x5607283ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 0x5607283ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 -- 172.21.15.122:0/1319953258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072841eae0 con 0x5607283a04d0 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7773fff640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 0x5607283ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7772ffd640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x5607283a7500 tx=0x7f776802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137f6bb7d6f4a0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768005d50 con 0x5607283a04d0 2024-04-02T13:47:47.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f776802ccc0 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.572+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77640a0ef0 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7768009520 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 10 client.19257 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56072841fde0 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77640a0ef0 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f7768046040 con 0x5607283a04d0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 10 client.19257 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 10 client.19257 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 20 client.19257 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x560728420390 0x5607284227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 4 client.19257 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560728307040 con 0x560728420390 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f77737fe640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x560728420390 0x5607284227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.573+0000 7f7752ffd640 10 client.19257 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f77737fe640 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x560728420390 0x5607284227b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9becc1c2b99936da server_cookie=7b6662801001bfd2 in_seq=0 out_seq=0 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f7752ffd640 1 -- 172.21.15.122:0/1319953258 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560728307040 con 0x560728420390 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f7752ffd640 10 client.19257 handle_command_reply: tid=0 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 client.19257 shutdown 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 msgr2=0x5607283ad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x5607283a7500 tx=0x7f776802c800 comp rx=0 tx=0).stop 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 shutdown_connections 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x560728420390 0x5607284227b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5607283a90b0 0x5607283ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5607283a04d0 0x5607283ad440 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f7752ffd640 0 client.19257 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 --2- 172.21.15.122:0/1319953258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56072839fbb0 0x5607283acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 >> 172.21.15.122:0/1319953258 conn(0x56072839a2e0 msgr2=0x5607283a5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 shutdown_connections 2024-04-02T13:47:47.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 1 -- 172.21.15.122:0/1319953258 wait complete. 2024-04-02T13:47:47.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.574+0000 7f777cc29740 20 client.19257 trim_cache size 0 max 16384 2024-04-02T13:47:47.586 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 msgr2=0x7f775c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 0x7f775c0a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f776c023000 tx=0x7f776c024000 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 msgr2=0x7f77741983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f776400e9f0 tx=0x7f776400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 shutdown_connections 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f775c0a3d20 0x7f775c0a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7774107b50 0x7f777419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740696c0 0x7f7774198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 --2- 172.21.15.122:0/1806913785 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7774068d00 0x7f77741983d0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 >> 172.21.15.122:0/1806913785 conn(0x7f7774077b00 msgr2=0x7f77740fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 shutdown_connections 2024-04-02T13:47:47.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.575+0000 7f777a03c640 1 -- 172.21.15.122:0/1806913785 wait complete. 2024-04-02T13:47:47.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:47:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.733+0000 7ff469700640 1 -- 172.21.15.122:0/586023678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641009c0 msgr2=0x7ff464100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.733+0000 7ff469700640 1 --2- 172.21.15.122:0/586023678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641009c0 0x7ff464100e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff458009860 tx=0x7ff45802bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.734+0000 7ff469700640 1 -- 172.21.15.122:0/586023678 shutdown_connections 2024-04-02T13:47:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.734+0000 7ff469700640 1 --2- 172.21.15.122:0/586023678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff464101360 0x7ff464069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.734+0000 7ff469700640 1 --2- 172.21.15.122:0/586023678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641009c0 0x7ff464100e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.734+0000 7ff469700640 1 --2- 172.21.15.122:0/586023678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff4641058a0 0x7ff464105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.734+0000 7ff469700640 1 -- 172.21.15.122:0/586023678 >> 172.21.15.122:0/586023678 conn(0x7ff464077a90 msgr2=0x7ff464077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.735+0000 7ff469700640 1 -- 172.21.15.122:0/586023678 shutdown_connections 2024-04-02T13:47:47.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.736+0000 7ff469700640 1 -- 172.21.15.122:0/586023678 wait complete. 2024-04-02T13:47:47.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.737+0000 7ff469700640 1 Processor -- start 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 -- start start 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4641009c0 0x7ff464116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff464101360 0x7ff46411bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410fd60 con 0x7ff4641009c0 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410fbe0 con 0x7ff4641058a0 2024-04-02T13:47:47.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.738+0000 7ff469700640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff46410fee0 con 0x7ff464101360 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35862/0 (socket says 172.21.15.122:35862) 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 -- 172.21.15.122:0/3259660634 learned_addr learned my addr 172.21.15.122:0/3259660634 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff462ffd640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4641009c0 0x7ff464116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 -- 172.21.15.122:0/3259660634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff464101360 msgr2=0x7ff46411bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff464101360 0x7ff46411bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 -- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4641009c0 msgr2=0x7ff464116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4641009c0 0x7ff464116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.739+0000 7ff4637fe640 1 -- 172.21.15.122:0/3259660634 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff464111540 con 0x7ff4641058a0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.740+0000 7ff4637fe640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.742+0000 7ff44bfff640 1 -- 172.21.15.122:0/3259660634 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c004b20 con 0x7ff4641058a0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.742+0000 7ff44bfff640 1 -- 172.21.15.122:0/3259660634 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45c003e50 con 0x7ff4641058a0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.742+0000 7ff44bfff640 1 -- 172.21.15.122:0/3259660634 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00b630 con 0x7ff4641058a0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.742+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff464111830 con 0x7ff4641058a0 2024-04-02T13:47:47.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.742+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff464111d30 con 0x7ff4641058a0 2024-04-02T13:47:47.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.745+0000 7ff46c3be740 1 Processor -- start 2024-04-02T13:47:47.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 -- start start 2024-04-02T13:47:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff44bfff640 1 -- 172.21.15.122:0/3259660634 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff45c01d020 con 0x7ff4641058a0 2024-04-02T13:47:47.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b797a0 0x556a96b79b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 0x556a96b85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556a96add7b0 con 0x556a96b797a0 2024-04-02T13:47:47.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556a96a7c570 con 0x556a96b7a0c0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.747+0000 7ff46c3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a96ad9760 con 0x556a96b82ca0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff44bfff640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 0x7ff44c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35870/0 (socket says 172.21.15.122:35870) 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 -- 172.21.15.122:0/4080874131 learned_addr learned my addr 172.21.15.122:0/4080874131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff44bfff640 1 -- 172.21.15.122:0/3259660634 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ff45c0d0c40 con 0x7ff4641058a0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff4627fc640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b797a0 0x556a96b79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff4627fc640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 0x556a96b85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff4637fe640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 0x7ff44c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 -- 172.21.15.122:0/4080874131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 msgr2=0x556a96b85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 0x556a96b85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 -- 172.21.15.122:0/4080874131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b797a0 msgr2=0x556a96b79b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b797a0 0x556a96b79b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.748+0000 7ff462ffd640 1 -- 172.21.15.122:0/4080874131 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a967f13f0 con 0x556a96b7a0c0 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.749+0000 7ff462ffd640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff454009a40 tx=0x7ff454009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9c0d544bee37535 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.749+0000 7ff4627fc640 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 0x556a96b85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.749+0000 7ff4497fa640 1 -- 172.21.15.122:0/4080874131 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454017070 con 0x556a96b7a0c0 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.749+0000 7ff4497fa640 1 -- 172.21.15.122:0/4080874131 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff454013470 con 0x556a96b7a0c0 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.749+0000 7ff4497fa640 1 -- 172.21.15.122:0/4080874131 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454012630 con 0x556a96b7a0c0 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.750+0000 7ff46c3be740 1 -- 172.21.15.122:0/4080874131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 msgr2=0x556a96b82760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.750+0000 7ff46c3be740 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff454009a40 tx=0x7ff454009f00 comp rx=0 tx=0).stop 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 -- 172.21.15.122:0/4080874131 shutdown_connections 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b82ca0 0x556a96b85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b7a0c0 0x556a96b82760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 --2- 172.21.15.122:0/4080874131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b797a0 0x556a96b79b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 -- 172.21.15.122:0/4080874131 >> 172.21.15.122:0/4080874131 conn(0x556a96b73ed0 msgr2=0x556a96b742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 -- 172.21.15.122:0/4080874131 shutdown_connections 2024-04-02T13:47:47.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.751+0000 7ff46c3be740 1 -- 172.21.15.122:0/4080874131 wait complete. 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 -- start start 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 Processor -- start 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556a96b85a90 con 0x556a96b7a0c0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556a96a98e80 con 0x556a96b82ca0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a96b85c10 con 0x556a96b797a0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.752+0000 7ff46c3be740 10 client.0 _ll_register_callbacks cb 0x556a96ad4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39776/0 (socket says 172.21.15.122:39776) 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 -- 172.21.15.122:0/267020511 learned_addr learned my addr 172.21.15.122:0/267020511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff4627fc640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff4627fc640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 -- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 msgr2=0x556a96b82750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b82750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 -- 172.21.15.122:0/267020511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 msgr2=0x556a96b80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b80220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 -- 172.21.15.122:0/267020511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a968bb7d0 con 0x556a96b7a0c0 2024-04-02T13:47:47.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.753+0000 7ff462ffd640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f32590b481f017a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff446ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff4637fe640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 0x7ff44c0a6320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00ffd0 tx=0x7ff45c004170 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454017050 con 0x556a96b7a0c0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff454013470 con 0x556a96b7a0c0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ff45401c450 con 0x556a96b7a0c0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff45402f050 con 0x556a96b7a0c0 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.754+0000 7ff46c3be740 10 client.19269 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a96b91ab0 con 0x556a96b7a0c0 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff45401cb30 con 0x556a96b7a0c0 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7ff454012930 con 0x556a96b7a0c0 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 10 client.19269 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 10 client.19269 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 20 client.19269 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b932b0 0x556a96b956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 4 client.19269 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.755+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff454013470 con 0x556a96b932b0 2024-04-02T13:47:47.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.756+0000 7ff4627fc640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b932b0 0x556a96b956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.757+0000 7ff4627fc640 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b932b0 0x556a96b956d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=312909adb5d14a60 server_cookie=612493b2cc28b4fc in_seq=0 out_seq=0 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.757+0000 7ff446ff5640 10 client.19269 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.757+0000 7ff446ff5640 1 -- 172.21.15.122:0/267020511 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ff454013470 con 0x556a96b932b0 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.757+0000 7ff446ff5640 10 client.19269 handle_command_reply: tid=0 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 client.19269 shutdown 2024-04-02T13:47:47.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 msgr2=0x556a96b7fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7ff454000c00 tx=0x7ff454013cd0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 shutdown_connections 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b932b0 0x556a96b956d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7fae0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 --2- 172.21.15.122:0/267020511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b82750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff446ff5640 0 client.19269 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 >> 172.21.15.122:0/267020511 conn(0x556a96b73ed0 msgr2=0x556a96b75c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 shutdown_connections 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 1 -- 172.21.15.122:0/267020511 wait complete. 2024-04-02T13:47:47.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.758+0000 7ff46c3be740 20 client.19269 trim_cache size 0 max 16384 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.785+0000 7ff46c3be740 1 Processor -- start 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.785+0000 7ff46c3be740 1 -- start start 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.785+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b82210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57076/0 (socket says 172.21.15.122:57076) 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b82210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff462ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b82210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39780/0 (socket says 172.21.15.122:39780) 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 -- 172.21.15.122:0/1579746460 learned_addr learned my addr 172.21.15.122:0/1579746460 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff46c3be740 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641c6950 con 0x556a96b7a0c0 2024-04-02T13:47:47.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641c8490 con 0x556a96b82ca0 2024-04-02T13:47:47.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641c7020 con 0x556a96b797a0 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 -- 172.21.15.122:0/1579746460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 msgr2=0x556a96b82750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b82750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 -- 172.21.15.122:0/1579746460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 msgr2=0x556a96b82210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b82210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.786+0000 7ff4627fc640 1 -- 172.21.15.122:0/1579746460 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a96b80590 con 0x556a96b797a0 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.787+0000 7ff4627fc640 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff458009250 tx=0x7ff45802c150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b528c55fbacbf4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.787+0000 7ff448ff9640 1 -- 172.21.15.122:0/1579746460 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45806e070 con 0x556a96b797a0 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.787+0000 7ff448ff9640 1 -- 172.21.15.122:0/1579746460 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff458034920 con 0x556a96b797a0 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.787+0000 7ff448ff9640 1 -- 172.21.15.122:0/1579746460 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458039040 con 0x556a96b797a0 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 msgr2=0x556a96b8e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff458009250 tx=0x7ff45802c150 comp rx=0 tx=0).stop 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 shutdown_connections 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b82750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b82210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 --2- 172.21.15.122:0/1579746460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8e950 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.788+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 >> 172.21.15.122:0/1579746460 conn(0x556a96b73ed0 msgr2=0x556a96b7e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.789+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 shutdown_connections 2024-04-02T13:47:47.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.789+0000 7ff46c3be740 1 -- 172.21.15.122:0/1579746460 wait complete. 2024-04-02T13:47:47.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.789+0000 7ff46c3be740 1 -- start start 2024-04-02T13:47:47.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.789+0000 7ff46c3be740 1 Processor -- start 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff4627fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57086/0 (socket says 172.21.15.122:57086) 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b7d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641c7dc0 con 0x556a96b7a0c0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff4641c76f0 con 0x556a96b82ca0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff464112880 con 0x556a96b797a0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff46c3be740 10 client.0 _ll_register_callbacks cb 0x7ff4540098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.790+0000 7ff4627fc640 1 -- 172.21.15.122:0/2663565671 learned_addr learned my addr 172.21.15.122:0/2663565671 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff4627fc640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b7d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 -- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 msgr2=0x556a96b8b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 -- 172.21.15.122:0/2663565671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 msgr2=0x556a96b7d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b7d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff462ffd640 1 -- 172.21.15.122:0/2663565671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a96b79ef0 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.791+0000 7ff4627fc640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff462ffd640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7cfd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e660 tx=0x7ff45401a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c513eb61a9384953 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff446ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff454017070 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7ff454013d40 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff458034920 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff4540138b0 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff46c3be740 10 client.19275 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.792+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556a96b89f20 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff458034920 con 0x556a96b7a0c0 2024-04-02T13:47:47.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7ff4540039f0 con 0x556a96b7a0c0 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 10 client.19275 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 10 client.19275 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 20 client.19275 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b8a6d0 0x556a96bfb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 4 client.19275 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.793+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a96aeacf0 con 0x556a96b8a6d0 2024-04-02T13:47:47.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.794+0000 7ff4627fc640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b8a6d0 0x556a96bfb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff446ff5640 10 client.19275 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff4627fc640 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b8a6d0 0x556a96bfb860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7f3fcd686eba362 server_cookie=6b08a4aadcc4d077 in_seq=0 out_seq=0 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff446ff5640 1 -- 172.21.15.122:0/2663565671 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a96aeacf0 con 0x556a96b8a6d0 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff446ff5640 10 client.19275 handle_command_reply: tid=0 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff46c3be740 1 client.19275 shutdown 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 msgr2=0x556a96b7cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.795+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7cfd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7ff45400e660 tx=0x7ff45401a5e0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 shutdown_connections 2024-04-02T13:47:47.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556a96b82ca0 0x556a96b7d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x556a96b8a6d0 0x556a96bfb860 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556a96b7a0c0 0x556a96b7cfd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 --2- 172.21.15.122:0/2663565671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a96b797a0 0x556a96b8b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff446ff5640 0 client.19275 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 >> 172.21.15.122:0/2663565671 conn(0x556a96b73ed0 msgr2=0x556a96b7f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 shutdown_connections 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 1 -- 172.21.15.122:0/2663565671 wait complete. 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff46c3be740 20 client.19275 trim_cache size 0 max 16384 2024-04-02T13:47:47.803 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 msgr2=0x7ff44c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 0x7ff44c0a6320 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00ffd0 tx=0x7ff45c004170 comp rx=0 tx=0).stop 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 msgr2=0x7ff4641167a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009fd0 tx=0x7ff45c00c800 comp rx=0 tx=0).stop 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 shutdown_connections 2024-04-02T13:47:47.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff44c0a3e60 0x7ff44c0a6320 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff4641058a0 0x7ff4641167a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff464101360 0x7ff46411bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 --2- 172.21.15.122:0/3259660634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4641009c0 0x7ff464116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.796+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 >> 172.21.15.122:0/3259660634 conn(0x7ff464077a90 msgr2=0x7ff4640ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.797+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 shutdown_connections 2024-04-02T13:47:47.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.797+0000 7ff469700640 1 -- 172.21.15.122:0/3259660634 wait complete. 2024-04-02T13:47:47.812 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- 172.21.15.122:0/3554609375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec100970 msgr2=0x7f65ec100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3554609375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec100970 0x7f65ec100df0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009930 tx=0x7f65dc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- 172.21.15.122:0/3554609375 shutdown_connections 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3554609375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec101330 0x7f65ec10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3554609375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec100970 0x7f65ec100df0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3554609375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec106980 0x7f65ec106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- 172.21.15.122:0/3554609375 >> 172.21.15.122:0/3554609375 conn(0x7f65ec077b00 msgr2=0x7f65ec077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- 172.21.15.122:0/3554609375 shutdown_connections 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- 172.21.15.122:0/3554609375 wait complete. 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 Processor -- start 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.922+0000 7f65f2b60640 1 -- start start 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec100970 0x7f65ec1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec101330 0x7f65ec1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110d90 con 0x7f65ec106980 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110c10 con 0x7f65ec101330 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110f10 con 0x7f65ec100970 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39802/0 (socket says 172.21.15.122:39802) 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 -- 172.21.15.122:0/3580023822 learned_addr learned my addr 172.21.15.122:0/3580023822 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65e3fff640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec101330 0x7f65ec1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec100970 0x7f65ec1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 -- 172.21.15.122:0/3580023822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec100970 msgr2=0x7f65ec1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec100970 0x7f65ec1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 -- 172.21.15.122:0/3580023822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec101330 msgr2=0x7f65ec1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec101330 0x7f65ec1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 -- 172.21.15.122:0/3580023822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19d3a0 con 0x7f65ec106980 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f65e400eaa0 tx=0x7f65e400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65e1ffb640 1 -- 172.21.15.122:0/3580023822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e400cdb0 con 0x7f65ec106980 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65e1ffb640 1 -- 172.21.15.122:0/3580023822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4005020 con 0x7f65ec106980 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec19d690 con 0x7f65ec106980 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.923+0000 7f65e1ffb640 1 -- 172.21.15.122:0/3580023822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40116b0 con 0x7f65ec106980 2024-04-02T13:47:47.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.924+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a4f20 con 0x7f65ec106980 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65e1ffb640 1 -- 172.21.15.122:0/3580023822 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f65e4005560 con 0x7f65ec106980 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 Processor -- start 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 -- start start 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 0x55634079db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65e1ffb640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 0x7f65d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5563407017b0 con 0x55634079d7a0 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5563406a0570 con 0x5563407a6ca0 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.926+0000 7f65f5826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5563406fd760 con 0x55634079e0c0 2024-04-02T13:47:47.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 0x7f65d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 0x55634079db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65e1ffb640 1 -- 172.21.15.122:0/3580023822 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f65e40d0d70 con 0x7f65ec106980 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35910/0 (socket says 172.21.15.122:35910) 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 0x55634079db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39816/0 (socket says 172.21.15.122:39816) 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 -- 172.21.15.122:0/4184150316 learned_addr learned my addr 172.21.15.122:0/4184150316 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65e3fff640 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 -- 172.21.15.122:0/4184150316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 msgr2=0x5563407a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 -- 172.21.15.122:0/4184150316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 msgr2=0x55634079db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 0x55634079db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 -- 172.21.15.122:0/4184150316 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563404153f0 con 0x5563407a6ca0 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65e3fff640 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:47:47.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 0x7f65d40a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009b50 tx=0x7f65e8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65f10fd640 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce130 tx=0x7f65e409d0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7304b09ea5bde78a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65d2ffd640 1 -- 172.21.15.122:0/4184150316 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e406e8b0 con 0x5563407a6ca0 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65d2ffd640 1 -- 172.21.15.122:0/4184150316 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e406eaf0 con 0x5563407a6ca0 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.927+0000 7f65d2ffd640 1 -- 172.21.15.122:0/4184150316 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e40ce350 con 0x5563407a6ca0 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- 172.21.15.122:0/4184150316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 msgr2=0x5563407a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce130 tx=0x7f65e409d0d0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- 172.21.15.122:0/4184150316 shutdown_connections 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563407a6ca0 0x5563407a9090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 --2- 172.21.15.122:0/4184150316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55634079d7a0 0x55634079db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- 172.21.15.122:0/4184150316 >> 172.21.15.122:0/4184150316 conn(0x556340797ed0 msgr2=0x5563407982b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- 172.21.15.122:0/4184150316 shutdown_connections 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- 172.21.15.122:0/4184150316 wait complete. 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 -- start start 2024-04-02T13:47:47.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.928+0000 7f65f5826740 1 Processor -- start 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079d7a0 0x5563407a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5563407a9a90 con 0x5563407a6ca0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5563406bce80 con 0x55634079d7a0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5563407a9c10 con 0x55634079e0c0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f5826740 10 client.0 _ll_register_callbacks cb 0x5563406f8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f08fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079d7a0 0x5563407a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39832/0 (socket says 172.21.15.122:39832) 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 -- 172.21.15.122:0/3460044558 learned_addr learned my addr 172.21.15.122:0/3460044558 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 -- 172.21.15.122:0/3460044558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 msgr2=0x5563407a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 -- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079d7a0 msgr2=0x5563407a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079d7a0 0x5563407a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 -- 172.21.15.122:0/3460044558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563404df7d0 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce180 tx=0x7f65e40058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73084737fcd7fc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.929+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e401f070 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f65e4099040 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e406eaf0 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f65e40a0cd0 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 10 client.19287 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563407b5bc0 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e406eaf0 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f65e400f040 con 0x5563407a6ca0 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 10 client.19287 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 10 client.19287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 20 client.19287 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563407b6f10 0x5563407b9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 4 client.19287 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55634070ecf0 con 0x5563407b6f10 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.930+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563407b6f10 0x5563407b9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.931+0000 7f65cbfff640 10 client.19287 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.931+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563407b6f10 0x5563407b9300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba5a324385cdde5a server_cookie=620f7d9445beb59c in_seq=0 out_seq=0 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65cbfff640 1 -- 172.21.15.122:0/3460044558 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55634070ecf0 con 0x5563407b6f10 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65cbfff640 10 client.19287 handle_command_reply: tid=0 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 client.19287 shutdown 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 msgr2=0x5563407a45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f65e40ce180 tx=0x7f65e40058c0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 shutdown_connections 2024-04-02T13:47:47.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407a45e0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079e0c0 0x5563407a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563407b6f10 0x5563407b9300 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 --2- 172.21.15.122:0/3460044558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079d7a0 0x5563407a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65cbfff640 0 client.19287 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 >> 172.21.15.122:0/3460044558 conn(0x556340797ed0 msgr2=0x556340799c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 shutdown_connections 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 1 -- 172.21.15.122:0/3460044558 wait complete. 2024-04-02T13:47:47.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.932+0000 7f65f5826740 20 client.19287 trim_cache size 0 max 16384 2024-04-02T13:47:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 Processor -- start 2024-04-02T13:47:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 -- start start 2024-04-02T13:47:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1af110 con 0x5563407a6ca0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1b0c50 con 0x55634079e0c0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.946+0000 7f65f5826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1af7e0 con 0x55634079d7a0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39836/0 (socket says 172.21.15.122:39836) 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f08fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 -- 172.21.15.122:0/3843878409 learned_addr learned my addr 172.21.15.122:0/3843878409 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65e3fff640 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 -- 172.21.15.122:0/3843878409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 msgr2=0x5563407b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 -- 172.21.15.122:0/3843878409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 msgr2=0x5563407a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 -- 172.21.15.122:0/3843878409 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55634079cac0 con 0x5563407a6ca0 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f08fc640 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65f10fd640 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f65e400efd0 tx=0x7f65e40054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d923a02a7f457b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65d27fc640 1 -- 172.21.15.122:0/3843878409 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e401f070 con 0x5563407a6ca0 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65d27fc640 1 -- 172.21.15.122:0/3843878409 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e40a5590 con 0x5563407a6ca0 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.947+0000 7f65d27fc640 1 -- 172.21.15.122:0/3843878409 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4098aa0 con 0x5563407a6ca0 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- 172.21.15.122:0/3843878409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 msgr2=0x5563407be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f65e400efd0 tx=0x7f65e40054d0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- 172.21.15.122:0/3843878409 shutdown_connections 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407be8a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 --2- 172.21.15.122:0/3843878409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- 172.21.15.122:0/3843878409 >> 172.21.15.122:0/3843878409 conn(0x556340797ed0 msgr2=0x556340799aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- 172.21.15.122:0/3843878409 shutdown_connections 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- 172.21.15.122:0/3843878409 wait complete. 2024-04-02T13:47:47.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 -- start start 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.948+0000 7f65f5826740 1 Processor -- start 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407aef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407aa300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1b0580 con 0x5563407a6ca0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1afeb0 con 0x55634079e0c0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1ad110 con 0x55634079d7a0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f5826740 10 client.0 _ll_register_callbacks cb 0x7f65dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65e3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39848/0 (socket says 172.21.15.122:39848) 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 -- 172.21.15.122:0/905678591 learned_addr learned my addr 172.21.15.122:0/905678591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:47.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f08fc640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 -- 172.21.15.122:0/905678591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 msgr2=0x5563407aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 -- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 msgr2=0x5563407aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 -- 172.21.15.122:0/905678591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563407aad80 con 0x5563407a6ca0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65e3fff640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65f10fd640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f65e409ac80 tx=0x7f65e409d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392a71393e35e50f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.949+0000 7f65cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e401f070 con 0x5563407a6ca0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f65e4087280 con 0x5563407a6ca0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e40a5590 con 0x5563407a6ca0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f65e40991c0 con 0x5563407a6ca0 2024-04-02T13:47:47.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 10 client.19293 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563407aad80 con 0x5563407a6ca0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e40a5590 con 0x5563407a6ca0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f65e40a1e60 con 0x5563407a6ca0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 10 client.19293 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 10 client.19293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 20 client.19293 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563408258f0 0x556340827ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 4 client.19293 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556340709040 con 0x5563408258f0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.950+0000 7f65f08fc640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563408258f0 0x556340827ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f08fc640 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563408258f0 0x556340827ce0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fab0383fa3bcd5c8 server_cookie=830675422de2455 in_seq=0 out_seq=0 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65cbfff640 10 client.19293 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:47.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65cbfff640 1 -- 172.21.15.122:0/905678591 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556340709040 con 0x5563408258f0 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65cbfff640 10 client.19293 handle_command_reply: tid=0 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 client.19293 shutdown 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 msgr2=0x5563407aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f65e409ac80 tx=0x7f65e409d200 comp rx=0 tx=0).stop 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 shutdown_connections 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563407a6ca0 0x5563407aa840 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5563408258f0 0x556340827ce0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55634079e0c0 0x5563407aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 --2- 172.21.15.122:0/905678591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55634079d7a0 0x5563407aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 >> 172.21.15.122:0/905678591 conn(0x556340797ed0 msgr2=0x5563407966a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65cbfff640 0 client.19293 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 shutdown_connections 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 1 -- 172.21.15.122:0/905678591 wait complete. 2024-04-02T13:47:47.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.951+0000 7f65f5826740 20 client.19293 trim_cache size 0 max 16384 2024-04-02T13:47:47.962 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 msgr2=0x7f65d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 0x7f65d40a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009b50 tx=0x7f65e8005f90 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 msgr2=0x7f65ec19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f65e400eaa0 tx=0x7f65e400ef60 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 shutdown_connections 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f65d40a3d20 0x7f65d40a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ec106980 0x7f65ec19cc70 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f65ec101330 0x7f65ec1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 --2- 172.21.15.122:0/3580023822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ec100970 0x7f65ec1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 >> 172.21.15.122:0/3580023822 conn(0x7f65ec077b00 msgr2=0x7f65ec068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 shutdown_connections 2024-04-02T13:47:47.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:47.952+0000 7f65f2b60640 1 -- 172.21.15.122:0/3580023822 wait complete. 2024-04-02T13:47:47.966 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:47:47.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 527199 4.23 MB/sec execute 548 sec latency 62.797 ms 2024-04-02T13:47:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 -- 172.21.15.122:0/2130549586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194100970 msgr2=0x7f4194100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 --2- 172.21.15.122:0/2130549586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194100970 0x7f4194100df0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f4184009930 tx=0x7f418402bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 -- 172.21.15.122:0/2130549586 shutdown_connections 2024-04-02T13:47:48.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 --2- 172.21.15.122:0/2130549586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194101330 0x7f419410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 --2- 172.21.15.122:0/2130549586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194100970 0x7f4194100df0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 --2- 172.21.15.122:0/2130549586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194106980 0x7f4194106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 -- 172.21.15.122:0/2130549586 >> 172.21.15.122:0/2130549586 conn(0x7f4194077b00 msgr2=0x7f4194077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 -- 172.21.15.122:0/2130549586 shutdown_connections 2024-04-02T13:47:48.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.076+0000 7f419c21b640 1 -- 172.21.15.122:0/2130549586 wait complete. 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 Processor -- start 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 -- start start 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194100970 0x7f4194196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194106980 0x7f419419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f419410fcd0 con 0x7f4194101330 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f419410fb50 con 0x7f4194100970 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419c21b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f419410fe50 con 0x7f4194106980 2024-04-02T13:47:48.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f4199fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194100970 0x7f4194196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f41997b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f41997b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39866/0 (socket says 172.21.15.122:39866) 2024-04-02T13:47:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f41997b6640 1 -- 172.21.15.122:0/3969393156 learned_addr learned my addr 172.21.15.122:0/3969393156 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.077+0000 7f419a7b8640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194106980 0x7f419419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 -- 172.21.15.122:0/3969393156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194106980 msgr2=0x7f419419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194106980 0x7f419419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 -- 172.21.15.122:0/3969393156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194100970 msgr2=0x7f4194196170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194100970 0x7f4194196170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 -- 172.21.15.122:0/3969393156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419419b160 con 0x7f4194101330 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.078+0000 7f41997b6640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4184009900 tx=0x7f4184004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.079+0000 7f418affd640 1 -- 172.21.15.122:0/3969393156 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184043020 con 0x7f4194101330 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.079+0000 7f418affd640 1 -- 172.21.15.122:0/3969393156 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41840054e0 con 0x7f4194101330 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.079+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f419419b3f0 con 0x7f4194101330 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.079+0000 7f418affd640 1 -- 172.21.15.122:0/3969393156 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184003bb0 con 0x7f4194101330 2024-04-02T13:47:48.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.079+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941a70f0 con 0x7f4194101330 2024-04-02T13:47:48.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.082+0000 7f418affd640 1 -- 172.21.15.122:0/3969393156 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4184046020 con 0x7f4194101330 2024-04-02T13:47:48.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.083+0000 7f419ee02740 1 Processor -- start 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.083+0000 7f419ee02740 1 -- start start 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.083+0000 7f418affd640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 0x7f41800a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2756990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d276a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e1d26ba7b0 con 0x55e1d27684d0 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e1d2659570 con 0x55e1d2756f60 2024-04-02T13:47:48.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419ee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1d26b6760 con 0x55e1d27565b0 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f418affd640 1 -- 172.21.15.122:0/3969393156 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f41840f2a30 con 0x7f4194101330 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35948/0 (socket says 172.21.15.122:35948) 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 -- 172.21.15.122:0/1558245238 learned_addr learned my addr 172.21.15.122:0/1558245238 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 -- 172.21.15.122:0/1558245238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 msgr2=0x55e1d2756990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2756990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 -- 172.21.15.122:0/1558245238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 msgr2=0x55e1d276a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d276a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f419a7b8640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 0x7f41800a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.084+0000 7f41997b6640 1 -- 172.21.15.122:0/1558245238 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d23ce3f0 con 0x55e1d2756f60 2024-04-02T13:47:48.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.085+0000 7f41997b6640 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41840be680 tx=0x7f41840035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91643b27d140f23e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.086+0000 7f417ffff640 1 -- 172.21.15.122:0/1558245238 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418404ac70 con 0x55e1d2756f60 2024-04-02T13:47:48.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.086+0000 7f417ffff640 1 -- 172.21.15.122:0/1558245238 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184004c80 con 0x55e1d2756f60 2024-04-02T13:47:48.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.087+0000 7f419ee02740 1 -- 172.21.15.122:0/1558245238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 msgr2=0x55e1d2767f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.087+0000 7f419ee02740 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f41840be680 tx=0x7f41840035f0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419ee02740 1 -- 172.21.15.122:0/1558245238 shutdown_connections 2024-04-02T13:47:48.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419ee02740 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d276a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419ee02740 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2767f90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419ee02740 1 --2- 172.21.15.122:0/1558245238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2756990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419ee02740 1 -- 172.21.15.122:0/1558245238 >> 172.21.15.122:0/1558245238 conn(0x55e1d2750e10 msgr2=0x55e1d27511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.088+0000 7f419a7b8640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 0x7f41800a6250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e1d27522f0 tx=0x7f418c009c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.089+0000 7f419ee02740 1 -- 172.21.15.122:0/1558245238 shutdown_connections 2024-04-02T13:47:48.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.089+0000 7f419ee02740 1 -- 172.21.15.122:0/1558245238 wait complete. 2024-04-02T13:47:48.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.089+0000 7f419ee02740 1 -- start start 2024-04-02T13:47:48.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.089+0000 7f419ee02740 1 Processor -- start 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2767e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d27651e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e1d276b020 con 0x55e1d27684d0 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e1d2675e80 con 0x55e1d2756f60 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1d276b1a0 con 0x55e1d27565b0 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419ee02740 10 client.0 _ll_register_callbacks cb 0x55e1d26b1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39886/0 (socket says 172.21.15.122:39886) 2024-04-02T13:47:48.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 -- 172.21.15.122:0/1254633620 learned_addr learned my addr 172.21.15.122:0/1254633620 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f4199fb7640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2767e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 -- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 msgr2=0x55e1d2767e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2767e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 -- 172.21.15.122:0/1254633620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 msgr2=0x55e1d27651e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.090+0000 7f419a7b8640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d27651e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.091+0000 7f419a7b8640 1 -- 172.21.15.122:0/1254633620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d24987d0 con 0x55e1d27684d0 2024-04-02T13:47:48.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.091+0000 7f419a7b8640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f418c01dac0 tx=0x7f418c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb7b8bdc4a6bd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.091+0000 7f417d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:48.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.091+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c02a060 con 0x55e1d27684d0 2024-04-02T13:47:48.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.091+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f418c00e850 con 0x55e1d27684d0 2024-04-02T13:47:48.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.092+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4184004c80 con 0x55e1d27684d0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.092+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f418c01d320 con 0x55e1d27684d0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.092+0000 7f419ee02740 10 client.19302 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.092+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1d275a2f0 con 0x55e1d27684d0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.092+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4184004c80 con 0x55e1d27684d0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f418c024030 con 0x55e1d27684d0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 10 client.19302 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 10 client.19302 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 20 client.19302 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d275bbc0 0x55e1d275e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 4 client.19302 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.093+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1d26c7cf0 con 0x55e1d275bbc0 2024-04-02T13:47:48.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.094+0000 7f4199fb7640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d275bbc0 0x55e1d275e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.096+0000 7f4199fb7640 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d275bbc0 0x55e1d275e080 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e12e3223bb05dea server_cookie=850f2b9ab9b4fe30 in_seq=0 out_seq=0 2024-04-02T13:47:48.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.096+0000 7f417d7fa640 10 client.19302 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:48.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.099+0000 7f417d7fa640 1 -- 172.21.15.122:0/1254633620 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1d26c7cf0 con 0x55e1d275bbc0 2024-04-02T13:47:48.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.099+0000 7f417d7fa640 10 client.19302 handle_command_reply: tid=0 2024-04-02T13:47:48.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.099+0000 7f419ee02740 1 client.19302 shutdown 2024-04-02T13:47:48.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.099+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 msgr2=0x55e1d2765760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.099+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f418c01dac0 tx=0x7f418c009500 comp rx=0 tx=0).stop 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 shutdown_connections 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d275bbc0 0x55e1d275e080 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2765760 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d27651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 --2- 172.21.15.122:0/1254633620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2767e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 >> 172.21.15.122:0/1254633620 conn(0x55e1d2750e10 msgr2=0x55e1d2752c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f417d7fa640 0 client.19302 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 shutdown_connections 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.100+0000 7f419ee02740 1 -- 172.21.15.122:0/1254633620 wait complete. 2024-04-02T13:47:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.101+0000 7f419ee02740 20 client.19302 trim_cache size 0 max 16384 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 Processor -- start 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 -- start start 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d2756f60 0x55e1d275f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2767910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41941acf60 con 0x55e1d27684d0 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41941aeaa0 con 0x55e1d27565b0 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f419ee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41941ad630 con 0x55e1d2756f60 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f4199fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f4199fb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35962/0 (socket says 172.21.15.122:35962) 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.121+0000 7f4199fb7640 1 -- 172.21.15.122:0/2358680878 learned_addr learned my addr 172.21.15.122:0/2358680878 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f41997b6640 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d2756f60 0x55e1d275f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 -- 172.21.15.122:0/2358680878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d2756f60 msgr2=0x55e1d275f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d2756f60 0x55e1d275f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 -- 172.21.15.122:0/2358680878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 msgr2=0x55e1d2767910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2767910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 -- 172.21.15.122:0/2358680878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1d2766100 con 0x55e1d27565b0 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f4199fb7640 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55e1d275e8c0 tx=0x7f4190005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7010b690f242b41c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.122+0000 7f417f7fe640 1 -- 172.21.15.122:0/2358680878 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419001f020 con 0x55e1d27565b0 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.123+0000 7f417f7fe640 1 -- 172.21.15.122:0/2358680878 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419001bdc0 con 0x55e1d27565b0 2024-04-02T13:47:48.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.123+0000 7f417f7fe640 1 -- 172.21.15.122:0/2358680878 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4190003850 con 0x55e1d27565b0 2024-04-02T13:47:48.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- 172.21.15.122:0/2358680878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 msgr2=0x55e1d2773ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55e1d275e8c0 tx=0x7f4190005f90 comp rx=0 tx=0).stop 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- 172.21.15.122:0/2358680878 shutdown_connections 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2767910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d2756f60 0x55e1d275f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 --2- 172.21.15.122:0/2358680878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d27565b0 0x55e1d2773ee0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- 172.21.15.122:0/2358680878 >> 172.21.15.122:0/2358680878 conn(0x55e1d2750e10 msgr2=0x55e1d27651a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- 172.21.15.122:0/2358680878 shutdown_connections 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- 172.21.15.122:0/2358680878 wait complete. 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 -- start start 2024-04-02T13:47:48.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.124+0000 7f419ee02740 1 Processor -- start 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2770680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2769f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41941ae3d0 con 0x55e1d27684d0 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41941add00 con 0x55e1d2756f60 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41941aaf80 con 0x55e1d27565b0 2024-04-02T13:47:48.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419ee02740 10 client.0 _ll_register_callbacks cb 0x7f418c01d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f41997b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f41997b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35966/0 (socket says 172.21.15.122:35966) 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f41997b6640 1 -- 172.21.15.122:0/3241922696 learned_addr learned my addr 172.21.15.122:0/3241922696 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.125+0000 7f419a7b8640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2769f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f4199fb7640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2770680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 -- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 msgr2=0x55e1d2770680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2770680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 -- 172.21.15.122:0/3241922696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 msgr2=0x55e1d2769f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2769f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 -- 172.21.15.122:0/3241922696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1d276a630 con 0x55e1d2756f60 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f41997b6640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41840eff50 tx=0x7f41840042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f58bfb1b1bf934cf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.126+0000 7f417d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:47:48.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.128+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184005690 con 0x55e1d2756f60 2024-04-02T13:47:48.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.128+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f418402cb90 con 0x55e1d2756f60 2024-04-02T13:47:48.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.129+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f419001bdc0 con 0x55e1d2756f60 2024-04-02T13:47:48.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.129+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f41840bedf0 con 0x55e1d2756f60 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.129+0000 7f419ee02740 10 client.28304 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.129+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e1d276f450 con 0x55e1d2756f60 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41840a90b0 con 0x55e1d2756f60 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f41840c6ed0 con 0x55e1d2756f60 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 10 client.28304 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 10 client.28304 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 20 client.28304 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:48.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d276f990 0x55e1d276fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 4 client.28304 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:48.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f419001bdc0 con 0x55e1d276f990 2024-04-02T13:47:48.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.130+0000 7f4199fb7640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d276f990 0x55e1d276fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.132+0000 7f4199fb7640 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d276f990 0x55e1d276fda0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0f3b5293c2a11bb server_cookie=8da670ff1df1838d in_seq=0 out_seq=0 2024-04-02T13:47:48.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.132+0000 7f417d7fa640 10 client.28304 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:48.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.132+0000 7f417d7fa640 1 -- 172.21.15.122:0/3241922696 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f419001bdc0 con 0x55e1d276f990 2024-04-02T13:47:48.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.132+0000 7f417d7fa640 10 client.28304 handle_command_reply: tid=0 2024-04-02T13:47:48.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.132+0000 7f419ee02740 1 client.28304 shutdown 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 msgr2=0x55e1d2769a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41840eff50 tx=0x7f41840042e0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 shutdown_connections 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e1d276f990 0x55e1d276fda0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e1d27684d0 0x55e1d2769f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f417d7fa640 0 client.28304 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e1d2756f60 0x55e1d2769a30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 --2- 172.21.15.122:0/3241922696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1d27565b0 0x55e1d2770680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 >> 172.21.15.122:0/3241922696 conn(0x55e1d2750e10 msgr2=0x55e1d2752320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 shutdown_connections 2024-04-02T13:47:48.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 1 -- 172.21.15.122:0/3241922696 wait complete. 2024-04-02T13:47:48.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.133+0000 7f419ee02740 20 client.28304 trim_cache size 0 max 16384 2024-04-02T13:47:48.139 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 msgr2=0x7f41800a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 0x7f41800a6250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55e1d27522f0 tx=0x7f418c009c90 comp rx=0 tx=0).stop 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 msgr2=0x7f41941966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4184009900 tx=0x7f4184004360 comp rx=0 tx=0).stop 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 shutdown_connections 2024-04-02T13:47:48.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f41800a3d90 0x7f41800a6250 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4194106980 0x7f419419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4194101330 0x7f41941966b0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 --2- 172.21.15.122:0/3969393156 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4194100970 0x7f4194196170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 >> 172.21.15.122:0/3969393156 conn(0x7f4194077b00 msgr2=0x7f41941030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 shutdown_connections 2024-04-02T13:47:48.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.134+0000 7f419c21b640 1 -- 172.21.15.122:0/3969393156 wait complete. 2024-04-02T13:47:48.149 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:47:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.314+0000 7f29cdd9b640 1 -- 172.21.15.122:0/259353792 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8039070 con 0x7f29d010f5d0 2024-04-02T13:47:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.314+0000 7f29d4959640 1 -- 172.21.15.122:0/259353792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d010f5d0 msgr2=0x7f29d01171a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.314+0000 7f29d4959640 1 --2- 172.21.15.122:0/259353792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d010f5d0 0x7f29d01171a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009ca0 tx=0x7f29c802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 -- 172.21.15.122:0/259353792 shutdown_connections 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 --2- 172.21.15.122:0/259353792 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d010f5d0 0x7f29d01171a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 --2- 172.21.15.122:0/259353792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d010aeb0 0x7f29d010eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 --2- 172.21.15.122:0/259353792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d010a500 0x7f29d010a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 -- 172.21.15.122:0/259353792 >> 172.21.15.122:0/259353792 conn(0x7f29d00730d0 msgr2=0x7f29d00734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 -- 172.21.15.122:0/259353792 shutdown_connections 2024-04-02T13:47:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.315+0000 7f29d4959640 1 -- 172.21.15.122:0/259353792 wait complete. 2024-04-02T13:47:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.316+0000 7f29d4959640 1 Processor -- start 2024-04-02T13:47:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 -- start start 2024-04-02T13:47:48.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d010a500 0x7f29d01a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d010f5d0 0x7f29d01a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d011a6e0 con 0x7f29d010f5d0 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29d011a560 con 0x7f29d0116280 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29d4959640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29d011a860 con 0x7f29d010a500 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29ced9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29ced9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35998/0 (socket says 172.21.15.122:35998) 2024-04-02T13:47:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.317+0000 7f29ced9d640 1 -- 172.21.15.122:0/70935045 learned_addr learned my addr 172.21.15.122:0/70935045 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.318+0000 7f29ced9d640 1 -- 172.21.15.122:0/70935045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d010a500 msgr2=0x7f29d01a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.318+0000 7f29ced9d640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d010a500 0x7f29d01a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.318+0000 7f29ced9d640 1 -- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d010f5d0 msgr2=0x7f29d01a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.318+0000 7f29ced9d640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d010f5d0 0x7f29d01a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.318+0000 7f29ced9d640 1 -- 172.21.15.122:0/70935045 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d0116fa0 con 0x7f29d0116280 2024-04-02T13:47:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29ced9d640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29c8004af0 tx=0x7f29c80344c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29c77fe640 1 -- 172.21.15.122:0/70935045 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8039050 con 0x7f29d0116280 2024-04-02T13:47:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29c77fe640 1 -- 172.21.15.122:0/70935045 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c8045070 con 0x7f29d0116280 2024-04-02T13:47:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29c77fe640 1 -- 172.21.15.122:0/70935045 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c8004510 con 0x7f29d0116280 2024-04-02T13:47:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d01a3240 con 0x7f29d0116280 2024-04-02T13:47:48.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.319+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01a3680 con 0x7f29d0116280 2024-04-02T13:47:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.323+0000 7f29c77fe640 1 -- 172.21.15.122:0/70935045 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f29c80046b0 con 0x7f29d0116280 2024-04-02T13:47:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.323+0000 7f29d7621740 1 Processor -- start 2024-04-02T13:47:48.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 -- start start 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f967ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f9686090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b4f95de7b0 con 0x55b4f967a7a0 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b4f957d570 con 0x55b4f9683ca0 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29d7621740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b4f95da760 con 0x55b4f967b0c0 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29c77fe640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 0x7f29bc0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.324+0000 7f29ce59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57204/0 (socket says 172.21.15.122:57204) 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 -- 172.21.15.122:0/1094521491 learned_addr learned my addr 172.21.15.122:0/1094521491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29c77fe640 1 -- 172.21.15.122:0/70935045 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f29c80f3010 con 0x7f29d0116280 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 -- 172.21.15.122:0/1094521491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 msgr2=0x55b4f9686090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f9686090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 -- 172.21.15.122:0/1094521491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 msgr2=0x55b4f967ab80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ced9d640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 0x7f29bc0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f967ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.325+0000 7f29ce59c640 1 -- 172.21.15.122:0/1094521491 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f92f23f0 con 0x55b4f967b0c0 2024-04-02T13:47:48.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.326+0000 7f29ce59c640 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009c40 tx=0x7f29c000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1255924e169a8901 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.326+0000 7f29c4ff9640 1 -- 172.21.15.122:0/1094521491 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0005020 con 0x55b4f967b0c0 2024-04-02T13:47:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.326+0000 7f29ced9d640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 0x7f29bc0a62e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f29c80c73b0 tx=0x7f29c8004040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.326+0000 7f29c4ff9640 1 -- 172.21.15.122:0/1094521491 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c00047a0 con 0x55b4f967b0c0 2024-04-02T13:47:48.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.326+0000 7f29c4ff9640 1 -- 172.21.15.122:0/1094521491 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0010720 con 0x55b4f967b0c0 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.327+0000 7f29d7621740 1 -- 172.21.15.122:0/1094521491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 msgr2=0x55b4f9683760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.327+0000 7f29d7621740 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009c40 tx=0x7f29c000e720 comp rx=0 tx=0).stop 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 -- 172.21.15.122:0/1094521491 shutdown_connections 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f9686090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967b0c0 0x55b4f9683760 secure :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009c40 tx=0x7f29c000e720 comp rx=0 tx=0).stop 2024-04-02T13:47:48.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 --2- 172.21.15.122:0/1094521491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f967ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 -- 172.21.15.122:0/1094521491 >> 172.21.15.122:0/1094521491 conn(0x55b4f9674ed0 msgr2=0x55b4f96752b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 -- 172.21.15.122:0/1094521491 shutdown_connections 2024-04-02T13:47:48.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.328+0000 7f29d7621740 1 -- 172.21.15.122:0/1094521491 wait complete. 2024-04-02T13:47:48.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.329+0000 7f29d7621740 1 -- start start 2024-04-02T13:47:48.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.329+0000 7f29d7621740 1 Processor -- start 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.329+0000 7f29d7621740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f96810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9681620 0x55b4f968ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b4f9686a90 con 0x55b4f967a7a0 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b4f9599e80 con 0x55b4f9683ca0 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b4f9686c10 con 0x55b4f9681620 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39932/0 (socket says 172.21.15.122:39932) 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29c7fff640 1 -- 172.21.15.122:0/2744416784 learned_addr learned my addr 172.21.15.122:0/2744416784 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.330+0000 7f29d7621740 10 client.0 _ll_register_callbacks cb 0x55b4f95d5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f96810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9681620 0x55b4f968ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 -- 172.21.15.122:0/2744416784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9681620 msgr2=0x55b4f968ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9681620 0x55b4f968ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 -- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 msgr2=0x55b4f96810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f96810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.331+0000 7f29c7fff640 1 -- 172.21.15.122:0/2744416784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f93bc7d0 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f29b8012800 tx=0x7f29b80097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb163792f30bf71 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8012e40 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f29b8012110 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29c00047a0 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f29b8023050 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29d7621740 10 client.19311 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b4f96930e0 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.332+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29c00047a0 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f29b8003680 con 0x55b4f967a7a0 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 10 client.19311 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:48.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 20 client.19311 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:48.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96945a0 0x55b4f96969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 4 client.19311 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:48.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4f95ebcf0 con 0x55b4f96945a0 2024-04-02T13:47:48.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.333+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96945a0 0x55b4f96969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.334+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96945a0 0x55b4f96969f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b94bd9a8e051311c server_cookie=c65254c7f53235bb in_seq=0 out_seq=0 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.334+0000 7f29b5ffb640 10 client.19311 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2744416784 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4f95ebcf0 con 0x55b4f96945a0 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29b5ffb640 10 client.19311 handle_command_reply: tid=0 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 client.19311 shutdown 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 msgr2=0x55b4f9680ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f29b8012800 tx=0x7f29b80097c0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 shutdown_connections 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96945a0 0x55b4f96969f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9681620 0x55b4f968ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9683ca0 0x55b4f96810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 --2- 172.21.15.122:0/2744416784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9680ba0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 >> 172.21.15.122:0/2744416784 conn(0x55b4f9674ed0 msgr2=0x55b4f9676c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29b5ffb640 0 client.19311 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 shutdown_connections 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 1 -- 172.21.15.122:0/2744416784 wait complete. 2024-04-02T13:47:48.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.335+0000 7f29d7621740 20 client.19311 trim_cache size 0 max 16384 2024-04-02T13:47:48.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 Processor -- start 2024-04-02T13:47:48.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 -- start start 2024-04-02T13:47:48.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967a7a0 0x55b4f96701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f967fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c6770 con 0x55b4f9683ca0 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c82b0 con 0x55b4f9681620 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29d7621740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c6e40 con 0x55b4f967a7a0 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39944/0 (socket says 172.21.15.122:39944) 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29ce59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f967fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 -- 172.21.15.122:0/1266799056 learned_addr learned my addr 172.21.15.122:0/1266799056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967a7a0 0x55b4f96701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 -- 172.21.15.122:0/1266799056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967a7a0 msgr2=0x55b4f96701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967a7a0 0x55b4f96701f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 -- 172.21.15.122:0/1266799056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 msgr2=0x55b4f967fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.355+0000 7f29c7fff640 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f967fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.356+0000 7f29c7fff640 1 -- 172.21.15.122:0/1266799056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4f967aeb0 con 0x55b4f9683ca0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.356+0000 7f29c7fff640 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f29b80059f0 tx=0x7f29b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5808f4b987d30b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.356+0000 7f29b7fff640 1 -- 172.21.15.122:0/1266799056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8003cc0 con 0x55b4f9683ca0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.356+0000 7f29b7fff640 1 -- 172.21.15.122:0/1266799056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b8004270 con 0x55b4f9683ca0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.356+0000 7f29b7fff640 1 -- 172.21.15.122:0/1266799056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8012110 con 0x55b4f9683ca0 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- 172.21.15.122:0/1266799056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 msgr2=0x55b4f969c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f29b80059f0 tx=0x7f29b80046c0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- 172.21.15.122:0/1266799056 shutdown_connections 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f9683ca0 0x55b4f969c2c0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f967fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 --2- 172.21.15.122:0/1266799056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f967a7a0 0x55b4f96701f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- 172.21.15.122:0/1266799056 >> 172.21.15.122:0/1266799056 conn(0x55b4f9674ed0 msgr2=0x55b4f967f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- 172.21.15.122:0/1266799056 shutdown_connections 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- 172.21.15.122:0/1266799056 wait complete. 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 -- start start 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.357+0000 7f29d7621740 1 Processor -- start 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f9687580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9683ca0 0x55b4f96f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c7be0 con 0x55b4f967a7a0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c7510 con 0x55b4f9681620 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29d01c4770 con 0x55b4f9683ca0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39946/0 (socket says 172.21.15.122:39946) 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 10 client.0 _ll_register_callbacks cb 0x7f29b8009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 -- 172.21.15.122:0/2763316322 learned_addr learned my addr 172.21.15.122:0/2763316322 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9683ca0 0x55b4f96f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f9687580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 -- 172.21.15.122:0/2763316322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9683ca0 msgr2=0x55b4f96f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9683ca0 0x55b4f96f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 -- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 msgr2=0x55b4f9687580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f9687580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 -- 172.21.15.122:0/2763316322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4f96f90e0 con 0x55b4f967a7a0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29c7fff640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009990 tx=0x7f29b8004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760e3f7f6a9e5ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8003800 con 0x55b4f967a7a0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f29b8012110 con 0x55b4f967a7a0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.358+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29b8004270 con 0x55b4f967a7a0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f29b8012310 con 0x55b4f967a7a0 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 10 client.19317 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:48.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b4f96fa3c0 con 0x55b4f967a7a0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b8004270 con 0x55b4f967a7a0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f29b8034060 con 0x55b4f967a7a0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 10 client.19317 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 10 client.19317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 20 client.19317 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96faa20 0x55b4f96fce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 4 client.19317 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4f95e6040 con 0x55b4f96faa20 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.359+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96faa20 0x55b4f96fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29b5ffb640 10 client.19317 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29ce59c640 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96faa20 0x55b4f96fce10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f777b2efcc4ec server_cookie=c128f2ddca20e326 in_seq=0 out_seq=0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29b5ffb640 1 -- 172.21.15.122:0/2763316322 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4f95e6040 con 0x55b4f96faa20 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29b5ffb640 10 client.19317 handle_command_reply: tid=0 2024-04-02T13:47:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 client.19317 shutdown 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 msgr2=0x55b4f9687040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f29b8009990 tx=0x7f29b8004000 comp rx=0 tx=0).stop 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 shutdown_connections 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4f96faa20 0x55b4f96fce10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4f9683ca0 0x55b4f96f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29b5ffb640 0 client.19317 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4f9681620 0x55b4f9687580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 --2- 172.21.15.122:0/2763316322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4f967a7a0 0x55b4f9687040 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 >> 172.21.15.122:0/2763316322 conn(0x55b4f9674ed0 msgr2=0x55b4f967fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 shutdown_connections 2024-04-02T13:47:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 1 -- 172.21.15.122:0/2763316322 wait complete. 2024-04-02T13:47:48.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.360+0000 7f29d7621740 20 client.19317 trim_cache size 0 max 16384 2024-04-02T13:47:48.377 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 msgr2=0x7f29bc0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 0x7f29bc0a62e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f29c80c73b0 tx=0x7f29c8004040 comp rx=0 tx=0).stop 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 msgr2=0x7f29d01166e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29c8004af0 tx=0x7f29c80344c0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 shutdown_connections 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29bc0a3e20 0x7f29bc0a62e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29d0116280 0x7f29d01166e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f29d010f5d0 0x7f29d01a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 --2- 172.21.15.122:0/70935045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d010a500 0x7f29d01a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 >> 172.21.15.122:0/70935045 conn(0x7f29d00730d0 msgr2=0x7f29d006e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 shutdown_connections 2024-04-02T13:47:48.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:48.361+0000 7f29d4959640 1 -- 172.21.15.122:0/70935045 wait complete. 2024-04-02T13:47:48.379 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:48 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2967081854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:48.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:48 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2967081854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:48.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:48 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2967081854' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:47:48.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 529395 4.25 MB/sec execute 549 sec latency 21.397 ms 2024-04-02T13:47:49.381 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:47:49.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:47:49.383 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:47:49.407 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:49 smithi122 ceph-mon[32114]: pgmap v394: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:49.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:49 smithi187 ceph-mon[36064]: pgmap v394: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 -- 172.21.15.122:0/2980297041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c1069a0 msgr2=0x7fd79c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/2980297041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c1069a0 0x7fd79c106d80 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009930 tx=0x7fd78c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 -- 172.21.15.122:0/2980297041 shutdown_connections 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/2980297041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/2980297041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c100990 0x7fd79c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/2980297041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c1069a0 0x7fd79c106d80 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 -- 172.21.15.122:0/2980297041 >> 172.21.15.122:0/2980297041 conn(0x7fd79c0777b0 msgr2=0x7fd79c0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 -- 172.21.15.122:0/2980297041 shutdown_connections 2024-04-02T13:47:49.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.492+0000 7fd7a29e9640 1 -- 172.21.15.122:0/2980297041 wait complete. 2024-04-02T13:47:49.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 Processor -- start 2024-04-02T13:47:49.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 -- start start 2024-04-02T13:47:49.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 0x7fd79c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c110db0 con 0x7fd79c100990 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c110c30 con 0x7fd79c101350 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a29e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c110f30 con 0x7fd79c1069a0 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36250/0 (socket says 172.21.15.122:36250) 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd7a0f86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 0x7fd79c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.493+0000 7fd79bfff640 1 -- 172.21.15.122:0/742290462 learned_addr learned my addr 172.21.15.122:0/742290462 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 -- 172.21.15.122:0/742290462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 msgr2=0x7fd79c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 0x7fd79c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 -- 172.21.15.122:0/742290462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 msgr2=0x7fd79c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 -- 172.21.15.122:0/742290462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c19d3e0 con 0x7fd79c100990 2024-04-02T13:47:49.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 0x7fd79c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd79bfff640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009230 tx=0x7fd78c02cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7997fa640 1 -- 172.21.15.122:0/742290462 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c041070 con 0x7fd79c100990 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c19d670 con 0x7fd79c100990 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7997fa640 1 -- 172.21.15.122:0/742290462 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004450 con 0x7fd79c100990 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7997fa640 1 -- 172.21.15.122:0/742290462 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c003e50 con 0x7fd79c100990 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.494+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1a4f60 con 0x7fd79c100990 2024-04-02T13:47:49.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.496+0000 7fd7997fa640 1 -- 172.21.15.122:0/742290462 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd78c044070 con 0x7fd79c100990 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 Processor -- start 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 -- start start 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7997fa640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 0x7fd7840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a9804be0 0x55d6a980d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a980fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6a9768770 con 0x55d6a98042c0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d6a9710960 con 0x55d6a980d7c0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d6a97648c0 con 0x55d6a9804be0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 0x7fd7840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7997fa640 1 -- 172.21.15.122:0/742290462 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fd78c0f2380 con 0x7fd79c100990 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36258/0 (socket says 172.21.15.122:36258) 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd7a0f86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a980fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 -- 172.21.15.122:0/2934661217 learned_addr learned my addr 172.21.15.122:0/2934661217 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a9804be0 0x55d6a980d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 -- 172.21.15.122:0/2934661217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a9804be0 msgr2=0x55d6a980d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a9804be0 0x55d6a980d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 -- 172.21.15.122:0/2934661217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 msgr2=0x55d6a980fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a980fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.497+0000 7fd79bfff640 1 -- 172.21.15.122:0/2934661217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a95447e0 con 0x55d6a98042c0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 0x7fd7840a6380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd7940099e0 tx=0x7fd794009ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd79bfff640 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0be080 tx=0x7fd78c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba43bd5081abc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7827fc640 1 -- 172.21.15.122:0/2934661217 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c04b070 con 0x55d6a98042c0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7827fc640 1 -- 172.21.15.122:0/2934661217 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004c80 con 0x55d6a98042c0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7827fc640 1 -- 172.21.15.122:0/2934661217 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0f20d0 con 0x55d6a98042c0 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7a55d4740 1 -- 172.21.15.122:0/2934661217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 msgr2=0x55d6a98046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.498+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0be080 tx=0x7fd78c009260 comp rx=0 tx=0).stop 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- 172.21.15.122:0/2934661217 shutdown_connections 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a980fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a9804be0 0x55d6a980d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/2934661217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a98042c0 0x55d6a98046a0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- 172.21.15.122:0/2934661217 >> 172.21.15.122:0/2934661217 conn(0x55d6a97fe9f0 msgr2=0x55d6a97fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- 172.21.15.122:0/2934661217 shutdown_connections 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- 172.21.15.122:0/2934661217 wait complete. 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- start start 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 Processor -- start 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a98042c0 0x55d6a980a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a980d7c0 0x55d6a980b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d6a9810370 con 0x55d6a9804be0 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d6a9723d20 con 0x55d6a98042c0 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d6a98104f0 con 0x55d6a980d7c0 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd7a55d4740 10 client.0 _ll_register_callbacks cb 0x55d6a96f0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.499+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36266/0 (socket says 172.21.15.122:36266) 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 -- 172.21.15.122:0/772472948 learned_addr learned my addr 172.21.15.122:0/772472948 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a980d7c0 0x55d6a980b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79bfff640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a98042c0 0x55d6a980a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 -- 172.21.15.122:0/772472948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a980d7c0 msgr2=0x55d6a980b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a980d7c0 0x55d6a980b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 -- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a98042c0 msgr2=0x55d6a980a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a98042c0 0x55d6a980a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 -- 172.21.15.122:0/772472948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a95a4f70 con 0x55d6a9804be0 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd790012d20 tx=0x7fd79000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f68ec9da570bf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:49.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790012110 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fd790004050 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd78c004c80 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd7900122b0 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd7a55d4740 10 client.19329 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6a943c170 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.500+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd78c004c80 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fd790010430 con 0x55d6a9804be0 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 10 client.19329 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 20 client.19329 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a981d990 0x55d6a981fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 4 client.19329 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:49.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6a9775cb0 con 0x55d6a981d990 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd79bfff640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a981d990 0x55d6a981fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd77b7fe640 10 client.19329 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.501+0000 7fd79bfff640 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a981d990 0x55d6a981fde0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=227a1684f0d0f676 server_cookie=e0c373847c9fa8d7 in_seq=0 out_seq=0 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd77b7fe640 1 -- 172.21.15.122:0/772472948 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6a9775cb0 con 0x55d6a981d990 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd77b7fe640 10 client.19329 handle_command_reply: tid=0 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 client.19329 shutdown 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 msgr2=0x55d6a980ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fd790012d20 tx=0x7fd79000f790 comp rx=0 tx=0).stop 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 shutdown_connections 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a981d990 0x55d6a981fde0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a980d7c0 0x55d6a980b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a980ade0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd77b7fe640 0 client.19329 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/772472948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a98042c0 0x55d6a980a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 >> 172.21.15.122:0/772472948 conn(0x55d6a97fe9f0 msgr2=0x55d6a9800740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 shutdown_connections 2024-04-02T13:47:49.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 1 -- 172.21.15.122:0/772472948 wait complete. 2024-04-02T13:47:49.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.502+0000 7fd7a55d4740 20 client.19329 trim_cache size 0 max 16384 2024-04-02T13:47:49.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:49 smithi188 ceph-mon[36117]: pgmap v394: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 Processor -- start 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 -- start start 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a980cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1af130 con 0x55d6a980d7c0 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1b0c70 con 0x55d6a9804be0 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1af800 con 0x55d6a98042c0 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a0f86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a0f86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36270/0 (socket says 172.21.15.122:36270) 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:34750/0 (socket says 172.21.15.122:34750) 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd7a0f86640 1 -- 172.21.15.122:0/3159485092 learned_addr learned my addr 172.21.15.122:0/3159485092 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.521+0000 7fd79bfff640 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a980cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3159485092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 msgr2=0x55d6a980cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a980cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3159485092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 msgr2=0x55d6a98256a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3159485092 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6a9809850 con 0x55d6a9804be0 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7900108d0 tx=0x7fd79000fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b62ab2c881d70f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd781ffb640 1 -- 172.21.15.122:0/3159485092 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790010ac0 con 0x55d6a9804be0 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd781ffb640 1 -- 172.21.15.122:0/3159485092 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79002e070 con 0x55d6a9804be0 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.522+0000 7fd781ffb640 1 -- 172.21.15.122:0/3159485092 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790003870 con 0x55d6a9804be0 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3159485092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 msgr2=0x55d6a9819150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd7900108d0 tx=0x7fd79000fc90 comp rx=0 tx=0).stop 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3159485092 shutdown_connections 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a980d7c0 0x55d6a98256a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a9804be0 0x55d6a9819150 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3159485092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a980cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3159485092 >> 172.21.15.122:0/3159485092 conn(0x55d6a97fe9f0 msgr2=0x55d6a97fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3159485092 shutdown_connections 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3159485092 wait complete. 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 -- start start 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.523+0000 7fd7a55d4740 1 Processor -- start 2024-04-02T13:47:49.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a9815aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a9812380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1b05a0 con 0x55d6a9804be0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1afed0 con 0x55d6a980d7c0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1ad130 con 0x55d6a98042c0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 10 client.0 _ll_register_callbacks cb 0x7fd790009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36284/0 (socket says 172.21.15.122:36284) 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3482846841 learned_addr learned my addr 172.21.15.122:0/3482846841 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a0f86640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a9812380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79bfff640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a9815aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3482846841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 msgr2=0x55d6a9815aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a9815aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 msgr2=0x55d6a9812380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a9812380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 -- 172.21.15.122:0/3482846841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6a9812bb0 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd79b7fe640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd790000c00 tx=0x7fd7900128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6a188a2d3dca89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd77b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790036070 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7fd7900052d0 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79002e070 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd790042050 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 10 client.19335 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.524+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d6a9884ff0 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79002e070 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fd790029480 con 0x55d6a9804be0 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 10 client.19335 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:49.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 20 client.19335 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a98855f0 0x55d6a98879e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 4 client.19335 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d6a9770000 con 0x55d6a98855f0 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd79bfff640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a98855f0 0x55d6a98879e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd79bfff640 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a98855f0 0x55d6a98879e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413a6f11c1bb5910 server_cookie=e86cef216ddf04bb in_seq=0 out_seq=0 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.525+0000 7fd77b7fe640 10 client.19335 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.526+0000 7fd77b7fe640 1 -- 172.21.15.122:0/3482846841 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d6a9770000 con 0x55d6a98855f0 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.526+0000 7fd77b7fe640 10 client.19335 handle_command_reply: tid=0 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.526+0000 7fd7a55d4740 1 client.19335 shutdown 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 msgr2=0x55d6a9811e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd790000c00 tx=0x7fd7900128a0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 shutdown_connections 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d6a98855f0 0x55d6a98879e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d6a980d7c0 0x55d6a9812380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d6a9804be0 0x55d6a9811e20 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd77b7fe640 0 client.19335 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 --2- 172.21.15.122:0/3482846841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d6a98042c0 0x55d6a9815aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 >> 172.21.15.122:0/3482846841 conn(0x55d6a97fe9f0 msgr2=0x55d6a97ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 shutdown_connections 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 1 -- 172.21.15.122:0/3482846841 wait complete. 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a55d4740 20 client.19335 trim_cache size 0 max 16384 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "b68aa0bf-79dd-492a-8402-37dcbbf2a162", 2024-04-02T13:47:49.538 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:47:49.539 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:49.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 msgr2=0x7fd7840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 0x7fd7840a6380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd7940099e0 tx=0x7fd794009ef0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 msgr2=0x7fd79c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.527+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd78c009230 tx=0x7fd78c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 shutdown_connections 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7840a3ec0 0x7fd7840a6380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd79c1069a0 0x7fd79c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd79c101350 0x7fd79c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 --2- 172.21.15.122:0/742290462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd79c100990 0x7fd79c198440 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 >> 172.21.15.122:0/742290462 conn(0x7fd79c0777b0 msgr2=0x7fd79c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 shutdown_connections 2024-04-02T13:47:49.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.528+0000 7fd7a29e9640 1 -- 172.21.15.122:0/742290462 wait complete. 2024-04-02T13:47:49.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:47:49.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.697+0000 7f06631e9640 1 -- 172.21.15.122:0/818100839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c107a90 msgr2=0x7f065c107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.697+0000 7f06631e9640 1 --2- 172.21.15.122:0/818100839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c107a90 0x7f065c107e70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0650009930 tx=0x7f065002bc30 comp rx=0 tx=0).stop 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 -- 172.21.15.122:0/818100839 shutdown_connections 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- 172.21.15.122:0/818100839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c102440 0x7f065c10e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- 172.21.15.122:0/818100839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c101a80 0x7f065c101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- 172.21.15.122:0/818100839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c107a90 0x7f065c107e70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 -- 172.21.15.122:0/818100839 >> 172.21.15.122:0/818100839 conn(0x7f065c0fd590 msgr2=0x7f065c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 -- 172.21.15.122:0/818100839 shutdown_connections 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 -- 172.21.15.122:0/818100839 wait complete. 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 Processor -- start 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 -- start start 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c102440 0x7f065c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.698+0000 7f06631e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c107a90 0x7f065c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f06631e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f065c111e70 con 0x7f065c102440 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f06631e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f065c111cf0 con 0x7f065c101a80 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f06631e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f065c111ff0 con 0x7f065c107a90 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0661786640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c107a90 0x7f065c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:34778/0 (socket says 172.21.15.122:34778) 2024-04-02T13:47:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 -- 172.21.15.122:0/2663253626 learned_addr learned my addr 172.21.15.122:0/2663253626 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f065bfff640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c102440 0x7f065c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 -- 172.21.15.122:0/2663253626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c107a90 msgr2=0x7f065c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c107a90 0x7f065c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 -- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c102440 msgr2=0x7f065c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c102440 0x7f065c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 -- 172.21.15.122:0/2663253626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19d400 con 0x7f065c101a80 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.699+0000 7f0660f85640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0650009230 tx=0x7f065002cba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.700+0000 7f0659ffb640 1 -- 172.21.15.122:0/2663253626 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650041070 con 0x7f065c101a80 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.700+0000 7f0659ffb640 1 -- 172.21.15.122:0/2663253626 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650004450 con 0x7f065c101a80 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.700+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19d690 con 0x7f065c101a80 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.700+0000 7f0659ffb640 1 -- 172.21.15.122:0/2663253626 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650034440 con 0x7f065c101a80 2024-04-02T13:47:49.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.700+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1a4f80 con 0x7f065c101a80 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.702+0000 7f0665dd3740 1 Processor -- start 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.702+0000 7f0665dd3740 1 -- start start 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.702+0000 7f0659ffb640 1 -- 172.21.15.122:0/2663253626 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0650044070 con 0x7f065c101a80 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.702+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0665dd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56130773e830 con 0x5613077db1f0 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0665dd3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5613076e6820 con 0x5613077e3dd0 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0665dd3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56130773a7e0 con 0x5613077da8d0 2024-04-02T13:47:49.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0661786640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0661786640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54782/0 (socket says 172.21.15.122:54782) 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0661786640 1 -- 172.21.15.122:0/2870441 learned_addr learned my addr 172.21.15.122:0/2870441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f065bfff640 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 -- 172.21.15.122:0/2870441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 msgr2=0x5613077dacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 -- 172.21.15.122:0/2870441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 msgr2=0x5613077e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 -- 172.21.15.122:0/2870441 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130751c7d0 con 0x5613077db1f0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0661786640 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0659ffb640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 0x7f064c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0660f85640 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077e3890 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f06500ef9a0 tx=0x7f06500f8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a967bd9b0e1be89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0661786640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 0x7f064c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0659ffb640 1 -- 172.21.15.122:0/2663253626 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f06500f2530 con 0x7f065c101a80 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.703+0000 7f0646ffd640 1 -- 172.21.15.122:0/2870441 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500ba490 con 0x5613077db1f0 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0646ffd640 1 -- 172.21.15.122:0/2870441 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650004c80 con 0x5613077db1f0 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0646ffd640 1 -- 172.21.15.122:0/2870441 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06500ba490 con 0x5613077db1f0 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 -- 172.21.15.122:0/2870441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 msgr2=0x5613077e3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0661786640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 0x7f064c0a61e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0654009fd0 tx=0x7f0654009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077e3890 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f06500ef9a0 tx=0x7f06500f8640 comp rx=0 tx=0).stop 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 -- 172.21.15.122:0/2870441 shutdown_connections 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077e3890 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2870441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 -- 172.21.15.122:0/2870441 >> 172.21.15.122:0/2870441 conn(0x5613077d5000 msgr2=0x5613077d53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 -- 172.21.15.122:0/2870441 shutdown_connections 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.704+0000 7f0665dd3740 1 -- 172.21.15.122:0/2870441 wait complete. 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 -- start start 2024-04-02T13:47:49.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 Processor -- start 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077e3dd0 0x5613077e0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613077e6980 con 0x5613077da8d0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5613076f9d30 con 0x5613077db1f0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5613077e6b00 con 0x5613077e3dd0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0665dd3740 10 client.0 _ll_register_callbacks cb 0x5613076f9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0660f85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0660f85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36314/0 (socket says 172.21.15.122:36314) 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0660f85640 1 -- 172.21.15.122:0/1543225187 learned_addr learned my addr 172.21.15.122:0/1543225187 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f065bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.705+0000 7f0661786640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077e3dd0 0x5613077e0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 -- 172.21.15.122:0/1543225187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077e3dd0 msgr2=0x5613077e0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077e3dd0 0x5613077e0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 -- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 msgr2=0x5613077e09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 -- 172.21.15.122:0/1543225187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613074523f0 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f065bfff640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0660f85640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f06500efa50 tx=0x7f0650009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9c346287f69124 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065004b070 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f06500035f0 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0650004c80 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f06500bdb50 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0665dd3740 10 client.19347 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5613074120f0 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650004c80 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.706+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f06500c2160 con 0x5613077da8d0 2024-04-02T13:47:49.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 10 client.19347 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 10 client.19347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 20 client.19347 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x5613077f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 4 client.19347 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130774bd70 con 0x5613077f38d0 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f065bfff640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x5613077f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f063bfff640 10 client.19347 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.707+0000 7f065bfff640 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x5613077f5cc0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fbd8636e877b594 server_cookie=449178c4c2dac33b in_seq=0 out_seq=0 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f063bfff640 1 -- 172.21.15.122:0/1543225187 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56130774bd70 con 0x5613077f38d0 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f063bfff640 10 client.19347 handle_command_reply: tid=0 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 client.19347 shutdown 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 msgr2=0x5613077e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f06500efa50 tx=0x7f0650009260 comp rx=0 tx=0).stop 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 shutdown_connections 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x5613077f5cc0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f063bfff640 0 client.19347 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077e3dd0 0x5613077e0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 --2- 172.21.15.122:0/1543225187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077da8d0 0x5613077e3630 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 >> 172.21.15.122:0/1543225187 conn(0x5613077d5000 msgr2=0x5613077d6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.708+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 shutdown_connections 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.709+0000 7f0665dd3740 1 -- 172.21.15.122:0/1543225187 wait complete. 2024-04-02T13:47:49.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.709+0000 7f0665dd3740 20 client.19347 trim_cache size 0 max 16384 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 Processor -- start 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 -- start start 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1af1c0 con 0x5613077db1f0 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1b0d00 con 0x5613077e3dd0 2024-04-02T13:47:49.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0665dd3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1af890 con 0x5613077da8d0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f065bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0660f85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f065bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36320/0 (socket says 172.21.15.122:36320) 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0660f85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54802/0 (socket says 172.21.15.122:54802) 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f065bfff640 1 -- 172.21.15.122:0/710745925 learned_addr learned my addr 172.21.15.122:0/710745925 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.729+0000 7f0661786640 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 -- 172.21.15.122:0/710745925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 msgr2=0x5613077dbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 -- 172.21.15.122:0/710745925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 msgr2=0x5613077e30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 -- 172.21.15.122:0/710745925 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613077d4e30 con 0x5613077db1f0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f0660f85640 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f065bfff640 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ffd20 tx=0x7f06480138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16af0a6624065 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f06467fc640 1 -- 172.21.15.122:0/710745925 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648004020 con 0x5613077db1f0 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f06467fc640 1 -- 172.21.15.122:0/710745925 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f064801d910 con 0x5613077db1f0 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.730+0000 7f06467fc640 1 -- 172.21.15.122:0/710745925 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648003c20 con 0x5613077db1f0 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- 172.21.15.122:0/710745925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 msgr2=0x5613077d40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f065c0ffd20 tx=0x7f06480138f0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- 172.21.15.122:0/710745925 shutdown_connections 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077e3dd0 0x5613077e30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077db1f0 0x5613077d40c0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 --2- 172.21.15.122:0/710745925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077dbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- 172.21.15.122:0/710745925 >> 172.21.15.122:0/710745925 conn(0x5613077d5000 msgr2=0x5613077d39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- 172.21.15.122:0/710745925 shutdown_connections 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- 172.21.15.122:0/710745925 wait complete. 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 -- start start 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.731+0000 7f0665dd3740 1 Processor -- start 2024-04-02T13:47:49.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1b0630 con 0x5613077e3dd0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1aff60 con 0x5613077db1f0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1ad1c0 con 0x5613077da8d0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0665dd3740 10 client.0 _ll_register_callbacks cb 0x7f06500f8400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f065bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36322/0 (socket says 172.21.15.122:36322) 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 -- 172.21.15.122:0/2511056243 learned_addr learned my addr 172.21.15.122:0/2511056243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f065bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:34814/0 (socket says 172.21.15.122:34814) 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0660f85640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 -- 172.21.15.122:0/2511056243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 msgr2=0x5613077e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 -- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 msgr2=0x5613077e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 -- 172.21.15.122:0/2511056243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561307858f60 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f0661786640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f065400ff20 tx=0x7f0654004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdb1997f5709a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.732+0000 7f063bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065402d070 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9525+0+0 (secure 0 0 0) 0x7f0654003c40 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f064801d910 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0654036050 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 10 client.19353 fetch_fsmap learned FSMap version 43 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56130785a290 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f064801d910 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f0654030020 con 0x5613077e3dd0 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 10 client.19353 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:47:49.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 10 client.19353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 20 client.19353 populate_metadata read hostname 'smithi122' 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x56130785cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 4 client.19353 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613077460c0 con 0x5613077f38d0 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.733+0000 7f0660f85640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x56130785cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f063bfff640 10 client.19353 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0660f85640 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x56130785cad0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=287554ea3c5ef670 server_cookie=d629bd6343fe156e in_seq=0 out_seq=0 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f063bfff640 1 -- 172.21.15.122:0/2511056243 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5613077460c0 con 0x5613077f38d0 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f063bfff640 10 client.19353 handle_command_reply: tid=0 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 client.19353 shutdown 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 msgr2=0x5613077e8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f065400ff20 tx=0x7f0654004ab0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 shutdown_connections 2024-04-02T13:47:49.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5613077f38d0 0x56130785cad0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613077e3dd0 0x5613077e8320 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613077db1f0 0x5613077e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 --2- 172.21.15.122:0/2511056243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613077da8d0 0x5613077e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f063bfff640 0 client.19353 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.734+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 >> 172.21.15.122:0/2511056243 conn(0x5613077d5000 msgr2=0x5613077dbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 shutdown_connections 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f0665dd3740 1 -- 172.21.15.122:0/2511056243 wait complete. 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f0665dd3740 20 client.19353 trim_cache size 0 max 16384 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:47:49.743 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: "b68aa0bf-79dd-492a-8402-37dcbbf2a162": { 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "b68aa0bf-79dd-492a-8402-37dcbbf2a162", 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:47:49.744 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 msgr2=0x7f064c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 0x7f064c0a61e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0654009fd0 tx=0x7f0654009500 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 msgr2=0x7f065c198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0650009230 tx=0x7f065002cba0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 shutdown_connections 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f064c0a3d20 0x7f064c0a61e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f065c107a90 0x7f065c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f065c102440 0x7f065c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 --2- 172.21.15.122:0/2663253626 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f065c101a80 0x7f065c198410 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:47:49.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 >> 172.21.15.122:0/2663253626 conn(0x7f065c0fd590 msgr2=0x7f065c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:47:49.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 shutdown_connections 2024-04-02T13:47:49.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:47:49.735+0000 7f06631e9640 1 -- 172.21.15.122:0/2663253626 wait complete. 2024-04-02T13:47:49.750 INFO:tasks.cephfs.filesystem:scrub status for tag:b68aa0bf-79dd-492a-8402-37dcbbf2a162 - {'path': '/', 'tag': 'b68aa0bf-79dd-492a-8402-37dcbbf2a162', 'options': 'recursive,force'} 2024-04-02T13:47:49.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 530493 4.25 MB/sec execute 550 sec latency 23.407 ms 2024-04-02T13:47:50.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:50.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:50.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:50.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:50.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:50.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:50.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:50.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:50.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:50.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:50.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 532827 4.27 MB/sec execute 551 sec latency 15.334 ms 2024-04-02T13:47:51.505 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:47:51.506 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.506 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:51.506 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:51.506 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.507 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:47:51.507 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.507 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.507 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:51 smithi187 ceph-mon[36064]: pgmap v395: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:51 smithi188 ceph-mon[36117]: pgmap v395: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:51.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:47:51.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:51.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:51.574 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:47:51.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:47:51.575 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:51 smithi122 ceph-mon[32114]: pgmap v395: 129 pgs: 129 active+clean; 500 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 203 op/s 2024-04-02T13:47:51.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 534013 4.27 MB/sec execute 552 sec latency 14.134 ms 2024-04-02T13:47:52.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 536083 4.28 MB/sec execute 553 sec latency 35.176 ms 2024-04-02T13:47:53.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:53 smithi187 ceph-mon[36064]: pgmap v396: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2024-04-02T13:47:53.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:53 smithi188 ceph-mon[36117]: pgmap v396: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2024-04-02T13:47:53.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:53 smithi122 ceph-mon[32114]: pgmap v396: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.2 MiB/s wr, 320 op/s 2024-04-02T13:47:53.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 537221 4.28 MB/sec execute 554 sec latency 22.588 ms 2024-04-02T13:47:54.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 539207 4.30 MB/sec execute 555 sec latency 29.134 ms 2024-04-02T13:47:55.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:55 smithi187 ceph-mon[36064]: pgmap v397: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-02T13:47:55.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:55 smithi188 ceph-mon[36117]: pgmap v397: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-02T13:47:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:55 smithi122 ceph-mon[32114]: pgmap v397: 129 pgs: 129 active+clean; 512 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 226 op/s 2024-04-02T13:47:55.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 540712 4.30 MB/sec execute 556 sec latency 14.897 ms 2024-04-02T13:47:56.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 542576 4.31 MB/sec execute 557 sec latency 19.868 ms 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: pgmap v398: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 341 op/s 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: pgmap v398: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 341 op/s 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:47:57.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: pgmap v398: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 341 op/s 2024-04-02T13:47:57.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:57.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:57.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:57.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:57.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:47:57.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 544082 4.32 MB/sec execute 558 sec latency 22.023 ms 2024-04-02T13:47:58.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 545856 4.33 MB/sec execute 559 sec latency 27.772 ms 2024-04-02T13:47:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:47:59 smithi187 ceph-mon[36064]: pgmap v399: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:47:59.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:47:59 smithi188 ceph-mon[36117]: pgmap v399: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:47:59.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:47:59 smithi122 ceph-mon[32114]: pgmap v399: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:47:59.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 547469 4.34 MB/sec execute 560 sec latency 13.241 ms 2024-04-02T13:48:00.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 548679 4.34 MB/sec execute 561 sec latency 168.951 ms 2024-04-02T13:48:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:01 smithi187 ceph-mon[36064]: pgmap v400: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:48:01.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:01 smithi188 ceph-mon[36117]: pgmap v400: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:48:01.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:01 smithi122 ceph-mon[32114]: pgmap v400: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 231 op/s 2024-04-02T13:48:01.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 549354 4.34 MB/sec execute 562 sec latency 620.676 ms 2024-04-02T13:48:02.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 550425 4.34 MB/sec execute 563 sec latency 67.122 ms 2024-04-02T13:48:03.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:03 smithi187 ceph-mon[36064]: pgmap v401: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-04-02T13:48:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:03 smithi188 ceph-mon[36117]: pgmap v401: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-04-02T13:48:03.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:03 smithi122 ceph-mon[32114]: pgmap v401: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 348 op/s 2024-04-02T13:48:03.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 551439 4.34 MB/sec execute 564 sec latency 41.820 ms 2024-04-02T13:48:04.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 553487 4.36 MB/sec execute 565 sec latency 19.163 ms 2024-04-02T13:48:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:05 smithi187 ceph-mon[36064]: pgmap v402: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-04-02T13:48:05.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:05 smithi188 ceph-mon[36117]: pgmap v402: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-04-02T13:48:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:05 smithi122 ceph-mon[32114]: pgmap v402: 129 pgs: 129 active+clean; 532 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 231 op/s 2024-04-02T13:48:05.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 554746 4.36 MB/sec execute 566 sec latency 11.645 ms 2024-04-02T13:48:06.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 556563 4.37 MB/sec execute 567 sec latency 17.522 ms 2024-04-02T13:48:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:07 smithi187 ceph-mon[36064]: pgmap v403: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 356 op/s 2024-04-02T13:48:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:07 smithi188 ceph-mon[36117]: pgmap v403: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 356 op/s 2024-04-02T13:48:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:07 smithi122 ceph-mon[32114]: pgmap v403: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 356 op/s 2024-04-02T13:48:07.980 INFO:tasks.workunit.client.0.smithi122.stdout: 1 558088 4.38 MB/sec execute 568 sec latency 58.995 ms 2024-04-02T13:48:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:08 smithi187 ceph-mon[36064]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:48:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:08 smithi188 ceph-mon[36117]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:48:08.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:08 smithi122 ceph-mon[32114]: osdmap e108: 12 total, 12 up, 12 in 2024-04-02T13:48:08.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 559264 4.38 MB/sec execute 569 sec latency 59.058 ms 2024-04-02T13:48:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:09 smithi187 ceph-mon[36064]: pgmap v405: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:09 smithi188 ceph-mon[36117]: pgmap v405: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:09 smithi122 ceph-mon[32114]: pgmap v405: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:09.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 561388 4.40 MB/sec execute 570 sec latency 19.286 ms 2024-04-02T13:48:10.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 562512 4.39 MB/sec execute 571 sec latency 20.586 ms 2024-04-02T13:48:11.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:11 smithi187 ceph-mon[36064]: pgmap v406: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:11 smithi188 ceph-mon[36117]: pgmap v406: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:11 smithi122 ceph-mon[32114]: pgmap v406: 129 pgs: 129 active+clean; 541 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.9 MiB/s wr, 290 op/s 2024-04-02T13:48:11.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 564628 4.41 MB/sec execute 572 sec latency 21.269 ms 2024-04-02T13:48:12.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 565763 4.41 MB/sec execute 573 sec latency 73.048 ms 2024-04-02T13:48:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:13 smithi187 ceph-mon[36064]: pgmap v407: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:13 smithi188 ceph-mon[36117]: pgmap v407: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:13 smithi122 ceph-mon[32114]: pgmap v407: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:13.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 567573 4.42 MB/sec execute 574 sec latency 26.411 ms 2024-04-02T13:48:14.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 568953 4.43 MB/sec execute 575 sec latency 20.270 ms 2024-04-02T13:48:15.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:15 smithi187 ceph-mon[36064]: pgmap v408: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:15.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:15 smithi188 ceph-mon[36117]: pgmap v408: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:15 smithi122 ceph-mon[32114]: pgmap v408: 129 pgs: 129 active+clean; 551 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.0 MiB/s wr, 289 op/s 2024-04-02T13:48:15.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 570799 4.43 MB/sec execute 576 sec latency 20.034 ms 2024-04-02T13:48:16.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 572330 4.44 MB/sec execute 577 sec latency 28.268 ms 2024-04-02T13:48:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:17 smithi187 ceph-mon[36064]: pgmap v409: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:17 smithi187 ceph-mon[36064]: osdmap e109: 12 total, 12 up, 12 in 2024-04-02T13:48:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:17 smithi188 ceph-mon[36117]: pgmap v409: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:17 smithi188 ceph-mon[36117]: osdmap e109: 12 total, 12 up, 12 in 2024-04-02T13:48:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:17 smithi122 ceph-mon[32114]: pgmap v409: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:17 smithi122 ceph-mon[32114]: osdmap e109: 12 total, 12 up, 12 in 2024-04-02T13:48:17.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 574088 4.45 MB/sec execute 578 sec latency 15.807 ms 2024-04-02T13:48:18.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 575723 4.46 MB/sec execute 579 sec latency 59.387 ms 2024-04-02T13:48:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:19 smithi187 ceph-mon[36064]: pgmap v411: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:19.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:19.751 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:19.766 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:19 smithi188 ceph-mon[36117]: pgmap v411: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:19.775 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:19 smithi122 ceph-mon[32114]: pgmap v411: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:19.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 -- 172.21.15.122:0/3277867452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 msgr2=0x7f391010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 --2- 172.21.15.122:0/3277867452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f391010d840 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f3908009cd0 tx=0x7f390802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 -- 172.21.15.122:0/3277867452 shutdown_connections 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 --2- 172.21.15.122:0/3277867452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f391010d840 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 --2- 172.21.15.122:0/3277867452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 0x7f3910100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 --2- 172.21.15.122:0/3277867452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f3910106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 -- 172.21.15.122:0/3277867452 >> 172.21.15.122:0/3277867452 conn(0x7f3910077b00 msgr2=0x7f3910077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 -- 172.21.15.122:0/3277867452 shutdown_connections 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.858+0000 7f3918055640 1 -- 172.21.15.122:0/3277867452 wait complete. 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 Processor -- start 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 -- start start 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 0x7f3910199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f39101a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110d90 con 0x7f3910101330 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110c10 con 0x7f3910106980 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3918055640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3910110f10 con 0x7f3910100970 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f39155f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f39165f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f39101a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f3915df1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 0x7f3910199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f39155f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52838/0 (socket says 172.21.15.122:52838) 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f39155f0640 1 -- 172.21.15.122:0/2458582285 learned_addr learned my addr 172.21.15.122:0/2458582285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.859+0000 7f39165f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f39101a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41200/0 (socket says 172.21.15.122:41200) 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 -- 172.21.15.122:0/2458582285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 msgr2=0x7f3910199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 0x7f3910199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 -- 172.21.15.122:0/2458582285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 msgr2=0x7f39101a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f39101a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 -- 172.21.15.122:0/2458582285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39101a4750 con 0x7f3910101330 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f39155f0640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f390000c970 tx=0x7f390000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f3906ffd640 1 -- 172.21.15.122:0/2458582285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900007bb0 con 0x7f3910101330 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39101a4920 con 0x7f3910101330 2024-04-02T13:48:19.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f3906ffd640 1 -- 172.21.15.122:0/2458582285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3900007d50 con 0x7f3910101330 2024-04-02T13:48:19.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f3906ffd640 1 -- 172.21.15.122:0/2458582285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390000f450 con 0x7f3910101330 2024-04-02T13:48:19.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.860+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39101a4e70 con 0x7f3910101330 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.862+0000 7f3906ffd640 1 -- 172.21.15.122:0/2458582285 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3900005020 con 0x7f3910101330 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.862+0000 7f391ac41740 1 Processor -- start 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 -- start start 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3906ffd640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 0x7f38fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c79c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d3c6f7830 con 0x556d3c7938d0 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d3c69f820 con 0x556d3c79cdd0 2024-04-02T13:48:19.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f391ac41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d3c6f37e0 con 0x556d3c7941f0 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f39165f2640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 0x7f38fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3906ffd640 1 -- 172.21.15.122:0/2458582285 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f39000d0950 con 0x7f3910101330 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52854/0 (socket says 172.21.15.122:52854) 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 -- 172.21.15.122:0/1895563837 learned_addr learned my addr 172.21.15.122:0/1895563837 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f39165f2640 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f39155f0640 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c79c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 -- 172.21.15.122:0/1895563837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 msgr2=0x556d3c79c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c79c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 -- 172.21.15.122:0/1895563837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 msgr2=0x556d3c79f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.863+0000 7f3915df1640 1 -- 172.21.15.122:0/1895563837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3c4d57d0 con 0x556d3c7938d0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.864+0000 7f39165f2640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 0x7f38fc0a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f390802c630 tx=0x7f3908032340 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.864+0000 7f3915df1640 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f390c009c70 tx=0x7f390c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebf489bc57438e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.864+0000 7f38fbfff640 1 -- 172.21.15.122:0/1895563837 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c004230 con 0x556d3c7938d0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.864+0000 7f38fbfff640 1 -- 172.21.15.122:0/1895563837 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c0043d0 con 0x556d3c7938d0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.864+0000 7f38fbfff640 1 -- 172.21.15.122:0/1895563837 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c019720 con 0x556d3c7938d0 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- 172.21.15.122:0/1895563837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 msgr2=0x556d3c793cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f390c009c70 tx=0x7f390c011810 comp rx=0 tx=0).stop 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- 172.21.15.122:0/1895563837 shutdown_connections 2024-04-02T13:48:19.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c79c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 --2- 172.21.15.122:0/1895563837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c793cb0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- 172.21.15.122:0/1895563837 >> 172.21.15.122:0/1895563837 conn(0x556d3c78e000 msgr2=0x556d3c78e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- 172.21.15.122:0/1895563837 shutdown_connections 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- 172.21.15.122:0/1895563837 wait complete. 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 -- start start 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.865+0000 7f391ac41740 1 Processor -- start 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c799960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c799ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d3c79fbc0 con 0x556d3c7938d0 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d3c6b2d30 con 0x556d3c79cdd0 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d3c79fd40 con 0x556d3c7941f0 2024-04-02T13:48:19.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 10 client.0 _ll_register_callbacks cb 0x556d3c6b2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52856/0 (socket says 172.21.15.122:52856) 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 -- 172.21.15.122:0/3142968200 learned_addr learned my addr 172.21.15.122:0/3142968200 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f39165f2640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c799ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f39155f0640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c799960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 -- 172.21.15.122:0/3142968200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 msgr2=0x556d3c799960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c799960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 -- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 msgr2=0x556d3c799ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c799ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 -- 172.21.15.122:0/3142968200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3c40b3f0 con 0x556d3c7938d0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f3915df1640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f390c005890 tx=0x7f390c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ba3234778732f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f38f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:19.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c027070 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f390c003800 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.866+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c0043d0 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f390c033050 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 10 client.19365 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556d3c3cb0f0 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c0043d0 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f390c018070 con 0x556d3c7938d0 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 10 client.19365 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 10 client.19365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 20 client.19365 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c7acb90 0x556d3c7aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 4 client.19365 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d3c704d70 con 0x556d3c7acb90 2024-04-02T13:48:19.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.867+0000 7f39155f0640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c7acb90 0x556d3c7aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f39155f0640 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c7acb90 0x556d3c7aef80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe2da17ed2e075b server_cookie=c29e283730512a4 in_seq=0 out_seq=0 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f38f97fa640 10 client.19365 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f38f97fa640 1 -- 172.21.15.122:0/3142968200 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d3c704d70 con 0x556d3c7acb90 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f38f97fa640 10 client.19365 handle_command_reply: tid=0 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 client.19365 shutdown 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 msgr2=0x556d3c79c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f390c005890 tx=0x7f390c005750 comp rx=0 tx=0).stop 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 shutdown_connections 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c7acb90 0x556d3c7aef80 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c799ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c799960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 --2- 172.21.15.122:0/3142968200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c79c5d0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f38f97fa640 0 client.19365 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 >> 172.21.15.122:0/3142968200 conn(0x556d3c78e000 msgr2=0x556d3c78fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.868+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 shutdown_connections 2024-04-02T13:48:19.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.869+0000 7f391ac41740 1 -- 172.21.15.122:0/3142968200 wait complete. 2024-04-02T13:48:19.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.869+0000 7f391ac41740 20 client.19365 trim_cache size 0 max 16384 2024-04-02T13:48:19.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 Processor -- start 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 -- start start 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c794710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39101af0b0 con 0x556d3c7938d0 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39101b0bf0 con 0x556d3c79cdd0 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f391ac41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39101af780 con 0x556d3c7941f0 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52862/0 (socket says 172.21.15.122:52862) 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 -- 172.21.15.122:0/2358119781 learned_addr learned my addr 172.21.15.122:0/2358119781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f39155f0640 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c794710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f39165f2640 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 -- 172.21.15.122:0/2358119781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 msgr2=0x556d3c794710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c794710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.887+0000 7f3915df1640 1 -- 172.21.15.122:0/2358119781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 msgr2=0x556d3c79c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f3915df1640 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f3915df1640 1 -- 172.21.15.122:0/2358119781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3c794020 con 0x556d3c7938d0 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f3915df1640 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f390c011d10 tx=0x7f390c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ebd3e3a39b61a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f38fb7fe640 1 -- 172.21.15.122:0/2358119781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c015070 con 0x556d3c7938d0 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f38fb7fe640 1 -- 172.21.15.122:0/2358119781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c003ce0 con 0x556d3c7938d0 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.888+0000 7f38fb7fe640 1 -- 172.21.15.122:0/2358119781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c027070 con 0x556d3c7938d0 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- 172.21.15.122:0/2358119781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 msgr2=0x556d3c7a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f390c011d10 tx=0x7f390c005f90 comp rx=0 tx=0).stop 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- 172.21.15.122:0/2358119781 shutdown_connections 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c79cdd0 0x556d3c79c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c7941f0 0x556d3c794710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- 172.21.15.122:0/2358119781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7938d0 0x556d3c7a8a80 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- 172.21.15.122:0/2358119781 >> 172.21.15.122:0/2358119781 conn(0x556d3c78e000 msgr2=0x556d3c78d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- 172.21.15.122:0/2358119781 shutdown_connections 2024-04-02T13:48:19.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- 172.21.15.122:0/2358119781 wait complete. 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- start start 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 Processor -- start 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c7938d0 0x556d3c796590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c79cdd0 0x556d3c797010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39101b0520 con 0x556d3c7941f0 2024-04-02T13:48:19.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39101afe50 con 0x556d3c7938d0 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39101ad0b0 con 0x556d3c79cdd0 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.889+0000 7f391ac41740 10 client.0 _ll_register_callbacks cb 0x7f390c004290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52864/0 (socket says 172.21.15.122:52864) 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 -- 172.21.15.122:0/2831482282 learned_addr learned my addr 172.21.15.122:0/2831482282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39165f2640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c79cdd0 0x556d3c797010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f3915df1640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c7938d0 0x556d3c796590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 -- 172.21.15.122:0/2831482282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c79cdd0 msgr2=0x556d3c797010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c79cdd0 0x556d3c797010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 -- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c7938d0 msgr2=0x556d3c796590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c7938d0 0x556d3c796590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 -- 172.21.15.122:0/2831482282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3c812020 con 0x556d3c7941f0 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f39155f0640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3900005740 tx=0x7f3900080040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2044b60afd04a2ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f38f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:19.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390001d040 con 0x556d3c7941f0 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f390009a8b0 con 0x556d3c7941f0 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c003ce0 con 0x556d3c7941f0 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f39000a08b0 con 0x556d3c7941f0 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f391ac41740 10 client.19371 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556d3c813270 con 0x556d3c7941f0 2024-04-02T13:48:19.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.890+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c003ce0 con 0x556d3c7941f0 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f390007b3d0 con 0x556d3c7941f0 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 10 client.19371 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 10 client.19371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 20 client.19371 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c8139c0 0x556d3c815de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:19.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 4 client.19371 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d3c6ff0c0 con 0x556d3c8139c0 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f3915df1640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c8139c0 0x556d3c815de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f38f97fa640 10 client.19371 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.891+0000 7f3915df1640 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c8139c0 0x556d3c815de0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f219f5e3446d4ff server_cookie=293e90146fd0d108 in_seq=0 out_seq=0 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f38f97fa640 1 -- 172.21.15.122:0/2831482282 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556d3c6ff0c0 con 0x556d3c8139c0 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f38f97fa640 10 client.19371 handle_command_reply: tid=0 2024-04-02T13:48:19.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 client.19371 shutdown 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 msgr2=0x556d3c796ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3900005740 tx=0x7f3900080040 comp rx=0 tx=0).stop 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 shutdown_connections 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556d3c8139c0 0x556d3c815de0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d3c79cdd0 0x556d3c797010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f38f97fa640 0 client.19371 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d3c7941f0 0x556d3c796ad0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 --2- 172.21.15.122:0/2831482282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d3c7938d0 0x556d3c796590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 >> 172.21.15.122:0/2831482282 conn(0x556d3c78e000 msgr2=0x556d3c798100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.892+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 shutdown_connections 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f391ac41740 1 -- 172.21.15.122:0/2831482282 wait complete. 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f391ac41740 20 client.19371 trim_cache size 0 max 16384 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:48:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:48:19.910 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 msgr2=0x7f38fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 0x7f38fc0a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f390802c630 tx=0x7f3908032340 comp rx=0 tx=0).stop 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 msgr2=0x7f39101996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f390000c970 tx=0x7f390000ce30 comp rx=0 tx=0).stop 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 shutdown_connections 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f38fc0a3d20 0x7f38fc0a61e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3910106980 0x7f39101a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3910101330 0x7f39101996d0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 --2- 172.21.15.122:0/2458582285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3910100970 0x7f3910199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:19.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 >> 172.21.15.122:0/2458582285 conn(0x7f3910077b00 msgr2=0x7f3910103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:19.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 shutdown_connections 2024-04-02T13:48:19.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:19.893+0000 7f3918055640 1 -- 172.21.15.122:0/2458582285 wait complete. 2024-04-02T13:48:19.912 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:48:19.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:48:19.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 576957 4.46 MB/sec execute 580 sec latency 24.360 ms 2024-04-02T13:48:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 -- 172.21.15.122:0/1638484712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52641013c0 msgr2=0x7f526410d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 --2- 172.21.15.122:0/1638484712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52641013c0 0x7f526410d8d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f525c009cd0 tx=0x7f525c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 -- 172.21.15.122:0/1638484712 shutdown_connections 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 --2- 172.21.15.122:0/1638484712 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52641013c0 0x7f526410d8d0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 --2- 172.21.15.122:0/1638484712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 0x7f5264100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 --2- 172.21.15.122:0/1638484712 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5264106a10 0x7f5264106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 -- 172.21.15.122:0/1638484712 >> 172.21.15.122:0/1638484712 conn(0x7f5264077b00 msgr2=0x7f5264077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 -- 172.21.15.122:0/1638484712 shutdown_connections 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.057+0000 7f526b538640 1 -- 172.21.15.122:0/1638484712 wait complete. 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 Processor -- start 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 -- start start 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 0x7f52641972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 0x7f5264197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 0x7f526419bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5264110e20 con 0x7f5264106a10 2024-04-02T13:48:20.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5264110ca0 con 0x7f52641013c0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f526b538640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5264110fa0 con 0x7f5264100a00 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5269ad5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 0x7f526419bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f52692d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 0x7f52641972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 0x7f5264197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5269ad5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 0x7f526419bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52880/0 (socket says 172.21.15.122:52880) 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5269ad5640 1 -- 172.21.15.122:0/357218893 learned_addr learned my addr 172.21.15.122:0/357218893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 -- 172.21.15.122:0/357218893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 msgr2=0x7f52641972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 0x7f52641972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 -- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 msgr2=0x7f526419bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 0x7f526419bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.058+0000 7f5268ad3640 1 -- 172.21.15.122:0/357218893 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526419c260 con 0x7f52641013c0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f5268ad3640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 0x7f5264197800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f525400d6d0 tx=0x7f525400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254017070 con 0x7f52641013c0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526419c550 con 0x7f52641013c0 2024-04-02T13:48:20.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5254009820 con 0x7f52641013c0 2024-04-02T13:48:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5254004cd0 con 0x7f52641013c0 2024-04-02T13:48:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.059+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52641a3de0 con 0x7f52641013c0 2024-04-02T13:48:20.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.060+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52640688f0 con 0x7f52641013c0 2024-04-02T13:48:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.061+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f52540099d0 con 0x7f52641013c0 2024-04-02T13:48:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.061+0000 7f525a7fc640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 0x7f52500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.062+0000 7f52692d4640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 0x7f52500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.062+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f52540d08c0 con 0x7f52641013c0 2024-04-02T13:48:20.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.062+0000 7f52692d4640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 0x7f52500a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5260009b50 tx=0x7f5260005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.066+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f525409c520 con 0x7f52641013c0 2024-04-02T13:48:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.323+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5264108ff0 con 0x7f52641013c0 2024-04-02T13:48:20.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.323+0000 7f525a7fc640 1 -- 172.21.15.122:0/357218893 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11946 (secure 0 0 0) 0x7f52540a0820 con 0x7f52641013c0 2024-04-02T13:48:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:48:20.327 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:47:03.399575+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:48:20.327 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 43 2024-04-02T13:48:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.325+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 msgr2=0x7f52500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.325+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 0x7f52500a61e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5260009b50 tx=0x7f5260005f90 comp rx=0 tx=0).stop 2024-04-02T13:48:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.325+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 msgr2=0x7f5264197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.325+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 0x7f5264197800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f525400d6d0 tx=0x7f525400db90 comp rx=0 tx=0).stop 2024-04-02T13:48:20.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 shutdown_connections 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52500a3d20 0x7f52500a61e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5264106a10 0x7f526419bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52641013c0 0x7f5264197800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 --2- 172.21.15.122:0/357218893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5264100a00 0x7f52641972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 >> 172.21.15.122:0/357218893 conn(0x7f5264077b00 msgr2=0x7f5264103200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 shutdown_connections 2024-04-02T13:48:20.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.326+0000 7f526b538640 1 -- 172.21.15.122:0/357218893 wait complete. 2024-04-02T13:48:20.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:48:20.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 -- 172.21.15.122:0/2185843381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9950100c20 msgr2=0x7f9950106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 --2- 172.21.15.122:0/2185843381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9950100c20 0x7f9950106750 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f9940009900 tx=0x7f994002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:20.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 -- 172.21.15.122:0/2185843381 shutdown_connections 2024-04-02T13:48:20.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 --2- 172.21.15.122:0/2185843381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950106e80 0x7f995010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 --2- 172.21.15.122:0/2185843381 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9950100c20 0x7f9950106750 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 --2- 172.21.15.122:0/2185843381 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950100270 0x7f9950100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 -- 172.21.15.122:0/2185843381 >> 172.21.15.122:0/2185843381 conn(0x7f9950076fc0 msgr2=0x7f99500fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 -- 172.21.15.122:0/2185843381 shutdown_connections 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.487+0000 7f995847b640 1 -- 172.21.15.122:0/2185843381 wait complete. 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 Processor -- start 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 -- start start 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9950111ee0 con 0x7f995019ccc0 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9950111d60 con 0x7f9950106e80 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f995847b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9950112060 con 0x7f9950100270 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956a18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956217640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9955a16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956217640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:35696/0 (socket says 172.21.15.122:35696) 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956a18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52892/0 (socket says 172.21.15.122:52892) 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9955a16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41278/0 (socket says 172.21.15.122:41278) 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956217640 1 -- 172.21.15.122:0/1060156016 learned_addr learned my addr 172.21.15.122:0/1060156016 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956217640 1 -- 172.21.15.122:0/1060156016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 msgr2=0x7f9950198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.488+0000 7f9956217640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9956217640 1 -- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 msgr2=0x7f995019d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9956217640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9956217640 1 -- 172.21.15.122:0/1060156016 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995019d7b0 con 0x7f9950100270 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9956a18640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9955a16640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f9956217640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f99477fe640 1 -- 172.21.15.122:0/1060156016 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00cd80 con 0x7f9950100270 2024-04-02T13:48:20.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995019daa0 con 0x7f9950100270 2024-04-02T13:48:20.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f99477fe640 1 -- 172.21.15.122:0/1060156016 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c010910 con 0x7f9950100270 2024-04-02T13:48:20.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f99477fe640 1 -- 172.21.15.122:0/1060156016 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c0188f0 con 0x7f9950100270 2024-04-02T13:48:20.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.489+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501a5250 con 0x7f9950100270 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.491+0000 7f99477fe640 1 -- 172.21.15.122:0/1060156016 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f994c010ac0 con 0x7f9950100270 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 Processor -- start 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 -- start start 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55770cbe37b0 con 0x55770cc8d0b0 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55770cb82570 con 0x55770cc7f180 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f995b068740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55770cbdf760 con 0x55770cc7fb30 2024-04-02T13:48:20.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f99477fe640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 0x7f993c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9956a18640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9955a16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9956a18640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41288/0 (socket says 172.21.15.122:41288) 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9956217640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9955a16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52900/0 (socket says 172.21.15.122:52900) 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9956a18640 1 -- 172.21.15.122:0/1497283932 learned_addr learned my addr 172.21.15.122:0/1497283932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.492+0000 7f9956a18640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 0x7f993c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f99477fe640 1 -- 172.21.15.122:0/1060156016 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f994c0d0fa0 con 0x7f9950100270 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 -- 172.21.15.122:0/1497283932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 msgr2=0x55770cc8cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc8cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 -- 172.21.15.122:0/1497283932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 msgr2=0x55770cc8f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 -- 172.21.15.122:0/1497283932 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770c8f73f0 con 0x55770cc7f180 2024-04-02T13:48:20.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9955a16640 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956217640 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc8cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 0x7f993c0a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99480099e0 tx=0x7f9948018000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9956a18640 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fd0 tx=0x7f994800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7a94272b4174f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9944bfe640 1 -- 172.21.15.122:0/1497283932 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948022070 con 0x55770cc7f180 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9944bfe640 1 -- 172.21.15.122:0/1497283932 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994800e4f0 con 0x55770cc7f180 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.493+0000 7f9944bfe640 1 -- 172.21.15.122:0/1497283932 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948011610 con 0x55770cc7f180 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- 172.21.15.122:0/1497283932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 msgr2=0x55770cc7f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9948009fd0 tx=0x7f994800fa00 comp rx=0 tx=0).stop 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- 172.21.15.122:0/1497283932 shutdown_connections 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc8d0b0 0x55770cc8f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc8cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 --2- 172.21.15.122:0/1497283932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7f180 0x55770cc7f560 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- 172.21.15.122:0/1497283932 >> 172.21.15.122:0/1497283932 conn(0x55770cc79ed0 msgr2=0x55770cc7a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- 172.21.15.122:0/1497283932 shutdown_connections 2024-04-02T13:48:20.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- 172.21.15.122:0/1497283932 wait complete. 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 -- start start 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.494+0000 7f995b068740 1 Processor -- start 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc86750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc86c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55770cc809c0 con 0x55770cc7f180 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55770cb9ee80 con 0x55770cc8d0b0 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55770cc80b40 con 0x55770cc7fb30 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 10 client.0 _ll_register_callbacks cb 0x55770cbdaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52908/0 (socket says 172.21.15.122:52908) 2024-04-02T13:48:20.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 -- 172.21.15.122:0/644788420 learned_addr learned my addr 172.21.15.122:0/644788420 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9955a16640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956a18640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc86c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 -- 172.21.15.122:0/644788420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 msgr2=0x55770cc86750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc86750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 -- 172.21.15.122:0/644788420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 msgr2=0x55770cc86c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc86c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 -- 172.21.15.122:0/644788420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770c9c17d0 con 0x55770cc7f180 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9956217640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f994c0097b0 tx=0x7f994c092090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b929706cba8511a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09cc00 con 0x55770cc7f180 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994800e4f0 con 0x55770cc7f180 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f994c099270 con 0x55770cc7f180 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f994c0a0f00 con 0x55770cc7f180 2024-04-02T13:48:20.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 10 client.19389 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.495+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55770cca0330 con 0x55770cc7f180 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994800e4f0 con 0x55770cc7f180 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f994c0a5330 con 0x55770cc7f180 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 10 client.19389 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 10 client.19389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 20 client.19389 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cca17f0 0x55770cca3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 4 client.19389 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55770cbf0cf0 con 0x55770cca17f0 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f9955a16640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cca17f0 0x55770cca3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f9939ffb640 10 client.19389 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.496+0000 7f9955a16640 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cca17f0 0x55770cca3be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc18956df39549ed server_cookie=dc808b7e048d3f17 in_seq=0 out_seq=0 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f9939ffb640 1 -- 172.21.15.122:0/644788420 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55770cbf0cf0 con 0x55770cca17f0 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f9939ffb640 10 client.19389 handle_command_reply: tid=0 2024-04-02T13:48:20.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 client.19389 shutdown 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 msgr2=0x55770cc86210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f994c0097b0 tx=0x7f994c092090 comp rx=0 tx=0).stop 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 shutdown_connections 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cca17f0 0x55770cca3be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc86c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc86750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 --2- 172.21.15.122:0/644788420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86210 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f9939ffb640 0 client.19389 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 >> 172.21.15.122:0/644788420 conn(0x55770cc79ed0 msgr2=0x55770cc7bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 shutdown_connections 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 1 -- 172.21.15.122:0/644788420 wait complete. 2024-04-02T13:48:20.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.497+0000 7f995b068740 20 client.19389 trim_cache size 0 max 16384 2024-04-02T13:48:20.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 Processor -- start 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 -- start start 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc8d0b0 0x55770cca93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99501acfd0 con 0x55770cc7f180 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99501aeb10 con 0x55770cc7fb30 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f995b068740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99501ad6a0 con 0x55770cc8d0b0 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9955a16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9955a16640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41296/0 (socket says 172.21.15.122:41296) 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9955a16640 1 -- 172.21.15.122:0/2712812799 learned_addr learned my addr 172.21.15.122:0/2712812799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956a18640 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc8d0b0 0x55770cca93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 -- 172.21.15.122:0/2712812799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc8d0b0 msgr2=0x55770cca93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc8d0b0 0x55770cca93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 -- 172.21.15.122:0/2712812799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 msgr2=0x55770cc78db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 -- 172.21.15.122:0/2712812799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770cc7f8d0 con 0x55770cc7f180 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9955a16640 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.512+0000 7f9956217640 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86ab0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f994c0a5730 tx=0x7f994c09b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59510d4b4b3cdc8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f993bfff640 1 -- 172.21.15.122:0/2712812799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c740 con 0x55770cc7f180 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f993bfff640 1 -- 172.21.15.122:0/2712812799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09cc00 con 0x55770cc7f180 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f993bfff640 1 -- 172.21.15.122:0/2712812799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c2b0 con 0x55770cc7f180 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 -- 172.21.15.122:0/2712812799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 msgr2=0x55770cc86ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86ab0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f994c0a5730 tx=0x7f994c09b110 comp rx=0 tx=0).stop 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 -- 172.21.15.122:0/2712812799 shutdown_connections 2024-04-02T13:48:20.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc8d0b0 0x55770cca93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc7fb30 0x55770cc78db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 --2- 172.21.15.122:0/2712812799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc86ab0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.513+0000 7f995b068740 1 -- 172.21.15.122:0/2712812799 >> 172.21.15.122:0/2712812799 conn(0x55770cc79ed0 msgr2=0x55770cc786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- 172.21.15.122:0/2712812799 shutdown_connections 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- 172.21.15.122:0/2712812799 wait complete. 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- start start 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 Processor -- start 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc98840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99501ae440 con 0x55770cc7f180 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f99501add70 con 0x55770cc8d0b0 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99501aafd0 con 0x55770cc7fb30 2024-04-02T13:48:20.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f995b068740 10 client.0 _ll_register_callbacks cb 0x7f994c00e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52920/0 (socket says 172.21.15.122:52920) 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 -- 172.21.15.122:0/1570983870 learned_addr learned my addr 172.21.15.122:0/1570983870 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9955a16640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc98840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956a18640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 -- 172.21.15.122:0/1570983870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 msgr2=0x55770cc98840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc98840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 -- 172.21.15.122:0/1570983870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 msgr2=0x55770cc98d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.514+0000 7f9956217640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc98d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9956217640 1 -- 172.21.15.122:0/1570983870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770cc9d500 con 0x55770cc7f180 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9956217640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ce900 tx=0x7f994c0a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d8b3494065e9f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c09c250 con 0x55770cc7f180 2024-04-02T13:48:20.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f994c010430 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c09cc00 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f994c0a57c0 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 10 client.19395 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55770cd080c0 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c09cc00 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f994c09bac0 con 0x55770cc7f180 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 10 client.19395 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 10 client.19395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 20 client.19395 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cd08590 0x55770cd0a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 4 client.19395 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55770cbeb040 con 0x55770cd08590 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.515+0000 7f9955a16640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cd08590 0x55770cd0a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f9955a16640 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cd08590 0x55770cd0a980 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b9430956db2b035f server_cookie=87f4183434f3f207 in_seq=0 out_seq=0 2024-04-02T13:48:20.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f9939ffb640 10 client.19395 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f9939ffb640 1 -- 172.21.15.122:0/1570983870 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55770cbeb040 con 0x55770cd08590 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f9939ffb640 10 client.19395 handle_command_reply: tid=0 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 client.19395 shutdown 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 msgr2=0x55770cc98300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f994c0ce900 tx=0x7f994c0a5630 comp rx=0 tx=0).stop 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 shutdown_connections 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55770cd08590 0x55770cd0a980 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55770cc8d0b0 0x55770cc98d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55770cc7fb30 0x55770cc98840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f9939ffb640 0 client.19395 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 --2- 172.21.15.122:0/1570983870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55770cc7f180 0x55770cc98300 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 >> 172.21.15.122:0/1570983870 conn(0x55770cc79ed0 msgr2=0x55770cc7baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 shutdown_connections 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 1 -- 172.21.15.122:0/1570983870 wait complete. 2024-04-02T13:48:20.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.516+0000 7f995b068740 20 client.19395 trim_cache size 0 max 16384 2024-04-02T13:48:20.527 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 msgr2=0x7f993c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 0x7f993c0a61e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f99480099e0 tx=0x7f9948018000 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 msgr2=0x7f9950198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 shutdown_connections 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f993c0a3d20 0x7f993c0a61e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f995019ccc0 0x7f995019d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9950106e80 0x7f9950198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 --2- 172.21.15.122:0/1060156016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9950100270 0x7f9950198400 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 >> 172.21.15.122:0/1060156016 conn(0x7f9950076fc0 msgr2=0x7f9950105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 shutdown_connections 2024-04-02T13:48:20.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.517+0000 7f995847b640 1 -- 172.21.15.122:0/1060156016 wait complete. 2024-04-02T13:48:20.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 -- 172.21.15.122:0/421227441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 msgr2=0x7fa610100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 --2- 172.21.15.122:0/421227441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610100df0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fa600009930 tx=0x7fa60002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 -- 172.21.15.122:0/421227441 shutdown_connections 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 --2- 172.21.15.122:0/421227441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa61010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 --2- 172.21.15.122:0/421227441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610100df0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 --2- 172.21.15.122:0/421227441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 0x7fa610106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 -- 172.21.15.122:0/421227441 >> 172.21.15.122:0/421227441 conn(0x7fa610077b00 msgr2=0x7fa610077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 -- 172.21.15.122:0/421227441 shutdown_connections 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.639+0000 7fa6158b1640 1 -- 172.21.15.122:0/421227441 wait complete. 2024-04-02T13:48:20.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 Processor -- start 2024-04-02T13:48:20.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 -- start start 2024-04-02T13:48:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa610199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 0x7fa6101a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110d90 con 0x7fa610100970 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110c10 con 0x7fa610101330 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa6158b1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110f10 con 0x7fa610106980 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52938/0 (socket says 172.21.15.122:52938) 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60effd640 1 -- 172.21.15.122:0/1113661468 learned_addr learned my addr 172.21.15.122:0/1113661468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa610199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 0x7fa6101a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.640+0000 7fa60effd640 1 -- 172.21.15.122:0/1113661468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 msgr2=0x7fa6101a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60effd640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 0x7fa6101a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60effd640 1 -- 172.21.15.122:0/1113661468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 msgr2=0x7fa610199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60effd640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa610199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60effd640 1 -- 172.21.15.122:0/1113661468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a47d0 con 0x7fa610100970 2024-04-02T13:48:20.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa610199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa60effd640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d770 tx=0x7fa5f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa6148af640 1 -- 172.21.15.122:0/1113661468 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8017070 con 0x7fa610100970 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa6148af640 1 -- 172.21.15.122:0/1113661468 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f8005500 con 0x7fa610100970 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a49a0 con 0x7fa610100970 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa6148af640 1 -- 172.21.15.122:0/1113661468 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8004e00 con 0x7fa610100970 2024-04-02T13:48:20.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.641+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a4e70 con 0x7fa610100970 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.643+0000 7fa6148af640 1 -- 172.21.15.122:0/1113661468 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa5f8004fa0 con 0x7fa610100970 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 Processor -- start 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 -- start start 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa6148af640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 0x7fa5fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56256cf427b0 con 0x56256cfe34d0 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56256cee1570 con 0x56256cfec0b0 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa618572740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56256cf3e760 con 0x56256cfe2bb0 2024-04-02T13:48:20.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 0x7fa5fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52950/0 (socket says 172.21.15.122:52950) 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60e7fc640 1 -- 172.21.15.122:0/928350192 learned_addr learned my addr 172.21.15.122:0/928350192 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:35756/0 (socket says 172.21.15.122:35756) 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa6148af640 1 -- 172.21.15.122:0/1113661468 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa5f80d0600 con 0x7fa610100970 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.644+0000 7fa60effd640 1 -- 172.21.15.122:0/928350192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 msgr2=0x56256cfee4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60effd640 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfee4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60effd640 1 -- 172.21.15.122:0/928350192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 msgr2=0x56256cfebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60effd640 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60effd640 1 -- 172.21.15.122:0/928350192 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256cc563f0 con 0x56256cfe2bb0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 0x7fa5fc0a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56256cfde470 tx=0x7fa608009b00 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa60effd640 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80cd960 tx=0x7fa5f800df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8fd208e40e140c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa605ffb640 1 -- 172.21.15.122:0/928350192 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8020070 con 0x56256cfe2bb0 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa605ffb640 1 -- 172.21.15.122:0/928350192 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f809c8b0 con 0x56256cfe2bb0 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.645+0000 7fa605ffb640 1 -- 172.21.15.122:0/928350192 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8009cc0 con 0x56256cfe2bb0 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- 172.21.15.122:0/928350192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 msgr2=0x56256cfe2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80cd960 tx=0x7fa5f800df90 comp rx=0 tx=0).stop 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- 172.21.15.122:0/928350192 shutdown_connections 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfee4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 --2- 172.21.15.122:0/928350192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cfe2f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- 172.21.15.122:0/928350192 >> 172.21.15.122:0/928350192 conn(0x56256cfdd2e0 msgr2=0x56256cfdd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- 172.21.15.122:0/928350192 shutdown_connections 2024-04-02T13:48:20.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- 172.21.15.122:0/928350192 wait complete. 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 -- start start 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.646+0000 7fa618572740 1 Processor -- start 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cfe8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfe95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56256cfeee40 con 0x56256cfe2bb0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56256cefde80 con 0x56256cfe34d0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56256cfeefc0 con 0x56256cfec0b0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa618572740 10 client.0 _ll_register_callbacks cb 0x56256cf39aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52952/0 (socket says 172.21.15.122:52952) 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 -- 172.21.15.122:0/2486688031 learned_addr learned my addr 172.21.15.122:0/2486688031 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cfe8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfe95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 -- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 msgr2=0x56256cfe95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfe95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 -- 172.21.15.122:0/2486688031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 msgr2=0x56256cfe8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cfe8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 -- 172.21.15.122:0/2486688031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256cd207d0 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfe95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa60effd640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa5f809c140 tx=0x7fa5f80a0670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ef5db61626e656 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.647+0000 7fa5eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8020070 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa5f8098820 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f809c8b0 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80a4e20 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 10 client.19407 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56256cfeee40 con 0x56256cfe2bb0 2024-04-02T13:48:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f809c8b0 con 0x56256cfe2bb0 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fa5f80d01b0 con 0x56256cfe2bb0 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 10 client.19407 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 10 client.19407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 20 client.19407 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256cffc050 0x56256cffe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 4 client.19407 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56256cf4fcf0 con 0x56256cffc050 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.648+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256cffc050 0x56256cffe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.649+0000 7fa5eeffd640 10 client.19407 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.649+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256cffc050 0x56256cffe440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c0359d767f163bc1 server_cookie=21debb77c450fd15 in_seq=0 out_seq=0 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.649+0000 7fa5eeffd640 1 -- 172.21.15.122:0/2486688031 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56256cf4fcf0 con 0x56256cffc050 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.649+0000 7fa5eeffd640 10 client.19407 handle_command_reply: tid=0 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.649+0000 7fa618572740 1 client.19407 shutdown 2024-04-02T13:48:20.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 msgr2=0x56256cfebb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa5f809c140 tx=0x7fa5f80a0670 comp rx=0 tx=0).stop 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 shutdown_connections 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfe95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cfe8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256cffc050 0x56256cffe440 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 --2- 172.21.15.122:0/2486688031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfebb40 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 >> 172.21.15.122:0/2486688031 conn(0x56256cfdd2e0 msgr2=0x56256cfdf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa5eeffd640 0 client.19407 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 shutdown_connections 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 1 -- 172.21.15.122:0/2486688031 wait complete. 2024-04-02T13:48:20.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.650+0000 7fa618572740 20 client.19407 trim_cache size 0 max 16384 2024-04-02T13:48:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 Processor -- start 2024-04-02T13:48:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 -- start start 2024-04-02T13:48:20.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cff7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfeb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101acec0 con 0x56256cfe2bb0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101aea00 con 0x56256cfe34d0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa618572740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ad590 con 0x56256cfec0b0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa60effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52962/0 (socket says 172.21.15.122:52962) 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa60effd640 1 -- 172.21.15.122:0/3978011502 learned_addr learned my addr 172.21.15.122:0/3978011502 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.665+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cff7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 -- 172.21.15.122:0/3978011502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 msgr2=0x56256cfeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfeb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 -- 172.21.15.122:0/3978011502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 msgr2=0x56256cff7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cff7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 -- 172.21.15.122:0/3978011502 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256cfe9930 con 0x56256cfe2bb0 2024-04-02T13:48:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa60effd640 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa5f809a460 tx=0x7fa5f80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ccfdf67d36e5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa6057fa640 1 -- 172.21.15.122:0/3978011502 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8020070 con 0x56256cfe2bb0 2024-04-02T13:48:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa6057fa640 1 -- 172.21.15.122:0/3978011502 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f809c8b0 con 0x56256cfe2bb0 2024-04-02T13:48:20.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.666+0000 7fa6057fa640 1 -- 172.21.15.122:0/3978011502 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80cdbe0 con 0x56256cfe2bb0 2024-04-02T13:48:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- 172.21.15.122:0/3978011502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 msgr2=0x56256cfe8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7fa5f809a460 tx=0x7fa5f80048a0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- 172.21.15.122:0/3978011502 shutdown_connections 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfec0b0 0x56256cfeb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfe34d0 0x56256cff7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- 172.21.15.122:0/3978011502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe2bb0 0x56256cfe8e70 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- 172.21.15.122:0/3978011502 >> 172.21.15.122:0/3978011502 conn(0x56256cfdd2e0 msgr2=0x56256cfe8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- 172.21.15.122:0/3978011502 shutdown_connections 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- 172.21.15.122:0/3978011502 wait complete. 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- start start 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 Processor -- start 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cff4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfefff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ae330 con 0x56256cfe34d0 2024-04-02T13:48:20.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101adc60 con 0x56256cfec0b0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.667+0000 7fa618572740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101aaec0 con 0x56256cfe2bb0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa618572740 10 client.0 _ll_register_callbacks cb 0x7fa600009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cff4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52972/0 (socket says 172.21.15.122:52972) 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 -- 172.21.15.122:0/757201711 learned_addr learned my addr 172.21.15.122:0/757201711 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60f7fe640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfefff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 -- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 msgr2=0x56256cff4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cff4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 -- 172.21.15.122:0/757201711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 msgr2=0x56256cfefff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfefff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 -- 172.21.15.122:0/757201711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256cff0820 con 0x56256cfe34d0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa60e7fc640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa600005ab0 tx=0x7fa600004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3643fec07b04e01b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa5eeffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa600004450 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa60002c880 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f809c8b0 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa600058050 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa618572740 10 client.19413 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.668+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56256d0624d0 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f809c8b0 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fa6000050c0 con 0x56256cfe34d0 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 10 client.19413 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 10 client.19413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 20 client.19413 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256d062ce0 0x56256d065100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 4 client.19413 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:20.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56256cf4a040 con 0x56256d062ce0 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa60effd640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256d062ce0 0x56256d065100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa60effd640 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256d062ce0 0x56256d065100 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d32cf15f0aa8b26 server_cookie=a87d05e6ae2b6225 in_seq=0 out_seq=0 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.669+0000 7fa5eeffd640 10 client.19413 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa5eeffd640 1 -- 172.21.15.122:0/757201711 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56256cf4a040 con 0x56256d062ce0 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa5eeffd640 10 client.19413 handle_command_reply: tid=0 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 client.19413 shutdown 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 msgr2=0x56256cfefa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa600005ab0 tx=0x7fa600004cf0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 shutdown_connections 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56256cfec0b0 0x56256cfefff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56256d062ce0 0x56256d065100 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56256cfe34d0 0x56256cfefa50 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 --2- 172.21.15.122:0/757201711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56256cfe2bb0 0x56256cff4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa5eeffd640 0 client.19413 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:20.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 >> 172.21.15.122:0/757201711 conn(0x56256cfdd2e0 msgr2=0x56256cfd76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 shutdown_connections 2024-04-02T13:48:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 1 -- 172.21.15.122:0/757201711 wait complete. 2024-04-02T13:48:20.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa618572740 20 client.19413 trim_cache size 0 max 16384 2024-04-02T13:48:20.680 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 msgr2=0x7fa5fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 0x7fa5fc0a61e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56256cfde470 tx=0x7fa608009b00 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.670+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 msgr2=0x7fa610199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800d770 tx=0x7fa5f800dc30 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 shutdown_connections 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5fc0a3d20 0x7fa5fc0a61e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa610106980 0x7fa6101a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa610101330 0x7fa610199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 --2- 172.21.15.122:0/1113661468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa610100970 0x7fa610199210 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 >> 172.21.15.122:0/1113661468 conn(0x7fa610077b00 msgr2=0x7fa6101031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 shutdown_connections 2024-04-02T13:48:20.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.671+0000 7fa6158b1640 1 -- 172.21.15.122:0/1113661468 wait complete. 2024-04-02T13:48:20.684 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:20 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/357218893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.685 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:48:20.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:20 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/357218893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:20 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/357218893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 -- 172.21.15.122:0/3160869822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 msgr2=0x7f047c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 --2- 172.21.15.122:0/3160869822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c100df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f046c009860 tx=0x7f046c02bb70 comp rx=0 tx=0).stop 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 -- 172.21.15.122:0/3160869822 shutdown_connections 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 --2- 172.21.15.122:0/3160869822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c101330 0x7f047c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 --2- 172.21.15.122:0/3160869822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c100df0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 --2- 172.21.15.122:0/3160869822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c106980 0x7f047c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.796+0000 7f0483e63640 1 -- 172.21.15.122:0/3160869822 >> 172.21.15.122:0/3160869822 conn(0x7f047c077b00 msgr2=0x7f047c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- 172.21.15.122:0/3160869822 shutdown_connections 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- 172.21.15.122:0/3160869822 wait complete. 2024-04-02T13:48:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 Processor -- start 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- start start 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c101330 0x7f047c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 0x7f047c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f047c110d90 con 0x7f047c106980 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f047c110c10 con 0x7f047c100970 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.797+0000 7f0483e63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f047c110f10 con 0x7f047c101330 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0482400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 0x7f047c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41356/0 (socket says 172.21.15.122:41356) 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 -- 172.21.15.122:0/4405275 learned_addr learned my addr 172.21.15.122:0/4405275 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f04813fe640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c101330 0x7f047c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 -- 172.21.15.122:0/4405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c101330 msgr2=0x7f047c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c101330 0x7f047c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 -- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 msgr2=0x7f047c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 0x7f047c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 -- 172.21.15.122:0/4405275 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c19d390 con 0x7f047c100970 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0482400640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 0x7f047c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0481bff640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f047800e9f0 tx=0x7f047800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0472ffd640 1 -- 172.21.15.122:0/4405275 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047800cd80 con 0x7f047c100970 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0472ffd640 1 -- 172.21.15.122:0/4405275 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0478010910 con 0x7f047c100970 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0472ffd640 1 -- 172.21.15.122:0/4405275 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478018990 con 0x7f047c100970 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047c19d680 con 0x7f047c100970 2024-04-02T13:48:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.798+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f047c1a4f10 con 0x7f047c100970 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.801+0000 7f0472ffd640 1 -- 172.21.15.122:0/4405275 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0478010ac0 con 0x7f047c100970 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.801+0000 7f0486a50740 1 Processor -- start 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.801+0000 7f0486a50740 1 -- start start 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0472ffd640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 0x7f04680a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c47a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c7c3d57b0 con 0x562c7c4720c0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562c7c374570 con 0x562c7c4717a0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0486a50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c7c3d1760 con 0x562c7c47aca0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0481bff640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 0x7f04680a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f04813fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c47a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41370/0 (socket says 172.21.15.122:41370) 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 -- 172.21.15.122:0/3166877203 learned_addr learned my addr 172.21.15.122:0/3166877203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0472ffd640 1 -- 172.21.15.122:0/4405275 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f04780d0dd0 con 0x7f047c100970 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 -- 172.21.15.122:0/3166877203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 msgr2=0x562c7c47d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 -- 172.21.15.122:0/3166877203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 msgr2=0x562c7c47a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c47a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 -- 172.21.15.122:0/3166877203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7c0e93f0 con 0x562c7c4717a0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f04813fe640 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c47a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0481bff640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 0x7f04680a6260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04780ce1e0 tx=0x7f04780bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.802+0000 7f0482400640 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f047400c3c0 tx=0x7f0474010040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b51a0a26acbbab3b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0467fff640 1 -- 172.21.15.122:0/3166877203 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474003970 con 0x562c7c4717a0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0467fff640 1 -- 172.21.15.122:0/3166877203 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0474004030 con 0x562c7c4717a0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0467fff640 1 -- 172.21.15.122:0/3166877203 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047401d630 con 0x562c7c4717a0 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 -- 172.21.15.122:0/3166877203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 msgr2=0x562c7c471b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f047400c3c0 tx=0x7f0474010040 comp rx=0 tx=0).stop 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 -- 172.21.15.122:0/3166877203 shutdown_connections 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c47a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 --2- 172.21.15.122:0/3166877203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4717a0 0x562c7c471b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 -- 172.21.15.122:0/3166877203 >> 172.21.15.122:0/3166877203 conn(0x562c7c46bed0 msgr2=0x562c7c46c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 -- 172.21.15.122:0/3166877203 shutdown_connections 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.803+0000 7f0486a50740 1 -- 172.21.15.122:0/3166877203 wait complete. 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 -- start start 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 Processor -- start 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c47aca0 0x562c7c478010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c7c47da30 con 0x562c7c4720c0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562c7c390e80 con 0x562c7c47aca0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562c7c47dbb0 con 0x562c7c4717a0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0486a50740 10 client.0 _ll_register_callbacks cb 0x562c7c3ccaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f04813fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0482400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f04813fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53004/0 (socket says 172.21.15.122:53004) 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f04813fe640 1 -- 172.21.15.122:0/2175963806 learned_addr learned my addr 172.21.15.122:0/2175963806 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.804+0000 7f0482400640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c47aca0 0x562c7c478010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 -- 172.21.15.122:0/2175963806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 msgr2=0x562c7c47a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 -- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c47aca0 msgr2=0x562c7c478010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c47aca0 0x562c7c478010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 -- 172.21.15.122:0/2175963806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7c1b37d0 con 0x562c7c4720c0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04813fe640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f046c02c0b0 tx=0x7f046c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da464a0910594ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c039040 con 0x562c7c4720c0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f046c004450 con 0x562c7c4720c0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0474004030 con 0x562c7c4720c0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f046c04e050 con 0x562c7c4720c0 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f0486a50740 10 client.19425 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562c7c489a00 con 0x562c7c4720c0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0474004030 con 0x562c7c4720c0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f046c02c510 con 0x562c7c4720c0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f0486a50740 10 client.19425 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.805+0000 7f0486a50740 10 client.19425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0486a50740 20 client.19425 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c48d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0486a50740 4 client.19425 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7c3e2cf0 con 0x562c7c48af10 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0482400640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c48d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f0482400640 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c48d350 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81851f600f8b8226 server_cookie=4a0bea600062324d in_seq=0 out_seq=0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.806+0000 7f04657fa640 10 client.19425 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f04657fa640 1 -- 172.21.15.122:0/2175963806 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562c7c3e2cf0 con 0x562c7c48af10 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f04657fa640 10 client.19425 handle_command_reply: tid=0 2024-04-02T13:48:20.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 client.19425 shutdown 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 msgr2=0x562c7c477a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f046c02c0b0 tx=0x7f046c009520 comp rx=0 tx=0).stop 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 shutdown_connections 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c48d350 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f04657fa640 0 client.19425 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c47aca0 0x562c7c478010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4720c0 0x562c7c477a00 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 --2- 172.21.15.122:0/2175963806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 >> 172.21.15.122:0/2175963806 conn(0x562c7c46bed0 msgr2=0x562c7c46dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 shutdown_connections 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 1 -- 172.21.15.122:0/2175963806 wait complete. 2024-04-02T13:48:20.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.807+0000 7f0486a50740 20 client.19425 trim_cache size 0 max 16384 2024-04-02T13:48:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 Processor -- start 2024-04-02T13:48:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 -- start start 2024-04-02T13:48:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c4868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1af0d0 con 0x562c7c4717a0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0c10 con 0x562c7c4720c0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0486a50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1af7a0 con 0x562c7c47aca0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53010/0 (socket says 172.21.15.122:53010) 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 -- 172.21.15.122:0/346405335 learned_addr learned my addr 172.21.15.122:0/346405335 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f04813fe640 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c4868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 -- 172.21.15.122:0/346405335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 msgr2=0x562c7c47a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 -- 172.21.15.122:0/346405335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 msgr2=0x562c7c4868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c4868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.823+0000 7f0482400640 1 -- 172.21.15.122:0/346405335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7c471ef0 con 0x562c7c4717a0 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.824+0000 7f0482400640 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f047400ec90 tx=0x7f0474010040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c4f3411d8d77c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.824+0000 7f04677fe640 1 -- 172.21.15.122:0/346405335 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0474003970 con 0x562c7c4717a0 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.824+0000 7f04677fe640 1 -- 172.21.15.122:0/346405335 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0474030030 con 0x562c7c4717a0 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.824+0000 7f04677fe640 1 -- 172.21.15.122:0/346405335 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047402ca20 con 0x562c7c4717a0 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- 172.21.15.122:0/346405335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 msgr2=0x562c7c48d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f047400ec90 tx=0x7f0474010040 comp rx=0 tx=0).stop 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- 172.21.15.122:0/346405335 shutdown_connections 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c47aca0 0x562c7c47a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c4868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- 172.21.15.122:0/346405335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c4717a0 0x562c7c48d330 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- 172.21.15.122:0/346405335 >> 172.21.15.122:0/346405335 conn(0x562c7c46bed0 msgr2=0x562c7c4762d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- 172.21.15.122:0/346405335 shutdown_connections 2024-04-02T13:48:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- 172.21.15.122:0/346405335 wait complete. 2024-04-02T13:48:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 -- start start 2024-04-02T13:48:20.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 Processor -- start 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.825+0000 7f0486a50740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c47b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1b0540 con 0x562c7c47aca0 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1afe70 con 0x562c7c4720c0 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f047c1ad0d0 con 0x562c7c4717a0 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 10 client.0 _ll_register_callbacks cb 0x7f0478005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0482400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0482400640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53026/0 (socket says 172.21.15.122:53026) 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0482400640 1 -- 172.21.15.122:0/2687667660 learned_addr learned my addr 172.21.15.122:0/2687667660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c47b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0482400640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 -- 172.21.15.122:0/2687667660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 msgr2=0x562c7c47b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 -- 172.21.15.122:0/2687667660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 msgr2=0x562c7c47bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 -- 172.21.15.122:0/2687667660 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7c481e00 con 0x562c7c4720c0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0482400640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04813fe640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c47b7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f046c005e00 tx=0x7f046c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406400ec496250dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04657fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c039040 con 0x562c7c4720c0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0474030030 con 0x562c7c4720c0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.826+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f046c004450 con 0x562c7c4720c0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f046c0096b0 con 0x562c7c4720c0 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 10 client.28433 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562c7c4830e0 con 0x562c7c4720c0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0474030030 con 0x562c7c4720c0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f046c005370 con 0x562c7c4720c0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 10 client.28433 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 10 client.28433 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 20 client.28433 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c4f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 4 client.28433 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c7c3dd040 con 0x562c7c48af10 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.827+0000 7f0482400640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c4f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f04657fa640 10 client.28433 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0482400640 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c4f34f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6907adf54a3e9d8 server_cookie=2ace2afedf9999eb in_seq=0 out_seq=0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f04657fa640 1 -- 172.21.15.122:0/2687667660 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c7c3dd040 con 0x562c7c48af10 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f04657fa640 10 client.28433 handle_command_reply: tid=0 2024-04-02T13:48:20.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 client.28433 shutdown 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 msgr2=0x562c7c47b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c47b7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f046c005e00 tx=0x7f046c005f10 comp rx=0 tx=0).stop 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 shutdown_connections 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c7c47aca0 0x562c7c47bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562c7c48af10 0x562c7c4f34f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562c7c4720c0 0x562c7c47b7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 --2- 172.21.15.122:0/2687667660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562c7c4717a0 0x562c7c47b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f04657fa640 0 client.28433 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 >> 172.21.15.122:0/2687667660 conn(0x562c7c46bed0 msgr2=0x562c7c477290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 shutdown_connections 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 1 -- 172.21.15.122:0/2687667660 wait complete. 2024-04-02T13:48:20.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.828+0000 7f0486a50740 20 client.28433 trim_cache size 0 max 16384 2024-04-02T13:48:20.839 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:20.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 msgr2=0x7f04680a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 0x7f04680a6260 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f04780ce1e0 tx=0x7f04780bd040 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 msgr2=0x7f047c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f047800e9f0 tx=0x7f047800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 shutdown_connections 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04680a3da0 0x7f04680a6260 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f047c106980 0x7f047c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f047c101330 0x7f047c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 --2- 172.21.15.122:0/4405275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f047c100970 0x7f047c1983a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 >> 172.21.15.122:0/4405275 conn(0x7f047c077b00 msgr2=0x7f047c068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 shutdown_connections 2024-04-02T13:48:20.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.829+0000 7f0483e63640 1 -- 172.21.15.122:0/4405275 wait complete. 2024-04-02T13:48:20.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:48:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 -- 172.21.15.122:0/717125713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 msgr2=0x7fd36010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 --2- 172.21.15.122:0/717125713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd36010e960 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd358009cd0 tx=0x7fd35802c0e0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 -- 172.21.15.122:0/717125713 shutdown_connections 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 --2- 172.21.15.122:0/717125713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd36010e960 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 --2- 172.21.15.122:0/717125713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360068d00 0x7fd360069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 --2- 172.21.15.122:0/717125713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360107b50 0x7fd360107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 -- 172.21.15.122:0/717125713 >> 172.21.15.122:0/717125713 conn(0x7fd360077b00 msgr2=0x7fd360077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 -- 172.21.15.122:0/717125713 shutdown_connections 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.949+0000 7fd366ad9640 1 -- 172.21.15.122:0/717125713 wait complete. 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 Processor -- start 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 -- start start 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 0x7fd360198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360107b50 0x7fd36019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd360111ea0 con 0x7fd3600696c0 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd360111d20 con 0x7fd360068d00 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd366ad9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd360112020 con 0x7fd360107b50 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53036/0 (socket says 172.21.15.122:53036) 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 -- 172.21.15.122:0/3586518043 learned_addr learned my addr 172.21.15.122:0/3586518043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd364875640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 0x7fd360198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd365076640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360107b50 0x7fd36019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 -- 172.21.15.122:0/3586518043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360107b50 msgr2=0x7fd36019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360107b50 0x7fd36019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 -- 172.21.15.122:0/3586518043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 msgr2=0x7fd360198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 0x7fd360198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 -- 172.21.15.122:0/3586518043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36019d2c0 con 0x7fd3600696c0 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd364875640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 0x7fd360198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.950+0000 7fd35ffff640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00e9f0 tx=0x7fd34c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.951+0000 7fd35dffb640 1 -- 172.21.15.122:0/3586518043 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c00cd80 con 0x7fd3600696c0 2024-04-02T13:48:20.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.951+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36019d5b0 con 0x7fd3600696c0 2024-04-02T13:48:20.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.951+0000 7fd35dffb640 1 -- 172.21.15.122:0/3586518043 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd34c010910 con 0x7fd3600696c0 2024-04-02T13:48:20.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.951+0000 7fd35dffb640 1 -- 172.21.15.122:0/3586518043 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0188f0 con 0x7fd3600696c0 2024-04-02T13:48:20.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.951+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a4e40 con 0x7fd3600696c0 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd35dffb640 1 -- 172.21.15.122:0/3586518043 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd34c010ac0 con 0x7fd3600696c0 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 Processor -- start 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 -- start start 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b60fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b61b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56442b5737b0 con 0x56442b6100c0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56442b512570 con 0x56442b618ca0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd3697a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56442b56f760 con 0x56442b60f7a0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.953+0000 7fd35dffb640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 0x7fd3500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53038/0 (socket says 172.21.15.122:53038) 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd35ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b61b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 -- 172.21.15.122:0/4169207419 learned_addr learned my addr 172.21.15.122:0/4169207419 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd365076640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b60fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd365076640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 0x7fd3500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd35dffb640 1 -- 172.21.15.122:0/3586518043 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fd34c0d0f80 con 0x7fd3600696c0 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 -- 172.21.15.122:0/4169207419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 msgr2=0x56442b60fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b60fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 -- 172.21.15.122:0/4169207419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 msgr2=0x56442b61b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b61b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 -- 172.21.15.122:0/4169207419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442b2873f0 con 0x56442b6100c0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd365076640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 0x7fd3500a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd358005cc0 tx=0x7fd358005c50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd364875640 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd3540115f0 tx=0x7fd354011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f768a50c12ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd34affd640 1 -- 172.21.15.122:0/4169207419 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35401e070 con 0x56442b6100c0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd34affd640 1 -- 172.21.15.122:0/4169207419 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35400fb20 con 0x56442b6100c0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.954+0000 7fd34affd640 1 -- 172.21.15.122:0/4169207419 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354019c40 con 0x56442b6100c0 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- 172.21.15.122:0/4169207419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 msgr2=0x56442b618760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd3540115f0 tx=0x7fd354011ab0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- 172.21.15.122:0/4169207419 shutdown_connections 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b61b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b6100c0 0x56442b618760 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 --2- 172.21.15.122:0/4169207419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b60fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- 172.21.15.122:0/4169207419 >> 172.21.15.122:0/4169207419 conn(0x56442b609ed0 msgr2=0x56442b60a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- 172.21.15.122:0/4169207419 shutdown_connections 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- 172.21.15.122:0/4169207419 wait complete. 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 -- start start 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.955+0000 7fd3697a1740 1 Processor -- start 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b615a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b616030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56442b61ba90 con 0x56442b60f7a0 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56442b52ee80 con 0x56442b618ca0 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56442b61bc10 con 0x56442b6100c0 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd3697a1740 10 client.0 _ll_register_callbacks cb 0x56442b56aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53040/0 (socket says 172.21.15.122:53040) 2024-04-02T13:48:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 -- 172.21.15.122:0/2237104032 learned_addr learned my addr 172.21.15.122:0/2237104032 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd35ffff640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b615a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd365076640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b616030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 -- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 msgr2=0x56442b615a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b615a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 -- 172.21.15.122:0/2237104032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 msgr2=0x56442b616030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b616030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 -- 172.21.15.122:0/2237104032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442b3517d0 con 0x56442b60f7a0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd364875640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd35401c040 tx=0x7fd35400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab350799a03a7bad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.956+0000 7fd33ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd354029060 con 0x56442b60f7a0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fd3540199c0 con 0x56442b60f7a0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35400fb20 con 0x56442b60f7a0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd354030050 con 0x56442b60f7a0 2024-04-02T13:48:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 10 client.19443 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56442b627b00 con 0x56442b60f7a0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35400fb20 con 0x56442b60f7a0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fd3540035f0 con 0x56442b60f7a0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 10 client.19443 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 10 client.19443 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 20 client.19443 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b628f60 0x56442b62b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 4 client.19443 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56442b580cf0 con 0x56442b628f60 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.957+0000 7fd35ffff640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b628f60 0x56442b62b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.958+0000 7fd35ffff640 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b628f60 0x56442b62b350 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8205a8647b0cf08 server_cookie=9d5111ddf599869 in_seq=0 out_seq=0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.958+0000 7fd33ffff640 10 client.19443 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd33ffff640 1 -- 172.21.15.122:0/2237104032 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56442b580cf0 con 0x56442b628f60 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd33ffff640 10 client.19443 handle_command_reply: tid=0 2024-04-02T13:48:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 client.19443 shutdown 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 msgr2=0x56442b618690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fd35401c040 tx=0x7fd35400f8d0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 shutdown_connections 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b628f60 0x56442b62b350 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b618ca0 0x56442b616030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b615a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd33ffff640 0 client.19443 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2237104032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b60f7a0 0x56442b618690 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 >> 172.21.15.122:0/2237104032 conn(0x56442b609ed0 msgr2=0x56442b60bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 shutdown_connections 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 1 -- 172.21.15.122:0/2237104032 wait complete. 2024-04-02T13:48:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.959+0000 7fd3697a1740 20 client.19443 trim_cache size 0 max 16384 2024-04-02T13:48:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 Processor -- start 2024-04-02T13:48:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 -- start start 2024-04-02T13:48:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b60f7a0 0x56442b624980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b618150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601af040 con 0x56442b618ca0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601b0b80 con 0x56442b60f7a0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd3697a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601af710 con 0x56442b6100c0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.973+0000 7fd365076640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53052/0 (socket says 172.21.15.122:53052) 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 -- 172.21.15.122:0/2583195942 learned_addr learned my addr 172.21.15.122:0/2583195942 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd364875640 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b60f7a0 0x56442b624980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd35ffff640 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b618150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 -- 172.21.15.122:0/2583195942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 msgr2=0x56442b618150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b618150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 -- 172.21.15.122:0/2583195942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b60f7a0 msgr2=0x56442b624980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b60f7a0 0x56442b624980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 -- 172.21.15.122:0/2583195942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56442b6163a0 con 0x56442b618ca0 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd365076640 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fd358009cd0 tx=0x7fd358009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc05486a8d983c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd34a7fc640 1 -- 172.21.15.122:0/2583195942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358055070 con 0x56442b618ca0 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd34a7fc640 1 -- 172.21.15.122:0/2583195942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd358048040 con 0x56442b618ca0 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.974+0000 7fd34a7fc640 1 -- 172.21.15.122:0/2583195942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358039030 con 0x56442b618ca0 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- 172.21.15.122:0/2583195942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 msgr2=0x56442b618690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fd358009cd0 tx=0x7fd358009360 comp rx=0 tx=0).stop 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- 172.21.15.122:0/2583195942 shutdown_connections 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b618690 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b6100c0 0x56442b618150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 --2- 172.21.15.122:0/2583195942 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b60f7a0 0x56442b624980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- 172.21.15.122:0/2583195942 >> 172.21.15.122:0/2583195942 conn(0x56442b609ed0 msgr2=0x56442b60a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- 172.21.15.122:0/2583195942 shutdown_connections 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- 172.21.15.122:0/2583195942 wait complete. 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 -- start start 2024-04-02T13:48:20.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 Processor -- start 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.975+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b621140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b6100c0 0x56442b619680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601b04b0 con 0x56442b618ca0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601afde0 con 0x56442b6100c0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601ad040 con 0x56442b60f7a0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 10 client.0 _ll_register_callbacks cb 0x7fd35401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53054/0 (socket says 172.21.15.122:53054) 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 -- 172.21.15.122:0/3299827075 learned_addr learned my addr 172.21.15.122:0/3299827075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd364875640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b621140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd35ffff640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b6100c0 0x56442b619680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 -- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 msgr2=0x56442b621140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b621140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 -- 172.21.15.122:0/3299827075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b6100c0 msgr2=0x56442b619680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b6100c0 0x56442b619680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 -- 172.21.15.122:0/3299827075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56442b61a200 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd365076640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd35802c6f0 tx=0x7fd3580323e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9590e8c5e3030032 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd33ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358055030 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fd358050df0 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd358048040 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd358039070 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 10 client.19449 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.976+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56442b68f2e0 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd358048040 con 0x56442b618ca0 2024-04-02T13:48:20.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fd35804d070 con 0x56442b618ca0 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 10 client.19449 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 10 client.19449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 20 client.19449 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b68f820 0x56442b691c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 4 client.19449 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56442b57b040 con 0x56442b68f820 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd364875640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b68f820 0x56442b691c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd33ffff640 10 client.19449 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.977+0000 7fd364875640 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b68f820 0x56442b691c40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e50524c0d54ede32 server_cookie=315769957e4cbba0 in_seq=0 out_seq=0 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd33ffff640 1 -- 172.21.15.122:0/3299827075 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56442b57b040 con 0x56442b68f820 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd33ffff640 10 client.19449 handle_command_reply: tid=0 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 client.19449 shutdown 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 msgr2=0x56442b619bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fd35802c6f0 tx=0x7fd3580323e0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 shutdown_connections 2024-04-02T13:48:20.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56442b68f820 0x56442b691c40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56442b618ca0 0x56442b619bc0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56442b6100c0 0x56442b619680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 --2- 172.21.15.122:0/3299827075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56442b60f7a0 0x56442b621140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.988 INFO:tasks.workunit.client.0.smithi122.stdout: 1 579096 4.48 MB/sec execute 581 sec latency 19.636 ms 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 >> 172.21.15.122:0/3299827075 conn(0x56442b609ed0 msgr2=0x56442b614210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd33ffff640 0 client.19449 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 shutdown_connections 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 1 -- 172.21.15.122:0/3299827075 wait complete. 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.978+0000 7fd3697a1740 20 client.19449 trim_cache size 0 max 16384 2024-04-02T13:48:20.989 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 msgr2=0x7fd3500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 0x7fd3500a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd358005cc0 tx=0x7fd358005c50 comp rx=0 tx=0).stop 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 msgr2=0x7fd360198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00e9f0 tx=0x7fd34c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 shutdown_connections 2024-04-02T13:48:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3500a3d20 0x7fd3500a61e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd360107b50 0x7fd36019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3600696c0 0x7fd360198860 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 --2- 172.21.15.122:0/3586518043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd360068d00 0x7fd360198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 >> 172.21.15.122:0/3586518043 conn(0x7fd360077b00 msgr2=0x7fd3600fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 shutdown_connections 2024-04-02T13:48:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:20.979+0000 7fd366ad9640 1 -- 172.21.15.122:0/3586518043 wait complete. 2024-04-02T13:48:20.993 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:48:21.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/3876594388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8100970 msgr2=0x7fb6e8100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/3876594388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8100970 0x7fb6e8100df0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009930 tx=0x7fb6d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/3876594388 shutdown_connections 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/3876594388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 0x7fb6e810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/3876594388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8100970 0x7fb6e8100df0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/3876594388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8106980 0x7fb6e8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/3876594388 >> 172.21.15.122:0/3876594388 conn(0x7fb6e8077b00 msgr2=0x7fb6e8077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/3876594388 shutdown_connections 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/3876594388 wait complete. 2024-04-02T13:48:21.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.101+0000 7fb6ecc7b640 1 Processor -- start 2024-04-02T13:48:21.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 -- start start 2024-04-02T13:48:21.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 0x7fb6e8198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 0x7fb6e81988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8110d90 con 0x7fb6e8106980 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8110c10 con 0x7fb6e8100970 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ecc7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8110f10 con 0x7fb6e8101330 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 0x7fb6e8198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53076/0 (socket says 172.21.15.122:53076) 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 -- 172.21.15.122:0/542323605 learned_addr learned my addr 172.21.15.122:0/542323605 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 0x7fb6e81988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 -- 172.21.15.122:0/542323605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 msgr2=0x7fb6e81988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 0x7fb6e81988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 -- 172.21.15.122:0/542323605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 msgr2=0x7fb6e8198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 0x7fb6e8198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e6d9d640 1 -- 172.21.15.122:0/542323605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e819d350 con 0x7fb6e8106980 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.102+0000 7fb6e659c640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 0x7fb6e8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6e6d9d640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000eb10 tx=0x7fb6e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6dffff640 1 -- 172.21.15.122:0/542323605 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0004b20 con 0x7fb6e8106980 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e819d640 con 0x7fb6e8106980 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6dffff640 1 -- 172.21.15.122:0/542323605 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6e0004cc0 con 0x7fb6e8106980 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6dffff640 1 -- 172.21.15.122:0/542323605 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0010760 con 0x7fb6e8106980 2024-04-02T13:48:21.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.103+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e81a4ed0 con 0x7fb6e8106980 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6dffff640 1 -- 172.21.15.122:0/542323605 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb6e0005020 con 0x7fb6e8106980 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 Processor -- start 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 -- start start 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4322830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af43324a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559af42867b0 con 0x559af4322d70 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559af4225570 con 0x559af4322450 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ef867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559af4282760 con 0x559af43300b0 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6dffff640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 0x7fb6d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4322830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53086/0 (socket says 172.21.15.122:53086) 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.106+0000 7fb6e659c640 1 -- 172.21.15.122:0/3259319139 learned_addr learned my addr 172.21.15.122:0/3259319139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e6d9d640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 0x7fb6d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6dffff640 1 -- 172.21.15.122:0/542323605 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb6e00d1170 con 0x7fb6e8106980 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af43324a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 -- 172.21.15.122:0/3259319139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 msgr2=0x559af43324a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af43324a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 -- 172.21.15.122:0/3259319139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 msgr2=0x559af4322830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4322830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 -- 172.21.15.122:0/3259319139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af3f9a3f0 con 0x559af4322d70 2024-04-02T13:48:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4322830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e659c640 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80114f0 tx=0x7fb6d80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58476db011c82606 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6e6d9d640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 0x7fb6d40a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00996f0 tx=0x7fb6e00bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6dcd99640 1 -- 172.21.15.122:0/3259319139 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d801e070 con 0x559af4322d70 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6dcd99640 1 -- 172.21.15.122:0/3259319139 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d800faa0 con 0x559af4322d70 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.107+0000 7fb6dcd99640 1 -- 172.21.15.122:0/3259319139 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8019b90 con 0x559af4322d70 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- 172.21.15.122:0/3259319139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 msgr2=0x559af432fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80114f0 tx=0x7fb6d80119b0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- 172.21.15.122:0/3259319139 shutdown_connections 2024-04-02T13:48:21.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af43324a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432fb70 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 --2- 172.21.15.122:0/3259319139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4322830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- 172.21.15.122:0/3259319139 >> 172.21.15.122:0/3259319139 conn(0x559af431ced0 msgr2=0x559af431d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- 172.21.15.122:0/3259319139 shutdown_connections 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- 172.21.15.122:0/3259319139 wait complete. 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.108+0000 7fb6ef867740 1 -- start start 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 Processor -- start 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322d70 0x559af432ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af432d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559af4332ef0 con 0x559af43300b0 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559af4241e80 con 0x559af4322d70 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559af4333070 con 0x559af4322450 2024-04-02T13:48:21.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ef867740 10 client.0 _ll_register_callbacks cb 0x559af427daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af432d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322d70 0x559af432ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:35936/0 (socket says 172.21.15.122:35936) 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/2347778207 learned_addr learned my addr 172.21.15.122:0/2347778207 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322d70 msgr2=0x559af432ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322d70 0x559af432ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/2347778207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 msgr2=0x559af432d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af432d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/2347778207 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af40647d0 con 0x559af4322450 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009930 tx=0x7fb6d0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=528942d398e0e6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.109+0000 7fb6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0004050 con 0x559af4322450 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d800faa0 con 0x559af4322450 2024-04-02T13:48:21.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb6d00041f0 con 0x559af4322450 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6d0055050 con 0x559af4322450 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 10 client.28216 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559af433f000 con 0x559af4322450 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d800faa0 con 0x559af4322450 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fb6d004c070 con 0x559af4322450 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 10 client.28216 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 10 client.28216 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 20 client.28216 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af4340410 0x559af4342800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 4 client.28216 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af4293cf0 con 0x559af4340410 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.110+0000 7fb6e659c640 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af4340410 0x559af4342800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.111+0000 7fb6cdffb640 10 client.28216 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.111+0000 7fb6e659c640 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af4340410 0x559af4342800 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b7a146bc9e6c21 server_cookie=6d5f2f6a52c8ecb8 in_seq=0 out_seq=0 2024-04-02T13:48:21.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6cdffb640 1 -- 172.21.15.122:0/2347778207 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559af4293cf0 con 0x559af4340410 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6cdffb640 10 client.28216 handle_command_reply: tid=0 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 client.28216 shutdown 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 msgr2=0x559af432fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0009930 tx=0x7fb6d0005b50 comp rx=0 tx=0).stop 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 shutdown_connections 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af4340410 0x559af4342800 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af432d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6cdffb640 0 client.28216 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322d70 0x559af432ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2347778207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322450 0x559af432fb00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 >> 172.21.15.122:0/2347778207 conn(0x559af431ced0 msgr2=0x559af431ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 shutdown_connections 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 1 -- 172.21.15.122:0/2347778207 wait complete. 2024-04-02T13:48:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.112+0000 7fb6ef867740 20 client.28216 trim_cache size 0 max 16384 2024-04-02T13:48:21.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.125+0000 7fb6ef867740 1 Processor -- start 2024-04-02T13:48:21.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.125+0000 7fb6ef867740 1 -- start start 2024-04-02T13:48:21.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.125+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af433be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af4348090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ef867740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81acee0 con 0x559af4322d70 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ef867740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81aea20 con 0x559af4322450 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ef867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81ad5b0 con 0x559af43300b0 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53092/0 (socket says 172.21.15.122:53092) 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 -- 172.21.15.122:0/2215133193 learned_addr learned my addr 172.21.15.122:0/2215133193 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af4348090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af433be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 -- 172.21.15.122:0/2215133193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 msgr2=0x559af4348090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af4348090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 -- 172.21.15.122:0/2215133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 msgr2=0x559af433be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af433be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 -- 172.21.15.122:0/2215133193 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af4322ba0 con 0x559af4322d70 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af4348090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6e659c640 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8019700 tx=0x7fb6d8019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0b6c7773d0eca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6cffff640 1 -- 172.21.15.122:0/2215133193 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8029070 con 0x559af4322d70 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6cffff640 1 -- 172.21.15.122:0/2215133193 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8009b50 con 0x559af4322d70 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.126+0000 7fb6cffff640 1 -- 172.21.15.122:0/2215133193 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d801e040 con 0x559af4322d70 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- 172.21.15.122:0/2215133193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 msgr2=0x559af432be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8019700 tx=0x7fb6d8019730 comp rx=0 tx=0).stop 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- 172.21.15.122:0/2215133193 shutdown_connections 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af43300b0 0x559af4348090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af4322d70 0x559af432be70 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 --2- 172.21.15.122:0/2215133193 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af433be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- 172.21.15.122:0/2215133193 >> 172.21.15.122:0/2215133193 conn(0x559af431ced0 msgr2=0x559af432b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- 172.21.15.122:0/2215133193 shutdown_connections 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- 172.21.15.122:0/2215133193 wait complete. 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 -- start start 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.127+0000 7fb6ef867740 1 Processor -- start 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4338630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322d70 0x559af4330b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81ae350 con 0x559af43300b0 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81adc80 con 0x559af4322450 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81aaee0 con 0x559af4322d70 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 10 client.0 _ll_register_callbacks cb 0x7fb6d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53098/0 (socket says 172.21.15.122:53098) 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/956971334 learned_addr learned my addr 172.21.15.122:0/956971334 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:21.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6e659c640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322d70 0x559af4330b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4338630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/956971334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322d70 msgr2=0x559af4330b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322d70 0x559af4330b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 msgr2=0x559af4338630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4338630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 -- 172.21.15.122:0/956971334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af4331800 con 0x559af43300b0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ddd9b640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb6d002c170 tx=0x7fb6d0042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329d142b02ba8ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0004120 con 0x559af43300b0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb6d0004f50 con 0x559af43300b0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.128+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6d8009b50 con 0x559af43300b0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb6d00042c0 con 0x559af43300b0 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 10 client.19467 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:21.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559af43a6900 con 0x559af43300b0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6d8009b50 con 0x559af43300b0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fb6d004c050 con 0x559af43300b0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 10 client.19467 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 20 client.19467 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af43a6dd0 0x559af43a91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 4 client.19467 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559af428e040 con 0x559af43a6dd0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6e659c640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af43a6dd0 0x559af43a91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6e659c640 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af43a6dd0 0x559af43a91c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45a590cfdb2bea8 server_cookie=31bc4be1bb45dc3 in_seq=0 out_seq=0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.129+0000 7fb6cdffb640 10 client.19467 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6cdffb640 1 -- 172.21.15.122:0/956971334 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559af428e040 con 0x559af43a6dd0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6cdffb640 10 client.19467 handle_command_reply: tid=0 2024-04-02T13:48:21.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 client.19467 shutdown 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 msgr2=0x559af4331090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb6d002c170 tx=0x7fb6d0042040 comp rx=0 tx=0).stop 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 shutdown_connections 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x559af43a6dd0 0x559af43a91c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559af43300b0 0x559af4331090 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6cdffb640 0 client.19467 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559af4322d70 0x559af4330b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 --2- 172.21.15.122:0/956971334 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559af4322450 0x559af4338630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 >> 172.21.15.122:0/956971334 conn(0x559af431ced0 msgr2=0x559af432bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 shutdown_connections 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 1 -- 172.21.15.122:0/956971334 wait complete. 2024-04-02T13:48:21.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ef867740 20 client.19467 trim_cache size 0 max 16384 2024-04-02T13:48:21.142 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:21.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 msgr2=0x7fb6d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 0x7fb6d40a61e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00996f0 tx=0x7fb6e00bd040 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 msgr2=0x7fb6e819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.130+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000eb10 tx=0x7fb6e000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 shutdown_connections 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb6d40a3d20 0x7fb6d40a61e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6e8106980 0x7fb6e819cc20 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6e8101330 0x7fb6e81988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 --2- 172.21.15.122:0/542323605 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb6e8100970 0x7fb6e8198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 >> 172.21.15.122:0/542323605 conn(0x7fb6e8077b00 msgr2=0x7fb6e8069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 shutdown_connections 2024-04-02T13:48:21.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:21.131+0000 7fb6ecc7b640 1 -- 172.21.15.122:0/542323605 wait complete. 2024-04-02T13:48:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:21 smithi187 ceph-mon[36064]: pgmap v412: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:21 smithi188 ceph-mon[36117]: pgmap v412: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:21 smithi122 ceph-mon[32114]: pgmap v412: 129 pgs: 129 active+clean; 562 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-04-02T13:48:21.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 580236 4.48 MB/sec execute 582 sec latency 15.704 ms 2024-04-02T13:48:22.145 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:48:22.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:48:22.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 -- 172.21.15.122:0/1608668481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5600696c0 msgr2=0x7fc56010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 --2- 172.21.15.122:0/1608668481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5600696c0 0x7fc56010e960 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fc558009c10 tx=0x7fc55802bef0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 -- 172.21.15.122:0/1608668481 shutdown_connections 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 --2- 172.21.15.122:0/1608668481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5600696c0 0x7fc56010e960 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 --2- 172.21.15.122:0/1608668481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc560068d00 0x7fc560069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 --2- 172.21.15.122:0/1608668481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 0x7fc560107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 -- 172.21.15.122:0/1608668481 >> 172.21.15.122:0/1608668481 conn(0x7fc560077b00 msgr2=0x7fc560077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 -- 172.21.15.122:0/1608668481 shutdown_connections 2024-04-02T13:48:22.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.259+0000 7fc56741b640 1 -- 172.21.15.122:0/1608668481 wait complete. 2024-04-02T13:48:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 Processor -- start 2024-04-02T13:48:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 -- start start 2024-04-02T13:48:22.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5600696c0 0x7fc560100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 0x7fc560101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc560111e70 con 0x7fc560068d00 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc560111cf0 con 0x7fc5600696c0 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc56741b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc560111ff0 con 0x7fc560107b50 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53116/0 (socket says 172.21.15.122:53116) 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 -- 172.21.15.122:0/767441417 learned_addr learned my addr 172.21.15.122:0/767441417 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5649b6640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5600696c0 0x7fc560100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5659b8640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 0x7fc560101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 -- 172.21.15.122:0/767441417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 msgr2=0x7fc560101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 0x7fc560101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 -- 172.21.15.122:0/767441417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5600696c0 msgr2=0x7fc560100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5600696c0 0x7fc560100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.260+0000 7fc5651b7640 1 -- 172.21.15.122:0/767441417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5601a6830 con 0x7fc560068d00 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc5651b7640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e9c0 tx=0x7fc55c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc5567fc640 1 -- 172.21.15.122:0/767441417 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c00ccc0 con 0x7fc560068d00 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc5567fc640 1 -- 172.21.15.122:0/767441417 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc55c005500 con 0x7fc560068d00 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5601a6b20 con 0x7fc560068d00 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc5567fc640 1 -- 172.21.15.122:0/767441417 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55c010430 con 0x7fc560068d00 2024-04-02T13:48:22.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.261+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5601a7070 con 0x7fc560068d00 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.263+0000 7fc56a001740 1 Processor -- start 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.263+0000 7fc56a001740 1 -- start start 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.263+0000 7fc5567fc640 1 -- 172.21.15.122:0/767441417 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc55c0046c0 con 0x7fc560068d00 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dbc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dbebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bf5d17770 con 0x562bf5dbc7c0 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562bf5cbf960 con 0x562bf5db32c0 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc56a001740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562bf5d138c0 con 0x562bf5db3be0 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5649b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dbebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5651b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dbc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:41544/0 (socket says 172.21.15.122:41544) 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5649b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dbebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53122/0 (socket says 172.21.15.122:53122) 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 -- 172.21.15.122:0/1339284927 learned_addr learned my addr 172.21.15.122:0/1339284927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 -- 172.21.15.122:0/1339284927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 msgr2=0x562bf5dbc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dbc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 -- 172.21.15.122:0/1339284927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 msgr2=0x562bf5dbebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dbebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 -- 172.21.15.122:0/1339284927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf5af37e0 con 0x562bf5db32c0 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5651b7640 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dbc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5567fc640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 0x7fc54c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc558009c10 tx=0x7fc5580098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f318f0330823533c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5567fc640 1 -- 172.21.15.122:0/767441417 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fc55c0d0e00 con 0x7fc560068d00 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.264+0000 7fc5651b7640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 0x7fc54c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc54b7fe640 1 -- 172.21.15.122:0/1339284927 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558043030 con 0x562bf5db32c0 2024-04-02T13:48:22.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc54b7fe640 1 -- 172.21.15.122:0/1339284927 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc558033cc0 con 0x562bf5db32c0 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc54b7fe640 1 -- 172.21.15.122:0/1339284927 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55802c890 con 0x562bf5db32c0 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 -- 172.21.15.122:0/1339284927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 msgr2=0x562bf5db36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc5651b7640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 0x7fc54c0a6210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0ce210 tx=0x7fc55c0bd040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc558009c10 tx=0x7fc5580098f0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 -- 172.21.15.122:0/1339284927 shutdown_connections 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dbebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dbc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 --2- 172.21.15.122:0/1339284927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5db36a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 -- 172.21.15.122:0/1339284927 >> 172.21.15.122:0/1339284927 conn(0x562bf5dad9f0 msgr2=0x562bf5daddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 -- 172.21.15.122:0/1339284927 shutdown_connections 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.265+0000 7fc56a001740 1 -- 172.21.15.122:0/1339284927 wait complete. 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 -- start start 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 Processor -- start 2024-04-02T13:48:22.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db32c0 0x562bf5db9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 0x562bf5db9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bf5dbf550 con 0x562bf5dbc7c0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562bf5cd2d20 con 0x562bf5db3be0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562bf5dbf6d0 con 0x562bf5db32c0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc56a001740 10 client.0 _ll_register_callbacks cb 0x562bf5c9fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc5649b6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 0x562bf5db9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53132/0 (socket says 172.21.15.122:53132) 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc5659b8640 1 -- 172.21.15.122:0/474021229 learned_addr learned my addr 172.21.15.122:0/474021229 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.266+0000 7fc5659b8640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db32c0 0x562bf5db9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 -- 172.21.15.122:0/474021229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db32c0 msgr2=0x562bf5db9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db32c0 0x562bf5db9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 -- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 msgr2=0x562bf5db9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 0x562bf5db9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 -- 172.21.15.122:0/474021229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf5b53f70 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5649b6640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 0x562bf5db9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc5659b8640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc558003fd0 tx=0x7fc558003df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7805a3969459105c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558043040 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fc55804d030 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc558033cc0 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc558004b40 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc56a001740 10 client.19479 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562bf59eb170 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc558033cc0 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.267+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fc558004150 con 0x562bf5dbc7c0 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 10 client.19479 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 10 client.19479 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:22.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 20 client.19479 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5dcc8d0 0x562bf5dcecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 4 client.19479 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bf5d24cb0 con 0x562bf5dcc8d0 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc5649b6640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5dcc8d0 0x562bf5dcecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc548ff9640 10 client.19479 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.268+0000 7fc5649b6640 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5dcc8d0 0x562bf5dcecc0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e11634a76d729b6 server_cookie=61408cec24c72c38 in_seq=0 out_seq=0 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.269+0000 7fc548ff9640 1 -- 172.21.15.122:0/474021229 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bf5d24cb0 con 0x562bf5dcc8d0 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.269+0000 7fc548ff9640 10 client.19479 handle_command_reply: tid=0 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.269+0000 7fc56a001740 1 client.19479 shutdown 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 msgr2=0x562bf5dba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fc558003fd0 tx=0x7fc558003df0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 shutdown_connections 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5dcc8d0 0x562bf5dcecc0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5dbc7c0 0x562bf5dba1d0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db3be0 0x562bf5db9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc548ff9640 0 client.19479 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 --2- 172.21.15.122:0/474021229 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db32c0 0x562bf5db9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 >> 172.21.15.122:0/474021229 conn(0x562bf5dad9f0 msgr2=0x562bf5daf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 shutdown_connections 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 1 -- 172.21.15.122:0/474021229 wait complete. 2024-04-02T13:48:22.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.270+0000 7fc56a001740 20 client.19479 trim_cache size 0 max 16384 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 Processor -- start 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 -- start start 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5dbbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5dbc7c0 0x562bf5dd42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601b1300 con 0x562bf5db3be0 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601b2e40 con 0x562bf5db32c0 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc56a001740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601b19d0 con 0x562bf5dbc7c0 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc5649b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5dbbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc5649b6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53134/0 (socket says 172.21.15.122:53134) 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.289+0000 7fc5649b6640 1 -- 172.21.15.122:0/1477047893 learned_addr learned my addr 172.21.15.122:0/1477047893 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5dbc7c0 0x562bf5dd42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 -- 172.21.15.122:0/1477047893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5dbc7c0 msgr2=0x562bf5dd42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5dbc7c0 0x562bf5dd42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 -- 172.21.15.122:0/1477047893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 msgr2=0x562bf5dbbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5dbbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 -- 172.21.15.122:0/1477047893 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bf5db2c90 con 0x562bf5db3be0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5dbbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc560102400 tx=0x7fc55000b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f07b4ba49b70513 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc54affd640 1 -- 172.21.15.122:0/1477047893 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550004220 con 0x562bf5db3be0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc54affd640 1 -- 172.21.15.122:0/1477047893 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc550024070 con 0x562bf5db3be0 2024-04-02T13:48:22.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.290+0000 7fc54affd640 1 -- 172.21.15.122:0/1477047893 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550020160 con 0x562bf5db3be0 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 -- 172.21.15.122:0/1477047893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 msgr2=0x562bf5dc8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fc560102400 tx=0x7fc55000b410 comp rx=0 tx=0).stop 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 -- 172.21.15.122:0/1477047893 shutdown_connections 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5dbc7c0 0x562bf5dd42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db3be0 0x562bf5dc8410 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 --2- 172.21.15.122:0/1477047893 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5db32c0 0x562bf5dbbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 -- 172.21.15.122:0/1477047893 >> 172.21.15.122:0/1477047893 conn(0x562bf5dad9f0 msgr2=0x562bf5dacab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 -- 172.21.15.122:0/1477047893 shutdown_connections 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.291+0000 7fc56a001740 1 -- 172.21.15.122:0/1477047893 wait complete. 2024-04-02T13:48:22.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 -- start start 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 Processor -- start 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dc4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5dbc7c0 0x562bf5dc0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601b2770 con 0x562bf5db32c0 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601b20a0 con 0x562bf5dbc7c0 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5601af300 con 0x562bf5db3be0 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc56a001740 10 client.0 _ll_register_callbacks cb 0x7fc55000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53138/0 (socket says 172.21.15.122:53138) 2024-04-02T13:48:22.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 -- 172.21.15.122:0/1284143139 learned_addr learned my addr 172.21.15.122:0/1284143139 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dc4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5dbc7c0 0x562bf5dc0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 -- 172.21.15.122:0/1284143139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 msgr2=0x562bf5dc4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dc4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 -- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5dbc7c0 msgr2=0x562bf5dc0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5dbc7c0 0x562bf5dc0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.292+0000 7fc5659b8640 1 -- 172.21.15.122:0/1284143139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bf5dc16e0 con 0x562bf5db32c0 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc5659b8640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc5580045c0 tx=0x7fc55804a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aaa942c7f24048e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc558043070 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fc55804d040 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc550024070 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc558033e50 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 10 client.19485 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562bf5e33810 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc550024070 con 0x562bf5db32c0 2024-04-02T13:48:22.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fc558034550 con 0x562bf5db32c0 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 10 client.19485 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 10 client.19485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 20 client.19485 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5e33e20 0x562bf5e36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 4 client.19485 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562bf5d1f000 con 0x562bf5e33e20 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.293+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5e33e20 0x562bf5e36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.294+0000 7fc548ff9640 10 client.19485 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.294+0000 7fc5649b6640 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5e33e20 0x562bf5e36210 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4450ae3d08b4c8b server_cookie=c6612b7491568927 in_seq=0 out_seq=0 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.295+0000 7fc548ff9640 1 -- 172.21.15.122:0/1284143139 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562bf5d1f000 con 0x562bf5e33e20 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.295+0000 7fc548ff9640 10 client.19485 handle_command_reply: tid=0 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.295+0000 7fc56a001740 1 client.19485 shutdown 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 msgr2=0x562bf5dc4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc5580045c0 tx=0x7fc55804a040 comp rx=0 tx=0).stop 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 shutdown_connections 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562bf5e33e20 0x562bf5e36210 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bf5dbc7c0 0x562bf5dc0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bf5db3be0 0x562bf5dc4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc548ff9640 0 client.19485 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 --2- 172.21.15.122:0/1284143139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bf5db32c0 0x562bf5dc4960 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 >> 172.21.15.122:0/1284143139 conn(0x562bf5dad9f0 msgr2=0x562bf5db9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 shutdown_connections 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 1 -- 172.21.15.122:0/1284143139 wait complete. 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56a001740 20 client.19485 trim_cache size 0 max 16384 2024-04-02T13:48:22.305 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:22.306 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:48:22.306 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "029fce59-af41-4a1b-b386-af7e008ac71d", 2024-04-02T13:48:22.306 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:48:22.306 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 msgr2=0x7fc54c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 0x7fc54c0a6210 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc55c0ce210 tx=0x7fc55c0bd040 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 msgr2=0x7fc5601009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e9c0 tx=0x7fc55c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 shutdown_connections 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc54c0a3d50 0x7fc54c0a6210 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc560107b50 0x7fc560101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc5600696c0 0x7fc560100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 --2- 172.21.15.122:0/767441417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc560068d00 0x7fc5601009c0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.296+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 >> 172.21.15.122:0/767441417 conn(0x7fc560077b00 msgr2=0x7fc5600fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.297+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 shutdown_connections 2024-04-02T13:48:22.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.297+0000 7fc56741b640 1 -- 172.21.15.122:0/767441417 wait complete. 2024-04-02T13:48:22.311 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:22.311 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4166065567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 msgr2=0x7f7fd4106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4166065567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd4106e80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4166065567 shutdown_connections 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4166065567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4166065567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd4100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4166065567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd4106e80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4166065567 >> 172.21.15.122:0/4166065567 conn(0x7f7fd4076f20 msgr2=0x7f7fd40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4166065567 shutdown_connections 2024-04-02T13:48:22.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4166065567 wait complete. 2024-04-02T13:48:22.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.439+0000 7f7fdb5e9640 1 Processor -- start 2024-04-02T13:48:22.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 -- start start 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4111f70 con 0x7f7fd40fffc0 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4111df0 con 0x7f7fd4106aa0 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fdb5e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41120f0 con 0x7f7fd4100980 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53154/0 (socket says 172.21.15.122:53154) 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 -- 172.21.15.122:0/4185410483 learned_addr learned my addr 172.21.15.122:0/4185410483 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 -- 172.21.15.122:0/4185410483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 msgr2=0x7f7fd4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 -- 172.21.15.122:0/4185410483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 msgr2=0x7f7fd419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 -- 172.21.15.122:0/4185410483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd419d4c0 con 0x7f7fd40fffc0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.440+0000 7f7fd9385640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0005940 tx=0x7f7fd0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.441+0000 7f7fca7fc640 1 -- 172.21.15.122:0/4185410483 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0041070 con 0x7f7fd40fffc0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.441+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd419d750 con 0x7f7fd40fffc0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.441+0000 7f7fca7fc640 1 -- 172.21.15.122:0/4185410483 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd00054e0 con 0x7f7fd40fffc0 2024-04-02T13:48:22.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.441+0000 7f7fca7fc640 1 -- 172.21.15.122:0/4185410483 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0003c30 con 0x7f7fd40fffc0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.441+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd41a5040 con 0x7f7fd40fffc0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.443+0000 7f7fde1cd740 1 Processor -- start 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.443+0000 7f7fde1cd740 1 -- start start 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fca7fc640 1 -- 172.21.15.122:0/4185410483 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7fd0044070 con 0x7f7fd40fffc0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a39a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a39d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f66a2f5830 con 0x55f66a39add0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f66a29d820 con 0x55f66a3921f0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f66a2f17e0 con 0x55f66a3918d0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9385640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a39a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36040/0 (socket says 172.21.15.122:36040) 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a39d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 -- 172.21.15.122:0/3734179177 learned_addr learned my addr 172.21.15.122:0/3734179177 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 -- 172.21.15.122:0/3734179177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 msgr2=0x55f66a39a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a39a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 -- 172.21.15.122:0/3734179177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 msgr2=0x55f66a39d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a39d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 -- 172.21.15.122:0/3734179177 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66a0d37d0 con 0x55f66a3918d0 2024-04-02T13:48:22.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9385640 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a39a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fca7fc640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 0x7f7fc00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc011790 tx=0x7f7fcc011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9c3f5daf9b3f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fd9385640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 0x7f7fc00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fca7fc640 1 -- 172.21.15.122:0/4185410483 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f7fd00f26a0 con 0x7f7fd40fffc0 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.444+0000 7f7fbf7fe640 1 -- 172.21.15.122:0/3734179177 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004d20 con 0x55f66a3918d0 2024-04-02T13:48:22.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fbf7fe640 1 -- 172.21.15.122:0/3734179177 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc003970 con 0x55f66a3918d0 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fbf7fe640 1 -- 172.21.15.122:0/3734179177 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004520 con 0x55f66a3918d0 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 -- 172.21.15.122:0/3734179177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 msgr2=0x55f66a391cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fd9385640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 0x7f7fc00a61e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00efbc0 tx=0x7f7fd00f8550 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc011790 tx=0x7f7fcc011c50 comp rx=0 tx=0).stop 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 -- 172.21.15.122:0/3734179177 shutdown_connections 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a39d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a39a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/3734179177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a391cb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 -- 172.21.15.122:0/3734179177 >> 172.21.15.122:0/3734179177 conn(0x55f66a38c000 msgr2=0x55f66a38c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 -- 172.21.15.122:0/3734179177 shutdown_connections 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.445+0000 7f7fde1cd740 1 -- 172.21.15.122:0/3734179177 wait complete. 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 -- start start 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 Processor -- start 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a39a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a3979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f66a39dbc0 con 0x55f66a39add0 2024-04-02T13:48:22.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f66a2b0d30 con 0x55f66a3921f0 2024-04-02T13:48:22.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f66a39dd40 con 0x55f66a3918d0 2024-04-02T13:48:22.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fde1cd740 10 client.0 _ll_register_callbacks cb 0x55f66a2b0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:22.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a3979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53168/0 (socket says 172.21.15.122:53168) 2024-04-02T13:48:22.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fd9b86640 1 -- 172.21.15.122:0/2418552853 learned_addr learned my addr 172.21.15.122:0/2418552853 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.446+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a39a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 -- 172.21.15.122:0/2418552853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 msgr2=0x55f66a39a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a39a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 -- 172.21.15.122:0/2418552853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 msgr2=0x55f66a397ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 -- 172.21.15.122:0/2418552853 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66a0093f0 con 0x55f66a3921f0 2024-04-02T13:48:22.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:22.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a3979b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40ff610 tx=0x7f7fc400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb61bcb10dd8c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:48:22.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4012110 con 0x55f66a3921f0 2024-04-02T13:48:22.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f7fc4004050 con 0x55f66a3921f0 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc003970 con 0x55f66a3921f0 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7fc40122b0 con 0x55f66a3921f0 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fde1cd740 10 client.28499 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f669fc90f0 con 0x55f66a3921f0 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.447+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc003970 con 0x55f66a3921f0 2024-04-02T13:48:22.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f7fc4010430 con 0x55f66a3921f0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 10 client.28499 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 10 client.28499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 20 client.28499 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a3aada0 0x55f66a3ad190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 4 client.28499 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66a302d70 con 0x55f66a3aada0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a3aada0 0x55f66a3ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fbcff9640 10 client.28499 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.448+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a3aada0 0x55f66a3ad190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce55da3993b1e6c server_cookie=6b538765c46fca2c in_seq=0 out_seq=0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2418552853 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f66a302d70 con 0x55f66a3aada0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fbcff9640 10 client.28499 handle_command_reply: tid=0 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 client.28499 shutdown 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 msgr2=0x55f66a3979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a3979b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40ff610 tx=0x7f7fc400f790 comp rx=0 tx=0).stop 2024-04-02T13:48:22.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 shutdown_connections 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a3aada0 0x55f66a3ad190 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a397ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3921f0 0x55f66a3979b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fbcff9640 0 client.28499 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2418552853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3918d0 0x55f66a39a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 >> 172.21.15.122:0/2418552853 conn(0x55f66a38c000 msgr2=0x55f66a38dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 shutdown_connections 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2418552853 wait complete. 2024-04-02T13:48:22.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.449+0000 7f7fde1cd740 20 client.28499 trim_cache size 0 max 16384 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 Processor -- start 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 -- start start 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a3a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 0x55f66a397f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41af0e0 con 0x55f66a3921f0 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41b0c20 con 0x55f66a3918d0 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41af7b0 con 0x55f66a39add0 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53178/0 (socket says 172.21.15.122:53178) 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 -- 172.21.15.122:0/961094667 learned_addr learned my addr 172.21.15.122:0/961094667 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a3a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 0x55f66a397f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 -- 172.21.15.122:0/961094667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 msgr2=0x55f66a397f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 0x55f66a397f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 -- 172.21.15.122:0/961094667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 msgr2=0x55f66a3a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a3a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd8b84640 1 -- 172.21.15.122:0/961094667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66a392020 con 0x55f66a3921f0 2024-04-02T13:48:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.468+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 0x55f66a397f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40108d0 tx=0x7f7fc40127f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f72ea7945354bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fbeffd640 1 -- 172.21.15.122:0/961094667 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40104f0 con 0x55f66a3921f0 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fbeffd640 1 -- 172.21.15.122:0/961094667 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc402e070 con 0x55f66a3921f0 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fbeffd640 1 -- 172.21.15.122:0/961094667 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40038e0 con 0x55f66a3921f0 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 -- 172.21.15.122:0/961094667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 msgr2=0x55f66a3927a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40108d0 tx=0x7f7fc40127f0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 -- 172.21.15.122:0/961094667 shutdown_connections 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a39add0 0x55f66a397f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a3921f0 0x55f66a3927a0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/961094667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a3a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.469+0000 7f7fde1cd740 1 -- 172.21.15.122:0/961094667 >> 172.21.15.122:0/961094667 conn(0x55f66a38c000 msgr2=0x55f66a38b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- 172.21.15.122:0/961094667 shutdown_connections 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- 172.21.15.122:0/961094667 wait complete. 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- start start 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 Processor -- start 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a394700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3921f0 0x55f66a394c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41b0550 con 0x55f66a39add0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41afe80 con 0x55f66a3918d0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41ad0e0 con 0x55f66a3921f0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.470+0000 7f7fde1cd740 10 client.0 _ll_register_callbacks cb 0x7f7fc4012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd8b84640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3921f0 0x55f66a394c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53194/0 (socket says 172.21.15.122:53194) 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 -- 172.21.15.122:0/2687891909 learned_addr learned my addr 172.21.15.122:0/2687891909 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a394700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 -- 172.21.15.122:0/2687891909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3921f0 msgr2=0x55f66a394c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3921f0 0x55f66a394c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 -- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 msgr2=0x55f66a394700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a394700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 -- 172.21.15.122:0/2687891909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66a410350 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fd9b86640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc011730 tx=0x7f7fcc011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c740ab21b28732 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc01d040 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f7fcc005030 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc402e070 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7fcc005230 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fde1cd740 10 client.19503 fetch_fsmap learned FSMap version 43 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.471+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f66a411660 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc0058b0 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f7fcc023070 con 0x55f66a39add0 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 10 client.19503 fetch_fsmap finished waiting for FSMap version 43 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 20 client.19503 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:22.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a411bf0 0x55f66a413fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 4 client.19503 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7fc402e070 con 0x55f66a411bf0 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a411bf0 0x55f66a413fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.472+0000 7f7fbcff9640 10 client.19503 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fd8b84640 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a411bf0 0x55f66a413fe0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbebff2095e5ee1 server_cookie=3a4c2a9653d0121a in_seq=0 out_seq=0 2024-04-02T13:48:22.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fbcff9640 1 -- 172.21.15.122:0/2687891909 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f7fc402e070 con 0x55f66a411bf0 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fbcff9640 10 client.19503 handle_command_reply: tid=0 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 client.19503 shutdown 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 msgr2=0x55f66a395180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc011730 tx=0x7f7fcc011c50 comp rx=0 tx=0).stop 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 shutdown_connections 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f66a411bf0 0x55f66a413fe0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f66a39add0 0x55f66a395180 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f66a3921f0 0x55f66a394c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 --2- 172.21.15.122:0/2687891909 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f66a3918d0 0x55f66a394700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 >> 172.21.15.122:0/2687891909 conn(0x55f66a38c000 msgr2=0x55f66a38a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fbcff9640 0 client.19503 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 shutdown_connections 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 1 -- 172.21.15.122:0/2687891909 wait complete. 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.473+0000 7f7fde1cd740 20 client.19503 trim_cache size 0 max 16384 2024-04-02T13:48:22.485 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "029fce59-af41-4a1b-b386-af7e008ac71d": { 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "029fce59-af41-4a1b-b386-af7e008ac71d", 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:48:22.486 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 msgr2=0x7f7fc00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 0x7f7fc00a61e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00efbc0 tx=0x7f7fd00f8550 comp rx=0 tx=0).stop 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 msgr2=0x7f7fd41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0005940 tx=0x7f7fd0004270 comp rx=0 tx=0).stop 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 shutdown_connections 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7fc00a3d20 0x7f7fc00a61e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7fd4106aa0 0x7f7fd419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd4100980 0x7f7fd4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 --2- 172.21.15.122:0/4185410483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fd40fffc0 0x7f7fd41984d0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:22.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 >> 172.21.15.122:0/4185410483 conn(0x7f7fd4076f20 msgr2=0x7f7fd4104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:22.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 shutdown_connections 2024-04-02T13:48:22.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:22.474+0000 7f7fdb5e9640 1 -- 172.21.15.122:0/4185410483 wait complete. 2024-04-02T13:48:22.489 INFO:tasks.cephfs.filesystem:scrub status for tag:029fce59-af41-4a1b-b386-af7e008ac71d - {'path': '/', 'tag': '029fce59-af41-4a1b-b386-af7e008ac71d', 'options': 'recursive,force'} 2024-04-02T13:48:22.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 582486 4.49 MB/sec execute 583 sec latency 28.791 ms 2024-04-02T13:48:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: pgmap v413: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:23.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: pgmap v413: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:48:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:23.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: pgmap v413: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:23.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 583564 4.49 MB/sec execute 584 sec latency 37.223 ms 2024-04-02T13:48:24.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 585551 4.50 MB/sec execute 585 sec latency 22.785 ms 2024-04-02T13:48:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:25 smithi187 ceph-mon[36064]: pgmap v414: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:25 smithi188 ceph-mon[36117]: pgmap v414: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:25 smithi122 ceph-mon[32114]: pgmap v414: 129 pgs: 129 active+clean; 573 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.9 MiB/s wr, 271 op/s 2024-04-02T13:48:25.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 586985 4.51 MB/sec execute 586 sec latency 18.292 ms 2024-04-02T13:48:26.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 589063 4.52 MB/sec execute 587 sec latency 18.327 ms 2024-04-02T13:48:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:27 smithi187 ceph-mon[36064]: pgmap v415: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-04-02T13:48:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:27 smithi188 ceph-mon[36117]: pgmap v415: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-04-02T13:48:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:27 smithi122 ceph-mon[32114]: pgmap v415: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.2 MiB/s wr, 283 op/s 2024-04-02T13:48:27.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 590342 4.52 MB/sec execute 588 sec latency 59.576 ms 2024-04-02T13:48:28.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 592051 4.53 MB/sec execute 589 sec latency 43.185 ms 2024-04-02T13:48:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:29 smithi187 ceph-mon[36064]: pgmap v416: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-04-02T13:48:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:29 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:29 smithi188 ceph-mon[36117]: pgmap v416: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-04-02T13:48:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:29 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:29 smithi122 ceph-mon[32114]: pgmap v416: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.6 MiB/s wr, 242 op/s 2024-04-02T13:48:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:29 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:48:29.981 INFO:tasks.workunit.client.0.smithi122.stdout: 1 593533 4.54 MB/sec execute 590 sec latency 30.425 ms 2024-04-02T13:48:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:30.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:48:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:30.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:48:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:48:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:48:30.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:48:30.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 594832 4.54 MB/sec execute 591 sec latency 21.805 ms 2024-04-02T13:48:31.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 596945 4.56 MB/sec execute 592 sec latency 16.445 ms 2024-04-02T13:48:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:31 smithi187 ceph-mon[36064]: pgmap v417: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-04-02T13:48:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:31 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:48:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:31 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:48:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:31 smithi188 ceph-mon[36117]: pgmap v417: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-04-02T13:48:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:31 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:48:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:31 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:48:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:31 smithi122 ceph-mon[32114]: pgmap v417: 129 pgs: 129 active+clean; 585 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 236 op/s 2024-04-02T13:48:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:31 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:48:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:31 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:48:32.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 598121 4.56 MB/sec execute 593 sec latency 20.673 ms 2024-04-02T13:48:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:48:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:48:32.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:32 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:48:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:48:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:48:33.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:32 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:48:33.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:48:33.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:48:33.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:32 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:48:33.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 600068 4.57 MB/sec execute 594 sec latency 28.678 ms 2024-04-02T13:48:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:33 smithi187 ceph-mon[36064]: pgmap v418: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-02T13:48:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:33 smithi188 ceph-mon[36117]: pgmap v418: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-02T13:48:34.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:33 smithi122 ceph-mon[32114]: pgmap v418: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 346 op/s 2024-04-02T13:48:34.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 601158 4.57 MB/sec execute 595 sec latency 25.357 ms 2024-04-02T13:48:35.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 603173 4.58 MB/sec execute 596 sec latency 23.851 ms 2024-04-02T13:48:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:35 smithi187 ceph-mon[36064]: pgmap v419: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-04-02T13:48:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:35 smithi188 ceph-mon[36117]: pgmap v419: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-04-02T13:48:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:35 smithi122 ceph-mon[32114]: pgmap v419: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 227 op/s 2024-04-02T13:48:36.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 604594 4.58 MB/sec execute 597 sec latency 14.506 ms 2024-04-02T13:48:37.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 606578 4.59 MB/sec execute 598 sec latency 17.843 ms 2024-04-02T13:48:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:37 smithi187 ceph-mon[36064]: pgmap v420: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-02T13:48:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:37 smithi188 ceph-mon[36117]: pgmap v420: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-02T13:48:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:37 smithi122 ceph-mon[32114]: pgmap v420: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 329 op/s 2024-04-02T13:48:38.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 607860 4.60 MB/sec execute 599 sec latency 26.844 ms 2024-04-02T13:48:38.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:38 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:38 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:39.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:38 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:39.982 INFO:tasks.workunit.client.0.smithi122.stdout: 1 cleanup 600 sec 2024-04-02T13:48:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:39 smithi187 ceph-mon[36064]: pgmap v421: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:39 smithi188 ceph-mon[36117]: pgmap v421: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:40.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:39 smithi122 ceph-mon[32114]: pgmap v421: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:40.217 INFO:tasks.workunit.client.0.smithi122.stdout: 0 cleanup 600 sec 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Operation Count AvgLat MaxLat 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: ---------------------------------------- 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: NTCreateX 87984 4.600 4960.628 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Close 64646 0.002 0.059 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Rename 3729 6.392 59.801 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Unlink 17756 2.513 3773.071 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Deltree 2 96.365 139.793 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Mkdir 1 0.005 0.005 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Qpathinfo 79740 1.065 620.669 2024-04-02T13:48:40.218 INFO:tasks.workunit.client.0.smithi122.stdout: Qfileinfo 13972 0.001 0.027 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: Qfsinfo 14620 0.172 5.612 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: Sfileinfo 7184 0.013 6.991 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: Find 30828 0.202 22.011 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: WriteX 43846 0.192 73.649 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: ReadX 137897 0.012 3.480 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: LockX 286 0.529 2.306 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: UnlockX 286 0.563 6.694 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: Flush 6176 3.703 127.328 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stdout:Throughput 4.59992 MB/sec 1 clients 1 procs max_latency=4960.638 ms 2024-04-02T13:48:40.219 INFO:tasks.workunit.client.0.smithi122.stderr:+ dbench 10 2024-04-02T13:48:40.220 INFO:tasks.workunit.client.0.smithi122.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-02T13:48:40.220 INFO:tasks.workunit.client.0.smithi122.stdout: 2024-04-02T13:48:40.220 INFO:tasks.workunit.client.0.smithi122.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-02T13:48:40.220 INFO:tasks.workunit.client.0.smithi122.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-02T13:48:40.319 INFO:tasks.workunit.client.0.smithi122.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-02T13:48:40.319 INFO:tasks.workunit.client.0.smithi122.stdout:releasing clients 2024-04-02T13:48:41.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 340 122.65 MB/sec warmup 1 sec latency 507.042 ms 2024-04-02T13:48:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:41 smithi187 ceph-mon[36064]: pgmap v422: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:41 smithi188 ceph-mon[36117]: pgmap v422: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:41 smithi122 ceph-mon[32114]: pgmap v422: 129 pgs: 129 active+clean; 608 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 211 op/s 2024-04-02T13:48:42.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 723 99.59 MB/sec warmup 2 sec latency 1507.092 ms 2024-04-02T13:48:43.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1147 72.43 MB/sec warmup 3 sec latency 2507.118 ms 2024-04-02T13:48:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:43 smithi187 ceph-mon[36064]: pgmap v423: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 317 op/s 2024-04-02T13:48:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:43 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:43 smithi188 ceph-mon[36117]: pgmap v423: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 317 op/s 2024-04-02T13:48:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:43 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:44.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:43 smithi122 ceph-mon[32114]: pgmap v423: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.7 MiB/s wr, 317 op/s 2024-04-02T13:48:44.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:43 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:44.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 1658 60.91 MB/sec warmup 4 sec latency 3507.151 ms 2024-04-02T13:48:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:44 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:44 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:45.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:44 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:45.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 2277 53.09 MB/sec warmup 5 sec latency 4507.197 ms 2024-04-02T13:48:45.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:45 smithi122 ceph-mon[32114]: pgmap v424: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-02T13:48:45.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:45 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:45 smithi187 ceph-mon[36064]: pgmap v424: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-02T13:48:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:45 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:45 smithi188 ceph-mon[36117]: pgmap v424: 129 pgs: 129 active+clean; 616 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 207 op/s 2024-04-02T13:48:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:45 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:46.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 2869 52.37 MB/sec warmup 6 sec latency 5507.244 ms 2024-04-02T13:48:47.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 3314 50.68 MB/sec warmup 7 sec latency 6507.276 ms 2024-04-02T13:48:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:47 smithi187 ceph-mon[36064]: pgmap v425: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-04-02T13:48:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:47 smithi188 ceph-mon[36117]: pgmap v425: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-04-02T13:48:48.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:47 smithi122 ceph-mon[32114]: pgmap v425: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.0 MiB/s wr, 307 op/s 2024-04-02T13:48:48.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 3500 45.59 MB/sec warmup 8 sec latency 7507.318 ms 2024-04-02T13:48:49.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 3655 41.29 MB/sec warmup 9 sec latency 7619.804 ms 2024-04-02T13:48:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:49 smithi187 ceph-mon[36064]: pgmap v426: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:49 smithi188 ceph-mon[36117]: pgmap v426: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:50.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:49 smithi122 ceph-mon[32114]: pgmap v426: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:50.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 3806 37.47 MB/sec warmup 10 sec latency 179.165 ms 2024-04-02T13:48:51.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 4108 34.91 MB/sec warmup 11 sec latency 251.937 ms 2024-04-02T13:48:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:51 smithi187 ceph-mon[36064]: pgmap v427: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:51 smithi188 ceph-mon[36117]: pgmap v427: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:51 smithi122 ceph-mon[32114]: pgmap v427: 129 pgs: 129 active+clean; 637 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 205 op/s 2024-04-02T13:48:52.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 4545 35.06 MB/sec warmup 12 sec latency 133.785 ms 2024-04-02T13:48:52.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:52.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.598+0000 7fb50c286640 1 -- 172.21.15.122:0/2657565293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504107ab0 msgr2=0x7fb504107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.598+0000 7fb50c286640 1 --2- 172.21.15.122:0/2657565293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504107ab0 0x7fb504107e90 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009930 tx=0x7fb4f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- 172.21.15.122:0/2657565293 shutdown_connections 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- 172.21.15.122:0/2657565293 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504102460 0x7fb50410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- 172.21.15.122:0/2657565293 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504101aa0 0x7fb504101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- 172.21.15.122:0/2657565293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504107ab0 0x7fb504107e90 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- 172.21.15.122:0/2657565293 >> 172.21.15.122:0/2657565293 conn(0x7fb5040fd590 msgr2=0x7fb5040ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- 172.21.15.122:0/2657565293 shutdown_connections 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- 172.21.15.122:0/2657565293 wait complete. 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 Processor -- start 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- start start 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 0x7fb50419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504107ab0 0x7fb50419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb504111ec0 con 0x7fb504101aa0 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb504111d40 con 0x7fb504107ab0 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50c286640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb504112040 con 0x7fb504102460 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50a022640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50a022640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33490/0 (socket says 172.21.15.122:33490) 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50a022640 1 -- 172.21.15.122:0/295328503 learned_addr learned my addr 172.21.15.122:0/295328503 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb509821640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 0x7fb50419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.599+0000 7fb50a823640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504107ab0 0x7fb50419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 -- 172.21.15.122:0/295328503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 msgr2=0x7fb50419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 0x7fb50419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 -- 172.21.15.122:0/295328503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504107ab0 msgr2=0x7fb50419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504107ab0 0x7fb50419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 -- 172.21.15.122:0/295328503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50419f5f0 con 0x7fb504101aa0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb509821640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 0x7fb50419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50a022640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009230 tx=0x7fb4f402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb4fb7fe640 1 -- 172.21.15.122:0/295328503 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4041070 con 0x7fb504101aa0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50419f880 con 0x7fb504101aa0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb4fb7fe640 1 -- 172.21.15.122:0/295328503 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f4004450 con 0x7fb504101aa0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb4fb7fe640 1 -- 172.21.15.122:0/295328503 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4003e50 con 0x7fb504101aa0 2024-04-02T13:48:52.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.600+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5041a7170 con 0x7fb504101aa0 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.602+0000 7fb4fb7fe640 1 -- 172.21.15.122:0/295328503 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb4f4044070 con 0x7fb504101aa0 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 Processor -- start 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 -- start start 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb4fb7fe640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 0x7fb4f00a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588f032d830 con 0x5588f03d2dd0 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5588f02d5820 con 0x5588f03ca1f0 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5588f03297e0 con 0x5588f03c98d0 2024-04-02T13:48:52.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50a022640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50a823640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb509821640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 0x7fb4f00a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50a823640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48394/0 (socket says 172.21.15.122:48394) 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50a823640 1 -- 172.21.15.122:0/2070572532 learned_addr learned my addr 172.21.15.122:0/2070572532 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.603+0000 7fb50a823640 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb4fb7fe640 1 -- 172.21.15.122:0/295328503 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb4f40f2380 con 0x7fb504101aa0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 -- 172.21.15.122:0/2070572532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 msgr2=0x5588f03d2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 -- 172.21.15.122:0/2070572532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 msgr2=0x5588f03d51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 -- 172.21.15.122:0/2070572532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f010b7d0 con 0x5588f03c98d0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a022640 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb50a823640 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005f40 tx=0x7fb4fc005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce697af1a497630 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb509821640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 0x7fb4f00a6380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb50410ee00 tx=0x7fb500005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb4f8bfe640 1 -- 172.21.15.122:0/2070572532 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004b20 con 0x5588f03c98d0 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb4f8bfe640 1 -- 172.21.15.122:0/2070572532 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc004cc0 con 0x5588f03c98d0 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.604+0000 7fb4f8bfe640 1 -- 172.21.15.122:0/2070572532 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc004030 con 0x5588f03c98d0 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2070572532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 msgr2=0x5588f03c9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc005f40 tx=0x7fb4fc005bd0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2070572532 shutdown_connections 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2070572532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03c9cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2070572532 >> 172.21.15.122:0/2070572532 conn(0x5588f03c4000 msgr2=0x5588f03c43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2070572532 shutdown_connections 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2070572532 wait complete. 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- start start 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 Processor -- start 2024-04-02T13:48:52.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5588f03d5bc0 con 0x5588f03d2dd0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5588f02e8d30 con 0x5588f03ca1f0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.605+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5588f03d5d40 con 0x5588f03c98d0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50ee6f740 10 client.0 _ll_register_callbacks cb 0x5588f02e8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48406/0 (socket says 172.21.15.122:48406) 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb509821640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 -- 172.21.15.122:0/4087154845 learned_addr learned my addr 172.21.15.122:0/4087154845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33502/0 (socket says 172.21.15.122:33502) 2024-04-02T13:48:52.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 -- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 msgr2=0x5588f03cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 -- 172.21.15.122:0/4087154845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 msgr2=0x5588f03cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 -- 172.21.15.122:0/4087154845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f00413f0 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a823640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50a022640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f4009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd410f4d9faa50b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f404b040 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb4f40ba760 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4fc004cc0 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40bda50 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50ee6f740 10 client.28270 fetch_fsmap learned FSMap version 47 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.606+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5588f00010f0 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4fc004cc0 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7fb4f40ef960 con 0x5588f03c98d0 2024-04-02T13:48:52.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 10 client.28270 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 10 client.28270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 20 client.28270 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f03e2bf0 0x5588f03e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 4 client.28270 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588f033ad70 con 0x5588f03e2bf0 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb509821640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f03e2bf0 0x5588f03e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb4edffb640 10 client.28270 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:52.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.607+0000 7fb509821640 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f03e2bf0 0x5588f03e5040 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6a376c9f10c7cec server_cookie=d12e88c5477400a6 in_seq=0 out_seq=0 2024-04-02T13:48:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb4edffb640 1 -- 172.21.15.122:0/4087154845 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588f033ad70 con 0x5588f03e2bf0 2024-04-02T13:48:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb4edffb640 10 client.28270 handle_command_reply: tid=0 2024-04-02T13:48:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 client.28270 shutdown 2024-04-02T13:48:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 msgr2=0x5588f03d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4006010 tx=0x7fb4f4009620 comp rx=0 tx=0).stop 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 shutdown_connections 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f03e2bf0 0x5588f03e5040 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb4edffb640 0 client.28270 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/4087154845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d25c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 >> 172.21.15.122:0/4087154845 conn(0x5588f03c4000 msgr2=0x5588f03c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 shutdown_connections 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 1 -- 172.21.15.122:0/4087154845 wait complete. 2024-04-02T13:48:52.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.610+0000 7fb50ee6f740 20 client.28270 trim_cache size 0 max 16384 2024-04-02T13:48:52.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 Processor -- start 2024-04-02T13:48:52.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 -- start start 2024-04-02T13:48:52.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03d2dd0 0x5588f03d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b1340 con 0x5588f03c98d0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b2ee0 con 0x5588f03ca1f0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b1a10 con 0x5588f03d2dd0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33508/0 (socket says 172.21.15.122:33508) 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 -- 172.21.15.122:0/393282118 learned_addr learned my addr 172.21.15.122:0/393282118 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a823640 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03d2dd0 0x5588f03d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb509821640 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 -- 172.21.15.122:0/393282118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03d2dd0 msgr2=0x5588f03d2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03d2dd0 0x5588f03d2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 -- 172.21.15.122:0/393282118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 msgr2=0x5588f03cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 -- 172.21.15.122:0/393282118 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588f03c3e30 con 0x5588f03c98d0 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.634+0000 7fb50a022640 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ba550 tx=0x7fb4f402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f062023ba6e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb4effff640 1 -- 172.21.15.122:0/393282118 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f404b040 con 0x5588f03c98d0 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb4effff640 1 -- 172.21.15.122:0/393282118 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f4004c80 con 0x5588f03c98d0 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb4effff640 1 -- 172.21.15.122:0/393282118 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f40be5c0 con 0x5588f03c98d0 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 -- 172.21.15.122:0/393282118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 msgr2=0x5588f03c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fb4f40ba550 tx=0x7fb4f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 -- 172.21.15.122:0/393282118 shutdown_connections 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03d2dd0 0x5588f03d2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/393282118 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03c98d0 0x5588f03c3240 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 -- 172.21.15.122:0/393282118 >> 172.21.15.122:0/393282118 conn(0x5588f03c4000 msgr2=0x5588f03ca5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.635+0000 7fb50ee6f740 1 -- 172.21.15.122:0/393282118 shutdown_connections 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 -- 172.21.15.122:0/393282118 wait complete. 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 -- start start 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 Processor -- start 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b2810 con 0x5588f03d2dd0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041b2110 con 0x5588f03ca1f0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5041af340 con 0x5588f03c98d0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50ee6f740 10 client.0 _ll_register_callbacks cb 0x7fb4f4005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50a823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50a022640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50a022640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48414/0 (socket says 172.21.15.122:48414) 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50a823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33512/0 (socket says 172.21.15.122:33512) 2024-04-02T13:48:52.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb509821640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.636+0000 7fb50a022640 1 -- 172.21.15.122:0/2091057298 learned_addr learned my addr 172.21.15.122:0/2091057298 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 -- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 msgr2=0x5588f03d6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 -- 172.21.15.122:0/2091057298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 msgr2=0x5588f03d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:52.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 -- 172.21.15.122:0/2091057298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588f0448250 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a823640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50a022640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4f402c820 tx=0x7fb4f4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4a22339cefe44c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb4edffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f404b040 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb4f40ef8a0 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4f4004c80 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4f40be5d0 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50ee6f740 10 client.28276 fetch_fsmap learned FSMap version 47 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.637+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5588f0448250 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4f4004c80 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 47) ==== 8382+0+0 (secure 0 0 0) 0x7fb4f40b9e40 con 0x5588f03c98d0 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 10 client.28276 fetch_fsmap finished waiting for FSMap version 47 2024-04-02T13:48:52.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 10 client.28276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 20 client.28276 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f0449890 0x5588f044bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 4 client.28276 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.638+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5588f03350c0 con 0x5588f0449890 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.639+0000 7fb509821640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f0449890 0x5588f044bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.639+0000 7fb509821640 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f0449890 0x5588f044bc80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c532bb79f69cfa08 server_cookie=6b2a933d43a87ccd in_seq=0 out_seq=0 2024-04-02T13:48:52.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.639+0000 7fb4edffb640 10 client.28276 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb4edffb640 1 -- 172.21.15.122:0/2091057298 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5588f03350c0 con 0x5588f0449890 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb4edffb640 10 client.28276 handle_command_reply: tid=0 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 client.28276 shutdown 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 msgr2=0x5588f03d6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4f402c820 tx=0x7fb4f4009260 comp rx=0 tx=0).stop 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 shutdown_connections 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5588f0449890 0x5588f044bc80 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5588f03d2dd0 0x5588f03d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb4edffb640 0 client.28276 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5588f03ca1f0 0x5588f03d6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 --2- 172.21.15.122:0/2091057298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5588f03c98d0 0x5588f03d6a70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 >> 172.21.15.122:0/2091057298 conn(0x5588f03c4000 msgr2=0x5588f03cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 shutdown_connections 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 1 -- 172.21.15.122:0/2091057298 wait complete. 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.646+0000 7fb50ee6f740 20 client.28276 trim_cache size 0 max 16384 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:48:52.651 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 msgr2=0x7fb4f00a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 0x7fb4f00a6380 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb50410ee00 tx=0x7fb500005f90 comp rx=0 tx=0).stop 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 msgr2=0x7fb50419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009230 tx=0x7fb4f402cb30 comp rx=0 tx=0).stop 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 shutdown_connections 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4f00a3ec0 0x7fb4f00a6380 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb504107ab0 0x7fb50419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb504102460 0x7fb50419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 --2- 172.21.15.122:0/295328503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb504101aa0 0x7fb50419a600 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 >> 172.21.15.122:0/295328503 conn(0x7fb5040fd590 msgr2=0x7fb5040fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 shutdown_connections 2024-04-02T13:48:52.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.647+0000 7fb50c286640 1 -- 172.21.15.122:0/295328503 wait complete. 2024-04-02T13:48:52.662 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:48:52.662 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:48:52.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 -- 172.21.15.122:0/643071032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 msgr2=0x7f5d18100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 --2- 172.21.15.122:0/643071032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18100e80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009930 tx=0x7f5d0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 -- 172.21.15.122:0/643071032 shutdown_connections 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 --2- 172.21.15.122:0/643071032 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d181013c0 0x7f5d1810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 --2- 172.21.15.122:0/643071032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18100e80 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 --2- 172.21.15.122:0/643071032 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d18106a10 0x7f5d18106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 -- 172.21.15.122:0/643071032 >> 172.21.15.122:0/643071032 conn(0x7f5d18077b00 msgr2=0x7f5d18077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 -- 172.21.15.122:0/643071032 shutdown_connections 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 -- 172.21.15.122:0/643071032 wait complete. 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.773+0000 7f5d1e932640 1 Processor -- start 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 -- start start 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d181013c0 0x7f5d18197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d18106a10 0x7f5d1819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18110e20 con 0x7f5d18100a00 2024-04-02T13:48:52.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18110ca0 con 0x7f5d18106a10 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1e932640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18110fa0 con 0x7f5d181013c0 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33520/0 (socket says 172.21.15.122:33520) 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 -- 172.21.15.122:0/3378093154 learned_addr learned my addr 172.21.15.122:0/3378093154 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d1cecf640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d18106a10 0x7f5d1819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d177fe640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d181013c0 0x7f5d18197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 -- 172.21.15.122:0/3378093154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d181013c0 msgr2=0x7f5d18197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d181013c0 0x7f5d18197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 -- 172.21.15.122:0/3378093154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d18106a10 msgr2=0x7f5d1819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d18106a10 0x7f5d1819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 -- 172.21.15.122:0/3378093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1819c330 con 0x7f5d18100a00 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.774+0000 7f5d17fff640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9f0 tx=0x7f5d0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.775+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800cd80 con 0x7f5d18100a00 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.775+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1819c620 con 0x7f5d18100a00 2024-04-02T13:48:52.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.775+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d08005500 con 0x7f5d18100a00 2024-04-02T13:48:52.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.775+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08010690 con 0x7f5d18100a00 2024-04-02T13:48:52.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.775+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181a3eb0 con 0x7f5d18100a00 2024-04-02T13:48:52.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.776+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d180688f0 con 0x7f5d18100a00 2024-04-02T13:48:52.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.777+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5d080046c0 con 0x7f5d18100a00 2024-04-02T13:48:52.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.777+0000 7f5d157fa640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 0x7f5d000a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:52.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.778+0000 7f5d177fe640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 0x7f5d000a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:52.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.778+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f5d080d1130 con 0x7f5d18100a00 2024-04-02T13:48:52.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.778+0000 7f5d177fe640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 0x7f5d000a61e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:52.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:52.784+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5d08053f20 con 0x7f5d18100a00 2024-04-02T13:48:53.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.043+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d18108fa0 con 0x7f5d18100a00 2024-04-02T13:48:53.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.044+0000 7f5d157fa640 1 -- 172.21.15.122:0/3378093154 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11950 (secure 0 0 0) 0x7f5d0809d3e0 con 0x7f5d18100a00 2024-04-02T13:48:53.047 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:48:53.047 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:48:51.709986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:48:53.048 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 48 2024-04-02T13:48:53.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 msgr2=0x7f5d000a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 0x7f5d000a61e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 msgr2=0x7f5d18197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9f0 tx=0x7f5d0800edf0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 shutdown_connections 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d000a3d20 0x7f5d000a61e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d18106a10 0x7f5d1819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d181013c0 0x7f5d18197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 --2- 172.21.15.122:0/3378093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d18100a00 0x7f5d18197340 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 >> 172.21.15.122:0/3378093154 conn(0x7f5d18077b00 msgr2=0x7f5d181033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 shutdown_connections 2024-04-02T13:48:53.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.046+0000 7f5d1e932640 1 -- 172.21.15.122:0/3378093154 wait complete. 2024-04-02T13:48:53.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2103459024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c101360 msgr2=0x7f6f3c10d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2103459024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c101360 0x7f6f3c10d820 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009ca0 tx=0x7f6f3402c180 comp rx=0 tx=0).stop 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2103459024 shutdown_connections 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2103459024 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c101360 0x7f6f3c10d820 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2103459024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c1009c0 0x7f6f3c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2103459024 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2103459024 >> 172.21.15.122:0/2103459024 conn(0x7f6f3c077a90 msgr2=0x7f6f3c077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2103459024 shutdown_connections 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.181+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2103459024 wait complete. 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 Processor -- start 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 -- start start 2024-04-02T13:48:53.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1009c0 0x7f6f3c197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 0x7f6f3c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c110d90 con 0x7f6f3c1058a0 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c110c10 con 0x7f6f3c1009c0 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f43c0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c110f10 con 0x7f6f3c101360 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f411a8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 0x7f6f3c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33536/0 (socket says 172.21.15.122:33536) 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f421aa640 1 -- 172.21.15.122:0/2190300612 learned_addr learned my addr 172.21.15.122:0/2190300612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.182+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1009c0 0x7f6f3c197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 -- 172.21.15.122:0/2190300612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 msgr2=0x7f6f3c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 0x7f6f3c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 -- 172.21.15.122:0/2190300612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1009c0 msgr2=0x7f6f3c197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1009c0 0x7f6f3c197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 -- 172.21.15.122:0/2190300612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3c19c200 con 0x7f6f3c1058a0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 0x7f6f3c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6f3402efd0 tx=0x7f6f34035d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f32ffd640 1 -- 172.21.15.122:0/2190300612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34044020 con 0x7f6f3c1058a0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3c19c490 con 0x7f6f3c1058a0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f32ffd640 1 -- 172.21.15.122:0/2190300612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f34003b70 con 0x7f6f3c1058a0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f32ffd640 1 -- 172.21.15.122:0/2190300612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f34039040 con 0x7f6f3c1058a0 2024-04-02T13:48:53.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.183+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f3c069e20 con 0x7f6f3c1058a0 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.185+0000 7f6f32ffd640 1 -- 172.21.15.122:0/2190300612 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6f34047020 con 0x7f6f3c1058a0 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 Processor -- start 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 -- start start 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f32ffd640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 0x7f6f280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 0x55c312e94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94d70 0x55c312ea1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c312df87b0 con 0x55c312e94d70 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c312d97570 con 0x55c312ea20b0 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f32ffd640 1 -- 172.21.15.122:0/2190300612 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6f340f53f0 con 0x7f6f3c1058a0 2024-04-02T13:48:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f467f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c312df4760 con 0x55c312e94450 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.186+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 0x7f6f280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49290/0 (socket says 172.21.15.122:49290) 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 -- 172.21.15.122:0/1646596295 learned_addr learned my addr 172.21.15.122:0/1646596295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f419a9640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 0x55c312e94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f411a8640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94d70 0x55c312ea1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 -- 172.21.15.122:0/1646596295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 msgr2=0x55c312e94830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 0x55c312e94830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 -- 172.21.15.122:0/1646596295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94d70 msgr2=0x55c312ea1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94d70 0x55c312ea1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 -- 172.21.15.122:0/1646596295 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c312b0c3f0 con 0x55c312ea20b0 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f419a9640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 0x55c312e94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 0x7f6f280a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009710 tx=0x7f6f2c011ca0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f421aa640 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f340bda80 tx=0x7f6f340c0a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86417a7a509d72d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f27fff640 1 -- 172.21.15.122:0/1646596295 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340c10b0 con 0x55c312ea20b0 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f27fff640 1 -- 172.21.15.122:0/1646596295 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f340c5460 con 0x55c312ea20b0 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.187+0000 7f6f27fff640 1 -- 172.21.15.122:0/1646596295 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340f29d0 con 0x55c312ea20b0 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- 172.21.15.122:0/1646596295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 msgr2=0x55c312ea44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6f340bda80 tx=0x7f6f340c0a10 comp rx=0 tx=0).stop 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- 172.21.15.122:0/1646596295 shutdown_connections 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312ea44a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94d70 0x55c312ea1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 --2- 172.21.15.122:0/1646596295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94450 0x55c312e94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- 172.21.15.122:0/1646596295 >> 172.21.15.122:0/1646596295 conn(0x55c312e8eed0 msgr2=0x55c312e8f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- 172.21.15.122:0/1646596295 shutdown_connections 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- 172.21.15.122:0/1646596295 wait complete. 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.188+0000 7f6f467f2740 1 -- start start 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 Processor -- start 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94450 0x55c312ea1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c312ea4cb0 con 0x55c312ea20b0 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c312db3e80 con 0x55c312e94450 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c312ea4e30 con 0x55c312e94d70 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f467f2740 10 client.0 _ll_register_callbacks cb 0x55c312defaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33548/0 (socket says 172.21.15.122:33548) 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 -- 172.21.15.122:0/2920198980 learned_addr learned my addr 172.21.15.122:0/2920198980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f419a9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94450 0x55c312ea1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 -- 172.21.15.122:0/2920198980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 msgr2=0x55c312e9eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 -- 172.21.15.122:0/2920198980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94450 msgr2=0x55c312ea1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94450 0x55c312ea1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.189+0000 7f6f421aa640 1 -- 172.21.15.122:0/2920198980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c312bd67d0 con 0x55c312ea20b0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6f340c1040 tx=0x7f6f3402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ea44d9cefe2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f340c10b0 con 0x55c312ea20b0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6f34003d20 con 0x55c312ea20b0 2024-04-02T13:48:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f340c5460 con 0x55c312ea20b0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6f340bd720 con 0x55c312ea20b0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 10 client.19539 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c312ea4cb0 con 0x55c312ea20b0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f340c5460 con 0x55c312ea20b0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f6f34004b40 con 0x55c312ea20b0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 10 client.19539 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 10 client.19539 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 20 client.19539 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312eb43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 4 client.19539 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.190+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c312e05cf0 con 0x55c312eb1fa0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312eb43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f257fa640 10 client.19539 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312eb43c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea52a1cc84b22d1b server_cookie=449479c4cd3c0d3e in_seq=0 out_seq=0 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f257fa640 1 -- 172.21.15.122:0/2920198980 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c312e05cf0 con 0x55c312eb1fa0 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f257fa640 10 client.19539 handle_command_reply: tid=0 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.191+0000 7f6f467f2740 1 client.19539 shutdown 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 msgr2=0x55c312e9f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f6f340c1040 tx=0x7f6f3402c180 comp rx=0 tx=0).stop 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 shutdown_connections 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312eb43c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312ea20b0 0x55c312e9f5d0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f257fa640 0 client.19539 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2920198980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94450 0x55c312ea1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 >> 172.21.15.122:0/2920198980 conn(0x55c312e8eed0 msgr2=0x55c312e90c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 shutdown_connections 2024-04-02T13:48:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 1 -- 172.21.15.122:0/2920198980 wait complete. 2024-04-02T13:48:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.192+0000 7f6f467f2740 20 client.19539 trim_cache size 0 max 16384 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.205+0000 7f6f467f2740 1 Processor -- start 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 -- start start 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 0x55c312ea1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312ea20b0 0x55c312ea1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1ace70 con 0x55c312e94450 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1aea10 con 0x55c312e94d70 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f467f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1ad540 con 0x55c312ea20b0 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33564/0 (socket says 172.21.15.122:33564) 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 -- 172.21.15.122:0/2442473703 learned_addr learned my addr 172.21.15.122:0/2442473703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 0x55c312ea1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f421aa640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312ea20b0 0x55c312ea1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 -- 172.21.15.122:0/2442473703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312ea20b0 msgr2=0x55c312ea1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312ea20b0 0x55c312ea1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 -- 172.21.15.122:0/2442473703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 msgr2=0x55c312ea1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 0x55c312ea1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 -- 172.21.15.122:0/2442473703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c312eb4730 con 0x55c312e94450 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f411a8640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 0x55c312ea1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f419a9640 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000c00 tx=0x7f6f38005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6495dfd5c4dea3a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f277fe640 1 -- 172.21.15.122:0/2442473703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f380200f0 con 0x55c312e94450 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f277fe640 1 -- 172.21.15.122:0/2442473703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f38020770 con 0x55c312e94450 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.206+0000 7f6f277fe640 1 -- 172.21.15.122:0/2442473703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38010d60 con 0x55c312e94450 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- 172.21.15.122:0/2442473703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 msgr2=0x55c312e9ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000c00 tx=0x7f6f38005f90 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- 172.21.15.122:0/2442473703 shutdown_connections 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312ea20b0 0x55c312ea1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312e94d70 0x55c312ea1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 --2- 172.21.15.122:0/2442473703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312e9ee80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- 172.21.15.122:0/2442473703 >> 172.21.15.122:0/2442473703 conn(0x55c312e8eed0 msgr2=0x55c312e9e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- 172.21.15.122:0/2442473703 shutdown_connections 2024-04-02T13:48:53.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- 172.21.15.122:0/2442473703 wait complete. 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.207+0000 7f6f467f2740 1 -- start start 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 Processor -- start 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312e9bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1ae340 con 0x55c312e94450 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1adc40 con 0x55c312ea20b0 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3c1aae70 con 0x55c312e94d70 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f467f2740 10 client.0 _ll_register_callbacks cb 0x7f6f2c009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33574/0 (socket says 172.21.15.122:33574) 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 -- 172.21.15.122:0/3780541326 learned_addr learned my addr 172.21.15.122:0/3780541326 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f411a8640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f421aa640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312e9bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 -- 172.21.15.122:0/3780541326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 msgr2=0x55c312e9b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 -- 172.21.15.122:0/3780541326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 msgr2=0x55c312e9bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312e9bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.208+0000 7f6f419a9640 1 -- 172.21.15.122:0/3780541326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c312e9c5b0 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f419a9640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6f380209b0 tx=0x7f6f380209e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aacafca53df83313 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f38010430 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6f38004020 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f38020770 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6f38038050 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 10 client.19545 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c312f183c0 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f38020770 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f6f38030020 con 0x55c312e94450 2024-04-02T13:48:53.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 10 client.19545 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 10 client.19545 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 20 client.19545 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312f1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 4 client.19545 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.209+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c312e00040 con 0x55c312eb1fa0 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f411a8640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312f1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f257fa640 10 client.19545 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f411a8640 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312f1abd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0968a7c14643e2b server_cookie=d463db72fe2cb297 in_seq=0 out_seq=0 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f257fa640 1 -- 172.21.15.122:0/3780541326 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c312e00040 con 0x55c312eb1fa0 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f257fa640 10 client.19545 handle_command_reply: tid=0 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 client.19545 shutdown 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 msgr2=0x55c312eaa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f6f380209b0 tx=0x7f6f380209e0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 shutdown_connections 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55c312eb1fa0 0x55c312f1abd0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c312ea20b0 0x55c312e9bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c312e94d70 0x55c312e9b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 --2- 172.21.15.122:0/3780541326 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c312e94450 0x55c312eaa360 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f257fa640 0 client.19545 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 >> 172.21.15.122:0/3780541326 conn(0x55c312e8eed0 msgr2=0x55c312e8d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 shutdown_connections 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 1 -- 172.21.15.122:0/3780541326 wait complete. 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.210+0000 7f6f467f2740 20 client.19545 trim_cache size 0 max 16384 2024-04-02T13:48:53.220 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 msgr2=0x7f6f280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 0x7f6f280a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009710 tx=0x7f6f2c011ca0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 msgr2=0x7f6f3c19bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f6f3402efd0 tx=0x7f6f34035d90 comp rx=0 tx=0).stop 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 shutdown_connections 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6f280a3d20 0x7f6f280a61e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f3c1058a0 0x7f6f3c19bad0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f3c101360 0x7f6f3c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 --2- 172.21.15.122:0/2190300612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f3c1009c0 0x7f6f3c197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 >> 172.21.15.122:0/2190300612 conn(0x7f6f3c077a90 msgr2=0x7f6f3c0fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 shutdown_connections 2024-04-02T13:48:53.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.211+0000 7f6f43c0d640 1 -- 172.21.15.122:0/2190300612 wait complete. 2024-04-02T13:48:53.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:48:53.319 INFO:tasks.workunit.client.0.smithi122.stdout: 10 5135 33.39 MB/sec warmup 13 sec latency 106.323 ms 2024-04-02T13:48:53.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- 172.21.15.122:0/2331908220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598107ab0 msgr2=0x7ff598107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 --2- 172.21.15.122:0/2331908220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598107ab0 0x7ff598107e90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7ff594009930 tx=0x7ff59402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:53.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- 172.21.15.122:0/2331908220 shutdown_connections 2024-04-02T13:48:53.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 --2- 172.21.15.122:0/2331908220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598102460 0x7ff59810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 --2- 172.21.15.122:0/2331908220 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff598101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 --2- 172.21.15.122:0/2331908220 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598107ab0 0x7ff598107e90 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- 172.21.15.122:0/2331908220 >> 172.21.15.122:0/2331908220 conn(0x7ff5980fd530 msgr2=0x7ff5980ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- 172.21.15.122:0/2331908220 shutdown_connections 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- 172.21.15.122:0/2331908220 wait complete. 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 Processor -- start 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.340+0000 7ff59eb00640 1 -- start start 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff5981983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598107ab0 0x7ff59819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff598111ec0 con 0x7ff598102460 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff598111d40 con 0x7ff598101aa0 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff598112040 con 0x7ff598107ab0 2024-04-02T13:48:53.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff5981983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33606/0 (socket says 172.21.15.122:33606) 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 -- 172.21.15.122:0/1268492580 learned_addr learned my addr 172.21.15.122:0/1268492580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59d09d640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598107ab0 0x7ff59819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 -- 172.21.15.122:0/1268492580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598107ab0 msgr2=0x7ff59819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598107ab0 0x7ff59819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 -- 172.21.15.122:0/1268492580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 msgr2=0x7ff5981983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff5981983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 -- 172.21.15.122:0/1268492580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59819d380 con 0x7ff598102460 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59c89c640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff5981983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58ffff640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff58800e9f0 tx=0x7ff58800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58dffb640 1 -- 172.21.15.122:0/1268492580 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff58800cd80 con 0x7ff598102460 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58dffb640 1 -- 172.21.15.122:0/1268492580 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588010910 con 0x7ff598102460 2024-04-02T13:48:53.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff58dffb640 1 -- 172.21.15.122:0/1268492580 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588018990 con 0x7ff598102460 2024-04-02T13:48:53.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.341+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59819d670 con 0x7ff598102460 2024-04-02T13:48:53.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.342+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5981a4f00 con 0x7ff598102460 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff58dffb640 1 -- 172.21.15.122:0/1268492580 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff588010ac0 con 0x7ff598102460 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 Processor -- start 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 -- start start 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e940c0 0x563582e9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563582df77b0 con 0x563582e940c0 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563582d96570 con 0x563582e937a0 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563582df3760 con 0x563582e9cca0 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff59d09d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.344+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e940c0 0x563582e9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49336/0 (socket says 172.21.15.122:49336) 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 -- 172.21.15.122:0/3757149813 learned_addr learned my addr 172.21.15.122:0/3757149813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff58dffb640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 0x7ff5800a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 -- 172.21.15.122:0/3757149813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 msgr2=0x563582e9f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 -- 172.21.15.122:0/3757149813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e940c0 msgr2=0x563582e9c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e940c0 0x563582e9c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 -- 172.21.15.122:0/3757149813 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563582b0b3f0 con 0x563582e937a0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 0x7ff5800a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff58dffb640 1 -- 172.21.15.122:0/1268492580 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7ff5880d0f20 con 0x7ff598102460 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff590010520 tx=0x7ff5900109e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24961110de1d0918 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff57effd640 1 -- 172.21.15.122:0/3757149813 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590019070 con 0x563582e937a0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff59d09d640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 0x7ff5800a6260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff590021000 tx=0x7ff59001e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff57effd640 1 -- 172.21.15.122:0/3757149813 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff590003e10 con 0x563582e937a0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.345+0000 7ff57effd640 1 -- 172.21.15.122:0/3757149813 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59001d710 con 0x563582e937a0 2024-04-02T13:48:53.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3757149813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 msgr2=0x563582e93b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff590010520 tx=0x7ff5900109e0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3757149813 shutdown_connections 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e940c0 0x563582e9c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3757149813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e93b80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3757149813 >> 172.21.15.122:0/3757149813 conn(0x563582e8ded0 msgr2=0x563582e8e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3757149813 shutdown_connections 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3757149813 wait complete. 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 -- start start 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.346+0000 7ff5a17c2740 1 Processor -- start 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563582e9fa90 con 0x563582e937a0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563582db2e80 con 0x563582e940c0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563582e9fc10 con 0x563582e9cca0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff5a17c2740 10 client.0 _ll_register_callbacks cb 0x563582deeaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33616/0 (socket says 172.21.15.122:33616) 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 -- 172.21.15.122:0/3693796165 learned_addr learned my addr 172.21.15.122:0/3693796165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff58ffff640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59d09d640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 -- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 msgr2=0x563582e9a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 -- 172.21.15.122:0/3693796165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 msgr2=0x563582e9a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 -- 172.21.15.122:0/3693796165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563582bd57d0 con 0x563582e937a0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff59c89c640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff594004ee0 tx=0x7ff59402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8dfede5e78defe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.347+0000 7ff577fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59402cbf0 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7ff594004040 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff590003e10 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff594004240 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 10 client.19557 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563582eaffe0 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff590003e10 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7ff594003860 con 0x563582e937a0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 10 client.19557 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 10 client.19557 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 20 client.19557 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582eb14f0 0x563582eb3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 4 client.19557 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563582e04cf0 con 0x563582eb14f0 2024-04-02T13:48:53.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.348+0000 7ff58ffff640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582eb14f0 0x563582eb3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff58ffff640 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582eb14f0 0x563582eb3990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1529a89e6f5430ad server_cookie=bbe49c75cc286a08 in_seq=0 out_seq=0 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff577fff640 10 client.19557 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff577fff640 1 -- 172.21.15.122:0/3693796165 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563582e04cf0 con 0x563582eb14f0 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff577fff640 10 client.19557 handle_command_reply: tid=0 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff5a17c2740 1 client.19557 shutdown 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 msgr2=0x563582e99b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.349+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7ff594004ee0 tx=0x7ff59402c880 comp rx=0 tx=0).stop 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 shutdown_connections 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582eb14f0 0x563582eb3990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/3693796165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e99b20 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff577fff640 0 client.19557 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 >> 172.21.15.122:0/3693796165 conn(0x563582e8ded0 msgr2=0x563582e8fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 shutdown_connections 2024-04-02T13:48:53.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 1 -- 172.21.15.122:0/3693796165 wait complete. 2024-04-02T13:48:53.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.350+0000 7ff5a17c2740 20 client.19557 trim_cache size 0 max 16384 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 Processor -- start 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 -- start start 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e9c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 0x563582e98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981acf00 con 0x563582e9cca0 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981aeaa0 con 0x563582e937a0 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981ad5d0 con 0x563582e940c0 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff59d09d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff59d09d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33618/0 (socket says 172.21.15.122:33618) 2024-04-02T13:48:53.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff58ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 0x563582e98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff59d09d640 1 -- 172.21.15.122:0/666448476 learned_addr learned my addr 172.21.15.122:0/666448476 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.366+0000 7ff59c89c640 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e9c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 -- 172.21.15.122:0/666448476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 msgr2=0x563582e9c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e9c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 -- 172.21.15.122:0/666448476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 msgr2=0x563582eb9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 -- 172.21.15.122:0/666448476 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563582eb4010 con 0x563582e940c0 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff59d09d640 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff58ffff640 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 0x563582e98b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff5880ce280 tx=0x7ff5880d34a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e7f6137ff9329b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff57e7fc640 1 -- 172.21.15.122:0/666448476 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880a5740 con 0x563582e940c0 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff57e7fc640 1 -- 172.21.15.122:0/666448476 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58809cad0 con 0x563582e940c0 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.367+0000 7ff57e7fc640 1 -- 172.21.15.122:0/666448476 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588099140 con 0x563582e940c0 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 -- 172.21.15.122:0/666448476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 msgr2=0x563582e98b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 0x563582e98b00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff5880ce280 tx=0x7ff5880d34a0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 -- 172.21.15.122:0/666448476 shutdown_connections 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e9cca0 0x563582eb9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e940c0 0x563582e98b00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/666448476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e937a0 0x563582e9c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 -- 172.21.15.122:0/666448476 >> 172.21.15.122:0/666448476 conn(0x563582e8ded0 msgr2=0x563582e983f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 -- 172.21.15.122:0/666448476 shutdown_connections 2024-04-02T13:48:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.368+0000 7ff5a17c2740 1 -- 172.21.15.122:0/666448476 wait complete. 2024-04-02T13:48:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.369+0000 7ff5a17c2740 1 -- start start 2024-04-02T13:48:53.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.369+0000 7ff5a17c2740 1 Processor -- start 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.369+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981ae3d0 con 0x563582e937a0 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981adcd0 con 0x563582e940c0 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5981aaf00 con 0x563582e9cca0 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff5a17c2740 10 client.0 _ll_register_callbacks cb 0x7ff594009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33628/0 (socket says 172.21.15.122:33628) 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 -- 172.21.15.122:0/2411002069 learned_addr learned my addr 172.21.15.122:0/2411002069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff58ffff640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59d09d640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 -- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 msgr2=0x563582e9db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 -- 172.21.15.122:0/2411002069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 msgr2=0x563582e9d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59c89c640 1 -- 172.21.15.122:0/2411002069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563582ea7f00 con 0x563582e937a0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.370+0000 7ff59d09d640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff59c89c640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ff5940345c0 tx=0x7ff59402c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318fdaea709bfd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594005d50 con 0x563582e937a0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7ff59402caf0 con 0x563582e937a0 2024-04-02T13:48:53.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff58809cad0 con 0x563582e937a0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff594064050 con 0x563582e937a0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff5a17c2740 10 client.19563 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563582ea91a0 con 0x563582e937a0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff58809cad0 con 0x563582e937a0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7ff594005260 con 0x563582e937a0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.371+0000 7ff5a17c2740 10 client.19563 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 10 client.19563 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 20 client.19563 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582f1c2c0 0x563582f1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 4 client.19563 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563582dff040 con 0x563582f1c2c0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff58ffff640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582f1c2c0 0x563582f1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff577fff640 10 client.19563 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff58ffff640 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582f1c2c0 0x563582f1e6b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c4611d2877794 server_cookie=d56719f2721802fc in_seq=0 out_seq=0 2024-04-02T13:48:53.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff577fff640 1 -- 172.21.15.122:0/2411002069 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563582dff040 con 0x563582f1c2c0 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff577fff640 10 client.19563 handle_command_reply: tid=0 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 client.19563 shutdown 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 msgr2=0x563582e9d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7ff5940345c0 tx=0x7ff59402c330 comp rx=0 tx=0).stop 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 shutdown_connections 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563582e9cca0 0x563582e9db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563582e940c0 0x563582e9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.372+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563582f1c2c0 0x563582f1e6b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff5a17c2740 1 --2- 172.21.15.122:0/2411002069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563582e937a0 0x563582e9d0e0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff577fff640 0 client.19563 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 >> 172.21.15.122:0/2411002069 conn(0x563582e8ded0 msgr2=0x563582e98c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 shutdown_connections 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff5a17c2740 1 -- 172.21.15.122:0/2411002069 wait complete. 2024-04-02T13:48:53.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff5a17c2740 20 client.19563 trim_cache size 0 max 16384 2024-04-02T13:48:53.381 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:53.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 msgr2=0x7ff5800a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 0x7ff5800a6260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff590021000 tx=0x7ff59001e000 comp rx=0 tx=0).stop 2024-04-02T13:48:53.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 msgr2=0x7ff598198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7ff58800e9f0 tx=0x7ff58800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 shutdown_connections 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff5800a3da0 0x7ff5800a6260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff598107ab0 0x7ff59819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff598102460 0x7ff598198920 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.373+0000 7ff59eb00640 1 --2- 172.21.15.122:0/1268492580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff598101aa0 0x7ff5981983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.374+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 >> 172.21.15.122:0/1268492580 conn(0x7ff5980fd530 msgr2=0x7ff5980fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.374+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 shutdown_connections 2024-04-02T13:48:53.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.374+0000 7ff59eb00640 1 -- 172.21.15.122:0/1268492580 wait complete. 2024-04-02T13:48:53.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:48:53.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 -- 172.21.15.122:0/1414533409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d98107b30 msgr2=0x7f6d98107f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 --2- 172.21.15.122:0/1414533409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d98107b30 0x7f6d98107f10 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009930 tx=0x7f6d9402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:53.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 -- 172.21.15.122:0/1414533409 shutdown_connections 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 --2- 172.21.15.122:0/1414533409 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d9806aa10 0x7f6d9810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 --2- 172.21.15.122:0/1414533409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806a070 0x7f6d9806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 --2- 172.21.15.122:0/1414533409 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d98107b30 0x7f6d98107f10 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 -- 172.21.15.122:0/1414533409 >> 172.21.15.122:0/1414533409 conn(0x7f6d980779e0 msgr2=0x7f6d98077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 -- 172.21.15.122:0/1414533409 shutdown_connections 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.499+0000 7f6d9f511640 1 -- 172.21.15.122:0/1414533409 wait complete. 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 Processor -- start 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 -- start start 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 0x7f6d98198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 0x7f6d9819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98111f10 con 0x7f6d9806a070 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98111d90 con 0x7f6d98107b30 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9f511640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d98112090 con 0x7f6d9806aa10 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9d2ad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.500+0000 7f6d9d2ad640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33654/0 (socket says 172.21.15.122:33654) 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/2795871494 learned_addr learned my addr 172.21.15.122:0/2795871494 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 0x7f6d98198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9daae640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 0x7f6d9819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/2795871494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 msgr2=0x7f6d98198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 0x7f6d98198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/2795871494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 msgr2=0x7f6d9819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 0x7f6d9819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/2795871494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9819d3c0 con 0x7f6d9806a070 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9daae640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 0x7f6d9819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 0x7f6d98198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009900 tx=0x7f6d940042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d8e7fc640 1 -- 172.21.15.122:0/2795871494 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94043020 con 0x7f6d9806a070 2024-04-02T13:48:53.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9819d650 con 0x7f6d9806a070 2024-04-02T13:48:53.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d8e7fc640 1 -- 172.21.15.122:0/2795871494 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d940044d0 con 0x7f6d9806a070 2024-04-02T13:48:53.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.501+0000 7f6d8e7fc640 1 -- 172.21.15.122:0/2795871494 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d94003be0 con 0x7f6d9806a070 2024-04-02T13:48:53.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.502+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d981a4f40 con 0x7f6d9806a070 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.504+0000 7f6d8e7fc640 1 -- 172.21.15.122:0/2795871494 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6d94046020 con 0x7f6d9806a070 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.505+0000 7f6da21d0740 1 Processor -- start 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.505+0000 7f6d8e7fc640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 0x7f6d840a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 -- start start 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d8e7fc640 1 -- 172.21.15.122:0/2795871494 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6d940f2c10 con 0x7f6d9806a070 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 0x7f6d840a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8685f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d867af7b0 con 0x556d8685d3d0 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d8674e570 con 0x556d8684bbc0 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6da21d0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d867ab760 con 0x556d8684b210 2024-04-02T13:48:53.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48548/0 (socket says 172.21.15.122:48548) 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/666925983 learned_addr learned my addr 172.21.15.122:0/666925983 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9daae640 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8685f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/666925983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 msgr2=0x556d8684c040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684c040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/666925983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 msgr2=0x556d8685f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8685f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9d2ad640 1 -- 172.21.15.122:0/666925983 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d864c33f0 con 0x556d8684b210 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.506+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 0x7f6d840a6220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6d880095f0 tx=0x7f6d88005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.507+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d940bb330 tx=0x7f6d9402bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4025a7d5cfc64e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.507+0000 7f6d837fe640 1 -- 172.21.15.122:0/666925983 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940d9020 con 0x556d8684b210 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.507+0000 7f6d837fe640 1 -- 172.21.15.122:0/666925983 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d940be9f0 con 0x556d8684b210 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.507+0000 7f6d837fe640 1 -- 172.21.15.122:0/666925983 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d940c71d0 con 0x556d8684b210 2024-04-02T13:48:53.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- 172.21.15.122:0/666925983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 msgr2=0x556d8684b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d940bb330 tx=0x7f6d9402bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- 172.21.15.122:0/666925983 shutdown_connections 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8685f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 --2- 172.21.15.122:0/666925983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d8684b5f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- 172.21.15.122:0/666925983 >> 172.21.15.122:0/666925983 conn(0x556d86845ed0 msgr2=0x556d868462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- 172.21.15.122:0/666925983 shutdown_connections 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- 172.21.15.122:0/666925983 wait complete. 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 -- start start 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.508+0000 7f6da21d0740 1 Processor -- start 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d86850230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556d86860040 con 0x556d8685d3d0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556d8676ae80 con 0x556d8684bbc0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556d868601c0 con 0x556d8684b210 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9caac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33674/0 (socket says 172.21.15.122:33674) 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 -- 172.21.15.122:0/3145003450 learned_addr learned my addr 172.21.15.122:0/3145003450 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d86850230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6da21d0740 10 client.0 _ll_register_callbacks cb 0x556d867a6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 -- 172.21.15.122:0/3145003450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 msgr2=0x556d86850230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d86850230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 -- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 msgr2=0x556d8684d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9daae640 1 -- 172.21.15.122:0/3145003450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8658d7d0 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.509+0000 7f6d9caac640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d9daae640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011d10 tx=0x7f6d900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e535b0f4bb19154 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90009e30 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6d90003970 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d940be9f0 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6d90003b70 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 10 client.19575 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556d8686c020 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d940be9f0 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f6d9001b5b0 con 0x556d8685d3d0 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 10 client.19575 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 10 client.19575 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:48:53.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 20 client.19575 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d8686d4d0 0x556d8686f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 4 client.19575 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.510+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d867bccf0 con 0x556d8686d4d0 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.511+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d8686d4d0 0x556d8686f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.511+0000 7f6d80ff9640 10 client.19575 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.511+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d8686d4d0 0x556d8686f8f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bcb74bc858c65039 server_cookie=ef3cfd1b5d2a03fc in_seq=0 out_seq=0 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6d80ff9640 1 -- 172.21.15.122:0/3145003450 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d867bccf0 con 0x556d8686d4d0 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6d80ff9640 10 client.19575 handle_command_reply: tid=0 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 client.19575 shutdown 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 msgr2=0x556d8684dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6d90011d10 tx=0x7f6d900107a0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 shutdown_connections 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684dd00 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d8686d4d0 0x556d8686f8f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684bbc0 0x556d8684d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 --2- 172.21.15.122:0/3145003450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684b210 0x556d86850230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6d80ff9640 0 client.19575 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.512+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 >> 172.21.15.122:0/3145003450 conn(0x556d86845ed0 msgr2=0x556d86847c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.513+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 shutdown_connections 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.513+0000 7f6da21d0740 1 -- 172.21.15.122:0/3145003450 wait complete. 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.513+0000 7f6da21d0740 20 client.19575 trim_cache size 0 max 16384 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 Processor -- start 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 -- start start 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d86868d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 0x556d86840d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981acfb0 con 0x556d8685d3d0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981aeb50 con 0x556d8684b210 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.526+0000 7f6da21d0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981ad680 con 0x556d8684bbc0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33682/0 (socket says 172.21.15.122:33682) 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 -- 172.21.15.122:0/1305904446 learned_addr learned my addr 172.21.15.122:0/1305904446 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d86868d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9caac640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 0x556d86840d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 -- 172.21.15.122:0/1305904446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 msgr2=0x556d86840d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 0x556d86840d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 -- 172.21.15.122:0/1305904446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 msgr2=0x556d86868d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d86868d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 -- 172.21.15.122:0/1305904446 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d8684b920 con 0x556d8685d3d0 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9caac640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 0x556d86840d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d9daae640 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005f90 tx=0x7f6d90003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522dc4cdacf166cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d82ffd640 1 -- 172.21.15.122:0/1305904446 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90009e30 con 0x556d8685d3d0 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d82ffd640 1 -- 172.21.15.122:0/1305904446 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9002e070 con 0x556d8685d3d0 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.527+0000 7f6d82ffd640 1 -- 172.21.15.122:0/1305904446 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90005320 con 0x556d8685d3d0 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- 172.21.15.122:0/1305904446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 msgr2=0x556d8684fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f6d90005f90 tx=0x7f6d90003b00 comp rx=0 tx=0).stop 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- 172.21.15.122:0/1305904446 shutdown_connections 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8685d3d0 0x556d8684fcf0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8684bbc0 0x556d86840d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 --2- 172.21.15.122:0/1305904446 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d86868d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- 172.21.15.122:0/1305904446 >> 172.21.15.122:0/1305904446 conn(0x556d86845ed0 msgr2=0x556d868462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- 172.21.15.122:0/1305904446 shutdown_connections 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- 172.21.15.122:0/1305904446 wait complete. 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 -- start start 2024-04-02T13:48:53.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.528+0000 7f6da21d0740 1 Processor -- start 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d868652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8685d3d0 0x556d8685aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981ae480 con 0x556d8684bbc0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981add80 con 0x556d8684b210 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d981aafb0 con 0x556d8685d3d0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6da21d0740 10 client.0 _ll_register_callbacks cb 0x7f6d90009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9d2ad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d868652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33698/0 (socket says 172.21.15.122:33698) 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 -- 172.21.15.122:0/2356394865 learned_addr learned my addr 172.21.15.122:0/2356394865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9daae640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8685d3d0 0x556d8685aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 -- 172.21.15.122:0/2356394865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8685d3d0 msgr2=0x556d8685aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8685d3d0 0x556d8685aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 -- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 msgr2=0x556d868652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d868652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 -- 172.21.15.122:0/2356394865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d8685b200 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d9caac640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x556d8684c6b0 tx=0x7f6d88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88c01b79d7626f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.529+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880273f0 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f6d88027590 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d9002e070 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f6d88035050 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 10 client.19581 fetch_fsmap learned FSMap version 48 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556d868d3740 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d9002e070 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f6d8801c790 con 0x556d8684bbc0 2024-04-02T13:48:53.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 10 client.19581 fetch_fsmap finished waiting for FSMap version 48 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 10 client.19581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 20 client.19581 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d868d3fa0 0x556d868d63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 4 client.19581 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d867b7040 con 0x556d868d3fa0 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.530+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d868d3fa0 0x556d868d63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6d80ff9640 10 client.19581 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6d9d2ad640 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d868d3fa0 0x556d868d63c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f132929499a425f8 server_cookie=4c20c712475915e3 in_seq=0 out_seq=0 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6d80ff9640 1 -- 172.21.15.122:0/2356394865 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d867b7040 con 0x556d868d3fa0 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6d80ff9640 10 client.19581 handle_command_reply: tid=0 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 client.19581 shutdown 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 msgr2=0x556d868657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x556d8684c6b0 tx=0x7f6d88004730 comp rx=0 tx=0).stop 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 shutdown_connections 2024-04-02T13:48:53.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556d8685d3d0 0x556d8685aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556d8684bbc0 0x556d868657e0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x556d868d3fa0 0x556d868d63c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 --2- 172.21.15.122:0/2356394865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556d8684b210 0x556d868652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6d80ff9640 0 client.19581 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 >> 172.21.15.122:0/2356394865 conn(0x556d86845ed0 msgr2=0x556d86840d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 shutdown_connections 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 1 -- 172.21.15.122:0/2356394865 wait complete. 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.531+0000 7f6da21d0740 20 client.19581 trim_cache size 0 max 16384 2024-04-02T13:48:53.800 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:53.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 msgr2=0x7f6d840a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 0x7f6d840a6220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6d880095f0 tx=0x7f6d88005f90 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 msgr2=0x7f6d981983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f6d94009900 tx=0x7f6d940042e0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 shutdown_connections 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6d840a3d60 0x7f6d840a6220 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6d98107b30 0x7f6d9819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d9806aa10 0x7f6d98198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 --2- 172.21.15.122:0/2795871494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d9806a070 0x7f6d981983d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 >> 172.21.15.122:0/2795871494 conn(0x7f6d980779e0 msgr2=0x7f6d980ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 shutdown_connections 2024-04-02T13:48:53.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.532+0000 7f6d9f511640 1 -- 172.21.15.122:0/2795871494 wait complete. 2024-04-02T13:48:53.803 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- 172.21.15.122:0/1135581045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c100970 msgr2=0x7fa58c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 --2- 172.21.15.122:0/1135581045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c100970 0x7fa58c100df0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fa580009930 tx=0x7fa58002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- 172.21.15.122:0/1135581045 shutdown_connections 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 --2- 172.21.15.122:0/1135581045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 --2- 172.21.15.122:0/1135581045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c100970 0x7fa58c100df0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 --2- 172.21.15.122:0/1135581045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c106980 0x7fa58c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- 172.21.15.122:0/1135581045 >> 172.21.15.122:0/1135581045 conn(0x7fa58c077b00 msgr2=0x7fa58c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- 172.21.15.122:0/1135581045 shutdown_connections 2024-04-02T13:48:53.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- 172.21.15.122:0/1135581045 wait complete. 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 Processor -- start 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.911+0000 7fa591ca5640 1 -- start start 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 0x7fa58c19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c110d90 con 0x7fa58c106980 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c110c10 con 0x7fa58c100970 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa591ca5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c110f10 con 0x7fa58c101330 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 0x7fa58c19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49422/0 (socket says 172.21.15.122:49422) 2024-04-02T13:48:53.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58b7fe640 1 -- 172.21.15.122:0/2306695914 learned_addr learned my addr 172.21.15.122:0/2306695914 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 -- 172.21.15.122:0/2306695914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 msgr2=0x7fa58c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 -- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 msgr2=0x7fa58c19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 0x7fa58c19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 -- 172.21.15.122:0/2306695914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c19f530 con 0x7fa58c101330 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 0x7fa58c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa58affd640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c19aad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa580009230 tx=0x7fa58002cb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.912+0000 7fa588ff9640 1 -- 172.21.15.122:0/2306695914 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580041070 con 0x7fa58c101330 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.913+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c19f7c0 con 0x7fa58c101330 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.913+0000 7fa588ff9640 1 -- 172.21.15.122:0/2306695914 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa580004450 con 0x7fa58c101330 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.913+0000 7fa588ff9640 1 -- 172.21.15.122:0/2306695914 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580003d30 con 0x7fa58c101330 2024-04-02T13:48:53.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.913+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c1a70b0 con 0x7fa58c101330 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa588ff9640 1 -- 172.21.15.122:0/2306695914 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa580044070 con 0x7fa58c101330 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 Processor -- start 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 -- start start 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 0x5639221d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56392212c7b0 con 0x5639221c90c0 2024-04-02T13:48:53.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5639220cb570 con 0x5639221d1ca0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa59496a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563922128760 con 0x5639221c87a0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33734/0 (socket says 172.21.15.122:33734) 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa58b7fe640 1 -- 172.21.15.122:0/3772597572 learned_addr learned my addr 172.21.15.122:0/3772597572 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa588ff9640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 0x7fa5740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.915+0000 7fa58affd640 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 0x5639221d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 0x7fa5740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 -- 172.21.15.122:0/3772597572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 msgr2=0x5639221c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 -- 172.21.15.122:0/3772597572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 msgr2=0x5639221d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 0x5639221d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 -- 172.21.15.122:0/3772597572 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563921e403f0 con 0x5639221c90c0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58affd640 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 0x5639221d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa588ff9640 1 -- 172.21.15.122:0/2306695914 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa5800f23e0 con 0x7fa58c101330 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0115c0 tx=0x7fa57c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3950f09cfcb8e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 0x7fa5740a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5840049d0 tx=0x7fa584004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa571ffb640 1 -- 172.21.15.122:0/3772597572 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00f660 con 0x5639221c90c0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa571ffb640 1 -- 172.21.15.122:0/3772597572 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c00fce0 con 0x5639221c90c0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.916+0000 7fa571ffb640 1 -- 172.21.15.122:0/3772597572 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57c019d80 con 0x5639221c90c0 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- 172.21.15.122:0/3772597572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 msgr2=0x5639221d1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fa57c0115c0 tx=0x7fa57c011a80 comp rx=0 tx=0).stop 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- 172.21.15.122:0/3772597572 shutdown_connections 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221d1ca0 0x5639221d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221c90c0 0x5639221d1760 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 --2- 172.21.15.122:0/3772597572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- 172.21.15.122:0/3772597572 >> 172.21.15.122:0/3772597572 conn(0x5639221c2ed0 msgr2=0x5639221c32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- 172.21.15.122:0/3772597572 shutdown_connections 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- 172.21.15.122:0/3772597572 wait complete. 2024-04-02T13:48:53.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 -- start start 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.917+0000 7fa59496a740 1 Processor -- start 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639221d4a90 con 0x5639221d1ca0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5639220e7e80 con 0x5639221c87a0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639221d4c10 con 0x5639221c90c0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa59496a740 10 client.0 _ll_register_callbacks cb 0x563922123aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33748/0 (socket says 172.21.15.122:33748) 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 -- 172.21.15.122:0/697471668 learned_addr learned my addr 172.21.15.122:0/697471668 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 -- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 msgr2=0x5639221ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 -- 172.21.15.122:0/697471668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 msgr2=0x5639221d1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 -- 172.21.15.122:0/697471668 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563921f0a7d0 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58affd640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa56affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa58bfff640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa584010b90 tx=0x7fa584004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b74fa17cc039e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.918+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840208f0 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c00fce0 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa584020a90 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa584034050 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 10 client.19593 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5639221e0ba0 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa57c00fce0 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7fa58401f040 con 0x5639221d1ca0 2024-04-02T13:48:53.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 10 client.19593 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 10 client.19593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 20 client.19593 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5639221e2060 0x5639221e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 4 client.19593 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563922139cf0 con 0x5639221e2060 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.919+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5639221e2060 0x5639221e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa56affd640 10 client.19593 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5639221e2060 0x5639221e44e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a72b10ee5c1f6775 server_cookie=14f64e3819399b57 in_seq=0 out_seq=0 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa56affd640 1 -- 172.21.15.122:0/697471668 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563922139cf0 con 0x5639221e2060 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa56affd640 10 client.19593 handle_command_reply: tid=0 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa59496a740 1 client.19593 shutdown 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 msgr2=0x5639221cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fa584010b90 tx=0x7fa584004a40 comp rx=0 tx=0).stop 2024-04-02T13:48:53.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.920+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 shutdown_connections 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5639221e2060 0x5639221e44e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221cf230 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa56affd640 0 client.19593 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 --2- 172.21.15.122:0/697471668 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 >> 172.21.15.122:0/697471668 conn(0x5639221c2ed0 msgr2=0x5639221c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 shutdown_connections 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 1 -- 172.21.15.122:0/697471668 wait complete. 2024-04-02T13:48:53.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.921+0000 7fa59496a740 20 client.19593 trim_cache size 0 max 16384 2024-04-02T13:48:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.935+0000 7fa59496a740 1 Processor -- start 2024-04-02T13:48:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.935+0000 7fa59496a740 1 -- start start 2024-04-02T13:48:53.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221dd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c90c0 0x5639221c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b1270 con 0x5639221d1ca0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b2e40 con 0x5639221c90c0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa59496a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b1940 con 0x5639221c87a0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221dd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33758/0 (socket says 172.21.15.122:33758) 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 -- 172.21.15.122:0/2297338255 learned_addr learned my addr 172.21.15.122:0/2297338255 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58affd640 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c90c0 0x5639221c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 -- 172.21.15.122:0/2297338255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 msgr2=0x5639221dd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221dd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 -- 172.21.15.122:0/2297338255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c90c0 msgr2=0x5639221c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c90c0 0x5639221c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 -- 172.21.15.122:0/2297338255 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639221e4b60 con 0x5639221d1ca0 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221dd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa58bfff640 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa58402bcf0 tx=0x7fa584022660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b9e7b9fd5390f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa5717fa640 1 -- 172.21.15.122:0/2297338255 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58403c020 con 0x5639221d1ca0 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa5717fa640 1 -- 172.21.15.122:0/2297338255 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5840200c0 con 0x5639221d1ca0 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.936+0000 7fa5717fa640 1 -- 172.21.15.122:0/2297338255 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5840209a0 con 0x5639221d1ca0 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- 172.21.15.122:0/2297338255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 msgr2=0x5639221d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa58402bcf0 tx=0x7fa584022660 comp rx=0 tx=0).stop 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- 172.21.15.122:0/2297338255 shutdown_connections 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x5639221d1220 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c90c0 0x5639221c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 --2- 172.21.15.122:0/2297338255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c87a0 0x5639221dd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- 172.21.15.122:0/2297338255 >> 172.21.15.122:0/2297338255 conn(0x5639221c2ed0 msgr2=0x5639221cd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- 172.21.15.122:0/2297338255 shutdown_connections 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- 172.21.15.122:0/2297338255 wait complete. 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.937+0000 7fa59496a740 1 -- start start 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 Processor -- start 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b2770 con 0x5639221d1ca0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1b2040 con 0x5639221c87a0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1af270 con 0x5639221c90c0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa59496a740 10 client.0 _ll_register_callbacks cb 0x7fa584010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33768/0 (socket says 172.21.15.122:33768) 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 -- 172.21.15.122:0/3055887146 learned_addr learned my addr 172.21.15.122:0/3055887146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58affd640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 -- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 msgr2=0x5639221d5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221d5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 -- 172.21.15.122:0/3055887146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 msgr2=0x5639221d50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 -- 172.21.15.122:0/3055887146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563922240c90 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa58bfff640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa584012040 tx=0x7fa584005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23e35c33b820d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.938+0000 7fa56affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58403c070 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5840200c0 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fa58402b690 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa58404e050 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 10 client.19599 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x563922248000 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5840200c0 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7fa584049030 con 0x5639221d1ca0 2024-04-02T13:48:53.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 10 client.19599 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 10 client.19599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 20 client.19599 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x563922248630 0x56392224aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 4 client.19599 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563922134040 con 0x563922248630 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.939+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x563922248630 0x56392224aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa56affd640 10 client.19599 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa58b7fe640 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x563922248630 0x56392224aa50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ba666336466a5df server_cookie=bcf99a10da000af2 in_seq=0 out_seq=0 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa56affd640 1 -- 172.21.15.122:0/3055887146 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563922134040 con 0x563922248630 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa56affd640 10 client.19599 handle_command_reply: tid=0 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 client.19599 shutdown 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 msgr2=0x563922240650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa584012040 tx=0x7fa584005bd0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 shutdown_connections 2024-04-02T13:48:53.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x563922248630 0x56392224aa50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639221d1ca0 0x563922240650 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639221c90c0 0x5639221d5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 --2- 172.21.15.122:0/3055887146 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639221c87a0 0x5639221d50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa56affd640 0 client.19599 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 >> 172.21.15.122:0/3055887146 conn(0x5639221c2ed0 msgr2=0x5639221c1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 shutdown_connections 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 1 -- 172.21.15.122:0/3055887146 wait complete. 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.940+0000 7fa59496a740 20 client.19599 trim_cache size 0 max 16384 2024-04-02T13:48:53.950 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 msgr2=0x7fa5740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 0x7fa5740a61e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa5840049d0 tx=0x7fa584004ab0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 msgr2=0x7fa58c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c19aad0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa580009230 tx=0x7fa58002cb20 comp rx=0 tx=0).stop 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 shutdown_connections 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa5740a3d20 0x7fa5740a61e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa58c106980 0x7fa58c19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa58c101330 0x7fa58c19aad0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 --2- 172.21.15.122:0/2306695914 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa58c100970 0x7fa58c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:53.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 >> 172.21.15.122:0/2306695914 conn(0x7fa58c077b00 msgr2=0x7fa58c068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:53.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 shutdown_connections 2024-04-02T13:48:53.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:53.941+0000 7fa591ca5640 1 -- 172.21.15.122:0/2306695914 wait complete. 2024-04-02T13:48:53.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:53 smithi122 ceph-mon[32114]: pgmap v428: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-04-02T13:48:53.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:53 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:53.954 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:53 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3378093154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:53.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:48:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:53 smithi187 ceph-mon[36064]: pgmap v428: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-04-02T13:48:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:53 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:53 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3378093154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:53 smithi188 ceph-mon[36117]: pgmap v428: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 293 op/s 2024-04-02T13:48:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:53 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:53 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3378093154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.062+0000 7fb485888640 1 -- 172.21.15.122:0/1028789974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480101330 msgr2=0x7fb48010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.062+0000 7fb485888640 1 --2- 172.21.15.122:0/1028789974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480101330 0x7fb48010d840 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb470009930 tx=0x7fb47002bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.062+0000 7fb485888640 1 -- 172.21.15.122:0/1028789974 shutdown_connections 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 --2- 172.21.15.122:0/1028789974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480101330 0x7fb48010d840 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 --2- 172.21.15.122:0/1028789974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480100970 0x7fb480100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 --2- 172.21.15.122:0/1028789974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480106980 0x7fb480106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 -- 172.21.15.122:0/1028789974 >> 172.21.15.122:0/1028789974 conn(0x7fb480077b00 msgr2=0x7fb480077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 -- 172.21.15.122:0/1028789974 shutdown_connections 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 -- 172.21.15.122:0/1028789974 wait complete. 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 Processor -- start 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 -- start start 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.063+0000 7fb485888640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480100970 0x7fb480198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480101330 0x7fb480198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb480110d90 con 0x7fb480106980 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb480110c10 con 0x7fb480100970 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb480110f10 con 0x7fb480101330 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33790/0 (socket says 172.21.15.122:33790) 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2714589440 learned_addr learned my addr 172.21.15.122:0/2714589440 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47effd640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480100970 0x7fb480198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480101330 0x7fb480198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2714589440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480101330 msgr2=0x7fb480198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480101330 0x7fb480198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2714589440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480100970 msgr2=0x7fb480198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480100970 0x7fb480198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2714589440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48019d300 con 0x7fb480106980 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb470005940 tx=0x7fb47002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb484886640 1 -- 172.21.15.122:0/2714589440 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470041070 con 0x7fb480106980 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48019d590 con 0x7fb480106980 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb484886640 1 -- 172.21.15.122:0/2714589440 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb470004450 con 0x7fb480106980 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb484886640 1 -- 172.21.15.122:0/2714589440 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb470003d30 con 0x7fb480106980 2024-04-02T13:48:54.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.064+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4801a4e80 con 0x7fb480106980 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.067+0000 7fb488479740 1 Processor -- start 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.067+0000 7fb484886640 1 -- 172.21.15.122:0/2714589440 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb470044070 con 0x7fb480106980 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.067+0000 7fb488479740 1 -- start start 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b77a0 0x5573622b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622c0ca0 0x5573622c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55736221b7b0 con 0x5573622b80c0 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5573621ba570 con 0x5573622c0ca0 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb488479740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557362217760 con 0x5573622b77a0 2024-04-02T13:48:54.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33792/0 (socket says 172.21.15.122:33792) 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 -- 172.21.15.122:0/3771489325 learned_addr learned my addr 172.21.15.122:0/3771489325 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622c0ca0 0x5573622c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b77a0 0x5573622b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 -- 172.21.15.122:0/3771489325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b77a0 msgr2=0x5573622b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b77a0 0x5573622b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 -- 172.21.15.122:0/3771489325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622c0ca0 msgr2=0x5573622c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622c0ca0 0x5573622c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 -- 172.21.15.122:0/3771489325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557361f2f3f0 con 0x5573622b80c0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb484886640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 0x7fb4680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47effd640 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb478009fd0 tx=0x7fb4780118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a056bfcdbd3c4b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 0x7fb4680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb465ffb640 1 -- 172.21.15.122:0/3771489325 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47801e070 con 0x5573622b80c0 2024-04-02T13:48:54.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb484886640 1 -- 172.21.15.122:0/2714589440 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb4700f2500 con 0x7fb480106980 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb465ffb640 1 -- 172.21.15.122:0/3771489325 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478003c30 con 0x5573622b80c0 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.068+0000 7fb465ffb640 1 -- 172.21.15.122:0/3771489325 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47800f9e0 con 0x5573622b80c0 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- 172.21.15.122:0/3771489325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 msgr2=0x5573622c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb478009fd0 tx=0x7fb4780118d0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 0x7fb4680a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5573622b3060 tx=0x7fb474009640 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- 172.21.15.122:0/3771489325 shutdown_connections 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622c0ca0 0x5573622c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622c0760 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 --2- 172.21.15.122:0/3771489325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b77a0 0x5573622b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- 172.21.15.122:0/3771489325 >> 172.21.15.122:0/3771489325 conn(0x5573622b1ed0 msgr2=0x5573622b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- 172.21.15.122:0/3771489325 shutdown_connections 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- 172.21.15.122:0/3771489325 wait complete. 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.069+0000 7fb488479740 1 -- start start 2024-04-02T13:48:54.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 Processor -- start 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5573622c3a90 con 0x5573622c0ca0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5573621d6e80 con 0x5573622b77a0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5573622c3c10 con 0x5573622b80c0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb488479740 10 client.0 _ll_register_callbacks cb 0x557362212aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33802/0 (socket says 172.21.15.122:33802) 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 -- 172.21.15.122:0/606698781 learned_addr learned my addr 172.21.15.122:0/606698781 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 -- 172.21.15.122:0/606698781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 msgr2=0x5573622be050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622be050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 -- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 msgr2=0x5573622bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 -- 172.21.15.122:0/606698781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557361ff97d0 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47effd640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb470005ab0 tx=0x7fb4700bde60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf8d3d87455c7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.070+0000 7fb45effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47004b040 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb4700d76a0 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478003c30 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb4700dc050 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 10 client.19611 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5573622cfb80 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478003c30 con 0x5573622c0ca0 2024-04-02T13:48:54.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7fb4700c6a40 con 0x5573622c0ca0 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 10 client.19611 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 10 client.19611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 20 client.19611 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x5573622d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 4 client.19611 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.071+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557362228cf0 con 0x5573622d0fe0 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.072+0000 7fb47effd640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x5573622d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.072+0000 7fb47effd640 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x5573622d3410 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af22d2bbf1e9df2e server_cookie=9563b0d432503fb5 in_seq=0 out_seq=0 2024-04-02T13:48:54.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.072+0000 7fb45effd640 10 client.19611 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:54.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb45effd640 1 -- 172.21.15.122:0/606698781 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557362228cf0 con 0x5573622d0fe0 2024-04-02T13:48:54.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb45effd640 10 client.19611 handle_command_reply: tid=0 2024-04-02T13:48:54.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 client.19611 shutdown 2024-04-02T13:48:54.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 msgr2=0x5573622be590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb470005ab0 tx=0x7fb4700bde60 comp rx=0 tx=0).stop 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 shutdown_connections 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x5573622d3410 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb45effd640 0 client.19611 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622be590 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622be050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.074+0000 7fb488479740 1 --2- 172.21.15.122:0/606698781 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.075+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 >> 172.21.15.122:0/606698781 conn(0x5573622b1ed0 msgr2=0x5573622b3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.075+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 shutdown_connections 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.075+0000 7fb488479740 1 -- 172.21.15.122:0/606698781 wait complete. 2024-04-02T13:48:54.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.075+0000 7fb488479740 20 client.19611 trim_cache size 0 max 16384 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.088+0000 7fb488479740 1 Processor -- start 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 -- start start 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801aefe0 con 0x5573622c0ca0 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b0bb0 con 0x5573622b77a0 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb488479740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801af6b0 con 0x5573622b80c0 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33812/0 (socket says 172.21.15.122:33812) 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48670/0 (socket says 172.21.15.122:48670) 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49502/0 (socket says 172.21.15.122:49502) 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2822576050 learned_addr learned my addr 172.21.15.122:0/2822576050 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2822576050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 msgr2=0x5573622cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2822576050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 msgr2=0x5573622c0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 -- 172.21.15.122:0/2822576050 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573622bc060 con 0x5573622c0ca0 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb470006010 tx=0x7fb4700bded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04acc391c22e516 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb4657fa640 1 -- 172.21.15.122:0/2822576050 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47004b070 con 0x5573622c0ca0 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb4657fa640 1 -- 172.21.15.122:0/2822576050 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4700a0030 con 0x5573622c0ca0 2024-04-02T13:48:54.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.089+0000 7fb4657fa640 1 -- 172.21.15.122:0/2822576050 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4700be030 con 0x5573622c0ca0 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 -- 172.21.15.122:0/2822576050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 msgr2=0x5573622b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fb470006010 tx=0x7fb4700bded0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 -- 172.21.15.122:0/2822576050 shutdown_connections 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622c0ca0 0x5573622b0db0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622b80c0 0x5573622cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 --2- 172.21.15.122:0/2822576050 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 -- 172.21.15.122:0/2822576050 >> 172.21.15.122:0/2822576050 conn(0x5573622b1ed0 msgr2=0x5573622b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 -- 172.21.15.122:0/2822576050 shutdown_connections 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.090+0000 7fb488479740 1 -- 172.21.15.122:0/2822576050 wait complete. 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 -- start start 2024-04-02T13:48:54.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 Processor -- start 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 0x5573622baa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801b04e0 con 0x5573622b80c0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801afdb0 con 0x5573622b77a0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801acfe0 con 0x5573622c0ca0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb488479740 10 client.0 _ll_register_callbacks cb 0x5573622c0bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33828/0 (socket says 172.21.15.122:33828) 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb47e7fc640 1 -- 172.21.15.122:0/791510234 learned_addr learned my addr 172.21.15.122:0/791510234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb47effd640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.091+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 0x5573622baa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 -- 172.21.15.122:0/791510234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 msgr2=0x5573622baa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 0x5573622baa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 -- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 msgr2=0x5573622c9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 -- 172.21.15.122:0/791510234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573622bb280 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47f7fe640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 0x5573622baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb47e7fc640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb47400fd90 tx=0x7fb474004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2d943142028a64 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474022b30 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7fb474022cd0 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4700a0030 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fb474036050 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb488479740 10 client.19617 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5573623375c0 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4700a0030 con 0x5573622b80c0 2024-04-02T13:48:54.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.092+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7fb474003c70 con 0x5573622b80c0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 10 client.19617 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 10 client.19617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 20 client.19617 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x557362339e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 4 client.19617 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557362223040 con 0x5573622d0fe0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb47effd640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x557362339e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb47effd640 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x557362339e30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8979616f59f44fa4 server_cookie=b79b29f779569191 in_seq=0 out_seq=0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.093+0000 7fb45effd640 10 client.19617 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb45effd640 1 -- 172.21.15.122:0/791510234 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557362223040 con 0x5573622d0fe0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb45effd640 10 client.19617 handle_command_reply: tid=0 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 client.19617 shutdown 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 msgr2=0x5573622ba510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fb47400fd90 tx=0x7fb474004730 comp rx=0 tx=0).stop 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 shutdown_connections 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5573622d0fe0 0x557362339e30 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5573622c0ca0 0x5573622baa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5573622b80c0 0x5573622ba510 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 --2- 172.21.15.122:0/791510234 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5573622b77a0 0x5573622c9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb45effd640 0 client.19617 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 >> 172.21.15.122:0/791510234 conn(0x5573622b1ed0 msgr2=0x5573622be360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 shutdown_connections 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 1 -- 172.21.15.122:0/791510234 wait complete. 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.095+0000 7fb488479740 20 client.19617 trim_cache size 0 max 16384 2024-04-02T13:48:54.103 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:48:54.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 msgr2=0x7fb4680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 0x7fb4680a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5573622b3060 tx=0x7fb474009640 comp rx=0 tx=0).stop 2024-04-02T13:48:54.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 msgr2=0x7fb48019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb470005940 tx=0x7fb47002cb30 comp rx=0 tx=0).stop 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 shutdown_connections 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4680a3d20 0x7fb4680a61e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb480106980 0x7fb48019cbd0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb480101330 0x7fb480198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 --2- 172.21.15.122:0/2714589440 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb480100970 0x7fb480198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 >> 172.21.15.122:0/2714589440 conn(0x7fb480077b00 msgr2=0x7fb480068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 shutdown_connections 2024-04-02T13:48:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:54.096+0000 7fb485888640 1 -- 172.21.15.122:0/2714589440 wait complete. 2024-04-02T13:48:54.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 5617 32.89 MB/sec warmup 14 sec latency 86.905 ms 2024-04-02T13:48:54.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:54 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:55.016 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:54 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:55.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:54 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:55.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:48:55.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:48:55.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 -- 172.21.15.122:0/1183126310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc1059f0 msgr2=0x7f49bc105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 --2- 172.21.15.122:0/1183126310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc1059f0 0x7f49bc105dd0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009900 tx=0x7f49b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 -- 172.21.15.122:0/1183126310 shutdown_connections 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 --2- 172.21.15.122:0/1183126310 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101ad0 0x7f49bc10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 --2- 172.21.15.122:0/1183126310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc101110 0x7f49bc101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 --2- 172.21.15.122:0/1183126310 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc1059f0 0x7f49bc105dd0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 -- 172.21.15.122:0/1183126310 >> 172.21.15.122:0/1183126310 conn(0x7f49bc076fc0 msgr2=0x7f49bc0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 -- 172.21.15.122:0/1183126310 shutdown_connections 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.220+0000 7f49c3881640 1 -- 172.21.15.122:0/1183126310 wait complete. 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 Processor -- start 2024-04-02T13:48:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 -- start start 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101110 0x7f49bc199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc1059f0 0x7f49bc1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110e20 con 0x7f49bc101ad0 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110ca0 con 0x7f49bc101110 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c3881640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc110fa0 con 0x7f49bc1059f0 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33850/0 (socket says 172.21.15.122:33850) 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 -- 172.21.15.122:0/233898639 learned_addr learned my addr 172.21.15.122:0/233898639 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c161d640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101110 0x7f49bc199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc1059f0 0x7f49bc1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 -- 172.21.15.122:0/233898639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc1059f0 msgr2=0x7f49bc1a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc1059f0 0x7f49bc1a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 -- 172.21.15.122:0/233898639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101110 msgr2=0x7f49bc199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101110 0x7f49bc199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.221+0000 7f49c0e1c640 1 -- 172.21.15.122:0/233898639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49bc1a4740 con 0x7f49bc101ad0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49b27fc640 1 -- 172.21.15.122:0/233898639 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00cd80 con 0x7f49bc101ad0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49b27fc640 1 -- 172.21.15.122:0/233898639 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49ac010910 con 0x7f49bc101ad0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49bc1a4910 con 0x7f49bc101ad0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49b27fc640 1 -- 172.21.15.122:0/233898639 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac018990 con 0x7f49bc101ad0 2024-04-02T13:48:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.222+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49bc1a4e60 con 0x7f49bc101ad0 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.224+0000 7f49b27fc640 1 -- 172.21.15.122:0/233898639 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f49ac010ac0 con 0x7f49bc101ad0 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 Processor -- start 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 -- start start 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49b27fc640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8286280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8288bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b4e81e1770 con 0x55b4e827d2c0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b4e8189960 con 0x55b4e827dbe0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c6461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b4e81dd8c0 con 0x55b4e82867c0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c0e1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8286280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33852/0 (socket says 172.21.15.122:33852) 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 -- 172.21.15.122:0/465853076 learned_addr learned my addr 172.21.15.122:0/465853076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8288bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49b27fc640 1 -- 172.21.15.122:0/233898639 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49ac0d0f80 con 0x7f49bc101ad0 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 -- 172.21.15.122:0/465853076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 msgr2=0x55b4e8288bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8288bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 -- 172.21.15.122:0/465853076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 msgr2=0x55b4e8286280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8286280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.225+0000 7f49c161d640 1 -- 172.21.15.122:0/465853076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e7fbd7e0 con 0x55b4e827d2c0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 0x7f49a80a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005fd0 tx=0x7f49b4009e90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49c161d640 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009230 tx=0x7f49b8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1735834897a665ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49a77fe640 1 -- 172.21.15.122:0/465853076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b804c070 con 0x55b4e827d2c0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49a77fe640 1 -- 172.21.15.122:0/465853076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b802c5d0 con 0x55b4e827d2c0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49a77fe640 1 -- 172.21.15.122:0/465853076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004a20 con 0x55b4e827d2c0 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49c6461740 1 -- 172.21.15.122:0/465853076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 msgr2=0x55b4e827d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.226+0000 7f49c6461740 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009230 tx=0x7f49b8005d70 comp rx=0 tx=0).stop 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- 172.21.15.122:0/465853076 shutdown_connections 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8288bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8286280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- 172.21.15.122:0/465853076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e827d6a0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- 172.21.15.122:0/465853076 >> 172.21.15.122:0/465853076 conn(0x55b4e82779f0 msgr2=0x55b4e8277dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- 172.21.15.122:0/465853076 shutdown_connections 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- 172.21.15.122:0/465853076 wait complete. 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- start start 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 Processor -- start 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8283cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8284200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b4e8289550 con 0x55b4e827d2c0 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b4e819cd20 con 0x55b4e827dbe0 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b4e82896d0 con 0x55b4e82867c0 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c6461740 10 client.0 _ll_register_callbacks cb 0x55b4e8169a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33858/0 (socket says 172.21.15.122:33858) 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c161d640 1 -- 172.21.15.122:0/1639943323 learned_addr learned my addr 172.21.15.122:0/1639943323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8284200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.227+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8283cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 -- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 msgr2=0x55b4e8283cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8283cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 -- 172.21.15.122:0/1639943323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 msgr2=0x55b4e8283780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 -- 172.21.15.122:0/1639943323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e801df70 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c161d640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8284200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55b4e8278b80 tx=0x7f49b401d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a36be0c57141420 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b4024070 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b802c5d0 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49b4003e00 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f49b4004790 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c6461740 10 client.28384 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b4e7eb5170 con 0x55b4e82867c0 2024-04-02T13:48:55.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.228+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b802c5d0 con 0x55b4e82867c0 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7f49b4005820 con 0x55b4e82867c0 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 10 client.28384 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 10 client.28384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 20 client.28384 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e8296b20 0x55b4e8298f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 4 client.28384 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4e81eecb0 con 0x55b4e8296b20 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c161d640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e8296b20 0x55b4e8298f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49a4ff9640 10 client.28384 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.229+0000 7f49c161d640 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e8296b20 0x55b4e8298f70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8865ade8f564d7ce server_cookie=c03fe76cc5e8c08d in_seq=0 out_seq=0 2024-04-02T13:48:55.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49a4ff9640 1 -- 172.21.15.122:0/1639943323 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4e81eecb0 con 0x55b4e8296b20 2024-04-02T13:48:55.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49a4ff9640 10 client.28384 handle_command_reply: tid=0 2024-04-02T13:48:55.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 client.28384 shutdown 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 msgr2=0x55b4e8284200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8284200 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55b4e8278b80 tx=0x7f49b401d000 comp rx=0 tx=0).stop 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 shutdown_connections 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e8296b20 0x55b4e8298f70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e8284200 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8283cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 --2- 172.21.15.122:0/1639943323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8283780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49a4ff9640 0 client.28384 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 >> 172.21.15.122:0/1639943323 conn(0x55b4e82779f0 msgr2=0x55b4e8279740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 shutdown_connections 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 1 -- 172.21.15.122:0/1639943323 wait complete. 2024-04-02T13:48:55.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.233+0000 7f49c6461740 20 client.28384 trim_cache size 0 max 16384 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 Processor -- start 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 -- start start 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8292470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e829e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1acea0 con 0x55b4e827d2c0 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1aea70 con 0x55b4e827dbe0 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c6461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ad570 con 0x55b4e82867c0 2024-04-02T13:48:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33864/0 (socket says 172.21.15.122:33864) 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 -- 172.21.15.122:0/1156151113 learned_addr learned my addr 172.21.15.122:0/1156151113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8292470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e829e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 -- 172.21.15.122:0/1156151113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 msgr2=0x55b4e829e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e829e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 -- 172.21.15.122:0/1156151113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 msgr2=0x55b4e8292470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8292470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.251+0000 7f49c161d640 1 -- 172.21.15.122:0/1156151113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4e8277820 con 0x55b4e827d2c0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c161d640 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009230 tx=0x7f49b802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f3d1b312caf6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49a6ffd640 1 -- 172.21.15.122:0/1156151113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b804c040 con 0x55b4e827d2c0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49a6ffd640 1 -- 172.21.15.122:0/1156151113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b8009520 con 0x55b4e827d2c0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49a6ffd640 1 -- 172.21.15.122:0/1156151113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8004f80 con 0x55b4e827d2c0 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 -- 172.21.15.122:0/1156151113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 msgr2=0x55b4e8285eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009230 tx=0x7f49b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 -- 172.21.15.122:0/1156151113 shutdown_connections 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e829e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827dbe0 0x55b4e8292470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 --2- 172.21.15.122:0/1156151113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827d2c0 0x55b4e8285eb0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 -- 172.21.15.122:0/1156151113 >> 172.21.15.122:0/1156151113 conn(0x55b4e82779f0 msgr2=0x55b4e82796b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 -- 172.21.15.122:0/1156151113 shutdown_connections 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.252+0000 7f49c6461740 1 -- 172.21.15.122:0/1156151113 wait complete. 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 -- start start 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 Processor -- start 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e828be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1ae3a0 con 0x55b4e827dbe0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1adc70 con 0x55b4e827d2c0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49bc1aaea0 con 0x55b4e82867c0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c6461740 10 client.0 _ll_register_callbacks cb 0x7f49b4009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c0e1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c161d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c161d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49574/0 (socket says 172.21.15.122:49574) 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c161d640 1 -- 172.21.15.122:0/3722234753 learned_addr learned my addr 172.21.15.122:0/3722234753 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e828be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.253+0000 7f49c0e1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33872/0 (socket says 172.21.15.122:33872) 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 -- 172.21.15.122:0/3722234753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 msgr2=0x55b4e828be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e828be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 -- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 msgr2=0x55b4e828eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 -- 172.21.15.122:0/3722234753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4e8288bb0 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c161d640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c1e1e640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e828be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c0e1c640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0ce280 tx=0x7f49ac025360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978943f78b77b306 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0afd70 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f49ac09ae80 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b8009520 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f49ac0afb90 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 10 client.19635 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b4e82fdc80 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b8009520 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7f49ac0991f0 con 0x55b4e827dbe0 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 10 client.19635 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 10 client.19635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 20 client.19635 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e82fe3d0 0x55b4e83007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 4 client.19635 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.254+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4e81e9000 con 0x55b4e82fe3d0 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.255+0000 7f49c161d640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e82fe3d0 0x55b4e83007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.255+0000 7f49a4ff9640 10 client.19635 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.255+0000 7f49c161d640 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e82fe3d0 0x55b4e83007f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cecfdfea27b4809 server_cookie=b911bdbf133c613c in_seq=0 out_seq=0 2024-04-02T13:48:55.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49a4ff9640 1 -- 172.21.15.122:0/3722234753 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4e81e9000 con 0x55b4e82fe3d0 2024-04-02T13:48:55.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49a4ff9640 10 client.19635 handle_command_reply: tid=0 2024-04-02T13:48:55.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 client.19635 shutdown 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 msgr2=0x55b4e828b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f49ac0ce280 tx=0x7f49ac025360 comp rx=0 tx=0).stop 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 shutdown_connections 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b4e82fe3d0 0x55b4e83007f0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b4e82867c0 0x55b4e828be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49a4ff9640 0 client.19635 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b4e827dbe0 0x55b4e828b950 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 --2- 172.21.15.122:0/3722234753 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b4e827d2c0 0x55b4e828eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 >> 172.21.15.122:0/3722234753 conn(0x55b4e82779f0 msgr2=0x55b4e8276220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 shutdown_connections 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 1 -- 172.21.15.122:0/3722234753 wait complete. 2024-04-02T13:48:55.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.264+0000 7f49c6461740 20 client.19635 trim_cache size 0 max 16384 2024-04-02T13:48:55.270 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:55.270 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:48:55.270 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "035a0531-231c-46ee-805f-82b00de94c84", 2024-04-02T13:48:55.270 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:48:55.270 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 msgr2=0x7f49a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 0x7f49a80a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005fd0 tx=0x7f49b4009e90 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 msgr2=0x7f49bc1996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00e9f0 tx=0x7f49ac00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 shutdown_connections 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49a80a3d20 0x7f49a80a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49bc1059f0 0x7f49bc1a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49bc101ad0 0x7f49bc1996c0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 --2- 172.21.15.122:0/233898639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f49bc101110 0x7f49bc199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 >> 172.21.15.122:0/233898639 conn(0x7f49bc076fc0 msgr2=0x7f49bc1038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 shutdown_connections 2024-04-02T13:48:55.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.265+0000 7f49c3881640 1 -- 172.21.15.122:0/233898639 wait complete. 2024-04-02T13:48:55.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:48:55.280 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:48:55.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 6097 33.65 MB/sec warmup 15 sec latency 129.147 ms 2024-04-02T13:48:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2698817615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 msgr2=0x7f5af0116460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2698817615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af0116460 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009ca0 tx=0x7f5ae802c100 comp rx=0 tx=0).stop 2024-04-02T13:48:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2698817615 shutdown_connections 2024-04-02T13:48:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2698817615 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af0116460 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2698817615 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 0x7f5af0109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2698817615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.426+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2698817615 >> 172.21.15.122:0/2698817615 conn(0x7f5af006d040 msgr2=0x7f5af006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.427+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2698817615 shutdown_connections 2024-04-02T13:48:55.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.427+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2698817615 wait complete. 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.427+0000 7f5af6cb1640 1 Processor -- start 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.427+0000 7f5af6cb1640 1 -- start start 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 0x7f5af01a2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af01a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0118530 con 0x7f5af0108b00 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01183b0 con 0x7f5af00695f0 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af6cb1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01186b0 con 0x7f5af010e590 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33876/0 (socket says 172.21.15.122:33876) 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5aeffff640 1 -- 172.21.15.122:0/2707741033 learned_addr learned my addr 172.21.15.122:0/2707741033 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 0x7f5af01a2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.428+0000 7f5af524e640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af01a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 -- 172.21.15.122:0/2707741033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 msgr2=0x7f5af01a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af01a7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 -- 172.21.15.122:0/2707741033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 msgr2=0x7f5af01a2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 0x7f5af01a2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 -- 172.21.15.122:0/2707741033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af01a7d30 con 0x7f5af0108b00 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5aeffff640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e9f0 tx=0x7f5ae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.429+0000 7f5af524e640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af01a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.430+0000 7f5aedffb640 1 -- 172.21.15.122:0/2707741033 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000ccc0 con 0x7f5af0108b00 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.430+0000 7f5aedffb640 1 -- 172.21.15.122:0/2707741033 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae0010910 con 0x7f5af0108b00 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.430+0000 7f5aedffb640 1 -- 172.21.15.122:0/2707741033 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00188e0 con 0x7f5af0108b00 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.430+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01a8020 con 0x7f5af0108b00 2024-04-02T13:48:55.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.430+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01af8b0 con 0x7f5af0108b00 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 Processor -- start 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 -- start start 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d52dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d536dd0 0x557e9d5391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557e9d491830 con 0x557e9d52d8d0 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557e9d439820 con 0x557e9d52e1f0 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af9971740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557e9d48d7e0 con 0x557e9d536dd0 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.434+0000 7f5af524e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d52dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49600/0 (socket says 172.21.15.122:49600) 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af4a4d640 1 -- 172.21.15.122:0/961583975 learned_addr learned my addr 172.21.15.122:0/961583975 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5aedffb640 1 -- 172.21.15.122:0/2707741033 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5ae0010ac0 con 0x7f5af0108b00 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5aeffff640 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d536dd0 0x557e9d5391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 -- 172.21.15.122:0/961583975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d536dd0 msgr2=0x557e9d5391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d536dd0 0x557e9d5391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 -- 172.21.15.122:0/961583975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 msgr2=0x557e9d536890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 -- 172.21.15.122:0/961583975 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d26f7d0 con 0x557e9d52d8d0 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.435+0000 7f5af524e640 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d52dcb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009bf0 tx=0x7f5ae8032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a357553267d9eb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.436+0000 7f5aedffb640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 0x7f5ad80a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.436+0000 7f5aedffb640 1 -- 172.21.15.122:0/2707741033 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f5ae00d1050 con 0x7f5af0108b00 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.436+0000 7f5ad6ffd640 1 -- 172.21.15.122:0/961583975 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8044020 con 0x557e9d52d8d0 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.436+0000 7f5ad6ffd640 1 -- 172.21.15.122:0/961583975 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8003e50 con 0x557e9d52d8d0 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.437+0000 7f5ad6ffd640 1 -- 172.21.15.122:0/961583975 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8039040 con 0x557e9d52d8d0 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.437+0000 7f5af9971740 1 -- 172.21.15.122:0/961583975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 msgr2=0x557e9d52dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.437+0000 7f5af9971740 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d52dcb0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009bf0 tx=0x7f5ae8032340 comp rx=0 tx=0).stop 2024-04-02T13:48:55.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.437+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 0x7f5ad80a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 -- 172.21.15.122:0/961583975 shutdown_connections 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d536dd0 0x557e9d5391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d536890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 --2- 172.21.15.122:0/961583975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d52dcb0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 -- 172.21.15.122:0/961583975 >> 172.21.15.122:0/961583975 conn(0x557e9d528000 msgr2=0x557e9d5283e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 -- 172.21.15.122:0/961583975 shutdown_connections 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 0x7f5ad80a6360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557e9d52d3a0 tx=0x7f5ae4008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.438+0000 7f5af9971740 1 -- 172.21.15.122:0/961583975 wait complete. 2024-04-02T13:48:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 -- start start 2024-04-02T13:48:55.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 Processor -- start 2024-04-02T13:48:55.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d533a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d536dd0 0x557e9d533fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557e9d539bb0 con 0x557e9d536dd0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557e9d44cd30 con 0x557e9d52e1f0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557e9d539d30 con 0x557e9d52d8d0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.439+0000 7f5af9971740 10 client.0 _ll_register_callbacks cb 0x557e9d44c720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:48800/0 (socket says 172.21.15.122:48800) 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 -- 172.21.15.122:0/3155575117 learned_addr learned my addr 172.21.15.122:0/3155575117 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af524e640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d536dd0 0x557e9d533fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d533a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 -- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 msgr2=0x557e9d533a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d533a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 -- 172.21.15.122:0/3155575117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d536dd0 msgr2=0x557e9d533fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d536dd0 0x557e9d533fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 -- 172.21.15.122:0/3155575117 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d1a53f0 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4006fd0 tx=0x7f5ae40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ebd7b9f19091e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.440+0000 7f5acffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4014930 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae8003e50 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f5ae4014ad0 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5ae402e050 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 10 client.28399 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557e9d1650f0 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae8003e50 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7f5ae4029030 con 0x557e9d52d8d0 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 10 client.28399 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:55.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 10 client.28399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 20 client.28399 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d547240 0x557e9d5496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 4 client.28399 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.441+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e9d49ed70 con 0x557e9d547240 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.443+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d547240 0x557e9d5496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.443+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d547240 0x557e9d5496c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49c8cd4e151b94d4 server_cookie=ddc4bdde480b31c9 in_seq=0 out_seq=0 2024-04-02T13:48:55.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.443+0000 7f5acffff640 10 client.28399 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5acffff640 1 -- 172.21.15.122:0/3155575117 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e9d49ed70 con 0x557e9d547240 2024-04-02T13:48:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5acffff640 10 client.28399 handle_command_reply: tid=0 2024-04-02T13:48:55.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 client.28399 shutdown 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 msgr2=0x557e9d5366f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4006fd0 tx=0x7f5ae40046c0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 shutdown_connections 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d547240 0x557e9d5496c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d536dd0 0x557e9d533fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d52e1f0 0x557e9d533a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 --2- 172.21.15.122:0/3155575117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d5366f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 >> 172.21.15.122:0/3155575117 conn(0x557e9d528000 msgr2=0x557e9d529d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5acffff640 0 client.28399 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 shutdown_connections 2024-04-02T13:48:55.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 1 -- 172.21.15.122:0/3155575117 wait complete. 2024-04-02T13:48:55.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.449+0000 7f5af9971740 20 client.28399 trim_cache size 0 max 16384 2024-04-02T13:48:55.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 Processor -- start 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 -- start start 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d542ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52e1f0 0x557e9d5361b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01b9ac0 con 0x557e9d52d8d0 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01bb690 con 0x557e9d528000 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.471+0000 7f5af9971740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ba190 con 0x557e9d52e1f0 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33906/0 (socket says 172.21.15.122:33906) 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 -- 172.21.15.122:0/1411615833 learned_addr learned my addr 172.21.15.122:0/1411615833 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d542ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5af524e640 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52e1f0 0x557e9d5361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 -- 172.21.15.122:0/1411615833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52e1f0 msgr2=0x557e9d5361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52e1f0 0x557e9d5361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 -- 172.21.15.122:0/1411615833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 msgr2=0x557e9d542ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d542ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 -- 172.21.15.122:0/1411615833 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e9d545a60 con 0x557e9d52d8d0 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5aeffff640 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00995d0 tx=0x7f5ae00d4ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1a765248332f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5ad67fc640 1 -- 172.21.15.122:0/1411615833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae009c560 con 0x557e9d52d8d0 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5ad67fc640 1 -- 172.21.15.122:0/1411615833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae009d300 con 0x557e9d52d8d0 2024-04-02T13:48:55.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.472+0000 7f5ad67fc640 1 -- 172.21.15.122:0/1411615833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00ce630 con 0x557e9d52d8d0 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- 172.21.15.122:0/1411615833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 msgr2=0x557e9d532350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00995d0 tx=0x7f5ae00d4ba0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- 172.21.15.122:0/1411615833 shutdown_connections 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52e1f0 0x557e9d5361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52d8d0 0x557e9d532350 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 --2- 172.21.15.122:0/1411615833 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d542ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- 172.21.15.122:0/1411615833 >> 172.21.15.122:0/1411615833 conn(0x557e9d534120 msgr2=0x557e9d527240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- 172.21.15.122:0/1411615833 shutdown_connections 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- 172.21.15.122:0/1411615833 wait complete. 2024-04-02T13:48:55.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.473+0000 7f5af9971740 1 -- start start 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 Processor -- start 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d530b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d531090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 0x557e9d5ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01bafc0 con 0x557e9d52e1f0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ba890 con 0x557e9d528000 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01b7ac0 con 0x557e9d52d8d0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af9971740 10 client.0 _ll_register_callbacks cb 0x7f5ae40166b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af524e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 0x557e9d5ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d530b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d531090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af4a4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d530b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:49628/0 (socket says 172.21.15.122:49628) 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af4a4d640 1 -- 172.21.15.122:0/3717510471 learned_addr learned my addr 172.21.15.122:0/3717510471 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:48:55.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 -- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 msgr2=0x557e9d530b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d530b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 -- 172.21.15.122:0/3717510471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 msgr2=0x557e9d5ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 0x557e9d5ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5aeffff640 1 -- 172.21.15.122:0/3717510471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e9d527e30 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.474+0000 7f5af524e640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 0x557e9d5ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5aeffff640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d531090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x557e9d532930 tx=0x7f5ae000cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb35b7313a486818 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae00a0fb0 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9525+0+0 (secure 0 0 0) 0x7f5ae0099270 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae009d300 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5ae00a5870 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 10 client.28405 fetch_fsmap learned FSMap version 49 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x557e9d5add80 con 0x557e9d52d8d0 2024-04-02T13:48:55.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ae009d300 con 0x557e9d52d8d0 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7f5ae009c050 con 0x557e9d52d8d0 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 10 client.28405 fetch_fsmap finished waiting for FSMap version 49 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 10 client.28405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 20 client.28405 populate_metadata read hostname 'smithi122' 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d536dd0 0x557e9d5b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 4 client.28405 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.475+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557e9d4990c0 con 0x557e9d536dd0 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.476+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d536dd0 0x557e9d5b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.476+0000 7f5af4a4d640 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d536dd0 0x557e9d5b0430 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c1ede4fccdb9cc server_cookie=e08c18fc194be54f in_seq=0 out_seq=0 2024-04-02T13:48:55.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.476+0000 7f5acffff640 10 client.28405 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5acffff640 1 -- 172.21.15.122:0/3717510471 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x557e9d4990c0 con 0x557e9d536dd0 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5acffff640 10 client.28405 handle_command_reply: tid=0 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 client.28405 shutdown 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 msgr2=0x557e9d531090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d531090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x557e9d532930 tx=0x7f5ae000cee0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 shutdown_connections 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557e9d536dd0 0x557e9d5b0430 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557e9d52e1f0 0x557e9d5ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e9d52d8d0 0x557e9d531090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5acffff640 0 client.28405 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 --2- 172.21.15.122:0/3717510471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557e9d528000 0x557e9d530b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.480+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 >> 172.21.15.122:0/3717510471 conn(0x557e9d534120 msgr2=0x557e9d527240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 shutdown_connections 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af9971740 1 -- 172.21.15.122:0/3717510471 wait complete. 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af9971740 20 client.28405 trim_cache size 0 max 16384 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "scrub active (151 inodes in the stack)", 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "035a0531-231c-46ee-805f-82b00de94c84": { 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "035a0531-231c-46ee-805f-82b00de94c84", 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:48:55.487 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:48:55.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 msgr2=0x7f5ad80a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 0x7f5ad80a6360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x557e9d52d3a0 tx=0x7f5ae4008040 comp rx=0 tx=0).stop 2024-04-02T13:48:55.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 msgr2=0x7f5af01a3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:48:55.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000e9f0 tx=0x7f5ae000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 shutdown_connections 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5ad80a3ea0 0x7f5ad80a6360 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5af010e590 0x7f5af01a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5af0108b00 0x7f5af01a3280 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 --2- 172.21.15.122:0/2707741033 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5af00695f0 0x7f5af01a2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 >> 172.21.15.122:0/2707741033 conn(0x7f5af006d040 msgr2=0x7f5af0071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 shutdown_connections 2024-04-02T13:48:55.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:48:55.481+0000 7f5af6cb1640 1 -- 172.21.15.122:0/2707741033 wait complete. 2024-04-02T13:48:55.495 INFO:tasks.cephfs.filesystem:scrub status for tag:035a0531-231c-46ee-805f-82b00de94c84 - {'path': '/', 'tag': '035a0531-231c-46ee-805f-82b00de94c84', 'options': 'recursive,force'} 2024-04-02T13:48:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: pgmap v429: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 187 op/s 2024-04-02T13:48:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:48:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:55 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: pgmap v429: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 187 op/s 2024-04-02T13:48:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:48:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:55 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: pgmap v429: 129 pgs: 129 active+clean; 651 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 187 op/s 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:55 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:56.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 6515 33.52 MB/sec warmup 16 sec latency 199.212 ms 2024-04-02T13:48:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:56.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:56 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:48:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:56.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:56 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:56 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:48:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:56 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:56 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:48:57.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:56 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:48:57.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 6856 32.83 MB/sec warmup 17 sec latency 208.336 ms 2024-04-02T13:48:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:57 smithi187 ceph-mon[36064]: pgmap v430: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 289 op/s 2024-04-02T13:48:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:57 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:57 smithi188 ceph-mon[36117]: pgmap v430: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 289 op/s 2024-04-02T13:48:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:57 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:57 smithi122 ceph-mon[32114]: pgmap v430: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.8 MiB/s wr, 289 op/s 2024-04-02T13:48:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:57 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:48:58.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7174 31.78 MB/sec warmup 18 sec latency 73.765 ms 2024-04-02T13:48:59.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7526 30.80 MB/sec warmup 19 sec latency 94.987 ms 2024-04-02T13:48:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:48:59 smithi187 ceph-mon[36064]: pgmap v431: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:48:59 smithi188 ceph-mon[36117]: pgmap v431: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:00.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:48:59 smithi122 ceph-mon[32114]: pgmap v431: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:00.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7606 29.33 MB/sec warmup 20 sec latency 329.542 ms 2024-04-02T13:49:01.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7631 27.98 MB/sec warmup 21 sec latency 1154.538 ms 2024-04-02T13:49:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:01 smithi187 ceph-mon[36064]: pgmap v432: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:01 smithi188 ceph-mon[36117]: pgmap v432: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:01 smithi122 ceph-mon[32114]: pgmap v432: 129 pgs: 129 active+clean; 660 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 188 op/s 2024-04-02T13:49:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:02.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7631 26.71 MB/sec warmup 22 sec latency 2154.580 ms 2024-04-02T13:49:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:02.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:02 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:02 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:02 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:03.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7675 25.69 MB/sec warmup 23 sec latency 2535.534 ms 2024-04-02T13:49:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:03 smithi187 ceph-mon[36064]: pgmap v433: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 363 op/s 2024-04-02T13:49:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:03 smithi188 ceph-mon[36117]: pgmap v433: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 363 op/s 2024-04-02T13:49:04.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:03 smithi122 ceph-mon[32114]: pgmap v433: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 20 MiB/s wr, 363 op/s 2024-04-02T13:49:04.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7675 24.62 MB/sec warmup 24 sec latency 1621.842 ms 2024-04-02T13:49:05.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7675 23.64 MB/sec warmup 25 sec latency 2621.961 ms 2024-04-02T13:49:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:05 smithi187 ceph-mon[36064]: pgmap v434: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-04-02T13:49:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:05 smithi188 ceph-mon[36117]: pgmap v434: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-04-02T13:49:06.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:05 smithi122 ceph-mon[32114]: pgmap v434: 129 pgs: 129 active+clean; 825 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 18 MiB/s wr, 275 op/s 2024-04-02T13:49:06.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7675 22.73 MB/sec warmup 26 sec latency 3622.064 ms 2024-04-02T13:49:07.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7903 22.10 MB/sec warmup 27 sec latency 4619.493 ms 2024-04-02T13:49:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:07 smithi187 ceph-mon[36064]: pgmap v435: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 18 MiB/s wr, 301 op/s 2024-04-02T13:49:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:07 smithi188 ceph-mon[36117]: pgmap v435: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 18 MiB/s wr, 301 op/s 2024-04-02T13:49:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:07 smithi122 ceph-mon[32114]: pgmap v435: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 18 MiB/s wr, 301 op/s 2024-04-02T13:49:08.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 7975 21.53 MB/sec warmup 28 sec latency 5619.532 ms 2024-04-02T13:49:09.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 8443 21.67 MB/sec warmup 29 sec latency 6619.567 ms 2024-04-02T13:49:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:09 smithi187 ceph-mon[36064]: pgmap v436: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:09 smithi188 ceph-mon[36117]: pgmap v436: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:09 smithi122 ceph-mon[32114]: pgmap v436: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:10.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 9299 22.03 MB/sec warmup 30 sec latency 7619.603 ms 2024-04-02T13:49:11.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 9963 23.08 MB/sec warmup 31 sec latency 8619.656 ms 2024-04-02T13:49:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:11 smithi187 ceph-mon[36064]: pgmap v437: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:11 smithi188 ceph-mon[36117]: pgmap v437: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:11 smithi122 ceph-mon[32114]: pgmap v437: 129 pgs: 129 active+clean; 826 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 200 op/s 2024-04-02T13:49:12.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 10493 23.33 MB/sec warmup 32 sec latency 9619.698 ms 2024-04-02T13:49:13.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:12 smithi188 ceph-mon[36117]: osdmap e110: 12 total, 12 up, 12 in 2024-04-02T13:49:13.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:12 smithi122 ceph-mon[32114]: osdmap e110: 12 total, 12 up, 12 in 2024-04-02T13:49:13.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:12 smithi187 ceph-mon[36064]: osdmap e110: 12 total, 12 up, 12 in 2024-04-02T13:49:13.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 10852 23.26 MB/sec warmup 33 sec latency 10619.726 ms 2024-04-02T13:49:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:13 smithi188 ceph-mon[36117]: pgmap v439: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:13 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:13 smithi122 ceph-mon[32114]: pgmap v439: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:13 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:14.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:13 smithi187 ceph-mon[36064]: pgmap v439: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:14.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:13 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:14.320 INFO:tasks.workunit.client.0.smithi122.stdout: 10 11288 22.99 MB/sec warmup 34 sec latency 11619.759 ms 2024-04-02T13:49:15.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 11968 23.37 MB/sec warmup 35 sec latency 12619.803 ms 2024-04-02T13:49:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:15 smithi188 ceph-mon[36117]: pgmap v440: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:16.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:15 smithi122 ceph-mon[32114]: pgmap v440: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:16.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:15 smithi187 ceph-mon[36064]: pgmap v440: 129 pgs: 129 active+clean; 850 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 368 op/s 2024-04-02T13:49:16.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 12594 23.71 MB/sec warmup 36 sec latency 13619.841 ms 2024-04-02T13:49:17.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 13127 23.99 MB/sec warmup 37 sec latency 13904.144 ms 2024-04-02T13:49:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:17 smithi188 ceph-mon[36117]: pgmap v441: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:17 smithi122 ceph-mon[32114]: pgmap v441: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:18.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:17 smithi187 ceph-mon[36064]: pgmap v441: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:18.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 13586 24.11 MB/sec warmup 38 sec latency 6886.629 ms 2024-04-02T13:49:19.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 14028 23.77 MB/sec warmup 39 sec latency 7886.663 ms 2024-04-02T13:49:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:19 smithi188 ceph-mon[36117]: pgmap v442: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:20.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:19 smithi122 ceph-mon[32114]: pgmap v442: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:20.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:19 smithi187 ceph-mon[36064]: pgmap v442: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:20.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 14675 24.04 MB/sec warmup 40 sec latency 8886.701 ms 2024-04-02T13:49:21.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 15386 24.65 MB/sec warmup 41 sec latency 9886.736 ms 2024-04-02T13:49:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:21 smithi188 ceph-mon[36117]: pgmap v443: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:21 smithi188 ceph-mon[36117]: osdmap e111: 12 total, 12 up, 12 in 2024-04-02T13:49:22.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:21 smithi122 ceph-mon[32114]: pgmap v443: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:22.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:21 smithi122 ceph-mon[32114]: osdmap e111: 12 total, 12 up, 12 in 2024-04-02T13:49:22.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:21 smithi187 ceph-mon[36064]: pgmap v443: 129 pgs: 129 active+clean; 874 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.4 MiB/s wr, 534 op/s 2024-04-02T13:49:22.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:21 smithi187 ceph-mon[36064]: osdmap e111: 12 total, 12 up, 12 in 2024-04-02T13:49:22.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 15896 24.69 MB/sec warmup 42 sec latency 10272.104 ms 2024-04-02T13:49:23.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 16432 24.59 MB/sec warmup 43 sec latency 10308.885 ms 2024-04-02T13:49:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:23 smithi188 ceph-mon[36117]: pgmap v445: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:24.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:23 smithi122 ceph-mon[32114]: pgmap v445: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:24.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:23 smithi187 ceph-mon[36064]: pgmap v445: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:24.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 17478 25.10 MB/sec warmup 44 sec latency 68.248 ms 2024-04-02T13:49:25.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 18575 26.16 MB/sec warmup 45 sec latency 56.523 ms 2024-04-02T13:49:25.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:49:25.496 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:49:25.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 -- 172.21.15.122:0/3765131795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b041048c0 msgr2=0x7f7b04108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 --2- 172.21.15.122:0/3765131795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b041048c0 0x7f7b04108900 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 -- 172.21.15.122:0/3765131795 shutdown_connections 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 --2- 172.21.15.122:0/3765131795 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b04108fe0 0x7f7b04110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 --2- 172.21.15.122:0/3765131795 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b041048c0 0x7f7b04108900 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 --2- 172.21.15.122:0/3765131795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 -- 172.21.15.122:0/3765131795 >> 172.21.15.122:0/3765131795 conn(0x7f7b040ff7b0 msgr2=0x7f7b04101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 -- 172.21.15.122:0/3765131795 shutdown_connections 2024-04-02T13:49:25.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.639+0000 7f7b0958d640 1 -- 172.21.15.122:0/3765131795 wait complete. 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 Processor -- start 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 -- start start 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0419a5f0 0x7f7b0419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04114090 con 0x7f7b04108fe0 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04113f10 con 0x7f7b0419a5f0 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.640+0000 7f7b0958d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04114210 con 0x7f7b04103f10 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52264/0 (socket says 172.21.15.122:52264) 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 -- 172.21.15.122:0/2986282295 learned_addr learned my addr 172.21.15.122:0/2986282295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7b037fe640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0419a5f0 0x7f7b0419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 -- 172.21.15.122:0/2986282295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 msgr2=0x7f7b041a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 -- 172.21.15.122:0/2986282295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0419a5f0 msgr2=0x7f7b0419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0419a5f0 0x7f7b0419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 -- 172.21.15.122:0/2986282295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b0419b110 con 0x7f7b04108fe0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.641+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.642+0000 7f7b00ff9640 1 -- 172.21.15.122:0/2986282295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec041070 con 0x7f7b04108fe0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.642+0000 7f7b00ff9640 1 -- 172.21.15.122:0/2986282295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0054e0 con 0x7f7b04108fe0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.642+0000 7f7b00ff9640 1 -- 172.21.15.122:0/2986282295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec003c50 con 0x7f7b04108fe0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.642+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b0419b3a0 con 0x7f7b04108fe0 2024-04-02T13:49:25.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.642+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a7590 con 0x7f7b04108fe0 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 Processor -- start 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 -- start start 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b00ff9640 1 -- 172.21.15.122:0/2986282295 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7aec044070 con 0x7f7b04108fe0 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ebe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5586f7e19830 con 0x5586f7ec75e0 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5586f7dc1820 con 0x5586f7ebe0e0 2024-04-02T13:49:25.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.645+0000 7f7b0c175740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586f7e157e0 con 0x5586f7ebea00 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52278/0 (socket says 172.21.15.122:52278) 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 -- 172.21.15.122:0/1776312181 learned_addr learned my addr 172.21.15.122:0/1776312181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7b037fe640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ebe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7b00ff9640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 -- 172.21.15.122:0/1776312181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 msgr2=0x5586f7ec70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 -- 172.21.15.122:0/1776312181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 msgr2=0x5586f7ebe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ebe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7afa7fc640 1 -- 172.21.15.122:0/1776312181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7bf77d0 con 0x5586f7ec75e0 2024-04-02T13:49:25.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.646+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ef760 tx=0x7f7aec0ba190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd195fade577ab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7b037fe640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7b00ff9640 1 -- 172.21.15.122:0/2986282295 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f7aec0f2350 con 0x7f7b04108fe0 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7af97fa640 1 -- 172.21.15.122:0/1776312181 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec04b070 con 0x5586f7ec75e0 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7af97fa640 1 -- 172.21.15.122:0/1776312181 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec005690 con 0x5586f7ec75e0 2024-04-02T13:49:25.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.647+0000 7f7af97fa640 1 -- 172.21.15.122:0/1776312181 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0bde90 con 0x5586f7ec75e0 2024-04-02T13:49:25.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- 172.21.15.122:0/1776312181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 msgr2=0x5586f7ec99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b037fe640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 0x7f7af00a6260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005fd0 tx=0x7f7afc009b30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0ef760 tx=0x7f7aec0ba190 comp rx=0 tx=0).stop 2024-04-02T13:49:25.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- 172.21.15.122:0/1776312181 shutdown_connections 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec99d0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 --2- 172.21.15.122:0/1776312181 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ebe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- 172.21.15.122:0/1776312181 >> 172.21.15.122:0/1776312181 conn(0x5586f7eb8810 msgr2=0x5586f7eb8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- 172.21.15.122:0/1776312181 shutdown_connections 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- 172.21.15.122:0/1776312181 wait complete. 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.648+0000 7f7b0c175740 1 -- start start 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 Processor -- start 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ec6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5586f7eca190 con 0x5586f7ec75e0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5586f7dd4d30 con 0x5586f7ebe0e0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586f7eca310 con 0x5586f7ebea00 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b0c175740 10 client.0 _ll_register_callbacks cb 0x5586f7dd4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ec6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52290/0 (socket says 172.21.15.122:52290) 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b037fe640 1 -- 172.21.15.122:0/569067424 learned_addr learned my addr 172.21.15.122:0/569067424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 -- 172.21.15.122:0/569067424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 msgr2=0x5586f7ec4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 -- 172.21.15.122:0/569067424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 msgr2=0x5586f7ec46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.649+0000 7f7b02ffd640 1 -- 172.21.15.122:0/569067424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7b2d3f0 con 0x5586f7ebe0e0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7b037fe640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ec6df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7af40139a0 tx=0x7f7af4013da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abb411e0c735c9b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7aea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011d50 con 0x5586f7ebe0e0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586f7e26d70 con 0x5586f7ebe0e0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f7af4005020 con 0x5586f7ebe0e0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7af4026050 con 0x5586f7ebe0e0 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7b0c175740 10 client.28661 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:25.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5586f7aed0f0 con 0x5586f7ebe0e0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.650+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5586f7e26d70 con 0x5586f7ebe0e0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f7af401b410 con 0x5586f7ebe0e0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 10 client.28661 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 10 client.28661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 20 client.28661 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7ed72b0 0x5586f7ed96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 4 client.28661 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586f7e210c0 con 0x5586f7ed72b0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7ed72b0 0x5586f7ed96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7aea7fc640 10 client.28661 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.651+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7ed72b0 0x5586f7ed96e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95000a15172e0d73 server_cookie=53efea1808254fa2 in_seq=0 out_seq=0 2024-04-02T13:49:25.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.653+0000 7f7aea7fc640 1 -- 172.21.15.122:0/569067424 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586f7e210c0 con 0x5586f7ed72b0 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.653+0000 7f7aea7fc640 10 client.28661 handle_command_reply: tid=0 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.653+0000 7f7b0c175740 1 client.28661 shutdown 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 msgr2=0x5586f7ec6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ec6df0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7af40139a0 tx=0x7f7af4013da0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 shutdown_connections 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7ed72b0 0x5586f7ed96e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ec4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7aea7fc640 0 client.28661 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 --2- 172.21.15.122:0/569067424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ec6df0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 >> 172.21.15.122:0/569067424 conn(0x5586f7eb8810 msgr2=0x5586f7eb9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 shutdown_connections 2024-04-02T13:49:25.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 1 -- 172.21.15.122:0/569067424 wait complete. 2024-04-02T13:49:25.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.654+0000 7f7b0c175740 20 client.28661 trim_cache size 0 max 16384 2024-04-02T13:49:25.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.674+0000 7f7b0c175740 1 Processor -- start 2024-04-02T13:49:25.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.674+0000 7f7b0c175740 1 -- start start 2024-04-02T13:49:25.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebea00 0x5586f7eaf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ec75e0 0x5586f7ec3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad440 con 0x5586f7ebe0e0 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041af100 con 0x5586f7ebea00 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b0c175740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041adb10 con 0x5586f7ec75e0 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52292/0 (socket says 172.21.15.122:52292) 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 -- 172.21.15.122:0/3152843205 learned_addr learned my addr 172.21.15.122:0/3152843205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebea00 0x5586f7eaf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b037fe640 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ec75e0 0x5586f7ec3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 -- 172.21.15.122:0/3152843205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ec75e0 msgr2=0x5586f7ec3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ec75e0 0x5586f7ec3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 -- 172.21.15.122:0/3152843205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebea00 msgr2=0x5586f7eaf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebea00 0x5586f7eaf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 -- 172.21.15.122:0/3152843205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586f7ebe830 con 0x5586f7ebe0e0 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009340 tx=0x7f7af4005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ade2555d153e533 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.675+0000 7f7af8ff9640 1 -- 172.21.15.122:0/3152843205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4030020 con 0x5586f7ebe0e0 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7af8ff9640 1 -- 172.21.15.122:0/3152843205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af4011e90 con 0x5586f7ebe0e0 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7af8ff9640 1 -- 172.21.15.122:0/3152843205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004b00 con 0x5586f7ebe0e0 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 -- 172.21.15.122:0/3152843205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 msgr2=0x5586f7ed2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009340 tx=0x7f7af4005c50 comp rx=0 tx=0).stop 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 -- 172.21.15.122:0/3152843205 shutdown_connections 2024-04-02T13:49:25.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ec75e0 0x5586f7ec3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebea00 0x5586f7eaf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3152843205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ebe0e0 0x5586f7ed2fc0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 -- 172.21.15.122:0/3152843205 >> 172.21.15.122:0/3152843205 conn(0x5586f7eb8810 msgr2=0x5586f7ec2940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.676+0000 7f7b0c175740 1 -- 172.21.15.122:0/3152843205 shutdown_connections 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 -- 172.21.15.122:0/3152843205 wait complete. 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 -- start start 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 Processor -- start 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ecf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ae970 con 0x5586f7ec75e0 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ae210 con 0x5586f7ebe0e0 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ab440 con 0x5586f7ebea00 2024-04-02T13:49:25.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b0c175740 10 client.0 _ll_register_callbacks cb 0x7f7af4013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7afa7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ecf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b02ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b02ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60290/0 (socket says 172.21.15.122:60290) 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.677+0000 7f7b02ffd640 1 -- 172.21.15.122:0/3859680339 learned_addr learned my addr 172.21.15.122:0/3859680339 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7b037fe640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 -- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 msgr2=0x5586f7ecf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 -- 172.21.15.122:0/3859680339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 msgr2=0x5586f7ec0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 -- 172.21.15.122:0/3859680339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586f7ec0d80 con 0x5586f7ebea00 2024-04-02T13:49:25.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7afa7fc640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ecf9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0f5c40 tx=0x7f7aec004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd3bcff24ef34ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7aea7fc640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec04b040 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f7aec0de040 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af4011e90 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f7aec0bc680 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.678+0000 7f7b0c175740 10 client.28417 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5586f7f3d330 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aec0be0c0 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f7aec0ba730 con 0x5586f7ebea00 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 10 client.28417 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 10 client.28417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:25.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 20 client.28417 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7f3d9a0 0x5586f7f3fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 4 client.28417 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7af4011e90 con 0x5586f7f3d9a0 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.679+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7f3d9a0 0x5586f7f3fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.680+0000 7f7b02ffd640 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7f3d9a0 0x5586f7f3fdc0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d8ff73e774a724 server_cookie=43df85bd800ded7f in_seq=0 out_seq=0 2024-04-02T13:49:25.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.680+0000 7f7aea7fc640 10 client.28417 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:25.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7aea7fc640 1 -- 172.21.15.122:0/3859680339 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f7af4011e90 con 0x5586f7f3d9a0 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7aea7fc640 10 client.28417 handle_command_reply: tid=0 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 client.28417 shutdown 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 msgr2=0x5586f7ecf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ecf9b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7aec0f5c40 tx=0x7f7aec004ec0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 shutdown_connections 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5586f7f3d9a0 0x5586f7f3fdc0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5586f7ec75e0 0x5586f7ec0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586f7ebea00 0x5586f7ecf9b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 --2- 172.21.15.122:0/3859680339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5586f7ebe0e0 0x5586f7ecf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 >> 172.21.15.122:0/3859680339 conn(0x5586f7eb8810 msgr2=0x5586f7ec31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7aea7fc640 0 client.28417 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:25.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 shutdown_connections 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 1 -- 172.21.15.122:0/3859680339 wait complete. 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.681+0000 7f7b0c175740 20 client.28417 trim_cache size 0 max 16384 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:49:25.691 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 msgr2=0x7f7af00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 0x7f7af00a6260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7afc005fd0 tx=0x7f7afc009b30 comp rx=0 tx=0).stop 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 msgr2=0x7f7b041a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009900 tx=0x7f7aec02cab0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 shutdown_connections 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7af00a3da0 0x7f7af00a6260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7b0419a5f0 0x7f7b0419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7b04108fe0 0x7f7b041a0980 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 --2- 172.21.15.122:0/2986282295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b04103f10 0x7f7b041a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 >> 172.21.15.122:0/2986282295 conn(0x7f7b040ff7b0 msgr2=0x7f7b041000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 shutdown_connections 2024-04-02T13:49:25.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.682+0000 7f7b0958d640 1 -- 172.21.15.122:0/2986282295 wait complete. 2024-04-02T13:49:25.700 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:49:25.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:49:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:25 smithi122 ceph-mon[32114]: pgmap v446: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:25.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.840+0000 7ff342fdb640 1 -- 172.21.15.122:0/1236857961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 msgr2=0x7ff33c115e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.840+0000 7ff342fdb640 1 --2- 172.21.15.122:0/1236857961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 0x7ff33c115e40 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009860 tx=0x7ff32c02bb70 comp rx=0 tx=0).stop 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.840+0000 7ff342fdb640 1 -- 172.21.15.122:0/1236857961 shutdown_connections 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.841+0000 7ff342fdb640 1 --2- 172.21.15.122:0/1236857961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 0x7ff33c115e40 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.841+0000 7ff342fdb640 1 --2- 172.21.15.122:0/1236857961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10adc0 0x7ff33c113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.841+0000 7ff342fdb640 1 --2- 172.21.15.122:0/1236857961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10a4a0 0x7ff33c10a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.841+0000 7ff342fdb640 1 -- 172.21.15.122:0/1236857961 >> 172.21.15.122:0/1236857961 conn(0x7ff33c06d050 msgr2=0x7ff33c06d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:25.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.842+0000 7ff342fdb640 1 -- 172.21.15.122:0/1236857961 shutdown_connections 2024-04-02T13:49:25.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.842+0000 7ff342fdb640 1 -- 172.21.15.122:0/1236857961 wait complete. 2024-04-02T13:49:25.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.843+0000 7ff342fdb640 1 Processor -- start 2024-04-02T13:49:25.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.843+0000 7ff342fdb640 1 -- start start 2024-04-02T13:49:25.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10a4a0 0x7ff33c19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 0x7ff33c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c119560 con 0x7ff33c113a50 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1193e0 con 0x7ff33c10adc0 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff342fdb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1196e0 con 0x7ff33c10a4a0 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff33bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff33bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60310/0 (socket says 172.21.15.122:60310) 2024-04-02T13:49:25.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.844+0000 7ff33bfff640 1 -- 172.21.15.122:0/805932403 learned_addr learned my addr 172.21.15.122:0/805932403 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff340d77640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10a4a0 0x7ff33c19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 -- 172.21.15.122:0/805932403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10a4a0 msgr2=0x7ff33c19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10a4a0 0x7ff33c19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 -- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 msgr2=0x7ff33c1a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 0x7ff33c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 -- 172.21.15.122:0/805932403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c1a4ba0 con 0x7ff33c10adc0 2024-04-02T13:49:25.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.845+0000 7ff33bfff640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9f0 tx=0x7ff32400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.846+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32400cd80 con 0x7ff33c10adc0 2024-04-02T13:49:25.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.846+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff324005500 con 0x7ff33c10adc0 2024-04-02T13:49:25.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.846+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324010690 con 0x7ff33c10adc0 2024-04-02T13:49:25.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.846+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c1a4e90 con 0x7ff33c10adc0 2024-04-02T13:49:25.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.846+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a5390 con 0x7ff33c10adc0 2024-04-02T13:49:25.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.849+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c0688f0 con 0x7ff33c10adc0 2024-04-02T13:49:25.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.850+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff3240046c0 con 0x7ff33c10adc0 2024-04-02T13:49:25.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.851+0000 7ff339ffb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 0x7ff3280a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:25.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.851+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7ff3240d0e90 con 0x7ff33c10adc0 2024-04-02T13:49:25.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.852+0000 7ff340d77640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 0x7ff3280a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:25.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.853+0000 7ff340d77640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 0x7ff3280a63e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff334005fd0 tx=0x7ff334010040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:25.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:25.861+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff32409d0e0 con 0x7ff33c10adc0 2024-04-02T13:49:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:25 smithi188 ceph-mon[36117]: pgmap v446: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:26.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.183+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff33c10eae0 con 0x7ff33c10adc0 2024-04-02T13:49:26.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.184+0000 7ff339ffb640 1 -- 172.21.15.122:0/805932403 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11961 (secure 0 0 0) 0x7ff32409c9e0 con 0x7ff33c10adc0 2024-04-02T13:49:26.187 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:49:26.187 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:49:12.671718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:49:26.187 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 52 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.187+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 msgr2=0x7ff3280a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.187+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 0x7ff3280a63e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff334005fd0 tx=0x7ff334010040 comp rx=0 tx=0).stop 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.187+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 msgr2=0x7ff33c19fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.187+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff32400e9f0 tx=0x7ff32400edf0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 shutdown_connections 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff3280a3f20 0x7ff3280a63e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff33c113a50 0x7ff33c1a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff33c10adc0 0x7ff33c19fb50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 --2- 172.21.15.122:0/805932403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c10a4a0 0x7ff33c19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 >> 172.21.15.122:0/805932403 conn(0x7ff33c06d050 msgr2=0x7ff33c070730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 shutdown_connections 2024-04-02T13:49:26.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.188+0000 7ff342fdb640 1 -- 172.21.15.122:0/805932403 wait complete. 2024-04-02T13:49:26.206 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:49:26.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:25 smithi187 ceph-mon[36064]: pgmap v446: 129 pgs: 129 active+clean; 905 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 450 op/s 2024-04-02T13:49:26.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 19239 26.63 MB/sec warmup 46 sec latency 58.483 ms 2024-04-02T13:49:26.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2738877513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414101a90 msgr2=0x7fc414101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2738877513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414101a90 0x7fc414101f10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fc404009890 tx=0x7fc40402bb70 comp rx=0 tx=0).stop 2024-04-02T13:49:26.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2738877513 shutdown_connections 2024-04-02T13:49:26.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2738877513 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414102450 0x7fc41410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2738877513 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414101a90 0x7fc414101f10 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2738877513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 0x7fc414107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2738877513 >> 172.21.15.122:0/2738877513 conn(0x7fc414078f00 msgr2=0x7fc414079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2738877513 shutdown_connections 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.363+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2738877513 wait complete. 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 Processor -- start 2024-04-02T13:49:26.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 -- start start 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414101a90 0x7fc414198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 0x7fc41419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc414111eb0 con 0x7fc414102450 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc414111d30 con 0x7fc414101a90 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.364+0000 7fc41c1c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc414112030 con 0x7fc414107aa0 2024-04-02T13:49:26.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52338/0 (socket says 172.21.15.122:52338) 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 -- 172.21.15.122:0/2377356926 learned_addr learned my addr 172.21.15.122:0/2377356926 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc41a765640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 0x7fc41419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419f64640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414101a90 0x7fc414198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 -- 172.21.15.122:0/2377356926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 msgr2=0x7fc41419cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 0x7fc41419cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 -- 172.21.15.122:0/2377356926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414101a90 msgr2=0x7fc414198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414101a90 0x7fc414198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 -- 172.21.15.122:0/2377356926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41419d380 con 0x7fc414102450 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc419763640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc4040042f0 tx=0x7fc404004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc40affd640 1 -- 172.21.15.122:0/2377356926 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404041020 con 0x7fc414102450 2024-04-02T13:49:26.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.365+0000 7fc40affd640 1 -- 172.21.15.122:0/2377356926 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc40402c9f0 con 0x7fc414102450 2024-04-02T13:49:26.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.366+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41419d610 con 0x7fc414102450 2024-04-02T13:49:26.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.366+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141a4f00 con 0x7fc414102450 2024-04-02T13:49:26.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.366+0000 7fc40affd640 1 -- 172.21.15.122:0/2377356926 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404039040 con 0x7fc414102450 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.368+0000 7fc40affd640 1 -- 172.21.15.122:0/2377356926 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc404044020 con 0x7fc414102450 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 Processor -- start 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 -- start start 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc40affd640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 0x7fc4000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b07a0 0x555eaa3b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555eaa3147b0 con 0x555eaa3b9ca0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555eaa2b3570 con 0x555eaa3b07a0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.369+0000 7fc41edad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555eaa310760 con 0x555eaa3b10c0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc41a765640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 0x7fc4000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc40affd640 1 -- 172.21.15.122:0/2377356926 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc4040f23a0 con 0x7fc414102450 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57290/0 (socket says 172.21.15.122:57290) 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 -- 172.21.15.122:0/2731468242 learned_addr learned my addr 172.21.15.122:0/2731468242 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419f64640 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b07a0 0x555eaa3b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc41a765640 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 -- 172.21.15.122:0/2731468242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b07a0 msgr2=0x555eaa3b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b07a0 0x555eaa3b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 -- 172.21.15.122:0/2731468242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 msgr2=0x555eaa3bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3bc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 -- 172.21.15.122:0/2731468242 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eaa0283f0 con 0x555eaa3b10c0 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc419763640 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4040c67a0 tx=0x7fc4040ba3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4f7ce705e5a5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.370+0000 7fc41a765640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 0x7fc4000a6260 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009fd0 tx=0x7fc40c011be0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc3fffff640 1 -- 172.21.15.122:0/2731468242 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040bd9c0 con 0x555eaa3b10c0 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc3fffff640 1 -- 172.21.15.122:0/2731468242 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4040efe50 con 0x555eaa3b10c0 2024-04-02T13:49:26.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc3fffff640 1 -- 172.21.15.122:0/2731468242 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040a2bb0 con 0x555eaa3b10c0 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 -- 172.21.15.122:0/2731468242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 msgr2=0x555eaa3b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4040c67a0 tx=0x7fc4040ba3c0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 -- 172.21.15.122:0/2731468242 shutdown_connections 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b9760 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.371+0000 7fc41edad740 1 --2- 172.21.15.122:0/2731468242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b07a0 0x555eaa3b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- 172.21.15.122:0/2731468242 >> 172.21.15.122:0/2731468242 conn(0x555eaa3aaed0 msgr2=0x555eaa3ab2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- 172.21.15.122:0/2731468242 shutdown_connections 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- 172.21.15.122:0/2731468242 wait complete. 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- start start 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 Processor -- start 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555eaa3bc850 con 0x555eaa3b07a0 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555eaa2cfe80 con 0x555eaa3b9ca0 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555eaa3bc9d0 con 0x555eaa3b10c0 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.372+0000 7fc41edad740 10 client.0 _ll_register_callbacks cb 0x555eaa30baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc41a765640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52344/0 (socket says 172.21.15.122:52344) 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 -- 172.21.15.122:0/1134470638 learned_addr learned my addr 172.21.15.122:0/1134470638 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 -- 172.21.15.122:0/1134470638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 msgr2=0x555eaa3b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 -- 172.21.15.122:0/1134470638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 msgr2=0x555eaa3b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 -- 172.21.15.122:0/1134470638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eaa0f27d0 con 0x555eaa3b07a0 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc41a765640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419763640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc419f64640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc410009fd0 tx=0x7fc410012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8895953606cfb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc3fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41000f920 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc41000fac0 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4040efe50 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc41000fcc0 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc41edad740 10 client.19686 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.373+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555eaa3ccbe0 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4040efe50 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7fc41001c420 con 0x555eaa3b07a0 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 10 client.19686 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 10 client.19686 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:49:26.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 20 client.19686 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa3ce110 0x555eaa3d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 4 client.19686 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eaa321cf0 con 0x555eaa3ce110 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc419763640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa3ce110 0x555eaa3d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc3fd7fa640 10 client.19686 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.374+0000 7fc419763640 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa3ce110 0x555eaa3d05b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=10db8738b5a752c6 server_cookie=a35d52dffa654983 in_seq=0 out_seq=0 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/1134470638 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555eaa321cf0 con 0x555eaa3ce110 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc3fd7fa640 10 client.19686 handle_command_reply: tid=0 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 client.19686 shutdown 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 msgr2=0x555eaa3b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc410009fd0 tx=0x7fc410012bd0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 shutdown_connections 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa3ce110 0x555eaa3d05b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 --2- 172.21.15.122:0/1134470638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3b9640 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc3fd7fa640 0 client.19686 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 >> 172.21.15.122:0/1134470638 conn(0x555eaa3aaed0 msgr2=0x555eaa3acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 shutdown_connections 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.375+0000 7fc41edad740 1 -- 172.21.15.122:0/1134470638 wait complete. 2024-04-02T13:49:26.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.376+0000 7fc41edad740 20 client.19686 trim_cache size 0 max 16384 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.390+0000 7fc41edad740 1 Processor -- start 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 -- start start 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3bbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141aceb0 con 0x555eaa3b07a0 2024-04-02T13:49:26.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141aeb70 con 0x555eaa3b9ca0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41edad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141ad580 con 0x555eaa3b10c0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52348/0 (socket says 172.21.15.122:52348) 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 -- 172.21.15.122:0/3579740588 learned_addr learned my addr 172.21.15.122:0/3579740588 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419763640 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3bbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc41a765640 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 -- 172.21.15.122:0/3579740588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 msgr2=0x555eaa3bbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3bbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 -- 172.21.15.122:0/3579740588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 msgr2=0x555eaa3b9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 -- 172.21.15.122:0/3579740588 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eaa3b4f20 con 0x555eaa3b07a0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.391+0000 7fc419f64640 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc41001c890 tx=0x7fc410009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314c69efda4d253e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc3ff7fe640 1 -- 172.21.15.122:0/3579740588 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100122e0 con 0x555eaa3b07a0 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc3ff7fe640 1 -- 172.21.15.122:0/3579740588 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4100055c0 con 0x555eaa3b07a0 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc3ff7fe640 1 -- 172.21.15.122:0/3579740588 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100035f0 con 0x555eaa3b07a0 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 -- 172.21.15.122:0/3579740588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 msgr2=0x555eaa3c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc41001c890 tx=0x7fc410009e70 comp rx=0 tx=0).stop 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 -- 172.21.15.122:0/3579740588 shutdown_connections 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b9ca0 0x555eaa3b9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b10c0 0x555eaa3bbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 --2- 172.21.15.122:0/3579740588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b07a0 0x555eaa3c5640 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.392+0000 7fc41edad740 1 -- 172.21.15.122:0/3579740588 >> 172.21.15.122:0/3579740588 conn(0x555eaa3aaed0 msgr2=0x555eaa3b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- 172.21.15.122:0/3579740588 shutdown_connections 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- 172.21.15.122:0/3579740588 wait complete. 2024-04-02T13:49:26.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- start start 2024-04-02T13:49:26.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 Processor -- start 2024-04-02T13:49:26.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b10c0 0x555eaa3b3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141ae3e0 con 0x555eaa3b9ca0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141adc80 con 0x555eaa3b10c0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.393+0000 7fc41edad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4141aaeb0 con 0x555eaa3b07a0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57318/0 (socket says 172.21.15.122:57318) 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 -- 172.21.15.122:0/222397132 learned_addr learned my addr 172.21.15.122:0/222397132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc41a765640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419763640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b10c0 0x555eaa3b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc41edad740 10 client.0 _ll_register_callbacks cb 0x7fc410009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 -- 172.21.15.122:0/222397132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b10c0 msgr2=0x555eaa3b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b10c0 0x555eaa3b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 -- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 msgr2=0x555eaa3c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 -- 172.21.15.122:0/222397132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eaa3c1070 con 0x555eaa3b07a0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc41a765640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4100055c0 con 0x555eaa3b07a0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc419f64640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc41000f890 tx=0x7fc410009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785b5949891a2f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc3fd7fa640 10 client.28441 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.394+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/222397132 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100122e0 con 0x555eaa3b07a0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/222397132 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc41001c630 con 0x555eaa3b07a0 2024-04-02T13:49:26.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/222397132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc4100050c0 con 0x555eaa3b07a0 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 10 client.28441 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=52}) -- 0x555eaa3c21c0 con 0x555eaa3b07a0 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/222397132 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7fc410031070 con 0x555eaa3b07a0 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 10 client.28441 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 10 client.28441 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 20 client.28441 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa434850 0x555eaa436c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 4 client.28441 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555eaa31c040 con 0x555eaa434850 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.395+0000 7fc419763640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa434850 0x555eaa436c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc419763640 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa434850 0x555eaa436c80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2f470317811c9f16 server_cookie=6a38dd73cdc3d316 in_seq=0 out_seq=0 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc3fd7fa640 10 client.28441 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc3fd7fa640 1 -- 172.21.15.122:0/222397132 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555eaa31c040 con 0x555eaa434850 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc3fd7fa640 10 client.28441 handle_command_reply: tid=0 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 client.28441 shutdown 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 msgr2=0x555eaa3b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc41000f890 tx=0x7fc410009240 comp rx=0 tx=0).stop 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 shutdown_connections 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x555eaa434850 0x555eaa436c80 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555eaa3b9ca0 0x555eaa3c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555eaa3b10c0 0x555eaa3b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc3fd7fa640 0 client.28441 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:26.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 --2- 172.21.15.122:0/222397132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555eaa3b07a0 0x555eaa3b3430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 >> 172.21.15.122:0/222397132 conn(0x555eaa3aaed0 msgr2=0x555eaa3a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 shutdown_connections 2024-04-02T13:49:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 1 -- 172.21.15.122:0/222397132 wait complete. 2024-04-02T13:49:26.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.396+0000 7fc41edad740 20 client.28441 trim_cache size 0 max 16384 2024-04-02T13:49:26.406 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 msgr2=0x7fc4000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 0x7fc4000a6260 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009fd0 tx=0x7fc40c011be0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 msgr2=0x7fc4141988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fc4040042f0 tx=0x7fc404004320 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 shutdown_connections 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc4000a3da0 0x7fc4000a6260 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc414107aa0 0x7fc41419cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc414102450 0x7fc4141988d0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 --2- 172.21.15.122:0/2377356926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc414101a90 0x7fc414198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 >> 172.21.15.122:0/2377356926 conn(0x7fc414078f00 msgr2=0x7fc4141044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 shutdown_connections 2024-04-02T13:49:26.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.397+0000 7fc41c1c8640 1 -- 172.21.15.122:0/2377356926 wait complete. 2024-04-02T13:49:26.411 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:49:26.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 -- 172.21.15.122:0/43877355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134100970 msgr2=0x7f4134100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 --2- 172.21.15.122:0/43877355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134100970 0x7f4134100df0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4128009930 tx=0x7f412802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:26.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 -- 172.21.15.122:0/43877355 shutdown_connections 2024-04-02T13:49:26.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 --2- 172.21.15.122:0/43877355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134101330 0x7f413410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 --2- 172.21.15.122:0/43877355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134100970 0x7f4134100df0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 --2- 172.21.15.122:0/43877355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134106980 0x7f4134106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 -- 172.21.15.122:0/43877355 >> 172.21.15.122:0/43877355 conn(0x7f4134077b00 msgr2=0x7f4134077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.541+0000 7f4138bc8640 1 -- 172.21.15.122:0/43877355 shutdown_connections 2024-04-02T13:49:26.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.542+0000 7f4138bc8640 1 -- 172.21.15.122:0/43877355 wait complete. 2024-04-02T13:49:26.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.542+0000 7f4138bc8640 1 Processor -- start 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.542+0000 7f4138bc8640 1 -- start start 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134100970 0x7f41341983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 0x7f4134198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4134069b60 con 0x7f4134106980 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41340699e0 con 0x7f4134100970 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4138bc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41341120c0 con 0x7f4134101330 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 0x7f4134198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f413259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134100970 0x7f41341983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52360/0 (socket says 172.21.15.122:52360) 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 -- 172.21.15.122:0/298439380 learned_addr learned my addr 172.21.15.122:0/298439380 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 0x7f4134198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57332/0 (socket says 172.21.15.122:57332) 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 -- 172.21.15.122:0/298439380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 msgr2=0x7f4134198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 0x7f4134198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 -- 172.21.15.122:0/298439380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134100970 msgr2=0x7f41341983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134100970 0x7f41341983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 -- 172.21.15.122:0/298439380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413419d3e0 con 0x7f4134106980 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f4132d9d640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.543+0000 7f41237fe640 1 -- 172.21.15.122:0/298439380 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c017070 con 0x7f4134106980 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.544+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f413419d6d0 con 0x7f4134106980 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.544+0000 7f41237fe640 1 -- 172.21.15.122:0/298439380 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412c003e50 con 0x7f4134106980 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.544+0000 7f41237fe640 1 -- 172.21.15.122:0/298439380 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c0106c0 con 0x7f4134106980 2024-04-02T13:49:26.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.544+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41341a4f60 con 0x7f4134106980 2024-04-02T13:49:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.546+0000 7f41237fe640 1 -- 172.21.15.122:0/298439380 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f412c003970 con 0x7f4134106980 2024-04-02T13:49:26.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f41237fe640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 0x7f411c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f413b7ad740 1 Processor -- start 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f41237fe640 1 -- 172.21.15.122:0/298439380 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f412c0d08c0 con 0x7f4134106980 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f413b7ad740 1 -- start start 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8025760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.547+0000 7f413259c640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 0x7f411c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8028090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413b7ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557f7f807b0 con 0x5557f8025ca0 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413b7ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5557f7f1f570 con 0x5557f801c7a0 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413b7ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5557f7f7c760 con 0x5557f801d0c0 2024-04-02T13:49:26.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8028090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60400/0 (socket says 172.21.15.122:60400) 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 -- 172.21.15.122:0/3635028165 learned_addr learned my addr 172.21.15.122:0/3635028165 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f4131d9b640 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8025760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 0x7f411c0a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4124012440 tx=0x7f4124012840 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 -- 172.21.15.122:0/3635028165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 msgr2=0x5557f8025760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8025760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 -- 172.21.15.122:0/3635028165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 msgr2=0x5557f8028090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8028090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 -- 172.21.15.122:0/3635028165 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f7c943f0 con 0x5557f801c7a0 2024-04-02T13:49:26.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f4132d9d640 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8028090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.548+0000 7f413259c640 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f412400e6b0 tx=0x7f412401e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19cb7e5696a0a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.549+0000 7f4120ff9640 1 -- 172.21.15.122:0/3635028165 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124025070 con 0x5557f801c7a0 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.549+0000 7f4120ff9640 1 -- 172.21.15.122:0/3635028165 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412401fcb0 con 0x5557f801c7a0 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.549+0000 7f4120ff9640 1 -- 172.21.15.122:0/3635028165 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412401ed50 con 0x5557f801c7a0 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.549+0000 7f413b7ad740 1 -- 172.21.15.122:0/3635028165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 msgr2=0x5557f801cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.549+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f412400e6b0 tx=0x7f412401e000 comp rx=0 tx=0).stop 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- 172.21.15.122:0/3635028165 shutdown_connections 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8028090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8025760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3635028165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f801cb80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- 172.21.15.122:0/3635028165 >> 172.21.15.122:0/3635028165 conn(0x5557f8016ed0 msgr2=0x5557f80172b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- 172.21.15.122:0/3635028165 shutdown_connections 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- 172.21.15.122:0/3635028165 wait complete. 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- start start 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 Processor -- start 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801c7a0 0x5557f8025600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801d0c0 0x5557f8022990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557f802cea0 con 0x5557f8025ca0 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5557f7f3be80 con 0x5557f801d0c0 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5557f802d020 con 0x5557f801c7a0 2024-04-02T13:49:26.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413b7ad740 10 client.0 _ll_register_callbacks cb 0x5557f7f77aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52370/0 (socket says 172.21.15.122:52370) 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801d0c0 0x5557f8022990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.550+0000 7f413259c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801c7a0 0x5557f8025600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 -- 172.21.15.122:0/1102965113 learned_addr learned my addr 172.21.15.122:0/1102965113 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 -- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801c7a0 msgr2=0x5557f8025600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801c7a0 0x5557f8025600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 -- 172.21.15.122:0/1102965113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801d0c0 msgr2=0x5557f8022990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801d0c0 0x5557f8022990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 -- 172.21.15.122:0/1102965113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f7d5e7d0 con 0x5557f8025ca0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4132d9d640 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f412c005c40 tx=0x7f412c09c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef31005ab75c3125 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c020070 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f412c098b90 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412401fcb0 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f412c003c60 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f413b7ad740 10 client.19704 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5557f8038e10 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.551+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412401fcb0 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f412c09b3e0 con 0x5557f8025ca0 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 10 client.19704 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 10 client.19704 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 20 client.19704 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f803a270 0x5557f803c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 4 client.19704 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557f7f8dcf0 con 0x5557f803a270 2024-04-02T13:49:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413259c640 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f803a270 0x5557f803c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f413259c640 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f803a270 0x5557f803c690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=25e507d627377007 server_cookie=2b0359d8a5f27180 in_seq=0 out_seq=0 2024-04-02T13:49:26.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.552+0000 7f4119ffb640 10 client.19704 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f4119ffb640 1 -- 172.21.15.122:0/1102965113 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557f7f8dcf0 con 0x5557f803a270 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f4119ffb640 10 client.19704 handle_command_reply: tid=0 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 client.19704 shutdown 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 msgr2=0x5557f8022ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f412c005c40 tx=0x7f412c09c520 comp rx=0 tx=0).stop 2024-04-02T13:49:26.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 shutdown_connections 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f803a270 0x5557f803c690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f8022ef0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801d0c0 0x5557f8022990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f4119ffb640 0 client.19704 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1102965113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801c7a0 0x5557f8025600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 >> 172.21.15.122:0/1102965113 conn(0x5557f8016ed0 msgr2=0x5557f8018c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 shutdown_connections 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 1 -- 172.21.15.122:0/1102965113 wait complete. 2024-04-02T13:49:26.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.557+0000 7f413b7ad740 20 client.19704 trim_cache size 0 max 16384 2024-04-02T13:49:26.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 Processor -- start 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 -- start start 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8015db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f8025ca0 0x5557f8025600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41341acf10 con 0x5557f801d0c0 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41341aebd0 con 0x5557f801c7a0 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413b7ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ad5e0 con 0x5557f8025ca0 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52374/0 (socket says 172.21.15.122:52374) 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f4131d9b640 1 -- 172.21.15.122:0/1622159857 learned_addr learned my addr 172.21.15.122:0/1622159857 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f413259c640 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8015db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.573+0000 7f4132d9d640 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f8025ca0 0x5557f8025600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 -- 172.21.15.122:0/1622159857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f8025ca0 msgr2=0x5557f8025600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f8025ca0 0x5557f8025600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 -- 172.21.15.122:0/1622159857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 msgr2=0x5557f8015db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8015db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 -- 172.21.15.122:0/1622159857 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557f801bb20 con 0x5557f801d0c0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f4131d9b640 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4128000c00 tx=0x7f412802c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515be98a8d14bb96 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f411bfff640 1 -- 172.21.15.122:0/1622159857 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128004040 con 0x5557f801d0c0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f411bfff640 1 -- 172.21.15.122:0/1622159857 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41280041e0 con 0x5557f801d0c0 2024-04-02T13:49:26.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.574+0000 7f411bfff640 1 -- 172.21.15.122:0/1622159857 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128004ee0 con 0x5557f801d0c0 2024-04-02T13:49:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- 172.21.15.122:0/1622159857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 msgr2=0x5557f80250c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4128000c00 tx=0x7f412802c760 comp rx=0 tx=0).stop 2024-04-02T13:49:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- 172.21.15.122:0/1622159857 shutdown_connections 2024-04-02T13:49:26.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f8025ca0 0x5557f8025600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f801d0c0 0x5557f80250c0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 --2- 172.21.15.122:0/1622159857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8015db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- 172.21.15.122:0/1622159857 >> 172.21.15.122:0/1622159857 conn(0x5557f8016ed0 msgr2=0x5557f80156a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- 172.21.15.122:0/1622159857 shutdown_connections 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- 172.21.15.122:0/1622159857 wait complete. 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 -- start start 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.575+0000 7f413b7ad740 1 Processor -- start 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8026b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8027080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f41341ae440 con 0x5557f8025ca0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f41341adce0 con 0x5557f801c7a0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41341aaf10 con 0x5557f801d0c0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413b7ad740 10 client.0 _ll_register_callbacks cb 0x7f412c09be80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4132d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52380/0 (socket says 172.21.15.122:52380) 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8027080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4132d9d640 1 -- 172.21.15.122:0/3960562870 learned_addr learned my addr 172.21.15.122:0/3960562870 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f413259c640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8026b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 -- 172.21.15.122:0/3960562870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 msgr2=0x5557f8026b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8026b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 -- 172.21.15.122:0/3960562870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 msgr2=0x5557f80275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4131d9b640 1 -- 172.21.15.122:0/3960562870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557f80305a0 con 0x5557f801d0c0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.576+0000 7f4132d9d640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4131d9b640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8027080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f412802c240 tx=0x7f41280045c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21d924e1e6a4b3b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4119ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412802cc90 con 0x5557f801d0c0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41280041e0 con 0x5557f801d0c0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f4128004ee0 con 0x5557f801d0c0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f41280050e0 con 0x5557f801d0c0 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f413b7ad740 10 client.28459 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5557f80a0b50 con 0x5557f801d0c0 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.577+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41280041e0 con 0x5557f801d0c0 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f4128047070 con 0x5557f801d0c0 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 10 client.28459 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 10 client.28459 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 20 client.28459 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f80a1330 0x5557f80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 4 client.28459 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557f7f88040 con 0x5557f80a1330 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413259c640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f80a1330 0x5557f80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f4119ffb640 10 client.28459 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.578+0000 7f413259c640 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f80a1330 0x5557f80a3750 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ea6ad2fa6cf0790 server_cookie=4c7a179f15628601 in_seq=0 out_seq=0 2024-04-02T13:49:26.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f4119ffb640 1 -- 172.21.15.122:0/3960562870 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557f7f88040 con 0x5557f80a1330 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f4119ffb640 10 client.28459 handle_command_reply: tid=0 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 client.28459 shutdown 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 msgr2=0x5557f8027080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8027080 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f412802c240 tx=0x7f41280045c0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 shutdown_connections 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5557f80a1330 0x5557f80a3750 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557f8025ca0 0x5557f80275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f4119ffb640 0 client.28459 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557f801d0c0 0x5557f8027080 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 --2- 172.21.15.122:0/3960562870 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557f801c7a0 0x5557f8026b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 >> 172.21.15.122:0/3960562870 conn(0x5557f8016ed0 msgr2=0x5557f8015f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 shutdown_connections 2024-04-02T13:49:26.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 1 -- 172.21.15.122:0/3960562870 wait complete. 2024-04-02T13:49:26.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.579+0000 7f413b7ad740 20 client.28459 trim_cache size 0 max 16384 2024-04-02T13:49:26.589 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 msgr2=0x7f411c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 0x7f411c0a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4124012440 tx=0x7f4124012840 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 msgr2=0x7f413419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f412c009fd0 tx=0x7f412c00b260 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 shutdown_connections 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f411c0a3d20 0x7f411c0a61e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4134106980 0x7f413419ccb0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4134101330 0x7f4134198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 --2- 172.21.15.122:0/298439380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4134100970 0x7f41341983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 >> 172.21.15.122:0/298439380 conn(0x7f4134077b00 msgr2=0x7f4134103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 shutdown_connections 2024-04-02T13:49:26.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.580+0000 7f4138bc8640 1 -- 172.21.15.122:0/298439380 wait complete. 2024-04-02T13:49:26.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:49:26.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.722+0000 7ffaa9560640 1 -- 172.21.15.122:0/4254919603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 msgr2=0x7ffaa410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.722+0000 7ffaa9560640 1 --2- 172.21.15.122:0/4254919603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa410d820 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009ca0 tx=0x7ffa9c02c180 comp rx=0 tx=0).stop 2024-04-02T13:49:26.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 -- 172.21.15.122:0/4254919603 shutdown_connections 2024-04-02T13:49:26.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 --2- 172.21.15.122:0/4254919603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa410d820 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 --2- 172.21.15.122:0/4254919603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 --2- 172.21.15.122:0/4254919603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa4105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 -- 172.21.15.122:0/4254919603 >> 172.21.15.122:0/4254919603 conn(0x7ffaa4077a90 msgr2=0x7ffaa4077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 -- 172.21.15.122:0/4254919603 shutdown_connections 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 -- 172.21.15.122:0/4254919603 wait complete. 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.723+0000 7ffaa9560640 1 Processor -- start 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 -- start start 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4110dd0 con 0x7ffaa4101360 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4110c50 con 0x7ffaa41009c0 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa9560640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4110f50 con 0x7ffaa41058a0 2024-04-02T13:49:26.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60444/0 (socket says 172.21.15.122:60444) 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/2683368039 learned_addr learned my addr 172.21.15.122:0/2683368039 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.724+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 -- 172.21.15.122:0/2683368039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 msgr2=0x7ffaa419bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa419bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 -- 172.21.15.122:0/2683368039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 msgr2=0x7ffaa4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 -- 172.21.15.122:0/2683368039 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa419c240 con 0x7ffaa4101360 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa4197790 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00b3f0 tx=0x7ffa8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa0ff9640 1 -- 172.21.15.122:0/2683368039 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c015020 con 0x7ffaa4101360 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa419c530 con 0x7ffaa4101360 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa0ff9640 1 -- 172.21.15.122:0/2683368039 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c011d10 con 0x7ffaa4101360 2024-04-02T13:49:26.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa0ff9640 1 -- 172.21.15.122:0/2683368039 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00f2e0 con 0x7ffaa4101360 2024-04-02T13:49:26.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.725+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a3dc0 con 0x7ffaa4101360 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.727+0000 7ffaa0ff9640 1 -- 172.21.15.122:0/2683368039 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ffa8c018020 con 0x7ffaa4101360 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 Processor -- start 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 -- start start 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaa0ff9640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 0x7ffa900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaa0ff9640 1 -- 172.21.15.122:0/2683368039 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7ffa8c0d0310 con 0x7ffaa4101360 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 0x7ffa900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268180 0x55fc3d268560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d2762a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d1cc7b0 con 0x55fc3d268b30 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d16b570 con 0x55fc3d273eb0 2024-04-02T13:49:26.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffaac228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d1c8760 con 0x55fc3d268180 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.728+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52398/0 (socket says 172.21.15.122:52398) 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 -- 172.21.15.122:0/1732370655 learned_addr learned my addr 172.21.15.122:0/1732370655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d2762a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268180 0x55fc3d268560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 0x7ffa900a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005e30 tx=0x7ffa9c005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 -- 172.21.15.122:0/1732370655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268180 msgr2=0x55fc3d268560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268180 0x55fc3d268560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 -- 172.21.15.122:0/1732370655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 msgr2=0x55fc3d2762a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d2762a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 -- 172.21.15.122:0/1732370655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3cee03f0 con 0x55fc3d268b30 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c098430 tx=0x7ffa8c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc33b5d41ed7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa997fa640 1 -- 172.21.15.122:0/1732370655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0b5070 con 0x55fc3d268b30 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.729+0000 7ffa997fa640 1 -- 172.21.15.122:0/1732370655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c09bfd0 con 0x55fc3d268b30 2024-04-02T13:49:26.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffa997fa640 1 -- 172.21.15.122:0/1732370655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c098640 con 0x55fc3d268b30 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 -- 172.21.15.122:0/1732370655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 msgr2=0x55fc3d273970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c098430 tx=0x7ffa8c0092c0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 -- 172.21.15.122:0/1732370655 shutdown_connections 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d2762a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d273970 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 --2- 172.21.15.122:0/1732370655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268180 0x55fc3d268560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.730+0000 7ffaac228740 1 -- 172.21.15.122:0/1732370655 >> 172.21.15.122:0/1732370655 conn(0x55fc3d262ed0 msgr2=0x55fc3d2632b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 -- 172.21.15.122:0/1732370655 shutdown_connections 2024-04-02T13:49:26.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 -- 172.21.15.122:0/1732370655 wait complete. 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 -- start start 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 Processor -- start 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268180 0x55fc3d270d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.731+0000 7ffaac228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d271790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d276c70 con 0x55fc3d268b30 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d187e80 con 0x55fc3d268180 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc3d276df0 con 0x55fc3d273eb0 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268180 0x55fc3d270d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52406/0 (socket says 172.21.15.122:52406) 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 10 client.0 _ll_register_callbacks cb 0x55fc3d1c3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 -- 172.21.15.122:0/283570612 learned_addr learned my addr 172.21.15.122:0/283570612 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d271790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 -- 172.21.15.122:0/283570612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 msgr2=0x55fc3d271790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d271790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 -- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268180 msgr2=0x55fc3d270d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268180 0x55fc3d270d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 -- 172.21.15.122:0/283570612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3cfaa7d0 con 0x55fc3d268b30 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0d0630 tx=0x7ffa8c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a51b9e28df9464 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c0b9070 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7ffa8c098640 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.732+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa8c09bfd0 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c0b5040 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 10 client.19722 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fc3d276c70 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c09bfd0 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7ffa8c09b540 con 0x55fc3d268b30 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 10 client.19722 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 10 client.19722 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 20 client.19722 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d283f00 0x55fc3d2862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 4 client.19722 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc3d1d9cf0 con 0x55fc3d283f00 2024-04-02T13:49:26.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.733+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d283f00 0x55fc3d2862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.734+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d283f00 0x55fc3d2862f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de995e69d3a8fd76 server_cookie=165c02ec8c183673 in_seq=0 out_seq=0 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.734+0000 7ffa8a7fc640 10 client.19722 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/283570612 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc3d1d9cf0 con 0x55fc3d283f00 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffa8a7fc640 10 client.19722 handle_command_reply: tid=0 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 client.19722 shutdown 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 msgr2=0x55fc3d271250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0d0630 tx=0x7ffa8c0047d0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 shutdown_connections 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d271790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268b30 0x55fc3d271250 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d283f00 0x55fc3d2862f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 --2- 172.21.15.122:0/283570612 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268180 0x55fc3d270d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 >> 172.21.15.122:0/283570612 conn(0x55fc3d262ed0 msgr2=0x55fc3d264c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffa8a7fc640 0 client.19722 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:49:26.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 shutdown_connections 2024-04-02T13:49:26.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 1 -- 172.21.15.122:0/283570612 wait complete. 2024-04-02T13:49:26.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.735+0000 7ffaac228740 20 client.19722 trim_cache size 0 max 16384 2024-04-02T13:49:26.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 Processor -- start 2024-04-02T13:49:26.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 -- start start 2024-04-02T13:49:26.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268b30 0x55fc3d27fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d264bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41acf50 con 0x55fc3d268180 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41aec10 con 0x55fc3d273eb0 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaac228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41ad620 con 0x55fc3d268b30 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52414/0 (socket says 172.21.15.122:52414) 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/231448921 learned_addr learned my addr 172.21.15.122:0/231448921 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d264bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.751+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268b30 0x55fc3d27fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/231448921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268b30 msgr2=0x55fc3d27fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268b30 0x55fc3d27fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/231448921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 msgr2=0x55fc3d264bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d264bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/231448921 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3d2688d0 con 0x55fc3d268180 2024-04-02T13:49:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d264bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.752+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e00 tx=0x7ffa94005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd93c5038a8b20df server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.754+0000 7ffa98ff9640 1 -- 172.21.15.122:0/231448921 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94022050 con 0x55fc3d268180 2024-04-02T13:49:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.754+0000 7ffa98ff9640 1 -- 172.21.15.122:0/231448921 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa940226d0 con 0x55fc3d268180 2024-04-02T13:49:26.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.754+0000 7ffa98ff9640 1 -- 172.21.15.122:0/231448921 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94010c90 con 0x55fc3d268180 2024-04-02T13:49:26.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.754+0000 7ffaac228740 1 -- 172.21.15.122:0/231448921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 msgr2=0x55fc3d273440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.754+0000 7ffaac228740 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009e00 tx=0x7ffa94005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:26.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 -- 172.21.15.122:0/231448921 shutdown_connections 2024-04-02T13:49:26.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d273eb0 0x55fc3d264bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d268b30 0x55fc3d27fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 --2- 172.21.15.122:0/231448921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d273440 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 -- 172.21.15.122:0/231448921 >> 172.21.15.122:0/231448921 conn(0x55fc3d262ed0 msgr2=0x55fc3d26f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 -- 172.21.15.122:0/231448921 shutdown_connections 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 -- 172.21.15.122:0/231448921 wait complete. 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 -- start start 2024-04-02T13:49:26.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 Processor -- start 2024-04-02T13:49:26.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.755+0000 7ffaac228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268b30 0x55fc3d277880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d277e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41ae480 con 0x55fc3d268180 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41add20 con 0x55fc3d268b30 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41aaf50 con 0x55fc3d273eb0 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaac228740 10 client.0 _ll_register_callbacks cb 0x7ffa8c0b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52422/0 (socket says 172.21.15.122:52422) 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/3701080472 learned_addr learned my addr 172.21.15.122:0/3701080472 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268b30 0x55fc3d277880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d277e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/3701080472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 msgr2=0x55fc3d277e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d277e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268b30 msgr2=0x55fc3d277880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268b30 0x55fc3d277880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 -- 172.21.15.122:0/3701080472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3d278690 con 0x55fc3d268180 2024-04-02T13:49:26.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa37fe640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d277e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffaa2ffd640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ffa94012fa0 tx=0x7ffa94022900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea718104d9d89d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.756+0000 7ffa8a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.761+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94022050 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.761+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7ffa940221f0 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa940226d0 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffa9403e050 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 10 client.19728 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55fc3d269700 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa940226d0 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7ffa940052f0 con 0x55fc3d268180 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 10 client.19728 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 10 client.19728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 20 client.19728 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d269e50 0x55fc3d26a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 4 client.19728 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:26.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.762+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc3d1d4040 con 0x55fc3d269e50 2024-04-02T13:49:26.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.763+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d269e50 0x55fc3d26a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.763+0000 7ffa8a7fc640 10 client.19728 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:49:26.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.763+0000 7ffa9bfff640 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d269e50 0x55fc3d26a260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e7268fe18cd586c server_cookie=ce45cecdd904959f in_seq=0 out_seq=0 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffa8a7fc640 1 -- 172.21.15.122:0/3701080472 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc3d1d4040 con 0x55fc3d269e50 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffa8a7fc640 10 client.19728 handle_command_reply: tid=0 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 client.19728 shutdown 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 msgr2=0x55fc3d27c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ffa94012fa0 tx=0x7ffa94022900 comp rx=0 tx=0).stop 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 shutdown_connections 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc3d273eb0 0x55fc3d277e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55fc3d269e50 0x55fc3d26a260 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc3d268b30 0x55fc3d277880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffa8a7fc640 0 client.19728 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 --2- 172.21.15.122:0/3701080472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc3d268180 0x55fc3d27c510 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 >> 172.21.15.122:0/3701080472 conn(0x55fc3d262ed0 msgr2=0x55fc3d26fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 shutdown_connections 2024-04-02T13:49:26.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 1 -- 172.21.15.122:0/3701080472 wait complete. 2024-04-02T13:49:26.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.766+0000 7ffaac228740 20 client.19728 trim_cache size 0 max 16384 2024-04-02T13:49:26.770 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 msgr2=0x7ffa900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 0x7ffa900a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005e30 tx=0x7ffa9c005d80 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 msgr2=0x7ffaa4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa4197790 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00b3f0 tx=0x7ffa8c00b8b0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 shutdown_connections 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ffa900a3d20 0x7ffa900a61e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa41058a0 0x7ffaa419bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaa4101360 0x7ffaa4197790 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 --2- 172.21.15.122:0/2683368039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaa41009c0 0x7ffaa4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 >> 172.21.15.122:0/2683368039 conn(0x7ffaa4077a90 msgr2=0x7ffaa40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 shutdown_connections 2024-04-02T13:49:26.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.767+0000 7ffaa9560640 1 -- 172.21.15.122:0/2683368039 wait complete. 2024-04-02T13:49:26.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:49:26.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- 172.21.15.122:0/2899378572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 msgr2=0x7fc634107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 --2- 172.21.15.122:0/2899378572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc634107f60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc624009900 tx=0x7fc62402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- 172.21.15.122:0/2899378572 shutdown_connections 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 --2- 172.21.15.122:0/2899378572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 0x7fc63410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 --2- 172.21.15.122:0/2899378572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc63406a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 --2- 172.21.15.122:0/2899378572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc634107f60 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- 172.21.15.122:0/2899378572 >> 172.21.15.122:0/2899378572 conn(0x7fc634077a90 msgr2=0x7fc634077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- 172.21.15.122:0/2899378572 shutdown_connections 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- 172.21.15.122:0/2899378572 wait complete. 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 Processor -- start 2024-04-02T13:49:26.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.910+0000 7fc63941b640 1 -- start start 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc634198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 0x7fc6341986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111eb0 con 0x7fc634107b80 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc634111d30 con 0x7fc63406a860 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc63941b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc634112030 con 0x7fc634069ec0 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52452/0 (socket says 172.21.15.122:52452) 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc6337fe640 1 -- 172.21.15.122:0/2086817654 learned_addr learned my addr 172.21.15.122:0/2086817654 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc634198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc62bfff640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 0x7fc6341986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 -- 172.21.15.122:0/2086817654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 msgr2=0x7fc6341986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 0x7fc6341986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 -- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 msgr2=0x7fc63419ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc632ffd640 1 -- 172.21.15.122:0/2086817654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419d160 con 0x7fc634069ec0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.911+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc632ffd640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc634198170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc624036040 tx=0x7fc62402ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc630ff9640 1 -- 172.21.15.122:0/2086817654 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624041070 con 0x7fc634069ec0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc630ff9640 1 -- 172.21.15.122:0/2086817654 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624004450 con 0x7fc634069ec0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc630ff9640 1 -- 172.21.15.122:0/2086817654 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624034440 con 0x7fc634069ec0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419d3f0 con 0x7fc634069ec0 2024-04-02T13:49:26.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.912+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63419d860 con 0x7fc634069ec0 2024-04-02T13:49:26.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.915+0000 7fc63c000740 1 Processor -- start 2024-04-02T13:49:26.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.915+0000 7fc630ff9640 1 -- 172.21.15.122:0/2086817654 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc624044070 con 0x7fc634069ec0 2024-04-02T13:49:26.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.915+0000 7fc63c000740 1 -- start start 2024-04-02T13:49:26.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f912760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f915090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a15f86d7b0 con 0x55a15f9097a0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a15f80c570 con 0x55a15f912ca0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc63c000740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a15f869760 con 0x55a15f90a0c0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52458/0 (socket says 172.21.15.122:52458) 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 -- 172.21.15.122:0/1294125655 learned_addr learned my addr 172.21.15.122:0/1294125655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc632ffd640 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f912760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc62bfff640 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f915090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc630ff9640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 -- 172.21.15.122:0/1294125655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 msgr2=0x55a15f912760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f912760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 -- 172.21.15.122:0/1294125655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 msgr2=0x55a15f915090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f915090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc6337fe640 1 -- 172.21.15.122:0/1294125655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a15f5813f0 con 0x55a15f9097a0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.916+0000 7fc630ff9640 1 -- 172.21.15.122:0/2086817654 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc6240f2770 con 0x7fc634069ec0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.917+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc62c011850 tx=0x7fc62c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de06a1cd92cd9dd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.917+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.917+0000 7fc6297fa640 1 -- 172.21.15.122:0/1294125655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c00fb00 con 0x55a15f9097a0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.917+0000 7fc6297fa640 1 -- 172.21.15.122:0/1294125655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c010e30 con 0x55a15f9097a0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.917+0000 7fc6297fa640 1 -- 172.21.15.122:0/1294125655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c01cc30 con 0x55a15f9097a0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 -- 172.21.15.122:0/1294125655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 msgr2=0x55a15f909b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 0x7fc6200a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0114d0 tx=0x7fc62c021000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc62c011850 tx=0x7fc62c011d10 comp rx=0 tx=0).stop 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 -- 172.21.15.122:0/1294125655 shutdown_connections 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f915090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f912760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 --2- 172.21.15.122:0/1294125655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f909b80 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 -- 172.21.15.122:0/1294125655 >> 172.21.15.122:0/1294125655 conn(0x55a15f903ed0 msgr2=0x55a15f9042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 -- 172.21.15.122:0/1294125655 shutdown_connections 2024-04-02T13:49:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.918+0000 7fc63c000740 1 -- 172.21.15.122:0/1294125655 wait complete. 2024-04-02T13:49:26.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 -- start start 2024-04-02T13:49:26.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 Processor -- start 2024-04-02T13:49:26.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f90f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f90ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a15f915a90 con 0x55a15f9097a0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a15f828e80 con 0x55a15f912ca0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a15f915c10 con 0x55a15f90a0c0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc63c000740 10 client.0 _ll_register_callbacks cb 0x55a15f864aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52466/0 (socket says 172.21.15.122:52466) 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc632ffd640 1 -- 172.21.15.122:0/4141763469 learned_addr learned my addr 172.21.15.122:0/4141763469 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc62bfff640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f90f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.919+0000 7fc6337fe640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f90ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 -- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 msgr2=0x55a15f90f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f90f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 -- 172.21.15.122:0/4141763469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 msgr2=0x55a15f90ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f90ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 -- 172.21.15.122:0/4141763469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a15f64b7d0 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc632ffd640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc6240be180 tx=0x7fc624005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaacdc4fcab968a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc61a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62404b070 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc62c010e30 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.920+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc6240c6bb0 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.921+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc6240bdde0 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.921+0000 7fc63c000740 10 client.19740 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.921+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a15f921810 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c010e30 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7fc6240f22c0 con 0x55a15f9097a0 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 10 client.19740 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 10 client.19740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 20 client.19740 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f922cb0 0x55a15f9250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 4 client.19740 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.922+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a15f87acf0 con 0x55a15f922cb0 2024-04-02T13:49:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.923+0000 7fc62bfff640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f922cb0 0x55a15f9250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.923+0000 7fc62bfff640 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f922cb0 0x55a15f9250f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f79073017a9ed302 server_cookie=f6e717609fc6f2ca in_seq=0 out_seq=0 2024-04-02T13:49:26.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.923+0000 7fc61a7fc640 10 client.19740 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:49:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.927+0000 7fc61a7fc640 1 -- 172.21.15.122:0/4141763469 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a15f87acf0 con 0x55a15f922cb0 2024-04-02T13:49:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.927+0000 7fc61a7fc640 10 client.19740 handle_command_reply: tid=0 2024-04-02T13:49:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.927+0000 7fc63c000740 1 client.19740 shutdown 2024-04-02T13:49:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.927+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 msgr2=0x55a15f912620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc6240be180 tx=0x7fc624005fd0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 shutdown_connections 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f922cb0 0x55a15f9250f0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f912ca0 0x55a15f90ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f90f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 --2- 172.21.15.122:0/4141763469 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f9097a0 0x55a15f912620 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 >> 172.21.15.122:0/4141763469 conn(0x55a15f903ed0 msgr2=0x55a15f905c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc61a7fc640 0 client.19740 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:49:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 shutdown_connections 2024-04-02T13:49:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.928+0000 7fc63c000740 1 -- 172.21.15.122:0/4141763469 wait complete. 2024-04-02T13:49:26.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.929+0000 7fc63c000740 20 client.19740 trim_cache size 0 max 16384 2024-04-02T13:49:26.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 Processor -- start 2024-04-02T13:49:26.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 -- start start 2024-04-02T13:49:26.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 0x55a15f91e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 0x55a15f9120e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341acc60 con 0x55a15f912ca0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ae920 con 0x55a15f90a0c0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.950+0000 7fc63c000740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ad330 con 0x55a15f9097a0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 0x55a15f91e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 0x55a15f9120e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 0x55a15f91e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57488/0 (socket says 172.21.15.122:57488) 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc632ffd640 1 -- 172.21.15.122:0/2919887054 learned_addr learned my addr 172.21.15.122:0/2919887054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52470/0 (socket says 172.21.15.122:52470) 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 -- 172.21.15.122:0/2919887054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 msgr2=0x55a15f91e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 0x55a15f91e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 -- 172.21.15.122:0/2919887054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 msgr2=0x55a15f912620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 -- 172.21.15.122:0/2919887054 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a15f909ef0 con 0x55a15f90a0c0 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc6337fe640 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc62bfff640 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 0x55a15f9120e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a15f925970 tx=0x7fc61c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=972cae87903c3047 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc628ff9640 1 -- 172.21.15.122:0/2919887054 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c01f630 con 0x55a15f90a0c0 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc628ff9640 1 -- 172.21.15.122:0/2919887054 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c005500 con 0x55a15f90a0c0 2024-04-02T13:49:26.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.951+0000 7fc628ff9640 1 -- 172.21.15.122:0/2919887054 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004d00 con 0x55a15f90a0c0 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 -- 172.21.15.122:0/2919887054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 msgr2=0x55a15f9120e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 0x55a15f9120e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55a15f925970 tx=0x7fc61c005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 -- 172.21.15.122:0/2919887054 shutdown_connections 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f912620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f90a0c0 0x55a15f9120e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 --2- 172.21.15.122:0/2919887054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f9097a0 0x55a15f91e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.952+0000 7fc63c000740 1 -- 172.21.15.122:0/2919887054 >> 172.21.15.122:0/2919887054 conn(0x55a15f903ed0 msgr2=0x55a15f90e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.953+0000 7fc63c000740 1 -- 172.21.15.122:0/2919887054 shutdown_connections 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.953+0000 7fc63c000740 1 -- 172.21.15.122:0/2919887054 wait complete. 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.953+0000 7fc63c000740 1 -- start start 2024-04-02T13:49:26.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.953+0000 7fc63c000740 1 Processor -- start 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f9097a0 0x55a15f91b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f913760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ae190 con 0x55a15f912ca0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341ada30 con 0x55a15f9097a0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6341aac60 con 0x55a15f90a0c0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc63c000740 10 client.0 _ll_register_callbacks cb 0x7fc624003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc632ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f9097a0 0x55a15f91b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc62bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f913760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52474/0 (socket says 172.21.15.122:52474) 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc62bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f913760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57494/0 (socket says 172.21.15.122:57494) 2024-04-02T13:49:26.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 -- 172.21.15.122:0/1839096509 learned_addr learned my addr 172.21.15.122:0/1839096509 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 -- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 msgr2=0x55a15f913760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f913760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 -- 172.21.15.122:0/1839096509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f9097a0 msgr2=0x55a15f91b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f9097a0 0x55a15f91b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 -- 172.21.15.122:0/1839096509 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a15f9144d0 con 0x55a15f912ca0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc6337fe640 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc62c011850 tx=0x7fc62c010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c9126be48f5084 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc61a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.954+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c037020 con 0x55a15f912ca0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c005500 con 0x55a15f912ca0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fc62c022040 con 0x55a15f912ca0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc62c042050 con 0x55a15f912ca0 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 10 client.19746 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:26.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55a15f989320 con 0x55a15f912ca0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c005500 con 0x55a15f912ca0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7fc62c028480 con 0x55a15f912ca0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 10 client.19746 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 10 client.19746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 20 client.19746 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f989960 0x55a15f98bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 4 client.19746 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.955+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a15f875040 con 0x55a15f989960 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc632ffd640 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f989960 0x55a15f98bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc632ffd640 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f989960 0x55a15f98bd80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1932a3a332605343 server_cookie=8214facc566933f1 in_seq=0 out_seq=0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc61a7fc640 10 client.19746 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc61a7fc640 1 -- 172.21.15.122:0/1839096509 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a15f875040 con 0x55a15f989960 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc61a7fc640 10 client.19746 handle_command_reply: tid=0 2024-04-02T13:49:26.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.956+0000 7fc63c000740 1 client.19746 shutdown 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 msgr2=0x55a15f913ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fc62c011850 tx=0x7fc62c010b00 comp rx=0 tx=0).stop 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 shutdown_connections 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a15f989960 0x55a15f98bd80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a15f912ca0 0x55a15f913ca0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a15f90a0c0 0x55a15f913760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc61a7fc640 0 client.19746 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 --2- 172.21.15.122:0/1839096509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a15f9097a0 0x55a15f91b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 >> 172.21.15.122:0/1839096509 conn(0x55a15f903ed0 msgr2=0x55a15f902f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 shutdown_connections 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 1 -- 172.21.15.122:0/1839096509 wait complete. 2024-04-02T13:49:26.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.957+0000 7fc63c000740 20 client.19746 trim_cache size 0 max 16384 2024-04-02T13:49:26.966 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:26.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 msgr2=0x7fc6200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 0x7fc6200a61e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0114d0 tx=0x7fc62c021000 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 msgr2=0x7fc634198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc634198170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc624036040 tx=0x7fc62402ce90 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 shutdown_connections 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc6200a3d20 0x7fc6200a61e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc634107b80 0x7fc63419ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc63406a860 0x7fc6341986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 --2- 172.21.15.122:0/2086817654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc634069ec0 0x7fc634198170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 >> 172.21.15.122:0/2086817654 conn(0x7fc634077a90 msgr2=0x7fc6340ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 shutdown_connections 2024-04-02T13:49:26.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:26.958+0000 7fc63941b640 1 -- 172.21.15.122:0/2086817654 wait complete. 2024-04-02T13:49:26.972 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:26 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/805932403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:26.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:49:27.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:26 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/805932403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/882038029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0068d00 msgr2=0x7f0fc0069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/882038029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0068d00 0x7f0fc0069180 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009930 tx=0x7f0fa802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/882038029 shutdown_connections 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/882038029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc00696c0 0x7f0fc010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/882038029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0068d00 0x7f0fc0069180 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/882038029 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc0107b50 0x7f0fc0107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/882038029 >> 172.21.15.122:0/882038029 conn(0x7f0fc0077b00 msgr2=0x7f0fc0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/882038029 shutdown_connections 2024-04-02T13:49:27.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.102+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/882038029 wait complete. 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.103+0000 7f0fc4dfa640 1 Processor -- start 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.103+0000 7f0fc4dfa640 1 -- start start 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc0068d00 0x7f0fc0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0107b50 0x7f0fc019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0111e70 con 0x7f0fc0107b50 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0111cf0 con 0x7f0fc00696c0 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fc4dfa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0111ff0 con 0x7f0fc0068d00 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60570/0 (socket says 172.21.15.122:60570) 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 -- 172.21.15.122:0/671712378 learned_addr learned my addr 172.21.15.122:0/671712378 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:27.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0107b50 0x7f0fc019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc0068d00 0x7f0fc0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 -- 172.21.15.122:0/671712378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc0068d00 msgr2=0x7f0fc0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc0068d00 0x7f0fc0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 -- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0107b50 msgr2=0x7f0fc019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0107b50 0x7f0fc019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.104+0000 7f0fb7fff640 1 -- 172.21.15.122:0/671712378 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc019d340 con 0x7f0fc00696c0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.105+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009230 tx=0x7f0fa8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.106+0000 7f0fb77fe640 1 -- 172.21.15.122:0/671712378 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8041070 con 0x7f0fc00696c0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.106+0000 7f0fb77fe640 1 -- 172.21.15.122:0/671712378 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa80054e0 con 0x7f0fc00696c0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.106+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc019d5d0 con 0x7f0fc00696c0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.106+0000 7f0fb77fe640 1 -- 172.21.15.122:0/671712378 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa8003c90 con 0x7f0fc00696c0 2024-04-02T13:49:27.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.106+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc01a4ec0 con 0x7f0fc00696c0 2024-04-02T13:49:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.108+0000 7f0fb77fe640 1 -- 172.21.15.122:0/671712378 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0fa8044070 con 0x7f0fc00696c0 2024-04-02T13:49:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 Processor -- start 2024-04-02T13:49:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 -- start start 2024-04-02T13:49:27.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c45bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c467090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56440c3bf7b0 con 0x56440c464ca0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56440c35e570 con 0x56440c45c0c0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56440c3bb760 con 0x56440c45b7a0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fb77fe640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 0x7f0fac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fbe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fbe59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60578/0 (socket says 172.21.15.122:60578) 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fbe59c640 1 -- 172.21.15.122:0/242234493 learned_addr learned my addr 172.21.15.122:0/242234493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c45bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c467090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.109+0000 7f0fb77fe640 1 -- 172.21.15.122:0/671712378 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0fa80f25f0 con 0x7f0fc00696c0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 0x7f0fac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 -- 172.21.15.122:0/242234493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 msgr2=0x56440c45bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c45bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 -- 172.21.15.122:0/242234493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 msgr2=0x56440c467090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c467090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 -- 172.21.15.122:0/242234493 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440c0d33f0 con 0x56440c45c0c0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c467090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009cf0 tx=0x7f0fb0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ccae2ccc4ccbf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 0x7f0fac0a6260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8005c40 tx=0x7f0fb8004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fb4ff9640 1 -- 172.21.15.122:0/242234493 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0004230 con 0x56440c45c0c0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fb4ff9640 1 -- 172.21.15.122:0/242234493 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb00043d0 con 0x56440c45c0c0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.110+0000 7f0fb4ff9640 1 -- 172.21.15.122:0/242234493 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00198c0 con 0x56440c45c0c0 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 -- 172.21.15.122:0/242234493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 msgr2=0x56440c464760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0009cf0 tx=0x7f0fb0011810 comp rx=0 tx=0).stop 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 -- 172.21.15.122:0/242234493 shutdown_connections 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c467090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c464760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/242234493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c45bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 -- 172.21.15.122:0/242234493 >> 172.21.15.122:0/242234493 conn(0x56440c455ed0 msgr2=0x56440c4562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.111+0000 7f0fc79dd740 1 -- 172.21.15.122:0/242234493 shutdown_connections 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 -- 172.21.15.122:0/242234493 wait complete. 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 -- start start 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 Processor -- start 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 0x56440c464600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c461990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 0x56440c461ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fbe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 0x56440c464600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.112+0000 7f0fbe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 0x56440c464600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52514/0 (socket says 172.21.15.122:52514) 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fbe59c640 1 -- 172.21.15.122:0/955510018 learned_addr learned my addr 172.21.15.122:0/955510018 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56440c467a90 con 0x56440c45b7a0 2024-04-02T13:49:27.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56440c37ae80 con 0x56440c45c0c0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56440c467c10 con 0x56440c464ca0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fc79dd740 10 client.0 _ll_register_callbacks cb 0x56440c3b6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c461990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 0x56440c461ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 -- 172.21.15.122:0/955510018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 msgr2=0x56440c461ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 0x56440c461ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 -- 172.21.15.122:0/955510018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 msgr2=0x56440c464600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 0x56440c464600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440c19d7d0 con 0x56440c45c0c0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 0x56440c461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c461990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80efa60 tx=0x7f0fa804d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b860d32fd8116631 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.113+0000 7f0fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa804b040 con 0x56440c45c0c0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0fa8041780 con 0x56440c45c0c0 2024-04-02T13:49:27.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb00043d0 con 0x56440c45c0c0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0fa80bdcc0 con 0x56440c45c0c0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fc79dd740 10 client.28754 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56440c473970 con 0x56440c45c0c0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb00043d0 con 0x56440c45c0c0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.114+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f0fa80dd0d0 con 0x56440c45c0c0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 10 client.28754 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 10 client.28754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 20 client.28754 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c477200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 4 client.28754 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56440c3cccf0 con 0x56440c474dd0 2024-04-02T13:49:27.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c477200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.115+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c477200 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d651c8894d45c093 server_cookie=558292d988e51dae in_seq=0 out_seq=0 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.116+0000 7f0fa5ffb640 10 client.28754 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/955510018 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56440c3cccf0 con 0x56440c474dd0 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fa5ffb640 10 client.28754 handle_command_reply: tid=0 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 client.28754 shutdown 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 msgr2=0x56440c461990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c461990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80efa60 tx=0x7f0fa804d000 comp rx=0 tx=0).stop 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 shutdown_connections 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c477200 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fa5ffb640 0 client.28754 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c464ca0 0x56440c461ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c461990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/955510018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45b7a0 0x56440c464600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 >> 172.21.15.122:0/955510018 conn(0x56440c455ed0 msgr2=0x56440c457c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 shutdown_connections 2024-04-02T13:49:27.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.117+0000 7f0fc79dd740 1 -- 172.21.15.122:0/955510018 wait complete. 2024-04-02T13:49:27.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.118+0000 7f0fc79dd740 20 client.28754 trim_cache size 0 max 16384 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 Processor -- start 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 -- start start 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c461950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 0x56440c4562b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01acd90 con 0x56440c45c0c0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01aea50 con 0x56440c464ca0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ad460 con 0x56440c45b7a0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52528/0 (socket says 172.21.15.122:52528) 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 -- 172.21.15.122:0/929855026 learned_addr learned my addr 172.21.15.122:0/929855026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c461950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 -- 172.21.15.122:0/929855026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 msgr2=0x56440c461950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 0x56440c4562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c461950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 -- 172.21.15.122:0/929855026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 msgr2=0x56440c4562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 0x56440c4562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fb7fff640 1 -- 172.21.15.122:0/929855026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56440c462260 con 0x56440c45c0c0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.137+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 0x56440c4562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.138+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80eff50 tx=0x7f0fa8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2d5b1706bcec67 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.138+0000 7f0fa7fff640 1 -- 172.21.15.122:0/929855026 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80a6020 con 0x56440c45c0c0 2024-04-02T13:49:27.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.138+0000 7f0fa7fff640 1 -- 172.21.15.122:0/929855026 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa809e020 con 0x56440c45c0c0 2024-04-02T13:49:27.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.138+0000 7f0fa7fff640 1 -- 172.21.15.122:0/929855026 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa804b070 con 0x56440c45c0c0 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- 172.21.15.122:0/929855026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 msgr2=0x56440c470950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f0fa80eff50 tx=0x7f0fa8009260 comp rx=0 tx=0).stop 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- 172.21.15.122:0/929855026 shutdown_connections 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c464ca0 0x56440c4562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c45c0c0 0x56440c470950 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/929855026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c461950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- 172.21.15.122:0/929855026 >> 172.21.15.122:0/929855026 conn(0x56440c455ed0 msgr2=0x56440c461220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- 172.21.15.122:0/929855026 shutdown_connections 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- 172.21.15.122:0/929855026 wait complete. 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 -- start start 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.139+0000 7f0fc79dd740 1 Processor -- start 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c46d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c465ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ae2c0 con 0x56440c464ca0 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01adb60 con 0x56440c45c0c0 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01aad90 con 0x56440c45b7a0 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fc79dd740 10 client.0 _ll_register_callbacks cb 0x7f0fa80057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52534/0 (socket says 172.21.15.122:52534) 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbe59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c46d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 -- 172.21.15.122:0/2156482374 learned_addr learned my addr 172.21.15.122:0/2156482374 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 -- 172.21.15.122:0/2156482374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 msgr2=0x56440c46d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c46d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 -- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 msgr2=0x56440c465ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c465ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c465ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 -- 172.21.15.122:0/2156482374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56440c4667d0 con 0x56440c464ca0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fb7fff640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c465ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fbed9d640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80048c0 tx=0x7f0fb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc217f7a0604eec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.140+0000 7f0fa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb8019070 con 0x56440c464ca0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fa809e020 con 0x56440c464ca0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0fb8004b20 con 0x56440c464ca0 2024-04-02T13:49:27.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f0fb8004d20 con 0x56440c464ca0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 10 client.19764 fetch_fsmap learned FSMap version 52 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56440c4db380 con 0x56440c464ca0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa809e020 con 0x56440c464ca0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 52) ==== 8410+0+0 (secure 0 0 0) 0x7f0fb8005490 con 0x56440c464ca0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 10 client.19764 fetch_fsmap finished waiting for FSMap version 52 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 10 client.19764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 20 client.19764 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c4ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 4 client.19764 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.141+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56440c3c7040 con 0x56440c474dd0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.142+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c4ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.142+0000 7f0fbe59c640 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c4ddc50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83afc37a81d88388 server_cookie=81df1856b3f39d43 in_seq=0 out_seq=0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.142+0000 7f0fa5ffb640 10 client.19764 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fa5ffb640 1 -- 172.21.15.122:0/2156482374 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56440c3c7040 con 0x56440c474dd0 2024-04-02T13:49:27.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fa5ffb640 10 client.19764 handle_command_reply: tid=0 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 client.19764 shutdown 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 msgr2=0x56440c465ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f0fb80048c0 tx=0x7f0fb8004a40 comp rx=0 tx=0).stop 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 shutdown_connections 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56440c474dd0 0x56440c4ddc50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fa5ffb640 0 client.19764 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56440c464ca0 0x56440c465ff0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56440c45c0c0 0x56440c465ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 --2- 172.21.15.122:0/2156482374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56440c45b7a0 0x56440c46d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.143+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 >> 172.21.15.122:0/2156482374 conn(0x56440c455ed0 msgr2=0x56440c461100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 shutdown_connections 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc79dd740 1 -- 172.21.15.122:0/2156482374 wait complete. 2024-04-02T13:49:27.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc79dd740 20 client.19764 trim_cache size 0 max 16384 2024-04-02T13:49:27.151 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 msgr2=0x7f0fac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 0x7f0fac0a6260 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8005c40 tx=0x7f0fb8004ab0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 msgr2=0x7f0fc0198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.144+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8009230 tx=0x7f0fa8004270 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 shutdown_connections 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0fac0a3da0 0x7f0fac0a6260 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fc0107b50 0x7f0fc019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0fc00696c0 0x7f0fc0198890 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 --2- 172.21.15.122:0/671712378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0fc0068d00 0x7f0fc0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 >> 172.21.15.122:0/671712378 conn(0x7f0fc0077b00 msgr2=0x7f0fc00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 shutdown_connections 2024-04-02T13:49:27.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:27.145+0000 7f0fc4dfa640 1 -- 172.21.15.122:0/671712378 wait complete. 2024-04-02T13:49:27.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:26 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/805932403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:49:27.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 19779 26.33 MB/sec warmup 47 sec latency 88.493 ms 2024-04-02T13:49:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:27 smithi188 ceph-mon[36117]: pgmap v447: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:27 smithi122 ceph-mon[32114]: pgmap v447: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:28.162 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:49:28.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:49:28.163 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:49:28.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:27 smithi187 ceph-mon[36064]: pgmap v447: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:28.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 -- 172.21.15.122:0/2793698542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35281048c0 msgr2=0x7f3528108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 --2- 172.21.15.122:0/2793698542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35281048c0 0x7f3528108900 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f3518009900 tx=0x7f351802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 -- 172.21.15.122:0/2793698542 shutdown_connections 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 --2- 172.21.15.122:0/2793698542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528108fe0 0x7f3528110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 --2- 172.21.15.122:0/2793698542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35281048c0 0x7f3528108900 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 --2- 172.21.15.122:0/2793698542 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3528103f10 0x7f35281042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 -- 172.21.15.122:0/2793698542 >> 172.21.15.122:0/2793698542 conn(0x7f35280ff7b0 msgr2=0x7f3528101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 -- 172.21.15.122:0/2793698542 shutdown_connections 2024-04-02T13:49:28.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.309+0000 7f3530504640 1 -- 172.21.15.122:0/2793698542 wait complete. 2024-04-02T13:49:28.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 Processor -- start 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 -- start start 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 0x7f352819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 0x7f352819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 0x7f352819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57548/0 (socket says 172.21.15.122:57548) 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52552/0 (socket says 172.21.15.122:52552) 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352e2a0640 1 -- 172.21.15.122:0/1405457862 learned_addr learned my addr 172.21.15.122:0/1405457862 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f352819ee70 0x7f352819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3528114090 con 0x7f3528108fe0 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3528113f10 con 0x7f352819ee70 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3528114210 con 0x7f3528103f10 2024-04-02T13:49:28.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.310+0000 7f352eaa1640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f352819ee70 0x7f352819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 -- 172.21.15.122:0/1405457862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 msgr2=0x7f352819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 0x7f352819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 -- 172.21.15.122:0/1405457862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f352819ee70 msgr2=0x7f352819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f352819ee70 0x7f352819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352819f960 con 0x7f3528108fe0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f352da9f640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3518036040 tx=0x7f351802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f351f7fe640 1 -- 172.21.15.122:0/1405457862 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518041070 con 0x7f3528108fe0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352819fbf0 con 0x7f3528108fe0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f351f7fe640 1 -- 172.21.15.122:0/1405457862 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3518004450 con 0x7f3528108fe0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f351f7fe640 1 -- 172.21.15.122:0/1405457862 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3518003d90 con 0x7f3528108fe0 2024-04-02T13:49:28.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.311+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35281a7410 con 0x7f3528108fe0 2024-04-02T13:49:28.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.315+0000 7f35331c9740 1 Processor -- start 2024-04-02T13:49:28.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.315+0000 7f351f7fe640 1 -- 172.21.15.122:0/1405457862 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3518044070 con 0x7f3528108fe0 2024-04-02T13:49:28.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.315+0000 7f35331c9740 1 -- start start 2024-04-02T13:49:28.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff65e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 0x55b3ff669bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff5c2770 con 0x55b3ff6677c0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff56a960 con 0x55b3ff65ebe0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f35331c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff5be8c0 con 0x55b3ff65e2c0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 0x55b3ff669bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352eaa1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff65e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60622/0 (socket says 172.21.15.122:60622) 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 0x55b3ff669bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52562/0 (socket says 172.21.15.122:52562) 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 -- 172.21.15.122:0/313273476 learned_addr learned my addr 172.21.15.122:0/313273476 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f351f7fe640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 0x7f35140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352eaa1640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 0x7f35140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 -- 172.21.15.122:0/313273476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 msgr2=0x55b3ff65e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff65e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 -- 172.21.15.122:0/313273476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 msgr2=0x55b3ff669bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 0x55b3ff669bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352e2a0640 1 -- 172.21.15.122:0/313273476 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ff39e7e0 con 0x55b3ff65ebe0 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.316+0000 7f352eaa1640 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff65e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:28.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.317+0000 7f351f7fe640 1 -- 172.21.15.122:0/1405457862 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f35180f2440 con 0x7f3528108fe0 2024-04-02T13:49:28.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.317+0000 7f352e2a0640 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3524009dc0 tx=0x7f3524011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aee0154d37ba151 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.317+0000 7f352eaa1640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 0x7f35140a6260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3520009fd0 tx=0x7f3520004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.317+0000 7f351cbfe640 1 -- 172.21.15.122:0/313273476 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524005220 con 0x55b3ff65ebe0 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.317+0000 7f351cbfe640 1 -- 172.21.15.122:0/313273476 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524004c80 con 0x55b3ff65ebe0 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.318+0000 7f351cbfe640 1 -- 172.21.15.122:0/313273476 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524019970 con 0x55b3ff65ebe0 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.318+0000 7f35331c9740 1 -- 172.21.15.122:0/313273476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 msgr2=0x55b3ff667280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.318+0000 7f35331c9740 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3524009dc0 tx=0x7f3524011810 comp rx=0 tx=0).stop 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 -- 172.21.15.122:0/313273476 shutdown_connections 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff6677c0 0x55b3ff669bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65ebe0 0x55b3ff667280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 --2- 172.21.15.122:0/313273476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff65e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 -- 172.21.15.122:0/313273476 >> 172.21.15.122:0/313273476 conn(0x55b3ff6589f0 msgr2=0x55b3ff658dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 -- 172.21.15.122:0/313273476 shutdown_connections 2024-04-02T13:49:28.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 -- 172.21.15.122:0/313273476 wait complete. 2024-04-02T13:49:28.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 20730 26.70 MB/sec warmup 48 sec latency 65.173 ms 2024-04-02T13:49:28.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 -- start start 2024-04-02T13:49:28.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.319+0000 7f35331c9740 1 Processor -- start 2024-04-02T13:49:28.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff664880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff665300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff66a5b0 con 0x55b3ff65ebe0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff57dd20 con 0x55b3ff65e2c0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b3ff66a730 con 0x55b3ff6677c0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff664880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52566/0 (socket says 172.21.15.122:52566) 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 -- 172.21.15.122:0/4273731687 learned_addr learned my addr 172.21.15.122:0/4273731687 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352eaa1640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff665300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f35331c9740 10 client.0 _ll_register_callbacks cb 0x55b3ff54aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 -- 172.21.15.122:0/4273731687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 msgr2=0x55b3ff665300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff665300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 -- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 msgr2=0x55b3ff664880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff664880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 -- 172.21.15.122:0/4273731687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ff3fef70 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f352da9f640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f351804c000 tx=0x7f35180f5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d160c7f70c5099a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.320+0000 7f3511ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351804b040 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f35180ba820 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3524004c80 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f35180bdf80 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 10 client.19776 fetch_fsmap learned FSMap version 53 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55b3ff296170 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524004c80 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 53) ==== 8414+0+0 (secure 0 0 0) 0x7f35180b9f00 con 0x55b3ff65ebe0 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 10 client.19776 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 10 client.19776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:28.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 20 client.19776 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff677ba0 0x55b3ff67a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 4 client.19776 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.321+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3ff5cfcb0 con 0x55b3ff677ba0 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.322+0000 7f352e2a0640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff677ba0 0x55b3ff67a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.322+0000 7f3511ffb640 10 client.19776 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.322+0000 7f352e2a0640 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff677ba0 0x55b3ff67a060 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ff7f5ac60719f1 server_cookie=84efcadea222d3b8 in_seq=0 out_seq=0 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f3511ffb640 1 -- 172.21.15.122:0/4273731687 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b3ff5cfcb0 con 0x55b3ff677ba0 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f3511ffb640 10 client.19776 handle_command_reply: tid=0 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 client.19776 shutdown 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 msgr2=0x55b3ff664dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f351804c000 tx=0x7f35180f5ac0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 shutdown_connections 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff677ba0 0x55b3ff67a060 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f3511ffb640 0 client.19776 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff665300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff664dc0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 --2- 172.21.15.122:0/4273731687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff664880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 >> 172.21.15.122:0/4273731687 conn(0x55b3ff6589f0 msgr2=0x55b3ff65a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 shutdown_connections 2024-04-02T13:49:28.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 1 -- 172.21.15.122:0/4273731687 wait complete. 2024-04-02T13:49:28.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.324+0000 7f35331c9740 20 client.19776 trim_cache size 0 max 16384 2024-04-02T13:49:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 Processor -- start 2024-04-02T13:49:28.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 -- start start 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff673470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff67f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35281b12f0 con 0x55b3ff65ebe0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35281b2fe0 con 0x55b3ff65e2c0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f35331c9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35281b19c0 con 0x55b3ff6677c0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff673470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f352da9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52570/0 (socket says 172.21.15.122:52570) 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.351+0000 7f352da9f640 1 -- 172.21.15.122:0/662147837 learned_addr learned my addr 172.21.15.122:0/662147837 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352eaa1640 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff67f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 -- 172.21.15.122:0/662147837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 msgr2=0x55b3ff67f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff67f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 -- 172.21.15.122:0/662147837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 msgr2=0x55b3ff673470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff673470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 -- 172.21.15.122:0/662147837 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3ff65ea10 con 0x55b3ff65ebe0 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f352da9f640 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f351802c820 tx=0x7f3518004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba93701b6136ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f3513fff640 1 -- 172.21.15.122:0/662147837 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351804b070 con 0x55b3ff65ebe0 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f3513fff640 1 -- 172.21.15.122:0/662147837 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35180be1b0 con 0x55b3ff65ebe0 2024-04-02T13:49:28.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.352+0000 7f3513fff640 1 -- 172.21.15.122:0/662147837 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35180d7020 con 0x55b3ff65ebe0 2024-04-02T13:49:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.353+0000 7f35331c9740 1 -- 172.21.15.122:0/662147837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 msgr2=0x55b3ff657930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.353+0000 7f35331c9740 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f351802c820 tx=0x7f3518004ee0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 -- 172.21.15.122:0/662147837 shutdown_connections 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff6677c0 0x55b3ff67f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff657930 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 --2- 172.21.15.122:0/662147837 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff65e2c0 0x55b3ff673470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 -- 172.21.15.122:0/662147837 >> 172.21.15.122:0/662147837 conn(0x55b3ff6589f0 msgr2=0x55b3ff657220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 -- 172.21.15.122:0/662147837 shutdown_connections 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 -- 172.21.15.122:0/662147837 wait complete. 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.354+0000 7f35331c9740 1 -- start start 2024-04-02T13:49:28.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 Processor -- start 2024-04-02T13:49:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff66c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 0x55b3ff6dd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f352e2a0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57564/0 (socket says 172.21.15.122:57564) 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f352e2a0640 1 -- 172.21.15.122:0/4067892717 learned_addr learned my addr 172.21.15.122:0/4067892717 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35281b2850 con 0x55b3ff65ebe0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35281b20c0 con 0x55b3ff6677c0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35281af2f0 con 0x55b3ff65e2c0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f352da9f640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff66c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.355+0000 7f352eaa1640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 0x55b3ff6dd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f35331c9740 10 client.0 _ll_register_callbacks cb 0x7f3518009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 -- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 msgr2=0x55b3ff6dd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 0x55b3ff6dd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 -- 172.21.15.122:0/4067892717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 msgr2=0x55b3ff66c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff66c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3ff6ee780 con 0x55b3ff65e2c0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352eaa1640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 0x55b3ff6dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35180be1b0 con 0x55b3ff65e2c0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f352e2a0640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3524009dc0 tx=0x7f3524009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e17a88bc616773c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f3511ffb640 10 client.28531 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.356+0000 7f3511ffb640 1 -- 172.21.15.122:0/4067892717 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524039050 con 0x55b3ff65e2c0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.357+0000 7f3511ffb640 1 -- 172.21.15.122:0/4067892717 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f352402bd50 con 0x55b3ff65e2c0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.357+0000 7f3511ffb640 1 -- 172.21.15.122:0/4067892717 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f352402f070 con 0x55b3ff65e2c0 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.357+0000 7f35331c9740 10 client.28531 fetch_fsmap learned FSMap version 53 2024-04-02T13:49:28.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.357+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=53}) -- 0x55b3ff6ef9c0 con 0x55b3ff65e2c0 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f3511ffb640 1 -- 172.21.15.122:0/4067892717 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 53) ==== 8414+0+0 (secure 0 0 0) 0x7f3524004040 con 0x55b3ff65e2c0 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 10 client.28531 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 10 client.28531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 20 client.28531 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff6effd0 0x55b3ff6f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 4 client.28531 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:49:28.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.359+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b3ff5ca000 con 0x55b3ff6effd0 2024-04-02T13:49:28.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.360+0000 7f352da9f640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff6effd0 0x55b3ff6f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.361+0000 7f3511ffb640 10 client.28531 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.361+0000 7f352da9f640 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff6effd0 0x55b3ff6f23c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e0e4b1fc0a9231 server_cookie=37936c1280eb7127 in_seq=0 out_seq=0 2024-04-02T13:49:28.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f3511ffb640 1 -- 172.21.15.122:0/4067892717 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b3ff5ca000 con 0x55b3ff6effd0 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f3511ffb640 10 client.28531 handle_command_reply: tid=0 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 client.28531 shutdown 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 msgr2=0x55b3ff66bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3524009dc0 tx=0x7f3524009f70 comp rx=0 tx=0).stop 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 shutdown_connections 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b3ff6effd0 0x55b3ff6f23c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b3ff6677c0 0x55b3ff6dd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3ff65ebe0 0x55b3ff66c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f3511ffb640 0 client.28531 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 --2- 172.21.15.122:0/4067892717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b3ff65e2c0 0x55b3ff66bb20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 >> 172.21.15.122:0/4067892717 conn(0x55b3ff6589f0 msgr2=0x55b3ff6650d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 shutdown_connections 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 1 -- 172.21.15.122:0/4067892717 wait complete. 2024-04-02T13:49:28.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.364+0000 7f35331c9740 20 client.28531 trim_cache size 0 max 16384 2024-04-02T13:49:28.368 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:49:28.368 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:49:28.368 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "dc93041e-ac37-4850-ad64-1f9ddfd5dbe5", 2024-04-02T13:49:28.368 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:49:28.368 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 msgr2=0x7f35140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 0x7f35140a6260 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3520009fd0 tx=0x7f3520004ab0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 msgr2=0x7f352819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f3518036040 tx=0x7f351802cb20 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 shutdown_connections 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f35140a3da0 0x7f35140a6260 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f352819ee70 0x7f352819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3528108fe0 0x7f352819aaf0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 --2- 172.21.15.122:0/1405457862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3528103f10 0x7f352819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.365+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 >> 172.21.15.122:0/1405457862 conn(0x7f35280ff7b0 msgr2=0x7f3528101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.366+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 shutdown_connections 2024-04-02T13:49:28.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.366+0000 7f3530504640 1 -- 172.21.15.122:0/1405457862 wait complete. 2024-04-02T13:49:28.383 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:49:28.383 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:49:28.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.539+0000 7f921dffb640 1 -- 172.21.15.122:0/358354534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218039070 con 0x7f9220101330 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 -- 172.21.15.122:0/358354534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220101330 msgr2=0x7f922010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 --2- 172.21.15.122:0/358354534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220101330 0x7f922010d840 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9218009cd0 tx=0x7f921802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 -- 172.21.15.122:0/358354534 shutdown_connections 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 --2- 172.21.15.122:0/358354534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220101330 0x7f922010d840 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 --2- 172.21.15.122:0/358354534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220100970 0x7f9220100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 --2- 172.21.15.122:0/358354534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220106980 0x7f9220106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 -- 172.21.15.122:0/358354534 >> 172.21.15.122:0/358354534 conn(0x7f9220077b00 msgr2=0x7f9220077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 -- 172.21.15.122:0/358354534 shutdown_connections 2024-04-02T13:49:28.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.540+0000 7f92253c8640 1 -- 172.21.15.122:0/358354534 wait complete. 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 Processor -- start 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 -- start start 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220101330 0x7f92201988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220106980 0x7f922019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9220110de0 con 0x7f9220100970 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9220110c60 con 0x7f9220101330 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f92253c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9220110f60 con 0x7f9220106980 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f921effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f921effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33026/0 (socket says 172.21.15.122:33026) 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f921effd640 1 -- 172.21.15.122:0/218606775 learned_addr learned my addr 172.21.15.122:0/218606775 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f921f7fe640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220106980 0x7f922019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.541+0000 7f921e7fc640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220101330 0x7f92201988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 -- 172.21.15.122:0/218606775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220106980 msgr2=0x7f922019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220106980 0x7f922019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 -- 172.21.15.122:0/218606775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220101330 msgr2=0x7f92201988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220101330 0x7f92201988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 -- 172.21.15.122:0/218606775 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922019d360 con 0x7f9220100970 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f921effd640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f921000e9f0 tx=0x7f921000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f9207fff640 1 -- 172.21.15.122:0/218606775 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921000cd80 con 0x7f9220100970 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f9207fff640 1 -- 172.21.15.122:0/218606775 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9210010910 con 0x7f9220100970 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922019d650 con 0x7f9220100970 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f9207fff640 1 -- 172.21.15.122:0/218606775 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018990 con 0x7f9220100970 2024-04-02T13:49:28.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.542+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92201a4ee0 con 0x7f9220100970 2024-04-02T13:49:28.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.545+0000 7f9207fff640 1 -- 172.21.15.122:0/218606775 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9210018b30 con 0x7f9220100970 2024-04-02T13:49:28.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 Processor -- start 2024-04-02T13:49:28.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 -- start start 2024-04-02T13:49:28.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 0x563657648cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x563657651890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x5636576541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5636575ac830 con 0x563657651dd0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563657554820 con 0x5636576491f0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9227fab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5636575a87e0 con 0x5636576488d0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9207fff640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 0x563657648cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 0x563657648cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:50870/0 (socket says 172.21.15.122:50870) 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921f7fe640 1 -- 172.21.15.122:0/1297685416 learned_addr learned my addr 172.21.15.122:0/1297685416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f9207fff640 1 -- 172.21.15.122:0/218606775 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f92100d0f30 con 0x7f9220100970 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x563657651890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921f7fe640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921e7fc640 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x5636576541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 -- 172.21.15.122:0/1297685416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 msgr2=0x563657648cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 0x563657648cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 -- 172.21.15.122:0/1297685416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 msgr2=0x5636576541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x5636576541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.546+0000 7f921effd640 1 -- 172.21.15.122:0/1297685416 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365738a7d0 con 0x5636576491f0 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.547+0000 7f921f7fe640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 0x7f92080a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9218005fe0 tx=0x7f9218005dc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.547+0000 7f921effd640 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x563657651890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92100ce290 tx=0x7f921006c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb629b8ba02188e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.547+0000 7f92057fa640 1 -- 172.21.15.122:0/1297685416 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921009cae0 con 0x5636576491f0 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.547+0000 7f92057fa640 1 -- 172.21.15.122:0/1297685416 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92100a0e90 con 0x5636576491f0 2024-04-02T13:49:28.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.547+0000 7f92057fa640 1 -- 172.21.15.122:0/1297685416 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210099150 con 0x5636576491f0 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- 172.21.15.122:0/1297685416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 msgr2=0x563657651890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x563657651890 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92100ce290 tx=0x7f921006c480 comp rx=0 tx=0).stop 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- 172.21.15.122:0/1297685416 shutdown_connections 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x5636576541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x563657651890 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 --2- 172.21.15.122:0/1297685416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576488d0 0x563657648cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- 172.21.15.122:0/1297685416 >> 172.21.15.122:0/1297685416 conn(0x563657643000 msgr2=0x5636576433e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- 172.21.15.122:0/1297685416 shutdown_connections 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- 172.21.15.122:0/1297685416 wait complete. 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 -- start start 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.548+0000 7f9227fab740 1 Processor -- start 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x56365764ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563657651dd0 0x56365764efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563657654b60 con 0x5636576488d0 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563657567d30 con 0x5636576491f0 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563657654ce0 con 0x563657651dd0 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f9227fab740 10 client.0 _ll_register_callbacks cb 0x563657567720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33044/0 (socket says 172.21.15.122:33044) 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 -- 172.21.15.122:0/662332294 learned_addr learned my addr 172.21.15.122:0/662332294 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921f7fe640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563657651dd0 0x56365764efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921e7fc640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x56365764ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 -- 172.21.15.122:0/662332294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563657651dd0 msgr2=0x56365764efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563657651dd0 0x56365764efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 -- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 msgr2=0x56365764ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x56365764ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.549+0000 7f921effd640 1 -- 172.21.15.122:0/662332294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636572c03f0 con 0x5636576488d0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f921effd640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f92100ce890 tx=0x7f921000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042e4ef6e7f1b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921009cae0 con 0x5636576488d0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f92100acde0 con 0x5636576488d0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92100a0e90 con 0x5636576488d0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f92100ad060 con 0x5636576488d0 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9227fab740 10 client.19794 fetch_fsmap learned FSMap version 53 2024-04-02T13:49:28.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.550+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5636572800f0 con 0x5636576488d0 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.551+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92100a0e90 con 0x5636576488d0 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 53) ==== 8414+0+0 (secure 0 0 0) 0x7f921006e1d0 con 0x5636576488d0 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 10 client.19794 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 10 client.19794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 20 client.19794 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56365766a410 0x56365766c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 4 client.19794 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636575b9d70 con 0x56365766a410 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.552+0000 7f921e7fc640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56365766a410 0x56365766c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.553+0000 7f9202ff5640 10 client.19794 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.553+0000 7f921e7fc640 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56365766a410 0x56365766c800 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8958a88911e12b07 server_cookie=648287645904fe3f in_seq=0 out_seq=0 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9202ff5640 1 -- 172.21.15.122:0/662332294 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636575b9d70 con 0x56365766a410 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9202ff5640 10 client.19794 handle_command_reply: tid=0 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 client.19794 shutdown 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 msgr2=0x5636576516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f92100ce890 tx=0x7f921000c810 comp rx=0 tx=0).stop 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 shutdown_connections 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56365766a410 0x56365766c800 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563657651dd0 0x56365764efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576491f0 0x56365764ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 --2- 172.21.15.122:0/662332294 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x5636576516f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 >> 172.21.15.122:0/662332294 conn(0x563657643000 msgr2=0x563657644d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9202ff5640 0 client.19794 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 shutdown_connections 2024-04-02T13:49:28.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.556+0000 7f9227fab740 1 -- 172.21.15.122:0/662332294 wait complete. 2024-04-02T13:49:28.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.557+0000 7f9227fab740 20 client.19794 trim_cache size 0 max 16384 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 Processor -- start 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 -- start start 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x563657642240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x5636576511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92201acec0 con 0x5636576488d0 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92201aebb0 con 0x563657651dd0 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f9227fab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ad590 con 0x5636576491f0 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f921f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60180/0 (socket says 172.21.15.122:60180) 2024-04-02T13:49:28.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.582+0000 7f921f7fe640 1 -- 172.21.15.122:0/3711580882 learned_addr learned my addr 172.21.15.122:0/3711580882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 -- 172.21.15.122:0/3711580882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 msgr2=0x5636576511b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x5636576511b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 -- 172.21.15.122:0/3711580882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 msgr2=0x563657642240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x563657642240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 -- 172.21.15.122:0/3711580882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365763ec50 con 0x563657651dd0 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f921f7fe640 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f921802c630 tx=0x7f9218005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9557f4952d979873 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f9204ff9640 1 -- 172.21.15.122:0/3711580882 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218039030 con 0x563657651dd0 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.583+0000 7f9204ff9640 1 -- 172.21.15.122:0/3711580882 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9218048040 con 0x563657651dd0 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.584+0000 7f9204ff9640 1 -- 172.21.15.122:0/3711580882 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92180348d0 con 0x563657651dd0 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.584+0000 7f9227fab740 1 -- 172.21.15.122:0/3711580882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 msgr2=0x5636576516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.584+0000 7f9227fab740 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f921802c630 tx=0x7f9218005bd0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- 172.21.15.122:0/3711580882 shutdown_connections 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563657651dd0 0x5636576516f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x5636576511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- 172.21.15.122:0/3711580882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5636576488d0 0x563657642240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- 172.21.15.122:0/3711580882 >> 172.21.15.122:0/3711580882 conn(0x563657643000 msgr2=0x56365764d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- 172.21.15.122:0/3711580882 shutdown_connections 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- 172.21.15.122:0/3711580882 wait complete. 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- start start 2024-04-02T13:49:28.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 Processor -- start 2024-04-02T13:49:28.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x56365764b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x56365764b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f92201ae420 con 0x563657651dd0 2024-04-02T13:49:28.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92201adc90 con 0x5636576488d0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.585+0000 7f9227fab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92201aaec0 con 0x5636576491f0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f9227fab740 10 client.0 _ll_register_callbacks cb 0x7f921009c8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:60194/0 (socket says 172.21.15.122:60194) 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 -- 172.21.15.122:0/277608950 learned_addr learned my addr 172.21.15.122:0/277608950 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921f7fe640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x56365764b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921e7fc640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x56365764b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 -- 172.21.15.122:0/277608950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 msgr2=0x56365764b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x56365764b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 -- 172.21.15.122:0/277608950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 msgr2=0x56365764b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x56365764b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 -- 172.21.15.122:0/277608950 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365764bf30 con 0x5636576488d0 2024-04-02T13:49:28.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921f7fe640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x56365764b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f921effd640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f921009ca70 tx=0x7f92100acfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9c8de097df9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f9202ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921009c440 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9218048040 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.586+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f921006e0d0 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f92100a0e90 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9227fab740 10 client.28790 fetch_fsmap learned FSMap version 53 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x5636576d0560 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9218048040 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 53) ==== 8414+0+0 (secure 0 0 0) 0x7f921009bf30 con 0x5636576488d0 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9227fab740 10 client.28790 fetch_fsmap finished waiting for FSMap version 53 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.587+0000 7f9227fab740 10 client.28790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:28.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f9227fab740 20 client.28790 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5636576d0bd0 0x5636576d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f9227fab740 4 client.28790 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636575b40c0 con 0x5636576d0bd0 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f921e7fc640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5636576d0bd0 0x5636576d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f921e7fc640 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5636576d0bd0 0x5636576d2ff0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9d00074a65fd334 server_cookie=6bd543c44d549982 in_seq=0 out_seq=0 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.588+0000 7f9202ff5640 10 client.28790 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.589+0000 7f9202ff5640 1 -- 172.21.15.122:0/277608950 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5636575b40c0 con 0x5636576d0bd0 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.589+0000 7f9202ff5640 10 client.28790 handle_command_reply: tid=0 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.589+0000 7f9227fab740 1 client.28790 shutdown 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.589+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 msgr2=0x563657662a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.589+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f921009ca70 tx=0x7f92100acfc0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 shutdown_connections 2024-04-02T13:49:28.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5636576d0bd0 0x5636576d2ff0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563657651dd0 0x56365764b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9202ff5640 0 client.28790 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5636576491f0 0x56365764b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 --2- 172.21.15.122:0/277608950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5636576488d0 0x563657662a90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 >> 172.21.15.122:0/277608950 conn(0x563657643000 msgr2=0x563657642240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 shutdown_connections 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 1 -- 172.21.15.122:0/277608950 wait complete. 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.590+0000 7f9227fab740 20 client.28790 trim_cache size 0 max 16384 2024-04-02T13:49:28.596 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "dc93041e-ac37-4850-ad64-1f9ddfd5dbe5": { 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "dc93041e-ac37-4850-ad64-1f9ddfd5dbe5", 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:49:28.597 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:49:28.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 msgr2=0x7f92080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 0x7f92080a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9218005fe0 tx=0x7f9218005dc0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 msgr2=0x7f9220198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:28.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f921000e9f0 tx=0x7f921000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 shutdown_connections 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f92080a3d20 0x7f92080a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9220106980 0x7f922019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9220101330 0x7f92201988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 --2- 172.21.15.122:0/218606775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9220100970 0x7f9220198370 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 >> 172.21.15.122:0/218606775 conn(0x7f9220077b00 msgr2=0x7f9220068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 shutdown_connections 2024-04-02T13:49:28.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:28.591+0000 7f92253c8640 1 -- 172.21.15.122:0/218606775 wait complete. 2024-04-02T13:49:28.607 INFO:tasks.cephfs.filesystem:scrub status for tag:dc93041e-ac37-4850-ad64-1f9ddfd5dbe5 - {'path': '/', 'tag': 'dc93041e-ac37-4850-ad64-1f9ddfd5dbe5', 'options': 'recursive,force'} 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:29.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:28 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:29.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:28 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:49:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:29.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:28 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:29.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 21698 27.41 MB/sec warmup 49 sec latency 65.208 ms 2024-04-02T13:49:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: pgmap v448: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:29 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: pgmap v448: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:29 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: pgmap v448: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:49:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:29 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:49:30.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 22177 27.42 MB/sec warmup 50 sec latency 281.759 ms 2024-04-02T13:49:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:31.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:30 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:49:31.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:31.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:31.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:30 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:49:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:49:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:49:31.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:30 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:49:31.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 22372 27.12 MB/sec warmup 51 sec latency 454.928 ms 2024-04-02T13:49:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:31 smithi122 ceph-mon[32114]: pgmap v449: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:32.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:31 smithi187 ceph-mon[36064]: pgmap v449: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:32.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:31 smithi188 ceph-mon[36117]: pgmap v449: 129 pgs: 129 active+clean; 956 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.0 MiB/s wr, 558 op/s 2024-04-02T13:49:32.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 22946 27.25 MB/sec warmup 52 sec latency 70.579 ms 2024-04-02T13:49:33.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 23641 27.36 MB/sec warmup 53 sec latency 52.128 ms 2024-04-02T13:49:34.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:33 smithi122 ceph-mon[32114]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 629 op/s 2024-04-02T13:49:34.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:33 smithi187 ceph-mon[36064]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 629 op/s 2024-04-02T13:49:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:33 smithi188 ceph-mon[36117]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 629 op/s 2024-04-02T13:49:34.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 24514 27.52 MB/sec warmup 54 sec latency 73.018 ms 2024-04-02T13:49:35.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 25299 28.02 MB/sec warmup 55 sec latency 52.172 ms 2024-04-02T13:49:36.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:35 smithi188 ceph-mon[36117]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-04-02T13:49:36.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:35 smithi122 ceph-mon[32114]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-04-02T13:49:36.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:35 smithi187 ceph-mon[36064]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 562 op/s 2024-04-02T13:49:36.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 26227 28.29 MB/sec warmup 56 sec latency 81.219 ms 2024-04-02T13:49:37.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 27019 28.64 MB/sec warmup 57 sec latency 57.729 ms 2024-04-02T13:49:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:37 smithi122 ceph-mon[32114]: pgmap v452: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-02T13:49:38.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:37 smithi187 ceph-mon[36064]: pgmap v452: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-02T13:49:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:37 smithi188 ceph-mon[36117]: pgmap v452: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-02T13:49:38.321 INFO:tasks.workunit.client.0.smithi122.stdout: 10 27869 28.83 MB/sec warmup 58 sec latency 46.579 ms 2024-04-02T13:49:39.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 28566 29.01 MB/sec warmup 59 sec latency 136.533 ms 2024-04-02T13:49:40.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:39 smithi122 ceph-mon[32114]: pgmap v453: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:39 smithi187 ceph-mon[36064]: pgmap v453: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:40.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:39 smithi188 ceph-mon[36117]: pgmap v453: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:40.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 29378 29.22 MB/sec warmup 60 sec latency 143.004 ms 2024-04-02T13:49:41.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:40 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:41.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:40 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:41.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:40 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:41.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 29409 28.74 MB/sec warmup 61 sec latency 863.974 ms 2024-04-02T13:49:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:41 smithi122 ceph-mon[32114]: pgmap v454: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:41 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:42.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:41 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:42.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:41 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:42.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:41 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:42.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:41 smithi187 ceph-mon[36064]: pgmap v454: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:41 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:41 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:41 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:42.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:41 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:41 smithi188 ceph-mon[36117]: pgmap v454: 129 pgs: 129 active+clean; 1022 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 614 op/s 2024-04-02T13:49:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:41 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:41 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:41 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:41 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:42.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 29517 28.34 MB/sec warmup 62 sec latency 1542.581 ms 2024-04-02T13:49:43.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:49:43.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:49:43.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:49:43.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 29868 28.31 MB/sec warmup 63 sec latency 397.229 ms 2024-04-02T13:49:44.136 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:43 smithi187 ceph-mon[36064]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 966 op/s 2024-04-02T13:49:44.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:43 smithi188 ceph-mon[36117]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 966 op/s 2024-04-02T13:49:44.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:43 smithi122 ceph-mon[32114]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 21 MiB/s wr, 966 op/s 2024-04-02T13:49:44.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 30173 28.18 MB/sec warmup 64 sec latency 215.906 ms 2024-04-02T13:49:45.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 30587 28.15 MB/sec warmup 65 sec latency 263.186 ms 2024-04-02T13:49:46.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:45 smithi122 ceph-mon[32114]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-04-02T13:49:46.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:45 smithi187 ceph-mon[36064]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-04-02T13:49:46.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:45 smithi188 ceph-mon[36117]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-04-02T13:49:46.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 31186 28.06 MB/sec warmup 66 sec latency 75.607 ms 2024-04-02T13:49:47.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 32007 28.28 MB/sec warmup 67 sec latency 124.692 ms 2024-04-02T13:49:48.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:47 smithi187 ceph-mon[36064]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 945 op/s 2024-04-02T13:49:48.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:47 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:48.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:47 smithi188 ceph-mon[36117]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 945 op/s 2024-04-02T13:49:48.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:47 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:48.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:47 smithi122 ceph-mon[32114]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 945 op/s 2024-04-02T13:49:48.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:47 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:49:48.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 32946 28.63 MB/sec warmup 68 sec latency 210.582 ms 2024-04-02T13:49:49.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 33725 28.88 MB/sec warmup 69 sec latency 1210.618 ms 2024-04-02T13:49:50.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:49 smithi187 ceph-mon[36064]: pgmap v458: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:50.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:49 smithi188 ceph-mon[36117]: pgmap v458: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:50.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:49 smithi122 ceph-mon[32114]: pgmap v458: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:50.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 34504 29.05 MB/sec warmup 70 sec latency 2210.654 ms 2024-04-02T13:49:51.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 35237 29.21 MB/sec warmup 71 sec latency 3210.685 ms 2024-04-02T13:49:52.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:51 smithi187 ceph-mon[36064]: pgmap v459: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:51 smithi188 ceph-mon[36117]: pgmap v459: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:52.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:51 smithi122 ceph-mon[32114]: pgmap v459: 129 pgs: 129 active+clean; 1.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 20 MiB/s wr, 880 op/s 2024-04-02T13:49:52.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 35911 29.39 MB/sec warmup 72 sec latency 3662.651 ms 2024-04-02T13:49:53.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:52 smithi187 ceph-mon[36064]: pgmap v460: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T13:49:53.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:52 smithi188 ceph-mon[36117]: pgmap v460: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T13:49:53.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:52 smithi122 ceph-mon[32114]: pgmap v460: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 23 MiB/s wr, 1.16k op/s 2024-04-02T13:49:53.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 36548 29.25 MB/sec warmup 73 sec latency 213.226 ms 2024-04-02T13:49:54.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 37643 29.74 MB/sec warmup 74 sec latency 43.363 ms 2024-04-02T13:49:55.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 38428 29.91 MB/sec warmup 75 sec latency 138.369 ms 2024-04-02T13:49:55.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:55 smithi187 ceph-mon[36064]: pgmap v461: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 808 op/s 2024-04-02T13:49:55.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:55 smithi188 ceph-mon[36117]: pgmap v461: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 808 op/s 2024-04-02T13:49:55.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:55 smithi122 ceph-mon[32114]: pgmap v461: 129 pgs: 129 active+clean; 1.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 808 op/s 2024-04-02T13:49:56.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39021 29.96 MB/sec warmup 76 sec latency 94.595 ms 2024-04-02T13:49:57.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 39887 30.22 MB/sec warmup 77 sec latency 58.321 ms 2024-04-02T13:49:57.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:57 smithi187 ceph-mon[36064]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 930 op/s 2024-04-02T13:49:57.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:57 smithi188 ceph-mon[36117]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 930 op/s 2024-04-02T13:49:57.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:57 smithi122 ceph-mon[32114]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 930 op/s 2024-04-02T13:49:58.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 40922 30.53 MB/sec warmup 78 sec latency 44.404 ms 2024-04-02T13:49:58.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:49:58.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.763+0000 7f2a63fff640 1 -- 172.21.15.122:0/2695905457 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c02cc80 con 0x7f2a6c112580 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 -- 172.21.15.122:0/2695905457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 msgr2=0x7f2a6c112960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 --2- 172.21.15.122:0/2695905457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c112960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009930 tx=0x7f2a5c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 -- 172.21.15.122:0/2695905457 shutdown_connections 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 --2- 172.21.15.122:0/2695905457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c119340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 --2- 172.21.15.122:0/2695905457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c10c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 --2- 172.21.15.122:0/2695905457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c112960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.764+0000 7f2a73855640 1 -- 172.21.15.122:0/2695905457 >> 172.21.15.122:0/2695905457 conn(0x7f2a6c06cb00 msgr2=0x7f2a6c10ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.765+0000 7f2a73855640 1 -- 172.21.15.122:0/2695905457 shutdown_connections 2024-04-02T13:49:58.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.765+0000 7f2a73855640 1 -- 172.21.15.122:0/2695905457 wait complete. 2024-04-02T13:49:58.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 Processor -- start 2024-04-02T13:49:58.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 -- start start 2024-04-02T13:49:58.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c117870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c1b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c06e1f0 con 0x7f2a6c10ce90 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c06e070 con 0x7f2a6c112580 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a73855640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c06e370 con 0x7f2a6c10c4d0 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c117870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34754/0 (socket says 172.21.15.122:34754) 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c117870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33110/0 (socket says 172.21.15.122:33110) 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 -- 172.21.15.122:0/283843124 learned_addr learned my addr 172.21.15.122:0/283843124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a71df2640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c1b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 -- 172.21.15.122:0/283843124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 msgr2=0x7f2a6c117870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c117870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 -- 172.21.15.122:0/283843124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 msgr2=0x7f2a6c1b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c1b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a70df0640 1 -- 172.21.15.122:0/283843124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c1b3560 con 0x7f2a6c10ce90 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.766+0000 7f2a71df2640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c1b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a70df0640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a627fc640 1 -- 172.21.15.122:0/283843124 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800cd80 con 0x7f2a6c10ce90 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c1b3730 con 0x7f2a6c10ce90 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c1b3c80 con 0x7f2a6c10ce90 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a627fc640 1 -- 172.21.15.122:0/283843124 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a68010910 con 0x7f2a6c10ce90 2024-04-02T13:49:58.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.767+0000 7f2a627fc640 1 -- 172.21.15.122:0/283843124 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680188f0 con 0x7f2a6c10ce90 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.770+0000 7f2a76444740 1 Processor -- start 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.770+0000 7f2a76444740 1 -- start start 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab71f1f0 0x55b6ab727890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 0x55b6ab72a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab682830 con 0x55b6ab727dd0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab62a820 con 0x55b6ab71e8d0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a76444740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab67e7e0 con 0x55b6ab71f1f0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.771+0000 7f2a71df2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50128/0 (socket says 172.21.15.122:50128) 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 -- 172.21.15.122:0/3453767061 learned_addr learned my addr 172.21.15.122:0/3453767061 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a627fc640 1 -- 172.21.15.122:0/283843124 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2a68018bd0 con 0x7f2a6c10ce90 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a715f1640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab71f1f0 0x55b6ab727890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a70df0640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 0x55b6ab72a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 -- 172.21.15.122:0/3453767061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab71f1f0 msgr2=0x55b6ab727890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab71f1f0 0x55b6ab727890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 -- 172.21.15.122:0/3453767061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 msgr2=0x55b6ab72a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.772+0000 7f2a71df2640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 0x55b6ab72a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a71df2640 1 -- 172.21.15.122:0/3453767061 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ab4607d0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a70df0640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 0x55b6ab72a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a627fc640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 0x7f2a580a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a71df2640 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011790 tx=0x7f2a64011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c089f86842c1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a627fc640 1 -- 172.21.15.122:0/283843124 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f2a680d0e10 con 0x7f2a6c10ce90 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a577fe640 1 -- 172.21.15.122:0/3453767061 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64003c30 con 0x55b6ab71e8d0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.773+0000 7f2a577fe640 1 -- 172.21.15.122:0/3453767061 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6400fe80 con 0x55b6ab71e8d0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.774+0000 7f2a715f1640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 0x7f2a580a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.774+0000 7f2a577fe640 1 -- 172.21.15.122:0/3453767061 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a640193e0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.774+0000 7f2a76444740 1 -- 172.21.15.122:0/3453767061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 msgr2=0x55b6ab71ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.774+0000 7f2a76444740 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a64011790 tx=0x7f2a64011c50 comp rx=0 tx=0).stop 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a715f1640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 0x7f2a580a62e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c02df90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 -- 172.21.15.122:0/3453767061 shutdown_connections 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab727dd0 0x55b6ab72a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab71f1f0 0x55b6ab727890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 --2- 172.21.15.122:0/3453767061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab71ecb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 -- 172.21.15.122:0/3453767061 >> 172.21.15.122:0/3453767061 conn(0x55b6ab719000 msgr2=0x55b6ab7193e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 -- 172.21.15.122:0/3453767061 shutdown_connections 2024-04-02T13:49:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.775+0000 7f2a76444740 1 -- 172.21.15.122:0/3453767061 wait complete. 2024-04-02T13:49:58.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.776+0000 7f2a76444740 1 -- start start 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.776+0000 7f2a76444740 1 Processor -- start 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab724e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a71df2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab724e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50134/0 (socket says 172.21.15.122:50134) 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 -- 172.21.15.122:0/1919157447 learned_addr learned my addr 172.21.15.122:0/1919157447 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab73bc70 con 0x55b6ab71f1f0 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab63dd30 con 0x55b6ab71e8d0 2024-04-02T13:49:58.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b6ab73bdf0 con 0x55b6ab727dd0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a76444740 10 client.0 _ll_register_callbacks cb 0x55b6ab63d720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 -- 172.21.15.122:0/1919157447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 msgr2=0x55b6ab724e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab724e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 -- 172.21.15.122:0/1919157447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 msgr2=0x55b6ab7248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.777+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a715f1640 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ab3963f0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009930 tx=0x7f2a5c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0760c810a88f632 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c045070 con 0x55b6ab71e8d0 2024-04-02T13:49:58.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f2a5c02cc00 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.778+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6ab68fd70 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c0046c0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a76444740 10 client.28802 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b6ab3560f0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b6ab68fd70 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f2a5c0035f0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a76444740 10 client.28802 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a76444740 10 client.28802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.779+0000 7f2a76444740 20 client.28802 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab748e60 0x55b6ab74b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a76444740 4 client.28802 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ab68a0c0 con 0x55b6ab748e60 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab748e60 0x55b6ab74b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab748e60 0x55b6ab74b2c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80714ac51a40fea server_cookie=2230fe8a746ad842 in_seq=0 out_seq=0 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.780+0000 7f2a54ff9640 10 client.28802 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1919157447 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6ab68a0c0 con 0x55b6ab748e60 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a54ff9640 10 client.28802 handle_command_reply: tid=0 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 client.28802 shutdown 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 msgr2=0x55b6ab727560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009930 tx=0x7f2a5c009260 comp rx=0 tx=0).stop 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 shutdown_connections 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab748e60 0x55b6ab74b2c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab724e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 --2- 172.21.15.122:0/1919157447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab727560 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 >> 172.21.15.122:0/1919157447 conn(0x55b6ab719000 msgr2=0x55b6ab71ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a54ff9640 0 client.28802 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 shutdown_connections 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 1 -- 172.21.15.122:0/1919157447 wait complete. 2024-04-02T13:49:58.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.781+0000 7f2a76444740 20 client.28802 trim_cache size 0 max 16384 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a76444740 1 Processor -- start 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a76444740 1 -- start start 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a76444740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a76444740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab71ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a76444740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab7180c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.808+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50144/0 (socket says 172.21.15.122:50144) 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 -- 172.21.15.122:0/1132538184 learned_addr learned my addr 172.21.15.122:0/1132538184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab71ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500100e0 con 0x55b6ab71f1f0 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50011e60 con 0x55b6ab71e8d0 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500107b0 con 0x55b6ab727dd0 2024-04-02T13:49:58.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a71df2640 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab7180c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 -- 172.21.15.122:0/1132538184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 msgr2=0x55b6ab7180c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab7180c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 -- 172.21.15.122:0/1132538184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 msgr2=0x55b6ab71ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab71ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 -- 172.21.15.122:0/1132538184 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ab71f020 con 0x55b6ab71e8d0 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.809+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c170 tx=0x7f2a5c004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c05eb93c17a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.810+0000 7f2a56ffd640 1 -- 172.21.15.122:0/1132538184 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0035f0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.810+0000 7f2a56ffd640 1 -- 172.21.15.122:0/1132538184 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c003790 con 0x55b6ab71e8d0 2024-04-02T13:49:58.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.810+0000 7f2a56ffd640 1 -- 172.21.15.122:0/1132538184 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c005410 con 0x55b6ab71e8d0 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 msgr2=0x55b6ab744be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c170 tx=0x7f2a5c004610 comp rx=0 tx=0).stop 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 shutdown_connections 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab7180c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71f1f0 0x55b6ab71ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 --2- 172.21.15.122:0/1132538184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71e8d0 0x55b6ab744be0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 >> 172.21.15.122:0/1132538184 conn(0x55b6ab719000 msgr2=0x55b6ab7179b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 shutdown_connections 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.811+0000 7f2a76444740 1 -- 172.21.15.122:0/1132538184 wait complete. 2024-04-02T13:49:58.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 -- start start 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 Processor -- start 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab73c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a500116d0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50010f40 con 0x55b6ab71f1f0 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5000e0e0 con 0x55b6ab727dd0 2024-04-02T13:49:58.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.812+0000 7f2a76444740 10 client.0 _ll_register_callbacks cb 0x7f2a5c004500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:58.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34796/0 (socket says 172.21.15.122:34796) 2024-04-02T13:49:58.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 -- 172.21.15.122:0/1806083820 learned_addr learned my addr 172.21.15.122:0/1806083820 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 -- 172.21.15.122:0/1806083820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 msgr2=0x55b6ab73c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:58.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab73c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 -- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71f1f0 msgr2=0x55b6ab7412f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7412f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.813+0000 7f2a715f1640 1 -- 172.21.15.122:0/1806083820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ab73ca90 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.814+0000 7f2a715f1640 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c240 tx=0x7f2a5c02cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93013c02a013028e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.814+0000 7f2a54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.814+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c045040 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.814+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f2a5c004a70 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.814+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c003790 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.815+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c0035f0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.815+0000 7f2a76444740 10 client.19815 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.815+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b6ab7aeef0 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c003790 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f2a5c054740 con 0x55b6ab71e8d0 2024-04-02T13:49:58.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 10 client.19815 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 10 client.19815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:49:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 20 client.19815 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab7af440 0x55b6ab7b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 4 client.19815 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:49:58.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.816+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6ab607560 con 0x55b6ab7af440 2024-04-02T13:49:58.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.817+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab7af440 0x55b6ab7b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.817+0000 7f2a70df0640 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab7af440 0x55b6ab7b1860 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a293da46a84907 server_cookie=15bff6f284dbd18f in_seq=0 out_seq=0 2024-04-02T13:49:58.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.817+0000 7f2a54ff9640 10 client.19815 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:58.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a54ff9640 1 -- 172.21.15.122:0/1806083820 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6ab607560 con 0x55b6ab7af440 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a54ff9640 10 client.19815 handle_command_reply: tid=0 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 client.19815 shutdown 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 msgr2=0x55b6ab740db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c02c240 tx=0x7f2a5c02cdb0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 shutdown_connections 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55b6ab7af440 0x55b6ab7b1860 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b6ab727dd0 0x55b6ab73c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b6ab71f1f0 0x55b6ab7412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 --2- 172.21.15.122:0/1806083820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b6ab71e8d0 0x55b6ab740db0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 >> 172.21.15.122:0/1806083820 conn(0x55b6ab719000 msgr2=0x55b6ab718240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.819+0000 7f2a54ff9640 0 client.19815 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.820+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 shutdown_connections 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.820+0000 7f2a76444740 1 -- 172.21.15.122:0/1806083820 wait complete. 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.820+0000 7f2a76444740 20 client.19815 trim_cache size 0 max 16384 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:49:58.824 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.821+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 msgr2=0x7f2a580a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.821+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 0x7f2a580a62e0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009900 tx=0x7f2a5c02df90 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.821+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 msgr2=0x7f2a6c117db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.821+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800e9f0 tx=0x7f2a6800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 shutdown_connections 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a580a3e20 0x7f2a580a62e0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a6c112580 0x7f2a6c1b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a6c10ce90 0x7f2a6c117db0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 --2- 172.21.15.122:0/283843124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6c10c4d0 0x7f2a6c117870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 >> 172.21.15.122:0/283843124 conn(0x7f2a6c06cb00 msgr2=0x7f2a6c108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 shutdown_connections 2024-04-02T13:49:58.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.822+0000 7f2a73855640 1 -- 172.21.15.122:0/283843124 wait complete. 2024-04-02T13:49:58.841 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:49:58.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:49:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 -- 172.21.15.122:0/868756924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8112770 msgr2=0x7f74a8114b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 --2- 172.21.15.122:0/868756924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8112770 0x7f74a8114b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009c10 tx=0x7f74a002bef0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 -- 172.21.15.122:0/868756924 shutdown_connections 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 --2- 172.21.15.122:0/868756924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8112770 0x7f74a8114b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 --2- 172.21.15.122:0/868756924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8076950 0x7f74a8112230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 --2- 172.21.15.122:0/868756924 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a8076410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.989+0000 7f74a759e640 1 -- 172.21.15.122:0/868756924 >> 172.21.15.122:0/868756924 conn(0x7f74a80ff5a0 msgr2=0x7f74a81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.990+0000 7f74a759e640 1 -- 172.21.15.122:0/868756924 shutdown_connections 2024-04-02T13:49:58.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.990+0000 7f74a759e640 1 -- 172.21.15.122:0/868756924 wait complete. 2024-04-02T13:49:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.991+0000 7f74a759e640 1 Processor -- start 2024-04-02T13:49:58.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.991+0000 7f74a759e640 1 -- start start 2024-04-02T13:49:58.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a759e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 0x7f74a819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8112770 0x7f74a81a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50176/0 (socket says 172.21.15.122:50176) 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a659c640 1 -- 172.21.15.122:0/3629589641 learned_addr learned my addr 172.21.15.122:0/3629589641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:58.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8118290 con 0x7f74a8112770 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a6d9d640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8112770 0x7f74a81a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.992+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8118110 con 0x7f74a8076030 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f74a8118410 con 0x7f74a8076950 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a5d9b640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 0x7f74a819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a659c640 1 -- 172.21.15.122:0/3629589641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 msgr2=0x7f74a819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a659c640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 0x7f74a819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a659c640 1 -- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8112770 msgr2=0x7f74a81a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a659c640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8112770 0x7f74a81a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a659c640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a81a3740 con 0x7f74a8076030 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.993+0000 7f74a5d9b640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 0x7f74a819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.994+0000 7f74a659c640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f749800e9c0 tx=0x7f749800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.994+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749800ccc0 con 0x7f74a8076030 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.994+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7498005500 con 0x7f74a8076030 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.994+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498010430 con 0x7f74a8076030 2024-04-02T13:49:58.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.995+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a81a3a30 con 0x7f74a8076030 2024-04-02T13:49:59.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.995+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81ab2c0 con 0x7f74a8076030 2024-04-02T13:49:59.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.996+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a8068c60 con 0x7f74a8076030 2024-04-02T13:49:59.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.998+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f74980046c0 con 0x7f74a8076030 2024-04-02T13:49:59.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:58.999+0000 7f74977fe640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 0x7f74900a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.000+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f74980d0f50 con 0x7f74a8076030 2024-04-02T13:49:59.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.000+0000 7f74a5d9b640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 0x7f74900a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.001+0000 7f74a5d9b640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 0x7f74900a62e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f749c005e00 tx=0x7f749c005c70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.008+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f749809d1a0 con 0x7f74a8076030 2024-04-02T13:49:59.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 41678 30.74 MB/sec warmup 79 sec latency 65.421 ms 2024-04-02T13:49:59.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.354+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74a819faa0 con 0x7f74a8076030 2024-04-02T13:49:59.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.355+0000 7f74977fe640 1 -- 172.21.15.122:0/3629589641 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11973 (secure 0 0 0) 0x7f749809caa0 con 0x7f74a8076030 2024-04-02T13:49:59.358 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:49:59.358 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:49:46.715053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:49:59.358 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 55 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 msgr2=0x7f74900a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 0x7f74900a62e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f749c005e00 tx=0x7f749c005c70 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 msgr2=0x7f74a819e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f749800e9c0 tx=0x7f749800ee80 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 shutdown_connections 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74900a3e20 0x7f74900a62e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74a8112770 0x7f74a81a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74a8076950 0x7f74a819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 --2- 172.21.15.122:0/3629589641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f74a8076030 0x7f74a819e750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 >> 172.21.15.122:0/3629589641 conn(0x7f74a80ff5a0 msgr2=0x7f74a8101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 shutdown_connections 2024-04-02T13:49:59.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.358+0000 7f74a759e640 1 -- 172.21.15.122:0/3629589641 wait complete. 2024-04-02T13:49:59.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:49:59.401 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:49:59 smithi122 ceph-mon[32114]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-04-02T13:49:59.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:49:59 smithi187 ceph-mon[36064]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-04-02T13:49:59.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:49:59 smithi188 ceph-mon[36117]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 11 MiB/s wr, 771 op/s 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 -- 172.21.15.122:0/1024494052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54104e80 msgr2=0x7f4d54105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1024494052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54104e80 0x7f4d54105330 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009900 tx=0x7f4d4402bc80 comp rx=0 tx=0).stop 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 -- 172.21.15.122:0/1024494052 shutdown_connections 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1024494052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54104e80 0x7f4d54105330 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1024494052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d54100870 0x7f4d54104940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1024494052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d540fff50 0x7f4d54100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 -- 172.21.15.122:0/1024494052 >> 172.21.15.122:0/1024494052 conn(0x7f4d54076b50 msgr2=0x7f4d540fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 -- 172.21.15.122:0/1024494052 shutdown_connections 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.531+0000 7f4d5c463640 1 -- 172.21.15.122:0/1024494052 wait complete. 2024-04-02T13:49:59.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 Processor -- start 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 -- start start 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 0x7f4d5410d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 0x7f4d54109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54110e20 con 0x7f4d54100870 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54110ca0 con 0x7f4d540fff50 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5c463640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54110fa0 con 0x7f4d54104e80 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d599fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d599fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34830/0 (socket says 172.21.15.122:34830) 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 0x7f4d5410d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.532+0000 7f4d599fe640 1 -- 172.21.15.122:0/1438557671 learned_addr learned my addr 172.21.15.122:0/1438557671 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 -- 172.21.15.122:0/1438557671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 msgr2=0x7f4d54109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 0x7f4d54109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 0x7f4d54109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 -- 172.21.15.122:0/1438557671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 msgr2=0x7f4d5410d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 0x7f4d5410d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 -- 172.21.15.122:0/1438557671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5410a3c0 con 0x7f4d54100870 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 0x7f4d5410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800ea90 tx=0x7f4d4800ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 0x7f4d54109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d537fe640 1 -- 172.21.15.122:0/1438557671 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800ce40 con 0x7f4d54100870 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.533+0000 7f4d537fe640 1 -- 172.21.15.122:0/1438557671 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d480035f0 con 0x7f4d54100870 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.534+0000 7f4d537fe640 1 -- 172.21.15.122:0/1438557671 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48010690 con 0x7f4d54100870 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.534+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5406b580 con 0x7f4d54100870 2024-04-02T13:49:59.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.534+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5406ba90 con 0x7f4d54100870 2024-04-02T13:49:59.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.537+0000 7f4d5f050740 1 Processor -- start 2024-04-02T13:49:59.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.537+0000 7f4d537fe640 1 -- 172.21.15.122:0/1438557671 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4d480037a0 con 0x7f4d54100870 2024-04-02T13:49:59.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 -- start start 2024-04-02T13:49:59.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc50c0 0x55af9fbcd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 0x55af9fbd0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af9fb287b0 con 0x55af9fbc50c0 2024-04-02T13:49:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af9fac7570 con 0x55af9fbcdca0 2024-04-02T13:49:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5f050740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af9fb24760 con 0x55af9fbc47a0 2024-04-02T13:49:59.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33198/0 (socket says 172.21.15.122:33198) 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d5aa00640 1 -- 172.21.15.122:0/3015580254 learned_addr learned my addr 172.21.15.122:0/3015580254 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.538+0000 7f4d599fe640 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 0x55af9fbd0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 -- 172.21.15.122:0/3015580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 msgr2=0x55af9fbd0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 0x55af9fbd0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 -- 172.21.15.122:0/3015580254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc50c0 msgr2=0x55af9fbcd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc50c0 0x55af9fbcd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 -- 172.21.15.122:0/3015580254 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9f83c3f0 con 0x55af9fbc47a0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d599fe640 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 0x55af9fbd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d44005ab0 tx=0x7f4d44005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809c8bc248c6fb8b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d50bfe640 1 -- 172.21.15.122:0/3015580254 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44041070 con 0x55af9fbc47a0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d50bfe640 1 -- 172.21.15.122:0/3015580254 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44004240 con 0x55af9fbc47a0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d50bfe640 1 -- 172.21.15.122:0/3015580254 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d440046c0 con 0x55af9fbc47a0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.539+0000 7f4d537fe640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 0x7f4d3c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 0x7f4d3c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d537fe640 1 -- 172.21.15.122:0/1438557671 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f4d480d1090 con 0x7f4d54100870 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 -- 172.21.15.122:0/3015580254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 msgr2=0x55af9fbc4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4d44005ab0 tx=0x7f4d44005e60 comp rx=0 tx=0).stop 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 -- 172.21.15.122:0/3015580254 shutdown_connections 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 0x7f4d3c0a62e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55af9fbc4270 tx=0x7f4d4c009b40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbcdca0 0x55af9fbd0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc50c0 0x55af9fbcd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3015580254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc47a0 0x55af9fbc4b80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.540+0000 7f4d5f050740 1 -- 172.21.15.122:0/3015580254 >> 172.21.15.122:0/3015580254 conn(0x55af9fbbeed0 msgr2=0x55af9fbbf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.541+0000 7f4d5f050740 1 -- 172.21.15.122:0/3015580254 shutdown_connections 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.541+0000 7f4d5f050740 1 -- 172.21.15.122:0/3015580254 wait complete. 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 -- start start 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 Processor -- start 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc50c0 0x55af9fbcb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbcdca0 0x55af9fbcb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55af9fbd0a30 con 0x55af9fbcdca0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55af9fae3e80 con 0x55af9fbc47a0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5f050740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af9fbd0bb0 con 0x55af9fbc50c0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50210/0 (socket says 172.21.15.122:50210) 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1825738265 learned_addr learned my addr 172.21.15.122:0/1825738265 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.542+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbcdca0 0x55af9fbcb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5f050740 10 client.0 _ll_register_callbacks cb 0x55af9fb1faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1825738265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc50c0 msgr2=0x55af9fbcb0c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:49:59.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc50c0 0x55af9fbcb0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbcdca0 msgr2=0x55af9fbcb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbcdca0 0x55af9fbcb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1825738265 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9f9067d0 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d44004240 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c011c90 tx=0x7f4d4c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c410257aefe768b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d39ffb640 10 client.28823 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.543+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1825738265 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01e620 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1825738265 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f4d4c01e7c0 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1825738265 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4d4c01ea00 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d5f050740 10 client.28823 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=55}) -- 0x55af9fbdcb10 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1825738265 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f4d4c01eba0 con 0x55af9fbc47a0 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d5f050740 10 client.28823 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:59.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.544+0000 7f4d5f050740 10 client.28823 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d5f050740 20 client.28823 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fbe0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d5f050740 4 client.28823 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af9fb35cf0 con 0x55af9fbddec0 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fbe0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fbe0330 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c35a31519f524eee server_cookie=de9c13d53747710c in_seq=0 out_seq=0 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.545+0000 7f4d39ffb640 10 client.28823 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.546+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1825738265 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af9fb35cf0 con 0x55af9fbddec0 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.546+0000 7f4d39ffb640 10 client.28823 handle_command_reply: tid=0 2024-04-02T13:49:59.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.547+0000 7f4d5f050740 1 client.28823 shutdown 2024-04-02T13:49:59.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.547+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 msgr2=0x55af9fbcab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.547+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c011c90 tx=0x7f4d4c009500 comp rx=0 tx=0).stop 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.547+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 shutdown_connections 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.547+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fbe0330 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbcdca0 0x55af9fbcb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d39ffb640 0 client.28823 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbc50c0 0x55af9fbcb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1825738265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc47a0 0x55af9fbcab80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 >> 172.21.15.122:0/1825738265 conn(0x55af9fbbeed0 msgr2=0x55af9fbc0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 shutdown_connections 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 1 -- 172.21.15.122:0/1825738265 wait complete. 2024-04-02T13:49:59.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.548+0000 7f4d5f050740 20 client.28823 trim_cache size 0 max 16384 2024-04-02T13:49:59.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.569+0000 7f4d5f050740 1 Processor -- start 2024-04-02T13:49:59.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.569+0000 7f4d5f050740 1 -- start start 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbd9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34856/0 (socket says 172.21.15.122:34856) 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/3559384174 learned_addr learned my addr 172.21.15.122:0/3559384174 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fbc9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541be040 con 0x55af9fbc47a0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541bfdc0 con 0x55af9fbc50c0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541be710 con 0x55af9fbcdca0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fbc9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d599fe640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbd9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/3559384174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 msgr2=0x55af9fbc9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fbc9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/3559384174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 msgr2=0x55af9fbd9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbd9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.570+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/3559384174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af9fbc90d0 con 0x55af9fbc47a0 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.571+0000 7f4d599fe640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbd9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:59.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.571+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ef10 tx=0x7f4d4c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6111213fd2fbda8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.571+0000 7f4d3bfff640 1 -- 172.21.15.122:0/3559384174 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c011840 con 0x55af9fbc47a0 2024-04-02T13:49:59.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.571+0000 7f4d3bfff640 1 -- 172.21.15.122:0/3559384174 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44004240 con 0x55af9fbc47a0 2024-04-02T13:49:59.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.571+0000 7f4d3bfff640 1 -- 172.21.15.122:0/3559384174 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c030a90 con 0x55af9fbc47a0 2024-04-02T13:49:59.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 msgr2=0x55af9fbcd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ef10 tx=0x7f4d4c01f000 comp rx=0 tx=0).stop 2024-04-02T13:49:59.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 shutdown_connections 2024-04-02T13:49:59.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fbc9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbd9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 --2- 172.21.15.122:0/3559384174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbcd2b0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 >> 172.21.15.122:0/3559384174 conn(0x55af9fbbeed0 msgr2=0x55af9fbbd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 shutdown_connections 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.572+0000 7f4d5f050740 1 -- 172.21.15.122:0/3559384174 wait complete. 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 -- start start 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 Processor -- start 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbc8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fc43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541bf630 con 0x55af9fbc47a0 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5a1ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34870/0 (socket says 172.21.15.122:34870) 2024-04-02T13:49:59.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1168034287 learned_addr learned my addr 172.21.15.122:0/1168034287 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4d541beea0 con 0x55af9fbc50c0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbc8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5410ab50 con 0x55af9fbcdca0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5aa00640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fc43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.573+0000 7f4d5f050740 10 client.0 _ll_register_callbacks cb 0x7f4d4c009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1168034287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 msgr2=0x55af9fc43470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fc43470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1168034287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 msgr2=0x55af9fbc8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbc8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 -- 172.21.15.122:0/1168034287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9fc439b0 con 0x55af9fbc47a0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5a1ff640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ece0 tx=0x7f4d4c0304f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b8670b8c096a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0309e0 con 0x55af9fbc47a0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d44004240 con 0x55af9fbc47a0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f4d4c030b80 con 0x55af9fbc47a0 2024-04-02T13:49:59.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f4d4c04d050 con 0x55af9fbc47a0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5f050740 10 client.19836 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.574+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55af9fc44ab0 con 0x55af9fbc47a0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d44004240 con 0x55af9fbc47a0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f4d4c047530 con 0x55af9fbc47a0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 10 client.19836 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 10 client.19836 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 20 client.19836 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fc470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 4 client.19836 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af9fb30040 con 0x55af9fbddec0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.575+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fc470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d39ffb640 10 client.19836 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d599fe640 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fc470d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3547657eef940da9 server_cookie=1909a0da2a18a9ff in_seq=0 out_seq=0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d39ffb640 1 -- 172.21.15.122:0/1168034287 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af9fb30040 con 0x55af9fbddec0 2024-04-02T13:49:59.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d39ffb640 10 client.19836 handle_command_reply: tid=0 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 client.19836 shutdown 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 msgr2=0x55af9fbc7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c00ece0 tx=0x7f4d4c0304f0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 shutdown_connections 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55af9fbddec0 0x55af9fc470d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af9fbcdca0 0x55af9fc43470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d39ffb640 0 client.19836 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55af9fbc50c0 0x55af9fbc8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 --2- 172.21.15.122:0/1168034287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55af9fbc47a0 0x55af9fbc7bc0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 >> 172.21.15.122:0/1168034287 conn(0x55af9fbbeed0 msgr2=0x55af9fbc0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 shutdown_connections 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 1 -- 172.21.15.122:0/1168034287 wait complete. 2024-04-02T13:49:59.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.576+0000 7f4d5f050740 20 client.19836 trim_cache size 0 max 16384 2024-04-02T13:49:59.584 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:59.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.577+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 msgr2=0x7f4d3c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.577+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 0x7f4d3c0a62e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55af9fbc4270 tx=0x7f4d4c009b40 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.577+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 msgr2=0x7f4d54109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.577+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800ea90 tx=0x7f4d4800ef50 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 shutdown_connections 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4d3c0a3e20 0x7f4d3c0a62e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d54104e80 0x7f4d54109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4d54100870 0x7f4d54109670 secure :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800ea90 tx=0x7f4d4800ef50 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 --2- 172.21.15.122:0/1438557671 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4d540fff50 0x7f4d5410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 >> 172.21.15.122:0/1438557671 conn(0x7f4d54076b50 msgr2=0x7f4d54102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 shutdown_connections 2024-04-02T13:49:59.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.578+0000 7f4d5c463640 1 -- 172.21.15.122:0/1438557671 wait complete. 2024-04-02T13:49:59.594 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:49:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 -- 172.21.15.122:0/2603877797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406ddb0 msgr2=0x7fd83411ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 --2- 172.21.15.122:0/2603877797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406ddb0 0x7fd83411ac60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd828009900 tx=0x7fd82802bc30 comp rx=0 tx=0).stop 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 -- 172.21.15.122:0/2603877797 shutdown_connections 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 --2- 172.21.15.122:0/2603877797 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd83411d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 --2- 172.21.15.122:0/2603877797 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406ddb0 0x7fd83411ac60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 --2- 172.21.15.122:0/2603877797 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd83406d490 0x7fd83406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.811+0000 7fd839e93640 1 -- 172.21.15.122:0/2603877797 >> 172.21.15.122:0/2603877797 conn(0x7fd83406cb00 msgr2=0x7fd83410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.812+0000 7fd839e93640 1 -- 172.21.15.122:0/2603877797 shutdown_connections 2024-04-02T13:49:59.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.812+0000 7fd839e93640 1 -- 172.21.15.122:0/2603877797 wait complete. 2024-04-02T13:49:59.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.813+0000 7fd839e93640 1 Processor -- start 2024-04-02T13:49:59.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 -- start start 2024-04-02T13:49:59.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 0x7fd8341a71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd83406ddb0 0x7fd8341a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd834120cf0 con 0x7fd83411b1a0 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34888/0 (socket says 172.21.15.122:34888) 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 -- 172.21.15.122:0/3203750022 learned_addr learned my addr 172.21.15.122:0/3203750022 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.814+0000 7fd839e93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd834120b70 con 0x7fd83406ddb0 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 0x7fd8341a71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd834120e70 con 0x7fd83406d490 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 -- 172.21.15.122:0/3203750022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 msgr2=0x7fd8341a71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 0x7fd8341a71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 -- 172.21.15.122:0/3203750022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd83406ddb0 msgr2=0x7fd8341a76f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd83406ddb0 0x7fd8341a76f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd833fff640 1 -- 172.21.15.122:0/3203750022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8341ac1a0 con 0x7fd83411b1a0 2024-04-02T13:49:59.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.815+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 0x7fd8341a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd833fff640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00efd0 tx=0x7fd82c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd830ff9640 1 -- 172.21.15.122:0/3203750022 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c003b70 con 0x7fd83411b1a0 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd830ff9640 1 -- 172.21.15.122:0/3203750022 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c003d10 con 0x7fd83411b1a0 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd830ff9640 1 -- 172.21.15.122:0/3203750022 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0109f0 con 0x7fd83411b1a0 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8341ac490 con 0x7fd83411b1a0 2024-04-02T13:49:59.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.816+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8341b3d20 con 0x7fd83411b1a0 2024-04-02T13:49:59.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.820+0000 7fd830ff9640 1 -- 172.21.15.122:0/3203750022 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd82c004020 con 0x7fd83411b1a0 2024-04-02T13:49:59.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.820+0000 7fd83ca7e740 1 Processor -- start 2024-04-02T13:49:59.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.821+0000 7fd830ff9640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 0x7fd81c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.821+0000 7fd830ff9640 1 -- 172.21.15.122:0/3203750022 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fd82c0d1090 con 0x7fd83411b1a0 2024-04-02T13:49:59.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.821+0000 7fd83ca7e740 1 -- start start 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.821+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 0x7fd81c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8718b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ce867c7b0 con 0x563ce87190c0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563ce861b570 con 0x563ce8721ca0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ce8678760 con 0x563ce87187a0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50254/0 (socket says 172.21.15.122:50254) 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 -- 172.21.15.122:0/2426430132 learned_addr learned my addr 172.21.15.122:0/2426430132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd8337fe640 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8718b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd832ffd640 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 0x7fd81c0a6260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd828036040 tx=0x7fd828005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 -- 172.21.15.122:0/2426430132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 msgr2=0x563ce8718b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8718b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 -- 172.21.15.122:0/2426430132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 msgr2=0x563ce8721760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.822+0000 7fd833fff640 1 -- 172.21.15.122:0/2426430132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce83903f0 con 0x563ce8721ca0 2024-04-02T13:49:59.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.823+0000 7fd833fff640 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0995b0 tx=0x7fd82c010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6d050621194a62 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.823+0000 7fd819ffb640 1 -- 172.21.15.122:0/2426430132 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0b6070 con 0x563ce8721ca0 2024-04-02T13:49:59.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.823+0000 7fd819ffb640 1 -- 172.21.15.122:0/2426430132 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c09cbe0 con 0x563ce8721ca0 2024-04-02T13:49:59.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.823+0000 7fd819ffb640 1 -- 172.21.15.122:0/2426430132 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c099250 con 0x563ce8721ca0 2024-04-02T13:49:59.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2426430132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 msgr2=0x563ce8724090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0995b0 tx=0x7fd82c010430 comp rx=0 tx=0).stop 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2426430132 shutdown_connections 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8724090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2426430132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8718b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2426430132 >> 172.21.15.122:0/2426430132 conn(0x563ce8712ed0 msgr2=0x563ce87132b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2426430132 shutdown_connections 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.824+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2426430132 wait complete. 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.825+0000 7fd83ca7e740 1 -- start start 2024-04-02T13:49:59.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.825+0000 7fd83ca7e740 1 Processor -- start 2024-04-02T13:49:59.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.825+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8721740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce871ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ce8724a30 con 0x563ce87190c0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563ce8637e80 con 0x563ce8721ca0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ce8724bb0 con 0x563ce87187a0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd83ca7e740 10 client.0 _ll_register_callbacks cb 0x563ce8673aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50260/0 (socket says 172.21.15.122:50260) 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 -- 172.21.15.122:0/2972889931 learned_addr learned my addr 172.21.15.122:0/2972889931 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd8337fe640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8721740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 -- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 msgr2=0x563ce8721740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8721740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd833fff640 1 -- 172.21.15.122:0/2972889931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 msgr2=0x563ce871ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.826+0000 7fd832ffd640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce871ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd833fff640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce871ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd833fff640 1 -- 172.21.15.122:0/2972889931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce845a7d0 con 0x563ce8721ca0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd833fff640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd82c09ae90 tx=0x7fd82c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f813e4d3e3e0cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd832ffd640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce871ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82c0bb020 con 0x563ce8721ca0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fd82c0ce610 con 0x563ce8721ca0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82c09cbe0 con 0x563ce8721ca0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd82c0ce810 con 0x563ce8721ca0 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd83ca7e740 10 client.28838 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:59.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.827+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563ce8730b00 con 0x563ce8721ca0 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd82c09cbe0 con 0x563ce8721ca0 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fd82c01f020 con 0x563ce8721ca0 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 10 client.28838 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 10 client.28838 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 20 client.28838 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce8731e50 0x563ce8734240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 4 client.28838 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ce8689cf0 con 0x563ce8731e50 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.828+0000 7fd8337fe640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce8731e50 0x563ce8734240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.829+0000 7fd812ffd640 10 client.28838 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.829+0000 7fd8337fe640 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce8731e50 0x563ce8734240 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98f81862ac341e12 server_cookie=193e8304c91dc9fe in_seq=0 out_seq=0 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.830+0000 7fd812ffd640 1 -- 172.21.15.122:0/2972889931 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ce8689cf0 con 0x563ce8731e50 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.830+0000 7fd812ffd640 10 client.28838 handle_command_reply: tid=0 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.830+0000 7fd83ca7e740 1 client.28838 shutdown 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 msgr2=0x563ce871f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd82c09ae90 tx=0x7fd82c004ab0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 shutdown_connections 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce871f1c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce8731e50 0x563ce8734240 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce871ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/2972889931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8721740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 >> 172.21.15.122:0/2972889931 conn(0x563ce8712ed0 msgr2=0x563ce8714c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd812ffd640 0 client.28838 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:59.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 shutdown_connections 2024-04-02T13:49:59.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 1 -- 172.21.15.122:0/2972889931 wait complete. 2024-04-02T13:49:59.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.831+0000 7fd83ca7e740 20 client.28838 trim_cache size 0 max 16384 2024-04-02T13:49:59.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.851+0000 7fd83ca7e740 1 Processor -- start 2024-04-02T13:49:59.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.851+0000 7fd83ca7e740 1 -- start start 2024-04-02T13:49:59.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8721740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b5630 con 0x563ce87190c0 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b73b0 con 0x563ce8721ca0 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b5d00 con 0x563ce87187a0 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33256/0 (socket says 172.21.15.122:33256) 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 -- 172.21.15.122:0/3884550452 learned_addr learned my addr 172.21.15.122:0/3884550452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd833fff640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8721740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 -- 172.21.15.122:0/3884550452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 msgr2=0x563ce8721740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8721740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 -- 172.21.15.122:0/3884550452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 msgr2=0x563ce8721200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd8337fe640 1 -- 172.21.15.122:0/3884550452 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ce871d020 con 0x563ce87187a0 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.852+0000 7fd833fff640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8721740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:49:59.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.853+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8240233b0 tx=0x7fd824005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=698cb862050926d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.853+0000 7fd8197fa640 1 -- 172.21.15.122:0/3884550452 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824004020 con 0x563ce87187a0 2024-04-02T13:49:59.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.853+0000 7fd8197fa640 1 -- 172.21.15.122:0/3884550452 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd824036070 con 0x563ce87187a0 2024-04-02T13:49:59.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.853+0000 7fd8197fa640 1 -- 172.21.15.122:0/3884550452 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd824032740 con 0x563ce87187a0 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3884550452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 msgr2=0x563ce872d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd8240233b0 tx=0x7fd824005f90 comp rx=0 tx=0).stop 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3884550452 shutdown_connections 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8721740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8721200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3884550452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce872d950 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3884550452 >> 172.21.15.122:0/3884550452 conn(0x563ce8712ed0 msgr2=0x563ce871e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.854+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3884550452 shutdown_connections 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.855+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3884550452 wait complete. 2024-04-02T13:49:59.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.855+0000 7fd83ca7e740 1 -- start start 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.855+0000 7fd83ca7e740 1 Processor -- start 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8722660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8722ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33272/0 (socket says 172.21.15.122:33272) 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 -- 172.21.15.122:0/3911597179 learned_addr learned my addr 172.21.15.122:0/3911597179 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b6c20 con 0x563ce87190c0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd8341b6490 con 0x563ce8721ca0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd834115ca0 con 0x563ce87187a0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd83ca7e740 10 client.0 _ll_register_callbacks cb 0x7fd82c0b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8722660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 -- 172.21.15.122:0/3911597179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 msgr2=0x563ce8722ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:49:59.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8722ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 -- 172.21.15.122:0/3911597179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 msgr2=0x563ce8722660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8722660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.856+0000 7fd8337fe640 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ce8728f10 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.857+0000 7fd8337fe640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8240322c0 tx=0x7fd824009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cbf1b7fba50e28d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.857+0000 7fd812ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8240326c0 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fd824032860 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd824036070 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fd8240219a0 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd83ca7e740 10 client.28597 fetch_fsmap learned FSMap version 55 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.858+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x563ce872a1d0 con 0x563ce87187a0 2024-04-02T13:49:59.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd824036070 con 0x563ce87187a0 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fd82403abf0 con 0x563ce87187a0 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 10 client.28597 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 10 client.28597 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 20 client.28597 populate_metadata read hostname 'smithi122' 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce87980d0 0x563ce879a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:49:59.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 4 client.28597 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:49:59.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ce8684040 con 0x563ce87980d0 2024-04-02T13:49:59.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.859+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce87980d0 0x563ce879a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:49:59.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.860+0000 7fd812ffd640 10 client.28597 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:59.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.860+0000 7fd832ffd640 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce87980d0 0x563ce879a4c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1442eaed198f85f1 server_cookie=72065d60b7a11c99 in_seq=0 out_seq=0 2024-04-02T13:49:59.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd812ffd640 1 -- 172.21.15.122:0/3911597179 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ce8684040 con 0x563ce87980d0 2024-04-02T13:49:59.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd812ffd640 10 client.28597 handle_command_reply: tid=0 2024-04-02T13:49:59.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 client.28597 shutdown 2024-04-02T13:49:59.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 msgr2=0x563ce8722120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8240322c0 tx=0x7fd824009310 comp rx=0 tx=0).stop 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 shutdown_connections 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ce8721ca0 0x563ce8722ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ce87980d0 0x563ce879a4c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ce87190c0 0x563ce8722660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 --2- 172.21.15.122:0/3911597179 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ce87187a0 0x563ce8722120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd812ffd640 0 client.28597 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:49:59.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.867+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 >> 172.21.15.122:0/3911597179 conn(0x563ce8712ed0 msgr2=0x563ce8719de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 shutdown_connections 2024-04-02T13:49:59.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd83ca7e740 1 -- 172.21.15.122:0/3911597179 wait complete. 2024-04-02T13:49:59.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd83ca7e740 20 client.28597 trim_cache size 0 max 16384 2024-04-02T13:49:59.871 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 msgr2=0x7fd81c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 0x7fd81c0a6260 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd828036040 tx=0x7fd828005fd0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 msgr2=0x7fd8341aba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.868+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd82c00efd0 tx=0x7fd82c00c6a0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 shutdown_connections 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd81c0a3da0 0x7fd81c0a6260 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83411b1a0 0x7fd8341aba70 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd83406ddb0 0x7fd8341a76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 --2- 172.21.15.122:0/3203750022 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83406d490 0x7fd8341a71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:49:59.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 >> 172.21.15.122:0/3203750022 conn(0x7fd83406cb00 msgr2=0x7fd83411c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:49:59.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 shutdown_connections 2024-04-02T13:49:59.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:49:59.869+0000 7fd839e93640 1 -- 172.21.15.122:0/3203750022 wait complete. 2024-04-02T13:49:59.884 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 -- 172.21.15.122:0/2373271213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 msgr2=0x7fe19c117040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 --2- 172.21.15.122:0/2373271213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c117040 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe194009be0 tx=0x7fe19402bef0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 -- 172.21.15.122:0/2373271213 shutdown_connections 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 --2- 172.21.15.122:0/2373271213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c117040 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 --2- 172.21.15.122:0/2373271213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 --2- 172.21.15.122:0/2373271213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c1090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 -- 172.21.15.122:0/2373271213 >> 172.21.15.122:0/2373271213 conn(0x7fe19c06cbe0 msgr2=0x7fe19c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 -- 172.21.15.122:0/2373271213 shutdown_connections 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.035+0000 7fe1a1088640 1 -- 172.21.15.122:0/2373271213 wait complete. 2024-04-02T13:50:00.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.036+0000 7fe1a1088640 1 Processor -- start 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 -- start start 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c111890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c112310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10c080 con 0x7fe19c114c50 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.037+0000 7fe1a1088640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10bf00 con 0x7fe19c109600 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe1a1088640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10c200 con 0x7fe19c108ce0 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50292/0 (socket says 172.21.15.122:50292) 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 -- 172.21.15.122:0/522705654 learned_addr learned my addr 172.21.15.122:0/522705654 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c111890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19b59e640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c112310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 -- 172.21.15.122:0/522705654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 msgr2=0x7fe19c111890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c111890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 -- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 msgr2=0x7fe19c112310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c112310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19a59c640 1 -- 172.21.15.122:0/522705654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c1b3460 con 0x7fe19c109600 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19b59e640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.038+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:00.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe19a59c640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe19000e9f0 tx=0x7fe19000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe183fff640 1 -- 172.21.15.122:0/522705654 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19000ccc0 con 0x7fe19c109600 2024-04-02T13:50:00.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe183fff640 1 -- 172.21.15.122:0/522705654 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe19000ce60 con 0x7fe19c109600 2024-04-02T13:50:00.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe183fff640 1 -- 172.21.15.122:0/522705654 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190010690 con 0x7fe19c109600 2024-04-02T13:50:00.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1b3750 con 0x7fe19c109600 2024-04-02T13:50:00.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.039+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1b3bf0 con 0x7fe19c109600 2024-04-02T13:50:00.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe183fff640 1 -- 172.21.15.122:0/522705654 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe1900046c0 con 0x7fe19c109600 2024-04-02T13:50:00.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe183fff640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 0x7fe1840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe1a3c73740 1 Processor -- start 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe1a3c73740 1 -- start start 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe183fff640 1 -- 172.21.15.122:0/522705654 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fe1900d0de0 con 0x7fe19c109600 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.043+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 0x7fe1840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a9ca0 0x55d5166ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d5166047b0 con 0x55d5166a10c0 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d5165a3570 con 0x55d5166a9ca0 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d516600760 con 0x55d5166a07a0 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34968/0 (socket says 172.21.15.122:34968) 2024-04-02T13:50:00.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 -- 172.21.15.122:0/2673563695 learned_addr learned my addr 172.21.15.122:0/2673563695 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19b59e640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a9ca0 0x55d5166ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 0x7fe1840a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00bd30 tx=0x7fe18c00f730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 -- 172.21.15.122:0/2673563695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 msgr2=0x55d5166a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 -- 172.21.15.122:0/2673563695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a9ca0 msgr2=0x55d5166ac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a9ca0 0x55d5166ac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 -- 172.21.15.122:0/2673563695 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5163183f0 con 0x55d5166a10c0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.044+0000 7fe19a59c640 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe190099360 tx=0x7fe19009c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5472ffee423663aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.045+0000 7fe1817fa640 1 -- 172.21.15.122:0/2673563695 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1900a0d40 con 0x55d5166a10c0 2024-04-02T13:50:00.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.045+0000 7fe1817fa640 1 -- 172.21.15.122:0/2673563695 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1900a5600 con 0x55d5166a10c0 2024-04-02T13:50:00.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.045+0000 7fe1817fa640 1 -- 172.21.15.122:0/2673563695 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1900870b0 con 0x55d5166a10c0 2024-04-02T13:50:00.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- 172.21.15.122:0/2673563695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 msgr2=0x55d5166a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe190099360 tx=0x7fe19009c890 comp rx=0 tx=0).stop 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- 172.21.15.122:0/2673563695 shutdown_connections 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a9ca0 0x55d5166ac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a9760 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/2673563695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- 172.21.15.122:0/2673563695 >> 172.21.15.122:0/2673563695 conn(0x55d51669aed0 msgr2=0x55d51669b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- 172.21.15.122:0/2673563695 shutdown_connections 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- 172.21.15.122:0/2673563695 wait complete. 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 -- start start 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.046+0000 7fe1a3c73740 1 Processor -- start 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a9ca0 0x55d5166a71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d5166aca70 con 0x55d5166a10c0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d5165bfe80 con 0x55d5166a07a0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d5166acbf0 con 0x55d5166a9ca0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe1a3c73740 10 client.0 _ll_register_callbacks cb 0x55d5165fbaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50316/0 (socket says 172.21.15.122:50316) 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 -- 172.21.15.122:0/1365341354 learned_addr learned my addr 172.21.15.122:0/1365341354 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 -- 172.21.15.122:0/1365341354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a9ca0 msgr2=0x55d5166a71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a9ca0 0x55d5166a71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 -- 172.21.15.122:0/1365341354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 msgr2=0x55d5166a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 -- 172.21.15.122:0/1365341354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5163e27d0 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe18c012780 tx=0x7fe18c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b863534221df93 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.047+0000 7fe17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c02b060 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fe18c01db40 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1900a5600 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe18c032050 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 10 client.28853 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d5166b8b20 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1900a5600 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fe18c01e430 con 0x55d5166a07a0 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 10 client.28853 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 10 client.28853 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 20 client.28853 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d5166ba080 0x55d5166bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 4 client.28853 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:00.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.048+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d516611cf0 con 0x55d5166ba080 2024-04-02T13:50:00.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.049+0000 7fe19a59c640 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d5166ba080 0x55d5166bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.049+0000 7fe19a59c640 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d5166ba080 0x55d5166bc500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87eb6e9843a816e6 server_cookie=d4d36e4cb692b40e in_seq=0 out_seq=0 2024-04-02T13:50:00.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.049+0000 7fe17eff5640 10 client.28853 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:00.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.051+0000 7fe17eff5640 1 -- 172.21.15.122:0/1365341354 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d516611cf0 con 0x55d5166ba080 2024-04-02T13:50:00.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe17eff5640 10 client.28853 handle_command_reply: tid=0 2024-04-02T13:50:00.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 client.28853 shutdown 2024-04-02T13:50:00.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 msgr2=0x55d5166a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe18c012780 tx=0x7fe18c004730 comp rx=0 tx=0).stop 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 shutdown_connections 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a9ca0 0x55d5166a71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a10c0 0x55d5166a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d5166ba080 0x55d5166bc500 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/1365341354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166a96f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 >> 172.21.15.122:0/1365341354 conn(0x55d51669aed0 msgr2=0x55d51669cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe17eff5640 0 client.28853 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 shutdown_connections 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 1 -- 172.21.15.122:0/1365341354 wait complete. 2024-04-02T13:50:00.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.052+0000 7fe1a3c73740 20 client.28853 trim_cache size 0 max 16384 2024-04-02T13:50:00.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.077+0000 7fe1a3c73740 1 Processor -- start 2024-04-02T13:50:00.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe1a3c73740 1 -- start start 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 0x55d5166bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe19ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe19ad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50332/0 (socket says 172.21.15.122:50332) 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 0x55d5166bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.078+0000 7fe19a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 0x55d5166bc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33330/0 (socket says 172.21.15.122:33330) 2024-04-02T13:50:00.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 -- 172.21.15.122:0/4165619203 learned_addr learned my addr 172.21.15.122:0/4165619203 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1bdfa0 con 0x55d5166a9ca0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1bfd20 con 0x55d5166a07a0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1be670 con 0x55d5166a10c0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19b59e640 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 -- 172.21.15.122:0/4165619203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 msgr2=0x55d5166bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 0x55d5166bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 -- 172.21.15.122:0/4165619203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 msgr2=0x55d5166c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166c1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 -- 172.21.15.122:0/4165619203 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5166acb40 con 0x55d5166a07a0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00bb10 tx=0x7fe18c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41f9f6e2fb98d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe180ff9640 1 -- 172.21.15.122:0/4165619203 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c01ea70 con 0x55d5166a07a0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.079+0000 7fe180ff9640 1 -- 172.21.15.122:0/4165619203 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c01dc70 con 0x55d5166a07a0 2024-04-02T13:50:00.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe180ff9640 1 -- 172.21.15.122:0/4165619203 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0386a0 con 0x55d5166a07a0 2024-04-02T13:50:00.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 msgr2=0x55d5166b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00bb10 tx=0x7fe18c00b8d0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 shutdown_connections 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a10c0 0x55d5166bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/4165619203 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a07a0 0x55d5166b5930 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 >> 172.21.15.122:0/4165619203 conn(0x55d51669aed0 msgr2=0x55d5166a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.080+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 shutdown_connections 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 -- 172.21.15.122:0/4165619203 wait complete. 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 -- start start 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 Processor -- start 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a10c0 0x55d5166a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1bf590 con 0x55d5166a9ca0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1bee00 con 0x55d5166a10c0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1bbfa0 con 0x55d5166a07a0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe19b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe19b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35004/0 (socket says 172.21.15.122:35004) 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe19b59e640 1 -- 172.21.15.122:0/3678583617 learned_addr learned my addr 172.21.15.122:0/3678583617 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe19a59c640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a10c0 0x55d5166a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.081+0000 7fe1a3c73740 10 client.0 _ll_register_callbacks cb 0x7fe18c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 -- 172.21.15.122:0/3678583617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 msgr2=0x55d5166b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 -- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a10c0 msgr2=0x55d5166a3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a10c0 0x55d5166a3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 -- 172.21.15.122:0/3678583617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5166a3f90 con 0x55d5166a9ca0 2024-04-02T13:50:00.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe19b59e640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe19c06af20 tx=0x7fe194033480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10570fbfc63a1fea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe17eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1940335f0 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7fe194033790 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c01dc70 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe194003970 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe1a3c73740 10 client.19869 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55d516720680 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.082+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c01dc70 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fe1940040d0 con 0x55d5166a9ca0 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 10 client.19869 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 10 client.19869 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 20 client.19869 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d516720df0 0x55d516723210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 4 client.19869 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d51660c040 con 0x55d516720df0 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d516720df0 0x55d516723210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe17eff5640 10 client.19869 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.083+0000 7fe19ad9d640 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d516720df0 0x55d516723210 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35540470e88fd5aa server_cookie=e4fa04e620cd286 in_seq=0 out_seq=0 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe17eff5640 1 -- 172.21.15.122:0/3678583617 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d51660c040 con 0x55d516720df0 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe17eff5640 10 client.19869 handle_command_reply: tid=0 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 client.19869 shutdown 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 msgr2=0x55d5166a38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fe19c06af20 tx=0x7fe194033480 comp rx=0 tx=0).stop 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 shutdown_connections 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d5166a9ca0 0x55d5166a38d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55d516720df0 0x55d516723210 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d5166a10c0 0x55d5166a3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 --2- 172.21.15.122:0/3678583617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d5166a07a0 0x55d5166b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 >> 172.21.15.122:0/3678583617 conn(0x55d51669aed0 msgr2=0x55d5166a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe17eff5640 0 client.19869 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:00.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 shutdown_connections 2024-04-02T13:50:00.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.085+0000 7fe1a3c73740 1 -- 172.21.15.122:0/3678583617 wait complete. 2024-04-02T13:50:00.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.086+0000 7fe1a3c73740 20 client.19869 trim_cache size 0 max 16384 2024-04-02T13:50:00.092 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.087+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 msgr2=0x7fe1840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.087+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 0x7fe1840a6260 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00bd30 tx=0x7fe18c00f730 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.087+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 msgr2=0x7fe19c111dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.087+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe19000e9f0 tx=0x7fe19000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 shutdown_connections 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe1840a3da0 0x7fe1840a6260 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe19c114c50 0x7fe19c112310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe19c109600 0x7fe19c111dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 --2- 172.21.15.122:0/522705654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe19c108ce0 0x7fe19c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 >> 172.21.15.122:0/522705654 conn(0x7fe19c06cbe0 msgr2=0x7fe19c0705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 shutdown_connections 2024-04-02T13:50:00.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.088+0000 7fe1a1088640 1 -- 172.21.15.122:0/522705654 wait complete. 2024-04-02T13:50:00.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:50:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/3689240771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 msgr2=0x7f8cd0100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/3689240771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 0x7f8cd0100ea0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009900 tx=0x7f8cbc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/3689240771 shutdown_connections 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/3689240771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 0x7f8cd010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/3689240771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 0x7f8cd0100ea0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/3689240771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.257+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/3689240771 >> 172.21.15.122:0/3689240771 conn(0x7f8cd0077b60 msgr2=0x7f8cd0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.258+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/3689240771 shutdown_connections 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.258+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/3689240771 wait complete. 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.258+0000 7f8cd6bbc640 1 Processor -- start 2024-04-02T13:50:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.258+0000 7f8cd6bbc640 1 -- start start 2024-04-02T13:50:00.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 0x7f8cd010e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 0x7f8cd010e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0111f70 con 0x7f8cd0106a30 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd0111df0 con 0x7f8cd0100a20 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd6bbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01120f0 con 0x7f8cd01013e0 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35022/0 (socket says 172.21.15.122:35022) 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 -- 172.21.15.122:0/771470478 learned_addr learned my addr 172.21.15.122:0/771470478 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8ccffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 0x7f8cd010e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 -- 172.21.15.122:0/771470478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 msgr2=0x7f8cd010e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 0x7f8cd010e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 -- 172.21.15.122:0/771470478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 msgr2=0x7f8cd010e080 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 0x7f8cd010e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.259+0000 7f8cd5159640 1 -- 172.21.15.122:0/771470478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd01a6910 con 0x7f8cd0106a30 2024-04-02T13:50:00.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.260+0000 7f8cd5159640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800eaa0 tx=0x7f8cc800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.260+0000 7f8ccdffb640 1 -- 172.21.15.122:0/771470478 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800cdb0 con 0x7f8cd0106a30 2024-04-02T13:50:00.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.261+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd01a6c00 con 0x7f8cd0106a30 2024-04-02T13:50:00.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.261+0000 7f8ccdffb640 1 -- 172.21.15.122:0/771470478 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8005020 con 0x7f8cd0106a30 2024-04-02T13:50:00.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.261+0000 7f8ccdffb640 1 -- 172.21.15.122:0/771470478 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80116d0 con 0x7f8cd0106a30 2024-04-02T13:50:00.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.261+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd01a7120 con 0x7f8cd0106a30 2024-04-02T13:50:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8ccdffb640 1 -- 172.21.15.122:0/771470478 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8cc8005540 con 0x7f8cd0106a30 2024-04-02T13:50:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 Processor -- start 2024-04-02T13:50:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 -- start start 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d420390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d428f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b98d37b7b0 con 0x55b98d4208d0 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b98d31a570 con 0x55b98d4294b0 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd979b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b98d377760 con 0x55b98d41ffb0 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d428f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d428f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35024/0 (socket says 172.21.15.122:35024) 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8cd4958640 1 -- 172.21.15.122:0/1979153015 learned_addr learned my addr 172.21.15.122:0/1979153015 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8ccffff640 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d420390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.265+0000 7f8ccffff640 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccdffb640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 0x7f8cc00a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 -- 172.21.15.122:0/1979153015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 msgr2=0x55b98d42b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8cd5159640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 0x7f8cc00a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 -- 172.21.15.122:0/1979153015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 msgr2=0x55b98d428f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d428f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 -- 172.21.15.122:0/1979153015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98d08f3f0 con 0x55b98d41ffb0 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccdffb640 1 -- 172.21.15.122:0/771470478 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f8cc80d2180 con 0x7f8cd0106a30 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8ccffff640 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d420390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009850 tx=0x7f8cbc005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f9c9a4e2661d0d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.266+0000 7f8cd5159640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 0x7f8cc00a62a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80d1000 tx=0x7f8cc801d000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.267+0000 7f8cbaffd640 1 -- 172.21.15.122:0/1979153015 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc049070 con 0x55b98d41ffb0 2024-04-02T13:50:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.267+0000 7f8cbaffd640 1 -- 172.21.15.122:0/1979153015 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc005050 con 0x55b98d41ffb0 2024-04-02T13:50:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.267+0000 7f8cbaffd640 1 -- 172.21.15.122:0/1979153015 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc004920 con 0x55b98d41ffb0 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- 172.21.15.122:0/1979153015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 msgr2=0x55b98d420390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d420390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009850 tx=0x7f8cbc005d50 comp rx=0 tx=0).stop 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- 172.21.15.122:0/1979153015 shutdown_connections 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d428f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 --2- 172.21.15.122:0/1979153015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d420390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- 172.21.15.122:0/1979153015 >> 172.21.15.122:0/1979153015 conn(0x55b98d41a6e0 msgr2=0x55b98d41aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- 172.21.15.122:0/1979153015 shutdown_connections 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- 172.21.15.122:0/1979153015 wait complete. 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 -- start start 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.268+0000 7f8cd979b740 1 Processor -- start 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d4263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b98d42c240 con 0x55b98d4294b0 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b98d336e80 con 0x55b98d4208d0 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b98d42c3c0 con 0x55b98d41ffb0 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd979b740 10 client.0 _ll_register_callbacks cb 0x55b98d372aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8ccffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50368/0 (socket says 172.21.15.122:50368) 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8ccffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35030/0 (socket says 172.21.15.122:35030) 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.269+0000 7f8cd4958640 1 -- 172.21.15.122:0/131288100 learned_addr learned my addr 172.21.15.122:0/131288100 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8ccffff640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d4263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 -- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 msgr2=0x55b98d4263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d4263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 -- 172.21.15.122:0/131288100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 msgr2=0x55b98d426e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 -- 172.21.15.122:0/131288100 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98d1597d0 con 0x55b98d4208d0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8ccffff640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8cd4958640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40116f0 tx=0x7f8cc4011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da61441ee60f753 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.270+0000 7f8caffff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc401e060 con 0x55b98d4208d0 2024-04-02T13:50:00.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cbc005050 con 0x55b98d4208d0 2024-04-02T13:50:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f8cc400fd30 con 0x55b98d4208d0 2024-04-02T13:50:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8cc4019430 con 0x55b98d4208d0 2024-04-02T13:50:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8cd979b740 10 client.28868 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b98d438480 con 0x55b98d4208d0 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.271+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cbc005050 con 0x55b98d4208d0 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f8cc4011ce0 con 0x55b98d4208d0 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 10 client.28868 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 10 client.28868 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 20 client.28868 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d43bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 4 client.28868 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98d388cf0 con 0x55b98d439950 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8ccffff640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d43bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8caffff640 10 client.28868 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.272+0000 7f8ccffff640 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d43bd70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a170a713d011f8ec server_cookie=d8eae9a0697818e3 in_seq=0 out_seq=0 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8caffff640 1 -- 172.21.15.122:0/131288100 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b98d388cf0 con 0x55b98d439950 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8caffff640 10 client.28868 handle_command_reply: tid=0 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8cd979b740 1 client.28868 shutdown 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 msgr2=0x55b98d426910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40116f0 tx=0x7f8cc4011bb0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 shutdown_connections 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d43bd70 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.274+0000 7f8caffff640 0 client.28868 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4294b0 0x55b98d426e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4208d0 0x55b98d426910 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 --2- 172.21.15.122:0/131288100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d4263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 >> 172.21.15.122:0/131288100 conn(0x55b98d41a6e0 msgr2=0x55b98d41c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 shutdown_connections 2024-04-02T13:50:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 1 -- 172.21.15.122:0/131288100 wait complete. 2024-04-02T13:50:00.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.275+0000 7f8cd979b740 20 client.28868 trim_cache size 0 max 16384 2024-04-02T13:50:00.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 Processor -- start 2024-04-02T13:50:00.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 -- start start 2024-04-02T13:50:00.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d428b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d410f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01acf40 con 0x55b98d4208d0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01aecc0 con 0x55b98d4294b0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd979b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01ad610 con 0x55b98d41ffb0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35034/0 (socket says 172.21.15.122:35034) 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 -- 172.21.15.122:0/741458299 learned_addr learned my addr 172.21.15.122:0/741458299 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8ccffff640 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d428b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8ccffff640 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d410f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 -- 172.21.15.122:0/741458299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 msgr2=0x55b98d428b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d428b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 -- 172.21.15.122:0/741458299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 msgr2=0x55b98d410f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d410f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.294+0000 7f8cd4958640 1 -- 172.21.15.122:0/741458299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98d420700 con 0x55b98d4208d0 2024-04-02T13:50:00.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.295+0000 7f8cd4958640 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400a010 tx=0x7f8cc400e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f09064176d1b1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.295+0000 7f8cba7fc640 1 -- 172.21.15.122:0/741458299 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4028080 con 0x55b98d4208d0 2024-04-02T13:50:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.295+0000 7f8cba7fc640 1 -- 172.21.15.122:0/741458299 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4005b90 con 0x55b98d4208d0 2024-04-02T13:50:00.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.295+0000 7f8cba7fc640 1 -- 172.21.15.122:0/741458299 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4019a70 con 0x55b98d4208d0 2024-04-02T13:50:00.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- 172.21.15.122:0/741458299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 msgr2=0x55b98d435100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400a010 tx=0x7f8cc400e5e0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- 172.21.15.122:0/741458299 shutdown_connections 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d410f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d4208d0 0x55b98d435100 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- 172.21.15.122:0/741458299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d41ffb0 0x55b98d428b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- 172.21.15.122:0/741458299 >> 172.21.15.122:0/741458299 conn(0x55b98d41a6e0 msgr2=0x55b98d41aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- 172.21.15.122:0/741458299 shutdown_connections 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- 172.21.15.122:0/741458299 wait complete. 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 -- start start 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 Processor -- start 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.296+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d4208d0 0x55b98d42cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01ae530 con 0x55b98d41ffb0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01adda0 con 0x55b98d4294b0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd01aaf40 con 0x55b98d4208d0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 10 client.0 _ll_register_callbacks cb 0x7f8cc4009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35048/0 (socket says 172.21.15.122:35048) 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 -- 172.21.15.122:0/712109396 learned_addr learned my addr 172.21.15.122:0/712109396 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd4958640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d4208d0 0x55b98d42cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 -- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d4208d0 msgr2=0x55b98d42cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d4208d0 0x55b98d42cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 -- 172.21.15.122:0/712109396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 msgr2=0x55b98d42d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 -- 172.21.15.122:0/712109396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98d49e190 con 0x55b98d41ffb0 2024-04-02T13:50:00.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8ccffff640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005ab0 tx=0x7f8cc4009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e36ed21576da2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8caffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cbc049040 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f8cbc0562c0 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.297+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc4005b90 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8cbc0564c0 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 10 client.19887 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b98d49f4f0 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc4005b90 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f8cbc04f040 con 0x55b98d41ffb0 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 10 client.19887 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 10 client.19887 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 20 client.19887 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d4a1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 4 client.19887 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b98d383040 con 0x55b98d439950 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.298+0000 7f8cd4958640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d4a1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.299+0000 7f8caffff640 10 client.19887 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.299+0000 7f8cd4958640 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d4a1cd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff534a26ed093c4f server_cookie=ed51334526b8e682 in_seq=0 out_seq=0 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.299+0000 7f8caffff640 1 -- 172.21.15.122:0/712109396 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b98d383040 con 0x55b98d439950 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.299+0000 7f8caffff640 10 client.19887 handle_command_reply: tid=0 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 client.19887 shutdown 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 msgr2=0x55b98d42c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc005ab0 tx=0x7f8cc4009c90 comp rx=0 tx=0).stop 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 shutdown_connections 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55b98d439950 0x55b98d4a1cd0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8caffff640 0 client.19887 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b98d4294b0 0x55b98d42d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b98d4208d0 0x55b98d42cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 --2- 172.21.15.122:0/712109396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b98d41ffb0 0x55b98d42c5d0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 >> 172.21.15.122:0/712109396 conn(0x55b98d41a6e0 msgr2=0x55b98d425400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 shutdown_connections 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 1 -- 172.21.15.122:0/712109396 wait complete. 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.300+0000 7f8cd979b740 20 client.19887 trim_cache size 0 max 16384 2024-04-02T13:50:00.307 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 msgr2=0x7f8cc00a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 0x7f8cc00a62a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80d1000 tx=0x7f8cc801d000 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 msgr2=0x7f8cd010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800eaa0 tx=0x7f8cc800ef60 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 shutdown_connections 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8cc00a3de0 0x7f8cc00a62a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd0106a30 0x7f8cd010eb00 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd01013e0 0x7f8cd010e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 --2- 172.21.15.122:0/771470478 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cd0100a20 0x7f8cd010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 >> 172.21.15.122:0/771470478 conn(0x7f8cd0077b60 msgr2=0x7f8cd01035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.301+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 shutdown_connections 2024-04-02T13:50:00.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.302+0000 7f8cd6bbc640 1 -- 172.21.15.122:0/771470478 wait complete. 2024-04-02T13:50:00.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:50:00.322 INFO:tasks.workunit.client.0.smithi122.stdout: 10 42347 30.60 MB/sec warmup 80 sec latency 108.905 ms 2024-04-02T13:50:00.353 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:00 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3629589641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:00.353 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:00 smithi122 ceph-mon[32114]: overall HEALTH_OK 2024-04-02T13:50:00.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:00 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3629589641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:00.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:00 smithi187 ceph-mon[36064]: overall HEALTH_OK 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 -- 172.21.15.122:0/519477910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e781013e0 msgr2=0x7f0e7810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 --2- 172.21.15.122:0/519477910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e781013e0 0x7f0e7810d8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009cd0 tx=0x7f0e7002c100 comp rx=0 tx=0).stop 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 -- 172.21.15.122:0/519477910 shutdown_connections 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 --2- 172.21.15.122:0/519477910 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e781013e0 0x7f0e7810d8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 --2- 172.21.15.122:0/519477910 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e78100a20 0x7f0e78100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 --2- 172.21.15.122:0/519477910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78106a30 0x7f0e78106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 -- 172.21.15.122:0/519477910 >> 172.21.15.122:0/519477910 conn(0x7f0e78077b60 msgr2=0x7f0e78077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.484+0000 7f0e8022c640 1 -- 172.21.15.122:0/519477910 shutdown_connections 2024-04-02T13:50:00.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.485+0000 7f0e8022c640 1 -- 172.21.15.122:0/519477910 wait complete. 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.485+0000 7f0e8022c640 1 Processor -- start 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.485+0000 7f0e8022c640 1 -- start start 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 0x7f0e7806bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e78106a30 0x7f0e78072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7806b010 con 0x7f0e78100a20 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e7dfc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7806ae90 con 0x7f0e781013e0 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e8022c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7810ff80 con 0x7f0e78106a30 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e7dfc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35074/0 (socket says 172.21.15.122:35074) 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e7dfc8640 1 -- 172.21.15.122:0/3872229035 learned_addr learned my addr 172.21.15.122:0/3872229035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e78106a30 0x7f0e78072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.486+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 0x7f0e7806bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 -- 172.21.15.122:0/3872229035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e78106a30 msgr2=0x7f0e78072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e78106a30 0x7f0e78072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 -- 172.21.15.122:0/3872229035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 msgr2=0x7f0e7806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 0x7f0e7806bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 -- 172.21.15.122:0/3872229035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e78072bf0 con 0x7f0e78100a20 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 0x7f0e7806bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:00.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b4a0 tx=0x7f0e7400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e6effd640 1 -- 172.21.15.122:0/3872229035 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74014070 con 0x7f0e78100a20 2024-04-02T13:50:00.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e6effd640 1 -- 172.21.15.122:0/3872229035 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e74010af0 con 0x7f0e78100a20 2024-04-02T13:50:00.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e78072ee0 con 0x7f0e78100a20 2024-04-02T13:50:00.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e6effd640 1 -- 172.21.15.122:0/3872229035 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74017d10 con 0x7f0e78100a20 2024-04-02T13:50:00.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.487+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e78073480 con 0x7f0e78100a20 2024-04-02T13:50:00.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.490+0000 7f0e82e0f740 1 Processor -- start 2024-04-02T13:50:00.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 -- start start 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cafb757b0 con 0x562cafc1aca0 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562cafb14570 con 0x562cafc120c0 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562cafb71760 con 0x562cafc117a0 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e6effd640 1 -- 172.21.15.122:0/3872229035 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0e74017070 con 0x7f0e78100a20 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e7d7c7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e7e7c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e7e7c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33426/0 (socket says 172.21.15.122:33426) 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/694915162 learned_addr learned my addr 172.21.15.122:0/694915162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.491+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/694915162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 msgr2=0x562cafc1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/694915162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 msgr2=0x562cafc1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/694915162 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562caf8893f0 con 0x562cafc117a0 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009cd0 tx=0x7f0e70032450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f02393ff9f2aabc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e6effd640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 0x7f0e640a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e63fff640 1 -- 172.21.15.122:0/694915162 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70043070 con 0x562cafc117a0 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e63fff640 1 -- 172.21.15.122:0/694915162 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70003970 con 0x562cafc117a0 2024-04-02T13:50:00.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e63fff640 1 -- 172.21.15.122:0/694915162 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70039040 con 0x562cafc117a0 2024-04-02T13:50:00.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.492+0000 7f0e6effd640 1 -- 172.21.15.122:0/3872229035 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0e740cfde0 con 0x7f0e78100a20 2024-04-02T13:50:00.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 0x7f0e640a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e7dfc8640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 0x7f0e640a6310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x562cafc11270 tx=0x7f0e740047d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e82e0f740 1 -- 172.21.15.122:0/694915162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 msgr2=0x562cafc11b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009cd0 tx=0x7f0e70032450 comp rx=0 tx=0).stop 2024-04-02T13:50:00.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e82e0f740 1 -- 172.21.15.122:0/694915162 shutdown_connections 2024-04-02T13:50:00.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.493+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/694915162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc11b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 -- 172.21.15.122:0/694915162 >> 172.21.15.122:0/694915162 conn(0x562cafc0bed0 msgr2=0x562cafc0c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 -- 172.21.15.122:0/694915162 shutdown_connections 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 -- 172.21.15.122:0/694915162 wait complete. 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 -- start start 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.494+0000 7f0e82e0f740 1 Processor -- start 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc18050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cafc1da30 con 0x562cafc1aca0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562cafb30e80 con 0x562cafc120c0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562cafc1dbb0 con 0x562cafc117a0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e82e0f740 10 client.0 _ll_register_callbacks cb 0x562cafb6caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e7d7c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e7d7c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50422/0 (socket says 172.21.15.122:50422) 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.495+0000 7f0e7d7c7640 1 -- 172.21.15.122:0/1223131882 learned_addr learned my addr 172.21.15.122:0/1223131882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 -- 172.21.15.122:0/1223131882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 msgr2=0x562cafc1a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 -- 172.21.15.122:0/1223131882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 msgr2=0x562cafc18050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc18050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 -- 172.21.15.122:0/1223131882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562caf9537d0 con 0x562cafc120c0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800d410 tx=0x7f0e6800d8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b67e18cd4742096 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.496+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801c060 con 0x562cafc120c0 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e70003970 con 0x562cafc120c0 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0e68015750 con 0x562cafc120c0 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0e68023050 con 0x562cafc120c0 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e82e0f740 10 client.28883 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562cafc29b40 con 0x562cafc120c0 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e68017480 con 0x562cafc120c0 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f0e680163e0 con 0x562cafc120c0 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.497+0000 7f0e82e0f740 10 client.28883 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e82e0f740 10 client.28883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e82e0f740 20 client.28883 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc2d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e82e0f740 4 client.28883 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e70003970 con 0x562cafc2afc0 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc2d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e617fa640 10 client.28883 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:00.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.498+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc2d3e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ab45752a0d99ae server_cookie=a1c7c6fdfe96328f in_seq=0 out_seq=0 2024-04-02T13:50:00.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.502+0000 7f0e617fa640 1 -- 172.21.15.122:0/1223131882 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f0e70003970 con 0x562cafc2afc0 2024-04-02T13:50:00.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.502+0000 7f0e617fa640 10 client.28883 handle_command_reply: tid=0 2024-04-02T13:50:00.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.502+0000 7f0e82e0f740 1 client.28883 shutdown 2024-04-02T13:50:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 msgr2=0x562cafc17a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800d410 tx=0x7f0e6800d8d0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 shutdown_connections 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc2d3e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc18050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e617fa640 0 client.28883 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc17a40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/1223131882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 >> 172.21.15.122:0/1223131882 conn(0x562cafc0bed0 msgr2=0x562cafc0dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 shutdown_connections 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 1 -- 172.21.15.122:0/1223131882 wait complete. 2024-04-02T13:50:00.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.503+0000 7f0e82e0f740 20 client.28883 trim_cache size 0 max 16384 2024-04-02T13:50:00.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:00 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3629589641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:00.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:00 smithi188 ceph-mon[36117]: overall HEALTH_OK 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 Processor -- start 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 -- start start 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc26920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b9b40 con 0x562cafc1aca0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781bb8c0 con 0x562cafc120c0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781ba210 con 0x562cafc117a0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e7e7c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.520+0000 7f0e7d7c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7d7c7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc179e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50424/0 (socket says 172.21.15.122:50424) 2024-04-02T13:50:00.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7d7c7640 1 -- 172.21.15.122:0/496326664 learned_addr learned my addr 172.21.15.122:0/496326664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc26920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/496326664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 msgr2=0x562cafc26920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc26920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/496326664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 msgr2=0x562cafc179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/496326664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cafc183c0 con 0x562cafc1aca0 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.521+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1a170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009ca0 tx=0x7f0e700099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625efd688b2366db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.522+0000 7f0e637fe640 1 -- 172.21.15.122:0/496326664 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70039070 con 0x562cafc1aca0 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.522+0000 7f0e637fe640 1 -- 172.21.15.122:0/496326664 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e70032e70 con 0x562cafc1aca0 2024-04-02T13:50:00.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.522+0000 7f0e637fe640 1 -- 172.21.15.122:0/496326664 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70043040 con 0x562cafc1aca0 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- 172.21.15.122:0/496326664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 msgr2=0x562cafc1a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1a170 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009ca0 tx=0x7f0e700099b0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- 172.21.15.122:0/496326664 shutdown_connections 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc1aca0 0x562cafc1a170 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/496326664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc117a0 0x562cafc26920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- 172.21.15.122:0/496326664 >> 172.21.15.122:0/496326664 conn(0x562cafc0bed0 msgr2=0x562cafc172d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- 172.21.15.122:0/496326664 shutdown_connections 2024-04-02T13:50:00.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- 172.21.15.122:0/496326664 wait complete. 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 -- start start 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.523+0000 7f0e82e0f740 1 Processor -- start 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc1aca0 0x562cafc15200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781bb130 con 0x562cafc117a0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e7e7c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e7e7c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35112/0 (socket says 172.21.15.122:35112) 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/188290058 learned_addr learned my addr 172.21.15.122:0/188290058 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781ba9a0 con 0x562cafc120c0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e781b7b40 con 0x562cafc1aca0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc1aca0 0x562cafc15200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.524+0000 7f0e82e0f740 10 client.0 _ll_register_callbacks cb 0x7f0e6800b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/188290058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc1aca0 msgr2=0x562cafc15200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc1aca0 0x562cafc15200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 msgr2=0x562cafc14cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc14cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cafc11ef0 con 0x562cafc117a0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e7e7c9640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f0e7002f020 tx=0x7f0e700093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c9ba231090729d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70039030 con 0x562cafc117a0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0e70032590 con 0x562cafc117a0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.525+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562cafb82cf0 con 0x562cafc117a0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0e70043070 con 0x562cafc117a0 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 10 client.19899 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:00.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x562cafc1c350 con 0x562cafc117a0 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562cafb82cf0 con 0x562cafc117a0 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f0e70005040 con 0x562cafc117a0 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 10 client.19899 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 10 client.19899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 20 client.19899 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.526+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc9bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.527+0000 7f0e82e0f740 4 client.19899 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.527+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cafb7d040 con 0x562cafc2afc0 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.527+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc9bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:00.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.528+0000 7f0e7d7c7640 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc9bd90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc3ae8032df0176 server_cookie=98b90404d28c20a5 in_seq=0 out_seq=0 2024-04-02T13:50:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.528+0000 7f0e617fa640 10 client.19899 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e617fa640 1 -- 172.21.15.122:0/188290058 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cafb7d040 con 0x562cafc2afc0 2024-04-02T13:50:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e617fa640 10 client.19899 handle_command_reply: tid=0 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 client.19899 shutdown 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 msgr2=0x562cafc230d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f0e7002f020 tx=0x7f0e700093f0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 shutdown_connections 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562cafc2afc0 0x562cafc9bd90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cafc1aca0 0x562cafc15200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cafc120c0 0x562cafc14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 --2- 172.21.15.122:0/188290058 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cafc117a0 0x562cafc230d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 >> 172.21.15.122:0/188290058 conn(0x562cafc0bed0 msgr2=0x562cafc0dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e617fa640 0 client.19899 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 shutdown_connections 2024-04-02T13:50:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.531+0000 7f0e82e0f740 1 -- 172.21.15.122:0/188290058 wait complete. 2024-04-02T13:50:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.532+0000 7f0e82e0f740 20 client.19899 trim_cache size 0 max 16384 2024-04-02T13:50:00.535 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.532+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 msgr2=0x7f0e640a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.532+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 0x7f0e640a6310 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x562cafc11270 tx=0x7f0e740047d0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.532+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 msgr2=0x7f0e7806b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.532+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400b4a0 tx=0x7f0e7400b960 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 shutdown_connections 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0e640a3e50 0x7f0e640a6310 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e78106a30 0x7f0e78072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e781013e0 0x7f0e7806bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 --2- 172.21.15.122:0/3872229035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e78100a20 0x7f0e7806b700 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 >> 172.21.15.122:0/3872229035 conn(0x7f0e78077b60 msgr2=0x7f0e78104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 shutdown_connections 2024-04-02T13:50:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:00.533+0000 7f0e8022c640 1 -- 172.21.15.122:0/3872229035 wait complete. 2024-04-02T13:50:01.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 42886 30.58 MB/sec warmup 81 sec latency 364.936 ms 2024-04-02T13:50:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:01 smithi187 ceph-mon[36064]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 529 op/s 2024-04-02T13:50:01.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:01 smithi188 ceph-mon[36117]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 529 op/s 2024-04-02T13:50:01.552 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:50:01.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:50:01.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:50:01.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:01 smithi122 ceph-mon[32114]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.7 MiB/s wr, 529 op/s 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.723+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/605391481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 msgr2=0x7f0ba0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.723+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/605391481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0100df0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009930 tx=0x7f0b9002bc50 comp rx=0 tx=0).stop 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/605391481 shutdown_connections 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/605391481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0101330 0x7f0ba010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/605391481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0100df0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/605391481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0106980 0x7f0ba0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/605391481 >> 172.21.15.122:0/605391481 conn(0x7f0ba0077b00 msgr2=0x7f0ba0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/605391481 shutdown_connections 2024-04-02T13:50:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.724+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/605391481 wait complete. 2024-04-02T13:50:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.725+0000 7f0ba6e7b640 1 Processor -- start 2024-04-02T13:50:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 -- start start 2024-04-02T13:50:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0116420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0101330 0x7f0ba01104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010ee40 con 0x7f0ba0100970 2024-04-02T13:50:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010ecc0 con 0x7f0ba0106980 2024-04-02T13:50:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba6e7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba010efc0 con 0x7f0ba0101330 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba5418640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba5418640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50448/0 (socket says 172.21.15.122:50448) 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba5418640 1 -- 172.21.15.122:0/595082171 learned_addr learned my addr 172.21.15.122:0/595082171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.726+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0116420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0101330 0x7f0ba01104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 -- 172.21.15.122:0/595082171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0101330 msgr2=0x7f0ba01104e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0101330 0x7f0ba01104e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 -- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 msgr2=0x7f0ba0116420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0116420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 -- 172.21.15.122:0/595082171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01111f0 con 0x7f0ba0106980 2024-04-02T13:50:01.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.727+0000 7f0ba5418640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400d750 tx=0x7f0b9400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.728+0000 7f0b9dffb640 1 -- 172.21.15.122:0/595082171 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94017070 con 0x7f0ba0106980 2024-04-02T13:50:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.728+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01b37e0 con 0x7f0ba0106980 2024-04-02T13:50:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.728+0000 7f0b9dffb640 1 -- 172.21.15.122:0/595082171 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b94005500 con 0x7f0ba0106980 2024-04-02T13:50:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.728+0000 7f0b9dffb640 1 -- 172.21.15.122:0/595082171 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004d90 con 0x7f0ba0106980 2024-04-02T13:50:01.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.728+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01b3d00 con 0x7f0ba0106980 2024-04-02T13:50:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.732+0000 7f0ba9a66740 1 Processor -- start 2024-04-02T13:50:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.732+0000 7f0b9dffb640 1 -- 172.21.15.122:0/595082171 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0b94004f30 con 0x7f0ba0106980 2024-04-02T13:50:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0b9dffb640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 0x7f0b880a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 -- start start 2024-04-02T13:50:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0b9dffb640 1 -- 172.21.15.122:0/595082171 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0b940d0530 con 0x7f0ba0106980 2024-04-02T13:50:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3258fbbe0 0x55e325904280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3259047c0 0x55e325906bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e32585f770 con 0x55e3258fb2c0 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e325807960 con 0x55e3259047c0 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e32585b8c0 con 0x55e3258fbbe0 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35144/0 (socket says 172.21.15.122:35144) 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba4c17640 1 -- 172.21.15.122:0/4105568843 learned_addr learned my addr 172.21.15.122:0/4105568843 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 0x7f0b880a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3258fbbe0 0x55e325904280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.733+0000 7f0ba5418640 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3259047c0 0x55e325906bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 -- 172.21.15.122:0/4105568843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3258fbbe0 msgr2=0x55e325904280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3258fbbe0 0x55e325904280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 -- 172.21.15.122:0/4105568843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3259047c0 msgr2=0x55e325906bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3259047c0 0x55e325906bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 -- 172.21.15.122:0/4105568843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32563b7e0 con 0x55e3258fb2c0 2024-04-02T13:50:01.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fd0 tx=0x7f0b980118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d695587e738b7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 0x7f0b880a6320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002c5d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0b86ffd640 1 -- 172.21.15.122:0/4105568843 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9801e070 con 0x55e3258fb2c0 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0b86ffd640 1 -- 172.21.15.122:0/4105568843 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b98003c30 con 0x55e3258fb2c0 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.734+0000 7f0b86ffd640 1 -- 172.21.15.122:0/4105568843 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800f940 con 0x55e3258fb2c0 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 -- 172.21.15.122:0/4105568843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 msgr2=0x55e3258fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009fd0 tx=0x7f0b980118d0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 -- 172.21.15.122:0/4105568843 shutdown_connections 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3259047c0 0x55e325906bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3258fbbe0 0x55e325904280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/4105568843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fb6a0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 -- 172.21.15.122:0/4105568843 >> 172.21.15.122:0/4105568843 conn(0x55e3258f59f0 msgr2=0x55e3258f5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 -- 172.21.15.122:0/4105568843 shutdown_connections 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.735+0000 7f0ba9a66740 1 -- 172.21.15.122:0/4105568843 wait complete. 2024-04-02T13:50:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 -- start start 2024-04-02T13:50:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 Processor -- start 2024-04-02T13:50:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e325901d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e325902240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e3259075b0 con 0x55e3258fb2c0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e32581ad20 con 0x55e3258fbbe0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.736+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e325907730 con 0x55e3259047c0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba9a66740 10 client.0 _ll_register_callbacks cb 0x55e3257e7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35146/0 (socket says 172.21.15.122:35146) 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3883451427 learned_addr learned my addr 172.21.15.122:0/3883451427 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba5418640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e325902240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e325901d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3883451427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 msgr2=0x55e325902240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e325902240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 msgr2=0x55e325901d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e325901d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.737+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3883451427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32569bf70 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800f520 tx=0x7f0b98004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c260a41ca3ed5d9c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0b7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9801e050 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0b98004460 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b98003c30 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.738+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0b98032050 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 10 client.19911 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e325533170 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b98003c30 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f0b9802d030 con 0x55e3258fb2c0 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 10 client.19911 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 10 client.19911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 20 client.19911 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e325925e10 0x55e325928290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 4 client.19911 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:01.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e32586ccb0 con 0x55e325925e10 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.739+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e325925e10 0x55e325928290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.740+0000 7f0b7ffff640 10 client.19911 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.740+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e325925e10 0x55e325928290 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e583cb33c917d4f2 server_cookie=c9038af46b24a6db in_seq=0 out_seq=0 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3883451427 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e32586ccb0 con 0x55e325925e10 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0b7ffff640 10 client.19911 handle_command_reply: tid=0 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 client.19911 shutdown 2024-04-02T13:50:01.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 msgr2=0x55e3259017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800f520 tx=0x7f0b98004f60 comp rx=0 tx=0).stop 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 shutdown_connections 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e325925e10 0x55e325928290 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e325902240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0b7ffff640 0 client.19911 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e325901d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3883451427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3259017c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 >> 172.21.15.122:0/3883451427 conn(0x55e3258f59f0 msgr2=0x55e3258f7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 shutdown_connections 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3883451427 wait complete. 2024-04-02T13:50:01.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.741+0000 7f0ba9a66740 20 client.19911 trim_cache size 0 max 16384 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.771+0000 7f0ba9a66740 1 Processor -- start 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 -- start start 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fb2c0 0x55e3259104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e32592db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b5670 con 0x55e3258fbbe0 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b73f0 con 0x55e3258fb2c0 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b5d40 con 0x55e3259047c0 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35150/0 (socket says 172.21.15.122:35150) 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 -- 172.21.15.122:0/2997087858 learned_addr learned my addr 172.21.15.122:0/2997087858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:01.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fb2c0 0x55e3259104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0ba5418640 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e32592db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 -- 172.21.15.122:0/2997087858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 msgr2=0x55e32592db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e32592db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 -- 172.21.15.122:0/2997087858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fb2c0 msgr2=0x55e3259104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.772+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fb2c0 0x55e3259104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.773+0000 7f0b9ffff640 1 -- 172.21.15.122:0/2997087858 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3258f4900 con 0x55e3258fbbe0 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.773+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b90049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b4dfc8becf01a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.773+0000 7f0b867fc640 1 -- 172.21.15.122:0/2997087858 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90049070 con 0x55e3258fbbe0 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.773+0000 7f0b867fc640 1 -- 172.21.15.122:0/2997087858 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90059070 con 0x55e3258fbbe0 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.773+0000 7f0b867fc640 1 -- 172.21.15.122:0/2997087858 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004920 con 0x55e3258fbbe0 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 -- 172.21.15.122:0/2997087858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 msgr2=0x55e3259010c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0b90000c00 tx=0x7f0b90049000 comp rx=0 tx=0).stop 2024-04-02T13:50:01.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 -- 172.21.15.122:0/2997087858 shutdown_connections 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e32592db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fbbe0 0x55e3259010c0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/2997087858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fb2c0 0x55e3259104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.774+0000 7f0ba9a66740 1 -- 172.21.15.122:0/2997087858 >> 172.21.15.122:0/2997087858 conn(0x55e3258f59f0 msgr2=0x55e3258f74b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.775+0000 7f0ba9a66740 1 -- 172.21.15.122:0/2997087858 shutdown_connections 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.775+0000 7f0ba9a66740 1 -- 172.21.15.122:0/2997087858 wait complete. 2024-04-02T13:50:01.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.775+0000 7f0ba9a66740 1 -- start start 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.775+0000 7f0ba9a66740 1 Processor -- start 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e3258feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e3258ff030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b6c60 con 0x55e3258fb2c0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b64d0 con 0x55e3258fbbe0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0073460 con 0x55e3259047c0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba9a66740 10 client.0 _ll_register_callbacks cb 0x7f0b9801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba5418640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e3258ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba4c17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35164/0 (socket says 172.21.15.122:35164) 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3280874201 learned_addr learned my addr 172.21.15.122:0/3280874201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.776+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e3258feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3280874201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 msgr2=0x55e3258ff030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e3258ff030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 msgr2=0x55e3258feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e3258feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 -- 172.21.15.122:0/3280874201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32598b530 con 0x55e3258fb2c0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba4c17640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f0b98004fd0 tx=0x7f0b98005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e43d4c5957e4af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0b7ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9801e040 con 0x55e3258fb2c0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f0b98003970 con 0x55e3258fb2c0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b90059070 con 0x55e3258fb2c0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.777+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f0b98043050 con 0x55e3258fb2c0 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 10 client.19917 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:01.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55e32598c7f0 con 0x55e3258fb2c0 2024-04-02T13:50:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90059070 con 0x55e3258fb2c0 2024-04-02T13:50:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f0b9803e030 con 0x55e3258fb2c0 2024-04-02T13:50:01.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 10 client.19917 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 10 client.19917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 20 client.19917 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e32598d010 0x55e32598f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 4 client.19917 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.778+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e325867000 con 0x55e32598d010 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.779+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e32598d010 0x55e32598f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.779+0000 7f0b9ffff640 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e32598d010 0x55e32598f430 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed579e2e717358e server_cookie=c59e0a357eebaf29 in_seq=0 out_seq=0 2024-04-02T13:50:01.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.779+0000 7f0b7ffff640 10 client.19917 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.780+0000 7f0b7ffff640 1 -- 172.21.15.122:0/3280874201 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e325867000 con 0x55e32598d010 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.780+0000 7f0b7ffff640 10 client.19917 handle_command_reply: tid=0 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.780+0000 7f0ba9a66740 1 client.19917 shutdown 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 msgr2=0x55e3258fe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f0b98004fd0 tx=0x7f0b98005070 comp rx=0 tx=0).stop 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 shutdown_connections 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e32598d010 0x55e32598f430 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e3259047c0 0x55e3258ff030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e3258fbbe0 0x55e3258feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 --2- 172.21.15.122:0/3280874201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e3258fb2c0 0x55e3258fe5b0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 >> 172.21.15.122:0/3280874201 conn(0x55e3258f59f0 msgr2=0x55e3258f69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0b7ffff640 0 client.19917 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 shutdown_connections 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 1 -- 172.21.15.122:0/3280874201 wait complete. 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.781+0000 7f0ba9a66740 20 client.19917 trim_cache size 0 max 16384 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:50:01.785 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "105afd33-67ae-48e7-9ade-44ee7d012cad", 2024-04-02T13:50:01.786 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:50:01.786 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.782+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 msgr2=0x7f0b880a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.782+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 0x7f0b880a6320 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009900 tx=0x7f0b9002c5d0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.782+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 msgr2=0x7f0ba0110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.782+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400d750 tx=0x7f0b9400dc10 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 shutdown_connections 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b880a3e60 0x7f0b880a6320 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ba0106980 0x7f0ba0110a20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ba0101330 0x7f0ba01104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 --2- 172.21.15.122:0/595082171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ba0100970 0x7f0ba0116420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.783+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 >> 172.21.15.122:0/595082171 conn(0x7f0ba0077b00 msgr2=0x7f0ba0103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:01.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.784+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 shutdown_connections 2024-04-02T13:50:01.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:01.784+0000 7f0ba6e7b640 1 -- 172.21.15.122:0/595082171 wait complete. 2024-04-02T13:50:01.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:50:01.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:50:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 -- 172.21.15.122:0/1658017747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 msgr2=0x7f3418104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 --2- 172.21.15.122:0/1658017747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 0x7f3418104a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3410009c10 tx=0x7f341002bef0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 -- 172.21.15.122:0/1658017747 shutdown_connections 2024-04-02T13:50:02.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 --2- 172.21.15.122:0/1658017747 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 0x7f3418104a40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 --2- 172.21.15.122:0/1658017747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34180696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 --2- 172.21.15.122:0/1658017747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3418106a50 0x7f3418068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.019+0000 7f341ed15640 1 -- 172.21.15.122:0/1658017747 >> 172.21.15.122:0/1658017747 conn(0x7f3418077b00 msgr2=0x7f3418077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.020+0000 7f341ed15640 1 -- 172.21.15.122:0/1658017747 shutdown_connections 2024-04-02T13:50:02.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.020+0000 7f341ed15640 1 -- 172.21.15.122:0/1658017747 wait complete. 2024-04-02T13:50:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 Processor -- start 2024-04-02T13:50:02.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 -- start start 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 0x7f34181a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3418106a50 0x7f341819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3418118480 con 0x7f3418069240 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3418118300 con 0x7f34181045c0 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341ed15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3418118600 con 0x7f3418106a50 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35190/0 (socket says 172.21.15.122:35190) 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.021+0000 7f341cab1640 1 -- 172.21.15.122:0/3040423330 learned_addr learned my addr 172.21.15.122:0/3040423330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 -- 172.21.15.122:0/3040423330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3418106a50 msgr2=0x7f341819e780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3418106a50 0x7f341819e780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 -- 172.21.15.122:0/3040423330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 msgr2=0x7f34181a4bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 0x7f34181a4bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 -- 172.21.15.122:0/3040423330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341819ef60 con 0x7f3418069240 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f341cab1640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f340c00d980 tx=0x7f340c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f3415ffb640 1 -- 172.21.15.122:0/3040423330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c017070 con 0x7f3418069240 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.022+0000 7f3415ffb640 1 -- 172.21.15.122:0/3040423330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340c009c30 con 0x7f3418069240 2024-04-02T13:50:02.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.023+0000 7f3415ffb640 1 -- 172.21.15.122:0/3040423330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c00f620 con 0x7f3418069240 2024-04-02T13:50:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.023+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341819f250 con 0x7f3418069240 2024-04-02T13:50:02.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.023+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3418106f10 con 0x7f3418069240 2024-04-02T13:50:02.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.026+0000 7f3415ffb640 1 -- 172.21.15.122:0/3040423330 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f340c010460 con 0x7f3418069240 2024-04-02T13:50:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.027+0000 7f3415ffb640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 0x7f34080a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.027+0000 7f34219d8740 1 Processor -- start 2024-04-02T13:50:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.027+0000 7f3415ffb640 1 -- 172.21.15.122:0/3040423330 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f340c0d08d0 con 0x7f3418069240 2024-04-02T13:50:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.027+0000 7f3417fff640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 0x7f34080a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 -- start start 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce063ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0645890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dce05a0830 con 0x55dce063c8d0 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dce0548820 con 0x55dce063d1f0 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f34219d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dce059c7e0 con 0x55dce0645dd0 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f341d2b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f341d2b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:33520/0 (socket says 172.21.15.122:33520) 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce063ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f341d2b2640 1 -- 172.21.15.122:0/2948420000 learned_addr learned my addr 172.21.15.122:0/2948420000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:02.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.028+0000 7f3417fff640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0645890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 -- 172.21.15.122:0/2948420000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 msgr2=0x55dce06481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 -- 172.21.15.122:0/2948420000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 msgr2=0x55dce0645890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0645890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 -- 172.21.15.122:0/2948420000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce037e7d0 con 0x55dce063c8d0 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341d2b2640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f3417fff640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0645890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f3417fff640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 0x7f34080a6290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f34040058c0 tx=0x7f3404005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f341cab1640 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce063ccb0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f340c0cdbd0 tx=0x7f340c09c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193d816a450143d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f3402ffd640 1 -- 172.21.15.122:0/2948420000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c0b5070 con 0x55dce063c8d0 2024-04-02T13:50:02.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.029+0000 7f3402ffd640 1 -- 172.21.15.122:0/2948420000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340c09a720 con 0x55dce063c8d0 2024-04-02T13:50:02.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.030+0000 7f3402ffd640 1 -- 172.21.15.122:0/2948420000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c020070 con 0x55dce063c8d0 2024-04-02T13:50:02.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.030+0000 7f34219d8740 1 -- 172.21.15.122:0/2948420000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 msgr2=0x55dce063ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce063ccb0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f340c0cdbd0 tx=0x7f340c09c420 comp rx=0 tx=0).stop 2024-04-02T13:50:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 -- 172.21.15.122:0/2948420000 shutdown_connections 2024-04-02T13:50:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce06481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0645890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 --2- 172.21.15.122:0/2948420000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce063ccb0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.031+0000 7f34219d8740 1 -- 172.21.15.122:0/2948420000 >> 172.21.15.122:0/2948420000 conn(0x55dce0637000 msgr2=0x55dce06373e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.032+0000 7f34219d8740 1 -- 172.21.15.122:0/2948420000 shutdown_connections 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.032+0000 7f34219d8740 1 -- 172.21.15.122:0/2948420000 wait complete. 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.032+0000 7f34219d8740 1 -- start start 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.032+0000 7f34219d8740 1 Processor -- start 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0642a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce0642fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dce0648c60 con 0x55dce063c8d0 2024-04-02T13:50:02.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35210/0 (socket says 172.21.15.122:35210) 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f341cab1640 1 -- 172.21.15.122:0/2193519892 learned_addr learned my addr 172.21.15.122:0/2193519892 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dce055bd30 con 0x55dce063d1f0 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dce0648de0 con 0x55dce0645dd0 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.033+0000 7f34219d8740 10 client.0 _ll_register_callbacks cb 0x55dce055b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341d2b2640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce0642fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f3417fff640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0642a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 -- 172.21.15.122:0/2193519892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 msgr2=0x55dce0642fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce0642fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 -- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 msgr2=0x55dce0642a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0642a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce02b43f0 con 0x55dce063c8d0 2024-04-02T13:50:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f341cab1640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f340c098df0 tx=0x7f340c09c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4b07ecfebeee21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.034+0000 7f33f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c020070 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f340c0d0400 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f340c09a720 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f340c0b5070 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f34219d8740 10 client.19929 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.036+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dce02740f0 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f340c09a720 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f340c09b870 con 0x55dce063c8d0 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 10 client.19929 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 10 client.19929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 20 client.19929 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:02.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce0655d00 0x55dce06580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 4 client.19929 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.037+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dce05add70 con 0x55dce0655d00 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.038+0000 7f3417fff640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce0655d00 0x55dce06580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.039+0000 7f33f7fff640 10 client.19929 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.039+0000 7f3417fff640 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce0655d00 0x55dce06580f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c7fab82d1c4fc8 server_cookie=c6f9940b3ae9c8c4 in_seq=0 out_seq=0 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f33f7fff640 1 -- 172.21.15.122:0/2193519892 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dce05add70 con 0x55dce0655d00 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f33f7fff640 10 client.19929 handle_command_reply: tid=0 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f34219d8740 1 client.19929 shutdown 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 msgr2=0x55dce06456e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f340c098df0 tx=0x7f340c09c320 comp rx=0 tx=0).stop 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 shutdown_connections 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.040+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce0655d00 0x55dce06580f0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.041+0000 7f33f7fff640 0 client.19929 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.041+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce0645dd0 0x55dce0642fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.041+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce063d1f0 0x55dce0642a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.041+0000 7f34219d8740 1 --2- 172.21.15.122:0/2193519892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce06456e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.041+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 >> 172.21.15.122:0/2193519892 conn(0x55dce0637000 msgr2=0x55dce0638d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.042+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 shutdown_connections 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.042+0000 7f34219d8740 1 -- 172.21.15.122:0/2193519892 wait complete. 2024-04-02T13:50:02.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.042+0000 7f34219d8740 20 client.19929 trim_cache size 0 max 16384 2024-04-02T13:50:02.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.076+0000 7f34219d8740 1 Processor -- start 2024-04-02T13:50:02.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 -- start start 2024-04-02T13:50:02.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063c8d0 0x55dce06451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 0x55dce06456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34181be020 con 0x55dce063d1f0 2024-04-02T13:50:02.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34181bfda0 con 0x55dce0645dd0 2024-04-02T13:50:02.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f34219d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34181be6f0 con 0x55dce063c8d0 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f3417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 0x55dce06456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f341d2b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f341d2b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:50530/0 (socket says 172.21.15.122:50530) 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f341d2b2640 1 -- 172.21.15.122:0/862847453 learned_addr learned my addr 172.21.15.122:0/862847453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.077+0000 7f3417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 0x55dce06456e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35220/0 (socket says 172.21.15.122:35220) 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341cab1640 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063c8d0 0x55dce06451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 -- 172.21.15.122:0/862847453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063c8d0 msgr2=0x55dce06451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063c8d0 0x55dce06451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 -- 172.21.15.122:0/862847453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 msgr2=0x55dce06456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 0x55dce06456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 -- 172.21.15.122:0/862847453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce0648ce0 con 0x55dce0645dd0 2024-04-02T13:50:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.078+0000 7f341d2b2640 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3410033420 tx=0x7f3410033bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f9ac475d58319d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.079+0000 7f34027fc640 1 -- 172.21.15.122:0/862847453 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410033da0 con 0x55dce0645dd0 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.079+0000 7f34027fc640 1 -- 172.21.15.122:0/862847453 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3410003970 con 0x55dce0645dd0 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.079+0000 7f34027fc640 1 -- 172.21.15.122:0/862847453 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341002cdf0 con 0x55dce0645dd0 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.080+0000 7f34219d8740 1 -- 172.21.15.122:0/862847453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 msgr2=0x55dce065b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.080+0000 7f34219d8740 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3410033420 tx=0x7f3410033bf0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.081+0000 7f34219d8740 1 -- 172.21.15.122:0/862847453 shutdown_connections 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.081+0000 7f34219d8740 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce065b110 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.081+0000 7f34219d8740 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063d1f0 0x55dce06456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.081+0000 7f34219d8740 1 --2- 172.21.15.122:0/862847453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063c8d0 0x55dce06451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.081+0000 7f34219d8740 1 -- 172.21.15.122:0/862847453 >> 172.21.15.122:0/862847453 conn(0x55dce0637000 msgr2=0x55dce06518c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.082+0000 7f34219d8740 1 -- 172.21.15.122:0/862847453 shutdown_connections 2024-04-02T13:50:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.082+0000 7f34219d8740 1 -- 172.21.15.122:0/862847453 wait complete. 2024-04-02T13:50:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.082+0000 7f34219d8740 1 -- start start 2024-04-02T13:50:02.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.082+0000 7f34219d8740 1 Processor -- start 2024-04-02T13:50:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.083+0000 7f34219d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce0658c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.083+0000 7f34219d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 0x55dce06591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.083+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce0658c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f341cab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce0658c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35228/0 (socket says 172.21.15.122:35228) 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f341cab1640 1 -- 172.21.15.122:0/993023986 learned_addr learned my addr 172.21.15.122:0/993023986 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce0659700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f3417fff640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 0x55dce06591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34181bf610 con 0x55dce063c8d0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f341d2b2640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce0659700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f34181bee80 con 0x55dce0645dd0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34180688f0 con 0x55dce063d1f0 2024-04-02T13:50:02.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.084+0000 7f34219d8740 10 client.0 _ll_register_callbacks cb 0x7f340c09be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f341d2b2640 1 -- 172.21.15.122:0/993023986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 msgr2=0x55dce06591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f341d2b2640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 0x55dce06591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f341d2b2640 1 -- 172.21.15.122:0/993023986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 msgr2=0x55dce0658c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f341d2b2640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce0658c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f341d2b2640 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce063bc70 con 0x55dce0645dd0 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.085+0000 7f3417fff640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 0x55dce06591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.086+0000 7f341d2b2640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce0659700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3410009b30 tx=0x7f341002c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7371f7f7b0d712b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:02.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.086+0000 7f33f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:02.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410034430 con 0x55dce0645dd0 2024-04-02T13:50:02.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9525+0+0 (secure 0 0 0) 0x7f34100345d0 con 0x55dce0645dd0 2024-04-02T13:50:02.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3410003970 con 0x55dce0645dd0 2024-04-02T13:50:02.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f3410058050 con 0x55dce0645dd0 2024-04-02T13:50:02.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f34219d8740 10 client.28922 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:02.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.087+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55dce06bc0f0 con 0x55dce0645dd0 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3410034cb0 con 0x55dce0645dd0 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f3410053030 con 0x55dce0645dd0 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 10 client.28922 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 10 client.28922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 20 client.28922 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce06bc820 0x55dce06bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 4 client.28922 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3410003970 con 0x55dce06bc820 2024-04-02T13:50:02.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.088+0000 7f341cab1640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce06bc820 0x55dce06bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:02.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.093+0000 7f341cab1640 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce06bc820 0x55dce06bec40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c062cc0e2b0633f server_cookie=34a67f956b6b1b05 in_seq=0 out_seq=0 2024-04-02T13:50:02.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.093+0000 7f33f7fff640 10 client.28922 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:02.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.093+0000 7f33f7fff640 1 -- 172.21.15.122:0/993023986 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f3410003970 con 0x55dce06bc820 2024-04-02T13:50:02.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f33f7fff640 10 client.28922 handle_command_reply: tid=0 2024-04-02T13:50:02.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 client.28922 shutdown 2024-04-02T13:50:02.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 msgr2=0x55dce0659700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce0659700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3410009b30 tx=0x7f341002c890 comp rx=0 tx=0).stop 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 shutdown_connections 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55dce06bc820 0x55dce06bec40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dce0645dd0 0x55dce0659700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dce063d1f0 0x55dce06591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 --2- 172.21.15.122:0/993023986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dce063c8d0 0x55dce0658c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 >> 172.21.15.122:0/993023986 conn(0x55dce0637000 msgr2=0x55dce0651030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.094+0000 7f33f7fff640 0 client.28922 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:02.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.096+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 shutdown_connections 2024-04-02T13:50:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.097+0000 7f34219d8740 1 -- 172.21.15.122:0/993023986 wait complete. 2024-04-02T13:50:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.097+0000 7f34219d8740 20 client.28922 trim_cache size 0 max 16384 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "105afd33-67ae-48e7-9ade-44ee7d012cad": { 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "105afd33-67ae-48e7-9ade-44ee7d012cad", 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:50:02.100 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:50:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 msgr2=0x7f34080a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 0x7f34080a6290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f34040058c0 tx=0x7f3404005850 comp rx=0 tx=0).stop 2024-04-02T13:50:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 msgr2=0x7f34181a4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f340c00d980 tx=0x7f340c00de40 comp rx=0 tx=0).stop 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 shutdown_connections 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f34080a3dd0 0x7f34080a6290 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3418106a50 0x7f341819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f34181045c0 0x7f34181a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 --2- 172.21.15.122:0/3040423330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3418069240 0x7f34181a4670 secure :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f340c00d980 tx=0x7f340c00de40 comp rx=0 tx=0).stop 2024-04-02T13:50:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.099+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 >> 172.21.15.122:0/3040423330 conn(0x7f3418077b00 msgr2=0x7f34180ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.101+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 shutdown_connections 2024-04-02T13:50:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:02.101+0000 7f341ed15640 1 -- 172.21.15.122:0/3040423330 wait complete. 2024-04-02T13:50:02.123 INFO:tasks.cephfs.filesystem:scrub status for tag:105afd33-67ae-48e7-9ade-44ee7d012cad - {'path': '/', 'tag': '105afd33-67ae-48e7-9ade-44ee7d012cad', 'options': 'recursive,force'} 2024-04-02T13:50:02.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 43207 30.42 MB/sec warmup 82 sec latency 890.655 ms 2024-04-02T13:50:02.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:02 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:02.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:02 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:02.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:02 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 43536 30.26 MB/sec warmup 83 sec latency 190.946 ms 2024-04-02T13:50:03.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:03 smithi187 ceph-mon[36064]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.0 MiB/s wr, 828 op/s 2024-04-02T13:50:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:50:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:03 smithi188 ceph-mon[36117]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.0 MiB/s wr, 828 op/s 2024-04-02T13:50:03.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:03.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:03 smithi122 ceph-mon[32114]: pgmap v465: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 9.0 MiB/s wr, 828 op/s 2024-04-02T13:50:04.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 43696 30.01 MB/sec warmup 84 sec latency 195.047 ms 2024-04-02T13:50:05.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44204 30.09 MB/sec warmup 85 sec latency 200.932 ms 2024-04-02T13:50:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:05 smithi187 ceph-mon[36064]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-04-02T13:50:05.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:05 smithi188 ceph-mon[36117]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-04-02T13:50:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:05 smithi122 ceph-mon[32114]: pgmap v466: 129 pgs: 129 active+clean; 1.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-04-02T13:50:06.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44621 30.07 MB/sec warmup 86 sec latency 440.233 ms 2024-04-02T13:50:07.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44658 29.74 MB/sec warmup 87 sec latency 1248.675 ms 2024-04-02T13:50:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:07 smithi187 ceph-mon[36064]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-04-02T13:50:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:07 smithi188 ceph-mon[36117]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-04-02T13:50:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:07 smithi122 ceph-mon[32114]: pgmap v467: 129 pgs: 129 active+clean; 1.1 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 686 op/s 2024-04-02T13:50:08.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 44937 29.53 MB/sec warmup 88 sec latency 221.674 ms 2024-04-02T13:50:08.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:08 smithi187 ceph-mon[36064]: osdmap e112: 12 total, 12 up, 12 in 2024-04-02T13:50:08.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:08 smithi188 ceph-mon[36117]: osdmap e112: 12 total, 12 up, 12 in 2024-04-02T13:50:08.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:08 smithi122 ceph-mon[32114]: osdmap e112: 12 total, 12 up, 12 in 2024-04-02T13:50:09.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 45183 29.34 MB/sec warmup 89 sec latency 232.319 ms 2024-04-02T13:50:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:09 smithi187 ceph-mon[36064]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:09 smithi188 ceph-mon[36117]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:09 smithi122 ceph-mon[32114]: pgmap v469: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:10.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 45786 29.26 MB/sec warmup 90 sec latency 91.665 ms 2024-04-02T13:50:11.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 46568 29.41 MB/sec warmup 91 sec latency 808.389 ms 2024-04-02T13:50:11.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:11 smithi187 ceph-mon[36064]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:11 smithi188 ceph-mon[36117]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:11 smithi122 ceph-mon[32114]: pgmap v470: 129 pgs: 129 active+clean; 1.1 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 808 op/s 2024-04-02T13:50:12.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 47370 29.67 MB/sec warmup 92 sec latency 1808.423 ms 2024-04-02T13:50:13.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 47995 29.72 MB/sec warmup 93 sec latency 2808.456 ms 2024-04-02T13:50:13.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:13 smithi187 ceph-mon[36064]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:13.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:13 smithi188 ceph-mon[36117]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:13.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:13 smithi122 ceph-mon[32114]: pgmap v471: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:14.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 48666 29.79 MB/sec warmup 94 sec latency 3808.486 ms 2024-04-02T13:50:15.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 49390 29.76 MB/sec warmup 95 sec latency 4808.520 ms 2024-04-02T13:50:15.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:15 smithi187 ceph-mon[36064]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:15.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:15 smithi188 ceph-mon[36117]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:15.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:15 smithi122 ceph-mon[32114]: pgmap v472: 129 pgs: 129 active+clean; 1.1 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 19 MiB/s wr, 1.01k op/s 2024-04-02T13:50:16.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 50049 29.93 MB/sec warmup 96 sec latency 5808.557 ms 2024-04-02T13:50:17.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 50755 29.94 MB/sec warmup 97 sec latency 6808.585 ms 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: osdmap e113: 12 total, 12 up, 12 in 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:17.763 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: osdmap e113: 12 total, 12 up, 12 in 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: pgmap v473: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 1.03k op/s 2024-04-02T13:50:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: osdmap e113: 12 total, 12 up, 12 in 2024-04-02T13:50:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:18.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 51450 30.03 MB/sec warmup 98 sec latency 7808.613 ms 2024-04-02T13:50:19.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 52277 30.22 MB/sec warmup 99 sec latency 8808.641 ms 2024-04-02T13:50:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:19 smithi187 ceph-mon[36064]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:19 smithi188 ceph-mon[36117]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:19 smithi122 ceph-mon[32114]: pgmap v475: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:20.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 52933 30.24 MB/sec warmup 100 sec latency 9204.888 ms 2024-04-02T13:50:21.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 53555 30.30 MB/sec warmup 101 sec latency 99.338 ms 2024-04-02T13:50:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:21 smithi187 ceph-mon[36064]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:21 smithi188 ceph-mon[36117]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:21 smithi122 ceph-mon[32114]: pgmap v476: 129 pgs: 129 active+clean; 1.1 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 864 op/s 2024-04-02T13:50:22.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 54342 30.37 MB/sec warmup 102 sec latency 228.864 ms 2024-04-02T13:50:23.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 55239 30.57 MB/sec warmup 103 sec latency 78.340 ms 2024-04-02T13:50:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:23 smithi187 ceph-mon[36064]: pgmap v477: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:23 smithi188 ceph-mon[36117]: pgmap v477: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:23 smithi122 ceph-mon[32114]: pgmap v477: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:24.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 56100 30.82 MB/sec warmup 104 sec latency 57.918 ms 2024-04-02T13:50:25.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 56789 30.76 MB/sec warmup 105 sec latency 121.632 ms 2024-04-02T13:50:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:25 smithi187 ceph-mon[36064]: pgmap v478: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:25 smithi188 ceph-mon[36117]: pgmap v478: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:25 smithi122 ceph-mon[32114]: pgmap v478: 129 pgs: 129 active+clean; 1.2 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 599 op/s 2024-04-02T13:50:26.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 57459 30.85 MB/sec warmup 106 sec latency 111.165 ms 2024-04-02T13:50:27.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 57564 30.64 MB/sec warmup 107 sec latency 932.188 ms 2024-04-02T13:50:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:27 smithi187 ceph-mon[36064]: pgmap v479: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 579 op/s 2024-04-02T13:50:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:27 smithi188 ceph-mon[36117]: pgmap v479: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 579 op/s 2024-04-02T13:50:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:27 smithi122 ceph-mon[32114]: pgmap v479: 129 pgs: 129 active+clean; 1.2 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 579 op/s 2024-04-02T13:50:28.323 INFO:tasks.workunit.client.0.smithi122.stdout: 10 57886 30.48 MB/sec warmup 108 sec latency 1198.111 ms 2024-04-02T13:50:29.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 58174 30.31 MB/sec warmup 109 sec latency 237.699 ms 2024-04-02T13:50:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:29 smithi187 ceph-mon[36064]: pgmap v480: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 553 op/s 2024-04-02T13:50:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:29 smithi188 ceph-mon[36117]: pgmap v480: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 553 op/s 2024-04-02T13:50:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:29 smithi122 ceph-mon[32114]: pgmap v480: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.9 MiB/s wr, 553 op/s 2024-04-02T13:50:30.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 58852 30.40 MB/sec warmup 110 sec latency 88.827 ms 2024-04-02T13:50:31.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 59302 30.33 MB/sec warmup 111 sec latency 361.748 ms 2024-04-02T13:50:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:31 smithi187 ceph-mon[36064]: pgmap v481: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.5 MiB/s wr, 524 op/s 2024-04-02T13:50:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:31 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:31 smithi188 ceph-mon[36117]: pgmap v481: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.5 MiB/s wr, 524 op/s 2024-04-02T13:50:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:31 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:31 smithi122 ceph-mon[32114]: pgmap v481: 129 pgs: 129 active+clean; 1.2 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.5 MiB/s wr, 524 op/s 2024-04-02T13:50:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:31 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:50:32.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:50:32.125 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:50:32.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.276+0000 7f9f3be3c640 1 -- 172.21.15.122:0/780261456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f34101360 msgr2=0x7f9f3410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.276+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/780261456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f34101360 0x7f9f3410d820 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009900 tx=0x7f9f2c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 -- 172.21.15.122:0/780261456 shutdown_connections 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/780261456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f34101360 0x7f9f3410d820 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/780261456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f341009c0 0x7f9f34100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/780261456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 0x7f9f34105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 -- 172.21.15.122:0/780261456 >> 172.21.15.122:0/780261456 conn(0x7f9f34077a90 msgr2=0x7f9f34077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 -- 172.21.15.122:0/780261456 shutdown_connections 2024-04-02T13:50:32.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 -- 172.21.15.122:0/780261456 wait complete. 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.277+0000 7f9f3be3c640 1 Processor -- start 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 -- start start 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f34101360 0x7f9f34198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 0x7f9f3419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34110d90 con 0x7f9f341009c0 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34110c10 con 0x7f9f341058a0 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3be3c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f34110f10 con 0x7f9f34101360 2024-04-02T13:50:32.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39358/0 (socket says 172.21.15.122:39358) 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 -- 172.21.15.122:0/4109372106 learned_addr learned my addr 172.21.15.122:0/4109372106 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f393d7640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f34101360 0x7f9f34198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 0x7f9f3419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 -- 172.21.15.122:0/4109372106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f34101360 msgr2=0x7f9f34198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f34101360 0x7f9f34198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 -- 172.21.15.122:0/4109372106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 msgr2=0x7f9f3419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 0x7f9f3419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.278+0000 7f9f39bd8640 1 -- 172.21.15.122:0/4109372106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3419d310 con 0x7f9f341009c0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ebc0 tx=0x7f9f3000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f2affd640 1 -- 172.21.15.122:0/4109372106 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010630 con 0x7f9f341009c0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f2affd640 1 -- 172.21.15.122:0/4109372106 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f30010cb0 con 0x7f9f341009c0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f2affd640 1 -- 172.21.15.122:0/4109372106 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30018bd0 con 0x7f9f341009c0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3419d600 con 0x7f9f341009c0 2024-04-02T13:50:32.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.279+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f341a4e90 con 0x7f9f341009c0 2024-04-02T13:50:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.282+0000 7f9f2affd640 1 -- 172.21.15.122:0/4109372106 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9f300107d0 con 0x7f9f341009c0 2024-04-02T13:50:32.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.282+0000 7f9f3ea25740 1 Processor -- start 2024-04-02T13:50:32.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.282+0000 7f9f3ea25740 1 -- start start 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556389884920 0x556389870ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 0x556389871530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5563897da830 con 0x556389884920 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556389782820 con 0x5563898710d0 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5563897d67e0 con 0x556389883f70 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3a3d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556389884920 0x556389870ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3a3d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34560/0 (socket says 172.21.15.122:34560) 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f3a3d9640 1 -- 172.21.15.122:0/580241625 learned_addr learned my addr 172.21.15.122:0/580241625 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f2affd640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 0x7f9f200a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 -- 172.21.15.122:0/580241625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 msgr2=0x556389871530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.283+0000 7f9f393d7640 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 0x556389871530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 0x556389871530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 -- 172.21.15.122:0/580241625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556389884920 msgr2=0x556389870ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556389884920 0x556389870ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 -- 172.21.15.122:0/580241625 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563895b87d0 con 0x556389883f70 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f393d7640 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 0x556389871530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f2affd640 1 -- 172.21.15.122:0/4109372106 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f9f300d12d0 con 0x7f9f341009c0 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 0x7f9f200a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005dc0 tx=0x7f9f2c02c6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e7cabebbc0b15c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 0x7f9f200a6320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04a000 tx=0x7f9f2c047000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f1ffff640 1 -- 172.21.15.122:0/580241625 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c005040 con 0x556389883f70 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f1ffff640 1 -- 172.21.15.122:0/580241625 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c0046c0 con 0x556389883f70 2024-04-02T13:50:32.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.284+0000 7f9f1ffff640 1 -- 172.21.15.122:0/580241625 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0037f0 con 0x556389883f70 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 -- 172.21.15.122:0/580241625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 msgr2=0x556389884350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005dc0 tx=0x7f9f2c02c6c0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 -- 172.21.15.122:0/580241625 shutdown_connections 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5563898710d0 0x556389871530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556389884920 0x556389870ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/580241625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x556389884350 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 -- 172.21.15.122:0/580241625 >> 172.21.15.122:0/580241625 conn(0x556389873570 msgr2=0x556389873950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.285+0000 7f9f3ea25740 1 -- 172.21.15.122:0/580241625 shutdown_connections 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 -- 172.21.15.122:0/580241625 wait complete. 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 -- start start 2024-04-02T13:50:32.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 Processor -- start 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x556389887b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55638988b3b0 con 0x5563898710d0 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556389795d30 con 0x556389883f70 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.286+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55638988b530 con 0x556389884920 2024-04-02T13:50:32.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f3ea25740 10 client.0 _ll_register_callbacks cb 0x556389795720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39378/0 (socket says 172.21.15.122:39378) 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 -- 172.21.15.122:0/2595549135 learned_addr learned my addr 172.21.15.122:0/2595549135 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f393d7640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x556389887b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 -- 172.21.15.122:0/2595549135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 msgr2=0x5563898880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 -- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 msgr2=0x556389887b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x556389887b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 -- 172.21.15.122:0/2595549135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563894ee3f0 con 0x5563898710d0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cba0 tx=0x7f9f30009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f310643b260af583 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010e60 con 0x5563898710d0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c0046c0 con 0x5563898710d0 2024-04-02T13:50:32.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.287+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f9f300d4b50 con 0x5563898710d0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9f3009c780 con 0x5563898710d0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 10 client.19947 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5563894ae0f0 con 0x5563898710d0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c0046c0 con 0x5563898710d0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f9f3009c270 con 0x5563898710d0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 10 client.19947 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 10 client.19947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 20 client.19947 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898985a0 0x55638989aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 4 client.19947 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563897e7d70 con 0x5563898985a0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.288+0000 7f9f393d7640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898985a0 0x55638989aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.289+0000 7f9f393d7640 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898985a0 0x55638989aa60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6e6d8dbffb712fd server_cookie=8b39316282d00853 in_seq=0 out_seq=0 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.289+0000 7f9f1d7fa640 10 client.19947 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:32.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/2595549135 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563897e7d70 con 0x5563898985a0 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f1d7fa640 10 client.19947 handle_command_reply: tid=0 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 client.19947 shutdown 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 msgr2=0x55638988a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000cba0 tx=0x7f9f30009a30 comp rx=0 tx=0).stop 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 shutdown_connections 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898985a0 0x55638989aa60 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f1d7fa640 0 client.19947 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x556389887b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/2595549135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x55638988a7f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.291+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 >> 172.21.15.122:0/2595549135 conn(0x556389873570 msgr2=0x556389872440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.292+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 shutdown_connections 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.292+0000 7f9f3ea25740 1 -- 172.21.15.122:0/2595549135 wait complete. 2024-04-02T13:50:32.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.292+0000 7f9f3ea25740 20 client.19947 trim_cache size 0 max 16384 2024-04-02T13:50:32.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 Processor -- start 2024-04-02T13:50:32.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 -- start start 2024-04-02T13:50:32.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x5563898700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389884920 0x55638988a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341aced0 con 0x5563898710d0 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341aec50 con 0x556389884920 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341ad5a0 con 0x556389883f70 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39386/0 (socket says 172.21.15.122:39386) 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f393d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x5563898700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 -- 172.21.15.122:0/775361537 learned_addr learned my addr 172.21.15.122:0/775361537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389884920 0x55638988a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 -- 172.21.15.122:0/775361537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 msgr2=0x5563898700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x5563898700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 -- 172.21.15.122:0/775361537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389884920 msgr2=0x55638988a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389884920 0x55638988a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 -- 172.21.15.122:0/775361537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556389870e20 con 0x5563898710d0 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.315+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9f300997f0 tx=0x7f9f3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7e2e20c4180ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f1f7fe640 1 -- 172.21.15.122:0/775361537 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010e60 con 0x5563898710d0 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f1f7fe640 1 -- 172.21.15.122:0/775361537 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3009d520 con 0x5563898710d0 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f1f7fe640 1 -- 172.21.15.122:0/775361537 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3009bce0 con 0x5563898710d0 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f3ea25740 1 -- 172.21.15.122:0/775361537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 msgr2=0x5563898942b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9f300997f0 tx=0x7f9f3000cf90 comp rx=0 tx=0).stop 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f3ea25740 1 -- 172.21.15.122:0/775361537 shutdown_connections 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.316+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389884920 0x55638988a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389883f70 0x5563898700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/775361537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x5563898942b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- 172.21.15.122:0/775361537 >> 172.21.15.122:0/775361537 conn(0x556389873570 msgr2=0x55638986f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- 172.21.15.122:0/775361537 shutdown_connections 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- 172.21.15.122:0/775361537 wait complete. 2024-04-02T13:50:32.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- start start 2024-04-02T13:50:32.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 Processor -- start 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x55638988a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898fdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341ae4c0 con 0x5563898710d0 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341add30 con 0x556389883f70 2024-04-02T13:50:32.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f341aaed0 con 0x556389884920 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f3ea25740 10 client.0 _ll_register_callbacks cb 0x7f9f300d49f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.317+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39394/0 (socket says 172.21.15.122:39394) 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 -- 172.21.15.122:0/390999387 learned_addr learned my addr 172.21.15.122:0/390999387 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898fdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f393d7640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x55638988a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 -- 172.21.15.122:0/390999387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 msgr2=0x5563898fdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898fdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 -- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 msgr2=0x55638988a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x55638988a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 -- 172.21.15.122:0/390999387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563898fe120 con 0x5563898710d0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f3a3d9640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f39bd8640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000efe0 tx=0x7f9f3001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee80c96ed87b9a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:32.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30010e60 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f9f300b6260 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.318+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f3009d520 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f9f3009ce20 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 10 client.19953 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5563898ff3a0 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3009d520 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f9f300bd0b0 con 0x5563898710d0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 10 client.19953 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 10 client.19953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 20 client.19953 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898ff8c0 0x556389901ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 4 client.19953 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.319+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563897e20c0 con 0x5563898ff8c0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.320+0000 7f9f393d7640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898ff8c0 0x556389901ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.320+0000 7f9f1d7fa640 10 client.19953 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:32.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.320+0000 7f9f393d7640 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898ff8c0 0x556389901ce0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700d8a6751c1c77 server_cookie=29c8144776372153 in_seq=0 out_seq=0 2024-04-02T13:50:32.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 60070 30.47 MB/sec warmup 112 sec latency 56.712 ms 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f1d7fa640 1 -- 172.21.15.122:0/390999387 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563897e20c0 con 0x5563898ff8c0 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f1d7fa640 10 client.19953 handle_command_reply: tid=0 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 client.19953 shutdown 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 msgr2=0x556389889e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000efe0 tx=0x7f9f3001f000 comp rx=0 tx=0).stop 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 shutdown_connections 2024-04-02T13:50:32.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5563898ff8c0 0x556389901ce0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556389884920 0x5563898fdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556389883f70 0x55638988a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.322+0000 7f9f3ea25740 1 --2- 172.21.15.122:0/390999387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5563898710d0 0x556389889e50 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.322+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 >> 172.21.15.122:0/390999387 conn(0x556389873570 msgr2=0x55638986f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.321+0000 7f9f1d7fa640 0 client.19953 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.322+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 shutdown_connections 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.322+0000 7f9f3ea25740 1 -- 172.21.15.122:0/390999387 wait complete. 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.322+0000 7f9f3ea25740 20 client.19953 trim_cache size 0 max 16384 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:50:32.330 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:50:32.331 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:50:32.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.323+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 msgr2=0x7f9f200a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.323+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 0x7f9f200a6320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c04a000 tx=0x7f9f2c047000 comp rx=0 tx=0).stop 2024-04-02T13:50:32.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 msgr2=0x7f9f34198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ebc0 tx=0x7f9f3000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 shutdown_connections 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9f200a3e60 0x7f9f200a6320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f341058a0 0x7f9f3419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f34101360 0x7f9f34198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 --2- 172.21.15.122:0/4109372106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9f341009c0 0x7f9f34198320 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 >> 172.21.15.122:0/4109372106 conn(0x7f9f34077a90 msgr2=0x7f9f3406a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 shutdown_connections 2024-04-02T13:50:32.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.324+0000 7f9f3be3c640 1 -- 172.21.15.122:0/4109372106 wait complete. 2024-04-02T13:50:32.344 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:50:32.344 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:50:32.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.473+0000 7f0b46711640 1 -- 172.21.15.122:0/3849075078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b401013c0 msgr2=0x7f0b4010d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.473+0000 7f0b46711640 1 --2- 172.21.15.122:0/3849075078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b401013c0 0x7f0b4010d8d0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009cd0 tx=0x7f0b3802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.473+0000 7f0b46711640 1 -- 172.21.15.122:0/3849075078 shutdown_connections 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.473+0000 7f0b46711640 1 --2- 172.21.15.122:0/3849075078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b401013c0 0x7f0b4010d8d0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 --2- 172.21.15.122:0/3849075078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b40100a00 0x7f0b40100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 --2- 172.21.15.122:0/3849075078 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 0x7f0b40106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 -- 172.21.15.122:0/3849075078 >> 172.21.15.122:0/3849075078 conn(0x7f0b40077b00 msgr2=0x7f0b40077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 -- 172.21.15.122:0/3849075078 shutdown_connections 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 -- 172.21.15.122:0/3849075078 wait complete. 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 Processor -- start 2024-04-02T13:50:32.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.474+0000 7f0b46711640 1 -- start start 2024-04-02T13:50:32.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b401013c0 0x7f0b40197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 0x7f0b4019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40110e40 con 0x7f0b40100a00 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40110cc0 con 0x7f0b40106a10 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b46711640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b40110fc0 con 0x7f0b401013c0 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39408/0 (socket says 172.21.15.122:39408) 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 -- 172.21.15.122:0/2476916459 learned_addr learned my addr 172.21.15.122:0/2476916459 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3f7fe640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b401013c0 0x7f0b40197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b44cae640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 0x7f0b4019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 -- 172.21.15.122:0/2476916459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b401013c0 msgr2=0x7f0b40197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b401013c0 0x7f0b40197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 -- 172.21.15.122:0/2476916459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 msgr2=0x7f0b4019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 0x7f0b4019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.475+0000 7f0b3ffff640 1 -- 172.21.15.122:0/2476916459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4019c330 con 0x7f0b40100a00 2024-04-02T13:50:32.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b3ffff640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000d990 tx=0x7f0b3000de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3001b070 con 0x7f0b40100a00 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4019c620 con 0x7f0b40100a00 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b3000edf0 con 0x7f0b40100a00 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b30014870 con 0x7f0b40100a00 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.476+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b40068c60 con 0x7f0b40100a00 2024-04-02T13:50:32.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.477+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b400688f0 con 0x7f0b40100a00 2024-04-02T13:50:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.479+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0b3001e070 con 0x7f0b40100a00 2024-04-02T13:50:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.479+0000 7f0b3d7fa640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 0x7f0b280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.480+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f0b300d21b0 con 0x7f0b40100a00 2024-04-02T13:50:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.480+0000 7f0b3f7fe640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 0x7f0b280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.480+0000 7f0b3f7fe640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 0x7f0b280a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b40105070 tx=0x7f0b34005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.485+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0b3009df20 con 0x7f0b40100a00 2024-04-02T13:50:32.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.828+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b40069680 con 0x7f0b40100a00 2024-04-02T13:50:32.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.829+0000 7f0b3d7fa640 1 -- 172.21.15.122:0/2476916459 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11973 (secure 0 0 0) 0x7f0b300a2220 con 0x7f0b40100a00 2024-04-02T13:50:32.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:50:32.833 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:49:46.715053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:50:32.833 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 55 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 msgr2=0x7f0b280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 0x7f0b280a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0b40105070 tx=0x7f0b34005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 msgr2=0x7f0b40197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000d990 tx=0x7f0b3000de50 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 shutdown_connections 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0b280a3d20 0x7f0b280a61e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b40106a10 0x7f0b4019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b401013c0 0x7f0b40197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 --2- 172.21.15.122:0/2476916459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b40100a00 0x7f0b40197340 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 >> 172.21.15.122:0/2476916459 conn(0x7f0b40077b00 msgr2=0x7f0b401031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 shutdown_connections 2024-04-02T13:50:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.832+0000 7f0b46711640 1 -- 172.21.15.122:0/2476916459 wait complete. 2024-04-02T13:50:32.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:50:32.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/1648685495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc108310 msgr2=0x7fe5dc1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/1648685495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc108310 0x7fe5dc1086f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009900 tx=0x7fe5cc02bc60 comp rx=0 tx=0).stop 2024-04-02T13:50:32.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/1648685495 shutdown_connections 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/1648685495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/1648685495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108cc0 0x7fe5dc10cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/1648685495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc108310 0x7fe5dc1086f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/1648685495 >> 172.21.15.122:0/1648685495 conn(0x7fe5dc076040 msgr2=0x7fe5dc078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/1648685495 shutdown_connections 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.984+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/1648685495 wait complete. 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.985+0000 7fe5e2a2c640 1 Processor -- start 2024-04-02T13:50:32.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.985+0000 7fe5e2a2c640 1 -- start start 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108310 0x7fe5dc1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 0x7fe5dc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37764/0 (socket says 172.21.15.122:37764) 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e0fc9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 0x7fe5dc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e0fc9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 0x7fe5dc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39426/0 (socket says 172.21.15.122:39426) 2024-04-02T13:50:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108310 0x7fe5dc1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5db7fe640 1 -- 172.21.15.122:0/2821091312 learned_addr learned my addr 172.21.15.122:0/2821091312 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc118540 con 0x7fe5dc19e880 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1183c0 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.986+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1186c0 con 0x7fe5dc108310 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 -- 172.21.15.122:0/2821091312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108310 msgr2=0x7fe5dc1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108310 0x7fe5dc1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 -- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 msgr2=0x7fe5dc19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 0x7fe5dc19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc19f3a0 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b470 tx=0x7fe5c400b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5d97fa640 1 -- 172.21.15.122:0/2821091312 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4015020 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc19f690 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5d97fa640 1 -- 172.21.15.122:0/2821091312 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c4011d70 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.987+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1ab8e0 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.988+0000 7fe5d97fa640 1 -- 172.21.15.122:0/2821091312 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400f340 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.991+0000 7fe5d97fa640 1 -- 172.21.15.122:0/2821091312 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe5c4018020 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.991+0000 7fe5e560e740 1 Processor -- start 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.991+0000 7fe5e560e740 1 -- start start 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5d97fa640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 0x7fe5c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227b40c0 0x55b5227bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227bcca0 0x55b5227bf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5227177b0 con 0x55b5227bcca0 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b5226b6570 con 0x55b5227b37a0 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e560e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b522713760 con 0x55b5227b40c0 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37780/0 (socket says 172.21.15.122:37780) 2024-04-02T13:50:32.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 -- 172.21.15.122:0/1053935047 learned_addr learned my addr 172.21.15.122:0/1053935047 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 0x7fe5c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227b40c0 0x55b5227bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227bcca0 0x55b5227bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5d97fa640 1 -- 172.21.15.122:0/2821091312 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fe5c40d04f0 con 0x7fe5dc10d3e0 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 -- 172.21.15.122:0/1053935047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227b40c0 msgr2=0x55b5227bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227b40c0 0x55b5227bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 -- 172.21.15.122:0/1053935047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227bcca0 msgr2=0x55b5227bf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227bcca0 0x55b5227bf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:32.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.992+0000 7fe5e0fc9640 1 -- 172.21.15.122:0/1053935047 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b52242b3f0 con 0x55b5227b37a0 2024-04-02T13:50:33.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.993+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4014910 tx=0x7fe5d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c5e6dcae58906b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.993+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 0x7fe5c80a61e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02c270 tx=0x7fe5cc000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.993+0000 7fe5d27fc640 1 -- 172.21.15.122:0/1053935047 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4014050 con 0x55b5227b37a0 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.993+0000 7fe5d27fc640 1 -- 172.21.15.122:0/1053935047 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d40141f0 con 0x55b5227b37a0 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.993+0000 7fe5d27fc640 1 -- 172.21.15.122:0/1053935047 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4005280 con 0x55b5227b37a0 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 -- 172.21.15.122:0/1053935047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 msgr2=0x55b5227b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4014910 tx=0x7fe5d4009b80 comp rx=0 tx=0).stop 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 -- 172.21.15.122:0/1053935047 shutdown_connections 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227bcca0 0x55b5227bf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227b40c0 0x55b5227bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 --2- 172.21.15.122:0/1053935047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b3b80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 -- 172.21.15.122:0/1053935047 >> 172.21.15.122:0/1053935047 conn(0x55b5227aded0 msgr2=0x55b5227ae2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 -- 172.21.15.122:0/1053935047 shutdown_connections 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.994+0000 7fe5e560e740 1 -- 172.21.15.122:0/1053935047 wait complete. 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.995+0000 7fe5e560e740 1 -- start start 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.995+0000 7fe5e560e740 1 Processor -- start 2024-04-02T13:50:33.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37790/0 (socket says 172.21.15.122:37790) 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5dbfff640 1 -- 172.21.15.122:0/4162259892 learned_addr learned my addr 172.21.15.122:0/4162259892 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b5227bfa90 con 0x55b5227b40c0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b5226d2e80 con 0x55b5227b37a0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b5227bfc10 con 0x55b5227bcca0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.996+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5e560e740 10 client.0 _ll_register_callbacks cb 0x55b52270eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 -- 172.21.15.122:0/4162259892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 msgr2=0x55b5227ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 -- 172.21.15.122:0/4162259892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 msgr2=0x55b5227b9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5224f57d0 con 0x55b5227b40c0 2024-04-02T13:50:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227ba040 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cd960 tx=0x7fe5c4020ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755433bb80d1a74b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.997+0000 7fe5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c409c1b0 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fe5c40b9e10 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d40141f0 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe5c40a0560 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 10 client.19971 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b5227cbb40 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d40141f0 con 0x55b5227b40c0 2024-04-02T13:50:33.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fe5c40a0170 con 0x55b5227b40c0 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 10 client.19971 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 10 client.19971 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 20 client.19971 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5227ccf50 0x55b5227cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 4 client.19971 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.998+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b522724cf0 con 0x55b5227ccf50 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:32.999+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5227ccf50 0x55b5227cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.000+0000 7fe5bb7fe640 10 client.19971 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:50:33.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.000+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5227ccf50 0x55b5227cf410 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cbd0ae4eb38a3ea server_cookie=e0b4a3d3e4104e43 in_seq=0 out_seq=0 2024-04-02T13:50:33.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.004+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/4162259892 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b522724cf0 con 0x55b5227ccf50 2024-04-02T13:50:33.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.004+0000 7fe5bb7fe640 10 client.19971 handle_command_reply: tid=0 2024-04-02T13:50:33.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.004+0000 7fe5e560e740 1 client.19971 shutdown 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.004+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 msgr2=0x55b5227ba040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.004+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227ba040 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cd960 tx=0x7fe5c4020ef0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 shutdown_connections 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5227ccf50 0x55b5227cf410 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227ba040 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 --2- 172.21.15.122:0/4162259892 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227b9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 >> 172.21.15.122:0/4162259892 conn(0x55b5227aded0 msgr2=0x55b5227afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5bb7fe640 0 client.19971 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 shutdown_connections 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 1 -- 172.21.15.122:0/4162259892 wait complete. 2024-04-02T13:50:33.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.005+0000 7fe5e560e740 20 client.19971 trim_cache size 0 max 16384 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 Processor -- start 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 -- start start 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227bc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b56c0 con 0x55b5227b40c0 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b7440 con 0x55b5227b37a0 2024-04-02T13:50:33.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e560e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b5d90 con 0x55b5227bcca0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39452/0 (socket says 172.21.15.122:39452) 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5db7fe640 1 -- 172.21.15.122:0/3122467226 learned_addr learned my addr 172.21.15.122:0/3122467226 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227bc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.025+0000 7fe5e0fc9640 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 -- 172.21.15.122:0/3122467226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 msgr2=0x55b5227b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 -- 172.21.15.122:0/3122467226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 msgr2=0x55b5227bc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227bc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 -- 172.21.15.122:0/3122467226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5227b3ef0 con 0x55b5227b40c0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cde50 tx=0x7fe5c4010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf9890f2747ad0d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5d1ffb640 1 -- 172.21.15.122:0/3122467226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40a4990 con 0x55b5227b40c0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5d1ffb640 1 -- 172.21.15.122:0/3122467226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c409c1b0 con 0x55b5227b40c0 2024-04-02T13:50:33.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.026+0000 7fe5d1ffb640 1 -- 172.21.15.122:0/3122467226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40b9d70 con 0x55b5227b40c0 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 -- 172.21.15.122:0/3122467226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 msgr2=0x55b5227c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40cde50 tx=0x7fe5c4010970 comp rx=0 tx=0).stop 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 -- 172.21.15.122:0/3122467226 shutdown_connections 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b40c0 0x55b5227c8950 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 --2- 172.21.15.122:0/3122467226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b37a0 0x55b5227bc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.027+0000 7fe5e560e740 1 -- 172.21.15.122:0/3122467226 >> 172.21.15.122:0/3122467226 conn(0x55b5227aded0 msgr2=0x55b5227ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.028+0000 7fe5e560e740 1 -- 172.21.15.122:0/3122467226 shutdown_connections 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.028+0000 7fe5e560e740 1 -- 172.21.15.122:0/3122467226 wait complete. 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.028+0000 7fe5e560e740 1 -- start start 2024-04-02T13:50:33.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.028+0000 7fe5e560e740 1 Processor -- start 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 0x55b5227c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227c08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b6cb0 con 0x55b5227b37a0 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b6520 con 0x55b5227b40c0 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b36c0 con 0x55b5227bcca0 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5e560e740 10 client.0 _ll_register_callbacks cb 0x7fe5c40103e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5dbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39462/0 (socket says 172.21.15.122:39462) 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5dbfff640 1 -- 172.21.15.122:0/168246049 learned_addr learned my addr 172.21.15.122:0/168246049 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.029+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 0x55b5227c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 -- 172.21.15.122:0/168246049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 msgr2=0x55b5227c08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227c08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 -- 172.21.15.122:0/168246049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 msgr2=0x55b5227c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 0x55b5227c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 -- 172.21.15.122:0/168246049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b522831640 con 0x55b5227b37a0 2024-04-02T13:50:33.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 0x55b5227c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5dbfff640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009230 tx=0x7fe5cc02bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f9b3736d0511c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc004a40 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fe5cc004be0 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.030+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5c409c1b0 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fe5cc055300 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5e560e740 10 client.19977 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b5228329a0 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5c409c1b0 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fe5cc0035f0 con 0x55b5227b37a0 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5e560e740 10 client.19977 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.031+0000 7fe5e560e740 10 client.19977 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5e560e740 20 client.19977 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5228331a0 0x55b5228355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5e560e740 4 client.19977 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b52271f040 con 0x55b5228331a0 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5228331a0 0x55b5228355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5bb7fe640 10 client.19977 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5db7fe640 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5228331a0 0x55b5228355e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eccfa1ebb3e4159e server_cookie=811d9b82f9250b34 in_seq=0 out_seq=0 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5bb7fe640 1 -- 172.21.15.122:0/168246049 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b52271f040 con 0x55b5228331a0 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.032+0000 7fe5bb7fe640 10 client.19977 handle_command_reply: tid=0 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 client.19977 shutdown 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 msgr2=0x55b5227bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009230 tx=0x7fe5cc02bc60 comp rx=0 tx=0).stop 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 shutdown_connections 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55b5228331a0 0x55b5228355e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b5227bcca0 0x55b5227c08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5bb7fe640 0 client.19977 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:50:33.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b5227b40c0 0x55b5227c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 --2- 172.21.15.122:0/168246049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b5227b37a0 0x55b5227bfe20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 >> 172.21.15.122:0/168246049 conn(0x55b5227aded0 msgr2=0x55b5227ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 shutdown_connections 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.033+0000 7fe5e560e740 1 -- 172.21.15.122:0/168246049 wait complete. 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.034+0000 7fe5e560e740 20 client.19977 trim_cache size 0 max 16384 2024-04-02T13:50:33.040 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.034+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 msgr2=0x7fe5c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.034+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 0x7fe5c80a61e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc02c270 tx=0x7fe5cc000bc0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.034+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 msgr2=0x7fe5dc1a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.034+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400b470 tx=0x7fe5c400b930 comp rx=0 tx=0).stop 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 shutdown_connections 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe5c80a3d20 0x7fe5c80a61e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5dc19e880 0x7fe5dc19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5dc10d3e0 0x7fe5dc1a4c10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 --2- 172.21.15.122:0/2821091312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5dc108310 0x7fe5dc1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 >> 172.21.15.122:0/2821091312 conn(0x7fe5dc076040 msgr2=0x7fe5dc078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 shutdown_connections 2024-04-02T13:50:33.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.035+0000 7fe5e2a2c640 1 -- 172.21.15.122:0/2821091312 wait complete. 2024-04-02T13:50:33.050 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:50:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 -- 172.21.15.122:0/3729128353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 msgr2=0x7fa81c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 --2- 172.21.15.122:0/3729128353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c10e960 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fa814009cd0 tx=0x7fa81402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 -- 172.21.15.122:0/3729128353 shutdown_connections 2024-04-02T13:50:33.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 --2- 172.21.15.122:0/3729128353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c10e960 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 --2- 172.21.15.122:0/3729128353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c101a90 0x7fa81c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 --2- 172.21.15.122:0/3729128353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c107aa0 0x7fa81c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.199+0000 7fa821b04640 1 -- 172.21.15.122:0/3729128353 >> 172.21.15.122:0/3729128353 conn(0x7fa81c078ec0 msgr2=0x7fa81c0792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.200+0000 7fa821b04640 1 -- 172.21.15.122:0/3729128353 shutdown_connections 2024-04-02T13:50:33.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.200+0000 7fa821b04640 1 -- 172.21.15.122:0/3729128353 wait complete. 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.200+0000 7fa821b04640 1 Processor -- start 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 -- start start 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 0x7fa81c073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c107aa0 0x7fa81c074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111eb0 con 0x7fa81c102450 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c111d30 con 0x7fa81c101a90 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa821b04640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c112030 con 0x7fa81c107aa0 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39482/0 (socket says 172.21.15.122:39482) 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 -- 172.21.15.122:0/876995035 learned_addr learned my addr 172.21.15.122:0/876995035 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 0x7fa81c073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81bfff640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c107aa0 0x7fa81c074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 -- 172.21.15.122:0/876995035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c107aa0 msgr2=0x7fa81c074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c107aa0 0x7fa81c074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 -- 172.21.15.122:0/876995035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 msgr2=0x7fa81c073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 0x7fa81c073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.201+0000 7fa81affd640 1 -- 172.21.15.122:0/876995035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1a6870 con 0x7fa81c102450 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 0x7fa81c073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa81affd640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d950 tx=0x7fa81000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa818ff9640 1 -- 172.21.15.122:0/876995035 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810017070 con 0x7fa81c102450 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa818ff9640 1 -- 172.21.15.122:0/876995035 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa810009c30 con 0x7fa81c102450 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1a6b60 con 0x7fa81c102450 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa818ff9640 1 -- 172.21.15.122:0/876995035 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81000f6b0 con 0x7fa81c102450 2024-04-02T13:50:33.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.202+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1a7100 con 0x7fa81c102450 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 Processor -- start 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 -- start start 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df71aff7b0 con 0x55df71ba4ca0 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df71a9e570 con 0x55df71b9b7a0 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa8247c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55df71afb760 con 0x55df71b9c0c0 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa818ff9640 1 -- 172.21.15.122:0/876995035 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa81000f850 con 0x7fa81c102450 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa81affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37844/0 (socket says 172.21.15.122:37844) 2024-04-02T13:50:33.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa81bfff640 1 -- 172.21.15.122:0/1520748810 learned_addr learned my addr 172.21.15.122:0/1520748810 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.205+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 -- 172.21.15.122:0/1520748810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 msgr2=0x55df71ba4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 -- 172.21.15.122:0/1520748810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 msgr2=0x55df71ba7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 -- 172.21.15.122:0/1520748810 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df718133f0 con 0x55df71b9b7a0 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81affd640 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa81bfff640 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa81402c700 tx=0x7fa8140093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1857a609634d35de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa801ffb640 1 -- 172.21.15.122:0/1520748810 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814043020 con 0x55df71b9b7a0 2024-04-02T13:50:33.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa801ffb640 1 -- 172.21.15.122:0/1520748810 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81402cca0 con 0x55df71b9b7a0 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.206+0000 7fa801ffb640 1 -- 172.21.15.122:0/1520748810 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa814039040 con 0x55df71b9b7a0 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa818ff9640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 0x7fa8040a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa818ff9640 1 -- 172.21.15.122:0/876995035 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fa8100d07a0 con 0x7fa81c102450 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 0x7fa8040a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 -- 172.21.15.122:0/1520748810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 msgr2=0x55df71b9bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa81402c700 tx=0x7fa8140093f0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 -- 172.21.15.122:0/1520748810 shutdown_connections 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1520748810 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9b7a0 0x55df71b9bb80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 -- 172.21.15.122:0/1520748810 >> 172.21.15.122:0/1520748810 conn(0x55df71b95ed0 msgr2=0x55df71b962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 -- 172.21.15.122:0/1520748810 shutdown_connections 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.207+0000 7fa8247c7740 1 -- 172.21.15.122:0/1520748810 wait complete. 2024-04-02T13:50:33.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 0x7fa8040a6260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55df71b9b270 tx=0x7fa80c005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 -- start start 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 Processor -- start 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71ba46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 0x55df71ba1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55df71ba7a30 con 0x55df71ba4ca0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55df71abae80 con 0x55df71b9c0c0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55df71ba7bb0 con 0x55df71b9b7a0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.208+0000 7fa8247c7740 10 client.0 _ll_register_callbacks cb 0x55df71af6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71ba46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39504/0 (socket says 172.21.15.122:39504) 2024-04-02T13:50:33.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81bfff640 1 -- 172.21.15.122:0/576655441 learned_addr learned my addr 172.21.15.122:0/576655441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 0x55df71ba1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 -- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 msgr2=0x55df71ba46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71ba46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 -- 172.21.15.122:0/576655441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 msgr2=0x55df71ba2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 -- 172.21.15.122:0/576655441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df718dd7d0 con 0x55df71b9c0c0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81bfff640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71ba46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa81affd640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 0x55df71ba1a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa8100cdb00 tx=0x7fa81009be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61625384f72a9def server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa7faffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.209+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810020070 con 0x55df71b9c0c0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa81402cca0 con 0x55df71b9c0c0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fa8100989c0 con 0x55df71b9c0c0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa810009de0 con 0x55df71b9c0c0 2024-04-02T13:50:33.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 10 client.28970 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55df71bb3a70 con 0x55df71b9c0c0 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81402cca0 con 0x55df71b9c0c0 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fa8100d0030 con 0x55df71b9c0c0 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 10 client.28970 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 10 client.28970 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 20 client.28970 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71bb4f20 0x55df71bb7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 4 client.28970 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df71b0ccf0 con 0x55df71bb4f20 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.210+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71bb4f20 0x55df71bb7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.211+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71bb4f20 0x55df71bb7310 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1426223556b9dd7 server_cookie=661112063d29f8ac in_seq=0 out_seq=0 2024-04-02T13:50:33.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.211+0000 7fa7faffd640 10 client.28970 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:50:33.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa7faffd640 1 -- 172.21.15.122:0/576655441 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df71b0ccf0 con 0x55df71bb4f20 2024-04-02T13:50:33.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa7faffd640 10 client.28970 handle_command_reply: tid=0 2024-04-02T13:50:33.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 client.28970 shutdown 2024-04-02T13:50:33.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 msgr2=0x55df71ba1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 0x55df71ba1a30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa8100cdb00 tx=0x7fa81009be90 comp rx=0 tx=0).stop 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 shutdown_connections 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71bb4f20 0x55df71bb7310 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71ba4ca0 0x55df71ba2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71b9c0c0 0x55df71ba1a30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 --2- 172.21.15.122:0/576655441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71ba46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa7faffd640 0 client.28970 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.214+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 >> 172.21.15.122:0/576655441 conn(0x55df71b95ed0 msgr2=0x55df71b97c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.215+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 shutdown_connections 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.215+0000 7fa8247c7740 1 -- 172.21.15.122:0/576655441 wait complete. 2024-04-02T13:50:33.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.215+0000 7fa8247c7740 20 client.28970 trim_cache size 0 max 16384 2024-04-02T13:50:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.235+0000 7fa8247c7740 1 Processor -- start 2024-04-02T13:50:33.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.235+0000 7fa8247c7740 1 -- start start 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71bb08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9c0c0 0x55df71b962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b13a0 con 0x55df71b9c0c0 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b3120 con 0x55df71ba4ca0 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa8247c7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b1a70 con 0x55df71b9b7a0 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71bb08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa81bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37866/0 (socket says 172.21.15.122:37866) 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa81bfff640 1 -- 172.21.15.122:0/1237873905 learned_addr learned my addr 172.21.15.122:0/1237873905 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.236+0000 7fa81affd640 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9c0c0 0x55df71b962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1237873905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 msgr2=0x55df71ba4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1237873905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9c0c0 msgr2=0x55df71b962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9c0c0 0x55df71b962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1237873905 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df71ba23b0 con 0x55df71b9b7a0 2024-04-02T13:50:33.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71bb08f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55df71bb7b60 tx=0x7fa80c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab39e21a1da8ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa8017fa640 1 -- 172.21.15.122:0/1237873905 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c02b3b0 con 0x55df71b9b7a0 2024-04-02T13:50:33.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.237+0000 7fa8017fa640 1 -- 172.21.15.122:0/1237873905 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c00f480 con 0x55df71b9b7a0 2024-04-02T13:50:33.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8017fa640 1 -- 172.21.15.122:0/1237873905 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c02a660 con 0x55df71b9b7a0 2024-04-02T13:50:33.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 -- 172.21.15.122:0/1237873905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 msgr2=0x55df71bb08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71bb08f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55df71bb7b60 tx=0x7fa80c017000 comp rx=0 tx=0).stop 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 -- 172.21.15.122:0/1237873905 shutdown_connections 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9c0c0 0x55df71b962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1237873905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9b7a0 0x55df71bb08f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 -- 172.21.15.122:0/1237873905 >> 172.21.15.122:0/1237873905 conn(0x55df71b95ed0 msgr2=0x55df71ba12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 -- 172.21.15.122:0/1237873905 shutdown_connections 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.238+0000 7fa8247c7740 1 -- 172.21.15.122:0/1237873905 wait complete. 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.239+0000 7fa8247c7740 1 -- start start 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.239+0000 7fa8247c7740 1 Processor -- start 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa81b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39526/0 (socket says 172.21.15.122:39526) 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1074567672 learned_addr learned my addr 172.21.15.122:0/1074567672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b2990 con 0x55df71b9b7a0 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa81affd640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa81bfff640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b2200 con 0x55df71ba4ca0 2024-04-02T13:50:33.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.240+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1af3a0 con 0x55df71b9c0c0 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa8247c7740 10 client.0 _ll_register_callbacks cb 0x7fa810098680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 msgr2=0x55df71ba5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1074567672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 msgr2=0x55df71ba63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df71baaed0 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81bfff640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa81b7fe640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c20 tx=0x7fa80c004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cdc54a8170e5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa7faffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c02b7c0 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.241+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7fa80c00f630 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa80c00f480 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fa80c041050 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 10 client.19995 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55df71c1b630 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa80c038070 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7fa80c01f9f0 con 0x55df71b9b7a0 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 10 client.19995 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 10 client.19995 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 20 client.19995 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.242+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71c1bd00 0x55df71c1e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.243+0000 7fa8247c7740 4 client.19995 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:33.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.243+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa80c00f480 con 0x55df71c1bd00 2024-04-02T13:50:33.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.243+0000 7fa81affd640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71c1bd00 0x55df71c1e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.243+0000 7fa81affd640 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71c1bd00 0x55df71c1e120 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e6ef0cc114aa17c server_cookie=9dc8c2f257ccdefd in_seq=0 out_seq=0 2024-04-02T13:50:33.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.243+0000 7fa7faffd640 10 client.19995 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:50:33.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa7faffd640 1 -- 172.21.15.122:0/1074567672 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa80c00f480 con 0x55df71c1bd00 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa7faffd640 10 client.19995 handle_command_reply: tid=0 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 client.19995 shutdown 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 msgr2=0x55df71bad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005c20 tx=0x7fa80c004400 comp rx=0 tx=0).stop 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 shutdown_connections 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55df71ba4ca0 0x55df71ba63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55df71b9c0c0 0x55df71ba5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55df71c1bd00 0x55df71c1e120 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 --2- 172.21.15.122:0/1074567672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55df71b9b7a0 0x55df71bad340 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa7faffd640 0 client.19995 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 >> 172.21.15.122:0/1074567672 conn(0x55df71b95ed0 msgr2=0x55df71b962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.246+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 shutdown_connections 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa8247c7740 1 -- 172.21.15.122:0/1074567672 wait complete. 2024-04-02T13:50:33.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa8247c7740 20 client.19995 trim_cache size 0 max 16384 2024-04-02T13:50:33.252 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:33.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 msgr2=0x7fa8040a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 0x7fa8040a6260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55df71b9b270 tx=0x7fa80c005f90 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 msgr2=0x7fa81c074340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.247+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fa81000d950 tx=0x7fa81000de10 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 shutdown_connections 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8040a3da0 0x7fa8040a6260 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa81c107aa0 0x7fa81c074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa81c102450 0x7fa81c074340 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 --2- 172.21.15.122:0/876995035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa81c101a90 0x7fa81c073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 >> 172.21.15.122:0/876995035 conn(0x7fa81c078ec0 msgr2=0x7fa81c1044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 shutdown_connections 2024-04-02T13:50:33.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.248+0000 7fa821b04640 1 -- 172.21.15.122:0/876995035 wait complete. 2024-04-02T13:50:33.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:50:33.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 60811 30.53 MB/sec warmup 113 sec latency 72.976 ms 2024-04-02T13:50:33.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.366+0000 7f12c73ba640 1 -- 172.21.15.122:0/3217418560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0107a80 msgr2=0x7f12c0107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.366+0000 7f12c73ba640 1 --2- 172.21.15.122:0/3217418560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0107a80 0x7f12c0107e60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009930 tx=0x7f12bc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:33.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- 172.21.15.122:0/3217418560 shutdown_connections 2024-04-02T13:50:33.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- 172.21.15.122:0/3217418560 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 0x7f12c010e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- 172.21.15.122:0/3217418560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0068ce0 0x7f12c0069160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- 172.21.15.122:0/3217418560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0107a80 0x7f12c0107e60 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- 172.21.15.122:0/3217418560 >> 172.21.15.122:0/3217418560 conn(0x7f12c0077b00 msgr2=0x7f12c0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- 172.21.15.122:0/3217418560 shutdown_connections 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- 172.21.15.122:0/3217418560 wait complete. 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 Processor -- start 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- start start 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 0x7f12c019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0107a80 0x7f12c019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0111e90 con 0x7f12c0068ce0 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0111d10 con 0x7f12c00696a0 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c73ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0112010 con 0x7f12c0107a80 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c5156640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c5156640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39534/0 (socket says 172.21.15.122:39534) 2024-04-02T13:50:33.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.367+0000 7f12c5156640 1 -- 172.21.15.122:0/1176180499 learned_addr learned my addr 172.21.15.122:0/1176180499 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c4955640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 0x7f12c019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5957640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0107a80 0x7f12c019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 -- 172.21.15.122:0/1176180499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0107a80 msgr2=0x7f12c019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0107a80 0x7f12c019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 -- 172.21.15.122:0/1176180499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 msgr2=0x7f12c019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 0x7f12c019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 -- 172.21.15.122:0/1176180499 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c019f5e0 con 0x7f12c0068ce0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c5156640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009230 tx=0x7f12bc02cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12b67fc640 1 -- 172.21.15.122:0/1176180499 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc041070 con 0x7f12c0068ce0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12b67fc640 1 -- 172.21.15.122:0/1176180499 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc004450 con 0x7f12c0068ce0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c019f870 con 0x7f12c0068ce0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12b67fc640 1 -- 172.21.15.122:0/1176180499 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc034440 con 0x7f12c0068ce0 2024-04-02T13:50:33.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.368+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c01a7160 con 0x7f12c0068ce0 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.370+0000 7f12b67fc640 1 -- 172.21.15.122:0/1176180499 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f12bc044070 con 0x7f12c0068ce0 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 Processor -- start 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 -- start start 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda4b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12b67fc640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 0x7f12ac0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda4bca0 0x5648fda4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648fd9a67b0 con 0x5648fda427a0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648fd945570 con 0x5648fda4bca0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648fd9a2760 con 0x5648fda430c0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39538/0 (socket says 172.21.15.122:39538) 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c4955640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 0x7f12ac0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 -- 172.21.15.122:0/418662318 learned_addr learned my addr 172.21.15.122:0/418662318 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5156640 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda4b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda4bca0 0x5648fda4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12b67fc640 1 -- 172.21.15.122:0/1176180499 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f12bc0f2480 con 0x7f12c0068ce0 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 -- 172.21.15.122:0/418662318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 msgr2=0x5648fda4b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda4b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 -- 172.21.15.122:0/418662318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda4bca0 msgr2=0x5648fda4e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda4bca0 0x5648fda4e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 -- 172.21.15.122:0/418662318 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fd6ba3f0 con 0x5648fda427a0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c5957640 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005c40 tx=0x7f12b8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf660a3d2ad30f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.371+0000 7f12c4955640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 0x7f12ac0a6260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12c010ed80 tx=0x7f12b000b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12ab7fe640 1 -- 172.21.15.122:0/418662318 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b800e850 con 0x5648fda427a0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12ab7fe640 1 -- 172.21.15.122:0/418662318 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8010460 con 0x5648fda427a0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12ab7fe640 1 -- 172.21.15.122:0/418662318 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b800f620 con 0x5648fda427a0 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 -- 172.21.15.122:0/418662318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 msgr2=0x5648fda42b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005c40 tx=0x7f12b8009b30 comp rx=0 tx=0).stop 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 -- 172.21.15.122:0/418662318 shutdown_connections 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda4bca0 0x5648fda4e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda4b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/418662318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda427a0 0x5648fda42b80 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 -- 172.21.15.122:0/418662318 >> 172.21.15.122:0/418662318 conn(0x5648fda3ced0 msgr2=0x5648fda3d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 -- 172.21.15.122:0/418662318 shutdown_connections 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.372+0000 7f12c9fa0740 1 -- 172.21.15.122:0/418662318 wait complete. 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 -- start start 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 Processor -- start 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda427a0 0x5648fda4b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda48ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5648fda4ea90 con 0x5648fda4bca0 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5648fd961e80 con 0x5648fda427a0 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5648fda4ec10 con 0x5648fda430c0 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c9fa0740 10 client.0 _ll_register_callbacks cb 0x5648fd99daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39542/0 (socket says 172.21.15.122:39542) 2024-04-02T13:50:33.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c5957640 1 -- 172.21.15.122:0/2468913700 learned_addr learned my addr 172.21.15.122:0/2468913700 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c5156640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda427a0 0x5648fda4b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.373+0000 7f12c4955640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda48ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 -- 172.21.15.122:0/2468913700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 msgr2=0x5648fda48ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda48ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 -- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda427a0 msgr2=0x5648fda4b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda427a0 0x5648fda4b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 -- 172.21.15.122:0/2468913700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fd7847d0 con 0x5648fda4bca0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c5957640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f12b8012040 tx=0x7f12b80123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b296fa6961cb2360 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8010610 con 0x5648fda4bca0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f12b80107b0 con 0x5648fda4bca0 2024-04-02T13:50:33.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.374+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b8010460 con 0x5648fda4bca0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.377+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12b8033050 con 0x5648fda4bca0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.377+0000 7f12c9fa0740 10 client.20007 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.377+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5648fda5ab20 con 0x5648fda4bca0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.377+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8010460 con 0x5648fda4bca0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.377+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f12b8028020 con 0x5648fda4bca0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 10 client.20007 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 10 client.20007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 20 client.20007 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fda5be70 0x5648fda5e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 4 client.20007 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648fd9b3cf0 con 0x5648fda5be70 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c5156640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fda5be70 0x5648fda5e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12c5156640 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fda5be70 0x5648fda5e280 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=533b1978cd1fa3f3 server_cookie=62c6bd44c04e70dd in_seq=0 out_seq=0 2024-04-02T13:50:33.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.378+0000 7f12a8ff9640 10 client.20007 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2468913700 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648fd9b3cf0 con 0x5648fda5be70 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12a8ff9640 10 client.20007 handle_command_reply: tid=0 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 client.20007 shutdown 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 msgr2=0x5648fda49140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f12b8012040 tx=0x7f12b80123c0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 shutdown_connections 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda49140 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda430c0 0x5648fda48ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fda5be70 0x5648fda5e280 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2468913700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda427a0 0x5648fda4b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12a8ff9640 0 client.20007 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 >> 172.21.15.122:0/2468913700 conn(0x5648fda3ced0 msgr2=0x5648fda3ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 shutdown_connections 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2468913700 wait complete. 2024-04-02T13:50:33.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.379+0000 7f12c9fa0740 20 client.20007 trim_cache size 0 max 16384 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.392+0000 7f12c9fa0740 1 Processor -- start 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.392+0000 7f12c9fa0740 1 -- start start 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.392+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01b1320 con 0x5648fda4bca0 2024-04-02T13:50:33.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01b30a0 con 0x5648fda430c0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01b19f0 con 0x5648fda427a0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39558/0 (socket says 172.21.15.122:39558) 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 -- 172.21.15.122:0/1770734075 learned_addr learned my addr 172.21.15.122:0/1770734075 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c4955640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5156640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 -- 172.21.15.122:0/1770734075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 msgr2=0x5648fda57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 -- 172.21.15.122:0/1770734075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 msgr2=0x5648fda483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 -- 172.21.15.122:0/1770734075 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648fda42ef0 con 0x5648fda4bca0 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5156640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:33.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c4955640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12c5957640 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16e07ba77c2ace09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12aaffd640 1 -- 172.21.15.122:0/1770734075 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b802a440 con 0x5648fda4bca0 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12aaffd640 1 -- 172.21.15.122:0/1770734075 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8010e50 con 0x5648fda4bca0 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.393+0000 7f12aaffd640 1 -- 172.21.15.122:0/1770734075 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b800e720 con 0x5648fda4bca0 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- 172.21.15.122:0/1770734075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 msgr2=0x5648fda63b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b8009f90 comp rx=0 tx=0).stop 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- 172.21.15.122:0/1770734075 shutdown_connections 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda63b50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/1770734075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- 172.21.15.122:0/1770734075 >> 172.21.15.122:0/1770734075 conn(0x5648fda3ced0 msgr2=0x5648fda48a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- 172.21.15.122:0/1770734075 shutdown_connections 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- 172.21.15.122:0/1770734075 wait complete. 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 -- start start 2024-04-02T13:50:33.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.394+0000 7f12c9fa0740 1 Processor -- start 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda44d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01b2910 con 0x5648fda4bca0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01b2180 con 0x5648fda430c0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01af320 con 0x5648fda427a0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 10 client.0 _ll_register_callbacks cb 0x7f12bc0bdfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39572/0 (socket says 172.21.15.122:39572) 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 -- 172.21.15.122:0/2137354330 learned_addr learned my addr 172.21.15.122:0/2137354330 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5156640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda44d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c4955640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 -- 172.21.15.122:0/2137354330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 msgr2=0x5648fda44d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda44d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 -- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 msgr2=0x5648fda45290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda45290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 -- 172.21.15.122:0/2137354330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648fda52e60 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c5957640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009d50 tx=0x7f12b8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39fffe60c36140c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8012460 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f12b802aac0 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b8010e50 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f12b8047050 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 10 client.20013 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.395+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5648fda540b0 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8010e50 con 0x5648fda4bca0 2024-04-02T13:50:33.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f12b8028030 con 0x5648fda4bca0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 10 client.20013 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 10 client.20013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 20 client.20013 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fdac23c0 0x5648fdac47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 4 client.20013 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648fd9ae040 con 0x5648fdac23c0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c5156640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fdac23c0 0x5648fdac47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12a8ff9640 10 client.20013 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.396+0000 7f12c5156640 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fdac23c0 0x5648fdac47b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe103e5c409f32fb server_cookie=8570d148cdb6ece0 in_seq=0 out_seq=0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12a8ff9640 1 -- 172.21.15.122:0/2137354330 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648fd9ae040 con 0x5648fdac23c0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12a8ff9640 10 client.20013 handle_command_reply: tid=0 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 client.20013 shutdown 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 msgr2=0x5648fda457d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009d50 tx=0x7f12b8009c30 comp rx=0 tx=0).stop 2024-04-02T13:50:33.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 shutdown_connections 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5648fda4bca0 0x5648fda457d0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5648fdac23c0 0x5648fdac47b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5648fda430c0 0x5648fda45290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 --2- 172.21.15.122:0/2137354330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5648fda427a0 0x5648fda44d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12a8ff9640 0 client.20013 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 >> 172.21.15.122:0/2137354330 conn(0x5648fda3ced0 msgr2=0x5648fda3e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 shutdown_connections 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 1 -- 172.21.15.122:0/2137354330 wait complete. 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c9fa0740 20 client.20013 trim_cache size 0 max 16384 2024-04-02T13:50:33.408 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 msgr2=0x7f12ac0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 0x7f12ac0a6260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12c010ed80 tx=0x7f12b000b390 comp rx=0 tx=0).stop 2024-04-02T13:50:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 msgr2=0x7f12c019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009230 tx=0x7f12bc02cc70 comp rx=0 tx=0).stop 2024-04-02T13:50:33.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 shutdown_connections 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f12ac0a3da0 0x7f12ac0a6260 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12c0107a80 0x7f12c019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12c00696a0 0x7f12c019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 --2- 172.21.15.122:0/1176180499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12c0068ce0 0x7f12c019a5f0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.397+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 >> 172.21.15.122:0/1176180499 conn(0x7f12c0077b00 msgr2=0x7f12c010d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.398+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 shutdown_connections 2024-04-02T13:50:33.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.398+0000 7f12c73ba640 1 -- 172.21.15.122:0/1176180499 wait complete. 2024-04-02T13:50:33.411 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:50:33.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 -- 172.21.15.122:0/1185353869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c10c110 msgr2=0x7f505c10e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 --2- 172.21.15.122:0/1185353869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c10c110 0x7f505c10e540 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5054009be0 tx=0x7f505402bf80 comp rx=0 tx=0).stop 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 -- 172.21.15.122:0/1185353869 shutdown_connections 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 --2- 172.21.15.122:0/1185353869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c10c110 0x7f505c10e540 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 --2- 172.21.15.122:0/1185353869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c103660 0x7f505c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 --2- 172.21.15.122:0/1185353869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 -- 172.21.15.122:0/1185353869 >> 172.21.15.122:0/1185353869 conn(0x7f505c076d80 msgr2=0x7f505c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 -- 172.21.15.122:0/1185353869 shutdown_connections 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.538+0000 7f5061cb1640 1 -- 172.21.15.122:0/1185353869 wait complete. 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.539+0000 7f5061cb1640 1 Processor -- start 2024-04-02T13:50:33.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.539+0000 7f5061cb1640 1 -- start start 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 0x7f505c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c10c110 0x7f505c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f505c111ea0 con 0x7f505c102d40 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f505c111d20 con 0x7f505c103660 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f5061cb1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f505c112020 con 0x7f505c10c110 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 0x7f505c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39588/0 (socket says 172.21.15.122:39588) 2024-04-02T13:50:33.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 -- 172.21.15.122:0/325231628 learned_addr learned my addr 172.21.15.122:0/325231628 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c10c110 0x7f505c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 -- 172.21.15.122:0/325231628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c10c110 msgr2=0x7f505c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c10c110 0x7f505c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 -- 172.21.15.122:0/325231628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 msgr2=0x7f505c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 0x7f505c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 -- 172.21.15.122:0/325231628 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c19d340 con 0x7f505c102d40 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505affd640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 0x7f505c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.540+0000 7f505b7fe640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9c0 tx=0x7f504c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.541+0000 7f5058ff9640 1 -- 172.21.15.122:0/325231628 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c00cde0 con 0x7f505c102d40 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.541+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505c19d630 con 0x7f505c102d40 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.541+0000 7f5058ff9640 1 -- 172.21.15.122:0/325231628 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c0035f0 con 0x7f505c102d40 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.541+0000 7f5058ff9640 1 -- 172.21.15.122:0/325231628 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c010650 con 0x7f505c102d40 2024-04-02T13:50:33.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.541+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505c1a4ec0 con 0x7f505c102d40 2024-04-02T13:50:33.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 Processor -- start 2024-04-02T13:50:33.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 -- start start 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5058ff9640 1 -- 172.21.15.122:0/325231628 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f504c0037a0 con 0x7f505c102d40 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 0x55a090f20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a090e787b0 con 0x55a090f150c0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a090e17570 con 0x55a090f1dca0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5064975740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a090e74760 con 0x55a090f147a0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39598/0 (socket says 172.21.15.122:39598) 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 0x55a090f20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 -- 172.21.15.122:0/3895882306 learned_addr learned my addr 172.21.15.122:0/3895882306 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 -- 172.21.15.122:0/3895882306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 msgr2=0x55a090f14b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f14b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 -- 172.21.15.122:0/3895882306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 msgr2=0x55a090f20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 0x55a090f20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f5058ff9640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 0x7f50440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505b7fe640 1 -- 172.21.15.122:0/3895882306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a090b8c3f0 con 0x55a090f150c0 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505bfff640 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.544+0000 7f505affd640 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 0x55a090f20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f505bfff640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 0x7f50440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f5058ff9640 1 -- 172.21.15.122:0/325231628 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f504c0d0ef0 con 0x7f505c102d40 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f505b7fe640 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f504c0ce1c0 tx=0x7f504c0d94b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498228755a552477 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f5041ffb640 1 -- 172.21.15.122:0/3895882306 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c09c870 con 0x55a090f150c0 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f5041ffb640 1 -- 172.21.15.122:0/3895882306 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c09d1a0 con 0x55a090f150c0 2024-04-02T13:50:33.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.545+0000 7f5041ffb640 1 -- 172.21.15.122:0/3895882306 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c09c400 con 0x55a090f150c0 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- 172.21.15.122:0/3895882306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 msgr2=0x55a090f1d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f505bfff640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 0x7f50440a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5054006010 tx=0x7f5054009280 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f504c0ce1c0 tx=0x7f504c0d94b0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- 172.21.15.122:0/3895882306 shutdown_connections 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f1dca0 0x55a090f20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f150c0 0x55a090f1d760 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 --2- 172.21.15.122:0/3895882306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f14b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- 172.21.15.122:0/3895882306 >> 172.21.15.122:0/3895882306 conn(0x55a090f0eed0 msgr2=0x55a090f0f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- 172.21.15.122:0/3895882306 shutdown_connections 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- 172.21.15.122:0/3895882306 wait complete. 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.546+0000 7f5064975740 1 -- start start 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 Processor -- start 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f27cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a090f20a90 con 0x55a090f147a0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a090e33e80 con 0x55a090f150c0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a090f20c10 con 0x55a090f1dca0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f5064975740 10 client.0 _ll_register_callbacks cb 0x55a090e6faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f505affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f27cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f505affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37948/0 (socket says 172.21.15.122:37948) 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f505affd640 1 -- 172.21.15.122:0/1605030337 learned_addr learned my addr 172.21.15.122:0/1605030337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.547+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f27cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39610/0 (socket says 172.21.15.122:39610) 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 -- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 msgr2=0x55a090f220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 -- 172.21.15.122:0/1605030337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 msgr2=0x55a090f27cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f27cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 -- 172.21.15.122:0/1605030337 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a090c567d0 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f505affd640 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5050012d00 tx=0x7f505000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463c9f8153266cee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f503affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050009d90 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f504c09d1a0 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.548+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f50500120d0 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5050004030 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 10 client.29006 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a090f30fe0 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c09d1a0 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f505001c460 con 0x55a090f150c0 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 10 client.29006 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 10 client.29006 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 20 client.29006 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f32460 0x55a090f348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 4 client.29006 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:33.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.549+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a090e85cf0 con 0x55a090f32460 2024-04-02T13:50:33.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.550+0000 7f505b7fe640 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f32460 0x55a090f348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.550+0000 7f503affd640 10 client.29006 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:33.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.550+0000 7f505b7fe640 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f32460 0x55a090f348a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=190ac8f3c1f34ddf server_cookie=68981de313458026 in_seq=0 out_seq=0 2024-04-02T13:50:33.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f503affd640 1 -- 172.21.15.122:0/1605030337 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a090e85cf0 con 0x55a090f32460 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f503affd640 10 client.29006 handle_command_reply: tid=0 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 client.29006 shutdown 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 msgr2=0x55a090f281f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5050012d00 tx=0x7f505000f790 comp rx=0 tx=0).stop 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 shutdown_connections 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f32460 0x55a090f348a0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f281f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.553+0000 7f5064975740 1 --2- 172.21.15.122:0/1605030337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f27cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.554+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 >> 172.21.15.122:0/1605030337 conn(0x55a090f0eed0 msgr2=0x55a090f10c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.554+0000 7f503affd640 0 client.29006 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.554+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 shutdown_connections 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.554+0000 7f5064975740 1 -- 172.21.15.122:0/1605030337 wait complete. 2024-04-02T13:50:33.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.554+0000 7f5064975740 20 client.29006 trim_cache size 0 max 16384 2024-04-02T13:50:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:33 smithi122 ceph-mon[32114]: pgmap v482: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 965 op/s 2024-04-02T13:50:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:33 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2476916459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:33 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:33 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:33.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:33 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:50:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 Processor -- start 2024-04-02T13:50:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 -- start start 2024-04-02T13:50:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f34400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f3a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1acf70 con 0x55a090f147a0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1aecf0 con 0x55a090f150c0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f5064975740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1ad640 con 0x55a090f1dca0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39626/0 (socket says 172.21.15.122:39626) 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 -- 172.21.15.122:0/1198613080 learned_addr learned my addr 172.21.15.122:0/1198613080 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505bfff640 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f3a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505affd640 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f34400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 -- 172.21.15.122:0/1198613080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 msgr2=0x55a090f3a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f3a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 -- 172.21.15.122:0/1198613080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 msgr2=0x55a090f34400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f34400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.573+0000 7f505b7fe640 1 -- 172.21.15.122:0/1198613080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a090f14ef0 con 0x55a090f147a0 2024-04-02T13:50:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f505b7fe640 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f504c09ad50 tx=0x7f504c0d4a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c3e742a7faed15 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f50417fa640 1 -- 172.21.15.122:0/1198613080 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c09c5e0 con 0x55a090f147a0 2024-04-02T13:50:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f50417fa640 1 -- 172.21.15.122:0/1198613080 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c09d1a0 con 0x55a090f147a0 2024-04-02T13:50:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f50417fa640 1 -- 172.21.15.122:0/1198613080 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504c0a0e50 con 0x55a090f147a0 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f5064975740 1 -- 172.21.15.122:0/1198613080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 msgr2=0x55a090f22060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f5064975740 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f504c09ad50 tx=0x7f504c0d4a40 comp rx=0 tx=0).stop 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.574+0000 7f5064975740 1 -- 172.21.15.122:0/1198613080 shutdown_connections 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f1dca0 0x55a090f3a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f34400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- 172.21.15.122:0/1198613080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f147a0 0x55a090f22060 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- 172.21.15.122:0/1198613080 >> 172.21.15.122:0/1198613080 conn(0x55a090f0eed0 msgr2=0x55a090f21950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- 172.21.15.122:0/1198613080 shutdown_connections 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- 172.21.15.122:0/1198613080 wait complete. 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- start start 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 Processor -- start 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f2e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1ae560 con 0x55a090f1dca0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1addd0 con 0x55a090f150c0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.575+0000 7f5064975740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f505c1aaf70 con 0x55a090f147a0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f5064975740 10 client.0 _ll_register_callbacks cb 0x7f5050009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f2e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39628/0 (socket says 172.21.15.122:39628) 2024-04-02T13:50:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 -- 172.21.15.122:0/3602858981 learned_addr learned my addr 172.21.15.122:0/3602858981 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 -- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 msgr2=0x55a090f2e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f2e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 -- 172.21.15.122:0/3602858981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 msgr2=0x55a090f2e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f2e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 -- 172.21.15.122:0/3602858981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a090f29dd0 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f505bfff640 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f5054009be0 tx=0x7f505402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc195b509934245 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f503affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505402ce00 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f5054033730 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.576+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f504c09d1a0 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f505405f050 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 10 client.20031 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55a090f985f0 con 0x55a090f1dca0 2024-04-02T13:50:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f504c09d1a0 con 0x55a090f1dca0 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f5054034bb0 con 0x55a090f1dca0 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 10 client.20031 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 10 client.20031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 20 client.20031 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f98e30 0x55a090f9b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 4 client.20031 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a090e80040 con 0x55a090f98e30 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.577+0000 7f505b7fe640 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f98e30 0x55a090f9b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f503affd640 10 client.20031 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f505b7fe640 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f98e30 0x55a090f9b250 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=841de5087cd9e105 server_cookie=5b1e36abccb735c8 in_seq=0 out_seq=0 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f503affd640 1 -- 172.21.15.122:0/3602858981 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a090e80040 con 0x55a090f98e30 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f503affd640 10 client.20031 handle_command_reply: tid=0 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 client.20031 shutdown 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 msgr2=0x55a090f295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f5054009be0 tx=0x7f505402bf80 comp rx=0 tx=0).stop 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 shutdown_connections 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55a090f98e30 0x55a090f9b250 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a090f1dca0 0x55a090f295d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f503affd640 0 client.20031 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a090f150c0 0x55a090f2e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 --2- 172.21.15.122:0/3602858981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a090f147a0 0x55a090f2e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 >> 172.21.15.122:0/3602858981 conn(0x55a090f0eed0 msgr2=0x55a090f27800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 shutdown_connections 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 1 -- 172.21.15.122:0/3602858981 wait complete. 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.578+0000 7f5064975740 20 client.20031 trim_cache size 0 max 16384 2024-04-02T13:50:33.587 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 msgr2=0x7f50440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 0x7f50440a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5054006010 tx=0x7f5054009280 comp rx=0 tx=0).stop 2024-04-02T13:50:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 msgr2=0x7f505c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f504c00e9c0 tx=0x7f504c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 shutdown_connections 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f50440a3d20 0x7f50440a61e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c10c110 0x7f505c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f505c103660 0x7f505c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 --2- 172.21.15.122:0/325231628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f505c102d40 0x7f505c1983a0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 >> 172.21.15.122:0/325231628 conn(0x7f505c076d80 msgr2=0x7f505c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 shutdown_connections 2024-04-02T13:50:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.579+0000 7f5061cb1640 1 -- 172.21.15.122:0/325231628 wait complete. 2024-04-02T13:50:33.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 -- 172.21.15.122:0/2228834838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 msgr2=0x7f1abc106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/2228834838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc106e10 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009930 tx=0x7f1ab802bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 -- 172.21.15.122:0/2228834838 shutdown_connections 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/2228834838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 0x7f1abc10d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/2228834838 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 0x7f1abc100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/2228834838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc106e10 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 -- 172.21.15.122:0/2228834838 >> 172.21.15.122:0/2228834838 conn(0x7f1abc077b60 msgr2=0x7f1abc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 -- 172.21.15.122:0/2228834838 shutdown_connections 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.703+0000 7f1ac44f2640 1 -- 172.21.15.122:0/2228834838 wait complete. 2024-04-02T13:50:33.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 Processor -- start 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 -- start start 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 0x7f1abc197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 0x7f1abc197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110e10 con 0x7f1abc106a30 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110c90 con 0x7f1abc100a20 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac44f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc110f90 con 0x7f1abc1013e0 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39634/0 (socket says 172.21.15.122:39634) 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 -- 172.21.15.122:0/594678305 learned_addr learned my addr 172.21.15.122:0/594678305 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 0x7f1abc197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac228e640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 0x7f1abc197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 -- 172.21.15.122:0/594678305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 msgr2=0x7f1abc197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 0x7f1abc197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 -- 172.21.15.122:0/594678305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 msgr2=0x7f1abc197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.704+0000 7f1ac2a8f640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 0x7f1abc197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ac2a8f640 1 -- 172.21.15.122:0/594678305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc19c2d0 con 0x7f1abc106a30 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ac2a8f640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400eaa0 tx=0x7f1ab400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ab37fe640 1 -- 172.21.15.122:0/594678305 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004d20 con 0x7f1abc106a30 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ab37fe640 1 -- 172.21.15.122:0/594678305 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4005020 con 0x7f1abc106a30 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc19c5c0 con 0x7f1abc106a30 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ab37fe640 1 -- 172.21.15.122:0/594678305 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40116d0 con 0x7f1abc106a30 2024-04-02T13:50:33.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.705+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1a3e50 con 0x7f1abc106a30 2024-04-02T13:50:33.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.707+0000 7f1ab37fe640 1 -- 172.21.15.122:0/594678305 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1ab40051d0 con 0x7f1abc106a30 2024-04-02T13:50:33.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.707+0000 7f1ac71b1740 1 Processor -- start 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 -- start start 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f26970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f26eb0 0x564a74f292a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a74e7f7b0 con 0x564a74f1b460 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564a74e1e570 con 0x564a74f26eb0 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564a74e7b760 con 0x564a74f1bd80 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39640/0 (socket says 172.21.15.122:39640) 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/3361873399 learned_addr learned my addr 172.21.15.122:0/3361873399 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ab37fe640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 0x7f1aa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac228e640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f26970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f26eb0 0x564a74f292a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac2a8f640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 0x7f1aa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/3361873399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 msgr2=0x564a74f26970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f26970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/3361873399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f26eb0 msgr2=0x564a74f292a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ab37fe640 1 -- 172.21.15.122:0/594678305 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f1ab40d0d40 con 0x7f1abc106a30 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f26eb0 0x564a74f292a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/3361873399 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a74b933f0 con 0x564a74f1b460 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.708+0000 7f1ac228e640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f26970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0139b0 tx=0x7f1aac00a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4e5f1529508ff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ac2a8f640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 0x7f1aa80a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400efd0 tx=0x7f1ab406e330 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ab0bfe640 1 -- 172.21.15.122:0/3361873399 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0119e0 con 0x564a74f1b460 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ab0bfe640 1 -- 172.21.15.122:0/3361873399 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac011b80 con 0x564a74f1b460 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ab0bfe640 1 -- 172.21.15.122:0/3361873399 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00fe00 con 0x564a74f1b460 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ac71b1740 1 -- 172.21.15.122:0/3361873399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 msgr2=0x564a74f1b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.709+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0139b0 tx=0x7f1aac00a800 comp rx=0 tx=0).stop 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- 172.21.15.122:0/3361873399 shutdown_connections 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f26eb0 0x564a74f292a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f26970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/3361873399 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1b460 0x564a74f1b840 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- 172.21.15.122:0/3361873399 >> 172.21.15.122:0/3361873399 conn(0x564a74f15ed0 msgr2=0x564a74f162b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- 172.21.15.122:0/3361873399 shutdown_connections 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- 172.21.15.122:0/3361873399 wait complete. 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- start start 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 Processor -- start 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f268c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564a74f29ca0 con 0x564a74f1bd80 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564a74e3ae80 con 0x564a74f1b460 2024-04-02T13:50:33.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564a74f29e20 con 0x564a74f26eb0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac71b1740 10 client.0 _ll_register_callbacks cb 0x564a74e76aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac228e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac228e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39656/0 (socket says 172.21.15.122:39656) 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac228e640 1 -- 172.21.15.122:0/1210086703 learned_addr learned my addr 172.21.15.122:0/1210086703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f268c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.710+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 -- 172.21.15.122:0/1210086703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 msgr2=0x564a74f24260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f24260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 -- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 msgr2=0x564a74f268c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f268c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 -- 172.21.15.122:0/1210086703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a74c5d7d0 con 0x564a74f1bd80 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8036040 tx=0x7f1ab802cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885ed7f7f3f0ad1d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80042c0 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f1ab8004460 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aac011b80 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1ab80046c0 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 10 client.20043 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564a74f35d10 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac011b80 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f1ab80035f0 con 0x564a74f1bd80 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 10 client.20043 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 10 client.20043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 20 client.20043 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f37220 0x564a74f39630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 4 client.20043 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.711+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a74e8ccf0 con 0x564a74f37220 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.712+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f37220 0x564a74f39630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.712+0000 7f1aa5ffb640 10 client.20043 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.712+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f37220 0x564a74f39630 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347a59df475c123c server_cookie=e79f6b1c87e84b76 in_seq=0 out_seq=0 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.712+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1210086703 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a74e8ccf0 con 0x564a74f37220 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1aa5ffb640 10 client.20043 handle_command_reply: tid=0 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 client.20043 shutdown 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 msgr2=0x564a74f23c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8036040 tx=0x7f1ab802cdf0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 shutdown_connections 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f37220 0x564a74f39630 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f23c50 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1210086703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f268c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1aa5ffb640 0 client.20043 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 >> 172.21.15.122:0/1210086703 conn(0x564a74f15ed0 msgr2=0x564a74f17c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 shutdown_connections 2024-04-02T13:50:33.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1210086703 wait complete. 2024-04-02T13:50:33.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.713+0000 7f1ac71b1740 20 client.20043 trim_cache size 0 max 16384 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 Processor -- start 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 -- start start 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f39610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f26380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ace40 con 0x564a74f1bd80 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1aebc0 con 0x564a74f1b460 2024-04-02T13:50:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ad510 con 0x564a74f26eb0 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac228e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac228e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39662/0 (socket says 172.21.15.122:39662) 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac228e640 1 -- 172.21.15.122:0/700538964 learned_addr learned my addr 172.21.15.122:0/700538964 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.729+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f39610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f26380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 -- 172.21.15.122:0/700538964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 msgr2=0x564a74f26380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f26380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 -- 172.21.15.122:0/700538964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 msgr2=0x564a74f39610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f39610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 -- 172.21.15.122:0/700538964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a74f103a0 con 0x564a74f1bd80 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1ac228e640 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1ab802c170 tx=0x7f1ab802c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713f60772ab083ce server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1aa7fff640 1 -- 172.21.15.122:0/700538964 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8005d50 con 0x564a74f1bd80 2024-04-02T13:50:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1aa7fff640 1 -- 172.21.15.122:0/700538964 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8009520 con 0x564a74f1bd80 2024-04-02T13:50:33.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.730+0000 7f1aa7fff640 1 -- 172.21.15.122:0/700538964 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8005610 con 0x564a74f1bd80 2024-04-02T13:50:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:33 smithi187 ceph-mon[36064]: pgmap v482: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 965 op/s 2024-04-02T13:50:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:33 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2476916459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:33 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:33 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:33 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:50:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- 172.21.15.122:0/700538964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 msgr2=0x564a74f32ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f1ab802c170 tx=0x7f1ab802c330 comp rx=0 tx=0).stop 2024-04-02T13:50:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- 172.21.15.122:0/700538964 shutdown_connections 2024-04-02T13:50:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f26eb0 0x564a74f26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f1bd80 0x564a74f32ba0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/700538964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f39610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- 172.21.15.122:0/700538964 >> 172.21.15.122:0/700538964 conn(0x564a74f15ed0 msgr2=0x564a74f1eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- 172.21.15.122:0/700538964 shutdown_connections 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- 172.21.15.122:0/700538964 wait complete. 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- start start 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 Processor -- start 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f2f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f21100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1ae430 con 0x564a74f26eb0 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1adca0 con 0x564a74f1b460 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1aae40 con 0x564a74f1bd80 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.732+0000 7f1ac71b1740 10 client.0 _ll_register_callbacks cb 0x7f1ab8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39678/0 (socket says 172.21.15.122:39678) 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/1676165961 learned_addr learned my addr 172.21.15.122:0/1676165961 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f21100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f2f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/1676165961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 msgr2=0x564a74f21100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f21100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 msgr2=0x564a74f2f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f2f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 -- 172.21.15.122:0/1676165961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a74f27c30 con 0x564a74f26eb0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac1a8d640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1aac013980 tx=0x7f1aac00a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb07b22ae07e8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac013050 con 0x564a74f26eb0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f1aac0131f0 con 0x564a74f26eb0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab8009520 con 0x564a74f26eb0 2024-04-02T13:50:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.733+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1aac00fb50 con 0x564a74f26eb0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 10 client.20049 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x564a74f9daa0 con 0x564a74f26eb0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8009520 con 0x564a74f26eb0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f1aac005310 con 0x564a74f26eb0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 10 client.20049 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 10 client.20049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 20 client.20049 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f9e130 0x564a74fa0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 4 client.20049 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a74e87040 con 0x564a74f9e130 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.734+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f9e130 0x564a74fa0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.735+0000 7f1aa5ffb640 10 client.20049 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.735+0000 7f1ac228e640 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f9e130 0x564a74fa0550 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174fc38f8e214f8b server_cookie=59725d4460ca65a6 in_seq=0 out_seq=0 2024-04-02T13:50:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1aa5ffb640 1 -- 172.21.15.122:0/1676165961 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a74e87040 con 0x564a74f9e130 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1aa5ffb640 10 client.20049 handle_command_reply: tid=0 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 client.20049 shutdown 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 msgr2=0x564a74f21680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f1aac013980 tx=0x7f1aac00a800 comp rx=0 tx=0).stop 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 shutdown_connections 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564a74f9e130 0x564a74fa0550 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564a74f26eb0 0x564a74f21680 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1aa5ffb640 0 client.20049 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a74f1bd80 0x564a74f21100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 --2- 172.21.15.122:0/1676165961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564a74f1b460 0x564a74f2f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 >> 172.21.15.122:0/1676165961 conn(0x564a74f15ed0 msgr2=0x564a74f146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 shutdown_connections 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 1 -- 172.21.15.122:0/1676165961 wait complete. 2024-04-02T13:50:33.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.736+0000 7f1ac71b1740 20 client.20049 trim_cache size 0 max 16384 2024-04-02T13:50:33.745 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 msgr2=0x7f1aa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 0x7f1aa80a61e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400efd0 tx=0x7f1ab406e330 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 msgr2=0x7f1abc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400eaa0 tx=0x7f1ab400eea0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 shutdown_connections 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1aa80a3d20 0x7f1aa80a61e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1abc106a30 0x7f1abc19bba0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1abc1013e0 0x7f1abc197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 --2- 172.21.15.122:0/594678305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1abc100a20 0x7f1abc197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 >> 172.21.15.122:0/594678305 conn(0x7f1abc077b60 msgr2=0x7f1abc103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 shutdown_connections 2024-04-02T13:50:33.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:33.737+0000 7f1ac44f2640 1 -- 172.21.15.122:0/594678305 wait complete. 2024-04-02T13:50:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:33 smithi188 ceph-mon[36117]: pgmap v482: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 18 MiB/s wr, 965 op/s 2024-04-02T13:50:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:33 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2476916459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:50:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:33 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:50:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:33 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:50:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:33 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:50:34.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 61374 30.57 MB/sec warmup 114 sec latency 206.042 ms 2024-04-02T13:50:34.756 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:50:34.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:50:34.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.911+0000 7f82a23db640 1 -- 172.21.15.122:0/2724908039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c101330 msgr2=0x7f829c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.911+0000 7f82a23db640 1 --2- 172.21.15.122:0/2724908039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c101330 0x7f829c10d840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8294009ca0 tx=0x7f829402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 -- 172.21.15.122:0/2724908039 shutdown_connections 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 --2- 172.21.15.122:0/2724908039 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c101330 0x7f829c10d840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 --2- 172.21.15.122:0/2724908039 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 --2- 172.21.15.122:0/2724908039 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c106980 0x7f829c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 -- 172.21.15.122:0/2724908039 >> 172.21.15.122:0/2724908039 conn(0x7f829c077b00 msgr2=0x7f829c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 -- 172.21.15.122:0/2724908039 shutdown_connections 2024-04-02T13:50:34.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 -- 172.21.15.122:0/2724908039 wait complete. 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.912+0000 7f82a23db640 1 Processor -- start 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 -- start start 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c101330 0x7f829c06d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c106980 0x7f829c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f829c06bf30 con 0x7f829c101330 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f829c06bdb0 con 0x7f829c106980 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a23db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f829c06c0b0 con 0x7f829c100970 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f829bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f829bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34846/0 (socket says 172.21.15.122:34846) 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f829b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c101330 0x7f829c06d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f829bfff640 1 -- 172.21.15.122:0/963731882 learned_addr learned my addr 172.21.15.122:0/963731882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.913+0000 7f82a0978640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c106980 0x7f829c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 -- 172.21.15.122:0/963731882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c106980 msgr2=0x7f829c06da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c106980 0x7f829c06da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 -- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c101330 msgr2=0x7f829c06d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c101330 0x7f829c06d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 -- 172.21.15.122:0/963731882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f829c06e170 con 0x7f829c100970 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f829bfff640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f828c00ea60 tx=0x7f828c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f82997fa640 1 -- 172.21.15.122:0/963731882 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c00cde0 con 0x7f829c100970 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f829c068c60 con 0x7f829c100970 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f82997fa640 1 -- 172.21.15.122:0/963731882 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f828c0035f0 con 0x7f829c100970 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.914+0000 7f82997fa640 1 -- 172.21.15.122:0/963731882 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828c018690 con 0x7f829c100970 2024-04-02T13:50:34.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.915+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f829c0691a0 con 0x7f829c100970 2024-04-02T13:50:34.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.917+0000 7f82a4fc0740 1 Processor -- start 2024-04-02T13:50:34.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.917+0000 7f82a4fc0740 1 -- start start 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x561875810280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39700/0 (socket says 172.21.15.122:39700) 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758107c0 0x561875812bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56187576b770 con 0x5618758072c0 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561875713960 con 0x561875807be0 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618757678c0 con 0x5618758107c0 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82997fa640 1 -- 172.21.15.122:0/963731882 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f828c003a80 con 0x7f829c100970 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 -- 172.21.15.122:0/1414248886 learned_addr learned my addr 172.21.15.122:0/1414248886 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:34.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f829bfff640 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x561875810280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 -- 172.21.15.122:0/1414248886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758107c0 msgr2=0x561875812bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758107c0 0x561875812bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 -- 172.21.15.122:0/1414248886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 msgr2=0x561875810280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x561875810280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f82a0978640 1 -- 172.21.15.122:0/1414248886 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618755477e0 con 0x5618758072c0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.918+0000 7f829bfff640 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x561875810280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82a0978640 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f829402c700 tx=0x7f8294005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a84c5a2bb8ba8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82997fa640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 0x7f82840a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82827fc640 1 -- 172.21.15.122:0/1414248886 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294043020 con 0x5618758072c0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f829bfff640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 0x7f82840a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82827fc640 1 -- 172.21.15.122:0/1414248886 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294032c90 con 0x5618758072c0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82827fc640 1 -- 172.21.15.122:0/1414248886 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294039040 con 0x5618758072c0 2024-04-02T13:50:34.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.919+0000 7f82997fa640 1 -- 172.21.15.122:0/963731882 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f828c0d0e00 con 0x7f829c100970 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 -- 172.21.15.122:0/1414248886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 msgr2=0x5618758076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f829402c700 tx=0x7f8294005f80 comp rx=0 tx=0).stop 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f829bfff640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 0x7f82840a6260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ce210 tx=0x7f828c087040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 -- 172.21.15.122:0/1414248886 shutdown_connections 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758107c0 0x561875812bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x561875810280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/1414248886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758072c0 0x5618758076a0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.920+0000 7f82a4fc0740 1 -- 172.21.15.122:0/1414248886 >> 172.21.15.122:0/1414248886 conn(0x5618758019f0 msgr2=0x561875801dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.921+0000 7f82a4fc0740 1 -- 172.21.15.122:0/1414248886 shutdown_connections 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.921+0000 7f82a4fc0740 1 -- 172.21.15.122:0/1414248886 wait complete. 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.921+0000 7f82a4fc0740 1 -- start start 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 Processor -- start 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x56187580e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561875813550 con 0x5618758107c0 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a0978640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561875726d20 con 0x561875807be0 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618758136d0 con 0x5618758072c0 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a0978640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34854/0 (socket says 172.21.15.122:34854) 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a0978640 1 -- 172.21.15.122:0/3942601715 learned_addr learned my addr 172.21.15.122:0/3942601715 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:34.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a4fc0740 10 client.0 _ll_register_callbacks cb 0x5618756f3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f829b7fe640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.922+0000 7f82a0978640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x56187580e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.923+0000 7f82a0978640 1 -- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 msgr2=0x56187580dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.923+0000 7f82a0978640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.923+0000 7f82a0978640 1 -- 172.21.15.122:0/3942601715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 msgr2=0x56187580e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.923+0000 7f82a0978640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x56187580e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.923+0000 7f82a0978640 1 -- 172.21.15.122:0/3942601715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618755a7f70 con 0x5618758072c0 2024-04-02T13:50:34.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.924+0000 7f82a0978640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f829402c630 tx=0x7f8294003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe49e2b8d0640b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.924+0000 7f827b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.924+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294039030 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.924+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8294032c90 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.924+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f8294003ee0 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.925+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8294043070 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.925+0000 7f82a4fc0740 10 client.28783 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.925+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56187543f170 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.925+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294032c90 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f829403f8c0 con 0x5618758072c0 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 10 client.28783 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 10 client.28783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 20 client.28783 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561875824c80 0x561875827070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 4 client.28783 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:34.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561875778cb0 con 0x561875824c80 2024-04-02T13:50:34.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.926+0000 7f829b7fe640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561875824c80 0x561875827070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.927+0000 7f829b7fe640 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561875824c80 0x561875827070 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58f63963b6b425f server_cookie=9980f0215a2a812 in_seq=0 out_seq=0 2024-04-02T13:50:34.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.927+0000 7f827b7fe640 10 client.28783 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:34.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.928+0000 7f827b7fe640 1 -- 172.21.15.122:0/3942601715 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561875778cb0 con 0x561875824c80 2024-04-02T13:50:34.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.928+0000 7f827b7fe640 10 client.28783 handle_command_reply: tid=0 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.928+0000 7f82a4fc0740 1 client.28783 shutdown 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 msgr2=0x56187580d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f829402c630 tx=0x7f8294003d40 comp rx=0 tx=0).stop 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 shutdown_connections 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561875824c80 0x561875827070 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x56187580e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/3942601715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x56187580d7b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 >> 172.21.15.122:0/3942601715 conn(0x5618758019f0 msgr2=0x561875803740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f827b7fe640 0 client.28783 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 shutdown_connections 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 1 -- 172.21.15.122:0/3942601715 wait complete. 2024-04-02T13:50:34.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.929+0000 7f82a4fc0740 20 client.28783 trim_cache size 0 max 16384 2024-04-02T13:50:34.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 Processor -- start 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 -- start start 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x5618758208a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b5660 con 0x5618758107c0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b73e0 con 0x561875807be0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b5d30 con 0x5618758072c0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39710/0 (socket says 172.21.15.122:39710) 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a0978640 1 -- 172.21.15.122:0/2275431607 learned_addr learned my addr 172.21.15.122:0/2275431607 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f82a0978640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x5618758208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.955+0000 7f829b7fe640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 -- 172.21.15.122:0/2275431607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 msgr2=0x5618758208a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x5618758208a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 -- 172.21.15.122:0/2275431607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 msgr2=0x56187580e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 -- 172.21.15.122:0/2275431607 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618757fd2b0 con 0x5618758107c0 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f829b7fe640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f82a0978640 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8294004aa0 tx=0x7f82940095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0050a77f5d3306e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f8281ffb640 1 -- 172.21.15.122:0/2275431607 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294039040 con 0x5618758107c0 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f8281ffb640 1 -- 172.21.15.122:0/2275431607 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940098a0 con 0x5618758107c0 2024-04-02T13:50:34.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.956+0000 7f8281ffb640 1 -- 172.21.15.122:0/2275431607 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940040a0 con 0x5618758107c0 2024-04-02T13:50:34.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2275431607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 msgr2=0x561875800930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8294004aa0 tx=0x7f82940095c0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2275431607 shutdown_connections 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875800930 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561875807be0 0x56187580e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2275431607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618758072c0 0x5618758208a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2275431607 >> 172.21.15.122:0/2275431607 conn(0x5618758019f0 msgr2=0x561875800220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2275431607 shutdown_connections 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2275431607 wait complete. 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.957+0000 7f82a4fc0740 1 -- start start 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 Processor -- start 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618758072c0 0x56187581d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561875807be0 0x5618758184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b6c50 con 0x5618758107c0 2024-04-02T13:50:34.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b64c0 con 0x5618758072c0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f829c1b3660 con 0x561875807be0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a4fc0740 10 client.0 _ll_register_callbacks cb 0x7f8290005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39722/0 (socket says 172.21.15.122:39722) 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 -- 172.21.15.122:0/2209302358 learned_addr learned my addr 172.21.15.122:0/2209302358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f829b7fe640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561875807be0 0x5618758184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618758072c0 0x56187581d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 -- 172.21.15.122:0/2209302358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561875807be0 msgr2=0x5618758184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561875807be0 0x5618758184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 -- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618758072c0 msgr2=0x56187581d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.958+0000 7f82a0978640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618758072c0 0x56187581d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f82a0978640 1 -- 172.21.15.122:0/2209302358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561875819240 con 0x5618758107c0 2024-04-02T13:50:34.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f82a0978640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f829402c630 tx=0x7f8294032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce9da9d267ea30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294039040 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82940098a0 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f8294004050 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f8294004b40 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f82a4fc0740 10 client.20067 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x56187588aee0 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82940098a0 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.959+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f829406d540 con 0x5618758107c0 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 10 client.20067 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 10 client.20067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 20 client.20067 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56187588b420 0x56187588d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 4 client.20067 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:50:34.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561875773000 con 0x56187588b420 2024-04-02T13:50:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f829b7fe640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56187588b420 0x56187588d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f827b7fe640 10 client.20067 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:34.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.960+0000 7f829b7fe640 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56187588b420 0x56187588d840 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84b32fc90d41fec3 server_cookie=fe03b6b2d3943f6f in_seq=0 out_seq=0 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.962+0000 7f827b7fe640 1 -- 172.21.15.122:0/2209302358 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561875773000 con 0x56187588b420 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.962+0000 7f827b7fe640 10 client.20067 handle_command_reply: tid=0 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.962+0000 7f82a4fc0740 1 client.20067 shutdown 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.962+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 msgr2=0x561875818a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.962+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f829402c630 tx=0x7f8294032e10 comp rx=0 tx=0).stop 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 shutdown_connections 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56187588b420 0x56187588d840 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5618758107c0 0x561875818a10 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561875807be0 0x5618758184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f827b7fe640 0 client.20067 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 --2- 172.21.15.122:0/2209302358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5618758072c0 0x56187581d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 >> 172.21.15.122:0/2209302358 conn(0x5618758019f0 msgr2=0x561875800ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 shutdown_connections 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 1 -- 172.21.15.122:0/2209302358 wait complete. 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.963+0000 7f82a4fc0740 20 client.20067 trim_cache size 0 max 16384 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "58b6fcc6-c280-4f21-b1c8-0ab19a657cea", 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:50:34.970 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 msgr2=0x7f82840a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 0x7f82840a6260 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f828c0ce210 tx=0x7f828c087040 comp rx=0 tx=0).stop 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 msgr2=0x7f829c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f828c00ea60 tx=0x7f828c00ef20 comp rx=0 tx=0).stop 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 shutdown_connections 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f82840a3da0 0x7f82840a6260 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f829c106980 0x7f829c06da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f829c101330 0x7f829c06d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 --2- 172.21.15.122:0/963731882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f829c100970 0x7f829c073440 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:34.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 >> 172.21.15.122:0/963731882 conn(0x7f829c077b00 msgr2=0x7f829c103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:34.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 shutdown_connections 2024-04-02T13:50:34.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:34.964+0000 7f82a23db640 1 -- 172.21.15.122:0/963731882 wait complete. 2024-04-02T13:50:34.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:50:34.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:50:35.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.129+0000 7f584a7fc640 1 -- 172.21.15.122:0/1460192214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c02cc80 con 0x7f584c100210 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 -- 172.21.15.122:0/1460192214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c100210 msgr2=0x7f584c1005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 --2- 172.21.15.122:0/1460192214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c100210 0x7f584c1005f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f583c009900 tx=0x7f583c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 -- 172.21.15.122:0/1460192214 shutdown_connections 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 --2- 172.21.15.122:0/1460192214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c106dd0 0x7f584c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 --2- 172.21.15.122:0/1460192214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c100bc0 0x7f584c1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 --2- 172.21.15.122:0/1460192214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c100210 0x7f584c1005f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 -- 172.21.15.122:0/1460192214 >> 172.21.15.122:0/1460192214 conn(0x7f584c076f20 msgr2=0x7f584c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 -- 172.21.15.122:0/1460192214 shutdown_connections 2024-04-02T13:50:35.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 -- 172.21.15.122:0/1460192214 wait complete. 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.130+0000 7f58521a3640 1 Processor -- start 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 -- start start 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 0x7f584c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c19ee80 0x7f584c19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f584c111e80 con 0x7f584c106dd0 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f584c111d00 con 0x7f584c19ee80 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 0x7f584c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f58521a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f584c112000 con 0x7f584c100210 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39748/0 (socket says 172.21.15.122:39748) 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 0x7f584c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34884/0 (socket says 172.21.15.122:34884) 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 -- 172.21.15.122:0/3897550253 learned_addr learned my addr 172.21.15.122:0/3897550253 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584bfff640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c19ee80 0x7f584c19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 -- 172.21.15.122:0/3897550253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 msgr2=0x7f584c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 0x7f584c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 -- 172.21.15.122:0/3897550253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c19ee80 msgr2=0x7f584c19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.131+0000 7f584affd640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c19ee80 0x7f584c19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f584affd640 1 -- 172.21.15.122:0/3897550253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c19f970 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f584affd640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f584000e9c0 tx=0x7f584000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f5848ff9640 1 -- 172.21.15.122:0/3897550253 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584000cd80 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c19fc60 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c1a74a0 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f5848ff9640 1 -- 172.21.15.122:0/3897550253 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5840005500 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.132+0000 7f5848ff9640 1 -- 172.21.15.122:0/3897550253 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840010430 con 0x7f584c106dd0 2024-04-02T13:50:35.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.134+0000 7f5848ff9640 1 -- 172.21.15.122:0/3897550253 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f58400046c0 con 0x7f584c106dd0 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 Processor -- start 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 -- start start 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc1636acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc16373dd0 0x55cc163761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc162ce830 con 0x55cc1636b1f0 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cc16276820 con 0x55cc16373dd0 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5854d81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc162ca7e0 con 0x55cc1636a8d0 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f5848ff9640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.135+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39754/0 (socket says 172.21.15.122:39754) 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 -- 172.21.15.122:0/3911088976 learned_addr learned my addr 172.21.15.122:0/3911088976 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584bfff640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc1636acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584affd640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc16373dd0 0x55cc163761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584bfff640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 -- 172.21.15.122:0/3911088976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 msgr2=0x55cc1636acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc1636acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 -- 172.21.15.122:0/3911088976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc16373dd0 msgr2=0x55cc163761c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc16373dd0 0x55cc163761c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f5848ff9640 1 -- 172.21.15.122:0/3897550253 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f58400d0ff0 con 0x7f584c106dd0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 -- 172.21.15.122:0/3911088976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc160ac7d0 con 0x55cc1636b1f0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584bfff640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc1636acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.136+0000 7f584b7fe640 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f583c02cfd0 tx=0x7f583c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7292aaab2bbc9432 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.137+0000 7f584bfff640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 0x7f58340a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5844009fd0 tx=0x7f5844009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.137+0000 7f5831ffb640 1 -- 172.21.15.122:0/3911088976 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c042070 con 0x55cc1636b1f0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.137+0000 7f5831ffb640 1 -- 172.21.15.122:0/3911088976 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c02ca80 con 0x55cc1636b1f0 2024-04-02T13:50:35.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.138+0000 7f5831ffb640 1 -- 172.21.15.122:0/3911088976 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f583c004f00 con 0x55cc1636b1f0 2024-04-02T13:50:35.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.138+0000 7f5854d81740 1 -- 172.21.15.122:0/3911088976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 msgr2=0x55cc16373890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.138+0000 7f5854d81740 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f583c02cfd0 tx=0x7f583c005c00 comp rx=0 tx=0).stop 2024-04-02T13:50:35.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 -- 172.21.15.122:0/3911088976 shutdown_connections 2024-04-02T13:50:35.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc16373dd0 0x55cc163761c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc16373890 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 --2- 172.21.15.122:0/3911088976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc1636acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 -- 172.21.15.122:0/3911088976 >> 172.21.15.122:0/3911088976 conn(0x55cc16365000 msgr2=0x55cc163653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 -- 172.21.15.122:0/3911088976 shutdown_connections 2024-04-02T13:50:35.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 -- 172.21.15.122:0/3911088976 wait complete. 2024-04-02T13:50:35.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 -- start start 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.139+0000 7f5854d81740 1 Processor -- start 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc16373550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636b1f0 0x55cc163708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc16370e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc16376c00 con 0x55cc16373dd0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cc16289d30 con 0x55cc1636b1f0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc16376d80 con 0x55cc1636a8d0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc16373550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc16373550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34900/0 (socket says 172.21.15.122:34900) 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f584b7fe640 1 -- 172.21.15.122:0/3290511960 learned_addr learned my addr 172.21.15.122:0/3290511960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f584bfff640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc16370e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.140+0000 7f5854d81740 10 client.0 _ll_register_callbacks cb 0x55cc16289720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584affd640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636b1f0 0x55cc163708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 -- 172.21.15.122:0/3290511960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 msgr2=0x55cc16373550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc16373550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 -- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636b1f0 msgr2=0x55cc163708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636b1f0 0x55cc163708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 -- 172.21.15.122:0/3290511960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc15fe23f0 con 0x55cc16373dd0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f584bfff640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc16370e20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5844011a10 tx=0x7f5844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89db84f12ea7e65 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f582affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.141+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400fd80 con 0x55cc16373dd0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f5844004b20 con 0x55cc16373dd0 2024-04-02T13:50:35.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f583c02ca80 con 0x55cc16373dd0 2024-04-02T13:50:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5844004d20 con 0x55cc16373dd0 2024-04-02T13:50:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f5854d81740 10 client.20079 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:35.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cc15fa20f0 con 0x55cc16373dd0 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.142+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f583c02ca80 con 0x55cc16373dd0 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f584401d070 con 0x55cc16373dd0 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 10 client.20079 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 10 client.20079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 20 client.20079 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc16383d80 0x55cc16386220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 4 client.20079 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc162dbd70 con 0x55cc16383d80 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f584b7fe640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc16383d80 0x55cc16386220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.143+0000 7f584b7fe640 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc16383d80 0x55cc16386220 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b144e957815f79d server_cookie=9dbca1be1f4da9e9 in_seq=0 out_seq=0 2024-04-02T13:50:35.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.144+0000 7f582affd640 10 client.20079 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:35.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f582affd640 1 -- 172.21.15.122:0/3290511960 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc162dbd70 con 0x55cc16383d80 2024-04-02T13:50:35.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f582affd640 10 client.20079 handle_command_reply: tid=0 2024-04-02T13:50:35.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 client.20079 shutdown 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 msgr2=0x55cc16370e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc16370e20 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5844011a10 tx=0x7f5844004a40 comp rx=0 tx=0).stop 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 shutdown_connections 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc16383d80 0x55cc16386220 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc16370e20 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636b1f0 0x55cc163708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 --2- 172.21.15.122:0/3290511960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636a8d0 0x55cc16373550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 >> 172.21.15.122:0/3290511960 conn(0x55cc16365000 msgr2=0x55cc16366d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f582affd640 0 client.20079 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 shutdown_connections 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.145+0000 7f5854d81740 1 -- 172.21.15.122:0/3290511960 wait complete. 2024-04-02T13:50:35.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.146+0000 7f5854d81740 20 client.20079 trim_cache size 0 max 16384 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.172+0000 7f5854d81740 1 Processor -- start 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.172+0000 7f5854d81740 1 -- start start 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.172+0000 7f5854d81740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1636fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.172+0000 7f5854d81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636b1f0 0x55cc16366b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.172+0000 7f5854d81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc1638baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1636fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1636fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:38130/0 (socket says 172.21.15.122:38130) 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584b7fe640 1 -- 172.21.15.122:0/4019083539 learned_addr learned my addr 172.21.15.122:0/4019083539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584affd640 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636b1f0 0x55cc16366b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b1290 con 0x55cc16373dd0 2024-04-02T13:50:35.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b3010 con 0x55cc1636a8d0 2024-04-02T13:50:35.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b1960 con 0x55cc1636b1f0 2024-04-02T13:50:35.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc1638baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 -- 172.21.15.122:0/4019083539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636b1f0 msgr2=0x55cc16366b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636b1f0 0x55cc16366b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 -- 172.21.15.122:0/4019083539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 msgr2=0x55cc1636fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1636fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.173+0000 7f584bfff640 1 -- 172.21.15.122:0/4019083539 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc16364e30 con 0x55cc16373dd0 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.174+0000 7f584bfff640 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc1638baf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5844009270 tx=0x7f5844004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798b1dc45273ed38 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.174+0000 7f58317fa640 1 -- 172.21.15.122:0/4019083539 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584400fd80 con 0x55cc16373dd0 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.174+0000 7f58317fa640 1 -- 172.21.15.122:0/4019083539 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584403e070 con 0x55cc16373dd0 2024-04-02T13:50:35.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.174+0000 7f58317fa640 1 -- 172.21.15.122:0/4019083539 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440396e0 con 0x55cc16373dd0 2024-04-02T13:50:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 msgr2=0x55cc1638baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc1638baf0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f5844009270 tx=0x7f5844004cb0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 shutdown_connections 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc16373dd0 0x55cc1638baf0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc1636b1f0 0x55cc16366b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 --2- 172.21.15.122:0/4019083539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1636fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 >> 172.21.15.122:0/4019083539 conn(0x55cc16365000 msgr2=0x55cc16366430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 shutdown_connections 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- 172.21.15.122:0/4019083539 wait complete. 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.175+0000 7f5854d81740 1 -- start start 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.176+0000 7f5854d81740 1 Processor -- start 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.176+0000 7f5854d81740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1637c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.176+0000 7f5854d81740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc163776a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.176+0000 7f5854d81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f5854d81740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b2880 con 0x55cc1636b1f0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f5854d81740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1b20f0 con 0x55cc1636a8d0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f5854d81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f584c1af290 con 0x55cc16373dd0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34928/0 (socket says 172.21.15.122:34928) 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 -- 172.21.15.122:0/2845768528 learned_addr learned my addr 172.21.15.122:0/2845768528 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f5854d81740 10 client.0 _ll_register_callbacks cb 0x7f5844011870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584b7fe640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1637c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 -- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 msgr2=0x55cc1637c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1637c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 -- 172.21.15.122:0/2845768528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 msgr2=0x55cc163776a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc163776a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584bfff640 1 -- 172.21.15.122:0/2845768528 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc1636a050 con 0x55cc16373dd0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.177+0000 7f584b7fe640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1637c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f584bfff640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5844010710 tx=0x7f58440093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16624de2c648542 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f582affd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440055f0 con 0x55cc16373dd0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f5844025040 con 0x55cc16373dd0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584403e070 con 0x55cc16373dd0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f5844039480 con 0x55cc16373dd0 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f5854d81740 10 client.28807 fetch_fsmap learned FSMap version 55 2024-04-02T13:50:35.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.178+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55cc163ea0d0 con 0x55cc16373dd0 2024-04-02T13:50:35.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5844039b00 con 0x55cc16373dd0 2024-04-02T13:50:35.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 55) ==== 8434+0+0 (secure 0 0 0) 0x7f584401d040 con 0x55cc16373dd0 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 10 client.28807 fetch_fsmap finished waiting for FSMap version 55 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 10 client.28807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 20 client.28807 populate_metadata read hostname 'smithi122' 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc163ea790 0x55cc163ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 4 client.28807 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f584403e070 con 0x55cc163ea790 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.179+0000 7f584b7fe640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc163ea790 0x55cc163ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.180+0000 7f582affd640 10 client.28807 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.180+0000 7f584b7fe640 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc163ea790 0x55cc163ecbb0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4029ad29e2f608 server_cookie=ca71476dfc4f4ff1 in_seq=0 out_seq=0 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f582affd640 1 -- 172.21.15.122:0/2845768528 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f584403e070 con 0x55cc163ea790 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f582affd640 10 client.28807 handle_command_reply: tid=0 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 client.28807 shutdown 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 msgr2=0x55cc16377be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5844010710 tx=0x7f58440093e0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 shutdown_connections 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc163ea790 0x55cc163ecbb0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f582affd640 0 client.28807 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc16373dd0 0x55cc16377be0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc1636b1f0 0x55cc163776a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 --2- 172.21.15.122:0/2845768528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc1636a8d0 0x55cc1637c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 >> 172.21.15.122:0/2845768528 conn(0x55cc16365000 msgr2=0x55cc163639b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 shutdown_connections 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 1 -- 172.21.15.122:0/2845768528 wait complete. 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.181+0000 7f5854d81740 20 client.28807 trim_cache size 0 max 16384 2024-04-02T13:50:35.187 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "58b6fcc6-c280-4f21-b1c8-0ab19a657cea": { 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "58b6fcc6-c280-4f21-b1c8-0ab19a657cea", 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:50:35.188 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 msgr2=0x7f58340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 0x7f58340a61e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5844009fd0 tx=0x7f5844009500 comp rx=0 tx=0).stop 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 msgr2=0x7f584c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f584000e9c0 tx=0x7f584000ee80 comp rx=0 tx=0).stop 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 shutdown_connections 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f58340a3d20 0x7f58340a61e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f584c19ee80 0x7f584c19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f584c106dd0 0x7f584c19ab00 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 --2- 172.21.15.122:0/3897550253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f584c100210 0x7f584c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:50:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 >> 172.21.15.122:0/3897550253 conn(0x7f584c076f20 msgr2=0x7f584c105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:50:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 shutdown_connections 2024-04-02T13:50:35.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:50:35.182+0000 7f58521a3640 1 -- 172.21.15.122:0/3897550253 wait complete. 2024-04-02T13:50:35.198 INFO:tasks.cephfs.filesystem:scrub status for tag:58b6fcc6-c280-4f21-b1c8-0ab19a657cea - {'path': '/', 'tag': '58b6fcc6-c280-4f21-b1c8-0ab19a657cea', 'options': 'recursive,force'} 2024-04-02T13:50:35.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 61971 30.60 MB/sec warmup 115 sec latency 56.580 ms 2024-04-02T13:50:35.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:35 smithi122 ceph-mon[32114]: pgmap v483: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T13:50:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:35 smithi187 ceph-mon[36064]: pgmap v483: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T13:50:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:35 smithi188 ceph-mon[36117]: pgmap v483: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 715 op/s 2024-04-02T13:50:36.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 62747 30.62 MB/sec warmup 116 sec latency 64.826 ms 2024-04-02T13:50:36.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:50:36.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:36 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:50:36.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:36 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:50:36.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:36.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:36 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:50:37.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 63567 30.75 MB/sec warmup 117 sec latency 79.763 ms 2024-04-02T13:50:37.700 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:37 smithi188 ceph-mon[36117]: pgmap v484: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 783 op/s 2024-04-02T13:50:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:37 smithi187 ceph-mon[36064]: pgmap v484: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 783 op/s 2024-04-02T13:50:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:37 smithi122 ceph-mon[32114]: pgmap v484: 129 pgs: 129 active+clean; 1.2 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 783 op/s 2024-04-02T13:50:38.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 64469 30.91 MB/sec warmup 118 sec latency 43.462 ms 2024-04-02T13:50:39.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 65081 30.99 MB/sec warmup 119 sec latency 147.874 ms 2024-04-02T13:50:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:39 smithi187 ceph-mon[36064]: pgmap v485: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 821 op/s 2024-04-02T13:50:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:39 smithi188 ceph-mon[36117]: pgmap v485: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 821 op/s 2024-04-02T13:50:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:39 smithi122 ceph-mon[32114]: pgmap v485: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 821 op/s 2024-04-02T13:50:41.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 66566 44.77 MB/sec execute 1 sec latency 70.000 ms 2024-04-02T13:50:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:41 smithi187 ceph-mon[36064]: pgmap v486: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 677 op/s 2024-04-02T13:50:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:41 smithi188 ceph-mon[36117]: pgmap v486: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 677 op/s 2024-04-02T13:50:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:41 smithi122 ceph-mon[32114]: pgmap v486: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 13 MiB/s wr, 677 op/s 2024-04-02T13:50:42.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 66950 30.16 MB/sec execute 2 sec latency 610.060 ms 2024-04-02T13:50:43.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 67239 25.08 MB/sec execute 3 sec latency 920.209 ms 2024-04-02T13:50:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:43 smithi187 ceph-mon[36064]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 904 op/s 2024-04-02T13:50:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:43 smithi188 ceph-mon[36117]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 904 op/s 2024-04-02T13:50:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:43 smithi122 ceph-mon[32114]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 904 op/s 2024-04-02T13:50:44.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 67626 24.85 MB/sec execute 4 sec latency 119.587 ms 2024-04-02T13:50:45.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 68443 30.44 MB/sec execute 5 sec latency 80.693 ms 2024-04-02T13:50:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:45 smithi187 ceph-mon[36064]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 507 op/s 2024-04-02T13:50:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:45 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:50:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:45 smithi188 ceph-mon[36117]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 507 op/s 2024-04-02T13:50:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:45 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:50:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:45 smithi122 ceph-mon[32114]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 507 op/s 2024-04-02T13:50:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:45 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:50:46.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 69138 31.13 MB/sec execute 6 sec latency 151.149 ms 2024-04-02T13:50:47.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 69832 31.93 MB/sec execute 7 sec latency 45.693 ms 2024-04-02T13:50:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-04-02T13:50:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:47 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:47 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:50:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: pgmap v489: 129 pgs: 129 active+clean; 1.3 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 722 op/s 2024-04-02T13:50:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:47 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:50:48.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 70592 33.32 MB/sec execute 8 sec latency 82.860 ms 2024-04-02T13:50:49.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 71443 34.12 MB/sec execute 9 sec latency 80.345 ms 2024-04-02T13:50:49.709 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:49 smithi188 ceph-mon[36117]: pgmap v490: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-04-02T13:50:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:49 smithi187 ceph-mon[36064]: pgmap v490: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-04-02T13:50:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:49 smithi122 ceph-mon[32114]: pgmap v490: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 832 op/s 2024-04-02T13:50:50.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 72138 34.87 MB/sec execute 10 sec latency 52.881 ms 2024-04-02T13:50:51.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 72899 35.06 MB/sec execute 11 sec latency 72.528 ms 2024-04-02T13:50:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:51 smithi187 ceph-mon[36064]: pgmap v491: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-04-02T13:50:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:51 smithi188 ceph-mon[36117]: pgmap v491: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-04-02T13:50:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:51 smithi122 ceph-mon[32114]: pgmap v491: 129 pgs: 129 active+clean; 1.3 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-04-02T13:50:52.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 73649 35.35 MB/sec execute 12 sec latency 53.118 ms 2024-04-02T13:50:53.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 74409 36.34 MB/sec execute 13 sec latency 56.215 ms 2024-04-02T13:50:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:53 smithi187 ceph-mon[36064]: pgmap v492: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-04-02T13:50:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:53 smithi188 ceph-mon[36117]: pgmap v492: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-04-02T13:50:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:53 smithi122 ceph-mon[32114]: pgmap v492: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 891 op/s 2024-04-02T13:50:54.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 75184 36.49 MB/sec execute 14 sec latency 93.360 ms 2024-04-02T13:50:55.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 76005 36.82 MB/sec execute 15 sec latency 90.490 ms 2024-04-02T13:50:55.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:55 smithi187 ceph-mon[36064]: pgmap v493: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 695 op/s 2024-04-02T13:50:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:55 smithi188 ceph-mon[36117]: pgmap v493: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 695 op/s 2024-04-02T13:50:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:55 smithi122 ceph-mon[32114]: pgmap v493: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 695 op/s 2024-04-02T13:50:56.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 76817 37.85 MB/sec execute 16 sec latency 54.900 ms 2024-04-02T13:50:57.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 77321 36.74 MB/sec execute 17 sec latency 243.587 ms 2024-04-02T13:50:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:57 smithi187 ceph-mon[36064]: pgmap v494: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T13:50:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:57 smithi188 ceph-mon[36117]: pgmap v494: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T13:50:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:57 smithi122 ceph-mon[32114]: pgmap v494: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 766 op/s 2024-04-02T13:50:58.324 INFO:tasks.workunit.client.0.smithi122.stdout: 10 77429 35.19 MB/sec execute 18 sec latency 1033.650 ms 2024-04-02T13:50:59.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 77680 33.88 MB/sec execute 19 sec latency 490.470 ms 2024-04-02T13:50:59.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:50:59 smithi187 ceph-mon[36064]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 729 op/s 2024-04-02T13:50:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:50:59 smithi188 ceph-mon[36117]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 729 op/s 2024-04-02T13:50:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:50:59 smithi122 ceph-mon[32114]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 729 op/s 2024-04-02T13:51:00.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78071 33.25 MB/sec execute 20 sec latency 561.999 ms 2024-04-02T13:51:01.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78599 32.97 MB/sec execute 21 sec latency 381.444 ms 2024-04-02T13:51:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:01 smithi187 ceph-mon[36064]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-04-02T13:51:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:01 smithi188 ceph-mon[36117]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-04-02T13:51:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:01 smithi122 ceph-mon[32114]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 550 op/s 2024-04-02T13:51:02.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78710 31.67 MB/sec execute 22 sec latency 880.855 ms 2024-04-02T13:51:03.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 78710 30.29 MB/sec execute 23 sec latency 1429.467 ms 2024-04-02T13:51:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:03 smithi187 ceph-mon[36064]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 867 op/s 2024-04-02T13:51:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:03 smithi188 ceph-mon[36117]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 867 op/s 2024-04-02T13:51:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:03 smithi122 ceph-mon[32114]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 16 MiB/s wr, 867 op/s 2024-04-02T13:51:04.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 79208 29.93 MB/sec execute 24 sec latency 1542.763 ms 2024-04-02T13:51:04.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:04 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:04.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:04 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:04.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:04 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:05.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:51:05.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:51:05.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 79228 28.90 MB/sec execute 25 sec latency 236.595 ms 2024-04-02T13:51:05.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.365+0000 7f77ce7fc640 1 -- 172.21.15.122:0/1592238242 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0032a00 con 0x7f77d0100a30 2024-04-02T13:51:05.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 -- 172.21.15.122:0/1592238242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100a30 msgr2=0x7f77d01045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 --2- 172.21.15.122:0/1592238242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100a30 0x7f77d01045a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009ca0 tx=0x7f77c002c120 comp rx=0 tx=0).stop 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 -- 172.21.15.122:0/1592238242 shutdown_connections 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 --2- 172.21.15.122:0/1592238242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100a30 0x7f77d01045a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 --2- 172.21.15.122:0/1592238242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0100070 0x7f77d01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 --2- 172.21.15.122:0/1592238242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0106b40 0x7f77d0106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.366+0000 7f77d5b40640 1 -- 172.21.15.122:0/1592238242 >> 172.21.15.122:0/1592238242 conn(0x7f77d0077030 msgr2=0x7f77d00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.370+0000 7f77d5b40640 1 -- 172.21.15.122:0/1592238242 shutdown_connections 2024-04-02T13:51:05.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.372+0000 7f77d5b40640 1 -- 172.21.15.122:0/1592238242 wait complete. 2024-04-02T13:51:05.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.373+0000 7f77d5b40640 1 Processor -- start 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.373+0000 7f77d5b40640 1 -- start start 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100070 0x7f77d01a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 0x7f77d01a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 0x7f77d019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01185b0 con 0x7f77d0100a30 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0118430 con 0x7f77d0106b40 2024-04-02T13:51:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.374+0000 7f77d5b40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0118730 con 0x7f77d0100070 2024-04-02T13:51:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.375+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 0x7f77d01a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.375+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 0x7f77d01a4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59192/0 (socket says 172.21.15.122:59192) 2024-04-02T13:51:05.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.375+0000 7f77ceffd640 1 -- 172.21.15.122:0/2891485046 learned_addr learned my addr 172.21.15.122:0/2891485046 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.376+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 0x7f77d019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.376+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100070 0x7f77d01a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.377+0000 7f77cffff640 1 -- 172.21.15.122:0/2891485046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100070 msgr2=0x7f77d01a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.377+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100070 0x7f77d01a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.377+0000 7f77cffff640 1 -- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 msgr2=0x7f77d01a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.377+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 0x7f77d01a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.377+0000 7f77cffff640 1 -- 172.21.15.122:0/2891485046 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019ef50 con 0x7f77d0106b40 2024-04-02T13:51:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.378+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 0x7f77d019e780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f77c002c660 tx=0x7f77c0035d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.379+0000 7f77ccff9640 1 -- 172.21.15.122:0/2891485046 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0039040 con 0x7f77d0106b40 2024-04-02T13:51:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.379+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019f1e0 con 0x7f77d0106b40 2024-04-02T13:51:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.379+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d006b520 con 0x7f77d0106b40 2024-04-02T13:51:05.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.379+0000 7f77ccff9640 1 -- 172.21.15.122:0/2891485046 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c0044070 con 0x7f77d0106b40 2024-04-02T13:51:05.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.379+0000 7f77ccff9640 1 -- 172.21.15.122:0/2891485046 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0003e60 con 0x7f77d0106b40 2024-04-02T13:51:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.382+0000 7f77ccff9640 1 -- 172.21.15.122:0/2891485046 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f77c00042a0 con 0x7f77d0106b40 2024-04-02T13:51:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.382+0000 7f77d8807740 1 Processor -- start 2024-04-02T13:51:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.382+0000 7f77d8807740 1 -- start start 2024-04-02T13:51:05.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 0x5587a8dcbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5587a8d2f830 con 0x5587a8dd4dd0 2024-04-02T13:51:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587a8cd7820 con 0x5587a8dcb8d0 2024-04-02T13:51:05.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77d8807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587a8d2b7e0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59198/0 (socket says 172.21.15.122:59198) 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77ceffd640 1 -- 172.21.15.122:0/1517116717 learned_addr learned my addr 172.21.15.122:0/1517116717 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.383+0000 7f77ceffd640 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 0x5587a8dcbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ccff9640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 0x7f77b80a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 -- 172.21.15.122:0/1517116717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 msgr2=0x5587a8dd4890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd4890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 -- 172.21.15.122:0/1517116717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 msgr2=0x5587a8dd71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:05.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 -- 172.21.15.122:0/1517116717 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a8b0d7d0 con 0x5587a8dcb8d0 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ceffd640 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 0x5587a8dcbcb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005fd0 tx=0x7f77c4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e22f8fe3b86a60b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77ccff9640 1 -- 172.21.15.122:0/2891485046 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f77c00f2830 con 0x7f77d0106b40 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77b5ffb640 1 -- 172.21.15.122:0/1517116717 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4005020 con 0x5587a8dcb8d0 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77b5ffb640 1 -- 172.21.15.122:0/1517116717 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c40047a0 con 0x5587a8dcb8d0 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.384+0000 7f77b5ffb640 1 -- 172.21.15.122:0/1517116717 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c400e8c0 con 0x5587a8dcb8d0 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 -- 172.21.15.122:0/1517116717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 msgr2=0x5587a8dcbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 0x5587a8dcbcb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f77c4005fd0 tx=0x7f77c4005de0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 0x7f77b80a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 -- 172.21.15.122:0/1517116717 shutdown_connections 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 --2- 172.21.15.122:0/1517116717 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcb8d0 0x5587a8dcbcb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.386+0000 7f77d8807740 1 -- 172.21.15.122:0/1517116717 >> 172.21.15.122:0/1517116717 conn(0x5587a8dc6000 msgr2=0x5587a8dc63e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.387+0000 7f77d8807740 1 -- 172.21.15.122:0/1517116717 shutdown_connections 2024-04-02T13:51:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.387+0000 7f77d8807740 1 -- 172.21.15.122:0/1517116717 wait complete. 2024-04-02T13:51:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.387+0000 7f77d8807740 1 -- start start 2024-04-02T13:51:05.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.388+0000 7f77d8807740 1 Processor -- start 2024-04-02T13:51:05.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.388+0000 7f77d8807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8dd46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77d8807740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77d8807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77cffff640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 0x7f77b80a63b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fa0 tx=0x7f77c0005f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77cf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77cf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43048/0 (socket says 172.21.15.122:43048) 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77cf7fe640 1 -- 172.21.15.122:0/381782943 learned_addr learned my addr 172.21.15.122:0/381782943 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5587a8dd7bc0 con 0x5587a8dcb8d0 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587a8cead30 con 0x5587a8dcc1f0 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.389+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587a8dd7d40 con 0x5587a8dd4dd0 2024-04-02T13:51:05.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77d8807740 10 client.0 _ll_register_callbacks cb 0x5587a8cea720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:05.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77ceffd640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8dd46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 -- 172.21.15.122:0/381782943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 msgr2=0x5587a8dd1fc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd1fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 -- 172.21.15.122:0/381782943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 msgr2=0x5587a8dd46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8dd46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a8a433f0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.390+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e480 tx=0x7f77c800e940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd170c3a1426810 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.391+0000 7f77aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.391+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8017070 con 0x5587a8dcc1f0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.391+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f77c8013680 con 0x5587a8dcc1f0 2024-04-02T13:51:05.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.391+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c40047a0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.391+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f77c8021050 con 0x5587a8dcc1f0 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 10 client.29066 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5587a8a030f0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c40047a0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f77c801cb10 con 0x5587a8dcc1f0 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 10 client.29066 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 10 client.29066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 20 client.29066 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8de4f50 0x5587a8de7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 4 client.29066 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:05.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.392+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a8d3cd70 con 0x5587a8de4f50 2024-04-02T13:51:05.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.393+0000 7f77ceffd640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8de4f50 0x5587a8de7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.395+0000 7f77ceffd640 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8de4f50 0x5587a8de7370 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab243a48824eaf32 server_cookie=2334965e1626bbf in_seq=0 out_seq=0 2024-04-02T13:51:05.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.395+0000 7f77aeffd640 10 client.29066 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.398+0000 7f77aeffd640 1 -- 172.21.15.122:0/381782943 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587a8d3cd70 con 0x5587a8de4f50 2024-04-02T13:51:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.398+0000 7f77aeffd640 10 client.29066 handle_command_reply: tid=0 2024-04-02T13:51:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.398+0000 7f77d8807740 1 client.29066 shutdown 2024-04-02T13:51:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 msgr2=0x5587a8dd1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e480 tx=0x7f77c800e940 comp rx=0 tx=0).stop 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 shutdown_connections 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8de4f50 0x5587a8de7370 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dd1a80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 --2- 172.21.15.122:0/381782943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8dd46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 >> 172.21.15.122:0/381782943 conn(0x5587a8dc6000 msgr2=0x5587a8dc7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.399+0000 7f77aeffd640 0 client.29066 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:05.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.401+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 shutdown_connections 2024-04-02T13:51:05.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.401+0000 7f77d8807740 1 -- 172.21.15.122:0/381782943 wait complete. 2024-04-02T13:51:05.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.401+0000 7f77d8807740 20 client.29066 trim_cache size 0 max 16384 2024-04-02T13:51:05.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.437+0000 7f77d8807740 1 Processor -- start 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 -- start start 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc010060 con 0x5587a8dcb8d0 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc011de0 con 0x5587a8dcc1f0 2024-04-02T13:51:05.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.438+0000 7f77d8807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc010730 con 0x5587a8dd4dd0 2024-04-02T13:51:05.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59214/0 (socket says 172.21.15.122:59214) 2024-04-02T13:51:05.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 -- 172.21.15.122:0/3980651091 learned_addr learned my addr 172.21.15.122:0/3980651091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 -- 172.21.15.122:0/3980651091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 msgr2=0x5587a8dd41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 -- 172.21.15.122:0/3980651091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 msgr2=0x5587a8dccf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dccf40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.439+0000 7f77ceffd640 1 -- 172.21.15.122:0/3980651091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a8de3a30 con 0x5587a8dcb8d0 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.440+0000 7f77ceffd640 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c400bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0e4aa3ea564029 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.440+0000 7f77b57fa640 1 -- 172.21.15.122:0/3980651091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4026bc0 con 0x5587a8dcb8d0 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.440+0000 7f77b57fa640 1 -- 172.21.15.122:0/3980651091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4026d60 con 0x5587a8dcb8d0 2024-04-02T13:51:05.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.440+0000 7f77b57fa640 1 -- 172.21.15.122:0/3980651091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c400e660 con 0x5587a8dcb8d0 2024-04-02T13:51:05.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.441+0000 7f77d8807740 1 -- 172.21.15.122:0/3980651091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 msgr2=0x5587a8de0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.441+0000 7f77d8807740 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f77c4000c00 tx=0x7f77c400bbc0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.443+0000 7f77d8807740 1 -- 172.21.15.122:0/3980651091 shutdown_connections 2024-04-02T13:51:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.443+0000 7f77d8807740 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dd4dd0 0x5587a8dd41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.443+0000 7f77d8807740 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dcc1f0 0x5587a8dccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.443+0000 7f77d8807740 1 --2- 172.21.15.122:0/3980651091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8de0b10 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.443+0000 7f77d8807740 1 -- 172.21.15.122:0/3980651091 >> 172.21.15.122:0/3980651091 conn(0x5587a8dc6000 msgr2=0x5587a8dcc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.444+0000 7f77d8807740 1 -- 172.21.15.122:0/3980651091 shutdown_connections 2024-04-02T13:51:05.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.444+0000 7f77d8807740 1 -- 172.21.15.122:0/3980651091 wait complete. 2024-04-02T13:51:05.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.445+0000 7f77d8807740 1 -- start start 2024-04-02T13:51:05.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.445+0000 7f77d8807740 1 Processor -- start 2024-04-02T13:51:05.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8ddd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8ddd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 0x5587a8dcddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc011650 con 0x5587a8dcb8d0 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc010ec0 con 0x5587a8dd4dd0 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc00e060 con 0x5587a8dcc1f0 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.448+0000 7f77d8807740 10 client.0 _ll_register_callbacks cb 0x7f77c8009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.449+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8ddd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.449+0000 7f77ceffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8ddd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59224/0 (socket says 172.21.15.122:59224) 2024-04-02T13:51:05.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.449+0000 7f77ceffd640 1 -- 172.21.15.122:0/2995881641 learned_addr learned my addr 172.21.15.122:0/2995881641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.449+0000 7f77ceffd640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 0x5587a8dcddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8ddd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 -- 172.21.15.122:0/2995881641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 msgr2=0x5587a8ddd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8ddd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 -- 172.21.15.122:0/2995881641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 msgr2=0x5587a8ddd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8ddd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 -- 172.21.15.122:0/2995881641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a8dce4c0 con 0x5587a8dd4dd0 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8ddd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77ceffd640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 0x5587a8dcddd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f77c40267e0 tx=0x7f77c400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4d491e10c1b14b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.450+0000 7f77aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:05.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.457+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40264a0 con 0x5587a8dd4dd0 2024-04-02T13:51:05.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.457+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c4026d60 con 0x5587a8dd4dd0 2024-04-02T13:51:05.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.457+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f77c4004020 con 0x5587a8dd4dd0 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.459+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f77c4004220 con 0x5587a8dd4dd0 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.459+0000 7f77d8807740 10 client.29069 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.459+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5587a8e4aee0 con 0x5587a8dd4dd0 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.459+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4026d60 con 0x5587a8dd4dd0 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f77c40035f0 con 0x5587a8dd4dd0 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 10 client.29069 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 10 client.29069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 20 client.29069 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8e4b640 0x5587a8e4da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 4 client.29069 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:51:05.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.460+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587a8d370c0 con 0x5587a8e4b640 2024-04-02T13:51:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.461+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8e4b640 0x5587a8e4da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.462+0000 7f77cf7fe640 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8e4b640 0x5587a8e4da60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ef8e2e21035cbf server_cookie=cc5d5ec75a80de49 in_seq=0 out_seq=0 2024-04-02T13:51:05.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.462+0000 7f77aeffd640 10 client.29069 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:05.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.462+0000 7f77aeffd640 1 -- 172.21.15.122:0/2995881641 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5587a8d370c0 con 0x5587a8e4b640 2024-04-02T13:51:05.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.462+0000 7f77aeffd640 10 client.29069 handle_command_reply: tid=0 2024-04-02T13:51:05.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.462+0000 7f77d8807740 1 client.29069 shutdown 2024-04-02T13:51:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 msgr2=0x5587a8dcddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 0x5587a8dcddd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f77c40267e0 tx=0x7f77c400b480 comp rx=0 tx=0).stop 2024-04-02T13:51:05.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 shutdown_connections 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5587a8e4b640 0x5587a8e4da60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587a8dd4dd0 0x5587a8dcddd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587a8dcc1f0 0x5587a8ddd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 --2- 172.21.15.122:0/2995881641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587a8dcb8d0 0x5587a8ddd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 >> 172.21.15.122:0/2995881641 conn(0x5587a8dc6000 msgr2=0x5587a8dc5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.465+0000 7f77aeffd640 0 client.29069 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:05.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.467+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 shutdown_connections 2024-04-02T13:51:05.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.467+0000 7f77d8807740 1 -- 172.21.15.122:0/2995881641 wait complete. 2024-04-02T13:51:05.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.468+0000 7f77d8807740 20 client.29069 trim_cache size 0 max 16384 2024-04-02T13:51:05.471 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:05.472 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:51:05.472 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:51:05.472 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:05.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 msgr2=0x7f77b80a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 0x7f77b80a63b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fa0 tx=0x7f77c0005f10 comp rx=0 tx=0).stop 2024-04-02T13:51:05.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 msgr2=0x7f77d019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 0x7f77d019e780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f77c002c660 tx=0x7f77c0035d10 comp rx=0 tx=0).stop 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 shutdown_connections 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f77b80a3ef0 0x7f77b80a63b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f77d0106b40 0x7f77d019e780 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77d0100a30 0x7f77d01a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 --2- 172.21.15.122:0/2891485046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77d0100070 0x7f77d01a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.471+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 >> 172.21.15.122:0/2891485046 conn(0x7f77d0077030 msgr2=0x7f77d0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.472+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 shutdown_connections 2024-04-02T13:51:05.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.472+0000 7f77d5b40640 1 -- 172.21.15.122:0/2891485046 wait complete. 2024-04-02T13:51:05.494 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:51:05.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:51:05.495 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:05 smithi122 ceph-mon[32114]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.661+0000 7f369b49a640 1 -- 172.21.15.122:0/3193792873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694108c90 msgr2=0x7f3694109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.661+0000 7f369b49a640 1 --2- 172.21.15.122:0/3193792873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694108c90 0x7f3694109070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f368400dcc0 tx=0x7f368402fc60 comp rx=0 tx=0).stop 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.662+0000 7f369b49a640 1 -- 172.21.15.122:0/3193792873 shutdown_connections 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.662+0000 7f369b49a640 1 --2- 172.21.15.122:0/3193792873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3694114c50 0x7f3694117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.662+0000 7f369b49a640 1 --2- 172.21.15.122:0/3193792873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36941095b0 0x7f3694114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.662+0000 7f369b49a640 1 --2- 172.21.15.122:0/3193792873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694108c90 0x7f3694109070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.662+0000 7f369b49a640 1 -- 172.21.15.122:0/3193792873 >> 172.21.15.122:0/3193792873 conn(0x7f369406cbf0 msgr2=0x7f369406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:05.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.664+0000 7f369b49a640 1 -- 172.21.15.122:0/3193792873 shutdown_connections 2024-04-02T13:51:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.664+0000 7f369b49a640 1 -- 172.21.15.122:0/3193792873 wait complete. 2024-04-02T13:51:05.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.665+0000 7f369b49a640 1 Processor -- start 2024-04-02T13:51:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.665+0000 7f369b49a640 1 -- start start 2024-04-02T13:51:05.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3694108c90 0x7f36941a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 0x7f36941a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 0x7f36941aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3694118540 con 0x7f36941095b0 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36941183c0 con 0x7f3694108c90 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f369b49a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36941186c0 con 0x7f3694114c50 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f3699a37640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 0x7f36941aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f3699a37640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 0x7f36941aba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49494/0 (socket says 172.21.15.122:49494) 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f3699a37640 1 -- 172.21.15.122:0/3351255342 learned_addr learned my addr 172.21.15.122:0/3351255342 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:05.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.666+0000 7f3698a35640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 0x7f36941a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 -- 172.21.15.122:0/3351255342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 msgr2=0x7f36941aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 0x7f36941aba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 -- 172.21.15.122:0/3351255342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3694108c90 msgr2=0x7f36941a7190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:51:05.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3694108c90 0x7f36941a7190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:05.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 -- 172.21.15.122:0/3351255342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36941ac130 con 0x7f36941095b0 2024-04-02T13:51:05.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.667+0000 7f3698a35640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 0x7f36941a76d0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.668+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369000ccc0 con 0x7f36941095b0 2024-04-02T13:51:05.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.668+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690010910 con 0x7f36941095b0 2024-04-02T13:51:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.668+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690018960 con 0x7f36941095b0 2024-04-02T13:51:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.668+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36941ac420 con 0x7f36941095b0 2024-04-02T13:51:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.668+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36941b3cb0 con 0x7f36941095b0 2024-04-02T13:51:05.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.669+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3678005290 con 0x7f36941095b0 2024-04-02T13:51:05.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.672+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3690010ac0 con 0x7f36941095b0 2024-04-02T13:51:05.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.673+0000 7f368a7fc640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 0x7f36800a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:05.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.674+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f36900d1400 con 0x7f36941095b0 2024-04-02T13:51:05.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.676+0000 7f3699236640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 0x7f36800a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:05.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.679+0000 7f3699236640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 0x7f36800a63e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3684010090 tx=0x7f368400d670 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:05.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:05.680+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f369009d6d0 con 0x7f36941095b0 2024-04-02T13:51:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:05 smithi187 ceph-mon[36064]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:51:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:05 smithi188 ceph-mon[36117]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:51:06.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.112+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3678005880 con 0x7f36941095b0 2024-04-02T13:51:06.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.114+0000 7f368a7fc640 1 -- 172.21.15.122:0/3351255342 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11970 (secure 0 0 0) 0x7f369009cfd0 con 0x7f36941095b0 2024-04-02T13:51:06.118 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:51:06.118 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:51:03.416002+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:51:06.118 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 57 2024-04-02T13:51:06.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 msgr2=0x7f36800a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 0x7f36800a63e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3684010090 tx=0x7f368400d670 comp rx=0 tx=0).stop 2024-04-02T13:51:06.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 msgr2=0x7f36941a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 0x7f36941a76d0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f369000e9f0 tx=0x7f369000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 shutdown_connections 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36800a3f20 0x7f36800a63e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3694114c50 0x7f36941aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36941095b0 0x7f36941a76d0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 --2- 172.21.15.122:0/3351255342 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3694108c90 0x7f36941a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 >> 172.21.15.122:0/3351255342 conn(0x7f369406cbf0 msgr2=0x7f36940718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 shutdown_connections 2024-04-02T13:51:06.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.118+0000 7f369b49a640 1 -- 172.21.15.122:0/3351255342 wait complete. 2024-04-02T13:51:06.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:51:06.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.307+0000 7f289c8fa640 1 -- 172.21.15.122:0/1058951173 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890032990 con 0x7f2898113a50 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.307+0000 7f289fb60640 1 -- 172.21.15.122:0/1058951173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2898113a50 msgr2=0x7f2898115e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.307+0000 7f289fb60640 1 --2- 172.21.15.122:0/1058951173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2898113a50 0x7f2898115e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289002c100 comp rx=0 tx=0).stop 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.307+0000 7f289fb60640 1 -- 172.21.15.122:0/1058951173 shutdown_connections 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.308+0000 7f289fb60640 1 --2- 172.21.15.122:0/1058951173 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2898113a50 0x7f2898115e40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.308+0000 7f289fb60640 1 --2- 172.21.15.122:0/1058951173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f289810adc0 0x7f2898113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.308+0000 7f289fb60640 1 --2- 172.21.15.122:0/1058951173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289810a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.308+0000 7f289fb60640 1 -- 172.21.15.122:0/1058951173 >> 172.21.15.122:0/1058951173 conn(0x7f289806cbe0 msgr2=0x7f289806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.309+0000 7f289fb60640 1 -- 172.21.15.122:0/1058951173 shutdown_connections 2024-04-02T13:51:06.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.309+0000 7f289fb60640 1 -- 172.21.15.122:0/1058951173 wait complete. 2024-04-02T13:51:06.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 Processor -- start 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 -- start start 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 0x7f289811eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2898113a50 0x7f2898119b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2898118450 con 0x7f289810a4a0 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f28981182d0 con 0x7f289810adc0 2024-04-02T13:51:06.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.310+0000 7f289fb60640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f28981185d0 con 0x7f2898113a50 2024-04-02T13:51:06.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59276/0 (socket says 172.21.15.122:59276) 2024-04-02T13:51:06.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 -- 172.21.15.122:0/3618507508 learned_addr learned my addr 172.21.15.122:0/3618507508 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2898113a50 0x7f2898119b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d0fb640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 0x7f289811eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 -- 172.21.15.122:0/3618507508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2898113a50 msgr2=0x7f2898119b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2898113a50 0x7f2898119b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 -- 172.21.15.122:0/3618507508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 msgr2=0x7f289811eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 0x7f289811eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.311+0000 7f289d8fc640 1 -- 172.21.15.122:0/3618507508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289811a3e0 con 0x7f289810a4a0 2024-04-02T13:51:06.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.312+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f288800d950 tx=0x7f288800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f288effd640 1 -- 172.21.15.122:0/3618507508 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888017030 con 0x7f289810a4a0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289811a6d0 con 0x7f289810a4a0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2898124860 con 0x7f289810a4a0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f288effd640 1 -- 172.21.15.122:0/3618507508 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2888004020 con 0x7f289810a4a0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f288effd640 1 -- 172.21.15.122:0/3618507508 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888003d20 con 0x7f289810a4a0 2024-04-02T13:51:06.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.313+0000 7f289d0fb640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 0x7f289811eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:06.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.317+0000 7f28a2825740 1 Processor -- start 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 -- start start 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371470c0 0x564d3714f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d37152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d370aa7b0 con 0x564d371470c0 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d37049570 con 0x564d3714fca0 2024-04-02T13:51:06.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f28a2825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d370a6760 con 0x564d371467a0 2024-04-02T13:51:06.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.318+0000 7f288effd640 1 -- 172.21.15.122:0/3618507508 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f288801a070 con 0x7f289810a4a0 2024-04-02T13:51:06.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.319+0000 7f289e0fd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.319+0000 7f289e0fd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49526/0 (socket says 172.21.15.122:49526) 2024-04-02T13:51:06.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.319+0000 7f289e0fd640 1 -- 172.21.15.122:0/373639482 learned_addr learned my addr 172.21.15.122:0/373639482 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.319+0000 7f289d0fb640 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d37152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f288effd640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 0x7f28840a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 -- 172.21.15.122:0/373639482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 msgr2=0x564d37152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d37152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 -- 172.21.15.122:0/373639482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371470c0 msgr2=0x564d3714f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371470c0 0x564d3714f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 -- 172.21.15.122:0/373639482 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d36dbe3f0 con 0x564d371467a0 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f289e0fd640 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2890004af0 tx=0x7f2890009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a58e9c99d6ff3f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.320+0000 7f288effd640 1 -- 172.21.15.122:0/3618507508 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f28880d0700 con 0x7f289810a4a0 2024-04-02T13:51:06.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.321+0000 7f2883fff640 1 -- 172.21.15.122:0/373639482 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890044070 con 0x564d371467a0 2024-04-02T13:51:06.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.321+0000 7f2883fff640 1 -- 172.21.15.122:0/373639482 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890003970 con 0x564d371467a0 2024-04-02T13:51:06.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.321+0000 7f2883fff640 1 -- 172.21.15.122:0/373639482 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890039040 con 0x564d371467a0 2024-04-02T13:51:06.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 79311 28.10 MB/sec execute 26 sec latency 316.175 ms 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.321+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 0x7f28840a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 -- 172.21.15.122:0/373639482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 msgr2=0x564d37146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2890004af0 tx=0x7f2890009800 comp rx=0 tx=0).stop 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 -- 172.21.15.122:0/373639482 shutdown_connections 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d37152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371470c0 0x564d3714f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 --2- 172.21.15.122:0/373639482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d37146b80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.322+0000 7f28a2825740 1 -- 172.21.15.122:0/373639482 >> 172.21.15.122:0/373639482 conn(0x564d37140ed0 msgr2=0x564d371412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.323+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 0x7f28840a6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2890004af0 tx=0x7f2890009800 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.323+0000 7f28a2825740 1 -- 172.21.15.122:0/373639482 shutdown_connections 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.323+0000 7f28a2825740 1 -- 172.21.15.122:0/373639482 wait complete. 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.323+0000 7f28a2825740 1 -- start start 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 Processor -- start 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371467a0 0x564d3714cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d37152a30 con 0x564d371467a0 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d37065e80 con 0x564d371470c0 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d37152bb0 con 0x564d3714fca0 2024-04-02T13:51:06.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.324+0000 7f28a2825740 10 client.0 _ll_register_callbacks cb 0x564d370a1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43120/0 (socket says 172.21.15.122:43120) 2024-04-02T13:51:06.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 -- 172.21.15.122:0/1176218580 learned_addr learned my addr 172.21.15.122:0/1176218580 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289e0fd640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 -- 172.21.15.122:0/1176218580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 msgr2=0x564d3714d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 -- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371467a0 msgr2=0x564d3714cb10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371467a0 0x564d3714cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289d0fb640 1 -- 172.21.15.122:0/1176218580 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d36e887d0 con 0x564d371470c0 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.325+0000 7f289e0fd640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.326+0000 7f28817fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.326+0000 7f289d0fb640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2894010670 tx=0x7f2894010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecbd22ab376d4b66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.326+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894004220 con 0x564d371470c0 2024-04-02T13:51:06.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.326+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f28940043c0 con 0x564d371470c0 2024-04-02T13:51:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.326+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2890003970 con 0x564d371470c0 2024-04-02T13:51:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.327+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f2894010e70 con 0x564d371470c0 2024-04-02T13:51:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.327+0000 7f28a2825740 10 client.29081 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.327+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x564d3715ea30 con 0x564d371470c0 2024-04-02T13:51:06.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.329+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890003970 con 0x564d371470c0 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.329+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f289400f040 con 0x564d371470c0 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 10 client.29081 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 10 client.29081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 20 client.29081 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d3715fef0 0x564d37162360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 4 client.29081 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:06.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d370b7cf0 con 0x564d3715fef0 2024-04-02T13:51:06.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f289d8fc640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d3715fef0 0x564d37162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f289d8fc640 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d3715fef0 0x564d37162360 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a2d379ffc891b690 server_cookie=9ca5a5180b5f6d29 in_seq=0 out_seq=0 2024-04-02T13:51:06.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.330+0000 7f28817fa640 10 client.29081 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:06.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.332+0000 7f28817fa640 1 -- 172.21.15.122:0/1176218580 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d370b7cf0 con 0x564d3715fef0 2024-04-02T13:51:06.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.332+0000 7f28817fa640 10 client.29081 handle_command_reply: tid=0 2024-04-02T13:51:06.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.332+0000 7f28a2825740 1 client.29081 shutdown 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.333+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 msgr2=0x564d3714d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.333+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2894010670 tx=0x7f2894010b30 comp rx=0 tx=0).stop 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.333+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 shutdown_connections 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.333+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d3715fef0 0x564d37162360 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371470c0 0x564d3714d050 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2894010670 tx=0x7f2894010b30 comp rx=0 tx=0).stop 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 --2- 172.21.15.122:0/1176218580 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371467a0 0x564d3714cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 >> 172.21.15.122:0/1176218580 conn(0x564d37140ed0 msgr2=0x564d37142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28817fa640 0 client.29081 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 shutdown_connections 2024-04-02T13:51:06.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.334+0000 7f28a2825740 1 -- 172.21.15.122:0/1176218580 wait complete. 2024-04-02T13:51:06.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.335+0000 7f28a2825740 20 client.29081 trim_cache size 0 max 16384 2024-04-02T13:51:06.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 Processor -- start 2024-04-02T13:51:06.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 -- start start 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d3713c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d3715b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f287c010110 con 0x564d371484b0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f287c011e90 con 0x564d3714fca0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.364+0000 7f28a2825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f287c0107e0 con 0x564d371467a0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49544/0 (socket says 172.21.15.122:49544) 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 -- 172.21.15.122:0/3867673722 learned_addr learned my addr 172.21.15.122:0/3867673722 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d0fb640 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d3713c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d3715b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 -- 172.21.15.122:0/3867673722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 msgr2=0x564d3713c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d3713c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 -- 172.21.15.122:0/3867673722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 msgr2=0x564d3715b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d3715b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.365+0000 7f289d8fc640 1 -- 172.21.15.122:0/3867673722 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d371523b0 con 0x564d371467a0 2024-04-02T13:51:06.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.366+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2888005fd0 tx=0x7f288800d8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa7300e59eb66ede server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.366+0000 7f28837fe640 1 -- 172.21.15.122:0/3867673722 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288809a550 con 0x564d371467a0 2024-04-02T13:51:06.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.367+0000 7f28837fe640 1 -- 172.21.15.122:0/3867673722 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2888014040 con 0x564d371467a0 2024-04-02T13:51:06.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.367+0000 7f28837fe640 1 -- 172.21.15.122:0/3867673722 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880d0230 con 0x564d371467a0 2024-04-02T13:51:06.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- 172.21.15.122:0/3867673722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 msgr2=0x564d3714f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2888005fd0 tx=0x7f288800d8d0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- 172.21.15.122:0/3867673722 shutdown_connections 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d3715b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d3714fca0 0x564d3713c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 --2- 172.21.15.122:0/3867673722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d371467a0 0x564d3714f240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- 172.21.15.122:0/3867673722 >> 172.21.15.122:0/3867673722 conn(0x564d37140ed0 msgr2=0x564d37142300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- 172.21.15.122:0/3867673722 shutdown_connections 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- 172.21.15.122:0/3867673722 wait complete. 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 -- start start 2024-04-02T13:51:06.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.368+0000 7f28a2825740 1 Processor -- start 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371467a0 0x564d371496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d37149c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f287c011700 con 0x564d371484b0 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f287c010f70 con 0x564d371467a0 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f287c00e0d0 con 0x564d3714fca0 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f28a2825740 10 client.0 _ll_register_callbacks cb 0x564d3714fbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f289d0fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d37149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f289d0fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d37149c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59310/0 (socket says 172.21.15.122:59310) 2024-04-02T13:51:06.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.369+0000 7f289d0fb640 1 -- 172.21.15.122:0/3891736001 learned_addr learned my addr 172.21.15.122:0/3891736001 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371467a0 0x564d371496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 -- 172.21.15.122:0/3891736001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371467a0 msgr2=0x564d371496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371467a0 0x564d371496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 -- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 msgr2=0x564d37149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d37149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 -- 172.21.15.122:0/3891736001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d371c54e0 con 0x564d3714fca0 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f289e0fd640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714a140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289004b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ea0689f2fe9a6bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.370+0000 7f28817fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:06.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.371+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890044070 con 0x564d3714fca0 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.371+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2888014040 con 0x564d3714fca0 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.371+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9525+0+0 (secure 0 0 0) 0x7f2890003970 con 0x564d3714fca0 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.371+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f2890067050 con 0x564d3714fca0 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 10 client.28840 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x564d371c6690 con 0x564d3714fca0 2024-04-02T13:51:06.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2888014040 con 0x564d3714fca0 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f289003f070 con 0x564d3714fca0 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 10 client.28840 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 10 client.28840 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 20 client.28840 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d371c6e10 0x564d371c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 4 client.28840 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.372+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d370b2040 con 0x564d371c6e10 2024-04-02T13:51:06.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.373+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d371c6e10 0x564d371c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.373+0000 7f28817fa640 10 client.28840 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.373+0000 7f289d8fc640 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d371c6e10 0x564d371c9230 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=611e94c138f76ea7 server_cookie=605cf27c6af536ee in_seq=0 out_seq=0 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28817fa640 1 -- 172.21.15.122:0/3891736001 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d370b2040 con 0x564d371c6e10 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28817fa640 10 client.28840 handle_command_reply: tid=0 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 client.28840 shutdown 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 msgr2=0x564d3714a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714a140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2890009ca0 tx=0x7f289004b040 comp rx=0 tx=0).stop 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 shutdown_connections 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x564d371c6e10 0x564d371c9230 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d3714fca0 0x564d3714a140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d371484b0 0x564d37149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 --2- 172.21.15.122:0/3891736001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d371467a0 0x564d371496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 >> 172.21.15.122:0/3891736001 conn(0x564d37140ed0 msgr2=0x564d3713ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28817fa640 0 client.28840 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.374+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 shutdown_connections 2024-04-02T13:51:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.375+0000 7f28a2825740 1 -- 172.21.15.122:0/3891736001 wait complete. 2024-04-02T13:51:06.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.375+0000 7f28a2825740 20 client.28840 trim_cache size 0 max 16384 2024-04-02T13:51:06.380 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 msgr2=0x7f28840a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 0x7f28840a6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2890004af0 tx=0x7f2890009800 comp rx=0 tx=0).stop 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 msgr2=0x7f289811e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f288800d950 tx=0x7f288800de10 comp rx=0 tx=0).stop 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 shutdown_connections 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.376+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f28840a3e60 0x7f28840a6320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2898113a50 0x7f2898119b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f289810adc0 0x7f289811eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 --2- 172.21.15.122:0/3618507508 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f289810a4a0 0x7f289811e970 secure :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f288800d950 tx=0x7f288800de10 comp rx=0 tx=0).stop 2024-04-02T13:51:06.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 >> 172.21.15.122:0/3618507508 conn(0x7f289806cbe0 msgr2=0x7f2898070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 shutdown_connections 2024-04-02T13:51:06.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.377+0000 7f289fb60640 1 -- 172.21.15.122:0/3618507508 wait complete. 2024-04-02T13:51:06.399 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:51:06.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.571+0000 7fd599f1c640 1 -- 172.21.15.122:0/215820660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd594108cc0 msgr2=0x7fd59410cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.571+0000 7fd599f1c640 1 --2- 172.21.15.122:0/215820660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd594108cc0 0x7fd59410cd00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009ca0 tx=0x7fd58c02c100 comp rx=0 tx=0).stop 2024-04-02T13:51:06.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 -- 172.21.15.122:0/215820660 shutdown_connections 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 --2- 172.21.15.122:0/215820660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59410d3e0 0x7fd594114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 --2- 172.21.15.122:0/215820660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd594108cc0 0x7fd59410cd00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 --2- 172.21.15.122:0/215820660 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 0x7fd5941086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 -- 172.21.15.122:0/215820660 >> 172.21.15.122:0/215820660 conn(0x7fd594076040 msgr2=0x7fd594078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 -- 172.21.15.122:0/215820660 shutdown_connections 2024-04-02T13:51:06.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.573+0000 7fd599f1c640 1 -- 172.21.15.122:0/215820660 wait complete. 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.574+0000 7fd599f1c640 1 Processor -- start 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.574+0000 7fd599f1c640 1 -- start start 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.574+0000 7fd599f1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 0x7fd5940735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.574+0000 7fd599f1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 0x7fd59406d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd599f1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 0x7fd59406e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 0x7fd59406d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 0x7fd59406d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49562/0 (socket says 172.21.15.122:49562) 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd592ffd640 1 -- 172.21.15.122:0/1626010657 learned_addr learned my addr 172.21.15.122:0/1626010657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd59406bf70 con 0x7fd59406dbe0 2024-04-02T13:51:06.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd5937fe640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 0x7fd5940735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 0x7fd59406e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd59406bdf0 con 0x7fd594108310 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.575+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd59406c0f0 con 0x7fd59410d3e0 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 -- 172.21.15.122:0/1626010657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 msgr2=0x7fd59406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 0x7fd59406d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 -- 172.21.15.122:0/1626010657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 msgr2=0x7fd5940735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 0x7fd5940735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5941b3960 con 0x7fd59406dbe0 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.576+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 0x7fd59406e090 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd58400d680 tx=0x7fd58400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.577+0000 7fd590ff9640 1 -- 172.21.15.122:0/1626010657 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584017070 con 0x7fd59406dbe0 2024-04-02T13:51:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.577+0000 7fd590ff9640 1 -- 172.21.15.122:0/1626010657 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd584009820 con 0x7fd59406dbe0 2024-04-02T13:51:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.578+0000 7fd590ff9640 1 -- 172.21.15.122:0/1626010657 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584004e00 con 0x7fd59406dbe0 2024-04-02T13:51:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.579+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5941b3c50 con 0x7fd59406dbe0 2024-04-02T13:51:06.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.579+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941b4060 con 0x7fd59406dbe0 2024-04-02T13:51:06.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.582+0000 7fd59cbe5740 1 Processor -- start 2024-04-02T13:51:06.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.583+0000 7fd590ff9640 1 -- 172.21.15.122:0/1626010657 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd5840099d0 con 0x7fd59406dbe0 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.583+0000 7fd59cbe5740 1 -- start start 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5595c76097b0 con 0x5595c76a60c0 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5595c75a8570 con 0x5595c76a57a0 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5595c7605760 con 0x5595c76aeca0 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49576/0 (socket says 172.21.15.122:49576) 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd592ffd640 1 -- 172.21.15.122:0/1875528034 learned_addr learned my addr 172.21.15.122:0/1875528034 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd590ff9640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 0x7fd57c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd590ff9640 1 -- 172.21.15.122:0/1626010657 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fd5840d06c0 con 0x7fd59406dbe0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.584+0000 7fd5937fe640 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 -- 172.21.15.122:0/1875528034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 msgr2=0x5595c76a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 -- 172.21.15.122:0/1875528034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 msgr2=0x5595c76ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 -- 172.21.15.122:0/1875528034 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c731d3f0 con 0x5595c76aeca0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 0x7fd57c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.585+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02c710 tx=0x7fd58c02e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9136ce0508a5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.586+0000 7fd593fff640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 0x7fd57c0a6310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd584005c20 tx=0x7fd584005bb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.587+0000 7fd579ffb640 1 -- 172.21.15.122:0/1875528034 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c04a070 con 0x5595c76aeca0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.588+0000 7fd579ffb640 1 -- 172.21.15.122:0/1875528034 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c003e50 con 0x5595c76aeca0 2024-04-02T13:51:06.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.588+0000 7fd579ffb640 1 -- 172.21.15.122:0/1875528034 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c039040 con 0x5595c76aeca0 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1875528034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 msgr2=0x5595c76b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02c710 tx=0x7fd58c02e040 comp rx=0 tx=0).stop 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1875528034 shutdown_connections 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76aeca0 0x5595c76b1090 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd58c02c710 tx=0x7fd58c02e040 comp rx=0 tx=0).stop 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1875528034 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.590+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1875528034 >> 172.21.15.122:0/1875528034 conn(0x5595c769fed0 msgr2=0x5595c76a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.591+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1875528034 shutdown_connections 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.591+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1875528034 wait complete. 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.592+0000 7fd59cbe5740 1 -- start start 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.592+0000 7fd59cbe5740 1 Processor -- start 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76a57a0 0x5595c76abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a60c0 0x5595c76ac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5595c76b1a30 con 0x5595c76ac650 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5595c75c4e80 con 0x5595c76a60c0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5595c76b1bb0 con 0x5595c76a57a0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd59cbe5740 10 client.0 _ll_register_callbacks cb 0x5595c7600aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.594+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a60c0 0x5595c76ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.595+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.595+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59328/0 (socket says 172.21.15.122:59328) 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.595+0000 7fd592ffd640 1 -- 172.21.15.122:0/1370869007 learned_addr learned my addr 172.21.15.122:0/1370869007 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.595+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76a57a0 0x5595c76abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 -- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76a57a0 msgr2=0x5595c76abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76a57a0 0x5595c76abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 -- 172.21.15.122:0/1370869007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a60c0 msgr2=0x5595c76ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a60c0 0x5595c76ac110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 -- 172.21.15.122:0/1370869007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c73e77d0 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd592ffd640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd58c006010 tx=0x7fd58c004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871fe37860a07ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.596+0000 7fd572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c04a070 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fd58c004510 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c003e50 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd58c039040 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd59cbe5740 10 client.20124 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.597+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5595c76cf0b0 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58c003e50 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7fd58c03f030 con 0x5595c76ac650 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 10 client.20124 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 10 client.20124 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:51:06.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 20 client.20124 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c76d0540 0x5595c76d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 4 client.20124 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.598+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c7616cf0 con 0x5595c76d0540 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.600+0000 7fd5937fe640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c76d0540 0x5595c76d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.601+0000 7fd5937fe640 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c76d0540 0x5595c76d2960 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a4f0da156d3ac118 server_cookie=e076cb882c8125b1 in_seq=0 out_seq=0 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.601+0000 7fd572ffd640 10 client.20124 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.605+0000 7fd572ffd640 1 -- 172.21.15.122:0/1370869007 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595c7616cf0 con 0x5595c76d0540 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.605+0000 7fd572ffd640 10 client.20124 handle_command_reply: tid=0 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.605+0000 7fd59cbe5740 1 client.20124 shutdown 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.606+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 msgr2=0x5595c76baf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.606+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fd58c006010 tx=0x7fd58c004170 comp rx=0 tx=0).stop 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 shutdown_connections 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c76d0540 0x5595c76d2960 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76ac650 0x5595c76baf00 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a60c0 0x5595c76ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/1370869007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76a57a0 0x5595c76abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 >> 172.21.15.122:0/1370869007 conn(0x5595c769fed0 msgr2=0x5595c76a1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.607+0000 7fd572ffd640 0 client.20124 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:06.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.608+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 shutdown_connections 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.608+0000 7fd59cbe5740 1 -- 172.21.15.122:0/1370869007 wait complete. 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.608+0000 7fd59cbe5740 20 client.20124 trim_cache size 0 max 16384 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.635+0000 7fd59cbe5740 1 Processor -- start 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.635+0000 7fd59cbe5740 1 -- start start 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c769edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76d8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b58a0 con 0x5595c76a60c0 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b7620 con 0x5595c76a57a0 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b5f70 con 0x5595c76ac650 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59340/0 (socket says 172.21.15.122:59340) 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 -- 172.21.15.122:0/3960394788 learned_addr learned my addr 172.21.15.122:0/3960394788 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 -- 172.21.15.122:0/3960394788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 msgr2=0x5595c76d8790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76d8790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 -- 172.21.15.122:0/3960394788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 msgr2=0x5595c769edb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c769edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.636+0000 7fd5937fe640 1 -- 172.21.15.122:0/3960394788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c76a5ef0 con 0x5595c76a60c0 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.637+0000 7fd5937fe640 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd58800bb40 tx=0x7fd588005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9fd7cc7dbd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.638+0000 7fd5797fa640 1 -- 172.21.15.122:0/3960394788 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58801e070 con 0x5595c76a60c0 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.638+0000 7fd5797fa640 1 -- 172.21.15.122:0/3960394788 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58801adc0 con 0x5595c76a60c0 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.638+0000 7fd5797fa640 1 -- 172.21.15.122:0/3960394788 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588004730 con 0x5595c76a60c0 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 -- 172.21.15.122:0/3960394788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 msgr2=0x5595c76d8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fd58800bb40 tx=0x7fd588005f30 comp rx=0 tx=0).stop 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 -- 172.21.15.122:0/3960394788 shutdown_connections 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76d8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76d8250 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/3960394788 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c769edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 -- 172.21.15.122:0/3960394788 >> 172.21.15.122:0/3960394788 conn(0x5595c769fed0 msgr2=0x5595c769e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 -- 172.21.15.122:0/3960394788 shutdown_connections 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.639+0000 7fd59cbe5740 1 -- 172.21.15.122:0/3960394788 wait complete. 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.640+0000 7fd59cbe5740 1 -- start start 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.640+0000 7fd59cbe5740 1 Processor -- start 2024-04-02T13:51:06.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b6e90 con 0x5595c76a60c0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b6700 con 0x5595c76a57a0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd594082110 con 0x5595c76ac650 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd59cbe5740 10 client.0 _ll_register_callbacks cb 0x7fd58c048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd5937fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59354/0 (socket says 172.21.15.122:59354) 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd5937fe640 1 -- 172.21.15.122:0/4672260 learned_addr learned my addr 172.21.15.122:0/4672260 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.641+0000 7fd592ffd640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd592ffd640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 -- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 msgr2=0x5595c76b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 -- 172.21.15.122:0/4672260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 msgr2=0x5595c76b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 -- 172.21.15.122:0/4672260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c7734da0 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd592ffd640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd5937fe640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd58801a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f1fab1df3cfeac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58801e050 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.642+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fd58801aba0 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.643+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58801adc0 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.643+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fd588004030 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.643+0000 7fd59cbe5740 10 client.20130 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.643+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5595c77360a0 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.643+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd58801adc0 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7fd588029070 con 0x5595c76a60c0 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 10 client.20130 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 10 client.20130 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 20 client.20130 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c7736890 0x5595c7738cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 4 client.20130 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5595c7611040 con 0x5595c7736890 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.644+0000 7fd592ffd640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c7736890 0x5595c7738cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.647+0000 7fd592ffd640 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c7736890 0x5595c7738cb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d1c19b1eb463506 server_cookie=51374e8d4730774f in_seq=0 out_seq=0 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.648+0000 7fd572ffd640 10 client.20130 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.649+0000 7fd572ffd640 1 -- 172.21.15.122:0/4672260 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5595c7611040 con 0x5595c7736890 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.649+0000 7fd572ffd640 10 client.20130 handle_command_reply: tid=0 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.649+0000 7fd59cbe5740 1 client.20130 shutdown 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 msgr2=0x5595c76b2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fd588000c00 tx=0x7fd58801a420 comp rx=0 tx=0).stop 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 shutdown_connections 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5595c76ac650 0x5595c76b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5595c7736890 0x5595c7738cb0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5595c76a60c0 0x5595c76b2360 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 --2- 172.21.15.122:0/4672260 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5595c76a57a0 0x5595c76b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.650+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 >> 172.21.15.122:0/4672260 conn(0x5595c769fed0 msgr2=0x5595c769ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.651+0000 7fd572ffd640 0 client.20130 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.651+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 shutdown_connections 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.651+0000 7fd59cbe5740 1 -- 172.21.15.122:0/4672260 wait complete. 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.651+0000 7fd59cbe5740 20 client.20130 trim_cache size 0 max 16384 2024-04-02T13:51:06.676 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.655+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 msgr2=0x7fd57c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.655+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 0x7fd57c0a6310 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd584005c20 tx=0x7fd584005bb0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.655+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 msgr2=0x7fd59406e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.655+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 0x7fd59406e090 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd58400d680 tx=0x7fd58400db40 comp rx=0 tx=0).stop 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 shutdown_connections 2024-04-02T13:51:06.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd57c0a3e50 0x7fd57c0a6310 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd59406dbe0 0x7fd59406e090 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd59410d3e0 0x7fd59406d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 --2- 172.21.15.122:0/1626010657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd594108310 0x7fd5940735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 >> 172.21.15.122:0/1626010657 conn(0x7fd594076040 msgr2=0x7fd5940773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 shutdown_connections 2024-04-02T13:51:06.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.656+0000 7fd599f1c640 1 -- 172.21.15.122:0/1626010657 wait complete. 2024-04-02T13:51:06.678 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:06 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3351255342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:06.679 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:51:06.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:06 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3351255342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:06.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:06 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3351255342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.841+0000 7f1f6ff32640 1 -- 172.21.15.122:0/3824934484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68114c50 msgr2=0x7f1f68117040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/3824934484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68114c50 0x7f1f68117040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009ca0 tx=0x7f1f60034910 comp rx=0 tx=0).stop 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 -- 172.21.15.122:0/3824934484 shutdown_connections 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/3824934484 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68114c50 0x7f1f68117040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/3824934484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68109420 0x7f1f68114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/3824934484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68108b00 0x7f1f68108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.842+0000 7f1f6ff32640 1 -- 172.21.15.122:0/3824934484 >> 172.21.15.122:0/3824934484 conn(0x7f1f6806cbe0 msgr2=0x7f1f6806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.843+0000 7f1f6ff32640 1 -- 172.21.15.122:0/3824934484 shutdown_connections 2024-04-02T13:51:06.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.843+0000 7f1f6ff32640 1 -- 172.21.15.122:0/3824934484 wait complete. 2024-04-02T13:51:06.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.844+0000 7f1f6ff32640 1 Processor -- start 2024-04-02T13:51:06.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.844+0000 7f1f6ff32640 1 -- start start 2024-04-02T13:51:06.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68109420 0x7f1f681a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 0x7f1f6819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68118480 con 0x7f1f68114c50 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68118300 con 0x7f1f68108b00 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6ff32640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f68118600 con 0x7f1f68109420 2024-04-02T13:51:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43206/0 (socket says 172.21.15.122:43206) 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 -- 172.21.15.122:0/4012521471 learned_addr learned my addr 172.21.15.122:0/4012521471 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68109420 0x7f1f681a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 -- 172.21.15.122:0/4012521471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68109420 msgr2=0x7f1f681a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68109420 0x7f1f681a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 0x7f1f6819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 -- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 msgr2=0x7f1f6819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 0x7f1f6819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.845+0000 7f1f6dcce640 1 -- 172.21.15.122:0/4012521471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6819f060 con 0x7f1f68108b00 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.846+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 0x7f1f6819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.846+0000 7f1f6dcce640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f640099f0 tx=0x7f1f64009eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.846+0000 7f1f5effd640 1 -- 172.21.15.122:0/4012521471 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6401c070 con 0x7f1f68108b00 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.846+0000 7f1f5effd640 1 -- 172.21.15.122:0/4012521471 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6400cc60 con 0x7f1f68108b00 2024-04-02T13:51:06.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.847+0000 7f1f5effd640 1 -- 172.21.15.122:0/4012521471 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64016720 con 0x7f1f68108b00 2024-04-02T13:51:06.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.847+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6819f350 con 0x7f1f68108b00 2024-04-02T13:51:06.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.847+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f681ab610 con 0x7f1f68108b00 2024-04-02T13:51:06.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.851+0000 7f1f72bf9740 1 Processor -- start 2024-04-02T13:51:06.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.851+0000 7f1f72bf9740 1 -- start start 2024-04-02T13:51:06.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.851+0000 7f1f5effd640 1 -- 172.21.15.122:0/4012521471 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1f6401f070 con 0x7f1f68108b00 2024-04-02T13:51:06.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43222/0 (socket says 172.21.15.122:43222) 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f6dcce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dfeae007b0 con 0x55dfeae9d0c0 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/4075738348 learned_addr learned my addr 172.21.15.122:0/4075738348 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dfead9f570 con 0x55dfeae9c7a0 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.852+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dfeadfc760 con 0x55dfeaea5ca0 2024-04-02T13:51:06.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/4075738348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 msgr2=0x55dfeaea8090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea8090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/4075738348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 msgr2=0x55dfeaea5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/4075738348 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfeab143f0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009430 tx=0x7f1f60039b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9d2665881343e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.853+0000 7f1f5effd640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 0x7f1f540a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.854+0000 7f1f6dcce640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 0x7f1f540a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.854+0000 7f1f53fff640 1 -- 172.21.15.122:0/4075738348 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003e070 con 0x55dfeae9c7a0 2024-04-02T13:51:06.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.854+0000 7f1f53fff640 1 -- 172.21.15.122:0/4075738348 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f60003e50 con 0x55dfeae9c7a0 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.854+0000 7f1f53fff640 1 -- 172.21.15.122:0/4075738348 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003a910 con 0x55dfeae9c7a0 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.854+0000 7f1f5effd640 1 -- 172.21.15.122:0/4012521471 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1f640d08e0 con 0x7f1f68108b00 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.855+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 msgr2=0x55dfeae9cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.855+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009430 tx=0x7f1f60039b90 comp rx=0 tx=0).stop 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f6dcce640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 0x7f1f540a6320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dfeae9c270 tx=0x7f1f64017dd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 shutdown_connections 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/4075738348 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9cb80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.856+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 >> 172.21.15.122:0/4075738348 conn(0x55dfeae96ed0 msgr2=0x55dfeae972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.857+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 shutdown_connections 2024-04-02T13:51:06.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.857+0000 7f1f72bf9740 1 -- 172.21.15.122:0/4075738348 wait complete. 2024-04-02T13:51:06.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.857+0000 7f1f72bf9740 1 -- start start 2024-04-02T13:51:06.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 Processor -- start 2024-04-02T13:51:06.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dfeaea8a30 con 0x55dfeae9c7a0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55dfeadbbe80 con 0x55dfeae9d0c0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dfeaea8bb0 con 0x55dfeaea5ca0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f72bf9740 10 client.0 _ll_register_callbacks cb 0x55dfeadf7aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59396/0 (socket says 172.21.15.122:59396) 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/2768348295 learned_addr learned my addr 172.21.15.122:0/2768348295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.858+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/2768348295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 msgr2=0x55dfeaea3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9d0c0 msgr2=0x55dfeaea2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/2768348295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfeabde7d0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.859+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1f60039b30 tx=0x7f1f600398f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ef0afdc7bcf8e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003e050 con 0x55dfeae9c7a0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1f6003aab0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f60003e50 con 0x55dfeae9c7a0 2024-04-02T13:51:06.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1f6003acb0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.860+0000 7f1f72bf9740 10 client.20136 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.861+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55dfeaea8a30 con 0x55dfeae9c7a0 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.861+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f6003ae50 con 0x55dfeae9c7a0 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.861+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f1f6004c020 con 0x55dfeae9c7a0 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 10 client.20136 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 10 client.20136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 20 client.20136 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaeb5a80 0x55dfeaeb7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 4 client.20136 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:06.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f60003e50 con 0x55dfeaeb5a80 2024-04-02T13:51:06.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.862+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaeb5a80 0x55dfeaeb7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.863+0000 7f1f517fa640 10 client.20136 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.863+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaeb5a80 0x55dfeaeb7e70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aae7850a60039108 server_cookie=af8b6042908e9bfa in_seq=0 out_seq=0 2024-04-02T13:51:06.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.868+0000 7f1f517fa640 1 -- 172.21.15.122:0/2768348295 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f1f60003e50 con 0x55dfeaeb5a80 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.868+0000 7f1f517fa640 10 client.20136 handle_command_reply: tid=0 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.868+0000 7f1f72bf9740 1 client.20136 shutdown 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.868+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 msgr2=0x55dfeaea5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.868+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1f60039b30 tx=0x7f1f600398f0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 shutdown_connections 2024-04-02T13:51:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaeb5a80 0x55dfeaeb7e70 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/2768348295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9c7a0 0x55dfeaea5760 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 >> 172.21.15.122:0/2768348295 conn(0x55dfeae96ed0 msgr2=0x55dfeae98c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f517fa640 0 client.20136 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:06.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 shutdown_connections 2024-04-02T13:51:06.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 1 -- 172.21.15.122:0/2768348295 wait complete. 2024-04-02T13:51:06.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.869+0000 7f1f72bf9740 20 client.20136 trim_cache size 0 max 16384 2024-04-02T13:51:06.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.897+0000 7f1f72bf9740 1 Processor -- start 2024-04-02T13:51:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.900+0000 7f1f72bf9740 1 -- start start 2024-04-02T13:51:06.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.900+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeaeb1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.900+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f72bf9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b5900 con 0x55dfeae9d0c0 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f6d4cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f6d4cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59412/0 (socket says 172.21.15.122:59412) 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f6d4cd640 1 -- 172.21.15.122:0/171556786 learned_addr learned my addr 172.21.15.122:0/171556786 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b7680 con 0x55dfeae9c7a0 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.901+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b5fd0 con 0x55dfeaea5ca0 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.902+0000 7f1f6d4cd640 1 -- 172.21.15.122:0/171556786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 msgr2=0x55dfeaea5220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:06.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.902+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea5220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.902+0000 7f1f6d4cd640 1 -- 172.21.15.122:0/171556786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 msgr2=0x55dfeaeb1920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.902+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeaeb1920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.902+0000 7f1f6d4cd640 1 -- 172.21.15.122:0/171556786 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfeae9cef0 con 0x55dfeae9d0c0 2024-04-02T13:51:06.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.903+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x55dfeae98060 tx=0x7f1f5800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6163d27e3ce958 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.905+0000 7f1f537fe640 1 -- 172.21.15.122:0/171556786 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58021070 con 0x55dfeae9d0c0 2024-04-02T13:51:06.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.905+0000 7f1f537fe640 1 -- 172.21.15.122:0/171556786 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f58015a80 con 0x55dfeae9d0c0 2024-04-02T13:51:06.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.905+0000 7f1f537fe640 1 -- 172.21.15.122:0/171556786 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58013b60 con 0x55dfeae9d0c0 2024-04-02T13:51:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.907+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 msgr2=0x55dfeaea1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.907+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x55dfeae98060 tx=0x7f1f5800b6a0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.908+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 shutdown_connections 2024-04-02T13:51:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.908+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeaea5ca0 0x55dfeaea5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.908+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeaea1af0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.908+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/171556786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeae9c7a0 0x55dfeaeb1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.908+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 >> 172.21.15.122:0/171556786 conn(0x55dfeae96ed0 msgr2=0x55dfeaea13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.910+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 shutdown_connections 2024-04-02T13:51:06.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.911+0000 7f1f72bf9740 1 -- 172.21.15.122:0/171556786 wait complete. 2024-04-02T13:51:06.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.912+0000 7f1f72bf9740 1 -- start start 2024-04-02T13:51:06.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.912+0000 7f1f72bf9740 1 Processor -- start 2024-04-02T13:51:06.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.912+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.912+0000 7f1f72bf9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeae9f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f6e4cf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49632/0 (socket says 172.21.15.122:49632) 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/1583565569 learned_addr learned my addr 172.21.15.122:0/1583565569 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeae9f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeaea5ca0 0x55dfeae9fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b6ef0 con 0x55dfeae9d0c0 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b6760 con 0x55dfeaea5ca0 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.913+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f681b3900 con 0x55dfeae9c7a0 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f72bf9740 10 client.0 _ll_register_callbacks cb 0x7f1f60009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeaea5ca0 0x55dfeae9fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeaea5ca0 msgr2=0x55dfeae9fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeaea5ca0 0x55dfeae9fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/1583565569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 msgr2=0x55dfeae9f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:06.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeae9f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.914+0000 7f1f6e4cf640 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfeaead0f0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.915+0000 7f1f6e4cf640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009a60 tx=0x7f1f60034910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5401938896f4dbb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:06.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.918+0000 7f1f517fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:06.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.918+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6003e070 con 0x55dfeae9c7a0 2024-04-02T13:51:06.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.918+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1f60044430 con 0x55dfeae9c7a0 2024-04-02T13:51:06.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.918+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f58015a80 con 0x55dfeae9c7a0 2024-04-02T13:51:06.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.919+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1f6003ae50 con 0x55dfeae9c7a0 2024-04-02T13:51:06.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.920+0000 7f1f72bf9740 10 client.28867 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:06.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.920+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55dfeaeae3b0 con 0x55dfeae9c7a0 2024-04-02T13:51:06.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f58015a80 con 0x55dfeae9c7a0 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f1f60056070 con 0x55dfeae9c7a0 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 10 client.28867 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 10 client.28867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 20 client.28867 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaf2cf90 0x55dfeaf2f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 4 client.28867 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.922+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfeae0dcf0 con 0x55dfeaf2cf90 2024-04-02T13:51:06.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.923+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaf2cf90 0x55dfeaf2f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:06.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.924+0000 7f1f6d4cd640 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaf2cf90 0x55dfeaf2f3b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7c1690f76ae063f server_cookie=78dde8457e0278d0 in_seq=0 out_seq=0 2024-04-02T13:51:06.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.924+0000 7f1f517fa640 10 client.28867 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:06.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.924+0000 7f1f517fa640 1 -- 172.21.15.122:0/1583565569 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfeae0dcf0 con 0x55dfeaf2cf90 2024-04-02T13:51:06.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.924+0000 7f1f517fa640 10 client.28867 handle_command_reply: tid=0 2024-04-02T13:51:06.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.925+0000 7f1f72bf9740 1 client.28867 shutdown 2024-04-02T13:51:06.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.925+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 msgr2=0x55dfeae9ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.925+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f60009a60 tx=0x7f1f60034910 comp rx=0 tx=0).stop 2024-04-02T13:51:06.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.926+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 shutdown_connections 2024-04-02T13:51:06.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55dfeaf2cf90 0x55dfeaf2f3b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55dfeaea5ca0 0x55dfeae9fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dfeae9d0c0 0x55dfeae9f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 --2- 172.21.15.122:0/1583565569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dfeae9c7a0 0x55dfeae9ef60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 >> 172.21.15.122:0/1583565569 conn(0x55dfeae96ed0 msgr2=0x55dfeae98090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f517fa640 0 client.28867 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:06.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.927+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 shutdown_connections 2024-04-02T13:51:06.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.928+0000 7f1f72bf9740 1 -- 172.21.15.122:0/1583565569 wait complete. 2024-04-02T13:51:06.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.928+0000 7f1f72bf9740 20 client.28867 trim_cache size 0 max 16384 2024-04-02T13:51:06.933 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:06.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.930+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 msgr2=0x7f1f540a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.930+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 0x7f1f540a6320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dfeae9c270 tx=0x7f1f64017dd0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.930+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 msgr2=0x7f1f681a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:06.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.930+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f640099f0 tx=0x7f1f64009eb0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 shutdown_connections 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f540a3e60 0x7f1f540a6320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1f68114c50 0x7f1f6819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f68109420 0x7f1f681a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 --2- 172.21.15.122:0/4012521471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f68108b00 0x7f1f681a4690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 >> 172.21.15.122:0/4012521471 conn(0x7f1f6806cbe0 msgr2=0x7f1f680707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:06.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 shutdown_connections 2024-04-02T13:51:06.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:06.931+0000 7f1f6ff32640 1 -- 172.21.15.122:0/4012521471 wait complete. 2024-04-02T13:51:06.952 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:51:07.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 -- 172.21.15.122:0/3358223328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac103f10 msgr2=0x7f70ac1042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3358223328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac103f10 0x7f70ac1042f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009900 tx=0x7f70a802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:07.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 -- 172.21.15.122:0/3358223328 shutdown_connections 2024-04-02T13:51:07.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3358223328 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac108fe0 0x7f70ac110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3358223328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac1048c0 0x7f70ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3358223328 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac103f10 0x7f70ac1042f0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 -- 172.21.15.122:0/3358223328 >> 172.21.15.122:0/3358223328 conn(0x7f70ac0ff750 msgr2=0x7f70ac101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 -- 172.21.15.122:0/3358223328 shutdown_connections 2024-04-02T13:51:07.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.099+0000 7f70b38c5640 1 -- 172.21.15.122:0/3358223328 wait complete. 2024-04-02T13:51:07.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.100+0000 7f70b38c5640 1 Processor -- start 2024-04-02T13:51:07.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.100+0000 7f70b38c5640 1 -- start start 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b38c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b38c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b38c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 0x7f70ac19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b1661640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b1661640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43256/0 (socket says 172.21.15.122:43256) 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59442/0 (socket says 172.21.15.122:59442) 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b1661640 1 -- 172.21.15.122:0/3723512162 learned_addr learned my addr 172.21.15.122:0/3723512162 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac114090 con 0x7f70ac108fe0 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac113f10 con 0x7f70ac103f10 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.101+0000 7f70b1e62640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 0x7f70ac19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac114210 con 0x7f70ac19ef10 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1661640 1 -- 172.21.15.122:0/3723512162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 msgr2=0x7f70ac19f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1661640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 0x7f70ac19f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1661640 1 -- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 msgr2=0x7f70ac19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1661640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1661640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac19fa00 con 0x7f70ac103f10 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b0e60640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.102+0000 7f70b1e62640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 0x7f70ac19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.103+0000 7f70b1661640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70a8006010 tx=0x7f70a802cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.103+0000 7f70a27fc640 1 -- 172.21.15.122:0/3723512162 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8041070 con 0x7f70ac103f10 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.103+0000 7f70a27fc640 1 -- 172.21.15.122:0/3723512162 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8004450 con 0x7f70ac103f10 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.103+0000 7f70a27fc640 1 -- 172.21.15.122:0/3723512162 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003e00 con 0x7f70ac103f10 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.103+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac19fc90 con 0x7f70ac103f10 2024-04-02T13:51:07.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.104+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac1a7400 con 0x7f70ac103f10 2024-04-02T13:51:07.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.107+0000 7f70a27fc640 1 -- 172.21.15.122:0/3723512162 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f70a8044070 con 0x7f70ac103f10 2024-04-02T13:51:07.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.108+0000 7f70b6587740 1 Processor -- start 2024-04-02T13:51:07.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.108+0000 7f70a27fc640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 0x7f70980a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.108+0000 7f70a27fc640 1 -- 172.21.15.122:0/3723512162 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f70a80f24a0 con 0x7f70ac103f10 2024-04-02T13:51:07.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b1e62640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 0x7f70980a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 -- start start 2024-04-02T13:51:07.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b172390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b17d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f28b0cd7b0 con 0x55f28b171fb0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f28b06c570 con 0x55f28b1728d0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b6587740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f28b0c9760 con 0x55f28b17b4b0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b1661640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b172390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b1661640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b172390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59446/0 (socket says 172.21.15.122:59446) 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43268/0 (socket says 172.21.15.122:43268) 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.109+0000 7f70b1661640 1 -- 172.21.15.122:0/1797912031 learned_addr learned my addr 172.21.15.122:0/1797912031 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b1e62640 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b17d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b1e62640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 0x7f70980a6330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005fd0 tx=0x7f70a4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 -- 172.21.15.122:0/1797912031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 msgr2=0x55f28b17d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b17d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 -- 172.21.15.122:0/1797912031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 msgr2=0x55f28b172390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b172390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 -- 172.21.15.122:0/1797912031 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28ade13f0 con 0x55f28b1728d0 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f70ac06b150 tx=0x7f709c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=261f6ac118b64931 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70977fe640 1 -- 172.21.15.122:0/1797912031 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c004310 con 0x55f28b1728d0 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70977fe640 1 -- 172.21.15.122:0/1797912031 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c0044b0 con 0x55f28b1728d0 2024-04-02T13:51:07.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.110+0000 7f70977fe640 1 -- 172.21.15.122:0/1797912031 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0035f0 con 0x55f28b1728d0 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 -- 172.21.15.122:0/1797912031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 msgr2=0x55f28b17af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f70ac06b150 tx=0x7f709c0118d0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 -- 172.21.15.122:0/1797912031 shutdown_connections 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b17d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b1728d0 0x55f28b17af70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 --2- 172.21.15.122:0/1797912031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b172390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.111+0000 7f70b6587740 1 -- 172.21.15.122:0/1797912031 >> 172.21.15.122:0/1797912031 conn(0x55f28b16c6e0 msgr2=0x55f28b16cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.112+0000 7f70b6587740 1 -- 172.21.15.122:0/1797912031 shutdown_connections 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.112+0000 7f70b6587740 1 -- 172.21.15.122:0/1797912031 wait complete. 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.112+0000 7f70b6587740 1 -- start start 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.112+0000 7f70b6587740 1 Processor -- start 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b178340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b178dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f28b17e2a0 con 0x55f28b1728d0 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f28b088e80 con 0x55f28b171fb0 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f28b17e420 con 0x55f28b17b4b0 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b6587740 10 client.0 _ll_register_callbacks cb 0x55f28b0c4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59458/0 (socket says 172.21.15.122:59458) 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 -- 172.21.15.122:0/4045015355 learned_addr learned my addr 172.21.15.122:0/4045015355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b1e62640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b178dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 -- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 msgr2=0x55f28b178dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b178dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 -- 172.21.15.122:0/4045015355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 msgr2=0x55f28b178340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b178340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 -- 172.21.15.122:0/4045015355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28aeab7d0 con 0x55f28b1728d0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f70b0e60640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f709c01b040 tx=0x7f709c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5985884d378ac4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f7094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.113+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c029070 con 0x55f28b1728d0 2024-04-02T13:51:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.114+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f709c003a00 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.114+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f709c0044b0 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f709c033050 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 10 client.20154 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f28b18a3d0 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f709c0044b0 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f709c02e440 con 0x55f28b1728d0 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 10 client.20154 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 10 client.20154 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 20 client.20154 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b18bcd0 0x55f28b18e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 4 client.20154 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.115+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f28b0dacf0 con 0x55f28b18bcd0 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.116+0000 7f70b1661640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b18bcd0 0x55f28b18e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.116+0000 7f70b1661640 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b18bcd0 0x55f28b18e170 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=698fcb4818984f56 server_cookie=2494aabd2658ba74 in_seq=0 out_seq=0 2024-04-02T13:51:07.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.116+0000 7f7094ff9640 10 client.20154 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.118+0000 7f7094ff9640 1 -- 172.21.15.122:0/4045015355 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f28b0dacf0 con 0x55f28b18bcd0 2024-04-02T13:51:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.118+0000 7f7094ff9640 10 client.20154 handle_command_reply: tid=0 2024-04-02T13:51:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.118+0000 7f70b6587740 1 client.20154 shutdown 2024-04-02T13:51:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.118+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 msgr2=0x55f28b178880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.118+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f709c01b040 tx=0x7f709c0048f0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 shutdown_connections 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b18bcd0 0x55f28b18e170 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b17b4b0 0x55f28b178dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b1728d0 0x55f28b178880 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 --2- 172.21.15.122:0/4045015355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b178340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 >> 172.21.15.122:0/4045015355 conn(0x55f28b16c6e0 msgr2=0x55f28b16e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f7094ff9640 0 client.20154 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 shutdown_connections 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 1 -- 172.21.15.122:0/4045015355 wait complete. 2024-04-02T13:51:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.119+0000 7f70b6587740 20 client.20154 trim_cache size 0 max 16384 2024-04-02T13:51:07.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.144+0000 7f70b6587740 1 Processor -- start 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.144+0000 7f70b6587740 1 -- start start 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b17aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b187190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ad030 con 0x55f28b17b4b0 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b1e62640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b1e62640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59474/0 (socket says 172.21.15.122:59474) 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b1e62640 1 -- 172.21.15.122:0/2952474873 learned_addr learned my addr 172.21.15.122:0/2952474873 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b0e60640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b187190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1aedb0 con 0x55f28b171fb0 2024-04-02T13:51:07.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.145+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ad700 con 0x55f28b1728d0 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 -- 172.21.15.122:0/2952474873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 msgr2=0x55f28b187190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b187190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 -- 172.21.15.122:0/2952474873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 msgr2=0x55f28b17aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b17aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 -- 172.21.15.122:0/2952474873 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f28b172700 con 0x55f28b17b4b0 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f70b1e62640 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f70a4003d50 tx=0x7f70a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0be04abb38340 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f7096ffd640 1 -- 172.21.15.122:0/2952474873 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a400fde0 con 0x55f28b17b4b0 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.146+0000 7f7096ffd640 1 -- 172.21.15.122:0/2952474873 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a402d070 con 0x55f28b17b4b0 2024-04-02T13:51:07.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.147+0000 7f7096ffd640 1 -- 172.21.15.122:0/2952474873 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4028a30 con 0x55f28b17b4b0 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.147+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 msgr2=0x55f28b193a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.147+0000 7f70b6587740 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f70a4003d50 tx=0x7f70a4014000 comp rx=0 tx=0).stop 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 shutdown_connections 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b17b4b0 0x55f28b193a80 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b187190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 --2- 172.21.15.122:0/2952474873 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b171fb0 0x55f28b17aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 >> 172.21.15.122:0/2952474873 conn(0x55f28b16c6e0 msgr2=0x55f28b16cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 shutdown_connections 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 -- 172.21.15.122:0/2952474873 wait complete. 2024-04-02T13:51:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 -- start start 2024-04-02T13:51:07.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.148+0000 7f70b6587740 1 Processor -- start 2024-04-02T13:51:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b6587740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b6587740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b17e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b1661640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b1661640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59486/0 (socket says 172.21.15.122:59486) 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b1661640 1 -- 172.21.15.122:0/1424820020 learned_addr learned my addr 172.21.15.122:0/1424820020 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.149+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b17b4b0 0x55f28b17eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b17e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ae620 con 0x55f28b171fb0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ade90 con 0x55f28b17b4b0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac1ab050 con 0x55f28b1728d0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b6587740 10 client.0 _ll_register_callbacks cb 0x7f70a8005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b1e62640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b17b4b0 0x55f28b17eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 -- 172.21.15.122:0/1424820020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b17b4b0 msgr2=0x55f28b17eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b17b4b0 0x55f28b17eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 -- 172.21.15.122:0/1424820020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 msgr2=0x55f28b1838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f28b17f190 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b1661640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f70b0e60640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b17e660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f709c02e890 tx=0x7f709c009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b399c9859c9ab29 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.150+0000 7f7094ff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c029040 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f709c003a00 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a402d070 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f709c003c40 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f70b6587740 10 client.28885 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.151+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55f28b1f10c0 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a402d070 con 0x55f28b1728d0 2024-04-02T13:51:07.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f709c0050a0 con 0x55f28b1728d0 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 10 client.28885 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 10 client.28885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 20 client.28885 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b1f18f0 0x55f28b1f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 4 client.28885 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f28b0d5040 con 0x55f28b1f18f0 2024-04-02T13:51:07.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.152+0000 7f70b1661640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b1f18f0 0x55f28b1f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f7094ff9640 10 client.28885 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f70b1661640 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b1f18f0 0x55f28b1f3d10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c505ed46fc3de46 server_cookie=5367b689ae647c65 in_seq=0 out_seq=0 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f7094ff9640 1 -- 172.21.15.122:0/1424820020 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f28b0d5040 con 0x55f28b1f18f0 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f7094ff9640 10 client.28885 handle_command_reply: tid=0 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f70b6587740 1 client.28885 shutdown 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 msgr2=0x55f28b17e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.153+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b17e660 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f709c02e890 tx=0x7f709c009e20 comp rx=0 tx=0).stop 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 shutdown_connections 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55f28b1f18f0 0x55f28b1f3d10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f7094ff9640 0 client.28885 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f28b17b4b0 0x55f28b17eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f28b1728d0 0x55f28b17e660 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 --2- 172.21.15.122:0/1424820020 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f28b171fb0 0x55f28b1838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 >> 172.21.15.122:0/1424820020 conn(0x55f28b16c6e0 msgr2=0x55f28b176c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 shutdown_connections 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 1 -- 172.21.15.122:0/1424820020 wait complete. 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.154+0000 7f70b6587740 20 client.28885 trim_cache size 0 max 16384 2024-04-02T13:51:07.161 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 msgr2=0x7f70980a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 0x7f70980a6330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f70a4005fd0 tx=0x7f70a4009500 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 msgr2=0x7f70ac19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70a8006010 tx=0x7f70a802cb30 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 shutdown_connections 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f70980a3e70 0x7f70980a6330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac19ef10 0x7f70ac19f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70ac108fe0 0x7f70ac19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 --2- 172.21.15.122:0/3723512162 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f70ac103f10 0x7f70ac19a650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 >> 172.21.15.122:0/3723512162 conn(0x7f70ac0ff750 msgr2=0x7f70ac100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 shutdown_connections 2024-04-02T13:51:07.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.156+0000 7f70b38c5640 1 -- 172.21.15.122:0/3723512162 wait complete. 2024-04-02T13:51:07.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:51:07.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 79769 28.21 MB/sec execute 27 sec latency 334.695 ms 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 -- 172.21.15.122:0/1376064345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8101f30 msgr2=0x7fc3f810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/1376064345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8101f30 0x7fc3f810d890 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009cd0 tx=0x7fc3f002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 -- 172.21.15.122:0/1376064345 shutdown_connections 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/1376064345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8101f30 0x7fc3f810d890 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/1376064345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101570 0x7fc3f81019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/1376064345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8105a00 0x7fc3f8105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 -- 172.21.15.122:0/1376064345 >> 172.21.15.122:0/1376064345 conn(0x7fc3f8076b50 msgr2=0x7fc3f80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 -- 172.21.15.122:0/1376064345 shutdown_connections 2024-04-02T13:51:07.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.330+0000 7fc3fee3c640 1 -- 172.21.15.122:0/1376064345 wait complete. 2024-04-02T13:51:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 Processor -- start 2024-04-02T13:51:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 -- start start 2024-04-02T13:51:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 0x7fc3f8196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101f30 0x7fc3f8197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 0x7fc3f8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8110e00 con 0x7fc3f8105a00 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8110c80 con 0x7fc3f8101570 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.331+0000 7fc3fee3c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8110f80 con 0x7fc3f8101f30 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 0x7fc3f8196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 0x7fc3f8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 0x7fc3f8196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43302/0 (socket says 172.21.15.122:43302) 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/2643961322 learned_addr learned my addr 172.21.15.122:0/2643961322 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101f30 0x7fc3f8197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/2643961322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101f30 msgr2=0x7fc3f8197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101f30 0x7fc3f8197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/2643961322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 msgr2=0x7fc3f8196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 0x7fc3f8196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/2643961322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f81a67b0 con 0x7fc3f8105a00 2024-04-02T13:51:07.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.332+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 0x7fc3f8197a50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0037040 tx=0x7fc3f0032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.333+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/2643961322 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0041020 con 0x7fc3f8105a00 2024-04-02T13:51:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.333+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f81a6a40 con 0x7fc3f8105a00 2024-04-02T13:51:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.333+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81a6f50 con 0x7fc3f8105a00 2024-04-02T13:51:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.333+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/2643961322 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f0034cd0 con 0x7fc3f8105a00 2024-04-02T13:51:07.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.333+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/2643961322 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0039040 con 0x7fc3f8105a00 2024-04-02T13:51:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.335+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/2643961322 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc3f0044020 con 0x7fc3f8105a00 2024-04-02T13:51:07.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.336+0000 7fc3f5ffb640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 0x7fc3e00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.336+0000 7fc401a27740 1 Processor -- start 2024-04-02T13:51:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.336+0000 7fc3f5ffb640 1 -- 172.21.15.122:0/2643961322 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fc3f00f1f50 con 0x7fc3f8105a00 2024-04-02T13:51:07.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.336+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 0x7fc3e00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.336+0000 7fc401a27740 1 -- start start 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1807000c0 0x55c180708760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c180708ca0 0x55c18070b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c1806637b0 con 0x55c1806ff7a0 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c180602570 con 0x55c180708ca0 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc401a27740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c18065f760 con 0x55c1807000c0 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 0x7fc3e00a6260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009c90 tx=0x7fc3e8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59512/0 (socket says 172.21.15.122:59512) 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/339079011 learned_addr learned my addr 172.21.15.122:0/339079011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c180708ca0 0x55c18070b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1807000c0 0x55c180708760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/339079011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1807000c0 msgr2=0x55c180708760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1807000c0 0x55c180708760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/339079011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c180708ca0 msgr2=0x55c18070b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c180708ca0 0x55c18070b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.337+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/339079011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1803773f0 con 0x55c1806ff7a0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.338+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009fc0 tx=0x7fc3e8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d07c4d1ddfff58 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.338+0000 7fc3deffd640 1 -- 172.21.15.122:0/339079011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e80265f0 con 0x55c1806ff7a0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.338+0000 7fc3deffd640 1 -- 172.21.15.122:0/339079011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e8026c70 con 0x55c1806ff7a0 2024-04-02T13:51:07.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.338+0000 7fc3deffd640 1 -- 172.21.15.122:0/339079011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800ed40 con 0x55c1806ff7a0 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.339+0000 7fc401a27740 1 -- 172.21.15.122:0/339079011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 msgr2=0x55c1806ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.339+0000 7fc401a27740 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009fc0 tx=0x7fc3e8014000 comp rx=0 tx=0).stop 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 -- 172.21.15.122:0/339079011 shutdown_connections 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c180708ca0 0x55c18070b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1807000c0 0x55c180708760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 --2- 172.21.15.122:0/339079011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1806ffb80 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 -- 172.21.15.122:0/339079011 >> 172.21.15.122:0/339079011 conn(0x55c1806f9ed0 msgr2=0x55c1806fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 -- 172.21.15.122:0/339079011 shutdown_connections 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 -- 172.21.15.122:0/339079011 wait complete. 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 -- start start 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.340+0000 7fc401a27740 1 Processor -- start 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc401a27740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc401a27740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180705a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc401a27740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c1807061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180705a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc3f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180705a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43332/0 (socket says 172.21.15.122:43332) 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc3f7fff640 1 -- 172.21.15.122:0/3389162149 learned_addr learned my addr 172.21.15.122:0/3389162149 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c18070ba90 con 0x55c1806ff7a0 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c1807061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c18061ee80 con 0x55c1807000c0 2024-04-02T13:51:07.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c18070bc10 con 0x55c180708ca0 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.341+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59518/0 (socket says 172.21.15.122:59518) 2024-04-02T13:51:07.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc401a27740 10 client.0 _ll_register_callbacks cb 0x55c18065aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 msgr2=0x55c180705a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180705a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/3389162149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 msgr2=0x55c1807086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1804417d0 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c1807061b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00ef370 tx=0x7fc3f00bd520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad31e446183e8d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00bdbc0 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fc3f00c63a0 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.342+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3e8026c70 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc3f00c1f70 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 10 client.28897 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c18070ba90 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e8026c70 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7fc3f003c030 con 0x55c180708ca0 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 10 client.28897 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:07.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 10 client.28897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 20 client.28897 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180721870 0x55c180723cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 4 client.28897 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c180670cf0 con 0x55c180721870 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.343+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180721870 0x55c180723cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.344+0000 7fc3d7fff640 10 client.28897 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:07.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.344+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180721870 0x55c180723cd0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed72a8664f675d9 server_cookie=c57189e22c63dc58 in_seq=0 out_seq=0 2024-04-02T13:51:07.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc3d7fff640 1 -- 172.21.15.122:0/3389162149 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c180670cf0 con 0x55c180721870 2024-04-02T13:51:07.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc3d7fff640 10 client.28897 handle_command_reply: tid=0 2024-04-02T13:51:07.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 client.28897 shutdown 2024-04-02T13:51:07.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 msgr2=0x55c1807061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c1807061b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00ef370 tx=0x7fc3f00bd520 comp rx=0 tx=0).stop 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 shutdown_connections 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180721870 0x55c180723cd0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c1807061b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180705a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc3d7fff640 0 client.28897 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 --2- 172.21.15.122:0/3389162149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1806ff7a0 0x55c1807086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 >> 172.21.15.122:0/3389162149 conn(0x55c1806f9ed0 msgr2=0x55c1806fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 shutdown_connections 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.346+0000 7fc401a27740 1 -- 172.21.15.122:0/3389162149 wait complete. 2024-04-02T13:51:07.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.347+0000 7fc401a27740 20 client.28897 trim_cache size 0 max 16384 2024-04-02T13:51:07.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.362+0000 7fc401a27740 1 Processor -- start 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.362+0000 7fc401a27740 1 -- start start 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.362+0000 7fc401a27740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 0x55c180704a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc401a27740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 0x55c1807295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc401a27740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c180729ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 0x55c180704a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc401a27740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40100d0 con 0x55c1807000c0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc401a27740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4011e50 con 0x55c1806ff7a0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc401a27740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40107a0 con 0x55c180708ca0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 0x55c180704a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43334/0 (socket says 172.21.15.122:43334) 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/2694442721 learned_addr learned my addr 172.21.15.122:0/2694442721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 0x55c1807295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c180729ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 -- 172.21.15.122:0/2694442721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 msgr2=0x55c180729ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c180729ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 -- 172.21.15.122:0/2694442721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 msgr2=0x55c180704a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 0x55c180704a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.363+0000 7fc3f7fff640 1 -- 172.21.15.122:0/2694442721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1806ffef0 con 0x55c1807000c0 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.364+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 0x55c1807295a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55c1807052d0 tx=0x7fc3ec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48993f176796945c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.364+0000 7fc3de7fc640 1 -- 172.21.15.122:0/2694442721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec022070 con 0x55c1807000c0 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.364+0000 7fc3de7fc640 1 -- 172.21.15.122:0/2694442721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ec0035f0 con 0x55c1807000c0 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.364+0000 7fc3de7fc640 1 -- 172.21.15.122:0/2694442721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec01d6c0 con 0x55c1807000c0 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- 172.21.15.122:0/2694442721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 msgr2=0x55c1807295a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 0x55c1807295a0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55c1807052d0 tx=0x7fc3ec005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- 172.21.15.122:0/2694442721 shutdown_connections 2024-04-02T13:51:07.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c180708ca0 0x55c180729ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c1807000c0 0x55c1807295a0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 --2- 172.21.15.122:0/2694442721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1806ff7a0 0x55c180704a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- 172.21.15.122:0/2694442721 >> 172.21.15.122:0/2694442721 conn(0x55c1806f9ed0 msgr2=0x55c180704340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- 172.21.15.122:0/2694442721 shutdown_connections 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- 172.21.15.122:0/2694442721 wait complete. 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.365+0000 7fc401a27740 1 -- start start 2024-04-02T13:51:07.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 Processor -- start 2024-04-02T13:51:07.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180702d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c180708ca0 0x55c1807034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e40116c0 con 0x55c180708ca0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e4010f30 con 0x55c1807000c0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e400e0d0 con 0x55c1806ff7a0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc401a27740 10 client.0 _ll_register_callbacks cb 0x7fc3f00b9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc3fcbd8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:49748/0 (socket says 172.21.15.122:49748) 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/502933201 learned_addr learned my addr 172.21.15.122:0/502933201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180702d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.366+0000 7fc3fd3d9640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c180708ca0 0x55c1807034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 msgr2=0x55c180702d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180702d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/502933201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c180708ca0 msgr2=0x55c1807034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:07.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c180708ca0 0x55c1807034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 -- 172.21.15.122:0/502933201 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c180709910 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180702d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3fcbd8640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8028040 tx=0x7fc3e800e920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d33f0b5dd30dcc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e800e6c0 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ec0035f0 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.367+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7fc3e800ebc0 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fc3e8047050 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 10 client.28903 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c180787ff0 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3e803a030 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7fc3e8036550 con 0x55c1806ff7a0 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 10 client.28903 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:07.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 10 client.28903 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 20 client.28903 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180788520 0x55c18078a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 4 client.28903 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.368+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc3ec0035f0 con 0x55c180788520 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180788520 0x55c18078a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3d7fff640 10 client.28903 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3f7fff640 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180788520 0x55c18078a940 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf36bf884a0c106 server_cookie=ef36932893180ab7 in_seq=0 out_seq=0 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3d7fff640 1 -- 172.21.15.122:0/502933201 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc3ec0035f0 con 0x55c180788520 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3d7fff640 10 client.28903 handle_command_reply: tid=0 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 client.28903 shutdown 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 msgr2=0x55c180719c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8028040 tx=0x7fc3e800e920 comp rx=0 tx=0).stop 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 shutdown_connections 2024-04-02T13:51:07.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c180788520 0x55c18078a940 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c180708ca0 0x55c1807034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c1807000c0 0x55c180702d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc3d7fff640 0 client.28903 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 --2- 172.21.15.122:0/502933201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1806ff7a0 0x55c180719c70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.369+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 >> 172.21.15.122:0/502933201 conn(0x55c1806f9ed0 msgr2=0x55c1806fb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.370+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 shutdown_connections 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.370+0000 7fc401a27740 1 -- 172.21.15.122:0/502933201 wait complete. 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.370+0000 7fc401a27740 20 client.28903 trim_cache size 0 max 16384 2024-04-02T13:51:07.377 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:07.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 msgr2=0x7fc3e00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 0x7fc3e00a6260 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009c90 tx=0x7fc3e8005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 msgr2=0x7fc3f8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 0x7fc3f8197a50 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0037040 tx=0x7fc3f0032f20 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 shutdown_connections 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3e00a3da0 0x7fc3e00a6260 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3f8105a00 0x7fc3f8197a50 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f8101f30 0x7fc3f8197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 --2- 172.21.15.122:0/2643961322 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3f8101570 0x7fc3f8196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 >> 172.21.15.122:0/2643961322 conn(0x7fc3f8076b50 msgr2=0x7fc3f8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 shutdown_connections 2024-04-02T13:51:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:07.371+0000 7fc3fee3c640 1 -- 172.21.15.122:0/2643961322 wait complete. 2024-04-02T13:51:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:07 smithi187 ceph-mon[36064]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 756 op/s 2024-04-02T13:51:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:07 smithi187 ceph-mon[36064]: osdmap e114: 12 total, 12 up, 12 in 2024-04-02T13:51:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:07 smithi188 ceph-mon[36117]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 756 op/s 2024-04-02T13:51:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:07 smithi188 ceph-mon[36117]: osdmap e114: 12 total, 12 up, 12 in 2024-04-02T13:51:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:07 smithi122 ceph-mon[32114]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 756 op/s 2024-04-02T13:51:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:07 smithi122 ceph-mon[32114]: osdmap e114: 12 total, 12 up, 12 in 2024-04-02T13:51:08.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 80446 28.39 MB/sec execute 28 sec latency 163.853 ms 2024-04-02T13:51:08.389 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:51:08.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:08.390 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:51:08.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 -- 172.21.15.122:0/759155131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70101a90 msgr2=0x7f1e70101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/759155131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70101a90 0x7f1e70101f10 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009930 tx=0x7f1e6402bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 -- 172.21.15.122:0/759155131 shutdown_connections 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/759155131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70102450 0x7f1e7010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/759155131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70101a90 0x7f1e70101f10 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/759155131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70107aa0 0x7f1e70107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 -- 172.21.15.122:0/759155131 >> 172.21.15.122:0/759155131 conn(0x7f1e700fd5a0 msgr2=0x7f1e700ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 -- 172.21.15.122:0/759155131 shutdown_connections 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.533+0000 7f1e74d7b640 1 -- 172.21.15.122:0/759155131 wait complete. 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 Processor -- start 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 -- start start 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70101a90 0x7f1e701983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70102450 0x7f1e70198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70111eb0 con 0x7f1e70107aa0 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70111d30 con 0x7f1e70102450 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e74d7b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70112030 con 0x7f1e70101a90 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34286/0 (socket says 172.21.15.122:34286) 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/2676424425 learned_addr learned my addr 172.21.15.122:0/2676424425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70102450 0x7f1e70198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70101a90 0x7f1e701983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/2676424425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70101a90 msgr2=0x7f1e701983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70101a90 0x7f1e701983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/2676424425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70102450 msgr2=0x7f1e70198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70102450 0x7f1e70198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/2676424425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7019d3e0 con 0x7f1e70107aa0 2024-04-02T13:51:08.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.534+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800eb10 tx=0x7f1e6800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.535+0000 7f1e5f7fe640 1 -- 172.21.15.122:0/2676424425 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004b20 con 0x7f1e70107aa0 2024-04-02T13:51:08.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.535+0000 7f1e5f7fe640 1 -- 172.21.15.122:0/2676424425 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e68004cc0 con 0x7f1e70107aa0 2024-04-02T13:51:08.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.535+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7019d6d0 con 0x7f1e70107aa0 2024-04-02T13:51:08.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.535+0000 7f1e5f7fe640 1 -- 172.21.15.122:0/2676424425 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68010760 con 0x7f1e70107aa0 2024-04-02T13:51:08.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.535+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e701a4f60 con 0x7f1e70107aa0 2024-04-02T13:51:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.537+0000 7f1e5f7fe640 1 -- 172.21.15.122:0/2676424425 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1e68005020 con 0x7f1e70107aa0 2024-04-02T13:51:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.537+0000 7f1e77963740 1 Processor -- start 2024-04-02T13:51:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.537+0000 7f1e77963740 1 -- start start 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e5f7fe640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 0x7f1e580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03b7ef0 0x555ca03ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555ca030a770 con 0x555ca03a5d30 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ca02b2960 con 0x555ca03a66e0 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e77963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ca03068c0 con 0x555ca03b7ef0 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 0x7f1e580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34288/0 (socket says 172.21.15.122:34288) 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 -- 172.21.15.122:0/2678002754 learned_addr learned my addr 172.21.15.122:0/2678002754 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e5f7fe640 1 -- 172.21.15.122:0/2676424425 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1e680d13a0 con 0x7f1e70107aa0 2024-04-02T13:51:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 -- 172.21.15.122:0/2678002754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03b7ef0 msgr2=0x555ca03ba2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03b7ef0 0x555ca03ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 -- 172.21.15.122:0/2678002754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 msgr2=0x555ca03a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6dd9b640 1 -- 172.21.15.122:0/2678002754 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca00e67e0 con 0x555ca03a5d30 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.538+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 0x7f1e580a61e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009ae0 tx=0x7f1e60005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009900 tx=0x7f1e64005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47786847c62c0b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e5cff9640 1 -- 172.21.15.122:0/2678002754 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64049070 con 0x555ca03a5d30 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e5cff9640 1 -- 172.21.15.122:0/2678002754 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e6402cb00 con 0x555ca03a5d30 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e5cff9640 1 -- 172.21.15.122:0/2678002754 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64004f50 con 0x555ca03a5d30 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e77963740 1 -- 172.21.15.122:0/2678002754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 msgr2=0x555ca03a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.539+0000 7f1e77963740 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009900 tx=0x7f1e64005b50 comp rx=0 tx=0).stop 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- 172.21.15.122:0/2678002754 shutdown_connections 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03b7ef0 0x555ca03ba2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- 172.21.15.122:0/2678002754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03a6110 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- 172.21.15.122:0/2678002754 >> 172.21.15.122:0/2678002754 conn(0x555ca03a09f0 msgr2=0x555ca03a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- 172.21.15.122:0/2678002754 shutdown_connections 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- 172.21.15.122:0/2678002754 wait complete. 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- start start 2024-04-02T13:51:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 Processor -- start 2024-04-02T13:51:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a66e0 0x555ca03b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03b2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555ca03ab9d0 con 0x555ca03a5d30 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555ca02c5d20 con 0x555ca03b7ef0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.540+0000 7f1e77963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555ca03abb50 con 0x555ca03a66e0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e77963740 10 client.0 _ll_register_callbacks cb 0x555ca0292a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34300/0 (socket says 172.21.15.122:34300) 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6dd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a66e0 0x555ca03b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 -- 172.21.15.122:0/995783379 learned_addr learned my addr 172.21.15.122:0/995783379 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 -- 172.21.15.122:0/995783379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a66e0 msgr2=0x555ca03b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a66e0 0x555ca03b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 -- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 msgr2=0x555ca03b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03b2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 -- 172.21.15.122:0/995783379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca0146f70 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004630 tx=0x7f1e60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8835a166d48986 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.541+0000 7f1e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6000fad0 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1e6000fc70 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e6402cb00 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1e60030050 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 10 client.20190 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555c9ffde170 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e6402cb00 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f1e60027470 con 0x555ca03a5d30 2024-04-02T13:51:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 10 client.20190 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 10 client.20190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 20 client.20190 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca03e1d60 0x555ca03e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 4 client.20190 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca0317cb0 con 0x555ca03e1d60 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.542+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca03e1d60 0x555ca03e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.543+0000 7f1e55ffb640 10 client.20190 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.543+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca03e1d60 0x555ca03e4180 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=906657791c5a93a3 server_cookie=5ac7db6c830f5694 in_seq=0 out_seq=0 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e55ffb640 1 -- 172.21.15.122:0/995783379 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555ca0317cb0 con 0x555ca03e1d60 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e55ffb640 10 client.20190 handle_command_reply: tid=0 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 client.20190 shutdown 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 msgr2=0x555ca03b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1e60004630 tx=0x7f1e60004730 comp rx=0 tx=0).stop 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 shutdown_connections 2024-04-02T13:51:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca03e1d60 0x555ca03e4180 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a66e0 0x555ca03b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 --2- 172.21.15.122:0/995783379 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a5d30 0x555ca03b1b70 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 >> 172.21.15.122:0/995783379 conn(0x555ca03a09f0 msgr2=0x555ca03a2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e55ffb640 0 client.20190 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 shutdown_connections 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 1 -- 172.21.15.122:0/995783379 wait complete. 2024-04-02T13:51:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.544+0000 7f1e77963740 20 client.20190 trim_cache size 0 max 16384 2024-04-02T13:51:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.574+0000 7f1e77963740 1 Processor -- start 2024-04-02T13:51:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e77963740 1 -- start start 2024-04-02T13:51:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e77963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e77963740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 0x555ca03e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58044/0 (socket says 172.21.15.122:58044) 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.575+0000 7f1e6e59c640 1 -- 172.21.15.122:0/1270610359 learned_addr learned my addr 172.21.15.122:0/1270610359 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e77963740 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e700755e0 con 0x555ca03a66e0 2024-04-02T13:51:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e701a6550 con 0x555ca03b7ef0 2024-04-02T13:51:08.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e70075cb0 con 0x555ca03a5d30 2024-04-02T13:51:08.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.576+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 0x555ca03e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 -- 172.21.15.122:0/1270610359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 msgr2=0x555ca03e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 -- 172.21.15.122:0/1270610359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 msgr2=0x555ca03e9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 0x555ca03e9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 -- 172.21.15.122:0/1270610359 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca039c2b0 con 0x555ca03a5d30 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1e600099a0 tx=0x7f1e60005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd398e2e2668bad3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 0x555ca03e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e57fff640 1 -- 172.21.15.122:0/1270610359 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e60027ab0 con 0x555ca03a5d30 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e57fff640 1 -- 172.21.15.122:0/1270610359 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e60038030 con 0x555ca03a5d30 2024-04-02T13:51:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.577+0000 7f1e57fff640 1 -- 172.21.15.122:0/1270610359 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6001f680 con 0x555ca03a5d30 2024-04-02T13:51:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 msgr2=0x555ca03b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1e600099a0 tx=0x7f1e60005c50 comp rx=0 tx=0).stop 2024-04-02T13:51:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 shutdown_connections 2024-04-02T13:51:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03b7ef0 0x555ca03e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03a66e0 0x555ca03e9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 --2- 172.21.15.122:0/1270610359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03b1700 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 >> 172.21.15.122:0/1270610359 conn(0x555ca03a09f0 msgr2=0x555ca03a77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.578+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 shutdown_connections 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 -- 172.21.15.122:0/1270610359 wait complete. 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 -- start start 2024-04-02T13:51:08.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 Processor -- start 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.579+0000 7f1e77963740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e77963740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e701a5dc0 con 0x555ca03b7ef0 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e77963740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1e701a5630 con 0x555ca03a66e0 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e77963740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e700735e0 con 0x555ca03a5d30 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e77963740 10 client.0 _ll_register_callbacks cb 0x7f1e600098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6e59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58050/0 (socket says 172.21.15.122:58050) 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34322/0 (socket says 172.21.15.122:34322) 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6e59c640 1 -- 172.21.15.122:0/301697016 learned_addr learned my addr 172.21.15.122:0/301697016 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/301697016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 msgr2=0x555ca03a9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 msgr2=0x555ca03a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6ed9d640 1 -- 172.21.15.122:0/301697016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca03aa350 con 0x555ca03b7ef0 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6dd9b640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.580+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.581+0000 7f1e6ed9d640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1e680ce6b0 tx=0x7f1e6809cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd862651ae82b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.581+0000 7f1e55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.581+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68010900 con 0x555ca03b7ef0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.581+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f1e6809b200 con 0x555ca03b7ef0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.581+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e60038030 con 0x555ca03b7ef0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f1e6809cf00 con 0x555ca03b7ef0 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 10 client.20196 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x555ca0448950 con 0x555ca03b7ef0 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e680a12b0 con 0x555ca03b7ef0 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f1e68087bc0 con 0x555ca03b7ef0 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 10 client.20196 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 10 client.20196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 20 client.20196 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca04491d0 0x555ca044b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 4 client.20196 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.582+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f1e60038030 con 0x555ca04491d0 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.583+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca04491d0 0x555ca044b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.583+0000 7f1e55ffb640 10 client.20196 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.583+0000 7f1e6e59c640 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca04491d0 0x555ca044b5f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7d702f0aaabfd06 server_cookie=8b6ce78b87f9ab62 in_seq=0 out_seq=0 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.584+0000 7f1e55ffb640 1 -- 172.21.15.122:0/301697016 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f1e60038030 con 0x555ca04491d0 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.584+0000 7f1e55ffb640 10 client.20196 handle_command_reply: tid=0 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.584+0000 7f1e77963740 1 client.20196 shutdown 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 msgr2=0x555ca0447300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1e680ce6b0 tx=0x7f1e6809cd90 comp rx=0 tx=0).stop 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 shutdown_connections 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555ca04491d0 0x555ca044b5f0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555ca03b7ef0 0x555ca0447300 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555ca03a66e0 0x555ca03a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 --2- 172.21.15.122:0/301697016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555ca03a5d30 0x555ca03a9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e55ffb640 0 client.20196 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 >> 172.21.15.122:0/301697016 conn(0x555ca03a09f0 msgr2=0x555ca03a1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 shutdown_connections 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 1 -- 172.21.15.122:0/301697016 wait complete. 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.585+0000 7f1e77963740 20 client.20196 trim_cache size 0 max 16384 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "17d485e5-0315-4ca4-8477-01761785674d", 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:51:08.591 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 msgr2=0x7f1e580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 0x7f1e580a61e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e60009ae0 tx=0x7f1e60005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 msgr2=0x7f1e7019ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800eb10 tx=0x7f1e6800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 shutdown_connections 2024-04-02T13:51:08.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1e580a3d20 0x7f1e580a61e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e70107aa0 0x7f1e7019ccb0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1e70102450 0x7f1e70198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 --2- 172.21.15.122:0/2676424425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70101a90 0x7f1e701983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.586+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 >> 172.21.15.122:0/2676424425 conn(0x7f1e700fd5a0 msgr2=0x7f1e700fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.587+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 shutdown_connections 2024-04-02T13:51:08.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.587+0000 7f1e74d7b640 1 -- 172.21.15.122:0/2676424425 wait complete. 2024-04-02T13:51:08.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:51:08.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:51:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.782+0000 7f04d9ffb640 1 -- 172.21.15.122:0/1560133968 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc02cc70 con 0x7f04dc076950 2024-04-02T13:51:08.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.783+0000 7f04e1982640 1 -- 172.21.15.122:0/1560133968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc076950 msgr2=0x7f04dc112400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.783+0000 7f04e1982640 1 --2- 172.21.15.122:0/1560133968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc076950 0x7f04dc112400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009930 tx=0x7f04cc02bc20 comp rx=0 tx=0).stop 2024-04-02T13:51:08.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.784+0000 7f04e1982640 1 -- 172.21.15.122:0/1560133968 shutdown_connections 2024-04-02T13:51:08.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.784+0000 7f04e1982640 1 --2- 172.21.15.122:0/1560133968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc112940 0x7f04dc114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.784+0000 7f04e1982640 1 --2- 172.21.15.122:0/1560133968 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc076950 0x7f04dc112400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.784+0000 7f04e1982640 1 --2- 172.21.15.122:0/1560133968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076030 0x7f04dc076410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.784+0000 7f04e1982640 1 -- 172.21.15.122:0/1560133968 >> 172.21.15.122:0/1560133968 conn(0x7f04dc0ff770 msgr2=0x7f04dc101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.785+0000 7f04e1982640 1 -- 172.21.15.122:0/1560133968 shutdown_connections 2024-04-02T13:51:08.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.785+0000 7f04e1982640 1 -- 172.21.15.122:0/1560133968 wait complete. 2024-04-02T13:51:08.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.786+0000 7f04e1982640 1 Processor -- start 2024-04-02T13:51:08.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.786+0000 7f04e1982640 1 -- start start 2024-04-02T13:51:08.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc076030 0x7f04dc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc112940 0x7f04dc06d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06bb30 con 0x7f04dc076030 2024-04-02T13:51:08.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06b9b0 con 0x7f04dc112940 2024-04-02T13:51:08.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04e1982640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc06bcb0 con 0x7f04dc076950 2024-04-02T13:51:08.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04da7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04da7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58054/0 (socket says 172.21.15.122:58054) 2024-04-02T13:51:08.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.787+0000 7f04da7fc640 1 -- 172.21.15.122:0/2248082563 learned_addr learned my addr 172.21.15.122:0/2248082563 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04db7fe640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc112940 0x7f04dc06d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04da7fc640 1 -- 172.21.15.122:0/2248082563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc112940 msgr2=0x7f04dc06d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04da7fc640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc112940 0x7f04dc06d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04da7fc640 1 -- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc076030 msgr2=0x7f04dc073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04da7fc640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc076030 0x7f04dc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.788+0000 7f04da7fc640 1 -- 172.21.15.122:0/2248082563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc06db40 con 0x7f04dc076950 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.789+0000 7f04da7fc640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c00 tx=0x7f04cc004340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.789+0000 7f04e0980640 1 -- 172.21.15.122:0/2248082563 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc043020 con 0x7f04dc076950 2024-04-02T13:51:08.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.789+0000 7f04e0980640 1 -- 172.21.15.122:0/2248082563 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0054e0 con 0x7f04dc076950 2024-04-02T13:51:08.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.790+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc06ddd0 con 0x7f04dc076950 2024-04-02T13:51:08.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.790+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc1b3e00 con 0x7f04dc076950 2024-04-02T13:51:08.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.790+0000 7f04e0980640 1 -- 172.21.15.122:0/2248082563 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc003ab0 con 0x7f04dc076950 2024-04-02T13:51:08.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.793+0000 7f04e0980640 1 -- 172.21.15.122:0/2248082563 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f04cc046020 con 0x7f04dc076950 2024-04-02T13:51:08.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.794+0000 7f04e0980640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 0x7f04c40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.795+0000 7f04daffd640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 0x7f04c40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.795+0000 7f04e0980640 1 -- 172.21.15.122:0/2248082563 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f04cc0f2b80 con 0x7f04dc076950 2024-04-02T13:51:08.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.796+0000 7f04e456e740 1 Processor -- start 2024-04-02T13:51:08.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.796+0000 7f04e456e740 1 -- start start 2024-04-02T13:51:08.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca86830 con 0x55e39cb228d0 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca2e820 con 0x55e39cb2bdd0 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04e456e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca827e0 con 0x55e39cb231f0 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58060/0 (socket says 172.21.15.122:58060) 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 -- 172.21.15.122:0/7384944 learned_addr learned my addr 172.21.15.122:0/7384944 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04db7fe640 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 -- 172.21.15.122:0/7384944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 msgr2=0x55e39cb2e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 -- 172.21.15.122:0/7384944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 msgr2=0x55e39cb22cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.797+0000 7f04da7fc640 1 -- 172.21.15.122:0/7384944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39c8647d0 con 0x55e39cb231f0 2024-04-02T13:51:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.798+0000 7f04daffd640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 0x7f04c40a6320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005c40 tx=0x7f04d4005bd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.798+0000 7f04da7fc640 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0babf0 tx=0x7f04cc0be390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa686510adef576 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.800+0000 7f04c1ffb640 1 -- 172.21.15.122:0/7384944 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0be790 con 0x55e39cb231f0 2024-04-02T13:51:08.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.800+0000 7f04c1ffb640 1 -- 172.21.15.122:0/7384944 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0c2b40 con 0x55e39cb231f0 2024-04-02T13:51:08.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.801+0000 7f04c1ffb640 1 -- 172.21.15.122:0/7384944 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc003c50 con 0x55e39cb231f0 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.801+0000 7f04e456e740 1 -- 172.21.15.122:0/7384944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 msgr2=0x55e39cb2b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.801+0000 7f04e456e740 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0babf0 tx=0x7f04cc0be390 comp rx=0 tx=0).stop 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 -- 172.21.15.122:0/7384944 shutdown_connections 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb2b890 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 --2- 172.21.15.122:0/7384944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 -- 172.21.15.122:0/7384944 >> 172.21.15.122:0/7384944 conn(0x55e39cb1d000 msgr2=0x55e39cb1d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 -- 172.21.15.122:0/7384944 shutdown_connections 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 -- 172.21.15.122:0/7384944 wait complete. 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 -- start start 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.802+0000 7f04e456e740 1 Processor -- start 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb231f0 0x55e39cb28a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb28f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e39cb2e980 con 0x55e39cb231f0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e39ca41d30 con 0x55e39cb2bdd0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e39cb2eb00 con 0x55e39cb228d0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04e456e740 10 client.0 _ll_register_callbacks cb 0x55e39ca41720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04daffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04daffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58070/0 (socket says 172.21.15.122:58070) 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04daffd640 1 -- 172.21.15.122:0/3176776069 learned_addr learned my addr 172.21.15.122:0/3176776069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04da7fc640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb231f0 0x55e39cb28a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.803+0000 7f04db7fe640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb28f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 -- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 msgr2=0x55e39cb28f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb28f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 -- 172.21.15.122:0/3176776069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb231f0 msgr2=0x55e39cb28a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb231f0 0x55e39cb28a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 -- 172.21.15.122:0/3176776069 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39c79a3f0 con 0x55e39cb228d0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04db7fe640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04daffd640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04d4010b60 tx=0x7f04d4010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7104315ad079917a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04baffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:08.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4020070 con 0x55e39cb228d0 2024-04-02T13:51:08.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f04d400fd20 con 0x55e39cb228d0 2024-04-02T13:51:08.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.804+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04cc0c2b40 con 0x55e39cb228d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.805+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f04d402e050 con 0x55e39cb228d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.805+0000 7f04e456e740 10 client.28930 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.805+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e39c75a0f0 con 0x55e39cb228d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.805+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0c2b40 con 0x55e39cb228d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.805+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f04d401c5b0 con 0x55e39cb228d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 10 client.28930 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 10 client.28930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 20 client.28930 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cb3c0d0 0x55e39cb3e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 4 client.28930 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.806+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e39ca93d70 con 0x55e39cb3c0d0 2024-04-02T13:51:08.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.807+0000 7f04da7fc640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cb3c0d0 0x55e39cb3e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.809+0000 7f04baffd640 10 client.28930 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.809+0000 7f04da7fc640 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cb3c0d0 0x55e39cb3e530 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bd7a2632470a93 server_cookie=2b771bdb83b00fd5 in_seq=0 out_seq=0 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04baffd640 1 -- 172.21.15.122:0/3176776069 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e39ca93d70 con 0x55e39cb3c0d0 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04baffd640 10 client.28930 handle_command_reply: tid=0 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 client.28930 shutdown 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 msgr2=0x55e39cb2b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f04d4010b60 tx=0x7f04d4010f60 comp rx=0 tx=0).stop 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 shutdown_connections 2024-04-02T13:51:08.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cb3c0d0 0x55e39cb3e530 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb2bdd0 0x55e39cb28f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb231f0 0x55e39cb28a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 --2- 172.21.15.122:0/3176776069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb228d0 0x55e39cb2b6b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 >> 172.21.15.122:0/3176776069 conn(0x55e39cb1d000 msgr2=0x55e39cb1ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.811+0000 7f04baffd640 0 client.28930 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.812+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 shutdown_connections 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.812+0000 7f04e456e740 1 -- 172.21.15.122:0/3176776069 wait complete. 2024-04-02T13:51:08.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.812+0000 7f04e456e740 20 client.28930 trim_cache size 0 max 16384 2024-04-02T13:51:08.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 Processor -- start 2024-04-02T13:51:08.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 -- start start 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc121f60 con 0x55e39cb2bdd0 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc123ce0 con 0x55e39cb228d0 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04e456e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc122630 con 0x55e39cb231f0 2024-04-02T13:51:08.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04daffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04daffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:55350/0 (socket says 172.21.15.122:55350) 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04daffd640 1 -- 172.21.15.122:0/1202416814 learned_addr learned my addr 172.21.15.122:0/1202416814 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.845+0000 7f04da7fc640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04db7fe640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 -- 172.21.15.122:0/1202416814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 msgr2=0x55e39cb1eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb1eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 -- 172.21.15.122:0/1202416814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 msgr2=0x55e39cb2b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 -- 172.21.15.122:0/1202416814 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39cb23020 con 0x55e39cb228d0 2024-04-02T13:51:08.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04db7fe640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:08.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04daffd640 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006010 tx=0x7f04d4009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d840282fd71f9c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04c17fa640 1 -- 172.21.15.122:0/1202416814 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4036070 con 0x55e39cb228d0 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04c17fa640 1 -- 172.21.15.122:0/1202416814 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04d401cbf0 con 0x55e39cb228d0 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.846+0000 7f04c17fa640 1 -- 172.21.15.122:0/1202416814 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4020070 con 0x55e39cb228d0 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 -- 172.21.15.122:0/1202416814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 msgr2=0x55e39cb37890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f04d4006010 tx=0x7f04d4009b00 comp rx=0 tx=0).stop 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 -- 172.21.15.122:0/1202416814 shutdown_connections 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb2bdd0 0x55e39cb2b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb231f0 0x55e39cb1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 --2- 172.21.15.122:0/1202416814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb228d0 0x55e39cb37890 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 -- 172.21.15.122:0/1202416814 >> 172.21.15.122:0/1202416814 conn(0x55e39cb1d000 msgr2=0x55e39cb1e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.847+0000 7f04e456e740 1 -- 172.21.15.122:0/1202416814 shutdown_connections 2024-04-02T13:51:08.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.848+0000 7f04e456e740 1 -- 172.21.15.122:0/1202416814 wait complete. 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.848+0000 7f04e456e740 1 -- start start 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.848+0000 7f04e456e740 1 Processor -- start 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 0x55e39cb33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 0x55e39cb24cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc123550 con 0x55e39cb228d0 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc122dc0 con 0x55e39cb231f0 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc11ff60 con 0x55e39cb2bdd0 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04e456e740 10 client.0 _ll_register_callbacks cb 0x7f04d4010810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:08.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 0x55e39cb24cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04daffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 0x55e39cb24cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58090/0 (socket says 172.21.15.122:58090) 2024-04-02T13:51:08.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04db7fe640 1 -- 172.21.15.122:0/1286647901 learned_addr learned my addr 172.21.15.122:0/1286647901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:08.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 0x55e39cb33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 -- 172.21.15.122:0/1286647901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 msgr2=0x55e39cb24cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 0x55e39cb24cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 -- 172.21.15.122:0/1286647901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 msgr2=0x55e39cb26720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:08.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb26720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.849+0000 7f04da7fc640 1 -- 172.21.15.122:0/1286647901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39cb251f0 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.850+0000 7f04da7fc640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 0x55e39cb33da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0be490 tx=0x7f04cc0044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a08bfc880c5cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.850+0000 7f04baffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.850+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0a1070 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.850+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9613+0+0 (secure 0 0 0) 0x7f04cc02ca80 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.850+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04d401cbf0 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.851+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f04cc0dd040 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.851+0000 7f04e456e740 10 client.29174 fetch_fsmap learned FSMap version 57 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.851+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55e39cba1d70 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.851+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc0c2b40 con 0x55e39cb231f0 2024-04-02T13:51:08.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 57) ==== 8426+0+0 (secure 0 0 0) 0x7f04cc0a4020 con 0x55e39cb231f0 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 10 client.29174 fetch_fsmap finished waiting for FSMap version 57 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 10 client.29174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 20 client.29174 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cba2380 0x55e39cba47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 4 client.29174 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f04d401cbf0 con 0x55e39cba2380 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.852+0000 7f04daffd640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cba2380 0x55e39cba47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04daffd640 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cba2380 0x55e39cba47a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0f428ceb778cd9a server_cookie=f5c36e280be01dca in_seq=0 out_seq=0 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04baffd640 10 client.29174 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04baffd640 1 -- 172.21.15.122:0/1286647901 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f04d401cbf0 con 0x55e39cba2380 2024-04-02T13:51:08.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04baffd640 10 client.29174 handle_command_reply: tid=0 2024-04-02T13:51:08.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 client.29174 shutdown 2024-04-02T13:51:08.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 msgr2=0x55e39cb33da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 0x55e39cb33da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f04cc0be490 tx=0x7f04cc0044f0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 shutdown_connections 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e39cba2380 0x55e39cba47a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e39cb2bdd0 0x55e39cb24cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e39cb231f0 0x55e39cb33da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04baffd640 0 client.29174 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 --2- 172.21.15.122:0/1286647901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e39cb228d0 0x55e39cb26720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 >> 172.21.15.122:0/1286647901 conn(0x55e39cb1d000 msgr2=0x55e39cb1c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 shutdown_connections 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.853+0000 7f04e456e740 1 -- 172.21.15.122:0/1286647901 wait complete. 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e456e740 20 client.29174 trim_cache size 0 max 16384 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:51:08.860 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: "17d485e5-0315-4ca4-8477-01761785674d": { 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "17d485e5-0315-4ca4-8477-01761785674d", 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:51:08.861 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 msgr2=0x7f04c40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 0x7f04c40a6320 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f04d4005c40 tx=0x7f04d4005bd0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 msgr2=0x7f04dc073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f04cc000c00 tx=0x7f04cc004340 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 shutdown_connections 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04c40a3e60 0x7f04c40a6320 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04dc112940 0x7f04dc06d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04dc076950 0x7f04dc073670 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 --2- 172.21.15.122:0/2248082563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04dc076030 0x7f04dc073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:08.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 >> 172.21.15.122:0/2248082563 conn(0x7f04dc0ff770 msgr2=0x7f04dc101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:08.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 shutdown_connections 2024-04-02T13:51:08.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:08.854+0000 7f04e1982640 1 -- 172.21.15.122:0/2248082563 wait complete. 2024-04-02T13:51:08.872 INFO:tasks.cephfs.filesystem:scrub status for tag:17d485e5-0315-4ca4-8477-01761785674d - {'path': '/', 'tag': '17d485e5-0315-4ca4-8477-01761785674d', 'options': 'recursive,force'} 2024-04-02T13:51:09.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 81098 28.55 MB/sec execute 29 sec latency 63.174 ms 2024-04-02T13:51:09.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:09.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:09.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:09.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:09.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:09.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:09 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:09 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:09.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:09 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 81901 28.99 MB/sec execute 30 sec latency 44.378 ms 2024-04-02T13:51:10.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:10 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:10 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:10.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:10 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:11.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 82710 29.53 MB/sec execute 31 sec latency 76.984 ms 2024-04-02T13:51:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:11 smithi187 ceph-mon[36064]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:11 smithi188 ceph-mon[36117]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:11 smithi122 ceph-mon[32114]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 12 MiB/s wr, 714 op/s 2024-04-02T13:51:12.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 83459 29.81 MB/sec execute 32 sec latency 57.412 ms 2024-04-02T13:51:13.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 84282 30.36 MB/sec execute 33 sec latency 67.287 ms 2024-04-02T13:51:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:13 smithi187 ceph-mon[36064]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-04-02T13:51:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:13 smithi188 ceph-mon[36117]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-04-02T13:51:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:13 smithi122 ceph-mon[32114]: pgmap v503: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.7 MiB/s wr, 642 op/s 2024-04-02T13:51:14.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 84986 30.60 MB/sec execute 34 sec latency 105.190 ms 2024-04-02T13:51:14.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:14 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:15.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:14 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:15.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:14 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:15.325 INFO:tasks.workunit.client.0.smithi122.stdout: 10 85744 30.89 MB/sec execute 35 sec latency 71.891 ms 2024-04-02T13:51:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:15 smithi187 ceph-mon[36064]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-04-02T13:51:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:15 smithi188 ceph-mon[36117]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-04-02T13:51:16.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:15 smithi122 ceph-mon[32114]: pgmap v504: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 5.4 MiB/s wr, 607 op/s 2024-04-02T13:51:16.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 86464 30.99 MB/sec execute 36 sec latency 79.557 ms 2024-04-02T13:51:17.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 87159 31.30 MB/sec execute 37 sec latency 74.022 ms 2024-04-02T13:51:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:17 smithi187 ceph-mon[36064]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-04-02T13:51:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:17 smithi187 ceph-mon[36064]: osdmap e115: 12 total, 12 up, 12 in 2024-04-02T13:51:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:17 smithi188 ceph-mon[36117]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-04-02T13:51:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:17 smithi188 ceph-mon[36117]: osdmap e115: 12 total, 12 up, 12 in 2024-04-02T13:51:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:17 smithi122 ceph-mon[32114]: pgmap v505: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 693 op/s 2024-04-02T13:51:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:17 smithi122 ceph-mon[32114]: osdmap e115: 12 total, 12 up, 12 in 2024-04-02T13:51:18.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 88022 31.66 MB/sec execute 38 sec latency 77.485 ms 2024-04-02T13:51:19.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 88764 31.80 MB/sec execute 39 sec latency 66.792 ms 2024-04-02T13:51:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:19 smithi187 ceph-mon[36064]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:19 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:19 smithi188 ceph-mon[36117]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:19 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:19 smithi122 ceph-mon[32114]: pgmap v507: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:19 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:20.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 89508 32.12 MB/sec execute 40 sec latency 52.000 ms 2024-04-02T13:51:21.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 90277 32.43 MB/sec execute 41 sec latency 50.421 ms 2024-04-02T13:51:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:21 smithi187 ceph-mon[36064]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:21 smithi188 ceph-mon[36117]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:21 smithi122 ceph-mon[32114]: pgmap v508: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.9 MiB/s wr, 616 op/s 2024-04-02T13:51:22.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 90891 32.26 MB/sec execute 42 sec latency 57.274 ms 2024-04-02T13:51:22.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:22.381 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:22.382 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:22.382 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:22.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:22.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:23.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 91764 32.75 MB/sec execute 43 sec latency 49.866 ms 2024-04-02T13:51:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:23 smithi187 ceph-mon[36064]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 582 op/s 2024-04-02T13:51:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:23 smithi188 ceph-mon[36117]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 582 op/s 2024-04-02T13:51:23.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:23 smithi122 ceph-mon[32114]: pgmap v509: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.0 MiB/s wr, 582 op/s 2024-04-02T13:51:24.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 92469 32.57 MB/sec execute 44 sec latency 138.714 ms 2024-04-02T13:51:24.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:24 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:24.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:24 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:24.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:24 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:25.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 93285 33.07 MB/sec execute 45 sec latency 54.471 ms 2024-04-02T13:51:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:25 smithi187 ceph-mon[36064]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 631 op/s 2024-04-02T13:51:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:25 smithi188 ceph-mon[36117]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 631 op/s 2024-04-02T13:51:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:25 smithi122 ceph-mon[32114]: pgmap v510: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.8 MiB/s wr, 631 op/s 2024-04-02T13:51:26.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 93945 33.22 MB/sec execute 46 sec latency 149.126 ms 2024-04-02T13:51:27.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 94579 33.14 MB/sec execute 47 sec latency 74.762 ms 2024-04-02T13:51:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:27 smithi187 ceph-mon[36064]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 574 op/s 2024-04-02T13:51:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:27 smithi188 ceph-mon[36117]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 574 op/s 2024-04-02T13:51:27.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:27 smithi122 ceph-mon[32114]: pgmap v511: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.7 MiB/s wr, 574 op/s 2024-04-02T13:51:28.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 94954 32.85 MB/sec execute 48 sec latency 516.194 ms 2024-04-02T13:51:29.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 94984 32.18 MB/sec execute 49 sec latency 1331.703 ms 2024-04-02T13:51:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:29 smithi187 ceph-mon[36064]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 552 op/s 2024-04-02T13:51:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:29 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:29 smithi188 ceph-mon[36117]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 552 op/s 2024-04-02T13:51:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:29 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:29 smithi122 ceph-mon[32114]: pgmap v512: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 552 op/s 2024-04-02T13:51:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:29 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:30.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95216 31.78 MB/sec execute 50 sec latency 1629.430 ms 2024-04-02T13:51:31.326 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95549 31.43 MB/sec execute 51 sec latency 261.107 ms 2024-04-02T13:51:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:31 smithi187 ceph-mon[36064]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2024-04-02T13:51:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:31 smithi188 ceph-mon[36117]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2024-04-02T13:51:31.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:31 smithi122 ceph-mon[32114]: pgmap v513: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 528 op/s 2024-04-02T13:51:32.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 95913 31.31 MB/sec execute 52 sec latency 117.037 ms 2024-04-02T13:51:33.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 96513 31.22 MB/sec execute 53 sec latency 129.106 ms 2024-04-02T13:51:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:33 smithi187 ceph-mon[36064]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 871 op/s 2024-04-02T13:51:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:33 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:51:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:33 smithi188 ceph-mon[36117]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 871 op/s 2024-04-02T13:51:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:33 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:51:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:33 smithi122 ceph-mon[32114]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 19 MiB/s wr, 871 op/s 2024-04-02T13:51:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:33 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:51:34.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 97145 31.37 MB/sec execute 54 sec latency 67.120 ms 2024-04-02T13:51:35.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 97815 31.55 MB/sec execute 55 sec latency 173.955 ms 2024-04-02T13:51:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:35 smithi187 ceph-mon[36064]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 750 op/s 2024-04-02T13:51:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:35 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:35 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:35 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:51:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:35 smithi122 ceph-mon[32114]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 750 op/s 2024-04-02T13:51:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:35 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:35 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:35 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:51:36.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:35 smithi188 ceph-mon[36117]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 750 op/s 2024-04-02T13:51:36.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:35 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:51:36.009 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:35 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:51:36.010 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:35 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:51:36.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 98492 31.63 MB/sec execute 56 sec latency 43.229 ms 2024-04-02T13:51:37.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 99273 31.68 MB/sec execute 57 sec latency 60.245 ms 2024-04-02T13:51:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:37 smithi187 ceph-mon[36064]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-04-02T13:51:37.780 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:37 smithi188 ceph-mon[36117]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-04-02T13:51:37.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:37 smithi122 ceph-mon[32114]: pgmap v516: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 766 op/s 2024-04-02T13:51:38.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 100169 31.94 MB/sec execute 58 sec latency 46.953 ms 2024-04-02T13:51:38.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:51:38.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 -- 172.21.15.122:0/95532883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5011b1a0 msgr2=0x7f8e5011d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 --2- 172.21.15.122:0/95532883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5011b1a0 0x7f8e5011d590 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f8e4001a910 tx=0x7f8e4003cca0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 -- 172.21.15.122:0/95532883 shutdown_connections 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 --2- 172.21.15.122:0/95532883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5011b1a0 0x7f8e5011d590 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 --2- 172.21.15.122:0/95532883 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5006ddb0 0x7f8e5011ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 --2- 172.21.15.122:0/95532883 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5006d490 0x7f8e5006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 -- 172.21.15.122:0/95532883 >> 172.21.15.122:0/95532883 conn(0x7f8e5006cb00 msgr2=0x7f8e5010ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.046+0000 7f8e577d4640 1 -- 172.21.15.122:0/95532883 shutdown_connections 2024-04-02T13:51:39.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.047+0000 7f8e577d4640 1 -- 172.21.15.122:0/95532883 wait complete. 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.047+0000 7f8e577d4640 1 Processor -- start 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.047+0000 7f8e577d4640 1 -- start start 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5006d490 0x7f8e5011a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5006ddb0 0x7f8e50116a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010ee40 con 0x7f8e5006ddb0 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010ecc0 con 0x7f8e5006d490 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e577d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5010efc0 con 0x7f8e5011b1a0 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e55d71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e55d71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34038/0 (socket says 172.21.15.122:34038) 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.048+0000 7f8e55d71640 1 -- 172.21.15.122:0/704857333 learned_addr learned my addr 172.21.15.122:0/704857333 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 -- 172.21.15.122:0/704857333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5006d490 msgr2=0x7f8e5011a9b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5006d490 0x7f8e5011a9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 -- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5006ddb0 msgr2=0x7f8e50116a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5006ddb0 0x7f8e50116a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 -- 172.21.15.122:0/704857333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e50117890 con 0x7f8e5011b1a0 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.049+0000 7f8e55d71640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004570 tx=0x7f8e400045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.050+0000 7f8e467fc640 1 -- 172.21.15.122:0/704857333 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40052020 con 0x7f8e5011b1a0 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.050+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e501ab070 con 0x7f8e5011b1a0 2024-04-02T13:51:39.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.050+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e501ab5e0 con 0x7f8e5011b1a0 2024-04-02T13:51:39.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.050+0000 7f8e467fc640 1 -- 172.21.15.122:0/704857333 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4003d850 con 0x7f8e5011b1a0 2024-04-02T13:51:39.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.050+0000 7f8e467fc640 1 -- 172.21.15.122:0/704857333 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400038f0 con 0x7f8e5011b1a0 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.054+0000 7f8e5a3be740 1 Processor -- start 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.054+0000 7f8e5a3be740 1 -- start start 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.054+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb088d0 0x56110eb08cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e467fc640 1 -- 172.21.15.122:0/704857333 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8e40055020 con 0x7f8e5011b1a0 2024-04-02T13:51:39.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb11dd0 0x56110eb141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56110ea6c830 con 0x56110eb091f0 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56110ea14820 con 0x56110eb088d0 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.055+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56110ea687e0 con 0x56110eb11dd0 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35032/0 (socket says 172.21.15.122:35032) 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 -- 172.21.15.122:0/1305835586 learned_addr learned my addr 172.21.15.122:0/1305835586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb088d0 0x56110eb08cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 -- 172.21.15.122:0/1305835586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb11dd0 msgr2=0x56110eb141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb11dd0 0x56110eb141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 -- 172.21.15.122:0/1305835586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb088d0 msgr2=0x56110eb08cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb088d0 0x56110eb08cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.056+0000 7f8e55570640 1 -- 172.21.15.122:0/1305835586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110e84a7d0 con 0x56110eb091f0 2024-04-02T13:51:39.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.059+0000 7f8e467fc640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 0x7f8e3c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.059+0000 7f8e467fc640 1 -- 172.21.15.122:0/704857333 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f8e40103ac0 con 0x7f8e5011b1a0 2024-04-02T13:51:39.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.060+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 0x7f8e3c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.060+0000 7f8e55570640 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009780 tx=0x7f8e4c009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c10398b0a6414 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.060+0000 7f8e3b7fe640 1 -- 172.21.15.122:0/1305835586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c015020 con 0x56110eb091f0 2024-04-02T13:51:39.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.060+0000 7f8e3b7fe640 1 -- 172.21.15.122:0/1305835586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4c011420 con 0x56110eb091f0 2024-04-02T13:51:39.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.060+0000 7f8e3b7fe640 1 -- 172.21.15.122:0/1305835586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0105e0 con 0x56110eb091f0 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 -- 172.21.15.122:0/1305835586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 msgr2=0x56110eb11890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c009780 tx=0x7f8e4c009c40 comp rx=0 tx=0).stop 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 0x7f8e3c0a6320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x56110eb04190 tx=0x7f8e4800b040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 -- 172.21.15.122:0/1305835586 shutdown_connections 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb11dd0 0x56110eb141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb11890 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/1305835586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb088d0 0x56110eb08cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.062+0000 7f8e5a3be740 1 -- 172.21.15.122:0/1305835586 >> 172.21.15.122:0/1305835586 conn(0x56110eb03000 msgr2=0x56110eb033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.063+0000 7f8e5a3be740 1 -- 172.21.15.122:0/1305835586 shutdown_connections 2024-04-02T13:51:39.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.064+0000 7f8e5a3be740 1 -- 172.21.15.122:0/1305835586 wait complete. 2024-04-02T13:51:39.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.064+0000 7f8e5a3be740 1 -- start start 2024-04-02T13:51:39.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.064+0000 7f8e5a3be740 1 Processor -- start 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb11660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb0e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56110eb14bc0 con 0x56110eb091f0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56110ea27d30 con 0x56110eb11dd0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56110eb14d40 con 0x56110eb088d0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e5a3be740 10 client.0 _ll_register_callbacks cb 0x56110ea27720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e55570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb11660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e55d71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e55d71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36856/0 (socket says 172.21.15.122:36856) 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e55d71640 1 -- 172.21.15.122:0/4064804562 learned_addr learned my addr 172.21.15.122:0/4064804562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.065+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb0e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 -- 172.21.15.122:0/4064804562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 msgr2=0x56110eb11660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb11660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 -- 172.21.15.122:0/4064804562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 msgr2=0x56110eb0e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb0e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 -- 172.21.15.122:0/4064804562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110e7803f0 con 0x56110eb11dd0 2024-04-02T13:51:39.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.066+0000 7f8e55d71640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e400cc060 tx=0x7f8e400cf0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=796d05b3e37086d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.067+0000 7f8e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.067+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400ba070 con 0x56110eb11dd0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.067+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f8e400d7f00 con 0x56110eb11dd0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.067+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e4c011420 con 0x56110eb11dd0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.068+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8e400cfe20 con 0x56110eb11dd0 2024-04-02T13:51:39.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.069+0000 7f8e5a3be740 10 client.29180 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:39.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.069+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56110e7400f0 con 0x56110eb11dd0 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4c011420 con 0x56110eb11dd0 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f8e400cec90 con 0x56110eb11dd0 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 10 client.29180 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 10 client.29180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 20 client.29180 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb22030 0x56110eb244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 4 client.29180 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56110ea79d70 con 0x56110eb22030 2024-04-02T13:51:39.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.070+0000 7f8e55570640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb22030 0x56110eb244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.072+0000 7f8e38ff9640 10 client.29180 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:39.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.072+0000 7f8e55570640 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb22030 0x56110eb244b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a9f314fa8408e16 server_cookie=b0a54146202e729e in_seq=0 out_seq=0 2024-04-02T13:51:39.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.073+0000 7f8e38ff9640 1 -- 172.21.15.122:0/4064804562 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56110ea79d70 con 0x56110eb22030 2024-04-02T13:51:39.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.073+0000 7f8e38ff9640 10 client.29180 handle_command_reply: tid=0 2024-04-02T13:51:39.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.074+0000 7f8e5a3be740 1 client.29180 shutdown 2024-04-02T13:51:39.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.075+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 msgr2=0x56110eb0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.075+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8e400cc060 tx=0x7f8e400cf0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 shutdown_connections 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb22030 0x56110eb244b0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb0ef30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb091f0 0x56110eb0e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/4064804562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb11660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 >> 172.21.15.122:0/4064804562 conn(0x56110eb03000 msgr2=0x56110eb04d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.076+0000 7f8e38ff9640 0 client.29180 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:39.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.077+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 shutdown_connections 2024-04-02T13:51:39.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.078+0000 7f8e5a3be740 1 -- 172.21.15.122:0/4064804562 wait complete. 2024-04-02T13:51:39.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.078+0000 7f8e5a3be740 20 client.29180 trim_cache size 0 max 16384 2024-04-02T13:51:39.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.108+0000 7f8e5a3be740 1 Processor -- start 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 -- start start 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 0x56110eb09fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 0x56110eb11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e340100e0 con 0x56110eb11dd0 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34011da0 con 0x56110eb091f0 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e5a3be740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e340107b0 con 0x56110eb088d0 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e55570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e55570640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34068/0 (socket says 172.21.15.122:34068) 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e55570640 1 -- 172.21.15.122:0/3045544940 learned_addr learned my addr 172.21.15.122:0/3045544940 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 0x56110eb09fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.109+0000 7f8e55d71640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 0x56110eb11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 -- 172.21.15.122:0/3045544940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 msgr2=0x56110eb1db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 -- 172.21.15.122:0/3045544940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 msgr2=0x56110eb11120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 0x56110eb11120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 -- 172.21.15.122:0/3045544940 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56110eb02e30 con 0x56110eb091f0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e55d71640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 0x56110eb11120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e55570640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 0x56110eb09fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56110eb0e250 tx=0x7f8e4800e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4752a0c1e964e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e3affd640 1 -- 172.21.15.122:0/3045544940 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48016be0 con 0x56110eb091f0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e3affd640 1 -- 172.21.15.122:0/3045544940 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48027070 con 0x56110eb091f0 2024-04-02T13:51:39.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.110+0000 7f8e3affd640 1 -- 172.21.15.122:0/3045544940 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48005240 con 0x56110eb091f0 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 -- 172.21.15.122:0/3045544940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 msgr2=0x56110eb09fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 0x56110eb09fa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56110eb0e250 tx=0x7f8e4800e000 comp rx=0 tx=0).stop 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 -- 172.21.15.122:0/3045544940 shutdown_connections 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb11dd0 0x56110eb11120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb091f0 0x56110eb09fa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/3045544940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb088d0 0x56110eb1db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 -- 172.21.15.122:0/3045544940 >> 172.21.15.122:0/3045544940 conn(0x56110eb03000 msgr2=0x56110eb02250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 -- 172.21.15.122:0/3045544940 shutdown_connections 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.111+0000 7f8e5a3be740 1 -- 172.21.15.122:0/3045544940 wait complete. 2024-04-02T13:51:39.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 -- start start 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 Processor -- start 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb091f0 0x56110eb15f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e55570640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb16460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e55570640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35064/0 (socket says 172.21.15.122:35064) 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e55570640 1 -- 172.21.15.122:0/2297117366 learned_addr learned my addr 172.21.15.122:0/2297117366 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34011610 con 0x56110eb088d0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34010f40 con 0x56110eb11dd0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.112+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3400e0e0 con 0x56110eb091f0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e5a3be740 10 client.0 _ll_register_callbacks cb 0x7f8e400cf260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb091f0 0x56110eb15f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55d71640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb16460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 -- 172.21.15.122:0/2297117366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb091f0 msgr2=0x56110eb15f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb091f0 0x56110eb15f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 -- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 msgr2=0x56110eb16460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb16460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56110eb141c0 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e55570640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c006b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836db7c1a2c174ca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c02a020 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f8e4c011830 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.113+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e48027070 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8e4c035050 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 10 client.20223 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x56110eb88710 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e48027070 con 0x56110eb088d0 2024-04-02T13:51:39.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f8e4c0265a0 con 0x56110eb088d0 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 10 client.20223 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 10 client.20223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 20 client.20223 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb88dd0 0x56110eb8b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 4 client.20223 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56110ea740c0 con 0x56110eb88dd0 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.114+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb88dd0 0x56110eb8b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.115+0000 7f8e54d6f640 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb88dd0 0x56110eb8b1f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c27cfd252dae73ae server_cookie=35517b7e1bbfe669 in_seq=0 out_seq=0 2024-04-02T13:51:39.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.115+0000 7f8e38ff9640 10 client.20223 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e38ff9640 1 -- 172.21.15.122:0/2297117366 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56110ea740c0 con 0x56110eb88dd0 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e38ff9640 10 client.20223 handle_command_reply: tid=0 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 client.20223 shutdown 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 msgr2=0x56110eb159e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c000c00 tx=0x7f8e4c006b60 comp rx=0 tx=0).stop 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 shutdown_connections 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56110eb88dd0 0x56110eb8b1f0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e38ff9640 0 client.20223 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56110eb11dd0 0x56110eb16460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56110eb091f0 0x56110eb15f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 --2- 172.21.15.122:0/2297117366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56110eb088d0 0x56110eb159e0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 >> 172.21.15.122:0/2297117366 conn(0x56110eb03000 msgr2=0x56110eb09fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.118+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 shutdown_connections 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.119+0000 7f8e5a3be740 1 -- 172.21.15.122:0/2297117366 wait complete. 2024-04-02T13:51:39.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.119+0000 7f8e5a3be740 20 client.20223 trim_cache size 0 max 16384 2024-04-02T13:51:39.125 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:51:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:51:39.125 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 msgr2=0x7f8e3c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 0x7f8e3c0a6320 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x56110eb04190 tx=0x7f8e4800b040 comp rx=0 tx=0).stop 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 msgr2=0x7f8e50116fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004570 tx=0x7f8e400045a0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 shutdown_connections 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8e3c0a3e60 0x7f8e3c0a6320 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e5011b1a0 0x7f8e50116fd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e5006ddb0 0x7f8e50116a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 --2- 172.21.15.122:0/704857333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8e5006d490 0x7f8e5011a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 >> 172.21.15.122:0/704857333 conn(0x7f8e5006cb00 msgr2=0x7f8e5010ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 shutdown_connections 2024-04-02T13:51:39.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.120+0000 7f8e577d4640 1 -- 172.21.15.122:0/704857333 wait complete. 2024-04-02T13:51:39.137 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:51:39.137 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 -- 172.21.15.122:0/2947887976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c108340 msgr2=0x7fd43c108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 --2- 172.21.15.122:0/2947887976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c108340 0x7fd43c108720 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fd438009900 tx=0x7fd43802bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 -- 172.21.15.122:0/2947887976 shutdown_connections 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 --2- 172.21.15.122:0/2947887976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c10d410 0x7fd43c114fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 --2- 172.21.15.122:0/2947887976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c108cf0 0x7fd43c10cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 --2- 172.21.15.122:0/2947887976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c108340 0x7fd43c108720 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 -- 172.21.15.122:0/2947887976 >> 172.21.15.122:0/2947887976 conn(0x7fd43c076050 msgr2=0x7fd43c078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 -- 172.21.15.122:0/2947887976 shutdown_connections 2024-04-02T13:51:39.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.280+0000 7fd44453d640 1 -- 172.21.15.122:0/2947887976 wait complete. 2024-04-02T13:51:39.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 Processor -- start 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 -- start start 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 0x7fd43c1a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 0x7fd43c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1184f0 con 0x7fd43c10d410 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c118370 con 0x7fd43c19e8b0 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd44453d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c118670 con 0x7fd43c108340 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd4422d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd441ad8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 0x7fd43c1a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd4422d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34106/0 (socket says 172.21.15.122:34106) 2024-04-02T13:51:39.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.281+0000 7fd4422d9640 1 -- 172.21.15.122:0/1360491913 learned_addr learned my addr 172.21.15.122:0/1360491913 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd442ada640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 0x7fd43c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 -- 172.21.15.122:0/1360491913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 msgr2=0x7fd43c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 -- 172.21.15.122:0/1360491913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 msgr2=0x7fd43c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 0x7fd43c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 -- 172.21.15.122:0/1360491913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c19f3d0 con 0x7fd43c10d410 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd4422d9640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd442ada640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 0x7fd43c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.282+0000 7fd441ad8640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 0x7fd43c1a4bf0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e9c0 tx=0x7fd42c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.283+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00ccc0 con 0x7fd43c10d410 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.283+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42c010910 con 0x7fd43c10d410 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.283+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c19f6c0 con 0x7fd43c10d410 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.283+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c018880 con 0x7fd43c10d410 2024-04-02T13:51:39.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.283+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1ab8c0 con 0x7fd43c10d410 2024-04-02T13:51:39.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.285+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c109810 con 0x7fd43c10d410 2024-04-02T13:51:39.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.287+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd42c010ac0 con 0x7fd43c10d410 2024-04-02T13:51:39.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.288+0000 7fd4337fe640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.288+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fd42c0d0f20 con 0x7fd43c10d410 2024-04-02T13:51:39.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.289+0000 7fd4422d9640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.289+0000 7fd4422d9640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 0x7fd4280a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4380096c0 tx=0x7fd43802df90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.294+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd42c09cb80 con 0x7fd43c10d410 2024-04-02T13:51:39.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 100785 32.10 MB/sec execute 59 sec latency 153.323 ms 2024-04-02T13:51:39.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.637+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd43c108cf0 con 0x7fd43c10d410 2024-04-02T13:51:39.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.638+0000 7fd4337fe640 1 -- 172.21.15.122:0/1360491913 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11963 (secure 0 0 0) 0x7fd42c0a0e80 con 0x7fd43c10d410 2024-04-02T13:51:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:51:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:51:27.702176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:51:39.643 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 61 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 msgr2=0x7fd4280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 0x7fd4280a61e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4380096c0 tx=0x7fd43802df90 comp rx=0 tx=0).stop 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 msgr2=0x7fd43c1a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 0x7fd43c1a4bf0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00e9c0 tx=0x7fd42c00ee80 comp rx=0 tx=0).stop 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 shutdown_connections 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd4280a3d20 0x7fd4280a61e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd43c19e8b0 0x7fd43c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd43c10d410 0x7fd43c1a4bf0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 --2- 172.21.15.122:0/1360491913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd43c108340 0x7fd43c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 >> 172.21.15.122:0/1360491913 conn(0x7fd43c076050 msgr2=0x7fd43c0773d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 shutdown_connections 2024-04-02T13:51:39.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.641+0000 7fd44453d640 1 -- 172.21.15.122:0/1360491913 wait complete. 2024-04-02T13:51:39.662 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:51:39.692 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:39 smithi122 ceph-mon[32114]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 856 op/s 2024-04-02T13:51:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:39 smithi187 ceph-mon[36064]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 856 op/s 2024-04-02T13:51:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:39 smithi188 ceph-mon[36117]: pgmap v517: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 856 op/s 2024-04-02T13:51:39.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/2955687233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 msgr2=0x7fe2dc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/2955687233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc100df0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009930 tx=0x7fe2d002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/2955687233 shutdown_connections 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/2955687233 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 0x7fe2dc10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/2955687233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc100df0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/2955687233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 0x7fe2dc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/2955687233 >> 172.21.15.122:0/2955687233 conn(0x7fe2dc077b00 msgr2=0x7fe2dc077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/2955687233 shutdown_connections 2024-04-02T13:51:39.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.790+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/2955687233 wait complete. 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 Processor -- start 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 -- start start 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 0x7fe2dc1033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 0x7fe2dc103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc110db0 con 0x7fe2dc100970 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc110c30 con 0x7fe2dc101330 2024-04-02T13:51:39.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2e0bfa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc110f30 con 0x7fe2dc106980 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35106/0 (socket says 172.21.15.122:35106) 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 -- 172.21.15.122:0/1111732164 learned_addr learned my addr 172.21.15.122:0/1111732164 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 0x7fe2dc1033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 0x7fe2dc103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 -- 172.21.15.122:0/1111732164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 msgr2=0x7fe2dc103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 0x7fe2dc103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 -- 172.21.15.122:0/1111732164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 msgr2=0x7fe2dc1033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 0x7fe2dc1033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.791+0000 7fe2da59c640 1 -- 172.21.15.122:0/1111732164 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2dc1a6850 con 0x7fe2dc100970 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c920 tx=0x7fe2cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2cb7fe640 1 -- 172.21.15.122:0/1111732164 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc007c70 con 0x7fe2dc100970 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2dc1a6b40 con 0x7fe2dc100970 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2cb7fe640 1 -- 172.21.15.122:0/1111732164 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc007e10 con 0x7fe2dc100970 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2cb7fe640 1 -- 172.21.15.122:0/1111732164 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc018890 con 0x7fe2dc100970 2024-04-02T13:51:39.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.792+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2dc1a70b0 con 0x7fe2dc100970 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.794+0000 7fe2cb7fe640 1 -- 172.21.15.122:0/1111732164 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe2cc018a70 con 0x7fe2dc100970 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 Processor -- start 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 -- start start 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2cb7fe640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 0x7fe2c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd5121cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd511807b0 con 0x55cd51225ca0 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd5111f570 con 0x55cd5121d0c0 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd5117c760 con 0x55cd5121c7a0 2024-04-02T13:51:39.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 0x7fe2c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2cb7fe640 1 -- 172.21.15.122:0/1111732164 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe2cc0d0c10 con 0x7fe2dc100970 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2dad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36924/0 (socket says 172.21.15.122:36924) 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2dad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35114/0 (socket says 172.21.15.122:35114) 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/2515262103 learned_addr learned my addr 172.21.15.122:0/2515262103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.795+0000 7fe2da59c640 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd5121cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/2515262103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 msgr2=0x55cd5121cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd5121cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/2515262103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 msgr2=0x55cd51228090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/2515262103 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd50e943f0 con 0x55cd5121d0c0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 0x7fe2c40a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005fd0 tx=0x7fe2d4014bf0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0000c00 tx=0x7fe2d002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1043b10f57d4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2c8ff9640 1 -- 172.21.15.122:0/2515262103 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0045070 con 0x55cd5121d0c0 2024-04-02T13:51:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2c8ff9640 1 -- 172.21.15.122:0/2515262103 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d002c880 con 0x55cd5121d0c0 2024-04-02T13:51:39.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.796+0000 7fe2c8ff9640 1 -- 172.21.15.122:0/2515262103 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0004ee0 con 0x55cd5121d0c0 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 -- 172.21.15.122:0/2515262103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 msgr2=0x55cd51225760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0000c00 tx=0x7fe2d002c650 comp rx=0 tx=0).stop 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 -- 172.21.15.122:0/2515262103 shutdown_connections 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51228090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51225760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/2515262103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd5121cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 -- 172.21.15.122:0/2515262103 >> 172.21.15.122:0/2515262103 conn(0x55cd51216ed0 msgr2=0x55cd512172b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 -- 172.21.15.122:0/2515262103 shutdown_connections 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.797+0000 7fe2e37dd740 1 -- 172.21.15.122:0/2515262103 wait complete. 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 -- start start 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 Processor -- start 2024-04-02T13:51:39.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd51225670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51223010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd51228a30 con 0x55cd51225ca0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd5113be80 con 0x55cd5121d0c0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd51228bb0 con 0x55cd5121c7a0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2e37dd740 10 client.0 _ll_register_callbacks cb 0x55cd51177aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2dad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51223010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd51225670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2dad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51223010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35118/0 (socket says 172.21.15.122:35118) 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36934/0 (socket says 172.21.15.122:36934) 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd51225670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34130/0 (socket says 172.21.15.122:34130) 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.798+0000 7fe2dad9d640 1 -- 172.21.15.122:0/3168517390 learned_addr learned my addr 172.21.15.122:0/3168517390 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/3168517390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 msgr2=0x55cd51225670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd51225670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 msgr2=0x55cd51223010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51223010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/3168517390 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd50f5e7d0 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009900 tx=0x7fe2d00048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2c61efeef38692 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d004e070 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe2d0004380 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.799+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2d002c880 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe2d0045040 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 10 client.29204 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cd51234a60 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2d002c880 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fe2d0003960 con 0x55cd5121d0c0 2024-04-02T13:51:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 10 client.29204 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 10 client.29204 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 20 client.29204 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd51235db0 0x55cd512381a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 4 client.29204 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd5118dcf0 con 0x55cd51235db0 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.800+0000 7fe2da59c640 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd51235db0 0x55cd512381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.801+0000 7fe2c1ffb640 10 client.29204 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.801+0000 7fe2da59c640 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd51235db0 0x55cd512381a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea42cd4be660fe66 server_cookie=f2f408159ab1220f in_seq=0 out_seq=0 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/3168517390 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd5118dcf0 con 0x55cd51235db0 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2c1ffb640 10 client.29204 handle_command_reply: tid=0 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 client.29204 shutdown 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 msgr2=0x55cd51222a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009900 tx=0x7fe2d00048f0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 shutdown_connections 2024-04-02T13:51:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd51235db0 0x55cd512381a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51223010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2c1ffb640 0 client.29204 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121d0c0 0x55cd51222a00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/3168517390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121c7a0 0x55cd51225670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 >> 172.21.15.122:0/3168517390 conn(0x55cd51216ed0 msgr2=0x55cd51218c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 shutdown_connections 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.803+0000 7fe2e37dd740 1 -- 172.21.15.122:0/3168517390 wait complete. 2024-04-02T13:51:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.804+0000 7fe2e37dd740 20 client.29204 trim_cache size 0 max 16384 2024-04-02T13:51:39.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.824+0000 7fe2e37dd740 1 Processor -- start 2024-04-02T13:51:39.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 -- start start 2024-04-02T13:51:39.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 0x55cd51225130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51225670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b1140 con 0x55cd51225ca0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b2e00 con 0x55cd5121c7a0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b1810 con 0x55cd5121d0c0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36936/0 (socket says 172.21.15.122:36936) 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 -- 172.21.15.122:0/1756392128 learned_addr learned my addr 172.21.15.122:0/1756392128 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 0x55cd51225130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51225670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 -- 172.21.15.122:0/1756392128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 msgr2=0x55cd51225130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 0x55cd51225130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 -- 172.21.15.122:0/1756392128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 msgr2=0x55cd51225670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51225670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.825+0000 7fe2da59c640 1 -- 172.21.15.122:0/1756392128 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd5121cef0 con 0x55cd5121c7a0 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.826+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 0x55cd51225130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.826+0000 7fe2da59c640 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc098c20 tx=0x7fe2cc087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=943bf71b08caa266 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.826+0000 7fe2c3fff640 1 -- 172.21.15.122:0/1756392128 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc09c7c0 con 0x55cd5121c7a0 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.826+0000 7fe2c3fff640 1 -- 172.21.15.122:0/1756392128 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc0a0b70 con 0x55cd5121c7a0 2024-04-02T13:51:39.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.826+0000 7fe2c3fff640 1 -- 172.21.15.122:0/1756392128 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc018e30 con 0x55cd5121c7a0 2024-04-02T13:51:39.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- 172.21.15.122:0/1756392128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 msgr2=0x55cd51221b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc098c20 tx=0x7fe2cc087040 comp rx=0 tx=0).stop 2024-04-02T13:51:39.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- 172.21.15.122:0/1756392128 shutdown_connections 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd51225ca0 0x55cd51225670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd5121d0c0 0x55cd51225130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/1756392128 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd51221b60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- 172.21.15.122:0/1756392128 >> 172.21.15.122:0/1756392128 conn(0x55cd51216ed0 msgr2=0x55cd512156a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- 172.21.15.122:0/1756392128 shutdown_connections 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- 172.21.15.122:0/1756392128 wait complete. 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 -- start start 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.827+0000 7fe2e37dd740 1 Processor -- start 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd5122e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd51225ca0 0x55cd51226c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b2670 con 0x55cd5121d0c0 2024-04-02T13:51:39.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1b1fa0 con 0x55cd5121c7a0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2dc1af140 con 0x55cd51225ca0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2e37dd740 10 client.0 _ll_register_callbacks cb 0x7fe2d4009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2da59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd5122e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35132/0 (socket says 172.21.15.122:35132) 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/573484226 learned_addr learned my addr 172.21.15.122:0/573484226 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2dad9d640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd51225ca0 0x55cd51226c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/573484226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd51225ca0 msgr2=0x55cd51226c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd51225ca0 0x55cd51226c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/573484226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 msgr2=0x55cd5122e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd5122e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 -- 172.21.15.122:0/573484226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd512273b0 con 0x55cd5121d0c0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2da59c640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd5122e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2d9d9b640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x55cd51223fd0 tx=0x7fe2d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d047cd25f2bf251e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:39.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.828+0000 7fe2c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d004e040 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe2d00037f0 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2cc0a0b70 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe2d0064050 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 10 client.20247 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cd5129c070 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2cc0a0b70 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fe2d0005450 con 0x55cd5121d0c0 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 10 client.20247 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 10 client.20247 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 20 client.20247 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd5129c700 0x55cd5129eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:39.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 4 client.20247 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.829+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd51188040 con 0x55cd5129c700 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.830+0000 7fe2da59c640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd5129c700 0x55cd5129eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.830+0000 7fe2c1ffb640 10 client.20247 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.830+0000 7fe2da59c640 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd5129c700 0x55cd5129eb20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a075170b4ca5fbf5 server_cookie=492da9ed8528f8d0 in_seq=0 out_seq=0 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2c1ffb640 1 -- 172.21.15.122:0/573484226 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd51188040 con 0x55cd5129c700 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2c1ffb640 10 client.20247 handle_command_reply: tid=0 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 client.20247 shutdown 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 msgr2=0x55cd51226700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x55cd51223fd0 tx=0x7fe2d0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 shutdown_connections 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55cd5129c700 0x55cd5129eb20 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd51225ca0 0x55cd51226c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd5121d0c0 0x55cd51226700 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 --2- 172.21.15.122:0/573484226 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd5121c7a0 0x55cd5122e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 >> 172.21.15.122:0/573484226 conn(0x55cd51216ed0 msgr2=0x55cd512219e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2c1ffb640 0 client.20247 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:51:39.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 shutdown_connections 2024-04-02T13:51:39.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.831+0000 7fe2e37dd740 1 -- 172.21.15.122:0/573484226 wait complete. 2024-04-02T13:51:39.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e37dd740 20 client.20247 trim_cache size 0 max 16384 2024-04-02T13:51:39.840 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 msgr2=0x7fe2c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 0x7fe2c40a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005fd0 tx=0x7fe2d4014bf0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 msgr2=0x7fe2dc102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc00c920 tx=0x7fe2cc00cde0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 shutdown_connections 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe2c40a3d20 0x7fe2c40a61e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2dc106980 0x7fe2dc103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2dc101330 0x7fe2dc1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 --2- 172.21.15.122:0/1111732164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe2dc100970 0x7fe2dc102ea0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.832+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 >> 172.21.15.122:0/1111732164 conn(0x7fe2dc077b00 msgr2=0x7fe2dc068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.833+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 shutdown_connections 2024-04-02T13:51:39.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:39.833+0000 7fe2e0bfa640 1 -- 172.21.15.122:0/1111732164 wait complete. 2024-04-02T13:51:39.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:51:40.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 -- 172.21.15.122:0/605862538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f391409c550 msgr2=0x7f391409c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 --2- 172.21.15.122:0/605862538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f391409c550 0x7f391409c930 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3918009900 tx=0x7f391802bc50 comp rx=0 tx=0).stop 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 -- 172.21.15.122:0/605862538 shutdown_connections 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 --2- 172.21.15.122:0/605862538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140a1620 0x7f39140a91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 --2- 172.21.15.122:0/605862538 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409cf00 0x7f39140a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 --2- 172.21.15.122:0/605862538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f391409c550 0x7f391409c930 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.045+0000 7f3922c3e640 1 -- 172.21.15.122:0/605862538 >> 172.21.15.122:0/605862538 conn(0x7f391400c810 msgr2=0x7f391409a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.046+0000 7f3922c3e640 1 -- 172.21.15.122:0/605862538 shutdown_connections 2024-04-02T13:51:40.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.046+0000 7f3922c3e640 1 -- 172.21.15.122:0/605862538 wait complete. 2024-04-02T13:51:40.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.047+0000 7f3922c3e640 1 Processor -- start 2024-04-02T13:51:40.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.047+0000 7f3922c3e640 1 -- start start 2024-04-02T13:51:40.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140a1620 0x7f3914138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 0x7f3914132dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39140ac7a0 con 0x7f3914132920 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39140ac620 con 0x7f391409c550 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3922c3e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39140ac920 con 0x7f39140a1620 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36958/0 (socket says 172.21.15.122:36958) 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 -- 172.21.15.122:0/4122593130 learned_addr learned my addr 172.21.15.122:0/4122593130 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f392243d640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 0x7f3914132dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 -- 172.21.15.122:0/4122593130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140a1620 msgr2=0x7f3914138cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140a1620 0x7f3914138cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 -- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 msgr2=0x7f3914132dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 0x7f3914132dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f3921c3c640 1 -- 172.21.15.122:0/4122593130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3914133440 con 0x7f391409c550 2024-04-02T13:51:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.048+0000 7f392243d640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 0x7f3914132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.049+0000 7f3921c3c640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39180045e0 tx=0x7f3918004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.049+0000 7f3912ffd640 1 -- 172.21.15.122:0/4122593130 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918041070 con 0x7f391409c550 2024-04-02T13:51:40.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.049+0000 7f3912ffd640 1 -- 172.21.15.122:0/4122593130 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39180054e0 con 0x7f391409c550 2024-04-02T13:51:40.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.049+0000 7f3912ffd640 1 -- 172.21.15.122:0/4122593130 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918034440 con 0x7f391409c550 2024-04-02T13:51:40.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.050+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39141336d0 con 0x7f391409c550 2024-04-02T13:51:40.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.051+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391413f8d0 con 0x7f391409c550 2024-04-02T13:51:40.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.054+0000 7f3926a85740 1 Processor -- start 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 -- start start 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a22c0 0x563ad27aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aaea0 0x563ad27ad290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ad27057b0 con 0x563ad27a22c0 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563ad26a4570 con 0x563ad27aaea0 2024-04-02T13:51:40.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f3926a85740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ad2701760 con 0x563ad27a19a0 2024-04-02T13:51:40.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f392243d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f392243d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34160/0 (socket says 172.21.15.122:34160) 2024-04-02T13:51:40.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.055+0000 7f392243d640 1 -- 172.21.15.122:0/666298923 learned_addr learned my addr 172.21.15.122:0/666298923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.056+0000 7f392243d640 1 -- 172.21.15.122:0/666298923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aaea0 msgr2=0x563ad27ad290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:40.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.056+0000 7f392243d640 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aaea0 0x563ad27ad290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.056+0000 7f392243d640 1 -- 172.21.15.122:0/666298923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a22c0 msgr2=0x563ad27aa960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:51:40.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.056+0000 7f392243d640 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a22c0 0x563ad27aa960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.056+0000 7f392243d640 1 -- 172.21.15.122:0/666298923 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad24193f0 con 0x563ad27a19a0 2024-04-02T13:51:40.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.057+0000 7f3912ffd640 1 -- 172.21.15.122:0/4122593130 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3918044070 con 0x7f391409c550 2024-04-02T13:51:40.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.057+0000 7f392243d640 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f391c067230 tx=0x7f391c079520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e05956d2914ab03 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3907fff640 1 -- 172.21.15.122:0/666298923 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c082020 con 0x563ad27a19a0 2024-04-02T13:51:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3912ffd640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 0x7f39080a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3907fff640 1 -- 172.21.15.122:0/666298923 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391c079e10 con 0x563ad27a19a0 2024-04-02T13:51:40.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3907fff640 1 -- 172.21.15.122:0/666298923 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c07d8e0 con 0x563ad27a19a0 2024-04-02T13:51:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3921c3c640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 0x7f39080a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.059+0000 7f3912ffd640 1 -- 172.21.15.122:0/4122593130 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f39180f2450 con 0x7f391409c550 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 -- 172.21.15.122:0/666298923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 msgr2=0x563ad27a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f391c067230 tx=0x7f391c079520 comp rx=0 tx=0).stop 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3921c3c640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 0x7f39080a63e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563ad27a1470 tx=0x7f39180886a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 -- 172.21.15.122:0/666298923 shutdown_connections 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aaea0 0x563ad27ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a22c0 0x563ad27aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 --2- 172.21.15.122:0/666298923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27a1d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.060+0000 7f3926a85740 1 -- 172.21.15.122:0/666298923 >> 172.21.15.122:0/666298923 conn(0x563ad279c0d0 msgr2=0x563ad279c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.061+0000 7f3926a85740 1 -- 172.21.15.122:0/666298923 shutdown_connections 2024-04-02T13:51:40.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.061+0000 7f3926a85740 1 -- 172.21.15.122:0/666298923 wait complete. 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.061+0000 7f3926a85740 1 -- start start 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.061+0000 7f3926a85740 1 Processor -- start 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27aa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 0x563ad27a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27a8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563ad27adc90 con 0x563ad27aaea0 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563ad26c0e80 con 0x563ad27a22c0 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563ad27ade10 con 0x563ad27a19a0 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f3926a85740 10 client.0 _ll_register_callbacks cb 0x563ad26fcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f392243d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f392243d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27a8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35156/0 (socket says 172.21.15.122:35156) 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.062+0000 7f392243d640 1 -- 172.21.15.122:0/3417013137 learned_addr learned my addr 172.21.15.122:0/3417013137 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.063+0000 7f392243d640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27aa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.064+0000 7f392143b640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 0x563ad27a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.065+0000 7f392143b640 1 -- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 msgr2=0x563ad27aa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.065+0000 7f392143b640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27aa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.065+0000 7f392143b640 1 -- 172.21.15.122:0/3417013137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 msgr2=0x563ad27a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.065+0000 7f392143b640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.065+0000 7f392143b640 1 -- 172.21.15.122:0/3417013137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad24e37d0 con 0x563ad27a22c0 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.066+0000 7f392143b640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 0x563ad27a7ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f391409b630 tx=0x7f390c00e830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e88829ace4ccb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.066+0000 7f39057fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.066+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c004220 con 0x563ad27a22c0 2024-04-02T13:51:40.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.066+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f390c0043c0 con 0x563ad27a22c0 2024-04-02T13:51:40.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.068+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f391c079e10 con 0x563ad27a22c0 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.071+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f390c005020 con 0x563ad27a22c0 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.071+0000 7f3926a85740 10 client.29219 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.071+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563ad27cac50 con 0x563ad27a22c0 2024-04-02T13:51:40.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391c079e10 con 0x563ad27a22c0 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f390c010640 con 0x563ad27a22c0 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 10 client.29219 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 10 client.29219 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 20 client.29219 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad27cc1d0 0x563ad27ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 4 client.29219 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.072+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ad2712cf0 con 0x563ad27cc1d0 2024-04-02T13:51:40.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.073+0000 7f392243d640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad27cc1d0 0x563ad27ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.074+0000 7f392243d640 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad27cc1d0 0x563ad27ce5c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc927dc278b5b057 server_cookie=f69c8746627bfcfa in_seq=0 out_seq=0 2024-04-02T13:51:40.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.074+0000 7f39057fa640 10 client.29219 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:40.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.075+0000 7f39057fa640 1 -- 172.21.15.122:0/3417013137 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ad2712cf0 con 0x563ad27cc1d0 2024-04-02T13:51:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.075+0000 7f39057fa640 10 client.29219 handle_command_reply: tid=0 2024-04-02T13:51:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.075+0000 7f3926a85740 1 client.29219 shutdown 2024-04-02T13:51:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.075+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 msgr2=0x563ad27a7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.075+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 0x563ad27a7ae0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f391409b630 tx=0x7f390c00e830 comp rx=0 tx=0).stop 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 shutdown_connections 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad27cc1d0 0x563ad27ce5c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a22c0 0x563ad27a7ae0 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f391409b630 tx=0x7f390c00e830 comp rx=0 tx=0).stop 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 --2- 172.21.15.122:0/3417013137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27a19a0 0x563ad27aa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.076+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 >> 172.21.15.122:0/3417013137 conn(0x563ad279c0d0 msgr2=0x563ad279de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.077+0000 7f39057fa640 0 client.29219 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:40.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.078+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 shutdown_connections 2024-04-02T13:51:40.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.079+0000 7f3926a85740 1 -- 172.21.15.122:0/3417013137 wait complete. 2024-04-02T13:51:40.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.079+0000 7f3926a85740 20 client.29219 trim_cache size 0 max 16384 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.105+0000 7f3926a85740 1 Processor -- start 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.105+0000 7f3926a85740 1 -- start start 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.105+0000 7f3926a85740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.106+0000 7f3926a85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27aa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.106+0000 7f3926a85740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aa750 0x563ad27d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.106+0000 7f3926a85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3914149720 con 0x563ad27aaea0 2024-04-02T13:51:40.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.106+0000 7f3926a85740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f391414b3e0 con 0x563ad27a19a0 2024-04-02T13:51:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.106+0000 7f3926a85740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3914149df0 con 0x563ad27aa750 2024-04-02T13:51:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.107+0000 7f392243d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.107+0000 7f392243d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36978/0 (socket says 172.21.15.122:36978) 2024-04-02T13:51:40.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.107+0000 7f392243d640 1 -- 172.21.15.122:0/130417222 learned_addr learned my addr 172.21.15.122:0/130417222 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.107+0000 7f392243d640 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aa750 0x563ad27d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392143b640 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392243d640 1 -- 172.21.15.122:0/130417222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aa750 msgr2=0x563ad27d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392243d640 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aa750 0x563ad27d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392243d640 1 -- 172.21.15.122:0/130417222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 msgr2=0x563ad27aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392243d640 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.108+0000 7f392243d640 1 -- 172.21.15.122:0/130417222 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad27a8380 con 0x563ad27a19a0 2024-04-02T13:51:40.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.110+0000 7f392243d640 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f391c064ff0 tx=0x7f391c062fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f28caa4e1b50a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.112+0000 7f39077fe640 1 -- 172.21.15.122:0/130417222 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c082040 con 0x563ad27a19a0 2024-04-02T13:51:40.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.112+0000 7f39077fe640 1 -- 172.21.15.122:0/130417222 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391c07dda0 con 0x563ad27a19a0 2024-04-02T13:51:40.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.112+0000 7f39077fe640 1 -- 172.21.15.122:0/130417222 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c088690 con 0x563ad27a19a0 2024-04-02T13:51:40.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.113+0000 7f3926a85740 1 -- 172.21.15.122:0/130417222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 msgr2=0x563ad27b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.113+0000 7f3926a85740 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f391c064ff0 tx=0x7f391c062fe0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.114+0000 7f3926a85740 1 -- 172.21.15.122:0/130417222 shutdown_connections 2024-04-02T13:51:40.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.114+0000 7f3926a85740 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aa750 0x563ad27d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.114+0000 7f3926a85740 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27aaea0 0x563ad27aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.114+0000 7f3926a85740 1 --2- 172.21.15.122:0/130417222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27a19a0 0x563ad27b6be0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.114+0000 7f3926a85740 1 -- 172.21.15.122:0/130417222 >> 172.21.15.122:0/130417222 conn(0x563ad279c0d0 msgr2=0x563ad27a6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.115+0000 7f3926a85740 1 -- 172.21.15.122:0/130417222 shutdown_connections 2024-04-02T13:51:40.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.116+0000 7f3926a85740 1 -- 172.21.15.122:0/130417222 wait complete. 2024-04-02T13:51:40.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.116+0000 7f3926a85740 1 -- start start 2024-04-02T13:51:40.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 Processor -- start 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a19a0 0x563ad27a5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aaea0 0x563ad27abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f391414ac50 con 0x563ad27a19a0 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f391414a580 con 0x563ad27aa750 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3914147720 con 0x563ad27aaea0 2024-04-02T13:51:40.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.117+0000 7f3926a85740 10 client.0 _ll_register_callbacks cb 0x7f390c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:36986/0 (socket says 172.21.15.122:36986) 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 -- 172.21.15.122:0/1606827164 learned_addr learned my addr 172.21.15.122:0/1606827164 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392243d640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aaea0 0x563ad27abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 -- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aaea0 msgr2=0x563ad27abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aaea0 0x563ad27abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 -- 172.21.15.122:0/1606827164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a19a0 msgr2=0x563ad27a5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a19a0 0x563ad27a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.118+0000 7f392143b640 1 -- 172.21.15.122:0/1606827164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad27b3980 con 0x563ad27aa750 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.119+0000 7f392143b640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f390c00f010 tx=0x7f390c010870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc4f1f08ebcc1a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.119+0000 7f39057fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.119+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0053d0 con 0x563ad27aa750 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.119+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f390c005570 con 0x563ad27aa750 2024-04-02T13:51:40.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.119+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f391c07dda0 con 0x563ad27aa750 2024-04-02T13:51:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.120+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f390c005770 con 0x563ad27aa750 2024-04-02T13:51:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.120+0000 7f3926a85740 10 client.29225 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.120+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x563ad2843d30 con 0x563ad27aa750 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.124+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391c07dda0 con 0x563ad27aa750 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.124+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f390c018710 con 0x563ad27aa750 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.125+0000 7f3926a85740 10 client.29225 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.125+0000 7f3926a85740 10 client.29225 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.125+0000 7f3926a85740 20 client.29225 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.125+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad2844430 0x563ad2846850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.126+0000 7f3926a85740 4 client.29225 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.126+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ad270d040 con 0x563ad2844430 2024-04-02T13:51:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.126+0000 7f392243d640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad2844430 0x563ad2846850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.126+0000 7f392243d640 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad2844430 0x563ad2846850 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2d36d88e22abf4b9 server_cookie=a74105af1eb6ea56 in_seq=0 out_seq=0 2024-04-02T13:51:40.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.127+0000 7f39057fa640 10 client.29225 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:40.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.128+0000 7f39057fa640 1 -- 172.21.15.122:0/1606827164 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ad270d040 con 0x563ad2844430 2024-04-02T13:51:40.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.128+0000 7f39057fa640 10 client.29225 handle_command_reply: tid=0 2024-04-02T13:51:40.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.128+0000 7f3926a85740 1 client.29225 shutdown 2024-04-02T13:51:40.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 msgr2=0x563ad27b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f390c00f010 tx=0x7f390c010870 comp rx=0 tx=0).stop 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 shutdown_connections 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563ad27aaea0 0x563ad27abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563ad27aa750 0x563ad27b30f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x563ad2844430 0x563ad2846850 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 --2- 172.21.15.122:0/1606827164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563ad27a19a0 0x563ad27a5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 >> 172.21.15.122:0/1606827164 conn(0x563ad279c0d0 msgr2=0x563ad279a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f39057fa640 0 client.29225 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:51:40.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.129+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 shutdown_connections 2024-04-02T13:51:40.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.130+0000 7f3926a85740 1 -- 172.21.15.122:0/1606827164 wait complete. 2024-04-02T13:51:40.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.130+0000 7f3926a85740 20 client.29225 trim_cache size 0 max 16384 2024-04-02T13:51:40.136 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:40.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.133+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 msgr2=0x7f39080a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.133+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 0x7f39080a63e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x563ad27a1470 tx=0x7f39180886a0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.133+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 msgr2=0x7f3914138770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.133+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39180045e0 tx=0x7f3918004610 comp rx=0 tx=0).stop 2024-04-02T13:51:40.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 shutdown_connections 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3f20 0x7f39080a63e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3914132920 0x7f3914132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39140a1620 0x7f3914138cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 --2- 172.21.15.122:0/4122593130 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f391409c550 0x7f3914138770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.134+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 >> 172.21.15.122:0/4122593130 conn(0x7f391400c810 msgr2=0x7f3914008dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.135+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 shutdown_connections 2024-04-02T13:51:40.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.135+0000 7f3922c3e640 1 -- 172.21.15.122:0/4122593130 wait complete. 2024-04-02T13:51:40.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:51:40.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 101400 32.08 MB/sec execute 60 sec latency 106.242 ms 2024-04-02T13:51:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.324+0000 7fe066401640 1 -- 172.21.15.122:0/1790360358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 msgr2=0x7fe06010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.324+0000 7fe066401640 1 --2- 172.21.15.122:0/1790360358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe06010d680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe058009cd0 tx=0x7fe05802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 -- 172.21.15.122:0/1790360358 shutdown_connections 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 --2- 172.21.15.122:0/1790360358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe06010d680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 --2- 172.21.15.122:0/1790360358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 0x7fe060100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 --2- 172.21.15.122:0/1790360358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe060106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 -- 172.21.15.122:0/1790360358 >> 172.21.15.122:0/1790360358 conn(0x7fe0600fc2c0 msgr2=0x7fe0600fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 -- 172.21.15.122:0/1790360358 shutdown_connections 2024-04-02T13:51:40.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.325+0000 7fe066401640 1 -- 172.21.15.122:0/1790360358 wait complete. 2024-04-02T13:51:40.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 Processor -- start 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 -- start start 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 0x7fe060195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe060196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe06010fac0 con 0x7fe0601067c0 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe06010f940 con 0x7fe0601007b0 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe066401640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe06010fc40 con 0x7fe060101170 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe065c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe0653ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 0x7fe060195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe065c00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35202/0 (socket says 172.21.15.122:35202) 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe065c00640 1 -- 172.21.15.122:0/374846758 learned_addr learned my addr 172.21.15.122:0/374846758 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.326+0000 7fe064bfe640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe060196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 -- 172.21.15.122:0/374846758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 msgr2=0x7fe060196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe060196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 -- 172.21.15.122:0/374846758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 msgr2=0x7fe060195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 0x7fe060195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 -- 172.21.15.122:0/374846758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06019adf0 con 0x7fe0601067c0 2024-04-02T13:51:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.327+0000 7fe065c00640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe058005e30 tx=0x7fe058032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.328+0000 7fe0567fc640 1 -- 172.21.15.122:0/374846758 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058041020 con 0x7fe0601067c0 2024-04-02T13:51:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.328+0000 7fe0567fc640 1 -- 172.21.15.122:0/374846758 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe058034cd0 con 0x7fe0601067c0 2024-04-02T13:51:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.328+0000 7fe0567fc640 1 -- 172.21.15.122:0/374846758 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058039040 con 0x7fe0601067c0 2024-04-02T13:51:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.328+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06019b080 con 0x7fe0601067c0 2024-04-02T13:51:40.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.328+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06019b460 con 0x7fe0601067c0 2024-04-02T13:51:40.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.331+0000 7fe06a252740 1 Processor -- start 2024-04-02T13:51:40.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.331+0000 7fe0567fc640 1 -- 172.21.15.122:0/374846758 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe058044020 con 0x7fe0601067c0 2024-04-02T13:51:40.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 -- start start 2024-04-02T13:51:40.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa52ea0 0x5632baa55290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5632ba9ad7b0 con 0x5632baa499a0 2024-04-02T13:51:40.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5632ba94c570 con 0x5632baa52ea0 2024-04-02T13:51:40.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe06a252740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5632ba9a9760 con 0x5632baa4a2c0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe064bfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe064bfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35204/0 (socket says 172.21.15.122:35204) 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe064bfe640 1 -- 172.21.15.122:0/2674533077 learned_addr learned my addr 172.21.15.122:0/2674533077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.332+0000 7fe0567fc640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 0x7fe04c0a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa52ea0 0x5632baa55290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe0653ff640 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 -- 172.21.15.122:0/2674533077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 msgr2=0x5632baa52960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa52960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 -- 172.21.15.122:0/2674533077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa52ea0 msgr2=0x5632baa55290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa52ea0 0x5632baa55290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe064bfe640 1 -- 172.21.15.122:0/2674533077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ba6c13f0 con 0x5632baa499a0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe0567fc640 1 -- 172.21.15.122:0/374846758 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe0580f28a0 con 0x7fe0601067c0 2024-04-02T13:51:40.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.333+0000 7fe065c00640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 0x7fe04c0a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.335+0000 7fe065c00640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 0x7fe04c0a6320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe058037040 tx=0x7fe058003c30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.335+0000 7fe064bfe640 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe0500058b0 tx=0x7fe050005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc963431603a0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.335+0000 7fe04b7fe640 1 -- 172.21.15.122:0/2674533077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050004460 con 0x5632baa499a0 2024-04-02T13:51:40.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.336+0000 7fe04b7fe640 1 -- 172.21.15.122:0/2674533077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05000bb50 con 0x5632baa499a0 2024-04-02T13:51:40.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.336+0000 7fe04b7fe640 1 -- 172.21.15.122:0/2674533077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500035f0 con 0x5632baa499a0 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 -- 172.21.15.122:0/2674533077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 msgr2=0x5632baa49d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe0500058b0 tx=0x7fe050005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 -- 172.21.15.122:0/2674533077 shutdown_connections 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa52ea0 0x5632baa55290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa52960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 --2- 172.21.15.122:0/2674533077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa499a0 0x5632baa49d80 secure :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe0500058b0 tx=0x7fe050005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:40.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.337+0000 7fe06a252740 1 -- 172.21.15.122:0/2674533077 >> 172.21.15.122:0/2674533077 conn(0x5632baa440d0 msgr2=0x5632baa444b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.341+0000 7fe06a252740 1 -- 172.21.15.122:0/2674533077 shutdown_connections 2024-04-02T13:51:40.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.342+0000 7fe06a252740 1 -- 172.21.15.122:0/2674533077 wait complete. 2024-04-02T13:51:40.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.342+0000 7fe06a252740 1 -- start start 2024-04-02T13:51:40.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.342+0000 7fe06a252740 1 Processor -- start 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa4a2c0 0x5632baa4fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa78910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5632baa66c90 con 0x5632baa52ea0 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5632ba968e80 con 0x5632baa4a2c0 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5632baa66e10 con 0x5632baa507c0 2024-04-02T13:51:40.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.343+0000 7fe06a252740 10 client.0 _ll_register_callbacks cb 0x5632ba9a4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.344+0000 7fe0653ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.344+0000 7fe0653ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35208/0 (socket says 172.21.15.122:35208) 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.344+0000 7fe0653ff640 1 -- 172.21.15.122:0/72539375 learned_addr learned my addr 172.21.15.122:0/72539375 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.344+0000 7fe064bfe640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa4a2c0 0x5632baa4fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.344+0000 7fe064bfe640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa78910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 -- 172.21.15.122:0/72539375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 msgr2=0x5632baa78910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa78910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 -- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa4a2c0 msgr2=0x5632baa4fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa4a2c0 0x5632baa4fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 -- 172.21.15.122:0/72539375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ba78b7d0 con 0x5632baa52ea0 2024-04-02T13:51:40.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe0653ff640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe05c013710 tx=0x7fe05c013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe94fb3953ca1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.345+0000 7fe064bfe640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa78910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.346+0000 7fe048ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.346+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c020060 con 0x5632baa52ea0 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.346+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05000bb50 con 0x5632baa52ea0 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.346+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe05c01b430 con 0x5632baa52ea0 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.347+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe05c027050 con 0x5632baa52ea0 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.347+0000 7fe06a252740 10 client.20268 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.347+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5632baa69cf0 con 0x5632baa52ea0 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05000bb50 con 0x5632baa52ea0 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fe05c00e410 con 0x5632baa52ea0 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 10 client.20268 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 10 client.20268 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 20 client.20268 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baa6b580 0x5632baa6d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 4 client.20268 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:40.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.348+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ba9bacf0 con 0x5632baa6b580 2024-04-02T13:51:40.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.350+0000 7fe064bfe640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baa6b580 0x5632baa6d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.353+0000 7fe064bfe640 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baa6b580 0x5632baa6d9e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e87118351017dee2 server_cookie=32f623b6af6d6dd3 in_seq=0 out_seq=0 2024-04-02T13:51:40.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.353+0000 7fe048ff9640 10 client.20268 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:40.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.356+0000 7fe048ff9640 1 -- 172.21.15.122:0/72539375 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632ba9bacf0 con 0x5632baa6b580 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.356+0000 7fe048ff9640 10 client.20268 handle_command_reply: tid=0 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.356+0000 7fe06a252740 1 client.20268 shutdown 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.357+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 msgr2=0x5632baa50280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.357+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe05c013710 tx=0x7fe05c013bd0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 shutdown_connections 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa78910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa52ea0 0x5632baa50280 secure :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fe05c013710 tx=0x7fe05c013bd0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baa6b580 0x5632baa6d9e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 --2- 172.21.15.122:0/72539375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa4a2c0 0x5632baa4fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 >> 172.21.15.122:0/72539375 conn(0x5632baa440d0 msgr2=0x5632baa45e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.358+0000 7fe048ff9640 0 client.20268 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.359+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 shutdown_connections 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.359+0000 7fe06a252740 1 -- 172.21.15.122:0/72539375 wait complete. 2024-04-02T13:51:40.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.359+0000 7fe06a252740 20 client.20268 trim_cache size 0 max 16384 2024-04-02T13:51:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.388+0000 7fe06a252740 1 Processor -- start 2024-04-02T13:51:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.388+0000 7fe06a252740 1 -- start start 2024-04-02T13:51:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa4a2c0 0x5632baa3efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa460e0 0x5632baa8f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a42e0 con 0x5632baa4a2c0 2024-04-02T13:51:40.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a5fa0 con 0x5632baa460e0 2024-04-02T13:51:40.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe06a252740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a49b0 con 0x5632baa507c0 2024-04-02T13:51:40.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe0653ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe0653ff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34212/0 (socket says 172.21.15.122:34212) 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.389+0000 7fe0653ff640 1 -- 172.21.15.122:0/135497665 learned_addr learned my addr 172.21.15.122:0/135497665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 -- 172.21.15.122:0/135497665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa460e0 msgr2=0x5632baa8f3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa460e0 0x5632baa8f3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 -- 172.21.15.122:0/135497665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa4a2c0 msgr2=0x5632baa3efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa4a2c0 0x5632baa3efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 -- 172.21.15.122:0/135497665 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632baa3f7b0 con 0x5632baa507c0 2024-04-02T13:51:40.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe0653ff640 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c0118e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d219c88b6d16c2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.390+0000 7fe04affd640 1 -- 172.21.15.122:0/135497665 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00eea0 con 0x5632baa507c0 2024-04-02T13:51:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.391+0000 7fe04affd640 1 -- 172.21.15.122:0/135497665 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c030070 con 0x5632baa507c0 2024-04-02T13:51:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.391+0000 7fe04affd640 1 -- 172.21.15.122:0/135497665 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05c00ec60 con 0x5632baa507c0 2024-04-02T13:51:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 -- 172.21.15.122:0/135497665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 msgr2=0x5632baa8ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe05c000c00 tx=0x7fe05c0118e0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 -- 172.21.15.122:0/135497665 shutdown_connections 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa460e0 0x5632baa8f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa507c0 0x5632baa8ee90 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 --2- 172.21.15.122:0/135497665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa4a2c0 0x5632baa3efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 -- 172.21.15.122:0/135497665 >> 172.21.15.122:0/135497665 conn(0x5632baa440d0 msgr2=0x5632baa4e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 -- 172.21.15.122:0/135497665 shutdown_connections 2024-04-02T13:51:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.392+0000 7fe06a252740 1 -- 172.21.15.122:0/135497665 wait complete. 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.393+0000 7fe06a252740 1 -- start start 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.393+0000 7fe06a252740 1 Processor -- start 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa460e0 0x5632baa4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa4d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a5810 con 0x5632baa460e0 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe0601a5140 con 0x5632baa507c0 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe060197190 con 0x5632baa4a2c0 2024-04-02T13:51:40.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe06a252740 10 client.0 _ll_register_callbacks cb 0x7fe05c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe064bfe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe064bfe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37052/0 (socket says 172.21.15.122:37052) 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe064bfe640 1 -- 172.21.15.122:0/660971917 learned_addr learned my addr 172.21.15.122:0/660971917 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.394+0000 7fe064bfe640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa460e0 0x5632baa4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 -- 172.21.15.122:0/660971917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 msgr2=0x5632baa4d230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa4d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 -- 172.21.15.122:0/660971917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa460e0 msgr2=0x5632baa4ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa460e0 0x5632baa4ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 -- 172.21.15.122:0/660971917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632baaea9e0 con 0x5632baa507c0 2024-04-02T13:51:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe064bfe640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe050017c40 tx=0x7fe050005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b3203c1b48cadc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.395+0000 7fe048ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:40.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.396+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050017990 con 0x5632baa507c0 2024-04-02T13:51:40.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.396+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe05c030070 con 0x5632baa507c0 2024-04-02T13:51:40.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.396+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fe05002c3c0 con 0x5632baa507c0 2024-04-02T13:51:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.398+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fe05002c5c0 con 0x5632baa507c0 2024-04-02T13:51:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.398+0000 7fe06a252740 10 client.29237 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.398+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5632baaebce0 con 0x5632baa507c0 2024-04-02T13:51:40.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe05c030070 con 0x5632baa507c0 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fe050004680 con 0x5632baa507c0 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 10 client.29237 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 10 client.29237 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 20 client.29237 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baaec3e0 0x5632baaee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 4 client.29237 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:40.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.399+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632ba9b5040 con 0x5632baaec3e0 2024-04-02T13:51:40.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.400+0000 7fe0653ff640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baaec3e0 0x5632baaee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.403+0000 7fe048ff9640 10 client.29237 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:40.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.403+0000 7fe0653ff640 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baaec3e0 0x5632baaee800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8a86b76cfb06420 server_cookie=5151cc0add0ff631 in_seq=0 out_seq=0 2024-04-02T13:51:40.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.411+0000 7fe048ff9640 1 -- 172.21.15.122:0/660971917 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632ba9b5040 con 0x5632baaec3e0 2024-04-02T13:51:40.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.411+0000 7fe048ff9640 10 client.29237 handle_command_reply: tid=0 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.411+0000 7fe06a252740 1 client.29237 shutdown 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 msgr2=0x5632baa4d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe050017c40 tx=0x7fe050005f90 comp rx=0 tx=0).stop 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 shutdown_connections 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5632baaec3e0 0x5632baaee800 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5632baa507c0 0x5632baa4d7b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5632baa4a2c0 0x5632baa4d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 --2- 172.21.15.122:0/660971917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5632baa460e0 0x5632baa4ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 >> 172.21.15.122:0/660971917 conn(0x5632baa440d0 msgr2=0x5632baa77df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe048ff9640 0 client.29237 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:51:40.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 shutdown_connections 2024-04-02T13:51:40.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.412+0000 7fe06a252740 1 -- 172.21.15.122:0/660971917 wait complete. 2024-04-02T13:51:40.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.413+0000 7fe06a252740 20 client.29237 trim_cache size 0 max 16384 2024-04-02T13:51:40.417 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:40.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 msgr2=0x7fe04c0a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 0x7fe04c0a6320 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe058037040 tx=0x7fe058003c30 comp rx=0 tx=0).stop 2024-04-02T13:51:40.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 msgr2=0x7fe06019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fe058005e30 tx=0x7fe058032ea0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 shutdown_connections 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe04c0a3e60 0x7fe04c0a6320 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe0601067c0 0x7fe06019a6c0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe060101170 0x7fe060196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 --2- 172.21.15.122:0/374846758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe0601007b0 0x7fe060195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 >> 172.21.15.122:0/374846758 conn(0x7fe0600fc2c0 msgr2=0x7fe0600fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 shutdown_connections 2024-04-02T13:51:40.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.414+0000 7fe066401640 1 -- 172.21.15.122:0/374846758 wait complete. 2024-04-02T13:51:40.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:51:40.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 -- 172.21.15.122:0/4216365956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc101dc0 msgr2=0x7fcefc10dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/4216365956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc101dc0 0x7fcefc10dfd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009ca0 tx=0x7fcef402c100 comp rx=0 tx=0).stop 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 -- 172.21.15.122:0/4216365956 shutdown_connections 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/4216365956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc10e510 0x7fcefc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/4216365956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc101dc0 0x7fcefc10dfd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/4216365956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc1014a0 0x7fcefc101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 -- 172.21.15.122:0/4216365956 >> 172.21.15.122:0/4216365956 conn(0x7fcefc077a90 msgr2=0x7fcefc077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 -- 172.21.15.122:0/4216365956 shutdown_connections 2024-04-02T13:51:40.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.578+0000 7fcf01dcd640 1 -- 172.21.15.122:0/4216365956 wait complete. 2024-04-02T13:51:40.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 Processor -- start 2024-04-02T13:51:40.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 -- start start 2024-04-02T13:51:40.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc101dc0 0x7fcefc10afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 0x7fcefc10b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc101dc0 0x7fcefc10afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35244/0 (socket says 172.21.15.122:35244) 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcefb7fe640 1 -- 172.21.15.122:0/352914570 learned_addr learned my addr 172.21.15.122:0/352914570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1140a0 con 0x7fcefc1014a0 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 0x7fcefc10b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc113f20 con 0x7fcefc10e510 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.579+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc114220 con 0x7fcefc101dc0 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 -- 172.21.15.122:0/352914570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc101dc0 msgr2=0x7fcefc10afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc101dc0 0x7fcefc10afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 -- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 msgr2=0x7fcefc10dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcefc06b520 con 0x7fcefc10e510 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 0x7fcefc10b570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e940 tx=0x7fceec00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcef8ff9640 1 -- 172.21.15.122:0/352914570 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec00ccc0 con 0x7fcefc10e510 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcef8ff9640 1 -- 172.21.15.122:0/352914570 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceec005020 con 0x7fcefc10e510 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcefc06b810 con 0x7fcefc10e510 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcef8ff9640 1 -- 172.21.15.122:0/352914570 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec004a00 con 0x7fcefc10e510 2024-04-02T13:51:40.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.580+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcefc06bd30 con 0x7fcefc10e510 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcf049b3740 1 Processor -- start 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcef8ff9640 1 -- 172.21.15.122:0/352914570 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fceec0051d0 con 0x7fcefc10e510 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcf049b3740 1 -- start start 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9af97a0 0x560ac9af9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.583+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcf049b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ac9a5d7b0 con 0x560ac9b02ca0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcf049b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560ac99fc570 con 0x560ac9afa0c0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcf049b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560ac9a59760 con 0x560ac9af97a0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37062/0 (socket says 172.21.15.122:37062) 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3959142320 learned_addr learned my addr 172.21.15.122:0/3959142320 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefaffd640 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9af97a0 0x560ac9af9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcef8ff9640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 0x7fcee40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3959142320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9af97a0 msgr2=0x560ac9af9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9af97a0 0x560ac9af9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3959142320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 msgr2=0x560ac9b05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3959142320 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac97713f0 con 0x560ac9afa0c0 2024-04-02T13:51:40.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 0x7fcee40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcef0010580 tx=0x7fcef0010a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71dd697d372eb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcef8ff9640 1 -- 172.21.15.122:0/352914570 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fceec0d0a70 con 0x7fcefc10e510 2024-04-02T13:51:40.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.584+0000 7fcee1ffb640 1 -- 172.21.15.122:0/3959142320 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef000e890 con 0x560ac9afa0c0 2024-04-02T13:51:40.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.585+0000 7fcee1ffb640 1 -- 172.21.15.122:0/3959142320 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef001c480 con 0x560ac9afa0c0 2024-04-02T13:51:40.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.585+0000 7fcee1ffb640 1 -- 172.21.15.122:0/3959142320 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef00116a0 con 0x560ac9afa0c0 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.585+0000 7fcefbfff640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 0x7fcee40a6260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fceec0cde80 tx=0x7fceec0982a0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.585+0000 7fcf049b3740 1 -- 172.21.15.122:0/3959142320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 msgr2=0x560ac9b02760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.585+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcef0010580 tx=0x7fcef0010a40 comp rx=0 tx=0).stop 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 -- 172.21.15.122:0/3959142320 shutdown_connections 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9b02760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3959142320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9af97a0 0x560ac9af9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 -- 172.21.15.122:0/3959142320 >> 172.21.15.122:0/3959142320 conn(0x560ac9af3ed0 msgr2=0x560ac9af42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 -- 172.21.15.122:0/3959142320 shutdown_connections 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 -- 172.21.15.122:0/3959142320 wait complete. 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 -- start start 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.586+0000 7fcf049b3740 1 Processor -- start 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9affae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9b00220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35266/0 (socket says 172.21.15.122:35266) 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcefaffd640 1 -- 172.21.15.122:0/2116369864 learned_addr learned my addr 172.21.15.122:0/2116369864 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560ac9b05a90 con 0x560ac9af97a0 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9affae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560ac9a18e80 con 0x560ac9afa0c0 2024-04-02T13:51:40.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.587+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560ac9b05c10 con 0x560ac9b02ca0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9b00220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 -- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 msgr2=0x560ac9b00220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9b00220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 -- 172.21.15.122:0/2116369864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 msgr2=0x560ac9affae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9affae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcf049b3740 10 client.0 _ll_register_callbacks cb 0x560ac9a54aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac983b7d0 con 0x560ac9af97a0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef001c480 con 0x560ac9af97a0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcef402e010 tx=0x7fcef402c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7792f945c60ca322 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcedaffd640 10 client.20283 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.588+0000 7fcedaffd640 1 -- 172.21.15.122:0/2116369864 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4039040 con 0x560ac9af97a0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcedaffd640 1 -- 172.21.15.122:0/2116369864 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fcef4003d80 con 0x560ac9af97a0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcedaffd640 1 -- 172.21.15.122:0/2116369864 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcef4004040 con 0x560ac9af97a0 2024-04-02T13:51:40.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 10 client.20283 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=61}) -- 0x560ac9b119c0 con 0x560ac9af97a0 2024-04-02T13:51:40.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcedaffd640 1 -- 172.21.15.122:0/2116369864 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fcef403f070 con 0x560ac9af97a0 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 10 client.20283 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 10 client.20283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 20 client.20283 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b13280 0x560ac9b15700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 4 client.20283 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.589+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ac9a6acf0 con 0x560ac9b13280 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.590+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b13280 0x560ac9b15700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.590+0000 7fcedaffd640 10 client.20283 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:40.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.590+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b13280 0x560ac9b15700 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d132e17e821c5124 server_cookie=ae4d6e0b01b43029 in_seq=0 out_seq=0 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcedaffd640 1 -- 172.21.15.122:0/2116369864 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ac9a6acf0 con 0x560ac9b13280 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcedaffd640 10 client.20283 handle_command_reply: tid=0 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 client.20283 shutdown 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 msgr2=0x560ac9b02750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcef402e010 tx=0x7fcef402c100 comp rx=0 tx=0).stop 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 shutdown_connections 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b13280 0x560ac9b15700 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9b00220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9afa0c0 0x560ac9affae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcedaffd640 0 client.20283 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2116369864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9af97a0 0x560ac9b02750 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 >> 172.21.15.122:0/2116369864 conn(0x560ac9af3ed0 msgr2=0x560ac9af5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 shutdown_connections 2024-04-02T13:51:40.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 1 -- 172.21.15.122:0/2116369864 wait complete. 2024-04-02T13:51:40.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.604+0000 7fcf049b3740 20 client.20283 trim_cache size 0 max 16384 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 Processor -- start 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 -- start start 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b02210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9afcd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.623+0000 7fcf049b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1bde10 con 0x560ac9afa0c0 2024-04-02T13:51:40.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcf049b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1bfad0 con 0x560ac9af97a0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcf049b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1be4e0 con 0x560ac9b02ca0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b02210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35278/0 (socket says 172.21.15.122:35278) 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3564651296 learned_addr learned my addr 172.21.15.122:0/3564651296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefaffd640 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9afcd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3564651296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 msgr2=0x560ac9afcd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9afcd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3564651296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 msgr2=0x560ac9b02210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b02210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 -- 172.21.15.122:0/3564651296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ac9b00590 con 0x560ac9afa0c0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefaffd640 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b02210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fcef0010580 tx=0x7fcef001ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cad7f00122663e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcee17fa640 1 -- 172.21.15.122:0/3564651296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef002f070 con 0x560ac9afa0c0 2024-04-02T13:51:40.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcee17fa640 1 -- 172.21.15.122:0/3564651296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef001c630 con 0x560ac9afa0c0 2024-04-02T13:51:40.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.624+0000 7fcee17fa640 1 -- 172.21.15.122:0/3564651296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef002a310 con 0x560ac9afa0c0 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- 172.21.15.122:0/3564651296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 msgr2=0x560ac9b02750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fcef0010580 tx=0x7fcef001ef90 comp rx=0 tx=0).stop 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- 172.21.15.122:0/3564651296 shutdown_connections 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9b02ca0 0x560ac9afcd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9afa0c0 0x560ac9b02750 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 --2- 172.21.15.122:0/3564651296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b02210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- 172.21.15.122:0/3564651296 >> 172.21.15.122:0/3564651296 conn(0x560ac9af3ed0 msgr2=0x560ac9af42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- 172.21.15.122:0/3564651296 shutdown_connections 2024-04-02T13:51:40.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- 172.21.15.122:0/3564651296 wait complete. 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.625+0000 7fcf049b3740 1 -- start start 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 Processor -- start 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9afa0c0 0x560ac9b16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b17460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1bf340 con 0x560ac9b02ca0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1bec70 con 0x560ac9af97a0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcefc1bbe30 con 0x560ac9afa0c0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcf049b3740 10 client.0 _ll_register_callbacks cb 0x7fcef4032a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcefaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37104/0 (socket says 172.21.15.122:37104) 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcefaffd640 1 -- 172.21.15.122:0/2109215878 learned_addr learned my addr 172.21.15.122:0/2109215878 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9afa0c0 0x560ac9b16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.626+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b17460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 -- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9afa0c0 msgr2=0x560ac9b16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9afa0c0 0x560ac9b16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 -- 172.21.15.122:0/2109215878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 msgr2=0x560ac9b17460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b17460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 -- 172.21.15.122:0/2109215878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ac9b787f0 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcefaffd640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcef4005e30 tx=0x7fcef4046030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442a1344bc06ebd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcedaffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4039070 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef001c630 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7fcef4004770 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcef404f440 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcf049b3740 10 client.29255 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x560ac9b79aa0 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.627+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcef001c630 con 0x560ac9af97a0 2024-04-02T13:51:40.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7fcef403f040 con 0x560ac9af97a0 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 10 client.29255 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 10 client.29255 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 20 client.29255 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b7a100 0x560ac9b7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 4 client.29255 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ac9a65040 con 0x560ac9b7a100 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.628+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b7a100 0x560ac9b7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.629+0000 7fcedaffd640 10 client.29255 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:40.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.629+0000 7fcefb7fe640 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b7a100 0x560ac9b7c4f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=321afad5ca28ba8e server_cookie=d6018f4f28cc877d in_seq=0 out_seq=0 2024-04-02T13:51:40.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcedaffd640 1 -- 172.21.15.122:0/2109215878 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ac9a65040 con 0x560ac9b7a100 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcedaffd640 10 client.29255 handle_command_reply: tid=0 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcf049b3740 1 client.29255 shutdown 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 msgr2=0x560ac9b169e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcef4005e30 tx=0x7fcef4046030 comp rx=0 tx=0).stop 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 shutdown_connections 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560ac9b7a100 0x560ac9b7c4f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.634+0000 7fcedaffd640 0 client.29255 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560ac9b02ca0 0x560ac9b17460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560ac9afa0c0 0x560ac9b16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 --2- 172.21.15.122:0/2109215878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560ac9af97a0 0x560ac9b169e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 >> 172.21.15.122:0/2109215878 conn(0x560ac9af3ed0 msgr2=0x560ac9b0de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 shutdown_connections 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 1 -- 172.21.15.122:0/2109215878 wait complete. 2024-04-02T13:51:40.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf049b3740 20 client.29255 trim_cache size 0 max 16384 2024-04-02T13:51:40.640 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 msgr2=0x7fcee40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 0x7fcee40a6260 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fceec0cde80 tx=0x7fceec0982a0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 msgr2=0x7fcefc10b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.635+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 0x7fcefc10b570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fceec00e940 tx=0x7fceec00ee00 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 shutdown_connections 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcee40a3da0 0x7fcee40a6260 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcefc10e510 0x7fcefc10b570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcefc101dc0 0x7fcefc10afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 --2- 172.21.15.122:0/352914570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcefc1014a0 0x7fcefc10dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 >> 172.21.15.122:0/352914570 conn(0x7fcefc077a90 msgr2=0x7fcefc10fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 shutdown_connections 2024-04-02T13:51:40.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.636+0000 7fcf01dcd640 1 -- 172.21.15.122:0/352914570 wait complete. 2024-04-02T13:51:40.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:51:40.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:40 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1360491913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:40 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1360491913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:40.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.793+0000 7f5be6dbc640 1 -- 172.21.15.122:0/839117489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be010e4a0 msgr2=0x7f5be0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.793+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/839117489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be010e4a0 0x7f5be0110890 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009860 tx=0x7f5bd803cc20 comp rx=0 tx=0).stop 2024-04-02T13:51:40.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 -- 172.21.15.122:0/839117489 shutdown_connections 2024-04-02T13:51:40.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/839117489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be010e4a0 0x7f5be0110890 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/839117489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0069620 0x7f5be010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/839117489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be0068d00 0x7f5be00690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 -- 172.21.15.122:0/839117489 >> 172.21.15.122:0/839117489 conn(0x7f5be0077b00 msgr2=0x7f5be0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 -- 172.21.15.122:0/839117489 shutdown_connections 2024-04-02T13:51:40.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.794+0000 7f5be6dbc640 1 -- 172.21.15.122:0/839117489 wait complete. 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 Processor -- start 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 -- start start 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 0x7f5be010dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 0x7f5be010a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5be006deb0 con 0x7f5be010e4a0 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5be006dd30 con 0x7f5be0069620 2024-04-02T13:51:40.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be6dbc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5be006e030 con 0x7f5be0068d00 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5be5359640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 0x7f5be010a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37122/0 (socket says 172.21.15.122:37122) 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.795+0000 7f5bdffff640 1 -- 172.21.15.122:0/324471728 learned_addr learned my addr 172.21.15.122:0/324471728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be4b58640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 0x7f5be010dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 -- 172.21.15.122:0/324471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 msgr2=0x7f5be010dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 0x7f5be010dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 -- 172.21.15.122:0/324471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 msgr2=0x7f5be0109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 -- 172.21.15.122:0/324471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be010a6a0 con 0x7f5be010e4a0 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5bdffff640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:40.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be4b58640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 0x7f5be010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:51:40.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be5359640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 0x7f5be010a0e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009780 tx=0x7f5bd8045670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5bddffb640 1 -- 172.21.15.122:0/324471728 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8052030 con 0x7f5be010e4a0 2024-04-02T13:51:40.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be010a930 con 0x7f5be010e4a0 2024-04-02T13:51:40.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5bddffb640 1 -- 172.21.15.122:0/324471728 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd803de50 con 0x7f5be010e4a0 2024-04-02T13:51:40.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.796+0000 7f5bddffb640 1 -- 172.21.15.122:0/324471728 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd804a040 con 0x7f5be010e4a0 2024-04-02T13:51:40.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.797+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be019e6d0 con 0x7f5be010e4a0 2024-04-02T13:51:40.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.800+0000 7f5be999b740 1 Processor -- start 2024-04-02T13:51:40.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.800+0000 7f5bddffb640 1 -- 172.21.15.122:0/324471728 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5bd803d5c0 con 0x7f5be010e4a0 2024-04-02T13:51:40.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.800+0000 7f5be999b740 1 -- start start 2024-04-02T13:51:40.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.800+0000 7f5be999b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252b9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.800+0000 7f5be999b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5be999b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5be999b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d252147b0 con 0x561d252b8fb0 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5be999b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561d251b3570 con 0x561d252b98d0 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5be999b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561d25210760 con 0x561d252c24b0 2024-04-02T13:51:40.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5bddffb640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 0x7f5bc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.801+0000 7f5bddffb640 1 -- 172.21.15.122:0/324471728 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f5bd81024c0 con 0x7f5be010e4a0 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5be4b58640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5be4b58640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252c1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37128/0 (socket says 172.21.15.122:37128) 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5be4b58640 1 -- 172.21.15.122:0/1061308863 learned_addr learned my addr 172.21.15.122:0/1061308863 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5bdffff640 1 -- 172.21.15.122:0/1061308863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 msgr2=0x561d252c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.802+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.803+0000 7f5bdffff640 1 -- 172.21.15.122:0/1061308863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 msgr2=0x561d252c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.803+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.803+0000 7f5bdffff640 1 -- 172.21.15.122:0/1061308863 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d24f283f0 con 0x561d252b8fb0 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.803+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252b9390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5be010b3a0 tx=0x7f5bd0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa4926dfc82588f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.803+0000 7f5be5359640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 0x7f5bc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.804+0000 7f5bc6ffd640 1 -- 172.21.15.122:0/1061308863 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0011c10 con 0x561d252b8fb0 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.804+0000 7f5bc6ffd640 1 -- 172.21.15.122:0/1061308863 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0011db0 con 0x561d252b8fb0 2024-04-02T13:51:40.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.804+0000 7f5bc6ffd640 1 -- 172.21.15.122:0/1061308863 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd001f3d0 con 0x561d252b8fb0 2024-04-02T13:51:40.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.805+0000 7f5be999b740 1 -- 172.21.15.122:0/1061308863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 msgr2=0x561d252b9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.805+0000 7f5be999b740 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252b9390 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5be010b3a0 tx=0x7f5bd0009f90 comp rx=0 tx=0).stop 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.805+0000 7f5be5359640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 0x7f5bc80a61e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80ffa80 tx=0x7f5bd803cc20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.806+0000 7f5be999b740 1 -- 172.21.15.122:0/1061308863 shutdown_connections 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.806+0000 7f5be999b740 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.806+0000 7f5be999b740 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.806+0000 7f5be999b740 1 --2- 172.21.15.122:0/1061308863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252b9390 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.806+0000 7f5be999b740 1 -- 172.21.15.122:0/1061308863 >> 172.21.15.122:0/1061308863 conn(0x561d252b36e0 msgr2=0x561d252b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.807+0000 7f5be999b740 1 -- 172.21.15.122:0/1061308863 shutdown_connections 2024-04-02T13:51:40.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.807+0000 7f5be999b740 1 -- 172.21.15.122:0/1061308863 wait complete. 2024-04-02T13:51:40.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 -- start start 2024-04-02T13:51:40.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 Processor -- start 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252bf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561d252c52a0 con 0x561d252b8fb0 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561d251cfe80 con 0x561d252b98d0 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561d252c5420 con 0x561d252c24b0 2024-04-02T13:51:40.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.808+0000 7f5be999b740 10 client.0 _ll_register_callbacks cb 0x561d2520baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.809+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252bf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.809+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252bf330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35316/0 (socket says 172.21.15.122:35316) 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.809+0000 7f5bdffff640 1 -- 172.21.15.122:0/185375358 learned_addr learned my addr 172.21.15.122:0/185375358 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.809+0000 7f5bdffff640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 -- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 msgr2=0x561d252bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5be4b58640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 -- 172.21.15.122:0/185375358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 msgr2=0x561d252bf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252bf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 -- 172.21.15.122:0/185375358 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d24ff27d0 con 0x561d252c24b0 2024-04-02T13:51:40.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bdffff640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bfdb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0012010 tx=0x7f5bd0005590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9a57435f44e98f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.810+0000 7f5bbffff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:51:40.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.811+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005230 con 0x561d252c24b0 2024-04-02T13:51:40.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.811+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd0011db0 con 0x561d252c24b0 2024-04-02T13:51:40.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.811+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f5bd0004a40 con 0x561d252c24b0 2024-04-02T13:51:40.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.812+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5bd0004c40 con 0x561d252c24b0 2024-04-02T13:51:40.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.813+0000 7f5be999b740 10 client.29017 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.813+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561d252d1390 con 0x561d252c24b0 2024-04-02T13:51:40.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.813+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0011db0 con 0x561d252c24b0 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f5bd002c020 con 0x561d252c24b0 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 10 client.29017 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 10 client.29017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 20 client.29017 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d252d26e0 0x561d252d4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 4 client.29017 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:40.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d25221cf0 con 0x561d252d26e0 2024-04-02T13:51:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.814+0000 7f5be4b58640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d252d26e0 0x561d252d4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.815+0000 7f5be4b58640 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d252d26e0 0x561d252d4ad0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0248b243dc6f9fc server_cookie=811da1c07c589321 in_seq=0 out_seq=0 2024-04-02T13:51:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.815+0000 7f5bbffff640 10 client.29017 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.816+0000 7f5bbffff640 1 -- 172.21.15.122:0/185375358 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d25221cf0 con 0x561d252d26e0 2024-04-02T13:51:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.816+0000 7f5bbffff640 10 client.29017 handle_command_reply: tid=0 2024-04-02T13:51:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.816+0000 7f5be999b740 1 client.29017 shutdown 2024-04-02T13:51:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.816+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 msgr2=0x561d252bfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.816+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bfdb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0012010 tx=0x7f5bd0005590 comp rx=0 tx=0).stop 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 shutdown_connections 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d252d26e0 0x561d252d4ad0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bfdb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 --2- 172.21.15.122:0/185375358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252bf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 >> 172.21.15.122:0/185375358 conn(0x561d252b36e0 msgr2=0x561d252b5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5bbffff640 0 client.29017 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:40.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:40 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1360491913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:51:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 shutdown_connections 2024-04-02T13:51:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.817+0000 7f5be999b740 1 -- 172.21.15.122:0/185375358 wait complete. 2024-04-02T13:51:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.818+0000 7f5be999b740 20 client.29017 trim_cache size 0 max 16384 2024-04-02T13:51:40.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.844+0000 7f5be999b740 1 Processor -- start 2024-04-02T13:51:40.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.845+0000 7f5be999b740 1 -- start start 2024-04-02T13:51:40.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34342/0 (socket says 172.21.15.122:34342) 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252c24b0 0x561d252da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5bdffff640 1 -- 172.21.15.122:0/2367142901 learned_addr learned my addr 172.21.15.122:0/2367142901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a46c0 con 0x561d252c24b0 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a6380 con 0x561d252b98d0 2024-04-02T13:51:40.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.846+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a4d90 con 0x561d252b8fb0 2024-04-02T13:51:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.851+0000 7f5bdffff640 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252c24b0 0x561d252da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.851+0000 7f5be4b58640 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 -- 172.21.15.122:0/2367142901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 msgr2=0x561d252bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 -- 172.21.15.122:0/2367142901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252c24b0 msgr2=0x561d252da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252c24b0 0x561d252da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 -- 172.21.15.122:0/2367142901 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d252bd8a0 con 0x561d252b8fb0 2024-04-02T13:51:40.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.854+0000 7f5bdffff640 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009540 tx=0x7f5bd00021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c609ef165475b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.856+0000 7f5bc67fc640 1 -- 172.21.15.122:0/2367142901 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005600 con 0x561d252b8fb0 2024-04-02T13:51:40.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.856+0000 7f5bc67fc640 1 -- 172.21.15.122:0/2367142901 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0017040 con 0x561d252b8fb0 2024-04-02T13:51:40.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.856+0000 7f5bc67fc640 1 -- 172.21.15.122:0/2367142901 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd00135a0 con 0x561d252b8fb0 2024-04-02T13:51:40.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.857+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 msgr2=0x561d252ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.857+0000 7f5be999b740 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0009540 tx=0x7f5bd00021d0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.858+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 shutdown_connections 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.858+0000 7f5be999b740 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252c24b0 0x561d252da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.858+0000 7f5be999b740 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.858+0000 7f5be999b740 1 --2- 172.21.15.122:0/2367142901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252b8fb0 0x561d252ce160 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.858+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 >> 172.21.15.122:0/2367142901 conn(0x561d252b36e0 msgr2=0x561d252a9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.859+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 shutdown_connections 2024-04-02T13:51:40.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.860+0000 7f5be999b740 1 -- 172.21.15.122:0/2367142901 wait complete. 2024-04-02T13:51:40.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.860+0000 7f5be999b740 1 -- start start 2024-04-02T13:51:40.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 Processor -- start 2024-04-02T13:51:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5bdffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35336/0 (socket says 172.21.15.122:35336) 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5bdffff640 1 -- 172.21.15.122:0/1565503904 learned_addr learned my addr 172.21.15.122:0/1565503904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a5bf0 con 0x561d252b8fb0 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a5520 con 0x561d252b98d0 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.861+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5be01a26c0 con 0x561d252c24b0 2024-04-02T13:51:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.862+0000 7f5be999b740 10 client.0 _ll_register_callbacks cb 0x7f5bd0013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.863+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.864+0000 7f5bdffff640 1 -- 172.21.15.122:0/1565503904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 msgr2=0x561d252bc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.864+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.864+0000 7f5bdffff640 1 -- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 msgr2=0x561d252bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.864+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.864+0000 7f5bdffff640 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d252bc850 con 0x561d252b8fb0 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.865+0000 7f5bdffff640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0013bd0 tx=0x7f5bd0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a776d348369348a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.865+0000 7f5bbffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.867+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0013040 con 0x561d252b8fb0 2024-04-02T13:51:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.867+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bd0017040 con 0x561d252b8fb0 2024-04-02T13:51:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.867+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f5bd00131e0 con 0x561d252b8fb0 2024-04-02T13:51:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.867+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f5bd0005230 con 0x561d252b8fb0 2024-04-02T13:51:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.868+0000 7f5be999b740 10 client.20307 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.868+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x561d25338950 con 0x561d252b8fb0 2024-04-02T13:51:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.873+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bd0017040 con 0x561d252b8fb0 2024-04-02T13:51:40.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.873+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f5bd002c050 con 0x561d252b8fb0 2024-04-02T13:51:40.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 10 client.20307 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 10 client.20307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 20 client.20307 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d25338fd0 0x561d2533b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 4 client.20307 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:51:40.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.875+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d2521c040 con 0x561d25338fd0 2024-04-02T13:51:40.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.877+0000 7f5be4b58640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d25338fd0 0x561d2533b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:40.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.881+0000 7f5be4b58640 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d25338fd0 0x561d2533b3f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a415ea952562c392 server_cookie=5f794f2cac96528a in_seq=0 out_seq=0 2024-04-02T13:51:40.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.881+0000 7f5bbffff640 10 client.20307 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:40.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.884+0000 7f5bbffff640 1 -- 172.21.15.122:0/1565503904 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d2521c040 con 0x561d25338fd0 2024-04-02T13:51:40.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.884+0000 7f5bbffff640 10 client.20307 handle_command_reply: tid=0 2024-04-02T13:51:40.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 client.20307 shutdown 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 msgr2=0x561d252ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f5bd0013bd0 tx=0x7f5bd0005c50 comp rx=0 tx=0).stop 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 shutdown_connections 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561d25338fd0 0x561d2533b3f0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561d252c24b0 0x561d252bc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561d252b98d0 0x561d252bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 --2- 172.21.15.122:0/1565503904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561d252b8fb0 0x561d252ca910 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 >> 172.21.15.122:0/1565503904 conn(0x561d252b36e0 msgr2=0x561d252b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.886+0000 7f5bbffff640 0 client.20307 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.887+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 shutdown_connections 2024-04-02T13:51:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.887+0000 7f5be999b740 1 -- 172.21.15.122:0/1565503904 wait complete. 2024-04-02T13:51:40.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.887+0000 7f5be999b740 20 client.20307 trim_cache size 0 max 16384 2024-04-02T13:51:40.891 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:51:40.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.890+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 msgr2=0x7f5bc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.890+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 0x7f5bc80a61e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80ffa80 tx=0x7f5bd803cc20 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.890+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 msgr2=0x7f5be010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.890+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 0x7f5be010a0e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009780 tx=0x7f5bd8045670 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 shutdown_connections 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5bc80a3d20 0x7f5bc80a61e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be010e4a0 0x7f5be010a0e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5be0069620 0x7f5be0109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 --2- 172.21.15.122:0/324471728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5be0068d00 0x7f5be010dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:40.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.891+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 >> 172.21.15.122:0/324471728 conn(0x7f5be0077b00 msgr2=0x7f5be010f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:40.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.896+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 shutdown_connections 2024-04-02T13:51:40.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:40.898+0000 7f5be6dbc640 1 -- 172.21.15.122:0/324471728 wait complete. 2024-04-02T13:51:41.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 101971 32.11 MB/sec execute 61 sec latency 70.402 ms 2024-04-02T13:51:41.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:41 smithi187 ceph-mon[36064]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 739 op/s 2024-04-02T13:51:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:41 smithi188 ceph-mon[36117]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 739 op/s 2024-04-02T13:51:41.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:41 smithi122 ceph-mon[32114]: pgmap v518: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 17 MiB/s wr, 739 op/s 2024-04-02T13:51:41.920 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:51:41.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:51:41.921 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:51:42.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.059+0000 7f3e541c8640 1 -- 172.21.15.122:0/1026240402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c100970 msgr2=0x7f3e4c100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.059+0000 7f3e541c8640 1 --2- 172.21.15.122:0/1026240402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c100970 0x7f3e4c100df0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c009930 tx=0x7f3e3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 -- 172.21.15.122:0/1026240402 shutdown_connections 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 --2- 172.21.15.122:0/1026240402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c101330 0x7f3e4c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 --2- 172.21.15.122:0/1026240402 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c100970 0x7f3e4c100df0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 --2- 172.21.15.122:0/1026240402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 0x7f3e4c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 -- 172.21.15.122:0/1026240402 >> 172.21.15.122:0/1026240402 conn(0x7f3e4c077b00 msgr2=0x7f3e4c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 -- 172.21.15.122:0/1026240402 shutdown_connections 2024-04-02T13:51:42.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.060+0000 7f3e541c8640 1 -- 172.21.15.122:0/1026240402 wait complete. 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 Processor -- start 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 -- start start 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 0x7f3e4c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 0x7f3e4c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c0698b0 con 0x7f3e4c101330 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c069730 con 0x7f3e4c100970 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e541c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c069a30 con 0x7f3e4c106980 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e51763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e51763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35354/0 (socket says 172.21.15.122:35354) 2024-04-02T13:51:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e51763640 1 -- 172.21.15.122:0/3379328084 learned_addr learned my addr 172.21.15.122:0/3379328084 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 0x7f3e4c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.061+0000 7f3e52765640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 0x7f3e4c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 -- 172.21.15.122:0/3379328084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 msgr2=0x7f3e4c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 0x7f3e4c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 -- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 msgr2=0x7f3e4c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 -- 172.21.15.122:0/3379328084 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c19d3e0 con 0x7f3e4c100970 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51763640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 0x7f3e4c1983f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800b3f0 tx=0x7f3e4800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e42ffd640 1 -- 172.21.15.122:0/3379328084 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e48015020 con 0x7f3e4c100970 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c19d6d0 con 0x7f3e4c100970 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c110140 con 0x7f3e4c100970 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e42ffd640 1 -- 172.21.15.122:0/3379328084 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e48011d10 con 0x7f3e4c100970 2024-04-02T13:51:42.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.062+0000 7f3e42ffd640 1 -- 172.21.15.122:0/3379328084 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800f2e0 con 0x7f3e4c100970 2024-04-02T13:51:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.066+0000 7f3e42ffd640 1 -- 172.21.15.122:0/3379328084 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3e48018020 con 0x7f3e4c100970 2024-04-02T13:51:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.067+0000 7f3e56da9740 1 Processor -- start 2024-04-02T13:51:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.068+0000 7f3e56da9740 1 -- start start 2024-04-02T13:51:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.068+0000 7f3e42ffd640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 0x7f3e380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.068+0000 7f3e42ffd640 1 -- 172.21.15.122:0/3379328084 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f3e480d0250 con 0x7f3e4c100970 2024-04-02T13:51:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.068+0000 7f3e52765640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 0x7f3e380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e52765640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 0x7f3e380a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005fd0 tx=0x7f3e44009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d24be0 0x55c646d2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d2fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c646c88770 con 0x55c646d242c0 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c646c30960 con 0x55c646d24be0 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e56da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c646c848c0 con 0x55c646d2d7c0 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35366/0 (socket says 172.21.15.122:35366) 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e51f64640 1 -- 172.21.15.122:0/3313029124 learned_addr learned my addr 172.21.15.122:0/3313029124 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.069+0000 7f3e52765640 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d2fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 -- 172.21.15.122:0/3313029124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 msgr2=0x55c646d2fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d2fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 -- 172.21.15.122:0/3313029124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d24be0 msgr2=0x55c646d2d280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d24be0 0x55c646d2d280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 -- 172.21.15.122:0/3313029124 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c646a647e0 con 0x55c646d242c0 2024-04-02T13:51:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.070+0000 7f3e51f64640 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3e480a4650 tx=0x7f3e4809c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480893dfb6276d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.071+0000 7f3e37fff640 1 -- 172.21.15.122:0/3313029124 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480a02c0 con 0x55c646d242c0 2024-04-02T13:51:42.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.071+0000 7f3e37fff640 1 -- 172.21.15.122:0/3313029124 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e480a4b80 con 0x55c646d242c0 2024-04-02T13:51:42.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.071+0000 7f3e37fff640 1 -- 172.21.15.122:0/3313029124 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4809b360 con 0x55c646d242c0 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 -- 172.21.15.122:0/3313029124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 msgr2=0x55c646d246a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f3e480a4650 tx=0x7f3e4809c610 comp rx=0 tx=0).stop 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 -- 172.21.15.122:0/3313029124 shutdown_connections 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d2fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d24be0 0x55c646d2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 --2- 172.21.15.122:0/3313029124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d246a0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.072+0000 7f3e56da9740 1 -- 172.21.15.122:0/3313029124 >> 172.21.15.122:0/3313029124 conn(0x55c646d1e9f0 msgr2=0x55c646d1edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.073+0000 7f3e56da9740 1 -- 172.21.15.122:0/3313029124 shutdown_connections 2024-04-02T13:51:42.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.073+0000 7f3e56da9740 1 -- 172.21.15.122:0/3313029124 wait complete. 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.073+0000 7f3e56da9740 1 -- start start 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.073+0000 7f3e56da9740 1 Processor -- start 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d2add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d2d7c0 0x55c646d2b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c646d305b0 con 0x55c646d242c0 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c646c43d20 con 0x55c646d2d7c0 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c646d30730 con 0x55c646d24be0 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e56da9740 10 client.0 _ll_register_callbacks cb 0x55c646c10a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35382/0 (socket says 172.21.15.122:35382) 2024-04-02T13:51:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.074+0000 7f3e51f64640 1 -- 172.21.15.122:0/1319166623 learned_addr learned my addr 172.21.15.122:0/1319166623 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51763640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d2add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 -- 172.21.15.122:0/1319166623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 msgr2=0x55c646d2add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d2add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 -- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d2d7c0 msgr2=0x55c646d2b310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:51:42.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d2d7c0 0x55c646d2b310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 -- 172.21.15.122:0/1319166623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c646ac4f70 con 0x55c646d242c0 2024-04-02T13:51:42.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e51f64640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3e48098370 tx=0x7f3e480cda40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b04b095b5c99ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.075+0000 7f3e357fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:42.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.076+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e480a02c0 con 0x55c646d242c0 2024-04-02T13:51:42.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.076+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f3e480b5420 con 0x55c646d242c0 2024-04-02T13:51:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.076+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e480a4b80 con 0x55c646d242c0 2024-04-02T13:51:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.077+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3e4809b360 con 0x55c646d242c0 2024-04-02T13:51:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.077+0000 7f3e56da9740 10 client.20319 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.077+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c64695c170 con 0x55c646d242c0 2024-04-02T13:51:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.077+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4807f070 con 0x55c646d242c0 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f3e4809af70 con 0x55c646d242c0 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 10 client.20319 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 10 client.20319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 20 client.20319 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646d4e9c0 0x55c646d50db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 4 client.20319 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.078+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e480a4b80 con 0x55c646d4e9c0 2024-04-02T13:51:42.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.079+0000 7f3e51763640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646d4e9c0 0x55c646d50db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.079+0000 7f3e357fa640 10 client.20319 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.079+0000 7f3e51763640 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646d4e9c0 0x55c646d50db0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2daa96377e69c18 server_cookie=31c4ae958831ae10 in_seq=0 out_seq=0 2024-04-02T13:51:42.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.082+0000 7f3e357fa640 1 -- 172.21.15.122:0/1319166623 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3e480a4b80 con 0x55c646d4e9c0 2024-04-02T13:51:42.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.082+0000 7f3e357fa640 10 client.20319 handle_command_reply: tid=0 2024-04-02T13:51:42.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.082+0000 7f3e56da9740 1 client.20319 shutdown 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.082+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 msgr2=0x55c646d2a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.082+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f3e48098370 tx=0x7f3e480cda40 comp rx=0 tx=0).stop 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 shutdown_connections 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646d4e9c0 0x55c646d50db0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d2d7c0 0x55c646d2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d2add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 --2- 172.21.15.122:0/1319166623 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d242c0 0x55c646d2a890 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 >> 172.21.15.122:0/1319166623 conn(0x55c646d1e9f0 msgr2=0x55c646d20740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e357fa640 0 client.20319 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 shutdown_connections 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 1 -- 172.21.15.122:0/1319166623 wait complete. 2024-04-02T13:51:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.083+0000 7f3e56da9740 20 client.20319 trim_cache size 0 max 16384 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.113+0000 7f3e56da9740 1 Processor -- start 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.113+0000 7f3e56da9740 1 -- start start 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.113+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d2cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.113+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d2d7c0 0x55c646d20870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e56da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30010130 con 0x55c646d2d7c0 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e56da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30011df0 con 0x55c646d242c0 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e56da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30010800 con 0x55c646d24be0 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d2cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:34400/0 (socket says 172.21.15.122:34400) 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 -- 172.21.15.122:0/67749992 learned_addr learned my addr 172.21.15.122:0/67749992 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e52765640 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d2d7c0 0x55c646d20870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 -- 172.21.15.122:0/67749992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 msgr2=0x55c646d2cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d2cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 -- 172.21.15.122:0/67749992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d2d7c0 msgr2=0x55c646d20870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d2d7c0 0x55c646d20870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51763640 1 -- 172.21.15.122:0/67749992 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c646d1e220 con 0x55c646d24be0 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.114+0000 7f3e51f64640 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d2cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.115+0000 7f3e51763640 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c004ee0 tx=0x7f3e3c004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1539bf152f799424 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.115+0000 7f3e377fe640 1 -- 172.21.15.122:0/67749992 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c04c070 con 0x55c646d24be0 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.115+0000 7f3e377fe640 1 -- 172.21.15.122:0/67749992 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c046b50 con 0x55c646d24be0 2024-04-02T13:51:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.115+0000 7f3e377fe640 1 -- 172.21.15.122:0/67749992 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0462f0 con 0x55c646d24be0 2024-04-02T13:51:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 -- 172.21.15.122:0/67749992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 msgr2=0x55c646d20330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c004ee0 tx=0x7f3e3c004e70 comp rx=0 tx=0).stop 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 -- 172.21.15.122:0/67749992 shutdown_connections 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d2d7c0 0x55c646d20870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d24be0 0x55c646d20330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 --2- 172.21.15.122:0/67749992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d2cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 -- 172.21.15.122:0/67749992 >> 172.21.15.122:0/67749992 conn(0x55c646d1e9f0 msgr2=0x55c646d1fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 -- 172.21.15.122:0/67749992 shutdown_connections 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.116+0000 7f3e56da9740 1 -- 172.21.15.122:0/67749992 wait complete. 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.117+0000 7f3e56da9740 1 -- start start 2024-04-02T13:51:42.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.117+0000 7f3e56da9740 1 Processor -- start 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d36340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30011660 con 0x55c646d24be0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30010f90 con 0x55c646d242c0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000e130 con 0x55c646d2d7c0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e56da9740 10 client.0 _ll_register_callbacks cb 0x7f3e44009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35392/0 (socket says 172.21.15.122:35392) 2024-04-02T13:51:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51f64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37202/0 (socket says 172.21.15.122:37202) 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 -- 172.21.15.122:0/4184958065 learned_addr learned my addr 172.21.15.122:0/4184958065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e52765640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d36340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 -- 172.21.15.122:0/4184958065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 msgr2=0x55c646d36340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d36340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 -- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 msgr2=0x55c646d358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51763640 1 -- 172.21.15.122:0/4184958065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c646db4bb0 con 0x55c646d24be0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.118+0000 7f3e51f64640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.119+0000 7f3e51763640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c004a90 tx=0x7f3e3c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ffc9f83e4af70a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.119+0000 7f3e357fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.119+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c04c040 con 0x55c646d24be0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.119+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f3e3c004050 con 0x55c646d24be0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.119+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c046b50 con 0x55c646d24be0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f3e3c060050 con 0x55c646d24be0 2024-04-02T13:51:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 10 client.20325 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:42.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55c646db5e70 con 0x55c646d24be0 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e3c057070 con 0x55c646d24be0 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f3e3c004f70 con 0x55c646d24be0 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 10 client.20325 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 10 client.20325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 20 client.20325 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646db65a0 0x55c646db89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 4 client.20325 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3e3c046b50 con 0x55c646db65a0 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.120+0000 7f3e51f64640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646db65a0 0x55c646db89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.121+0000 7f3e357fa640 10 client.20325 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.121+0000 7f3e51f64640 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646db65a0 0x55c646db89c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b281b0f1f9d9c8 server_cookie=383588e8248949aa in_seq=0 out_seq=0 2024-04-02T13:51:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e357fa640 1 -- 172.21.15.122:0/4184958065 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3e3c046b50 con 0x55c646db65a0 2024-04-02T13:51:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e357fa640 10 client.20325 handle_command_reply: tid=0 2024-04-02T13:51:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 client.20325 shutdown 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 msgr2=0x55c646d35e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c004a90 tx=0x7f3e3c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 shutdown_connections 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c646db65a0 0x55c646db89c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c646d2d7c0 0x55c646d36340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c646d24be0 0x55c646d35e00 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 --2- 172.21.15.122:0/4184958065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c646d242c0 0x55c646d358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 >> 172.21.15.122:0/4184958065 conn(0x55c646d1e9f0 msgr2=0x55c646d20440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e357fa640 0 client.20325 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 shutdown_connections 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 1 -- 172.21.15.122:0/4184958065 wait complete. 2024-04-02T13:51:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.123+0000 7f3e56da9740 20 client.20325 trim_cache size 0 max 16384 2024-04-02T13:51:42.129 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:51:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "12e5c3b4-9048-460f-815e-74a9ae08a874", 2024-04-02T13:51:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:51:42.129 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 msgr2=0x7f3e380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 0x7f3e380a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3e44005fd0 tx=0x7f3e44009500 comp rx=0 tx=0).stop 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 msgr2=0x7f3e4c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 0x7f3e4c1983f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800b3f0 tx=0x7f3e4800b8b0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 shutdown_connections 2024-04-02T13:51:42.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e380a3d20 0x7f3e380a61e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e4c106980 0x7f3e4c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.124+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e4c101330 0x7f3e4c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.125+0000 7f3e541c8640 1 --2- 172.21.15.122:0/3379328084 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e4c100970 0x7f3e4c1983f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.125+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 >> 172.21.15.122:0/3379328084 conn(0x7f3e4c077b00 msgr2=0x7f3e4c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.125+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 shutdown_connections 2024-04-02T13:51:42.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.125+0000 7f3e541c8640 1 -- 172.21.15.122:0/3379328084 wait complete. 2024-04-02T13:51:42.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:51:42.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 -- 172.21.15.122:0/511086227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4101a90 msgr2=0x7f7df4101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/511086227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4101a90 0x7f7df4101f10 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009930 tx=0x7f7de002bc30 comp rx=0 tx=0).stop 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 -- 172.21.15.122:0/511086227 shutdown_connections 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/511086227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4102450 0x7f7df410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/511086227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4101a90 0x7f7df4101f10 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/511086227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4107aa0 0x7f7df4107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 -- 172.21.15.122:0/511086227 >> 172.21.15.122:0/511086227 conn(0x7f7df40fd5a0 msgr2=0x7f7df40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 -- 172.21.15.122:0/511086227 shutdown_connections 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.274+0000 7f7dfb10e640 1 -- 172.21.15.122:0/511086227 wait complete. 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 Processor -- start 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 -- start start 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4101a90 0x7f7df4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4102450 0x7f7df4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4111eb0 con 0x7f7df4107aa0 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4111d30 con 0x7f7df4102450 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7dfb10e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4112030 con 0x7f7df4101a90 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35408/0 (socket says 172.21.15.122:35408) 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7df96ab640 1 -- 172.21.15.122:0/3673417346 learned_addr learned my addr 172.21.15.122:0/3673417346 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4101a90 0x7f7df4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.275+0000 7f7df3fff640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4102450 0x7f7df4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 -- 172.21.15.122:0/3673417346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4101a90 msgr2=0x7f7df4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4101a90 0x7f7df4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 -- 172.21.15.122:0/3673417346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4102450 msgr2=0x7f7df4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4102450 0x7f7df4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 -- 172.21.15.122:0/3673417346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df419d3d0 con 0x7f7df4107aa0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df1ffb640 1 -- 172.21.15.122:0/3673417346 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec004b20 con 0x7f7df4107aa0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df419d6c0 con 0x7f7df4107aa0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.276+0000 7f7df1ffb640 1 -- 172.21.15.122:0/3673417346 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec004cc0 con 0x7f7df4107aa0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.277+0000 7f7df1ffb640 1 -- 172.21.15.122:0/3673417346 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010760 con 0x7f7df4107aa0 2024-04-02T13:51:42.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.277+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df41a4f50 con 0x7f7df4107aa0 2024-04-02T13:51:42.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.279+0000 7f7df1ffb640 1 -- 172.21.15.122:0/3673417346 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7dec005020 con 0x7f7df4107aa0 2024-04-02T13:51:42.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df1ffb640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 0x7f7de40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 Processor -- start 2024-04-02T13:51:42.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df1ffb640 1 -- 172.21.15.122:0/3673417346 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f7dec0d1220 con 0x7f7df4107aa0 2024-04-02T13:51:42.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 -- start start 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 0x7f7de40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76330 0x557267e76710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76ce0 0x557267e77160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557267dda830 con 0x557267e88500 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557267d82820 con 0x557267e76ce0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557267dd67e0 con 0x557267e76330 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35424/0 (socket says 172.21.15.122:35424) 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.280+0000 7f7df96ab640 1 -- 172.21.15.122:0/3485025549 learned_addr learned my addr 172.21.15.122:0/3485025549 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df3fff640 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76ce0 0x557267e77160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76330 0x557267e76710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 0x7f7de40a6260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7de800bb70 tx=0x7f7de800bf70 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df96ab640 1 -- 172.21.15.122:0/3485025549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76330 msgr2=0x557267e76710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76330 0x557267e76710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df96ab640 1 -- 172.21.15.122:0/3485025549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76ce0 msgr2=0x557267e77160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76ce0 0x557267e77160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.281+0000 7f7df96ab640 1 -- 172.21.15.122:0/3485025549 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557267bb87d0 con 0x557267e88500 2024-04-02T13:51:42.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.282+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0997a0 tx=0x7f7dec004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce5d063541489ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.282+0000 7f7ddeffd640 1 -- 172.21.15.122:0/3485025549 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec00f3c0 con 0x557267e88500 2024-04-02T13:51:42.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.282+0000 7f7ddeffd640 1 -- 172.21.15.122:0/3485025549 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dec09d4d0 con 0x557267e88500 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.282+0000 7f7ddeffd640 1 -- 172.21.15.122:0/3485025549 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec007b00 con 0x557267e88500 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3485025549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 msgr2=0x557267e8a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0997a0 tx=0x7f7dec004f20 comp rx=0 tx=0).stop 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3485025549 shutdown_connections 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e88500 0x557267e8a8f0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76ce0 0x557267e77160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3485025549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76330 0x557267e76710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3485025549 >> 172.21.15.122:0/3485025549 conn(0x557267e71000 msgr2=0x557267e713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3485025549 shutdown_connections 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3485025549 wait complete. 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 -- start start 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.283+0000 7f7dfddd1740 1 Processor -- start 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76330 0x557267e7b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e88500 0x557267e789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557267e8b390 con 0x557267e76ce0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557267d95d30 con 0x557267e76330 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557267e8b510 con 0x557267e88500 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7dfddd1740 10 client.0 _ll_register_callbacks cb 0x557267d95720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35434/0 (socket says 172.21.15.122:35434) 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df8eaa640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76330 0x557267e7b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 -- 172.21.15.122:0/497342295 learned_addr learned my addr 172.21.15.122:0/497342295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df96ab640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e88500 0x557267e789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 -- 172.21.15.122:0/497342295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e88500 msgr2=0x557267e789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e88500 0x557267e789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 -- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76330 msgr2=0x557267e7b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76330 0x557267e7b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.284+0000 7f7df3fff640 1 -- 172.21.15.122:0/497342295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557267aee3f0 con 0x557267e76ce0 2024-04-02T13:51:42.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7df3fff640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f317149f14e0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de00042c0 con 0x557267e76ce0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f7de0004460 con 0x557267e76ce0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dec09d4d0 con 0x557267e76ce0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7de00046c0 con 0x557267e76ce0 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dfddd1740 10 client.20337 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:42.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557267aae0f0 con 0x557267e76ce0 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.285+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de0004d40 con 0x557267e76ce0 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f7de0003700 con 0x557267e76ce0 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 10 client.20337 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 10 client.20337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 20 client.20337 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267e9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 4 client.20337 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:51:42.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7dec09d4d0 con 0x557267e98590 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267e9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267e9a9c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf03c60bc572c84e server_cookie=b6c13f0837205a9c in_seq=0 out_seq=0 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.286+0000 7f7dd3fff640 10 client.20337 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.288+0000 7f7dd3fff640 1 -- 172.21.15.122:0/497342295 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f7dec09d4d0 con 0x557267e98590 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.288+0000 7f7dd3fff640 10 client.20337 handle_command_reply: tid=0 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.288+0000 7f7dfddd1740 1 client.20337 shutdown 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 msgr2=0x557267e784b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009900 tx=0x7f7de0005dc0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 shutdown_connections 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267e9a9c0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e88500 0x557267e789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dd3fff640 0 client.20337 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76ce0 0x557267e784b0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/497342295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e76330 0x557267e7b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 >> 172.21.15.122:0/497342295 conn(0x557267e71000 msgr2=0x557267e72d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 shutdown_connections 2024-04-02T13:51:42.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 1 -- 172.21.15.122:0/497342295 wait complete. 2024-04-02T13:51:42.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.289+0000 7f7dfddd1740 20 client.20337 trim_cache size 0 max 16384 2024-04-02T13:51:42.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 Processor -- start 2024-04-02T13:51:42.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 -- start start 2024-04-02T13:51:42.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e94290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267ea02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41acfc0 con 0x557267e76330 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.318+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41aec80 con 0x557267e88500 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ad690 con 0x557267e76ce0 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35442/0 (socket says 172.21.15.122:35442) 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 -- 172.21.15.122:0/3170153496 learned_addr learned my addr 172.21.15.122:0/3170153496 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df3fff640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e94290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267ea02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 -- 172.21.15.122:0/3170153496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 msgr2=0x557267e94290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e94290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 -- 172.21.15.122:0/3170153496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 msgr2=0x557267ea02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267ea02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 -- 172.21.15.122:0/3170153496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557267e75b90 con 0x557267e76330 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df3fff640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e94290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df96ab640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267ea02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.319+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7de8010860 tx=0x7f7de8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777313d86a6d48da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dde7fc640 1 -- 172.21.15.122:0/3170153496 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8011dc0 con 0x557267e76330 2024-04-02T13:51:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dde7fc640 1 -- 172.21.15.122:0/3170153496 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de801fe10 con 0x557267e76330 2024-04-02T13:51:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dde7fc640 1 -- 172.21.15.122:0/3170153496 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de8005310 con 0x557267e76330 2024-04-02T13:51:42.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3170153496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 msgr2=0x557267e777d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f7de8010860 tx=0x7f7de8015000 comp rx=0 tx=0).stop 2024-04-02T13:51:42.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3170153496 shutdown_connections 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267ea02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e94290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.320+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/3170153496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e777d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.321+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3170153496 >> 172.21.15.122:0/3170153496 conn(0x557267e71000 msgr2=0x557267e770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.321+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3170153496 shutdown_connections 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.321+0000 7f7dfddd1740 1 -- 172.21.15.122:0/3170153496 wait complete. 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.321+0000 7f7dfddd1740 1 -- start start 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.321+0000 7f7dfddd1740 1 Processor -- start 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ae4f0 con 0x557267e76330 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41ade20 con 0x557267e88500 2024-04-02T13:51:42.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7df41aafc0 con 0x557267e76ce0 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7dfddd1740 10 client.0 _ll_register_callbacks cb 0x7f7de002cc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df96ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37258/0 (socket says 172.21.15.122:37258) 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:35458/0 (socket says 172.21.15.122:35458) 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df96ab640 1 -- 172.21.15.122:0/2521825415 learned_addr learned my addr 172.21.15.122:0/2521825415 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df3fff640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 -- 172.21.15.122:0/2521825415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 msgr2=0x557267e868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 -- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 msgr2=0x557267efdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df8eaa640 1 -- 172.21.15.122:0/2521825415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557267efe080 con 0x557267e76330 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.322+0000 7f7df96ab640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:51:42.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7df8eaa640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011560 tx=0x7f7de802a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961992faf99d5310 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de801fb40 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9525+0+0 (secure 0 0 0) 0x7f7de802a3b0 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de801fe10 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7de8041050 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dfddd1740 10 client.20343 fetch_fsmap learned FSMap version 61 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x557267eff230 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.323+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7de801fe10 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 61) ==== 8410+0+0 (secure 0 0 0) 0x7f7de80047c0 con 0x557267e76330 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 10 client.20343 fetch_fsmap finished waiting for FSMap version 61 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 10 client.20343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 20 client.20343 populate_metadata read hostname 'smithi122' 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267f01a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 4 client.20343 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:51:42.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557267de7d70 con 0x557267e98590 2024-04-02T13:51:42.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7df3fff640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267f01a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:51:42.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 102648 31.98 MB/sec execute 62 sec latency 82.662 ms 2024-04-02T13:51:42.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7df3fff640 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267f01a40 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ed861d644ae6a1 server_cookie=cc3185eddddce2bd in_seq=0 out_seq=0 2024-04-02T13:51:42.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.324+0000 7f7dd3fff640 10 client.20343 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.325+0000 7f7dd3fff640 1 -- 172.21.15.122:0/2521825415 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557267de7d70 con 0x557267e98590 2024-04-02T13:51:42.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.325+0000 7f7dd3fff640 10 client.20343 handle_command_reply: tid=0 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.325+0000 7f7dfddd1740 1 client.20343 shutdown 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.325+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 msgr2=0x557267e86370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.325+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f7de8011560 tx=0x7f7de802a910 comp rx=0 tx=0).stop 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 shutdown_connections 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557267e98590 0x557267f01a40 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557267e88500 0x557267efdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557267e76ce0 0x557267e868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 --2- 172.21.15.122:0/2521825415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557267e76330 0x557267e86370 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 >> 172.21.15.122:0/2521825415 conn(0x557267e71000 msgr2=0x557267e78450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dd3fff640 0 client.20343 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 shutdown_connections 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 1 -- 172.21.15.122:0/2521825415 wait complete. 2024-04-02T13:51:42.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.326+0000 7f7dfddd1740 20 client.20343 trim_cache size 0 max 16384 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "12e5c3b4-9048-460f-815e-74a9ae08a874": { 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "12e5c3b4-9048-460f-815e-74a9ae08a874", 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:51:42.335 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:51:42.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 msgr2=0x7f7de40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 0x7f7de40a6260 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7de800bb70 tx=0x7f7de800bf70 comp rx=0 tx=0).stop 2024-04-02T13:51:42.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 msgr2=0x7f7df419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:51:42.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 shutdown_connections 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7de40a3da0 0x7f7de40a6260 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7df4107aa0 0x7f7df419cca0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7df4102450 0x7f7df4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 --2- 172.21.15.122:0/3673417346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df4101a90 0x7f7df4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 >> 172.21.15.122:0/3673417346 conn(0x7f7df40fd5a0 msgr2=0x7f7df40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 shutdown_connections 2024-04-02T13:51:42.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:51:42.327+0000 7f7dfb10e640 1 -- 172.21.15.122:0/3673417346 wait complete. 2024-04-02T13:51:42.343 INFO:tasks.cephfs.filesystem:scrub status for tag:12e5c3b4-9048-460f-815e-74a9ae08a874 - {'path': '/', 'tag': '12e5c3b4-9048-460f-815e-74a9ae08a874', 'options': 'recursive,force'} 2024-04-02T13:51:43.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 103593 32.40 MB/sec execute 63 sec latency 95.957 ms 2024-04-02T13:51:43.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 976 op/s 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:43 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 976 op/s 2024-04-02T13:51:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:51:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:43 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: pgmap v519: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 976 op/s 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:43 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:51:44.327 INFO:tasks.workunit.client.0.smithi122.stdout: 10 104317 32.66 MB/sec execute 64 sec latency 81.644 ms 2024-04-02T13:51:45.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 104929 32.57 MB/sec execute 65 sec latency 44.318 ms 2024-04-02T13:51:45.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:45 smithi187 ceph-mon[36064]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 646 op/s 2024-04-02T13:51:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:45 smithi188 ceph-mon[36117]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 646 op/s 2024-04-02T13:51:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:45 smithi122 ceph-mon[32114]: pgmap v520: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.0 MiB/s wr, 646 op/s 2024-04-02T13:51:46.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 105472 32.50 MB/sec execute 66 sec latency 64.657 ms 2024-04-02T13:51:47.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 106381 32.57 MB/sec execute 67 sec latency 54.917 ms 2024-04-02T13:51:47.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:47 smithi187 ceph-mon[36064]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-04-02T13:51:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:47 smithi188 ceph-mon[36117]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-04-02T13:51:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:47 smithi122 ceph-mon[32114]: pgmap v521: 129 pgs: 129 active+clean; 1.5 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.7 MiB/s wr, 642 op/s 2024-04-02T13:51:48.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 107276 32.98 MB/sec execute 68 sec latency 55.407 ms 2024-04-02T13:51:49.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 107303 32.51 MB/sec execute 69 sec latency 1040.857 ms 2024-04-02T13:51:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:49 smithi187 ceph-mon[36064]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-04-02T13:51:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:49 smithi188 ceph-mon[36117]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-04-02T13:51:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:49 smithi122 ceph-mon[32114]: pgmap v522: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 717 op/s 2024-04-02T13:51:50.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 107526 32.29 MB/sec execute 70 sec latency 1064.601 ms 2024-04-02T13:51:51.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 107817 32.08 MB/sec execute 71 sec latency 150.732 ms 2024-04-02T13:51:51.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:51 smithi187 ceph-mon[36064]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 496 op/s 2024-04-02T13:51:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:51 smithi188 ceph-mon[36117]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 496 op/s 2024-04-02T13:51:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:51 smithi122 ceph-mon[32114]: pgmap v523: 129 pgs: 129 active+clean; 1.5 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 496 op/s 2024-04-02T13:51:52.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 108202 31.88 MB/sec execute 72 sec latency 150.463 ms 2024-04-02T13:51:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:52 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:52 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:52.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:52 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:53.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 108862 31.83 MB/sec execute 73 sec latency 54.389 ms 2024-04-02T13:51:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:53 smithi187 ceph-mon[36064]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 840 op/s 2024-04-02T13:51:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:53 smithi188 ceph-mon[36117]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 840 op/s 2024-04-02T13:51:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:53 smithi122 ceph-mon[32114]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 17 MiB/s wr, 840 op/s 2024-04-02T13:51:54.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 109639 32.00 MB/sec execute 74 sec latency 70.521 ms 2024-04-02T13:51:55.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 110414 32.13 MB/sec execute 75 sec latency 81.544 ms 2024-04-02T13:51:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:55 smithi187 ceph-mon[36064]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T13:51:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:55 smithi188 ceph-mon[36117]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T13:51:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:55 smithi122 ceph-mon[32114]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 686 op/s 2024-04-02T13:51:56.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 111113 32.19 MB/sec execute 76 sec latency 81.636 ms 2024-04-02T13:51:57.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 111806 32.34 MB/sec execute 77 sec latency 59.916 ms 2024-04-02T13:51:57.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 737 op/s 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:51:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:57 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 737 op/s 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:51:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:57 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: pgmap v526: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 737 op/s 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:51:57.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:57 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:58.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 112431 32.32 MB/sec execute 78 sec latency 77.542 ms 2024-04-02T13:51:58.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:58 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:58.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:58 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:58.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:58 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:51:59.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 113034 32.31 MB/sec execute 79 sec latency 106.105 ms 2024-04-02T13:51:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:51:59 smithi188 ceph-mon[36117]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 839 op/s 2024-04-02T13:51:59.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:51:59 smithi122 ceph-mon[32114]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 839 op/s 2024-04-02T13:51:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:51:59 smithi187 ceph-mon[36064]: pgmap v527: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 839 op/s 2024-04-02T13:52:00.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 113739 32.32 MB/sec execute 80 sec latency 81.008 ms 2024-04-02T13:52:01.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114477 32.36 MB/sec execute 81 sec latency 157.893 ms 2024-04-02T13:52:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:01 smithi122 ceph-mon[32114]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-04-02T13:52:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:01 smithi187 ceph-mon[36064]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-04-02T13:52:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:01 smithi188 ceph-mon[36117]: pgmap v528: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 697 op/s 2024-04-02T13:52:02.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114701 32.07 MB/sec execute 82 sec latency 354.680 ms 2024-04-02T13:52:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:02 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:02.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:02 smithi122 ceph-mon[32114]: osdmap e116: 12 total, 12 up, 12 in 2024-04-02T13:52:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:02 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:02.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:02 smithi187 ceph-mon[36064]: osdmap e116: 12 total, 12 up, 12 in 2024-04-02T13:52:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:02 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:02 smithi188 ceph-mon[36117]: osdmap e116: 12 total, 12 up, 12 in 2024-04-02T13:52:03.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114724 31.73 MB/sec execute 83 sec latency 324.652 ms 2024-04-02T13:52:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:03 smithi122 ceph-mon[32114]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-04-02T13:52:03.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:03 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:03 smithi187 ceph-mon[36064]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-04-02T13:52:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:03 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:03 smithi188 ceph-mon[36117]: pgmap v530: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 697 op/s 2024-04-02T13:52:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:03 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:04.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 114836 31.47 MB/sec execute 84 sec latency 274.518 ms 2024-04-02T13:52:05.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 115283 31.50 MB/sec execute 85 sec latency 258.907 ms 2024-04-02T13:52:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:05 smithi122 ceph-mon[32114]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-04-02T13:52:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:05 smithi187 ceph-mon[36064]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-04-02T13:52:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:05 smithi188 ceph-mon[36117]: pgmap v531: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.4 MiB/s wr, 642 op/s 2024-04-02T13:52:06.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 115932 31.53 MB/sec execute 86 sec latency 140.739 ms 2024-04-02T13:52:07.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 116614 31.59 MB/sec execute 87 sec latency 262.598 ms 2024-04-02T13:52:07.819 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:07 smithi188 ceph-mon[36117]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 704 op/s 2024-04-02T13:52:07.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:07 smithi122 ceph-mon[32114]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 704 op/s 2024-04-02T13:52:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:07 smithi187 ceph-mon[36064]: pgmap v532: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.9 MiB/s wr, 704 op/s 2024-04-02T13:52:08.328 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117127 31.52 MB/sec execute 88 sec latency 1262.634 ms 2024-04-02T13:52:08.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:08 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:08.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:08 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:09.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:08 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:09.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117477 31.39 MB/sec execute 89 sec latency 2262.675 ms 2024-04-02T13:52:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:09 smithi122 ceph-mon[32114]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:09 smithi187 ceph-mon[36064]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:09 smithi188 ceph-mon[36117]: pgmap v533: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:10.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117531 31.07 MB/sec execute 90 sec latency 3103.925 ms 2024-04-02T13:52:11.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 117782 30.83 MB/sec execute 91 sec latency 976.671 ms 2024-04-02T13:52:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:11 smithi187 ceph-mon[36064]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:11 smithi187 ceph-mon[36064]: osdmap e117: 12 total, 12 up, 12 in 2024-04-02T13:52:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:11 smithi188 ceph-mon[36117]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:11 smithi188 ceph-mon[36117]: osdmap e117: 12 total, 12 up, 12 in 2024-04-02T13:52:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:11 smithi122 ceph-mon[32114]: pgmap v534: 129 pgs: 129 active+clean; 1.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 672 op/s 2024-04-02T13:52:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:11 smithi122 ceph-mon[32114]: osdmap e117: 12 total, 12 up, 12 in 2024-04-02T13:52:12.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 118249 30.80 MB/sec execute 92 sec latency 1976.705 ms 2024-04-02T13:52:12.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:52:12.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 -- 172.21.15.122:0/3382946677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c94068d00 msgr2=0x7f6c94069180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 --2- 172.21.15.122:0/3382946677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c94068d00 0x7f6c94069180 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009930 tx=0x7f6c8802bc20 comp rx=0 tx=0).stop 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 -- 172.21.15.122:0/3382946677 shutdown_connections 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 --2- 172.21.15.122:0/3382946677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c940696c0 0x7f6c9410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 --2- 172.21.15.122:0/3382946677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c94068d00 0x7f6c94069180 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 --2- 172.21.15.122:0/3382946677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94107b50 0x7f6c94107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 -- 172.21.15.122:0/3382946677 >> 172.21.15.122:0/3382946677 conn(0x7f6c94077b00 msgr2=0x7f6c94077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 -- 172.21.15.122:0/3382946677 shutdown_connections 2024-04-02T13:52:12.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.472+0000 7f6c99063640 1 -- 172.21.15.122:0/3382946677 wait complete. 2024-04-02T13:52:12.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 Processor -- start 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 -- start start 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c94107b50 0x7f6c9419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.473+0000 7f6c99063640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94111ec0 con 0x7f6c940696c0 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c99063640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94111d40 con 0x7f6c94068d00 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c99063640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c94112040 con 0x7f6c94107b50 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46960/0 (socket says 172.21.15.122:46960) 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 -- 172.21.15.122:0/2764130970 learned_addr learned my addr 172.21.15.122:0/2764130970 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47124/0 (socket says 172.21.15.122:47124) 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c94107b50 0x7f6c9419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 -- 172.21.15.122:0/2764130970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c94107b50 msgr2=0x7f6c9419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c94107b50 0x7f6c9419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 -- 172.21.15.122:0/2764130970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 msgr2=0x7f6c94198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 -- 172.21.15.122:0/2764130970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9419d370 con 0x7f6c940696c0 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802c160 tx=0x7f6c880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c7bfff640 1 -- 172.21.15.122:0/2764130970 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88049070 con 0x7f6c940696c0 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.474+0000 7f6c7bfff640 1 -- 172.21.15.122:0/2764130970 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c880054e0 con 0x7f6c940696c0 2024-04-02T13:52:12.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.475+0000 7f6c7bfff640 1 -- 172.21.15.122:0/2764130970 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88003af0 con 0x7f6c940696c0 2024-04-02T13:52:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.475+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9419d600 con 0x7f6c940696c0 2024-04-02T13:52:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.475+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c941008f0 con 0x7f6c940696c0 2024-04-02T13:52:12.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.477+0000 7f6c7bfff640 1 -- 172.21.15.122:0/2764130970 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6c8804c070 con 0x7f6c940696c0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c7bfff640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 0x7f6c7c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 Processor -- start 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 -- start start 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 0x7f6c7c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c7bfff640 1 -- 172.21.15.122:0/2764130970 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f6c880fb030 con 0x7f6c940696c0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac07cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac10890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557dfab6b830 con 0x557dfac10dd0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557dfab13820 con 0x557dfac078d0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.478+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557dfab677e0 con 0x557dfac081f0 2024-04-02T13:52:12.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac07cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac07cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47132/0 (socket says 172.21.15.122:47132) 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:46972/0 (socket says 172.21.15.122:46972) 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c92d9d640 1 -- 172.21.15.122:0/482519200 learned_addr learned my addr 172.21.15.122:0/482519200 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9259c640 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac10890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 0x7f6c7c0a6260 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c840095f0 tx=0x7f6c84011810 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 -- 172.21.15.122:0/482519200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 msgr2=0x557dfac10890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac10890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 -- 172.21.15.122:0/482519200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 msgr2=0x557dfac07cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac07cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 -- 172.21.15.122:0/482519200 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfa9497d0 con 0x557dfac10dd0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c9359e640 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009fd0 tx=0x7f6c8c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a0d1ce17dabdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c797fa640 1 -- 172.21.15.122:0/482519200 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c011440 con 0x557dfac10dd0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c797fa640 1 -- 172.21.15.122:0/482519200 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8c015030 con 0x557dfac10dd0 2024-04-02T13:52:12.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.479+0000 7f6c797fa640 1 -- 172.21.15.122:0/482519200 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c010850 con 0x557dfac10dd0 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/482519200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 msgr2=0x557dfac131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009fd0 tx=0x7f6c8c003d40 comp rx=0 tx=0).stop 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/482519200 shutdown_connections 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac131c0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac10890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/482519200 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac07cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/482519200 >> 172.21.15.122:0/482519200 conn(0x557dfac02000 msgr2=0x557dfac023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/482519200 shutdown_connections 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.480+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/482519200 wait complete. 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 -- start start 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 Processor -- start 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac0df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557dfac13bc0 con 0x557dfac081f0 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557dfab26d30 con 0x557dfac10dd0 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557dfac13d40 con 0x557dfac078d0 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37244/0 (socket says 172.21.15.122:37244) 2024-04-02T13:52:12.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2297977225 learned_addr learned my addr 172.21.15.122:0/2297977225 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.481+0000 7f6c9bc4f740 10 client.0 _ll_register_callbacks cb 0x557dfab26720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c9359e640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac0df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 msgr2=0x557dfac0df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac0df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2297977225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 msgr2=0x557dfac0d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac0d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2297977225 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfa87f3f0 con 0x557dfac078d0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400fc10 tx=0x7f6c84016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e2ed9dc730898ea server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8402b060 con 0x557dfac078d0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f6c8401db50 con 0x557dfac078d0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.482+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c8c015030 con 0x557dfac078d0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f6c84032050 con 0x557dfac078d0 2024-04-02T13:52:12.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 10 client.29068 fetch_fsmap learned FSMap version 67 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x557dfa83f0f0 con 0x557dfac078d0 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c84019050 con 0x557dfac078d0 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 67) ==== 8382+0+0 (secure 0 0 0) 0x7f6c8401c3a0 con 0x557dfac078d0 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 10 client.29068 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 10 client.29068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 20 client.29068 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac23620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 4 client.29068 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.483+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6c8c015030 con 0x557dfac211a0 2024-04-02T13:52:12.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.484+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac23620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.485+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac23620 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c518121a4b7b6b server_cookie=f433593bdd854bf8 in_seq=0 out_seq=0 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.485+0000 7f6c76ff5640 10 client.29068 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2297977225 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6c8c015030 con 0x557dfac211a0 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c76ff5640 10 client.29068 handle_command_reply: tid=0 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 client.29068 shutdown 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 msgr2=0x557dfac10650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400fc10 tx=0x7f6c84016000 comp rx=0 tx=0).stop 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 shutdown_connections 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac23620 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac0df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c76ff5640 0 client.29068 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac0d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2297977225 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac10650 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 >> 172.21.15.122:0/2297977225 conn(0x557dfac02000 msgr2=0x557dfac03d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 shutdown_connections 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2297977225 wait complete. 2024-04-02T13:52:12.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.486+0000 7f6c9bc4f740 20 client.29068 trim_cache size 0 max 16384 2024-04-02T13:52:12.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.510+0000 7f6c9bc4f740 1 Processor -- start 2024-04-02T13:52:12.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.510+0000 7f6c9bc4f740 1 -- start start 2024-04-02T13:52:12.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac0cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac03b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47162/0 (socket says 172.21.15.122:47162) 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2108319401 learned_addr learned my addr 172.21.15.122:0/2108319401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac0cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ace50 con 0x557dfac10dd0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941aea50 con 0x557dfac078d0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ad520 con 0x557dfac081f0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.511+0000 7f6c9359e640 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac03b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2108319401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 msgr2=0x557dfac0cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac0cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2108319401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 msgr2=0x557dfac03b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac03b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2108319401 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dfac131c0 con 0x557dfac078d0 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.512+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009250 tx=0x7f6c8401d890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fd1dbf9763ef09 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.513+0000 7f6c78ff9640 1 -- 172.21.15.122:0/2108319401 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017040 con 0x557dfac078d0 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.513+0000 7f6c78ff9640 1 -- 172.21.15.122:0/2108319401 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c8401cbc0 con 0x557dfac078d0 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.513+0000 7f6c78ff9640 1 -- 172.21.15.122:0/2108319401 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8401ddb0 con 0x557dfac078d0 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 msgr2=0x557dfac1ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6c84009250 tx=0x7f6c8401d890 comp rx=0 tx=0).stop 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 shutdown_connections 2024-04-02T13:52:12.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac10dd0 0x557dfac03b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac081f0 0x557dfac0cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2108319401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac078d0 0x557dfac1ca80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 >> 172.21.15.122:0/2108319401 conn(0x557dfac02000 msgr2=0x557dfac03450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.514+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 shutdown_connections 2024-04-02T13:52:12.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.515+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2108319401 wait complete. 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.515+0000 7f6c9bc4f740 1 -- start start 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.515+0000 7f6c9bc4f740 1 Processor -- start 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac19190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941ae380 con 0x557dfac081f0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37252/0 (socket says 172.21.15.122:37252) 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac19190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2037027037 learned_addr learned my addr 172.21.15.122:0/2037027037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941adcb0 con 0x557dfac10dd0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c941aae50 con 0x557dfac078d0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9bc4f740 10 client.0 _ll_register_callbacks cb 0x7f6c8400fb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c9359e640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 msgr2=0x557dfac14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac14110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2037027037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 msgr2=0x557dfac19190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac19190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.516+0000 7f6c92d9d640 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dfac14910 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c92d9d640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c84011eb0 tx=0x7f6c8401edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e42ee2f073b26737 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c76ff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c84017040 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dfab78d70 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f6c84005250 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f6c8401ce50 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c9bc4f740 10 client.29074 fetch_fsmap learned FSMap version 67 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.517+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x557dfac871d0 con 0x557dfac078d0 2024-04-02T13:52:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557dfab78d70 con 0x557dfac078d0 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 67) ==== 8382+0+0 (secure 0 0 0) 0x7f6c84004c40 con 0x557dfac078d0 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 10 client.29074 fetch_fsmap finished waiting for FSMap version 67 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 10 client.29074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 20 client.29074 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 4 client.29074 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557dfab730c0 con 0x557dfac211a0 2024-04-02T13:52:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.518+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c76ff5640 10 client.29074 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:12.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9259c640 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac89840 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bbdc011de248260 server_cookie=95d6189fde7f637d in_seq=0 out_seq=0 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c76ff5640 1 -- 172.21.15.122:0/2037027037 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557dfab730c0 con 0x557dfac211a0 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c76ff5640 10 client.29074 handle_command_reply: tid=0 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 client.29074 shutdown 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 msgr2=0x557dfac18c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c84011eb0 tx=0x7f6c8401edf0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 shutdown_connections 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557dfac211a0 0x557dfac89840 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557dfac10dd0 0x557dfac14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557dfac081f0 0x557dfac19190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 --2- 172.21.15.122:0/2037027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557dfac078d0 0x557dfac18c50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 >> 172.21.15.122:0/2037027037 conn(0x557dfac02000 msgr2=0x557dfac033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c76ff5640 0 client.29074 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.519+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 shutdown_connections 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.520+0000 7f6c9bc4f740 1 -- 172.21.15.122:0/2037027037 wait complete. 2024-04-02T13:52:12.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.520+0000 7f6c9bc4f740 20 client.29074 trim_cache size 0 max 16384 2024-04-02T13:52:12.526 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:52:12.526 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:52:12.526 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 msgr2=0x7f6c7c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 0x7f6c7c0a6260 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6c840095f0 tx=0x7f6c84011810 comp rx=0 tx=0).stop 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 msgr2=0x7f6c941988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802c160 tx=0x7f6c880042f0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 shutdown_connections 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6c7c0a3da0 0x7f6c7c0a6260 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c94107b50 0x7f6c9419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c940696c0 0x7f6c941988c0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 --2- 172.21.15.122:0/2764130970 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c94068d00 0x7f6c94198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.521+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 >> 172.21.15.122:0/2764130970 conn(0x7f6c94077b00 msgr2=0x7f6c940fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.522+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 shutdown_connections 2024-04-02T13:52:12.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.522+0000 7f6c99063640 1 -- 172.21.15.122:0/2764130970 wait complete. 2024-04-02T13:52:12.539 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:52:12.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 -- 172.21.15.122:0/3065468296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801058a0 msgr2=0x7f1a80105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 --2- 172.21.15.122:0/3065468296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801058a0 0x7f1a80105c80 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009900 tx=0x7f1a7c02bcd0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 -- 172.21.15.122:0/3065468296 shutdown_connections 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 --2- 172.21.15.122:0/3065468296 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 0x7f1a8010d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 --2- 172.21.15.122:0/3065468296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801009c0 0x7f1a80100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 --2- 172.21.15.122:0/3065468296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801058a0 0x7f1a80105c80 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 -- 172.21.15.122:0/3065468296 >> 172.21.15.122:0/3065468296 conn(0x7f1a80077a90 msgr2=0x7f1a80077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:12.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.671+0000 7f1a88255640 1 -- 172.21.15.122:0/3065468296 shutdown_connections 2024-04-02T13:52:12.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.672+0000 7f1a88255640 1 -- 172.21.15.122:0/3065468296 wait complete. 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.672+0000 7f1a88255640 1 Processor -- start 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 -- start start 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 0x7f1a800fe410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 0x7f1a800fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80110d90 con 0x7f1a801009c0 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80110c10 con 0x7f1a80101360 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a88255640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a80110f10 con 0x7f1a801058a0 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a85ff1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 0x7f1a800fe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37262/0 (socket says 172.21.15.122:37262) 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a857f0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 0x7f1a800fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 -- 172.21.15.122:0/1232806414 learned_addr learned my addr 172.21.15.122:0/1232806414 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:12.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a85ff1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 0x7f1a800fe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47012/0 (socket says 172.21.15.122:47012) 2024-04-02T13:52:12.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 -- 172.21.15.122:0/1232806414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 msgr2=0x7f1a800fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 0x7f1a800fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 -- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 msgr2=0x7f1a800fe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 0x7f1a800fe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.673+0000 7f1a867f2640 1 -- 172.21.15.122:0/1232806414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a801a6880 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a867f2640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800eb10 tx=0x7f1a7800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004b20 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a801a6b70 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a801a70c0 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a78004cc0 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.674+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78010690 con 0x7f1a801058a0 2024-04-02T13:52:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.675+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a64005290 con 0x7f1a801058a0 2024-04-02T13:52:12.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.677+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1a78005000 con 0x7f1a801058a0 2024-04-02T13:52:12.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.677+0000 7f1a76ffd640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 0x7f1a6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:12.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.678+0000 7f1a85ff1640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 0x7f1a6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:12.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.678+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f1a780d0f50 con 0x7f1a801058a0 2024-04-02T13:52:12.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.678+0000 7f1a85ff1640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 0x7f1a6c0a6260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c02efd0 tx=0x7f1a7c02cf90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:12.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:12.685+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f1a7809d200 con 0x7f1a801058a0 2024-04-02T13:52:13.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.031+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a64005850 con 0x7f1a801058a0 2024-04-02T13:52:13.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.032+0000 7f1a76ffd640 1 -- 172.21.15.122:0/1232806414 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+11949 (secure 0 0 0) 0x7f1a7809cb00 con 0x7f1a801058a0 2024-04-02T13:52:13.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:52:13.035 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:52:11.735177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:52:13.036 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 68 2024-04-02T13:52:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 msgr2=0x7f1a6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 0x7f1a6c0a6260 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c02efd0 tx=0x7f1a7c02cf90 comp rx=0 tx=0).stop 2024-04-02T13:52:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 msgr2=0x7f1a800fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800eb10 tx=0x7f1a7800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 shutdown_connections 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1a6c0a3da0 0x7f1a6c0a6260 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a801058a0 0x7f1a800fee90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.037+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1a80101360 0x7f1a800fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.038+0000 7f1a88255640 1 --2- 172.21.15.122:0/1232806414 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a801009c0 0x7f1a800fe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.038+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 >> 172.21.15.122:0/1232806414 conn(0x7f1a80077a90 msgr2=0x7f1a8006a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.038+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 shutdown_connections 2024-04-02T13:52:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.038+0000 7f1a88255640 1 -- 172.21.15.122:0/1232806414 wait complete. 2024-04-02T13:52:13.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 -- 172.21.15.122:0/344766136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0101350 msgr2=0x7fbca010d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/344766136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0101350 0x7fbca010d860 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009cd0 tx=0x7fbc9802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 -- 172.21.15.122:0/344766136 shutdown_connections 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/344766136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0101350 0x7fbca010d860 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/344766136 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0100990 0x7fbca0100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/344766136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 0x7fbca0106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 -- 172.21.15.122:0/344766136 >> 172.21.15.122:0/344766136 conn(0x7fbca00777b0 msgr2=0x7fbca00feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 -- 172.21.15.122:0/344766136 shutdown_connections 2024-04-02T13:52:13.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.190+0000 7fbca5cc5640 1 -- 172.21.15.122:0/344766136 wait complete. 2024-04-02T13:52:13.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 Processor -- start 2024-04-02T13:52:13.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 -- start start 2024-04-02T13:52:13.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0101350 0x7fbca0199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 0x7fbca01a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110db0 con 0x7fbca0100990 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110c30 con 0x7fbca0101350 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbca5cc5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbca0110f30 con 0x7fbca01069a0 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47022/0 (socket says 172.21.15.122:47022) 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1542244211 learned_addr learned my addr 172.21.15.122:0/1542244211 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.191+0000 7fbc9effd640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0101350 0x7fbca0199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 0x7fbca01a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1542244211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 msgr2=0x7fbca01a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 0x7fbca01a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1542244211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0101350 msgr2=0x7fbca0199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0101350 0x7fbca0199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1542244211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca01a4810 con 0x7fbca0100990 2024-04-02T13:52:13.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9f0 tx=0x7fbc9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9cff9640 1 -- 172.21.15.122:0/1542244211 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000cd80 con 0x7fbca0100990 2024-04-02T13:52:13.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9cff9640 1 -- 172.21.15.122:0/1542244211 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc90005500 con 0x7fbca0100990 2024-04-02T13:52:13.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca01a49e0 con 0x7fbca0100990 2024-04-02T13:52:13.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbc9cff9640 1 -- 172.21.15.122:0/1542244211 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90010690 con 0x7fbca0100990 2024-04-02T13:52:13.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.192+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca01a4e50 con 0x7fbca0100990 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9cff9640 1 -- 172.21.15.122:0/1542244211 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbc900046c0 con 0x7fbca0100990 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 Processor -- start 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 -- start start 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d48ace7b0 con 0x563d48b6a7a0 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563d48a6d570 con 0x563d48b73ca0 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbca8985740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d48aca760 con 0x563d48b6b0c0 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47026/0 (socket says 172.21.15.122:47026) 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9ffff640 1 -- 172.21.15.122:0/1796732867 learned_addr learned my addr 172.21.15.122:0/1796732867 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9effd640 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9cff9640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 0x7fbc880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 0x7fbc880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.196+0000 7fbc9ffff640 1 -- 172.21.15.122:0/1796732867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 msgr2=0x563d48b73760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b73760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 -- 172.21.15.122:0/1796732867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 msgr2=0x563d48b76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 -- 172.21.15.122:0/1796732867 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d487e23f0 con 0x563d48b6a7a0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9cff9640 1 -- 172.21.15.122:0/1542244211 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fbc900d0f70 con 0x7fbca0100990 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fbc98005e30 tx=0x7fbc9802ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5df89b256169ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc85ffb640 1 -- 172.21.15.122:0/1796732867 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98050070 con 0x563d48b6a7a0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc85ffb640 1 -- 172.21.15.122:0/1796732867 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98003e50 con 0x563d48b6a7a0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc85ffb640 1 -- 172.21.15.122:0/1796732867 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98039040 con 0x563d48b6a7a0 2024-04-02T13:52:13.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.197+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 0x7fbc880a61e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009430 tx=0x7fbc98048000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- 172.21.15.122:0/1796732867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 msgr2=0x563d48b6ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fbc98005e30 tx=0x7fbc9802ce80 comp rx=0 tx=0).stop 2024-04-02T13:52:13.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- 172.21.15.122:0/1796732867 shutdown_connections 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 --2- 172.21.15.122:0/1796732867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b6ab80 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- 172.21.15.122:0/1796732867 >> 172.21.15.122:0/1796732867 conn(0x563d48b64ed0 msgr2=0x563d48b652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- 172.21.15.122:0/1796732867 shutdown_connections 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- 172.21.15.122:0/1796732867 wait complete. 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 -- start start 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.198+0000 7fbca8985740 1 Processor -- start 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b70a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b71020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563d48b76a90 con 0x563d48b6a7a0 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563d48a89e80 con 0x563d48b6b0c0 2024-04-02T13:52:13.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563d48b76c10 con 0x563d48b73ca0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 10 client.0 _ll_register_callbacks cb 0x563d48ac5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47032/0 (socket says 172.21.15.122:47032) 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b71020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/548467928 learned_addr learned my addr 172.21.15.122:0/548467928 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9effd640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b70a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/548467928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 msgr2=0x563d48b71020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b71020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/548467928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 msgr2=0x563d48b70a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b70a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/548467928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d488ac7d0 con 0x563d48b6a7a0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b71020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbc90098f80 tx=0x7fbc900d1270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36167e23a50d5b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc7effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9009d220 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fbc900d30f0 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc98003e50 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.199+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fbc9009cb20 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 10 client.20379 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x563d48b86e90 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98003e50 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fbc900bbce0 con 0x563d48b6a7a0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 10 client.20379 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 10 client.20379 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 20 client.20379 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48b882f0 0x563d48b8a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 4 client.20379 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d48adbcf0 con 0x563d48b882f0 2024-04-02T13:52:13.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbc9effd640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48b882f0 0x563d48b8a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.200+0000 7fbc7effd640 10 client.20379 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.201+0000 7fbc9effd640 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48b882f0 0x563d48b8a740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1386e58f2c76de04 server_cookie=3761b790832ebd60 in_seq=0 out_seq=0 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.201+0000 7fbc7effd640 1 -- 172.21.15.122:0/548467928 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d48adbcf0 con 0x563d48b882f0 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.201+0000 7fbc7effd640 10 client.20379 handle_command_reply: tid=0 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 client.20379 shutdown 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 msgr2=0x563d48b73680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbc90098f80 tx=0x7fbc900d1270 comp rx=0 tx=0).stop 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 shutdown_connections 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48b882f0 0x563d48b8a740 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b71020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b70a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 --2- 172.21.15.122:0/548467928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b73680 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbc7effd640 0 client.20379 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 >> 172.21.15.122:0/548467928 conn(0x563d48b64ed0 msgr2=0x563d48b66c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 shutdown_connections 2024-04-02T13:52:13.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 1 -- 172.21.15.122:0/548467928 wait complete. 2024-04-02T13:52:13.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.202+0000 7fbca8985740 20 client.20379 trim_cache size 0 max 16384 2024-04-02T13:52:13.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 Processor -- start 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 -- start start 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b6f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01acec0 con 0x563d48b6a7a0 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01aeac0 con 0x563d48b6b0c0 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbca8985740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01ad590 con 0x563d48b73ca0 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47044/0 (socket says 172.21.15.122:47044) 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1395079227 learned_addr learned my addr 172.21.15.122:0/1395079227 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.220+0000 7fbc9effd640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1395079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 msgr2=0x563d48b6f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b6f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b6f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1395079227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 msgr2=0x563d48b75fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b75fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/1395079227 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d48b6aef0 con 0x563d48b6a7a0 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9effd640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b75fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b6f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbc900994f0 tx=0x7fbc90009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31ed5e70e62d54a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc857fa640 1 -- 172.21.15.122:0/1395079227 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90098dc0 con 0x563d48b6a7a0 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc857fa640 1 -- 172.21.15.122:0/1395079227 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9009de50 con 0x563d48b6a7a0 2024-04-02T13:52:13.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.221+0000 7fbc857fa640 1 -- 172.21.15.122:0/1395079227 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a0ed0 con 0x563d48b6a7a0 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- 172.21.15.122:0/1395079227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 msgr2=0x563d48b7f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fbc900994f0 tx=0x7fbc90009310 comp rx=0 tx=0).stop 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- 172.21.15.122:0/1395079227 shutdown_connections 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b73ca0 0x563d48b6f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b6b0c0 0x563d48b75fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 --2- 172.21.15.122:0/1395079227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b7f950 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- 172.21.15.122:0/1395079227 >> 172.21.15.122:0/1395079227 conn(0x563d48b64ed0 msgr2=0x563d48b6f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- 172.21.15.122:0/1395079227 shutdown_connections 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- 172.21.15.122:0/1395079227 wait complete. 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.222+0000 7fbca8985740 1 -- start start 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 Processor -- start 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b74bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b75100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47056/0 (socket says 172.21.15.122:47056) 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/582614394 learned_addr learned my addr 172.21.15.122:0/582614394 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01ae3f0 con 0x563d48b6a7a0 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01add20 con 0x563d48b73ca0 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbca01aaec0 con 0x563d48b6b0c0 2024-04-02T13:52:13.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbca8985740 10 client.0 _ll_register_callbacks cb 0x7fbc9000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9effd640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b74bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9ffff640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b75100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/582614394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 msgr2=0x563d48b74bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b74bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/582614394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 msgr2=0x563d48b75100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b75100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.223+0000 7fbc9f7fe640 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d48b7afe0 con 0x563d48b6a7a0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc9f7fe640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000cfd0 tx=0x7fbc90005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909498ddd5337f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90098dc0 con 0x563d48b6a7a0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fbc90099190 con 0x563d48b6a7a0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9009de50 con 0x563d48b6a7a0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fbc9009c270 con 0x563d48b6a7a0 2024-04-02T13:52:13.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 10 client.20385 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x563d48b7c1f0 con 0x563d48b6a7a0 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9009de50 con 0x563d48b6a7a0 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fbc9009bb70 con 0x563d48b6a7a0 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 10 client.20385 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 10 client.20385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 20 client.20385 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48bee880 0x563d48bf0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 4 client.20385 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.224+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d48ad6040 con 0x563d48bee880 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.225+0000 7fbc9effd640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48bee880 0x563d48bf0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.225+0000 7fbc7effd640 10 client.20385 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.225+0000 7fbc9effd640 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48bee880 0x563d48bf0c70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69b79788819e5ba4 server_cookie=4c059e7dd9bb9f00 in_seq=0 out_seq=0 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbc7effd640 1 -- 172.21.15.122:0/582614394 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d48ad6040 con 0x563d48bee880 2024-04-02T13:52:13.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbc7effd640 10 client.20385 handle_command_reply: tid=0 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 client.20385 shutdown 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 msgr2=0x563d48b74680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000cfd0 tx=0x7fbc90005730 comp rx=0 tx=0).stop 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 shutdown_connections 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563d48bee880 0x563d48bf0c70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563d48b73ca0 0x563d48b75100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d48b6b0c0 0x563d48b74bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbc7effd640 0 client.20385 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 --2- 172.21.15.122:0/582614394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563d48b6a7a0 0x563d48b74680 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 >> 172.21.15.122:0/582614394 conn(0x563d48b64ed0 msgr2=0x563d48b6fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 shutdown_connections 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 1 -- 172.21.15.122:0/582614394 wait complete. 2024-04-02T13:52:13.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.226+0000 7fbca8985740 20 client.20385 trim_cache size 0 max 16384 2024-04-02T13:52:13.234 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:13.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 msgr2=0x7fbc880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 0x7fbc880a61e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009430 tx=0x7fbc98048000 comp rx=0 tx=0).stop 2024-04-02T13:52:13.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 msgr2=0x7fbca0199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9f0 tx=0x7fbc9000edf0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 shutdown_connections 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbc880a3d20 0x7fbc880a61e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbca01069a0 0x7fbca01a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbca0101350 0x7fbca0199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 --2- 172.21.15.122:0/1542244211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbca0100990 0x7fbca0199250 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 >> 172.21.15.122:0/1542244211 conn(0x7fbca00777b0 msgr2=0x7fbca0103860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 shutdown_connections 2024-04-02T13:52:13.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.227+0000 7fbca5cc5640 1 -- 172.21.15.122:0/1542244211 wait complete. 2024-04-02T13:52:13.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:52:13.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 118783 30.82 MB/sec execute 93 sec latency 2976.736 ms 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 -- 172.21.15.122:0/2319985504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c100d30 msgr2=0x7fc30c104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 --2- 172.21.15.122:0/2319985504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c100d30 0x7fc30c104d70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc300009860 tx=0x7fc30002bb70 comp rx=0 tx=0).stop 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 -- 172.21.15.122:0/2319985504 shutdown_connections 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 --2- 172.21.15.122:0/2319985504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 0x7fc30c10d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 --2- 172.21.15.122:0/2319985504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c100d30 0x7fc30c104d70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 --2- 172.21.15.122:0/2319985504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 -- 172.21.15.122:0/2319985504 >> 172.21.15.122:0/2319985504 conn(0x7fc30c076f20 msgr2=0x7fc30c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 -- 172.21.15.122:0/2319985504 shutdown_connections 2024-04-02T13:52:13.375 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.372+0000 7fc31180f640 1 -- 172.21.15.122:0/2319985504 wait complete. 2024-04-02T13:52:13.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 Processor -- start 2024-04-02T13:52:13.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 -- start start 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 0x7fc30c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c199d40 0x7fc30c1a4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c110d80 con 0x7fc30c100380 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c110c00 con 0x7fc30c199d40 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.373+0000 7fc31180f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc30c110f00 con 0x7fc30c105450 2024-04-02T13:52:13.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47082/0 (socket says 172.21.15.122:47082) 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 -- 172.21.15.122:0/1512031685 learned_addr learned my addr 172.21.15.122:0/1512031685 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c199d40 0x7fc30c1a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 0x7fc30c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 -- 172.21.15.122:0/1512031685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 msgr2=0x7fc30c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 0x7fc30c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 -- 172.21.15.122:0/1512031685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c199d40 msgr2=0x7fc30c1a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c199d40 0x7fc30c1a4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 -- 172.21.15.122:0/1512031685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30c1971e0 con 0x7fc30c100380 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc30affd640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00e9c0 tx=0x7fc2fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc31080d640 1 -- 172.21.15.122:0/1512031685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc00ccc0 con 0x7fc30c100380 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc31080d640 1 -- 172.21.15.122:0/1512031685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2fc005500 con 0x7fc30c100380 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.374+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30c1974d0 con 0x7fc30c100380 2024-04-02T13:52:13.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.375+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30c197a40 con 0x7fc30c100380 2024-04-02T13:52:13.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.375+0000 7fc31080d640 1 -- 172.21.15.122:0/1512031685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc010430 con 0x7fc30c100380 2024-04-02T13:52:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 Processor -- start 2024-04-02T13:52:13.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 -- start start 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc31080d640 1 -- 172.21.15.122:0/1512031685 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc2fc0105d0 con 0x7fc30c100380 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c2f82417b0 con 0x55c2f82e6ca0 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2f81e0570 con 0x55c2f82de0c0 2024-04-02T13:52:13.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.378+0000 7fc3143f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c2f823d760 con 0x55c2f82dd7a0 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47096/0 (socket says 172.21.15.122:47096) 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30a7fc640 1 -- 172.21.15.122:0/3970713426 learned_addr learned my addr 172.21.15.122:0/3970713426 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc31080d640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 0x7fc2f40a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30affd640 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 0x7fc2f40a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc31080d640 1 -- 172.21.15.122:0/1512031685 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fc2fc0d0d20 con 0x7fc30c100380 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 -- 172.21.15.122:0/3970713426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 msgr2=0x55c2f82e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 -- 172.21.15.122:0/3970713426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 msgr2=0x55c2f82e9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30b7fe640 1 -- 172.21.15.122:0/3970713426 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f7f553f0 con 0x55c2f82dd7a0 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.379+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.380+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ddb80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc304005fd0 tx=0x7fc304013e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba6cc38d8b8917e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.380+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 0x7fc2f40a6310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc30401e000 tx=0x7fc304020000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.380+0000 7fc2f1ffb640 1 -- 172.21.15.122:0/3970713426 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30400fc60 con 0x55c2f82dd7a0 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.380+0000 7fc2f1ffb640 1 -- 172.21.15.122:0/3970713426 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304023030 con 0x55c2f82dd7a0 2024-04-02T13:52:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.380+0000 7fc2f1ffb640 1 -- 172.21.15.122:0/3970713426 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304010c40 con 0x55c2f82dd7a0 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- 172.21.15.122:0/3970713426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 msgr2=0x55c2f82ddb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ddb80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc304005fd0 tx=0x7fc304013e50 comp rx=0 tx=0).stop 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- 172.21.15.122:0/3970713426 shutdown_connections 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3970713426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ddb80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- 172.21.15.122:0/3970713426 >> 172.21.15.122:0/3970713426 conn(0x55c2f82d7ed0 msgr2=0x55c2f82d82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- 172.21.15.122:0/3970713426 shutdown_connections 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- 172.21.15.122:0/3970713426 wait complete. 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.381+0000 7fc3143f1740 1 -- start start 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 Processor -- start 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 0x55c2f82e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c2f82e9a30 con 0x55c2f82de0c0 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c2f81fce80 con 0x55c2f82e6ca0 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c2f82e9bb0 con 0x55c2f82dd7a0 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.382+0000 7fc3143f1740 10 client.0 _ll_register_callbacks cb 0x55c2f8238aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 0x55c2f82e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 0x55c2f82e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47106/0 (socket says 172.21.15.122:47106) 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30a7fc640 1 -- 172.21.15.122:0/2759979669 learned_addr learned my addr 172.21.15.122:0/2759979669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 -- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 msgr2=0x55c2f82e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 -- 172.21.15.122:0/2759979669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 msgr2=0x55c2f82e3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 0x55c2f82e3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 -- 172.21.15.122:0/2759979669 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f801f7d0 con 0x55c2f82e6ca0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e4220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3040133a0 tx=0x7fc304009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9f7ba5ec638646 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.383+0000 7fc2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.384+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304036060 con 0x55c2f82e6ca0 2024-04-02T13:52:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.384+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc304023030 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.384+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fc30401fce0 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc30403d050 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 10 client.29351 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c2f82f5b90 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc304023030 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fc304028470 con 0x55c2f82e6ca0 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 10 client.29351 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 10 client.29351 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 20 client.29351 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f82f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 4 client.29351 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.385+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f824ecf0 con 0x55c2f82f7360 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.386+0000 7fc30affd640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f82f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.386+0000 7fc30affd640 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f82f97c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b818f73f7dbfc0e2 server_cookie=a6765b95af16b1a7 in_seq=0 out_seq=0 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.386+0000 7fc2eaffd640 10 client.29351 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.387+0000 7fc2eaffd640 1 -- 172.21.15.122:0/2759979669 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2f824ecf0 con 0x55c2f82f7360 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.387+0000 7fc2eaffd640 10 client.29351 handle_command_reply: tid=0 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.387+0000 7fc3143f1740 1 client.29351 shutdown 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 msgr2=0x55c2f82e4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e4220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3040133a0 tx=0x7fc304009510 comp rx=0 tx=0).stop 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 shutdown_connections 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e4220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f82f97c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82de0c0 0x55c2f82e3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc2eaffd640 0 client.29351 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 --2- 172.21.15.122:0/2759979669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82dd7a0 0x55c2f82e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 >> 172.21.15.122:0/2759979669 conn(0x55c2f82d7ed0 msgr2=0x55c2f82d9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 shutdown_connections 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.388+0000 7fc3143f1740 1 -- 172.21.15.122:0/2759979669 wait complete. 2024-04-02T13:52:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.389+0000 7fc3143f1740 20 client.29351 trim_cache size 0 max 16384 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.408+0000 7fc3143f1740 1 Processor -- start 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.408+0000 7fc3143f1740 1 -- start start 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 0x55c2f82f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8010000 con 0x55c2f82dd7a0 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8011c00 con 0x55c2f82de0c0 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc3143f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80106d0 con 0x55c2f82e6ca0 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 0x55c2f82f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47270/0 (socket says 172.21.15.122:47270) 2024-04-02T13:52:13.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 -- 172.21.15.122:0/3200680375 learned_addr learned my addr 172.21.15.122:0/3200680375 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 0x55c2f82f9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47122/0 (socket says 172.21.15.122:47122) 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 -- 172.21.15.122:0/3200680375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82e6ca0 msgr2=0x55c2f82e6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 -- 172.21.15.122:0/3200680375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 msgr2=0x55c2f82f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 0x55c2f82f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.409+0000 7fc30a7fc640 1 -- 172.21.15.122:0/3200680375 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f82ddef0 con 0x55c2f82de0c0 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.410+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc30002c180 tx=0x7fc300009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeee733495e3eb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.410+0000 7fc2f17fa640 1 -- 172.21.15.122:0/3200680375 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300034cb0 con 0x55c2f82de0c0 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.410+0000 7fc2f17fa640 1 -- 172.21.15.122:0/3200680375 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc30004a040 con 0x55c2f82de0c0 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.410+0000 7fc2f17fa640 1 -- 172.21.15.122:0/3200680375 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30002c770 con 0x55c2f82de0c0 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 -- 172.21.15.122:0/3200680375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 msgr2=0x55c2f82f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc30002c180 tx=0x7fc300009570 comp rx=0 tx=0).stop 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 -- 172.21.15.122:0/3200680375 shutdown_connections 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82de0c0 0x55c2f82f2950 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3200680375 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82dd7a0 0x55c2f82f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.411+0000 7fc3143f1740 1 -- 172.21.15.122:0/3200680375 >> 172.21.15.122:0/3200680375 conn(0x55c2f82d7ed0 msgr2=0x55c2f82d82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.412+0000 7fc3143f1740 1 -- 172.21.15.122:0/3200680375 shutdown_connections 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.412+0000 7fc3143f1740 1 -- 172.21.15.122:0/3200680375 wait complete. 2024-04-02T13:52:13.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.412+0000 7fc3143f1740 1 -- start start 2024-04-02T13:52:13.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 Processor -- start 2024-04-02T13:52:13.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 0x55c2f82e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8011530 con 0x55c2f82e6ca0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.413+0000 7fc3143f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f8010e60 con 0x55c2f82dd7a0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc3143f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f800e000 con 0x55c2f82de0c0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47272/0 (socket says 172.21.15.122:47272) 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 -- 172.21.15.122:0/3042383606 learned_addr learned my addr 172.21.15.122:0/3042383606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc3143f1740 10 client.0 _ll_register_callbacks cb 0x7fc2fc09c640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30b7fe640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 0x55c2f82e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 -- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 msgr2=0x55c2f82e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 0x55c2f82e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 -- 172.21.15.122:0/3042383606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 msgr2=0x55c2f82e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 -- 172.21.15.122:0/3042383606 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f82dfd50 con 0x55c2f82dd7a0 2024-04-02T13:52:13.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 0x55c2f82e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc30affd640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8a017ed2a8bc47 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc2eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0a54e0 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fc2fc0bb6c0 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.414+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30004a040 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc098ee0 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 10 client.29357 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c2f835cb70 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc30004a040 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fc2fc0a5050 con 0x55c2f82dd7a0 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 10 client.29357 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 10 client.29357 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 20 client.29357 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.415+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f835f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc3143f1740 4 client.29357 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:13.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2f8249040 con 0x55c2f82f7360 2024-04-02T13:52:13.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f835f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc2eaffd640 10 client.29357 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc30a7fc640 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f835f2e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8f1afc6d2cfb1cd7 server_cookie=2860e9d0c3a9f47c in_seq=0 out_seq=0 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc2eaffd640 1 -- 172.21.15.122:0/3042383606 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2f8249040 con 0x55c2f82f7360 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc2eaffd640 10 client.29357 handle_command_reply: tid=0 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.416+0000 7fc3143f1740 1 client.29357 shutdown 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 msgr2=0x55c2f82ef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc000c00 tx=0x7fc2fc014000 comp rx=0 tx=0).stop 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 shutdown_connections 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55c2f82f7360 0x55c2f835f2e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c2f82e6ca0 0x55c2f82e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc2eaffd640 0 client.29357 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c2f82de0c0 0x55c2f82e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 --2- 172.21.15.122:0/3042383606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c2f82dd7a0 0x55c2f82ef340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 >> 172.21.15.122:0/3042383606 conn(0x55c2f82d7ed0 msgr2=0x55c2f82e2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 shutdown_connections 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 1 -- 172.21.15.122:0/3042383606 wait complete. 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.417+0000 7fc3143f1740 20 client.29357 trim_cache size 0 max 16384 2024-04-02T13:52:13.423 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 msgr2=0x7fc2f40a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 0x7fc2f40a6310 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc30401e000 tx=0x7fc304020000 comp rx=0 tx=0).stop 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 msgr2=0x7fc30c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00e9c0 tx=0x7fc2fc00ee80 comp rx=0 tx=0).stop 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 shutdown_connections 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc2f40a3e50 0x7fc2f40a6310 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc30c199d40 0x7fc30c1a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc30c105450 0x7fc30c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 --2- 172.21.15.122:0/1512031685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc30c100380 0x7fc30c1992c0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 >> 172.21.15.122:0/1512031685 conn(0x7fc30c076f20 msgr2=0x7fc30c103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 shutdown_connections 2024-04-02T13:52:13.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.418+0000 7fc31180f640 1 -- 172.21.15.122:0/1512031685 wait complete. 2024-04-02T13:52:13.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:52:13.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.569+0000 7f93956ba640 1 -- 172.21.15.122:0/886616658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390103620 msgr2=0x7f939010bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.569+0000 7f93956ba640 1 --2- 172.21.15.122:0/886616658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390103620 0x7f939010bb00 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f9384009900 tx=0x7f938402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.569+0000 7f93956ba640 1 -- 172.21.15.122:0/886616658 shutdown_connections 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.569+0000 7f93956ba640 1 --2- 172.21.15.122:0/886616658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 --2- 172.21.15.122:0/886616658 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390103620 0x7f939010bb00 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 --2- 172.21.15.122:0/886616658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390102d00 0x7f93901030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 -- 172.21.15.122:0/886616658 >> 172.21.15.122:0/886616658 conn(0x7f93900777b0 msgr2=0x7f93900feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 -- 172.21.15.122:0/886616658 shutdown_connections 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 -- 172.21.15.122:0/886616658 wait complete. 2024-04-02T13:52:13.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 Processor -- start 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.570+0000 7f93956ba640 1 -- start start 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390103620 0x7f9390198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9390111e70 con 0x7f9390102d00 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9390111cf0 con 0x7f939010c0d0 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f93956ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9390111ff0 con 0x7f9390103620 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47138/0 (socket says 172.21.15.122:47138) 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 -- 172.21.15.122:0/707953678 learned_addr learned my addr 172.21.15.122:0/707953678 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938e7fc640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390103620 0x7f9390198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938f7fe640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 -- 172.21.15.122:0/707953678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390103620 msgr2=0x7f9390198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390103620 0x7f9390198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 -- 172.21.15.122:0/707953678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 msgr2=0x7f939019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 -- 172.21.15.122:0/707953678 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939019d3e0 con 0x7f9390102d00 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938f7fe640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.571+0000 7f938effd640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f938000e9f0 tx=0x7f938000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.572+0000 7f9377fff640 1 -- 172.21.15.122:0/707953678 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938000cd80 con 0x7f9390102d00 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.572+0000 7f9377fff640 1 -- 172.21.15.122:0/707953678 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9380005500 con 0x7f9390102d00 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.572+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939019d6d0 con 0x7f9390102d00 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.572+0000 7f9377fff640 1 -- 172.21.15.122:0/707953678 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380010690 con 0x7f9390102d00 2024-04-02T13:52:13.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.572+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93901a4f60 con 0x7f9390102d00 2024-04-02T13:52:13.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.574+0000 7f9377fff640 1 -- 172.21.15.122:0/707953678 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f93800046c0 con 0x7f9390102d00 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 Processor -- start 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 -- start start 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93a7a0 0x562bee93ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee943760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee946090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bee89e7b0 con 0x562bee93b0c0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562bee83d570 con 0x562bee943ca0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f939837d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562bee89a760 con 0x562bee93a7a0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee943760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93a7a0 0x562bee93ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee946090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee943760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47148/0 (socket says 172.21.15.122:47148) 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938effd640 1 -- 172.21.15.122:0/3640103525 learned_addr learned my addr 172.21.15.122:0/3640103525 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f9377fff640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 0x7f93780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938f7fe640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 0x7f93780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f9377fff640 1 -- 172.21.15.122:0/707953678 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f93800d0f80 con 0x7f9390102d00 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938e7fc640 1 -- 172.21.15.122:0/3640103525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93a7a0 msgr2=0x562bee93ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93a7a0 0x562bee93ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938e7fc640 1 -- 172.21.15.122:0/3640103525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 msgr2=0x562bee943760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.575+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee943760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f938e7fc640 1 -- 172.21.15.122:0/3640103525 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bee5b23f0 con 0x562bee943ca0 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f938f7fe640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 0x7f93780a6260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f93880094d0 tx=0x7f9388004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee946090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960454a494930a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f93757fa640 1 -- 172.21.15.122:0/3640103525 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938402c920 con 0x562bee943ca0 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f93757fa640 1 -- 172.21.15.122:0/3640103525 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938402cac0 con 0x562bee943ca0 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.576+0000 7f93757fa640 1 -- 172.21.15.122:0/3640103525 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93840417a0 con 0x562bee943ca0 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- 172.21.15.122:0/3640103525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 msgr2=0x562bee946090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee946090 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384009560 comp rx=0 tx=0).stop 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- 172.21.15.122:0/3640103525 shutdown_connections 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee946090 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee943760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 --2- 172.21.15.122:0/3640103525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93a7a0 0x562bee93ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- 172.21.15.122:0/3640103525 >> 172.21.15.122:0/3640103525 conn(0x562bee934ed0 msgr2=0x562bee9352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- 172.21.15.122:0/3640103525 shutdown_connections 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- 172.21.15.122:0/3640103525 wait complete. 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.577+0000 7f939837d740 1 -- start start 2024-04-02T13:52:13.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 Processor -- start 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 0x562bee9409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee940f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562bee946a30 con 0x562bee93a7a0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562bee859e80 con 0x562bee943ca0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562bee946bb0 con 0x562bee93b0c0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f939837d740 10 client.0 _ll_register_callbacks cb 0x562bee895aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee940f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47154/0 (socket says 172.21.15.122:47154) 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee940f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47288/0 (socket says 172.21.15.122:47288) 2024-04-02T13:52:13.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938effd640 1 -- 172.21.15.122:0/3310698090 learned_addr learned my addr 172.21.15.122:0/3310698090 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.578+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 0x562bee9409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 -- 172.21.15.122:0/3310698090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 msgr2=0x562bee9409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 0x562bee9409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 -- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 msgr2=0x562bee940f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee940f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 -- 172.21.15.122:0/3310698090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bee67c7d0 con 0x562bee93a7a0 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 0x562bee9409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f938effd640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f93800049d0 tx=0x7f93800056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dda8fd0b8fa988 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.579+0000 7f9372ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938009de60 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f93800991a0 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f938402cac0 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f938009c490 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f939837d740 10 client.20415 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.581+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x562bee952a50 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938402cac0 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7f93800d3110 con 0x562bee93a7a0 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 10 client.20415 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 10 client.20415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 20 client.20415 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee953f60 0x562bee956420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 4 client.20415 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:13.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bee8abcf0 con 0x562bee953f60 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.582+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee953f60 0x562bee956420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.583+0000 7f938e7fc640 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee953f60 0x562bee956420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e0422e5bb0ddf11e server_cookie=1e2bbccf9a4364ba in_seq=0 out_seq=0 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.583+0000 7f9372ff5640 10 client.20415 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.583+0000 7f9372ff5640 1 -- 172.21.15.122:0/3310698090 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562bee8abcf0 con 0x562bee953f60 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.583+0000 7f9372ff5640 10 client.20415 handle_command_reply: tid=0 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.583+0000 7f939837d740 1 client.20415 shutdown 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 msgr2=0x562bee943620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f93800049d0 tx=0x7f93800056b0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 shutdown_connections 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee953f60 0x562bee956420 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee943ca0 0x562bee940f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee93b0c0 0x562bee9409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f9372ff5640 0 client.20415 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 --2- 172.21.15.122:0/3310698090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee943620 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 >> 172.21.15.122:0/3310698090 conn(0x562bee934ed0 msgr2=0x562bee936c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 shutdown_connections 2024-04-02T13:52:13.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 1 -- 172.21.15.122:0/3310698090 wait complete. 2024-04-02T13:52:13.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.584+0000 7f939837d740 20 client.20415 trim_cache size 0 max 16384 2024-04-02T13:52:13.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.600+0000 7f939837d740 1 Processor -- start 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.600+0000 7f939837d740 1 -- start start 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93a7a0 0x562bee94f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee943620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93901ace40 con 0x562bee93b0c0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93901aea40 con 0x562bee93a7a0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f939837d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93901ad510 con 0x562bee943ca0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47158/0 (socket says 172.21.15.122:47158) 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 -- 172.21.15.122:0/969157609 learned_addr learned my addr 172.21.15.122:0/969157609 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938effd640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93a7a0 0x562bee94f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938f7fe640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee943620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 -- 172.21.15.122:0/969157609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 msgr2=0x562bee943620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee943620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 -- 172.21.15.122:0/969157609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93a7a0 msgr2=0x562bee94f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93a7a0 0x562bee94f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 -- 172.21.15.122:0/969157609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bee939940 con 0x562bee93b0c0 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938f7fe640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee943620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f938e7fc640 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f93840049b0 tx=0x7f938402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7293525381b84468 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.601+0000 7f9374ff9640 1 -- 172.21.15.122:0/969157609 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384005d50 con 0x562bee93b0c0 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f9374ff9640 1 -- 172.21.15.122:0/969157609 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93840046c0 con 0x562bee93b0c0 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f9374ff9640 1 -- 172.21.15.122:0/969157609 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384004f10 con 0x562bee93b0c0 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 -- 172.21.15.122:0/969157609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 msgr2=0x562bee9430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f93840049b0 tx=0x7f938402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 -- 172.21.15.122:0/969157609 shutdown_connections 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee943620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93b0c0 0x562bee9430e0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 --2- 172.21.15.122:0/969157609 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93a7a0 0x562bee94f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 -- 172.21.15.122:0/969157609 >> 172.21.15.122:0/969157609 conn(0x562bee934ed0 msgr2=0x562bee9352b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 -- 172.21.15.122:0/969157609 shutdown_connections 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.602+0000 7f939837d740 1 -- 172.21.15.122:0/969157609 wait complete. 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 -- start start 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 Processor -- start 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 0x562bee944e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee94ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93901ae370 con 0x562bee93a7a0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93901adca0 con 0x562bee93b0c0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93901aae40 con 0x562bee943ca0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f939837d740 10 client.0 _ll_register_callbacks cb 0x7f938000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f938e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 0x562bee944e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f938effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47162/0 (socket says 172.21.15.122:47162) 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f938effd640 1 -- 172.21.15.122:0/925879068 learned_addr learned my addr 172.21.15.122:0/925879068 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.603+0000 7f938f7fe640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee94ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 -- 172.21.15.122:0/925879068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 msgr2=0x562bee94ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee94ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 -- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 msgr2=0x562bee944e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 0x562bee944e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 -- 172.21.15.122:0/925879068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bee94b370 con 0x562bee93a7a0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938e7fc640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 0x562bee944e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f938effd640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9380099500 tx=0x7f93800a5520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3ef77b18d819cd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938009de60 con 0x562bee93a7a0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f93800d4380 con 0x562bee93a7a0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93840046c0 con 0x562bee93a7a0 2024-04-02T13:52:13.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f938009d230 con 0x562bee93a7a0 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 10 client.20421 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x562bee94c670 con 0x562bee93a7a0 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93840046c0 con 0x562bee93a7a0 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7f938009bf80 con 0x562bee93a7a0 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 10 client.20421 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 10 client.20421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 20 client.20421 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee9baa60 0x562bee9bce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 4 client.20421 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.604+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562bee8a6040 con 0x562bee9baa60 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.605+0000 7f938e7fc640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee9baa60 0x562bee9bce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.605+0000 7f9372ff5640 10 client.20421 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:13.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.605+0000 7f938e7fc640 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee9baa60 0x562bee9bce80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=664febed78d66c58 server_cookie=f5c6bb973d748e89 in_seq=0 out_seq=0 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f9372ff5640 1 -- 172.21.15.122:0/925879068 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562bee8a6040 con 0x562bee9baa60 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f9372ff5640 10 client.20421 handle_command_reply: tid=0 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 client.20421 shutdown 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 msgr2=0x562bee9448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f9380099500 tx=0x7f93800a5520 comp rx=0 tx=0).stop 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 shutdown_connections 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562bee943ca0 0x562bee94ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562bee9baa60 0x562bee9bce80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562bee93b0c0 0x562bee944e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f9372ff5640 0 client.20421 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 --2- 172.21.15.122:0/925879068 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562bee93a7a0 0x562bee9448f0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 >> 172.21.15.122:0/925879068 conn(0x562bee934ed0 msgr2=0x562bee93f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 shutdown_connections 2024-04-02T13:52:13.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 1 -- 172.21.15.122:0/925879068 wait complete. 2024-04-02T13:52:13.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.606+0000 7f939837d740 20 client.20421 trim_cache size 0 max 16384 2024-04-02T13:52:13.615 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 msgr2=0x7f93780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 0x7f93780a6260 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f93880094d0 tx=0x7f9388004ab0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 msgr2=0x7f93901983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f938000e9f0 tx=0x7f938000edf0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 shutdown_connections 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93780a3da0 0x7f93780a6260 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f939010c0d0 0x7f939019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9390103620 0x7f9390198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 --2- 172.21.15.122:0/707953678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9390102d00 0x7f93901983f0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 >> 172.21.15.122:0/707953678 conn(0x7f93900777b0 msgr2=0x7f93900fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 shutdown_connections 2024-04-02T13:52:13.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.607+0000 7f93956ba640 1 -- 172.21.15.122:0/707953678 wait complete. 2024-04-02T13:52:13.621 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 -- 172.21.15.122:0/2836057228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50102450 msgr2=0x7feb5010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 --2- 172.21.15.122:0/2836057228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50102450 0x7feb5010e960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feb48009be0 tx=0x7feb4802bef0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 -- 172.21.15.122:0/2836057228 shutdown_connections 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 --2- 172.21.15.122:0/2836057228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50102450 0x7feb5010e960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 --2- 172.21.15.122:0/2836057228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 --2- 172.21.15.122:0/2836057228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50107aa0 0x7feb50107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 -- 172.21.15.122:0/2836057228 >> 172.21.15.122:0/2836057228 conn(0x7feb50078f00 msgr2=0x7feb50079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 -- 172.21.15.122:0/2836057228 shutdown_connections 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 -- 172.21.15.122:0/2836057228 wait complete. 2024-04-02T13:52:13.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.746+0000 7feb56ed6640 1 Processor -- start 2024-04-02T13:52:13.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 -- start start 2024-04-02T13:52:13.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50102450 0x7feb50198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50107aa0 0x7feb5019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb50111eb0 con 0x7feb50101a90 2024-04-02T13:52:13.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb50111d30 con 0x7feb50107aa0 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb56ed6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb50112030 con 0x7feb50102450 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47190/0 (socket says 172.21.15.122:47190) 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 -- 172.21.15.122:0/3811791798 learned_addr learned my addr 172.21.15.122:0/3811791798 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb55473640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50107aa0 0x7feb5019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb4ffff640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50102450 0x7feb50198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 -- 172.21.15.122:0/3811791798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50102450 msgr2=0x7feb50198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50102450 0x7feb50198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 -- 172.21.15.122:0/3811791798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50107aa0 msgr2=0x7feb5019cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50107aa0 0x7feb5019cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 -- 172.21.15.122:0/3811791798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5019d2f0 con 0x7feb50101a90 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.747+0000 7feb54c72640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7feb4400e9f0 tx=0x7feb4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.748+0000 7feb4dffb640 1 -- 172.21.15.122:0/3811791798 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4400cd80 con 0x7feb50101a90 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.748+0000 7feb4dffb640 1 -- 172.21.15.122:0/3811791798 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb44005500 con 0x7feb50101a90 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.748+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5019d5e0 con 0x7feb50101a90 2024-04-02T13:52:13.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.748+0000 7feb4dffb640 1 -- 172.21.15.122:0/3811791798 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44010690 con 0x7feb50101a90 2024-04-02T13:52:13.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.748+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb501a4e70 con 0x7feb50101a90 2024-04-02T13:52:13.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.750+0000 7feb4dffb640 1 -- 172.21.15.122:0/3811791798 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7feb440046c0 con 0x7feb50101a90 2024-04-02T13:52:13.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 Processor -- start 2024-04-02T13:52:13.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 -- start start 2024-04-02T13:52:13.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be447a0 0x55e43be44b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43bda87b0 con 0x55e43be450c0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e43bd47570 con 0x55e43be4dca0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb59b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e43bda4760 con 0x55e43be447a0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4dffb640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 0x7feb400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb55473640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be447a0 0x55e43be44b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47330/0 (socket says 172.21.15.122:47330) 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 -- 172.21.15.122:0/493404943 learned_addr learned my addr 172.21.15.122:0/493404943 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb55473640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 0x7feb400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4dffb640 1 -- 172.21.15.122:0/3811791798 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7feb440d0f70 con 0x7feb50101a90 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 -- 172.21.15.122:0/493404943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be447a0 msgr2=0x55e43be44b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be447a0 0x55e43be44b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 -- 172.21.15.122:0/493404943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 msgr2=0x55e43be4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.751+0000 7feb4ffff640 1 -- 172.21.15.122:0/493404943 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43babc3f0 con 0x55e43be4dca0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.752+0000 7feb55473640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 0x7feb400a61e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb48009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.752+0000 7feb4ffff640 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009c80 tx=0x7feb3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b5bb2f45565fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.752+0000 7feb3affd640 1 -- 172.21.15.122:0/493404943 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c004220 con 0x55e43be4dca0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.752+0000 7feb3affd640 1 -- 172.21.15.122:0/493404943 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c009e30 con 0x55e43be4dca0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.752+0000 7feb3affd640 1 -- 172.21.15.122:0/493404943 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c019890 con 0x55e43be4dca0 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- 172.21.15.122:0/493404943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 msgr2=0x55e43be50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009c80 tx=0x7feb3c011810 comp rx=0 tx=0).stop 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- 172.21.15.122:0/493404943 shutdown_connections 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be4dca0 0x55e43be50090 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- 172.21.15.122:0/493404943 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be447a0 0x55e43be44b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- 172.21.15.122:0/493404943 >> 172.21.15.122:0/493404943 conn(0x55e43be3eed0 msgr2=0x55e43be3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- 172.21.15.122:0/493404943 shutdown_connections 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- 172.21.15.122:0/493404943 wait complete. 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 -- start start 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 Processor -- start 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.753+0000 7feb59b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43be50a90 con 0x55e43be447a0 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e43bd63e80 con 0x55e43be450c0 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e43be50c10 con 0x55e43be4dca0 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 10 client.0 _ll_register_callbacks cb 0x55e43bd9faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47216/0 (socket says 172.21.15.122:47216) 2024-04-02T13:52:13.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 -- 172.21.15.122:0/2882104198 learned_addr learned my addr 172.21.15.122:0/2882104198 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb55473640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb4ffff640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 -- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 msgr2=0x55e43be4b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 -- 172.21.15.122:0/2882104198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 msgr2=0x55e43be4aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 -- 172.21.15.122:0/2882104198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43bb867d0 con 0x55e43be447a0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb54c72640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7feb4409c820 tx=0x7feb4409c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54a1f9ed59e481c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440a0ed0 con 0x55e43be447a0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7feb440d30e0 con 0x55e43be447a0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb3c009e30 con 0x55e43be447a0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7feb440a5790 con 0x55e43be447a0 2024-04-02T13:52:13.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 10 client.20433 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.754+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55e43be5cb20 con 0x55e43be447a0 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c009e30 con 0x55e43be447a0 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7feb440bbce0 con 0x55e43be447a0 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 10 client.20433 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 10 client.20433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 20 client.20433 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43be5dfe0 0x55e43be603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 4 client.20433 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e43bdb5cf0 con 0x55e43be5dfe0 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.755+0000 7feb4ffff640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43be5dfe0 0x55e43be603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.756+0000 7feb2ffff640 10 client.20433 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:13.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.756+0000 7feb4ffff640 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43be5dfe0 0x55e43be603d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65b7fb8e8cea5346 server_cookie=3abb07572ed2a556 in_seq=0 out_seq=0 2024-04-02T13:52:13.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.757+0000 7feb2ffff640 1 -- 172.21.15.122:0/2882104198 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e43bdb5cf0 con 0x55e43be5dfe0 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.757+0000 7feb2ffff640 10 client.20433 handle_command_reply: tid=0 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 client.20433 shutdown 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 msgr2=0x55e43be4d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7feb4409c820 tx=0x7feb4409c480 comp rx=0 tx=0).stop 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 shutdown_connections 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43be5dfe0 0x55e43be603d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 --2- 172.21.15.122:0/2882104198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be4d710 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 >> 172.21.15.122:0/2882104198 conn(0x55e43be3eed0 msgr2=0x55e43be40c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb2ffff640 0 client.20433 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 shutdown_connections 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 1 -- 172.21.15.122:0/2882104198 wait complete. 2024-04-02T13:52:13.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.758+0000 7feb59b98740 20 client.20433 trim_cache size 0 max 16384 2024-04-02T13:52:13.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 Processor -- start 2024-04-02T13:52:13.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 -- start start 2024-04-02T13:52:13.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 0x55e43be59950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb501ace50 con 0x55e43be450c0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb501aea50 con 0x55e43be447a0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb59b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb501ad520 con 0x55e43be4dca0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb4ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47224/0 (socket says 172.21.15.122:47224) 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb4ffff640 1 -- 172.21.15.122:0/1231355569 learned_addr learned my addr 172.21.15.122:0/1231355569 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb55473640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.772+0000 7feb54c72640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 0x55e43be59950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 -- 172.21.15.122:0/1231355569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 msgr2=0x55e43be4d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 -- 172.21.15.122:0/1231355569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 msgr2=0x55e43be59950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 0x55e43be59950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 -- 172.21.15.122:0/1231355569 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43be44ef0 con 0x55e43be450c0 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb54c72640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 0x55e43be59950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb4ffff640 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009530 tx=0x7feb3c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba1b4c87c42a284 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb3a7fc640 1 -- 172.21.15.122:0/1231355569 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c004280 con 0x55e43be450c0 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb3a7fc640 1 -- 172.21.15.122:0/1231355569 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c011d10 con 0x55e43be450c0 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.773+0000 7feb3a7fc640 1 -- 172.21.15.122:0/1231355569 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c003a80 con 0x55e43be450c0 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- 172.21.15.122:0/1231355569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 msgr2=0x55e43be49a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009530 tx=0x7feb3c005fd0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- 172.21.15.122:0/1231355569 shutdown_connections 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be450c0 0x55e43be49a80 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- 172.21.15.122:0/1231355569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be447a0 0x55e43be59950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- 172.21.15.122:0/1231355569 >> 172.21.15.122:0/1231355569 conn(0x55e43be3eed0 msgr2=0x55e43be49370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- 172.21.15.122:0/1231355569 shutdown_connections 2024-04-02T13:52:13.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- 172.21.15.122:0/1231355569 wait complete. 2024-04-02T13:52:13.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- start start 2024-04-02T13:52:13.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 Processor -- start 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb501ae380 con 0x55e43be447a0 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feb501adcb0 con 0x55e43be450c0 2024-04-02T13:52:13.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb501aae50 con 0x55e43be4dca0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.774+0000 7feb59b98740 10 client.0 _ll_register_callbacks cb 0x7feb44009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47234/0 (socket says 172.21.15.122:47234) 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 -- 172.21.15.122:0/4117287805 learned_addr learned my addr 172.21.15.122:0/4117287805 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb4ffff640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb55473640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 -- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 msgr2=0x55e43be4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 -- 172.21.15.122:0/4117287805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 msgr2=0x55e43be4e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 -- 172.21.15.122:0/4117287805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43be53e50 con 0x55e43be447a0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb4ffff640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb54c72640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feb440994f0 tx=0x7feb440a5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13146f92f900ceb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440a0ed0 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7feb44098870 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb3c011d10 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7feb4409c8f0 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb59b98740 10 client.20439 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.775+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55e43bec45f0 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3c011d10 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7feb440bbce0 con 0x55e43be447a0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 10 client.20439 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 10 client.20439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 20 client.20439 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43bec4d10 0x55e43bec7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 4 client.20439 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e43bdb0040 con 0x55e43bec4d10 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.776+0000 7feb4ffff640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43bec4d10 0x55e43bec7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.777+0000 7feb2ffff640 10 client.20439 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.777+0000 7feb4ffff640 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43bec4d10 0x55e43bec7130 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d61e1b02cc89c3fd server_cookie=8e1cbc35c8a066ed in_seq=0 out_seq=0 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.778+0000 7feb2ffff640 1 -- 172.21.15.122:0/4117287805 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e43bdb0040 con 0x55e43bec4d10 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.778+0000 7feb2ffff640 10 client.20439 handle_command_reply: tid=0 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.778+0000 7feb59b98740 1 client.20439 shutdown 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 msgr2=0x55e43be56330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7feb440994f0 tx=0x7feb440a5530 comp rx=0 tx=0).stop 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 shutdown_connections 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e43bec4d10 0x55e43bec7130 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e43be4dca0 0x55e43be4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb2ffff640 0 client.20439 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e43be450c0 0x55e43be4e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 --2- 172.21.15.122:0/4117287805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43be447a0 0x55e43be56330 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 >> 172.21.15.122:0/4117287805 conn(0x55e43be3eed0 msgr2=0x55e43be40b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 shutdown_connections 2024-04-02T13:52:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 1 -- 172.21.15.122:0/4117287805 wait complete. 2024-04-02T13:52:13.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb59b98740 20 client.20439 trim_cache size 0 max 16384 2024-04-02T13:52:13.786 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 msgr2=0x7feb400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 0x7feb400a61e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb48009500 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 msgr2=0x7feb50198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.779+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7feb4400e9f0 tx=0x7feb4400edf0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 shutdown_connections 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7feb400a3d20 0x7feb400a61e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feb50107aa0 0x7feb5019cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb50102450 0x7feb50198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 --2- 172.21.15.122:0/3811791798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb50101a90 0x7feb50198350 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 >> 172.21.15.122:0/3811791798 conn(0x7feb50078f00 msgr2=0x7feb501043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 shutdown_connections 2024-04-02T13:52:13.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.780+0000 7feb56ed6640 1 -- 172.21.15.122:0/3811791798 wait complete. 2024-04-02T13:52:13.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:13 smithi122 ceph-mon[32114]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 772 op/s 2024-04-02T13:52:13.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:13 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:13.792 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:13 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1232806414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:13.793 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 -- 172.21.15.122:0/587408195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 msgr2=0x7fd994069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 --2- 172.21.15.122:0/587408195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd994069160 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fd988009930 tx=0x7fd98802bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 -- 172.21.15.122:0/587408195 shutdown_connections 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 --2- 172.21.15.122:0/587408195 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 0x7fd99410e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 --2- 172.21.15.122:0/587408195 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd994069160 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 --2- 172.21.15.122:0/587408195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 0x7fd994107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 -- 172.21.15.122:0/587408195 >> 172.21.15.122:0/587408195 conn(0x7fd994077b00 msgr2=0x7fd994077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 -- 172.21.15.122:0/587408195 shutdown_connections 2024-04-02T13:52:13.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.903+0000 7fd998c0f640 1 -- 172.21.15.122:0/587408195 wait complete. 2024-04-02T13:52:13.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 Processor -- start 2024-04-02T13:52:13.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 -- start start 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 0x7fd99419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 0x7fd99419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd994111e90 con 0x7fd994068ce0 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd994111d10 con 0x7fd9940696a0 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd998c0f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd994112010 con 0x7fd994107a80 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47240/0 (socket says 172.21.15.122:47240) 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 -- 172.21.15.122:0/3575455694 learned_addr learned my addr 172.21.15.122:0/3575455694 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd991d9b640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 0x7fd99419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd992d9d640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 0x7fd99419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 -- 172.21.15.122:0/3575455694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 msgr2=0x7fd99419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 0x7fd99419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 -- 172.21.15.122:0/3575455694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 msgr2=0x7fd99419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 0x7fd99419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 -- 172.21.15.122:0/3575455694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99419f5a0 con 0x7fd994068ce0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.904+0000 7fd99259c640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd98400e9f0 tx=0x7fd98400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.905+0000 7fd9837fe640 1 -- 172.21.15.122:0/3575455694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98400cd80 con 0x7fd994068ce0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.905+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99419f890 con 0x7fd994068ce0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.905+0000 7fd9837fe640 1 -- 172.21.15.122:0/3575455694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd984010910 con 0x7fd994068ce0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.905+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a7120 con 0x7fd994068ce0 2024-04-02T13:52:13.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.906+0000 7fd9837fe640 1 -- 172.21.15.122:0/3575455694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984018980 con 0x7fd994068ce0 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.908+0000 7fd9837fe640 1 -- 172.21.15.122:0/3575455694 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd984018b20 con 0x7fd994068ce0 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 Processor -- start 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 -- start start 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82934990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b829488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b828987b0 con 0x555b829345b0 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555b82837570 con 0x555b82934f60 2024-04-02T13:52:13.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b82894760 con 0x555b829464b0 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.909+0000 7fd99259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47392/0 (socket says 172.21.15.122:47392) 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd9837fe640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 0x7fd97c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 -- 172.21.15.122:0/982418703 learned_addr learned my addr 172.21.15.122:0/982418703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd992d9d640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82934990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd991d9b640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b829488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 -- 172.21.15.122:0/982418703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 msgr2=0x555b829488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b829488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 -- 172.21.15.122:0/982418703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 msgr2=0x555b82934990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd9837fe640 1 -- 172.21.15.122:0/3575455694 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd9840d0ec0 con 0x7fd994068ce0 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82934990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 -- 172.21.15.122:0/982418703 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b825ac3f0 con 0x555b82934f60 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd992d9d640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82934990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd992d9d640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 0x7fd97c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd99259c640 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9840994f0 tx=0x7fd984018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3edfea54c47a3c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.910+0000 7fd980ff9640 1 -- 172.21.15.122:0/982418703 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98409cb20 con 0x555b82934f60 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.911+0000 7fd980ff9640 1 -- 172.21.15.122:0/982418703 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9840a0e20 con 0x555b82934f60 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.911+0000 7fd980ff9640 1 -- 172.21.15.122:0/982418703 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984060740 con 0x555b82934f60 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- 172.21.15.122:0/982418703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 msgr2=0x555b82945f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd992d9d640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 0x7fd97c0a6260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd9840994f0 tx=0x7fd984018420 comp rx=0 tx=0).stop 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- 172.21.15.122:0/982418703 shutdown_connections 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b829488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b82934f60 0x555b82945f70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/982418703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82934990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- 172.21.15.122:0/982418703 >> 172.21.15.122:0/982418703 conn(0x555b8292eed0 msgr2=0x555b8292f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- 172.21.15.122:0/982418703 shutdown_connections 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- 172.21.15.122:0/982418703 wait complete. 2024-04-02T13:52:13.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 -- start start 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.912+0000 7fd99b7f1740 1 Processor -- start 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b829432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b829439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555b82949210 con 0x555b829345b0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555b82853e80 con 0x555b829464b0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555b82949390 con 0x555b82934f60 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 10 client.0 _ll_register_callbacks cb 0x555b8288faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47246/0 (socket says 172.21.15.122:47246) 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 -- 172.21.15.122:0/1740092593 learned_addr learned my addr 172.21.15.122:0/1740092593 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd991d9b640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b829432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd992d9d640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b829439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 -- 172.21.15.122:0/1740092593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 msgr2=0x555b829432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b829432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 -- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 msgr2=0x555b829439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b829439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 -- 172.21.15.122:0/1740092593 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b826767d0 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99259c640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd98409cab0 tx=0x7fd984099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc62ecab93bda56 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9840a56e0 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd984060860 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.913+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9840a0e20 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fd98409cb20 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 10 client.20451 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x555b829552e0 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9840a0e20 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fd98409b9e0 con 0x555b829345b0 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 10 client.20451 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 10 client.20451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:13.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 20 client.20451 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b82956630 0x555b82958a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 4 client.20451 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:13.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b828a5cf0 con 0x555b82956630 2024-04-02T13:52:13.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.914+0000 7fd991d9b640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b82956630 0x555b82958a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.915+0000 7fd991d9b640 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b82956630 0x555b82958a20 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17a6d50d3567d98 server_cookie=9774d35f8ff15f01 in_seq=0 out_seq=0 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.915+0000 7fd979ffb640 10 client.20451 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd979ffb640 1 -- 172.21.15.122:0/1740092593 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b828a5cf0 con 0x555b82956630 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd979ffb640 10 client.20451 handle_command_reply: tid=0 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 client.20451 shutdown 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 msgr2=0x555b82945f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd98409cab0 tx=0x7fd984099430 comp rx=0 tx=0).stop 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 shutdown_connections 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b82956630 0x555b82958a20 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b829439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b829432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd979ffb640 0 client.20451 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1740092593 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b82945f40 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.917+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 >> 172.21.15.122:0/1740092593 conn(0x555b8292eed0 msgr2=0x555b82930c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.918+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 shutdown_connections 2024-04-02T13:52:13.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.918+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1740092593 wait complete. 2024-04-02T13:52:13.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.918+0000 7fd99b7f1740 20 client.20451 trim_cache size 0 max 16384 2024-04-02T13:52:13.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.941+0000 7fd99b7f1740 1 Processor -- start 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.941+0000 7fd99b7f1740 1 -- start start 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.941+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 0x555b82943270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.941+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aceb0 con 0x555b82934f60 2024-04-02T13:52:13.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aeab0 con 0x555b829345b0 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ad580 con 0x555b829464b0 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd991d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 0x555b82943270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37512/0 (socket says 172.21.15.122:37512) 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd99259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 0x555b82943270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47422/0 (socket says 172.21.15.122:47422) 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 -- 172.21.15.122:0/1177795301 learned_addr learned my addr 172.21.15.122:0/1177795301 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd991d9b640 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47250/0 (socket says 172.21.15.122:47250) 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 -- 172.21.15.122:0/1177795301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 msgr2=0x555b82943270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 0x555b82943270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 -- 172.21.15.122:0/1177795301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 msgr2=0x555b82945a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd992d9d640 1 -- 172.21.15.122:0/1177795301 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b82943d30 con 0x555b829464b0 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.942+0000 7fd991d9b640 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.943+0000 7fd992d9d640 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00f9d0 tx=0x7fd98c013030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28ea6e57ca3b590f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.943+0000 7fd97bfff640 1 -- 172.21.15.122:0/1177795301 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c02e070 con 0x555b829464b0 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.943+0000 7fd97bfff640 1 -- 172.21.15.122:0/1177795301 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd98c01ee20 con 0x555b829464b0 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.943+0000 7fd97bfff640 1 -- 172.21.15.122:0/1177795301 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c029e20 con 0x555b829464b0 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1177795301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 msgr2=0x555b82945f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00f9d0 tx=0x7fd98c013030 comp rx=0 tx=0).stop 2024-04-02T13:52:13.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1177795301 shutdown_connections 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b829464b0 0x555b82945f40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b82934f60 0x555b82945a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/1177795301 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829345b0 0x555b82943270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1177795301 >> 172.21.15.122:0/1177795301 conn(0x555b8292eed0 msgr2=0x555b82942b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1177795301 shutdown_connections 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- 172.21.15.122:0/1177795301 wait complete. 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 -- start start 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.944+0000 7fd99b7f1740 1 Processor -- start 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b82959370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b82959ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941ae3e0 con 0x555b829345b0 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941add10 con 0x555b829464b0 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9941aaf30 con 0x555b82934f60 2024-04-02T13:52:13.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99b7f1740 10 client.0 _ll_register_callbacks cb 0x7fd98c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47260/0 (socket says 172.21.15.122:47260) 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 -- 172.21.15.122:0/2392027339 learned_addr learned my addr 172.21.15.122:0/2392027339 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd992d9d640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b82959ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 -- 172.21.15.122:0/2392027339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 msgr2=0x555b82959370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b82959370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 -- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 msgr2=0x555b82959ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b82959ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 -- 172.21.15.122:0/2392027339 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b829bb810 con 0x555b829345b0 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd99259c640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd984098f80 tx=0x7fd984099190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4c41f35fcb69fc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:13.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.945+0000 7fd979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:13.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984098dc0 con 0x555b829345b0 2024-04-02T13:52:13.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fd98409bf70 con 0x555b829345b0 2024-04-02T13:52:13.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c01ee20 con 0x555b829345b0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fd9840a56e0 con 0x555b829345b0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd99b7f1740 10 client.20457 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.946+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x555b829bc9d0 con 0x555b829345b0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9840a0e20 con 0x555b829345b0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fd984060070 con 0x555b829345b0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 10 client.20457 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 10 client.20457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 20 client.20457 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b829bcea0 0x555b829bf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 4 client.20457 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd98c01ee20 con 0x555b829bcea0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.947+0000 7fd991d9b640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b829bcea0 0x555b829bf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.948+0000 7fd991d9b640 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b829bcea0 0x555b829bf290 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6286256c1feec286 server_cookie=e43a701be045b37 in_seq=0 out_seq=0 2024-04-02T13:52:13.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.948+0000 7fd979ffb640 10 client.20457 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd979ffb640 1 -- 172.21.15.122:0/2392027339 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd98c01ee20 con 0x555b829bcea0 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd979ffb640 10 client.20457 handle_command_reply: tid=0 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 client.20457 shutdown 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 msgr2=0x555b8295bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd984098f80 tx=0x7fd984099190 comp rx=0 tx=0).stop 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 shutdown_connections 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555b829bcea0 0x555b829bf290 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd979ffb640 0 client.20457 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555b829464b0 0x555b82959ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555b82934f60 0x555b82959370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 --2- 172.21.15.122:0/2392027339 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555b829345b0 0x555b8295bfe0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 >> 172.21.15.122:0/2392027339 conn(0x555b8292eed0 msgr2=0x555b82951620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 shutdown_connections 2024-04-02T13:52:13.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 1 -- 172.21.15.122:0/2392027339 wait complete. 2024-04-02T13:52:13.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.949+0000 7fd99b7f1740 20 client.20457 trim_cache size 0 max 16384 2024-04-02T13:52:13.955 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 msgr2=0x7fd97c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 0x7fd97c0a6260 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c004a40 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 msgr2=0x7fd99419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fd98400e9f0 tx=0x7fd98400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 shutdown_connections 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd97c0a3da0 0x7fd97c0a6260 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd994107a80 0x7fd99419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd9940696a0 0x7fd99419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 --2- 172.21.15.122:0/3575455694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd994068ce0 0x7fd99419a5b0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 >> 172.21.15.122:0/3575455694 conn(0x7fd994077b00 msgr2=0x7fd994101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 shutdown_connections 2024-04-02T13:52:13.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:13.950+0000 7fd998c0f640 1 -- 172.21.15.122:0/3575455694 wait complete. 2024-04-02T13:52:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:13 smithi187 ceph-mon[36064]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 772 op/s 2024-04-02T13:52:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:13 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:13.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:13 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1232806414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:13 smithi188 ceph-mon[36117]: pgmap v536: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 772 op/s 2024-04-02T13:52:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:13 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:13 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1232806414' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:14.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 119157 30.70 MB/sec execute 94 sec latency 3976.765 ms 2024-04-02T13:52:14.966 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:52:14.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:52:14.966 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:52:15.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 -- 172.21.15.122:0/4024811368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec106870 msgr2=0x7fabec106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/4024811368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec106870 0x7fabec106c50 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009930 tx=0x7fabdc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 -- 172.21.15.122:0/4024811368 shutdown_connections 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/4024811368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec101220 0x7fabec10d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/4024811368 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec100860 0x7fabec100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/4024811368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec106870 0x7fabec106c50 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 -- 172.21.15.122:0/4024811368 >> 172.21.15.122:0/4024811368 conn(0x7fabec0fc350 msgr2=0x7fabec0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 -- 172.21.15.122:0/4024811368 shutdown_connections 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.114+0000 7fabf0cf7640 1 -- 172.21.15.122:0/4024811368 wait complete. 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 Processor -- start 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 -- start start 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec100860 0x7fabec198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec106870 0x7fabec19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabec110c80 con 0x7fabec101220 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabec110b00 con 0x7fabec106870 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabf0cf7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabec110e00 con 0x7fabec100860 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47270/0 (socket says 172.21.15.122:47270) 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 -- 172.21.15.122:0/959989338 learned_addr learned my addr 172.21.15.122:0/959989338 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec100860 0x7fabec198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabebfff640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec106870 0x7fabec19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 -- 172.21.15.122:0/959989338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec100860 msgr2=0x7fabec198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec100860 0x7fabec198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 -- 172.21.15.122:0/959989338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec106870 msgr2=0x7fabec19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec106870 0x7fabec19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.115+0000 7fabeaffd640 1 -- 172.21.15.122:0/959989338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabec19d250 con 0x7fabec101220 2024-04-02T13:52:15.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabeaffd640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e9f0 tx=0x7fabe000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabe8ff9640 1 -- 172.21.15.122:0/959989338 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe000cd80 con 0x7fabec101220 2024-04-02T13:52:15.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabe8ff9640 1 -- 172.21.15.122:0/959989338 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe0010910 con 0x7fabec101220 2024-04-02T13:52:15.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabec19d540 con 0x7fabec101220 2024-04-02T13:52:15.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabe8ff9640 1 -- 172.21.15.122:0/959989338 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0018990 con 0x7fabec101220 2024-04-02T13:52:15.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.116+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabec1a4dd0 con 0x7fabec101220 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.118+0000 7fabe8ff9640 1 -- 172.21.15.122:0/959989338 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fabe0010ac0 con 0x7fabec101220 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 Processor -- start 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 -- start start 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabe8ff9640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 0x7fabd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 0x5639a96b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639a960e770 con 0x5639a96b3990 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5639a95b6960 con 0x5639a96aa490 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabf4c21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639a960a8c0 con 0x5639a96aadb0 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabebfff640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 0x7fabd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabe8ff9640 1 -- 172.21.15.122:0/959989338 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fabe00d0f10 con 0x7fabec101220 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 0x5639a96b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabebfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 0x5639a96b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37528/0 (socket says 172.21.15.122:37528) 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeaffd640 1 -- 172.21.15.122:0/1453809106 learned_addr learned my addr 172.21.15.122:0/1453809106 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 -- 172.21.15.122:0/1453809106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 msgr2=0x5639a96b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 0x5639a96b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 -- 172.21.15.122:0/1453809106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 msgr2=0x5639a96b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.119+0000 7fabeb7fe640 1 -- 172.21.15.122:0/1453809106 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a93ea7e0 con 0x5639a96aa490 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.120+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96aa870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009230 tx=0x7fabdc009610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17a0cefc1ea5735 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.120+0000 7fabebfff640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 0x7fabd40a61e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009fd0 tx=0x7fabe4009de0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.120+0000 7fabd1ffb640 1 -- 172.21.15.122:0/1453809106 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc042070 con 0x5639a96aa490 2024-04-02T13:52:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.120+0000 7fabd1ffb640 1 -- 172.21.15.122:0/1453809106 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc02c690 con 0x5639a96aa490 2024-04-02T13:52:15.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.120+0000 7fabd1ffb640 1 -- 172.21.15.122:0/1453809106 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc004b90 con 0x5639a96aa490 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- 172.21.15.122:0/1453809106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 msgr2=0x5639a96aa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96aa870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009230 tx=0x7fabdc009610 comp rx=0 tx=0).stop 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- 172.21.15.122:0/1453809106 shutdown_connections 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aadb0 0x5639a96b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 --2- 172.21.15.122:0/1453809106 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96aa870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- 172.21.15.122:0/1453809106 >> 172.21.15.122:0/1453809106 conn(0x5639a96a4bc0 msgr2=0x5639a96a4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- 172.21.15.122:0/1453809106 shutdown_connections 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- 172.21.15.122:0/1453809106 wait complete. 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 -- start start 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.121+0000 7fabf4c21740 1 Processor -- start 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5639a96b6720 con 0x5639a96b3990 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5639a95c9d20 con 0x5639a96aadb0 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639a96b68a0 con 0x5639a96aa490 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47448/0 (socket says 172.21.15.122:47448) 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeaffd640 1 -- 172.21.15.122:0/3039916182 learned_addr learned my addr 172.21.15.122:0/3039916182 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabf4c21740 10 client.0 _ll_register_callbacks cb 0x5639a9596a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabebfff640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 -- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 msgr2=0x5639a96b0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 -- 172.21.15.122:0/3039916182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 msgr2=0x5639a96b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeb7fe640 1 -- 172.21.15.122:0/3039916182 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a944af70 con 0x5639a96aa490 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabebfff640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.122+0000 7fabeaffd640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96b09d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabdc036040 tx=0x7fabdc009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7303d014b29661a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc042040 con 0x5639a96aa490 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5639a961bcb0 con 0x5639a96aa490 2024-04-02T13:52:15.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fabdc004240 con 0x5639a96aa490 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fabdc057050 con 0x5639a96aa490 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 10 client.29173 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5639a92e2170 con 0x5639a96aa490 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5639a961bcb0 con 0x5639a96aa490 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fabdc0039f0 con 0x5639a96aa490 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 10 client.29173 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 10 client.29173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 20 client.29173 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a96c3b40 0x5639a96c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 4 client.29173 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.123+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639a9616000 con 0x5639a96c3b40 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.124+0000 7fabeaffd640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a96c3b40 0x5639a96c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.124+0000 7fabeaffd640 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a96c3b40 0x5639a96c5f30 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0ed242cfe123eae server_cookie=61b8b81345989af9 in_seq=0 out_seq=0 2024-04-02T13:52:15.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.124+0000 7fabcaffd640 10 client.29173 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabcaffd640 1 -- 172.21.15.122:0/3039916182 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639a9616000 con 0x5639a96c3b40 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabcaffd640 10 client.29173 handle_command_reply: tid=0 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 client.29173 shutdown 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 msgr2=0x5639a96b09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96b09d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fabdc036040 tx=0x7fabdc009680 comp rx=0 tx=0).stop 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 shutdown_connections 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a96c3b40 0x5639a96c5f30 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96b3990 0x5639a96b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aadb0 0x5639a96b0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 --2- 172.21.15.122:0/3039916182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96b09d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabcaffd640 0 client.29173 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 >> 172.21.15.122:0/3039916182 conn(0x5639a96a4bc0 msgr2=0x5639a96a6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 shutdown_connections 2024-04-02T13:52:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 1 -- 172.21.15.122:0/3039916182 wait complete. 2024-04-02T13:52:15.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.125+0000 7fabf4c21740 20 client.29173 trim_cache size 0 max 16384 2024-04-02T13:52:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 Processor -- start 2024-04-02T13:52:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 -- start start 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96b3990 0x5639a96afb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1b1010 con 0x5639a96aadb0 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1b2c10 con 0x5639a96aa490 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabf4c21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1b16e0 con 0x5639a96b3990 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabeb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabeaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabeb7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47456/0 (socket says 172.21.15.122:47456) 2024-04-02T13:52:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.150+0000 7fabeb7fe640 1 -- 172.21.15.122:0/111852960 learned_addr learned my addr 172.21.15.122:0/111852960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabebfff640 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96b3990 0x5639a96afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 -- 172.21.15.122:0/111852960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96b3990 msgr2=0x5639a96afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96b3990 0x5639a96afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 -- 172.21.15.122:0/111852960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 msgr2=0x5639a96bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 -- 172.21.15.122:0/111852960 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639a96a01c0 con 0x5639a96aa490 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57e7263e31646a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabd17fa640 1 -- 172.21.15.122:0/111852960 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc042070 con 0x5639a96aa490 2024-04-02T13:52:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabd17fa640 1 -- 172.21.15.122:0/111852960 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc051080 con 0x5639a96aa490 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.151+0000 7fabd17fa640 1 -- 172.21.15.122:0/111852960 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc005d50 con 0x5639a96aa490 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.152+0000 7fabf4c21740 1 -- 172.21.15.122:0/111852960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 msgr2=0x5639a96b3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.152+0000 7fabf4c21740 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fabdc006010 tx=0x7fabdc02c5d0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 -- 172.21.15.122:0/111852960 shutdown_connections 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96b3990 0x5639a96afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 --2- 172.21.15.122:0/111852960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96aa490 0x5639a96b3100 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 -- 172.21.15.122:0/111852960 >> 172.21.15.122:0/111852960 conn(0x5639a96a4bc0 msgr2=0x5639a96a3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 -- 172.21.15.122:0/111852960 shutdown_connections 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 -- 172.21.15.122:0/111852960 wait complete. 2024-04-02T13:52:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 -- start start 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.153+0000 7fabf4c21740 1 Processor -- start 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabeb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabeb7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96bbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37564/0 (socket says 172.21.15.122:37564) 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1b2540 con 0x5639a96aadb0 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1b1e70 con 0x5639a96b3990 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fabec1af010 con 0x5639a96aa490 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabf4c21740 10 client.0 _ll_register_callbacks cb 0x7fabe001df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabebfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabebfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47470/0 (socket says 172.21.15.122:47470) 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabebfff640 1 -- 172.21.15.122:0/2413047579 learned_addr learned my addr 172.21.15.122:0/2413047579 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.154+0000 7fabeaffd640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 -- 172.21.15.122:0/2413047579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 msgr2=0x5639a96bbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96bbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 -- 172.21.15.122:0/2413047579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 msgr2=0x5639a96b72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96b72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 -- 172.21.15.122:0/2413047579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639a96b8070 con 0x5639a96b3990 2024-04-02T13:52:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabeaffd640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.155+0000 7fabebfff640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fabe4011ce0 tx=0x7fabe401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e6e93c601b1f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.156+0000 7fabcaffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.156+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4004240 con 0x5639a96b3990 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.156+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fabe40043e0 con 0x5639a96b3990 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.157+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabdc051080 con 0x5639a96b3990 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.157+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fabe40045e0 con 0x5639a96b3990 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.157+0000 7fabf4c21740 10 client.29429 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.157+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5639a97326f0 con 0x5639a96b3990 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc051080 con 0x5639a96b3990 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7fabe401f070 con 0x5639a96b3990 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 10 client.29429 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 10 client.29429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 20 client.29429 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a9732e00 0x5639a9735220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 4 client.29429 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:52:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.158+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5639a9593be0 con 0x5639a9732e00 2024-04-02T13:52:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.160+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a9732e00 0x5639a9735220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.161+0000 7fabeb7fe640 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a9732e00 0x5639a9735220 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb14d6a99e1554 server_cookie=a277c0523e0a418b in_seq=0 out_seq=0 2024-04-02T13:52:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.161+0000 7fabcaffd640 10 client.29429 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabcaffd640 1 -- 172.21.15.122:0/2413047579 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5639a9593be0 con 0x5639a9732e00 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabcaffd640 10 client.29429 handle_command_reply: tid=0 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 client.29429 shutdown 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 msgr2=0x5639a96b7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fabe4011ce0 tx=0x7fabe401b000 comp rx=0 tx=0).stop 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 shutdown_connections 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5639a9732e00 0x5639a9735220 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5639a96b3990 0x5639a96b7840 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5639a96aadb0 0x5639a96b72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 --2- 172.21.15.122:0/2413047579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639a96aa490 0x5639a96bbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 >> 172.21.15.122:0/2413047579 conn(0x5639a96a4bc0 msgr2=0x5639a96b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabcaffd640 0 client.29429 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.163+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 shutdown_connections 2024-04-02T13:52:15.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.164+0000 7fabf4c21740 1 -- 172.21.15.122:0/2413047579 wait complete. 2024-04-02T13:52:15.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.164+0000 7fabf4c21740 20 client.29429 trim_cache size 0 max 16384 2024-04-02T13:52:15.167 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:15.168 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:52:15.168 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "33cb3335-4631-4eba-ba88-871602b50dd5", 2024-04-02T13:52:15.168 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:52:15.168 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.166+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 msgr2=0x7fabd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.166+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 0x7fabd40a61e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009fd0 tx=0x7fabe4009de0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.166+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 msgr2=0x7fabec1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.166+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fabe000e9f0 tx=0x7fabe000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.166+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 shutdown_connections 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fabd40a3d20 0x7fabd40a61e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fabec106870 0x7fabec19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fabec101220 0x7fabec1987a0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 --2- 172.21.15.122:0/959989338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabec100860 0x7fabec198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 >> 172.21.15.122:0/959989338 conn(0x7fabec0fc350 msgr2=0x7fabec104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 shutdown_connections 2024-04-02T13:52:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.167+0000 7fabf0cf7640 1 -- 172.21.15.122:0/959989338 wait complete. 2024-04-02T13:52:15.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:52:15.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:52:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 -- 172.21.15.122:0/337804587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701009b0 msgr2=0x7f7170100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 --2- 172.21.15.122:0/337804587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701009b0 0x7f7170100e30 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f7164009930 tx=0x7f716402bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:15.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 -- 172.21.15.122:0/337804587 shutdown_connections 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 --2- 172.21.15.122:0/337804587 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7170101370 0x7f717010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 --2- 172.21.15.122:0/337804587 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701009b0 0x7f7170100e30 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 --2- 172.21.15.122:0/337804587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71701069c0 0x7f7170106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.313+0000 7f7174877640 1 -- 172.21.15.122:0/337804587 >> 172.21.15.122:0/337804587 conn(0x7f7170076d80 msgr2=0x7f7170077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- 172.21.15.122:0/337804587 shutdown_connections 2024-04-02T13:52:15.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- 172.21.15.122:0/337804587 wait complete. 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 Processor -- start 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- start start 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71701009b0 0x7f7170197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 0x7f7170197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7170110dd0 con 0x7f71701069c0 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7170110c50 con 0x7f71701009b0 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.314+0000 7f7174877640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7170110f50 con 0x7f7170101370 2024-04-02T13:52:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47324/0 (socket says 172.21.15.122:47324) 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716dd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 0x7f7170197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 -- 172.21.15.122:0/93393670 learned_addr learned my addr 172.21.15.122:0/93393670 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71701009b0 0x7f7170197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 -- 172.21.15.122:0/93393670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 msgr2=0x7f7170197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 0x7f7170197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 -- 172.21.15.122:0/93393670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71701009b0 msgr2=0x7f7170197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71701009b0 0x7f7170197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 -- 172.21.15.122:0/93393670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717019c320 con 0x7f71701069c0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716dd9b640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 0x7f7170197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f716ed9d640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f716800efd0 tx=0x7f716800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f715f7fe640 1 -- 172.21.15.122:0/93393670 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168003b70 con 0x7f71701069c0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717019c610 con 0x7f71701069c0 2024-04-02T13:52:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f715f7fe640 1 -- 172.21.15.122:0/93393670 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7168003d10 con 0x7f71701069c0 2024-04-02T13:52:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f715f7fe640 1 -- 172.21.15.122:0/93393670 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71680109f0 con 0x7f71701069c0 2024-04-02T13:52:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.315+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717006a150 con 0x7f71701069c0 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.318+0000 7f715f7fe640 1 -- 172.21.15.122:0/93393670 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7168004020 con 0x7f71701069c0 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.318+0000 7f717745c740 1 Processor -- start 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 -- start start 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4965248d0 0x55f496524cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f715f7fe640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 0x7f71580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 0x55f4965301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f496488830 con 0x55f4965251f0 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f496430820 con 0x55f49652ddd0 2024-04-02T13:52:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f717745c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4964847e0 con 0x55f4965248d0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716dd9b640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 0x7f71580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716ed9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 0x55f4965301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47330/0 (socket says 172.21.15.122:47330) 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 -- 172.21.15.122:0/1558220552 learned_addr learned my addr 172.21.15.122:0/1558220552 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f715f7fe640 1 -- 172.21.15.122:0/93393670 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f71680d1050 con 0x7f71701069c0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716dd9b640 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4965248d0 0x55f496524cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 -- 172.21.15.122:0/1558220552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4965248d0 msgr2=0x55f496524cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4965248d0 0x55f496524cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 -- 172.21.15.122:0/1558220552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 msgr2=0x55f4965301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 0x55f4965301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716e59c640 1 -- 172.21.15.122:0/1558220552 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4962667d0 con 0x55f4965251f0 2024-04-02T13:52:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716ed9d640 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 0x55f4965301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.319+0000 7f716dd9b640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 0x7f71580a61e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7164005ab0 tx=0x7f7164005e80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f716e59c640 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7160009c20 tx=0x7f7160005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8922a3caddd691e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f715cff9640 1 -- 172.21.15.122:0/1558220552 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160018400 con 0x55f4965251f0 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f715cff9640 1 -- 172.21.15.122:0/1558220552 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7160018a80 con 0x55f4965251f0 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f715cff9640 1 -- 172.21.15.122:0/1558220552 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160017b40 con 0x55f4965251f0 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 -- 172.21.15.122:0/1558220552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 msgr2=0x55f49652d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7160009c20 tx=0x7f7160005f90 comp rx=0 tx=0).stop 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 -- 172.21.15.122:0/1558220552 shutdown_connections 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f49652ddd0 0x55f4965301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49652d890 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 --2- 172.21.15.122:0/1558220552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4965248d0 0x55f496524cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.320+0000 7f717745c740 1 -- 172.21.15.122:0/1558220552 >> 172.21.15.122:0/1558220552 conn(0x55f49651f000 msgr2=0x55f49651f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- 172.21.15.122:0/1558220552 shutdown_connections 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- 172.21.15.122:0/1558220552 wait complete. 2024-04-02T13:52:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- start start 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 Processor -- start 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f496538150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496532020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f496530bc0 con 0x55f4965251f0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f496443d30 con 0x55f4965248d0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f496530d40 con 0x55f49652ddd0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f717745c740 10 client.0 _ll_register_callbacks cb 0x55f496443720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f716e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f716dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f496538150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.321+0000 7f716e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:47506/0 (socket says 172.21.15.122:47506) 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 -- 172.21.15.122:0/3674882681 learned_addr learned my addr 172.21.15.122:0/3674882681 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716ed9d640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496532020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 -- 172.21.15.122:0/3674882681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 msgr2=0x55f496532020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496532020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 -- 172.21.15.122:0/3674882681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 msgr2=0x55f496538150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f496538150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 -- 172.21.15.122:0/3674882681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f49619c3f0 con 0x55f4965248d0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f496538150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f716e59c640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7160017840 tx=0x7f7160005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6bbe294376b1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f7155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160026070 con 0x55f4965248d0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7160018a80 con 0x55f4965248d0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f7160018610 con 0x55f4965248d0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f7160032050 con 0x55f4965248d0 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f717745c740 10 client.29441 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.322+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55f49615c0f0 con 0x55f4965248d0 2024-04-02T13:52:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7160018a80 con 0x55f4965248d0 2024-04-02T13:52:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7f71600046c0 con 0x55f4965248d0 2024-04-02T13:52:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 10 client.29441 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 10 client.29441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 20 client.29441 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f496544400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 4 client.29441 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f496495d70 con 0x55f496542010 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f496544400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f7155ffb640 10 client.29441 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.323+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f496544400 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b1954404b71c6c8 server_cookie=64f8c3bd7c143d18 in_seq=0 out_seq=0 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.324+0000 7f7155ffb640 1 -- 172.21.15.122:0/3674882681 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f496495d70 con 0x55f496542010 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.324+0000 7f7155ffb640 10 client.29441 handle_command_reply: tid=0 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 client.29441 shutdown 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 msgr2=0x55f496537c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7160017840 tx=0x7f7160005da0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 shutdown_connections 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f496544400 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f7155ffb640 0 client.29441 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496532020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f496538150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 --2- 172.21.15.122:0/3674882681 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f496537c10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 >> 172.21.15.122:0/3674882681 conn(0x55f49651f000 msgr2=0x55f496520d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 shutdown_connections 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 1 -- 172.21.15.122:0/3674882681 wait complete. 2024-04-02T13:52:15.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.325+0000 7f717745c740 20 client.29441 trim_cache size 0 max 16384 2024-04-02T13:52:15.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 119477 30.57 MB/sec execute 95 sec latency 4976.793 ms 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 Processor -- start 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 -- start start 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965251f0 0x55f49652a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496549a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71701ace80 con 0x55f4965248d0 2024-04-02T13:52:15.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71701aea80 con 0x55f4965251f0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f717745c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71701ad550 con 0x55f49652ddd0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f716e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f716e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47348/0 (socket says 172.21.15.122:47348) 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f716e59c640 1 -- 172.21.15.122:0/1872332324 learned_addr learned my addr 172.21.15.122:0/1872332324 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.345+0000 7f716dd9b640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965251f0 0x55f49652a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716ed9d640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496549a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 -- 172.21.15.122:0/1872332324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 msgr2=0x55f496549a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496549a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 -- 172.21.15.122:0/1872332324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965251f0 msgr2=0x55f49652a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965251f0 0x55f49652a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 -- 172.21.15.122:0/1872332324 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f49651ee30 con 0x55f4965248d0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716ed9d640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496549a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f716e59c640 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f7160009c20 tx=0x7f7160009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325c0393a47005b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f7157fff640 1 -- 172.21.15.122:0/1872332324 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160026040 con 0x55f4965248d0 2024-04-02T13:52:15.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f7157fff640 1 -- 172.21.15.122:0/1872332324 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7160009dd0 con 0x55f4965248d0 2024-04-02T13:52:15.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.346+0000 7f7157fff640 1 -- 172.21.15.122:0/1872332324 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160017e60 con 0x55f4965248d0 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- 172.21.15.122:0/1872332324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 msgr2=0x55f496537ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f7160009c20 tx=0x7f7160009660 comp rx=0 tx=0).stop 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- 172.21.15.122:0/1872332324 shutdown_connections 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f496549a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965251f0 0x55f49652a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 --2- 172.21.15.122:0/1872332324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965248d0 0x55f496537ff0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- 172.21.15.122:0/1872332324 >> 172.21.15.122:0/1872332324 conn(0x55f49651f000 msgr2=0x55f496529c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- 172.21.15.122:0/1872332324 shutdown_connections 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- 172.21.15.122:0/1872332324 wait complete. 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.347+0000 7f717745c740 1 -- start start 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 Processor -- start 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f49653e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f49653bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71701ae3b0 con 0x55f4965251f0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71701adce0 con 0x55f4965248d0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71701aae80 con 0x55f49652ddd0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f717745c740 10 client.0 _ll_register_callbacks cb 0x7f7160009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f49653e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:47362/0 (socket says 172.21.15.122:47362) 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716dd9b640 1 -- 172.21.15.122:0/3003392055 learned_addr learned my addr 172.21.15.122:0/3003392055 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:15.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716ed9d640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f49653bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716e59c640 1 -- 172.21.15.122:0/3003392055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 msgr2=0x55f49653bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.348+0000 7f716e59c640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f49653bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f716e59c640 1 -- 172.21.15.122:0/3003392055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 msgr2=0x55f49653b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f716e59c640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f716e59c640 1 -- 172.21.15.122:0/3003392055 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4965242d0 con 0x55f4965248d0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f716e59c640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f49653e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f496513c70 tx=0x7f71600096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7389117c89cb69a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160026030 con 0x55f4965248d0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7160009dd0 con 0x55f4965248d0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f7160017de0 con 0x55f4965248d0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f7160048050 con 0x55f4965248d0 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f717745c740 10 client.29447 fetch_fsmap learned FSMap version 68 2024-04-02T13:52:15.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55f4965a7e70 con 0x55f4965248d0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7160009dd0 con 0x55f4965248d0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.349+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 68) ==== 8378+0+0 (secure 0 0 0) 0x7f716003e030 con 0x55f4965248d0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 10 client.29447 fetch_fsmap finished waiting for FSMap version 68 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 10 client.29447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 20 client.29447 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f4965aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 4 client.29447 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4964900c0 con 0x55f496542010 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f4965aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f716dd9b640 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f4965aa460 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc0139bd78ea7de8 server_cookie=b4ee648c4a5fbce5 in_seq=0 out_seq=0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f7155ffb640 10 client.29447 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f7155ffb640 1 -- 172.21.15.122:0/3003392055 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55f4964900c0 con 0x55f496542010 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f7155ffb640 10 client.29447 handle_command_reply: tid=0 2024-04-02T13:52:15.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 client.29447 shutdown 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 msgr2=0x55f49653e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f49653e320 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f496513c70 tx=0x7f71600096d0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.350+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 shutdown_connections 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f496542010 0x55f4965aa460 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f49652ddd0 0x55f49653bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7155ffb640 0 client.29447 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f4965251f0 0x55f49653b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 --2- 172.21.15.122:0/3003392055 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f4965248d0 0x55f49653e320 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 >> 172.21.15.122:0/3003392055 conn(0x55f49651f000 msgr2=0x55f49651d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 shutdown_connections 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 1 -- 172.21.15.122:0/3003392055 wait complete. 2024-04-02T13:52:15.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f717745c740 20 client.29447 trim_cache size 0 max 16384 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "scrub active (1 inodes in the stack)", 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "33cb3335-4631-4eba-ba88-871602b50dd5": { 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "33cb3335-4631-4eba-ba88-871602b50dd5", 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:52:15.360 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 msgr2=0x7f71580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 0x7f71580a61e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7164005ab0 tx=0x7f7164005e80 comp rx=0 tx=0).stop 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 msgr2=0x7f717019bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f716800efd0 tx=0x7f716800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 shutdown_connections 2024-04-02T13:52:15.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71580a3d20 0x7f71580a61e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71701069c0 0x7f717019bbf0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7170101370 0x7f7170197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 --2- 172.21.15.122:0/93393670 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71701009b0 0x7f7170197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 >> 172.21.15.122:0/93393670 conn(0x7f7170076d80 msgr2=0x7f71700fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 shutdown_connections 2024-04-02T13:52:15.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:15.351+0000 7f7174877640 1 -- 172.21.15.122:0/93393670 wait complete. 2024-04-02T13:52:15.365 INFO:tasks.cephfs.filesystem:scrub status for tag:33cb3335-4631-4eba-ba88-871602b50dd5 - {'path': '/', 'tag': '33cb3335-4631-4eba-ba88-871602b50dd5', 'options': 'recursive,force'} 2024-04-02T13:52:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:15 smithi122 ceph-mon[32114]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:15 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:15 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:15.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:15 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:15 smithi187 ceph-mon[36064]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:15 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:15 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:15.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:15 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:15 smithi188 ceph-mon[36117]: pgmap v537: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:15 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:15 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:15 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:16.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 120004 30.52 MB/sec execute 96 sec latency 5976.821 ms 2024-04-02T13:52:16.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:16.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:16 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:17.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:16 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:52:17.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:17.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:16 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:17.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 120444 30.47 MB/sec execute 97 sec latency 6976.848 ms 2024-04-02T13:52:17.971 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:17 smithi122 ceph-mon[32114]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 754 op/s 2024-04-02T13:52:17.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:17 smithi187 ceph-mon[36064]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 754 op/s 2024-04-02T13:52:18.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:17 smithi188 ceph-mon[36117]: pgmap v538: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 15 MiB/s wr, 754 op/s 2024-04-02T13:52:18.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 120880 30.40 MB/sec execute 98 sec latency 7976.878 ms 2024-04-02T13:52:19.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 121322 30.32 MB/sec execute 99 sec latency 8976.921 ms 2024-04-02T13:52:19.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:19 smithi187 ceph-mon[36064]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:20.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:19 smithi188 ceph-mon[36117]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:20.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:19 smithi122 ceph-mon[32114]: pgmap v539: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:20.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 121702 30.26 MB/sec execute 100 sec latency 9976.964 ms 2024-04-02T13:52:21.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 122153 30.17 MB/sec execute 101 sec latency 10976.999 ms 2024-04-02T13:52:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:21 smithi187 ceph-mon[36064]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:21.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:21 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:21.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:21 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:21 smithi188 ceph-mon[36117]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:21 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:22.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:21 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:22.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:21 smithi122 ceph-mon[32114]: pgmap v540: 129 pgs: 129 active+clean; 1.7 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 15 MiB/s wr, 704 op/s 2024-04-02T13:52:22.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:21 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:22.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:21 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:22.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 122684 30.14 MB/sec execute 102 sec latency 11977.032 ms 2024-04-02T13:52:22.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:22 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:22 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:22.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:22 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:22 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:22 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:23.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:22 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:23.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:22 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:23.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:22 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:23.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:22 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:23.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 123258 30.13 MB/sec execute 103 sec latency 12977.070 ms 2024-04-02T13:52:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:23 smithi187 ceph-mon[36064]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-04-02T13:52:23.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:23 smithi188 ceph-mon[36117]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-04-02T13:52:24.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:24.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:23 smithi122 ceph-mon[32114]: pgmap v541: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.6 MiB/s wr, 483 op/s 2024-04-02T13:52:24.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:24.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 123927 30.20 MB/sec execute 104 sec latency 13194.755 ms 2024-04-02T13:52:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:24 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:24.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:24 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:25.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:25.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:24 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:25.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 124462 30.18 MB/sec execute 105 sec latency 90.309 ms 2024-04-02T13:52:25.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:25 smithi187 ceph-mon[36064]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-04-02T13:52:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:25 smithi188 ceph-mon[36117]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-04-02T13:52:26.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:25 smithi122 ceph-mon[32114]: pgmap v542: 129 pgs: 129 active+clean; 1.7 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.3 MiB/s wr, 455 op/s 2024-04-02T13:52:26.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 125192 30.25 MB/sec execute 106 sec latency 58.440 ms 2024-04-02T13:52:27.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 125873 30.28 MB/sec execute 107 sec latency 76.481 ms 2024-04-02T13:52:27.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:27 smithi187 ceph-mon[36064]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 501 op/s 2024-04-02T13:52:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:27 smithi188 ceph-mon[36117]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 501 op/s 2024-04-02T13:52:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:27 smithi122 ceph-mon[32114]: pgmap v543: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 501 op/s 2024-04-02T13:52:28.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 126555 30.38 MB/sec execute 108 sec latency 73.759 ms 2024-04-02T13:52:29.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127270 30.47 MB/sec execute 109 sec latency 92.531 ms 2024-04-02T13:52:29.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:29 smithi187 ceph-mon[36064]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-04-02T13:52:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:29 smithi188 ceph-mon[36117]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-04-02T13:52:30.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:29 smithi122 ceph-mon[32114]: pgmap v544: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 544 op/s 2024-04-02T13:52:30.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127283 30.19 MB/sec execute 110 sec latency 1092.574 ms 2024-04-02T13:52:31.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127500 30.02 MB/sec execute 111 sec latency 1382.431 ms 2024-04-02T13:52:31.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:31 smithi187 ceph-mon[36064]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.6 MiB/s wr, 407 op/s 2024-04-02T13:52:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:31 smithi188 ceph-mon[36117]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.6 MiB/s wr, 407 op/s 2024-04-02T13:52:32.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:31 smithi122 ceph-mon[32114]: pgmap v545: 129 pgs: 129 active+clean; 1.7 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 5.6 MiB/s wr, 407 op/s 2024-04-02T13:52:32.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 127714 29.86 MB/sec execute 112 sec latency 164.003 ms 2024-04-02T13:52:33.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 128242 29.86 MB/sec execute 113 sec latency 114.714 ms 2024-04-02T13:52:33.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:33 smithi187 ceph-mon[36064]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 679 op/s 2024-04-02T13:52:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:33 smithi188 ceph-mon[36117]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 679 op/s 2024-04-02T13:52:34.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:33 smithi122 ceph-mon[32114]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 679 op/s 2024-04-02T13:52:34.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 128953 29.90 MB/sec execute 114 sec latency 64.132 ms 2024-04-02T13:52:35.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 129557 29.96 MB/sec execute 115 sec latency 68.421 ms 2024-04-02T13:52:35.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:35 smithi187 ceph-mon[36064]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-04-02T13:52:36.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:35 smithi188 ceph-mon[36117]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-04-02T13:52:36.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:35 smithi122 ceph-mon[32114]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 579 op/s 2024-04-02T13:52:36.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 130115 29.91 MB/sec execute 116 sec latency 137.170 ms 2024-04-02T13:52:36.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:36 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:52:36.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:52:36.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:52:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:36 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:52:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:52:37.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:52:37.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:36 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:52:37.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:52:37.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:52:37.329 INFO:tasks.workunit.client.0.smithi122.stdout: 10 130613 29.91 MB/sec execute 117 sec latency 160.511 ms 2024-04-02T13:52:37.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:37 smithi187 ceph-mon[36064]: pgmap v548: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 627 op/s 2024-04-02T13:52:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:52:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:52:37.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:52:38.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:37 smithi188 ceph-mon[36117]: pgmap v548: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 627 op/s 2024-04-02T13:52:38.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:52:38.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:52:38.004 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:52:38.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:37 smithi122 ceph-mon[32114]: pgmap v548: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 627 op/s 2024-04-02T13:52:38.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:52:38.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:52:38.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:52:38.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 131342 30.05 MB/sec execute 118 sec latency 90.614 ms 2024-04-02T13:52:38.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:38 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:38 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:38.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:38 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:52:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:38 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:38 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:38 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:52:39.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:38 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:52:39.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:38 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:52:39.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:38 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:52:39.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 131847 29.95 MB/sec execute 119 sec latency 91.174 ms 2024-04-02T13:52:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:39 smithi187 ceph-mon[36064]: pgmap v549: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 730 op/s 2024-04-02T13:52:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:39 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:39 smithi188 ceph-mon[36117]: pgmap v549: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 730 op/s 2024-04-02T13:52:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:39 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:40.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:39 smithi122 ceph-mon[32114]: pgmap v549: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 730 op/s 2024-04-02T13:52:40.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:39 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:40.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 132498 30.01 MB/sec execute 120 sec latency 66.495 ms 2024-04-02T13:52:40.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:40 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:41.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:40 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:41.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:40 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:41.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 133240 30.09 MB/sec execute 121 sec latency 60.314 ms 2024-04-02T13:52:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:41 smithi187 ceph-mon[36064]: pgmap v550: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 608 op/s 2024-04-02T13:52:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:41 smithi188 ceph-mon[36117]: pgmap v550: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 608 op/s 2024-04-02T13:52:42.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:41 smithi122 ceph-mon[32114]: pgmap v550: 129 pgs: 129 active+clean; 1.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 608 op/s 2024-04-02T13:52:42.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 133707 30.05 MB/sec execute 122 sec latency 258.792 ms 2024-04-02T13:52:43.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 134362 30.11 MB/sec execute 123 sec latency 94.154 ms 2024-04-02T13:52:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:43 smithi187 ceph-mon[36064]: pgmap v551: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:44.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:43 smithi188 ceph-mon[36117]: pgmap v551: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:44.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:43 smithi122 ceph-mon[32114]: pgmap v551: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 799 op/s 2024-04-02T13:52:44.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 134981 30.14 MB/sec execute 124 sec latency 61.557 ms 2024-04-02T13:52:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:44 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:44 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:45.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:44 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:52:45.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 135150 29.94 MB/sec execute 125 sec latency 723.849 ms 2024-04-02T13:52:45.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:52:45.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:52:45.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.708+0000 7f291ecd5640 1 -- 172.21.15.122:0/2502969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918101a80 msgr2=0x7f2918101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.708+0000 7f291ecd5640 1 --2- 172.21.15.122:0/2502969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918101a80 0x7f2918101f00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f290c009930 tx=0x7f290c02bc50 comp rx=0 tx=0).stop 2024-04-02T13:52:45.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.709+0000 7f291ecd5640 1 -- 172.21.15.122:0/2502969264 shutdown_connections 2024-04-02T13:52:45.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.709+0000 7f291ecd5640 1 --2- 172.21.15.122:0/2502969264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918102440 0x7f291810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.709+0000 7f291ecd5640 1 --2- 172.21.15.122:0/2502969264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918101a80 0x7f2918101f00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.709+0000 7f291ecd5640 1 --2- 172.21.15.122:0/2502969264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918107a90 0x7f2918107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.709+0000 7f291ecd5640 1 -- 172.21.15.122:0/2502969264 >> 172.21.15.122:0/2502969264 conn(0x7f29180fd590 msgr2=0x7f29180ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.712+0000 7f291ecd5640 1 -- 172.21.15.122:0/2502969264 shutdown_connections 2024-04-02T13:52:45.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.715+0000 7f291ecd5640 1 -- 172.21.15.122:0/2502969264 wait complete. 2024-04-02T13:52:45.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.715+0000 7f291ecd5640 1 Processor -- start 2024-04-02T13:52:45.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 -- start start 2024-04-02T13:52:45.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 0x7f2918116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918107a90 0x7f29181113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f291810fd10 con 0x7f2918101a80 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f291810fb90 con 0x7f2918107a90 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.718+0000 7f291ecd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f291810fe90 con 0x7f2918102440 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.719+0000 7f291d272640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918107a90 0x7f29181113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.719+0000 7f2917fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.719+0000 7f2917fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57414/0 (socket says 172.21.15.122:57414) 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.719+0000 7f2917fff640 1 -- 172.21.15.122:0/4084877298 learned_addr learned my addr 172.21.15.122:0/4084877298 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 -- 172.21.15.122:0/4084877298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918107a90 msgr2=0x7f29181113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918107a90 0x7f29181113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f291ca71640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 0x7f2918116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 -- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 msgr2=0x7f2918116210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 0x7f2918116210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 -- 172.21.15.122:0/4084877298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918111ae0 con 0x7f2918102440 2024-04-02T13:52:45.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f291ca71640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 0x7f2918116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:45.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.721+0000 7f2917fff640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.723+0000 7f2915ffb640 1 -- 172.21.15.122:0/4084877298 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c063070 con 0x7f2918102440 2024-04-02T13:52:45.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.723+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2918111d70 con 0x7f2918102440 2024-04-02T13:52:45.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.724+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2918112200 con 0x7f2918102440 2024-04-02T13:52:45.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.724+0000 7f2915ffb640 1 -- 172.21.15.122:0/4084877298 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290c005410 con 0x7f2918102440 2024-04-02T13:52:45.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.724+0000 7f2915ffb640 1 -- 172.21.15.122:0/4084877298 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c05f900 con 0x7f2918102440 2024-04-02T13:52:45.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.728+0000 7f292198b740 1 Processor -- start 2024-04-02T13:52:45.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.728+0000 7f292198b740 1 -- start start 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f291d272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c23dd0 0x55ec88c261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec88b7e830 con 0x55ec88c1a8d0 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ec88b26820 con 0x55ec88c23dd0 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f292198b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ec88b7a7e0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f291d272640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50634/0 (socket says 172.21.15.122:50634) 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.729+0000 7f291d272640 1 -- 172.21.15.122:0/3474039802 learned_addr learned my addr 172.21.15.122:0/3474039802 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 -- 172.21.15.122:0/3474039802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 msgr2=0x55ec88c23890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c23890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 -- 172.21.15.122:0/3474039802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c23dd0 msgr2=0x55ec88c261c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c23dd0 0x55ec88c261c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 -- 172.21.15.122:0/3474039802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8895c7d0 con 0x55ec88c1a8d0 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.730+0000 7f291d272640 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f29080099c0 tx=0x7f2908009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861c4765bcb7e8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.731+0000 7f28feffd640 1 -- 172.21.15.122:0/3474039802 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908017070 con 0x55ec88c1a8d0 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.731+0000 7f28feffd640 1 -- 172.21.15.122:0/3474039802 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2908013470 con 0x55ec88c1a8d0 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.732+0000 7f28feffd640 1 -- 172.21.15.122:0/3474039802 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908012610 con 0x55ec88c1a8d0 2024-04-02T13:52:45.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.733+0000 7f2915ffb640 1 -- 172.21.15.122:0/4084877298 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f290c066070 con 0x7f2918102440 2024-04-02T13:52:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.733+0000 7f2915ffb640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 0x7f29000a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.733+0000 7f292198b740 1 -- 172.21.15.122:0/3474039802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 msgr2=0x55ec88c1acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f29080099c0 tx=0x7f2908009e80 comp rx=0 tx=0).stop 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 -- 172.21.15.122:0/3474039802 shutdown_connections 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c23dd0 0x55ec88c261c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 --2- 172.21.15.122:0/3474039802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c1acb0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f292198b740 1 -- 172.21.15.122:0/3474039802 >> 172.21.15.122:0/3474039802 conn(0x55ec88c15000 msgr2=0x55ec88c153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f2915ffb640 1 -- 172.21.15.122:0/4084877298 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f290c1141b0 con 0x7f2918102440 2024-04-02T13:52:45.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.734+0000 7f291ca71640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 0x7f29000a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.735+0000 7f291ca71640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 0x7f29000a63e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55ec88c1a3a0 tx=0x7f2910009400 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.736+0000 7f292198b740 1 -- 172.21.15.122:0/3474039802 shutdown_connections 2024-04-02T13:52:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.736+0000 7f292198b740 1 -- 172.21.15.122:0/3474039802 wait complete. 2024-04-02T13:52:45.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.737+0000 7f292198b740 1 -- start start 2024-04-02T13:52:45.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.737+0000 7f292198b740 1 Processor -- start 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.738+0000 7f292198b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c23610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.738+0000 7f292198b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c209a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.738+0000 7f292198b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c23dd0 0x55ec88c20ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f291ca71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c23610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f291ca71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c23610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:32992/0 (socket says 172.21.15.122:32992) 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f291ca71640 1 -- 172.21.15.122:0/3477510859 learned_addr learned my addr 172.21.15.122:0/3477510859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ec88c26bc0 con 0x55ec88c23dd0 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ec88b39d30 con 0x55ec88c1a8d0 2024-04-02T13:52:45.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ec88c26d40 con 0x55ec88c1b1f0 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.739+0000 7f292198b740 10 client.0 _ll_register_callbacks cb 0x55ec88b39720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.740+0000 7f2917fff640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c209a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 -- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 msgr2=0x55ec88c23610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c23610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 -- 172.21.15.122:0/3477510859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c23dd0 msgr2=0x55ec88c20ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c23dd0 0x55ec88c20ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec888923f0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f2917fff640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c209a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f290c1115c0 tx=0x7f290c0dfa10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c4d1b2ee349344 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.741+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2908013470 con 0x55ec88c1b1f0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.742+0000 7f28f7fff640 10 client.29206 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.742+0000 7f28f7fff640 1 -- 172.21.15.122:0/3477510859 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c0f9060 con 0x55ec88c1b1f0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.742+0000 7f28f7fff640 1 -- 172.21.15.122:0/3477510859 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f290c111730 con 0x55ec88c1b1f0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.742+0000 7f28f7fff640 1 -- 172.21.15.122:0/3477510859 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f290c004c80 con 0x55ec88c1b1f0 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.743+0000 7f292198b740 10 client.29206 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:45.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.743+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=73}) -- 0x55ec888520f0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.744+0000 7f28f7fff640 1 -- 172.21.15.122:0/3477510859 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f290c0e85f0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.744+0000 7f292198b740 10 client.29206 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:45.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.744+0000 7f292198b740 10 client.29206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:45.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.745+0000 7f292198b740 20 client.29206 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:45.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.745+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c33c30 0x55ec88c360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.745+0000 7f292198b740 4 client.29206 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:45.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.745+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec88b8bd70 con 0x55ec88c33c30 2024-04-02T13:52:45.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.748+0000 7f291ca71640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c33c30 0x55ec88c360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.751+0000 7f291ca71640 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c33c30 0x55ec88c360f0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f485f177395db37 server_cookie=25a6410f755a66e6 in_seq=0 out_seq=0 2024-04-02T13:52:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.751+0000 7f28f7fff640 10 client.29206 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:45.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.753+0000 7f28f7fff640 1 -- 172.21.15.122:0/3477510859 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec88b8bd70 con 0x55ec88c33c30 2024-04-02T13:52:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.753+0000 7f28f7fff640 10 client.29206 handle_command_reply: tid=0 2024-04-02T13:52:45.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.753+0000 7f292198b740 1 client.29206 shutdown 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.757+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 msgr2=0x55ec88c209a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.757+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c209a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f290c1115c0 tx=0x7f290c0dfa10 comp rx=0 tx=0).stop 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 shutdown_connections 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c33c30 0x55ec88c360f0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c23dd0 0x55ec88c20ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c209a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 --2- 172.21.15.122:0/3477510859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c23610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.759+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 >> 172.21.15.122:0/3477510859 conn(0x55ec88c15000 msgr2=0x55ec88c164c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.760+0000 7f28f7fff640 0 client.29206 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:45.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.763+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 shutdown_connections 2024-04-02T13:52:45.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.764+0000 7f292198b740 1 -- 172.21.15.122:0/3477510859 wait complete. 2024-04-02T13:52:45.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.766+0000 7f292198b740 20 client.29206 trim_cache size 0 max 16384 2024-04-02T13:52:45.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.800+0000 7f292198b740 1 Processor -- start 2024-04-02T13:52:45.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.800+0000 7f292198b740 1 -- start start 2024-04-02T13:52:45.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 0x55ec88c2fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c140c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 0x55ec88c230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2904010070 con 0x55ec88c1a8d0 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2904011c70 con 0x55ec88c1b1f0 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f292198b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2904010740 con 0x55ec88c15000 2024-04-02T13:52:45.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f291ca71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 0x55ec88c2fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.801+0000 7f291ca71640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 0x55ec88c2fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57426/0 (socket says 172.21.15.122:57426) 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.802+0000 7f291ca71640 1 -- 172.21.15.122:0/3832411773 learned_addr learned my addr 172.21.15.122:0/3832411773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.802+0000 7f291d272640 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 0x55ec88c230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.802+0000 7f2917fff640 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 -- 172.21.15.122:0/3832411773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 msgr2=0x55ec88c2fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 0x55ec88c2fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 -- 172.21.15.122:0/3832411773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 msgr2=0x55ec88c140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 -- 172.21.15.122:0/3832411773 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec88c199c0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.803+0000 7f291d272640 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 0x55ec88c230d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f290800a010 tx=0x7f29080096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a51146839077f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.804+0000 7f28fe7fc640 1 -- 172.21.15.122:0/3832411773 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2908017050 con 0x55ec88c1b1f0 2024-04-02T13:52:45.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.804+0000 7f28fe7fc640 1 -- 172.21.15.122:0/3832411773 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290800edf0 con 0x55ec88c1b1f0 2024-04-02T13:52:45.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 -- 172.21.15.122:0/3832411773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 msgr2=0x55ec88c230d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 0x55ec88c230d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f290800a010 tx=0x7f29080096f0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 -- 172.21.15.122:0/3832411773 shutdown_connections 2024-04-02T13:52:45.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1b1f0 0x55ec88c230d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c1a8d0 0x55ec88c140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 --2- 172.21.15.122:0/3832411773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c15000 0x55ec88c2fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.805+0000 7f292198b740 1 -- 172.21.15.122:0/3832411773 >> 172.21.15.122:0/3832411773 conn(0x55ec88c1f5e0 msgr2=0x55ec88c139b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.812+0000 7f292198b740 1 -- 172.21.15.122:0/3832411773 shutdown_connections 2024-04-02T13:52:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.813+0000 7f292198b740 1 -- 172.21.15.122:0/3832411773 wait complete. 2024-04-02T13:52:45.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.813+0000 7f292198b740 1 -- start start 2024-04-02T13:52:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.813+0000 7f292198b740 1 Processor -- start 2024-04-02T13:52:45.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.814+0000 7f292198b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 0x55ec88c2c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.814+0000 7f292198b740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.814+0000 7f291ca71640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 0x55ec88c2c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.814+0000 7f291ca71640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 0x55ec88c2c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50668/0 (socket says 172.21.15.122:50668) 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f291ca71640 1 -- 172.21.15.122:0/3171534782 learned_addr learned my addr 172.21.15.122:0/3171534782 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29040115a0 con 0x55ec88c15000 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2904010e10 con 0x55ec88c1a8d0 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f290400e070 con 0x55ec88c1b1f0 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f292198b740 10 client.0 _ll_register_callbacks cb 0x7f290c0dc020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:45.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.815+0000 7f2917fff640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.816+0000 7f2917fff640 1 -- 172.21.15.122:0/3171534782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 msgr2=0x55ec88c27d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.817+0000 7f2917fff640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.817+0000 7f2917fff640 1 -- 172.21.15.122:0/3171534782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 msgr2=0x55ec88c2c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.817+0000 7f2917fff640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 0x55ec88c2c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.817+0000 7f2917fff640 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec88c28590 con 0x55ec88c1a8d0 2024-04-02T13:52:45.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.818+0000 7f2917fff640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c277b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f290c0dfb10 tx=0x7f290c100070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b980d20a8e6f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.819+0000 7f28f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.819+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f290c1118a0 con 0x55ec88c1a8d0 2024-04-02T13:52:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.819+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f290c0df260 con 0x55ec88c1a8d0 2024-04-02T13:52:45.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.819+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f290800edf0 con 0x55ec88c1a8d0 2024-04-02T13:52:45.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.821+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f290c004c80 con 0x55ec88c1a8d0 2024-04-02T13:52:45.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.825+0000 7f292198b740 10 client.29459 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:45.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.825+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55ec88c28590 con 0x55ec88c1a8d0 2024-04-02T13:52:45.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.825+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f290800edf0 con 0x55ec88c1a8d0 2024-04-02T13:52:45.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.825+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f290c0decd0 con 0x55ec88c1a8d0 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 10 client.29459 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 10 client.29459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 20 client.29459 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c23dd0 0x55ec88c9c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 4 client.29459 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:52:45.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.829+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec88b860c0 con 0x55ec88c23dd0 2024-04-02T13:52:45.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.830+0000 7f291ca71640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c23dd0 0x55ec88c9c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:45.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.834+0000 7f291ca71640 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c23dd0 0x55ec88c9c660 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557652b9c08a7607 server_cookie=6ce2744ddafb407b in_seq=0 out_seq=0 2024-04-02T13:52:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.835+0000 7f28f7fff640 10 client.29459 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.835+0000 7f28f7fff640 1 -- 172.21.15.122:0/3171534782 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ec88b860c0 con 0x55ec88c23dd0 2024-04-02T13:52:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.835+0000 7f28f7fff640 10 client.29459 handle_command_reply: tid=0 2024-04-02T13:52:45.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.835+0000 7f292198b740 1 client.29459 shutdown 2024-04-02T13:52:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.843+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 msgr2=0x55ec88c277b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.843+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c277b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f290c0dfb10 tx=0x7f290c100070 comp rx=0 tx=0).stop 2024-04-02T13:52:45.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 shutdown_connections 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55ec88c23dd0 0x55ec88c9c660 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ec88c1b1f0 0x55ec88c27d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ec88c1a8d0 0x55ec88c277b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 --2- 172.21.15.122:0/3171534782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ec88c15000 0x55ec88c2c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 >> 172.21.15.122:0/3171534782 conn(0x55ec88c1f5e0 msgr2=0x55ec88c1b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f28f7fff640 0 client.29459 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 shutdown_connections 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 1 -- 172.21.15.122:0/3171534782 wait complete. 2024-04-02T13:52:45.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.845+0000 7f292198b740 20 client.29459 trim_cache size 0 max 16384 2024-04-02T13:52:45.852 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:45.852 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:52:45.853 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:52:45.853 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:45.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.851+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 msgr2=0x7f29000a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.851+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 0x7f29000a63e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x55ec88c1a3a0 tx=0x7f2910009400 comp rx=0 tx=0).stop 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.851+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 msgr2=0x7f2918116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.851+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f290c009900 tx=0x7f290c0042f0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.851+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 shutdown_connections 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f29000a3f20 0x7f29000a63e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2918107a90 0x7f29181113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918102440 0x7f2918116750 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 --2- 172.21.15.122:0/4084877298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2918101a80 0x7f2918116210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:45.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 >> 172.21.15.122:0/4084877298 conn(0x7f29180fd590 msgr2=0x7f29180fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:45.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 shutdown_connections 2024-04-02T13:52:45.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:45.852+0000 7f291ecd5640 1 -- 172.21.15.122:0/4084877298 wait complete. 2024-04-02T13:52:45.876 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:52:45.876 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:52:45.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:45 smithi187 ceph-mon[36064]: pgmap v552: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2024-04-02T13:52:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:45 smithi188 ceph-mon[36117]: pgmap v552: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2024-04-02T13:52:46.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 -- 172.21.15.122:0/4158319052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb364110330 msgr2=0x7fb36410de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 --2- 172.21.15.122:0/4158319052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb364110330 0x7fb36410de20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb354009900 tx=0x7fb35402bc50 comp rx=0 tx=0).stop 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 -- 172.21.15.122:0/4158319052 shutdown_connections 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 --2- 172.21.15.122:0/4158319052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb36411b2e0 0x7fb36411d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 --2- 172.21.15.122:0/4158319052 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36410e360 0x7fb36410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 --2- 172.21.15.122:0/4158319052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb364110330 0x7fb36410de20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.061+0000 7fb36a688640 1 -- 172.21.15.122:0/4158319052 >> 172.21.15.122:0/4158319052 conn(0x7fb36406cbf0 msgr2=0x7fb36406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.063+0000 7fb36a688640 1 -- 172.21.15.122:0/4158319052 shutdown_connections 2024-04-02T13:52:46.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.064+0000 7fb36a688640 1 -- 172.21.15.122:0/4158319052 wait complete. 2024-04-02T13:52:46.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.065+0000 7fb36a688640 1 Processor -- start 2024-04-02T13:52:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.066+0000 7fb36a688640 1 -- start start 2024-04-02T13:52:46.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36410e360 0x7fb36411aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:45 smithi122 ceph-mon[32114]: pgmap v552: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 542 op/s 2024-04-02T13:52:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb364110330 0x7fb364114b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb364111510 con 0x7fb36411b2e0 2024-04-02T13:52:46.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb364111390 con 0x7fb364110330 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb36a688640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb364111690 con 0x7fb36410e360 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb368c25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb368c25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50678/0 (socket says 172.21.15.122:50678) 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.067+0000 7fb368c25640 1 -- 172.21.15.122:0/3741185599 learned_addr learned my addr 172.21.15.122:0/3741185599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.068+0000 7fb368c25640 1 -- 172.21.15.122:0/3741185599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36410e360 msgr2=0x7fb36411aac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.068+0000 7fb368c25640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36410e360 0x7fb36411aac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.068+0000 7fb368c25640 1 -- 172.21.15.122:0/3741185599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb364110330 msgr2=0x7fb364114b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.068+0000 7fb368c25640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb364110330 0x7fb364114b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.068+0000 7fb368c25640 1 -- 172.21.15.122:0/3741185599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb364115840 con 0x7fb36411b2e0 2024-04-02T13:52:46.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.069+0000 7fb368c25640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00e970 tx=0x7fb34c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.071+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c00ccc0 con 0x7fb36411b2e0 2024-04-02T13:52:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.071+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34c00ce60 con 0x7fb36411b2e0 2024-04-02T13:52:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.071+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c010490 con 0x7fb36411b2e0 2024-04-02T13:52:46.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.071+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3641bc030 con 0x7fb36411b2e0 2024-04-02T13:52:46.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.071+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3641bc530 con 0x7fb36411b2e0 2024-04-02T13:52:46.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.075+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb34c0046c0 con 0x7fb36411b2e0 2024-04-02T13:52:46.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.074+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36406de90 con 0x7fb36411b2e0 2024-04-02T13:52:46.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.076+0000 7fb3617fa640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 0x7fb3500a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.076+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fb34c0d1040 con 0x7fb36411b2e0 2024-04-02T13:52:46.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.076+0000 7fb363fff640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 0x7fb3500a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.077+0000 7fb363fff640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 0x7fb3500a6330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb354036040 tx=0x7fb354005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.086+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb34c09d290 con 0x7fb36411b2e0 2024-04-02T13:52:46.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 135397 29.79 MB/sec execute 126 sec latency 978.377 ms 2024-04-02T13:52:46.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.538+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3641bc850 con 0x7fb36411b2e0 2024-04-02T13:52:46.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.540+0000 7fb3617fa640 1 -- 172.21.15.122:0/3741185599 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11949 (secure 0 0 0) 0x7fb34c09cb90 con 0x7fb36411b2e0 2024-04-02T13:52:46.544 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:52:46.545 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:52:42.709142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:52:46.545 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 73 2024-04-02T13:52:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.544+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 msgr2=0x7fb3500a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.544+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 0x7fb3500a6330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb354036040 tx=0x7fb354005fd0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.544+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 msgr2=0x7fb364115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.544+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00e970 tx=0x7fb34c00ed70 comp rx=0 tx=0).stop 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 shutdown_connections 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb3500a3e70 0x7fb3500a6330 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb36411b2e0 0x7fb364115070 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb364110330 0x7fb364114b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 --2- 172.21.15.122:0/3741185599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36410e360 0x7fb36411aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 >> 172.21.15.122:0/3741185599 conn(0x7fb36406cbf0 msgr2=0x7fb3640725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.547+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 shutdown_connections 2024-04-02T13:52:46.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.548+0000 7fb36a688640 1 -- 172.21.15.122:0/3741185599 wait complete. 2024-04-02T13:52:46.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:52:46.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.759+0000 7f1443b7c640 1 -- 172.21.15.122:0/920436522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 msgr2=0x7f143c10de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.759+0000 7f1443b7c640 1 --2- 172.21.15.122:0/920436522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c10de10 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f143801a910 tx=0x7f143803cc90 comp rx=0 tx=0).stop 2024-04-02T13:52:46.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.766+0000 7f1443b7c640 1 -- 172.21.15.122:0/920436522 shutdown_connections 2024-04-02T13:52:46.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.766+0000 7f1443b7c640 1 --2- 172.21.15.122:0/920436522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c11d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.766+0000 7f1443b7c640 1 --2- 172.21.15.122:0/920436522 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.766+0000 7f1443b7c640 1 --2- 172.21.15.122:0/920436522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c10de10 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.766+0000 7f1443b7c640 1 -- 172.21.15.122:0/920436522 >> 172.21.15.122:0/920436522 conn(0x7f143c06cbe0 msgr2=0x7f143c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.770+0000 7f1443b7c640 1 -- 172.21.15.122:0/920436522 shutdown_connections 2024-04-02T13:52:46.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.772+0000 7f1443b7c640 1 -- 172.21.15.122:0/920436522 wait complete. 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.773+0000 7f1443b7c640 1 Processor -- start 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 -- start start 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c11aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c116b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c1170a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f143c1134c0 con 0x7f143c110320 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f143c113340 con 0x7f143c10e350 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.774+0000 7f1443b7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f143c113640 con 0x7f143c11b2e0 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.776+0000 7f1441117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c116b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.776+0000 7f1441117640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c116b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50686/0 (socket says 172.21.15.122:50686) 2024-04-02T13:52:46.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.776+0000 7f1441117640 1 -- 172.21.15.122:0/3027513083 learned_addr learned my addr 172.21.15.122:0/3027513083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.777+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c11aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1442119640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c1170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1441918640 1 -- 172.21.15.122:0/3027513083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 msgr2=0x7f143c1170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c1170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1441918640 1 -- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 msgr2=0x7f143c116b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c116b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1441918640 1 -- 172.21.15.122:0/3027513083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c1177d0 con 0x7f143c10e350 2024-04-02T13:52:46.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.778+0000 7f1442119640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c1170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:46.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.779+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c11aa30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1438004ff0 tx=0x7f1438005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.781+0000 7f1432ffd640 1 -- 172.21.15.122:0/3027513083 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438052020 con 0x7f143c10e350 2024-04-02T13:52:46.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.781+0000 7f1432ffd640 1 -- 172.21.15.122:0/3027513083 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143803dd20 con 0x7f143c10e350 2024-04-02T13:52:46.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.781+0000 7f1432ffd640 1 -- 172.21.15.122:0/3027513083 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1438003d50 con 0x7f143c10e350 2024-04-02T13:52:46.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.782+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c073d30 con 0x7f143c10e350 2024-04-02T13:52:46.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.782+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c074280 con 0x7f143c10e350 2024-04-02T13:52:46.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 Processor -- start 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 -- start start 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e869090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d48e7c17b0 con 0x55d48e85e0c0 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d48e760570 con 0x55d48e85d7a0 2024-04-02T13:52:46.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.789+0000 7f14463fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d48e7bd760 con 0x55d48e866ca0 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1432ffd640 1 -- 172.21.15.122:0/3027513083 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1438055020 con 0x7f143c10e350 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33040/0 (socket says 172.21.15.122:33040) 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 -- 172.21.15.122:0/942588282 learned_addr learned my addr 172.21.15.122:0/942588282 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 -- 172.21.15.122:0/942588282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 msgr2=0x55d48e869090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e869090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 -- 172.21.15.122:0/942588282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 msgr2=0x55d48e866760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1442119640 1 -- 172.21.15.122:0/942588282 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48e4d53f0 con 0x55d48e85d7a0 2024-04-02T13:52:46.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.790+0000 7f1432ffd640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 0x7f14280a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.791+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 0x7f14280a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.791+0000 7f1442119640 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f142c00a940 tx=0x7f142c00ae00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=544a9d650c1d8b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.791+0000 7f1432ffd640 1 -- 172.21.15.122:0/3027513083 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f1438103940 con 0x7f143c10e350 2024-04-02T13:52:46.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.792+0000 7f1441918640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 0x7f14280a63e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55d48e85d270 tx=0x7f14380fa6b0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.793+0000 7f1427fff640 1 -- 172.21.15.122:0/942588282 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c015070 con 0x55d48e85d7a0 2024-04-02T13:52:46.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.793+0000 7f1427fff640 1 -- 172.21.15.122:0/942588282 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c00cc00 con 0x55d48e85d7a0 2024-04-02T13:52:46.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.793+0000 7f1427fff640 1 -- 172.21.15.122:0/942588282 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c0185a0 con 0x55d48e85d7a0 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 -- 172.21.15.122:0/942588282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 msgr2=0x55d48e85db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f142c00a940 tx=0x7f142c00ae00 comp rx=0 tx=0).stop 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 -- 172.21.15.122:0/942588282 shutdown_connections 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e869090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 --2- 172.21.15.122:0/942588282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e85db80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.795+0000 7f14463fd740 1 -- 172.21.15.122:0/942588282 >> 172.21.15.122:0/942588282 conn(0x55d48e857ed0 msgr2=0x55d48e8582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.796+0000 7f14463fd740 1 -- 172.21.15.122:0/942588282 shutdown_connections 2024-04-02T13:52:46.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.797+0000 7f14463fd740 1 -- 172.21.15.122:0/942588282 wait complete. 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.797+0000 7f14463fd740 1 -- start start 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.797+0000 7f14463fd740 1 Processor -- start 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e866750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e864220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d48e869a90 con 0x55d48e866ca0 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d48e77ce80 con 0x55d48e85e0c0 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d48e869c10 con 0x55d48e85d7a0 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f14463fd740 10 client.0 _ll_register_callbacks cb 0x55d48e7b8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f1441117640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f1441117640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33050/0 (socket says 172.21.15.122:33050) 2024-04-02T13:52:46.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.798+0000 7f1441117640 1 -- 172.21.15.122:0/2161790603 learned_addr learned my addr 172.21.15.122:0/2161790603 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.801+0000 7f1442119640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e866750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.802+0000 7f1442119640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e864220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1441117640 1 -- 172.21.15.122:0/2161790603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 msgr2=0x55d48e866750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1441117640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e866750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1441117640 1 -- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 msgr2=0x55d48e864220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1441117640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e864220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1441117640 1 -- 172.21.15.122:0/2161790603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48e59f7d0 con 0x55d48e85e0c0 2024-04-02T13:52:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.803+0000 7f1442119640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e864220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:46.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.804+0000 7f1441117640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f14340327c0 tx=0x7f1434032c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eaffcc38d38d4cb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.804+0000 7f14257fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.805+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434030cf0 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.805+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f143403e480 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.806+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142c00cc00 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.806+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f1434047050 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.806+0000 7f14463fd740 10 client.29474 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.806+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d48e897c10 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1434030e90 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f1434033800 con 0x55d48e85e0c0 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 10 client.29474 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:46.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 10 client.29474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 20 client.29474 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e899550 0x55d48e89b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 4 client.29474 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f142c00cc00 con 0x55d48e899550 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.807+0000 7f1442119640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e899550 0x55d48e89b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.808+0000 7f1442119640 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e899550 0x55d48e89b9b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b33847559ae20ac server_cookie=be32a80be6a587c1 in_seq=0 out_seq=0 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.808+0000 7f14257fa640 10 client.29474 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14257fa640 1 -- 172.21.15.122:0/2161790603 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f142c00cc00 con 0x55d48e899550 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14257fa640 10 client.29474 handle_command_reply: tid=0 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14463fd740 1 client.29474 shutdown 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 msgr2=0x55d48e863ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f14340327c0 tx=0x7f1434032c80 comp rx=0 tx=0).stop 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 shutdown_connections 2024-04-02T13:52:46.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.809+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e899550 0x55d48e89b9b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e864220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e863ae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14463fd740 1 --2- 172.21.15.122:0/2161790603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e866750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 >> 172.21.15.122:0/2161790603 conn(0x55d48e857ed0 msgr2=0x55d48e859c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14257fa640 0 client.29474 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:46.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.810+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 shutdown_connections 2024-04-02T13:52:46.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.811+0000 7f14463fd740 1 -- 172.21.15.122:0/2161790603 wait complete. 2024-04-02T13:52:46.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.812+0000 7f14463fd740 20 client.29474 trim_cache size 0 max 16384 2024-04-02T13:52:46.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:46 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3741185599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.841+0000 7f14463fd740 1 Processor -- start 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.841+0000 7f14463fd740 1 -- start start 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e8729b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07e460 con 0x55d48e85e0c0 2024-04-02T13:52:46.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f143c080060 con 0x55d48e85d7a0 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f14463fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07eb30 con 0x55d48e866ca0 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57464/0 (socket says 172.21.15.122:57464) 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 -- 172.21.15.122:0/1784975952 learned_addr learned my addr 172.21.15.122:0/1784975952 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e8729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 -- 172.21.15.122:0/1784975952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 msgr2=0x55d48e8729b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e8729b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.842+0000 7f1442119640 1 -- 172.21.15.122:0/1784975952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 msgr2=0x55d48e866210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:46.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.843+0000 7f1442119640 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.843+0000 7f1442119640 1 -- 172.21.15.122:0/1784975952 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d48e864590 con 0x55d48e866ca0 2024-04-02T13:52:46.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.845+0000 7f1442119640 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c520 tx=0x7f142c01c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40aa8c9b77413ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.850+0000 7f14277fe640 1 -- 172.21.15.122:0/1784975952 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c015040 con 0x55d48e866ca0 2024-04-02T13:52:46.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.850+0000 7f14277fe640 1 -- 172.21.15.122:0/1784975952 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c02a840 con 0x55d48e866ca0 2024-04-02T13:52:46.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.851+0000 7f14463fd740 1 -- 172.21.15.122:0/1784975952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 msgr2=0x55d48e866750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.851+0000 7f14463fd740 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c520 tx=0x7f142c01c040 comp rx=0 tx=0).stop 2024-04-02T13:52:46.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.855+0000 7f14463fd740 1 -- 172.21.15.122:0/1784975952 shutdown_connections 2024-04-02T13:52:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.855+0000 7f14463fd740 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e866ca0 0x55d48e866750 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.855+0000 7f14463fd740 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e85e0c0 0x55d48e866210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.855+0000 7f14463fd740 1 --2- 172.21.15.122:0/1784975952 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85d7a0 0x55d48e8729b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.855+0000 7f14463fd740 1 -- 172.21.15.122:0/1784975952 >> 172.21.15.122:0/1784975952 conn(0x55d48e857ed0 msgr2=0x55d48e8582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.857+0000 7f14463fd740 1 -- 172.21.15.122:0/1784975952 shutdown_connections 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.858+0000 7f14463fd740 1 -- 172.21.15.122:0/1784975952 wait complete. 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.858+0000 7f14463fd740 1 -- start start 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.858+0000 7f14463fd740 1 Processor -- start 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e860160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e8606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e86dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07f990 con 0x55d48e866ca0 2024-04-02T13:52:46.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07f200 con 0x55d48e85e0c0 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f143c07c460 con 0x55d48e85d7a0 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.859+0000 7f14463fd740 10 client.0 _ll_register_callbacks cb 0x7f14340096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.860+0000 7f1442119640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e86dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.860+0000 7f1442119640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e86dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50714/0 (socket says 172.21.15.122:50714) 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.860+0000 7f1442119640 1 -- 172.21.15.122:0/1691603436 learned_addr learned my addr 172.21.15.122:0/1691603436 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.861+0000 7f1441117640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e8606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.861+0000 7f1442119640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e860160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.863+0000 7f1441117640 1 -- 172.21.15.122:0/1691603436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 msgr2=0x55d48e860160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.863+0000 7f1441117640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e860160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.863+0000 7f1441117640 1 -- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 msgr2=0x55d48e86dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:46.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.863+0000 7f1441117640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e86dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.863+0000 7f1441117640 1 -- 172.21.15.122:0/1691603436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d48e85def0 con 0x55d48e85e0c0 2024-04-02T13:52:46.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.865+0000 7f1441117640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e8606a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d48e8644c0 tx=0x7f1434030d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec20fc2522bd62c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.867+0000 7f14257fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.867+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1434030e90 con 0x55d48e85e0c0 2024-04-02T13:52:46.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.867+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f143403c380 con 0x55d48e85e0c0 2024-04-02T13:52:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.867+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142c02a840 con 0x55d48e85e0c0 2024-04-02T13:52:46.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.868+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f1434009e80 con 0x55d48e85e0c0 2024-04-02T13:52:46.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.868+0000 7f14463fd740 10 client.29480 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:46.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.868+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d48e86f340 con 0x55d48e85e0c0 2024-04-02T13:52:46.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f142c02a840 con 0x55d48e85e0c0 2024-04-02T13:52:46.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f143403c650 con 0x55d48e85e0c0 2024-04-02T13:52:46.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 10 client.29480 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:46.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 10 client.29480 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:52:46.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 20 client.29480 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e8ffb90 0x55d48e901fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 4 client.29480 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:46.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.870+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d48e7cecf0 con 0x55d48e8ffb90 2024-04-02T13:52:46.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.875+0000 7f1442119640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e8ffb90 0x55d48e901fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.875+0000 7f1442119640 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e8ffb90 0x55d48e901fb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f68123355680c71 server_cookie=b50220da69701585 in_seq=0 out_seq=0 2024-04-02T13:52:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.875+0000 7f14257fa640 10 client.29480 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:46.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.876+0000 7f14257fa640 1 -- 172.21.15.122:0/1691603436 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d48e7cecf0 con 0x55d48e8ffb90 2024-04-02T13:52:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.876+0000 7f14257fa640 10 client.29480 handle_command_reply: tid=0 2024-04-02T13:52:46.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.876+0000 7f14463fd740 1 client.29480 shutdown 2024-04-02T13:52:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 msgr2=0x55d48e8606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e8606a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55d48e8644c0 tx=0x7f1434030d20 comp rx=0 tx=0).stop 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 shutdown_connections 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55d48e8ffb90 0x55d48e901fb0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d48e866ca0 0x55d48e86dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d48e85e0c0 0x55d48e8606a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 --2- 172.21.15.122:0/1691603436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d48e85d7a0 0x55d48e860160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 >> 172.21.15.122:0/1691603436 conn(0x55d48e857ed0 msgr2=0x55d48e862370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14257fa640 0 client.29480 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:52:46.883 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 shutdown_connections 2024-04-02T13:52:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.879+0000 7f14463fd740 1 -- 172.21.15.122:0/1691603436 wait complete. 2024-04-02T13:52:46.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.880+0000 7f14463fd740 20 client.29480 trim_cache size 0 max 16384 2024-04-02T13:52:46.884 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:46.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 msgr2=0x7f14280a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 0x7f14280a63e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x55d48e85d270 tx=0x7f14380fa6b0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 msgr2=0x7f143c11aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c11aa30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1438004ff0 tx=0x7f1438005770 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 shutdown_connections 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f14280a3f20 0x7f14280a63e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c11b2e0 0x7f143c1170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f143c110320 0x7f143c116b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.882+0000 7f1443b7c640 1 --2- 172.21.15.122:0/3027513083 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f143c10e350 0x7f143c11aa30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:46.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.883+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 >> 172.21.15.122:0/3027513083 conn(0x7f143c06cbe0 msgr2=0x7f143c11c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:46.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.883+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 shutdown_connections 2024-04-02T13:52:46.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:46.883+0000 7f1443b7c640 1 -- 172.21.15.122:0/3027513083 wait complete. 2024-04-02T13:52:46.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:52:46.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:46 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3741185599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:47.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:46 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3741185599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:52:47.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.106+0000 7f85af0a2640 1 -- 172.21.15.122:0/3179111756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8100870 msgr2=0x7f85a8100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.106+0000 7f85af0a2640 1 --2- 172.21.15.122:0/3179111756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8100870 0x7f85a8100c50 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8598009860 tx=0x7f859802bc70 comp rx=0 tx=0).stop 2024-04-02T13:52:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.109+0000 7f85af0a2640 1 -- 172.21.15.122:0/3179111756 shutdown_connections 2024-04-02T13:52:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.109+0000 7f85af0a2640 1 --2- 172.21.15.122:0/3179111756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a8105c70 0x7f85a8106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.109+0000 7f85af0a2640 1 --2- 172.21.15.122:0/3179111756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8101190 0x7f85a8105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.109+0000 7f85af0a2640 1 --2- 172.21.15.122:0/3179111756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8100870 0x7f85a8100c50 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.109+0000 7f85af0a2640 1 -- 172.21.15.122:0/3179111756 >> 172.21.15.122:0/3179111756 conn(0x7f85a80fc2a0 msgr2=0x7f85a80fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.113+0000 7f85af0a2640 1 -- 172.21.15.122:0/3179111756 shutdown_connections 2024-04-02T13:52:47.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.116+0000 7f85af0a2640 1 -- 172.21.15.122:0/3179111756 wait complete. 2024-04-02T13:52:47.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.116+0000 7f85af0a2640 1 Processor -- start 2024-04-02T13:52:47.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 -- start start 2024-04-02T13:52:47.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a8100870 0x7f85a81a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8105c70 0x7f85a819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85a81182c0 con 0x7f85a8101190 2024-04-02T13:52:47.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8118140 con 0x7f85a8105c70 2024-04-02T13:52:47.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.117+0000 7f85af0a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8118440 con 0x7f85a8100870 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.118+0000 7f85ae8a1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8105c70 0x7f85a819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.118+0000 7f85ad89f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.118+0000 7f85ad89f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50736/0 (socket says 172.21.15.122:50736) 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.118+0000 7f85ad89f640 1 -- 172.21.15.122:0/2523695337 learned_addr learned my addr 172.21.15.122:0/2523695337 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.119+0000 7f85ad89f640 1 -- 172.21.15.122:0/2523695337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a8100870 msgr2=0x7f85a81a4580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:52:47.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.119+0000 7f85ad89f640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a8100870 0x7f85a81a4580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.119+0000 7f85ad89f640 1 -- 172.21.15.122:0/2523695337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8105c70 msgr2=0x7f85a819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.119+0000 7f85ad89f640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8105c70 0x7f85a819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.119+0000 7f85ad89f640 1 -- 172.21.15.122:0/2523695337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a819ee50 con 0x7f85a8101190 2024-04-02T13:52:47.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.120+0000 7f85ad89f640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f85a000ea00 tx=0x7f85a000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.120+0000 7f859f7fe640 1 -- 172.21.15.122:0/2523695337 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a000cd80 con 0x7f85a8101190 2024-04-02T13:52:47.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.120+0000 7f859f7fe640 1 -- 172.21.15.122:0/2523695337 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a0011950 con 0x7f85a8101190 2024-04-02T13:52:47.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.120+0000 7f859f7fe640 1 -- 172.21.15.122:0/2523695337 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a001aaa0 con 0x7f85a8101190 2024-04-02T13:52:47.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.121+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a819f140 con 0x7f85a8101190 2024-04-02T13:52:47.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.121+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a8106c20 con 0x7f85a8101190 2024-04-02T13:52:47.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.124+0000 7f85b2fca740 1 Processor -- start 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 -- start start 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e959960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e95c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d29e8b47b0 con 0x55d29e959ea0 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d29e853570 con 0x55d29e9512c0 2024-04-02T13:52:47.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.125+0000 7f85b2fca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d29e8b0760 con 0x55d29e9509a0 2024-04-02T13:52:47.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.126+0000 7f85ae8a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.126+0000 7f85ae8a1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57498/0 (socket says 172.21.15.122:57498) 2024-04-02T13:52:47.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.126+0000 7f85ae8a1640 1 -- 172.21.15.122:0/3316818151 learned_addr learned my addr 172.21.15.122:0/3316818151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f85ae8a1640 1 -- 172.21.15.122:0/3316818151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 msgr2=0x55d29e959960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f85ae8a1640 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e959960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f85ae8a1640 1 -- 172.21.15.122:0/3316818151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 msgr2=0x55d29e95c290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f85ae8a1640 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e95c290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f85ae8a1640 1 -- 172.21.15.122:0/3316818151 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29e5c83f0 con 0x55d29e9509a0 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.127+0000 7f859f7fe640 1 -- 172.21.15.122:0/2523695337 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f85a0011b00 con 0x7f85a8101190 2024-04-02T13:52:47.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.128+0000 7f85ae8a1640 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009bb0 tx=0x7f85a40107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5a423081f8b180 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.132+0000 7f859cbfe640 1 -- 172.21.15.122:0/3316818151 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a401c480 con 0x55d29e9509a0 2024-04-02T13:52:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.132+0000 7f859cbfe640 1 -- 172.21.15.122:0/3316818151 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a4011920 con 0x55d29e9509a0 2024-04-02T13:52:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.132+0000 7f859cbfe640 1 -- 172.21.15.122:0/3316818151 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a401a870 con 0x55d29e9509a0 2024-04-02T13:52:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.133+0000 7f85b2fca740 1 -- 172.21.15.122:0/3316818151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 msgr2=0x55d29e950d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.133+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009bb0 tx=0x7f85a40107a0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.134+0000 7f85b2fca740 1 -- 172.21.15.122:0/3316818151 shutdown_connections 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.134+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e95c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.134+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e959960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.134+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3316818151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e950d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.134+0000 7f85b2fca740 1 -- 172.21.15.122:0/3316818151 >> 172.21.15.122:0/3316818151 conn(0x55d29e94b0d0 msgr2=0x55d29e94b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.135+0000 7f859f7fe640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 0x7f85940a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.136+0000 7f859f7fe640 1 -- 172.21.15.122:0/2523695337 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f85a00d10d0 con 0x7f85a8101190 2024-04-02T13:52:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.137+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 0x7f85940a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.138+0000 7f85b2fca740 1 -- 172.21.15.122:0/3316818151 shutdown_connections 2024-04-02T13:52:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.141+0000 7f85b2fca740 1 -- 172.21.15.122:0/3316818151 wait complete. 2024-04-02T13:52:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.141+0000 7f85b2fca740 1 -- start start 2024-04-02T13:52:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.141+0000 7f85b2fca740 1 Processor -- start 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.148+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 0x7f85940a6360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85980095f0 tx=0x7f8598005760 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e956d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e957800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d29e96dca0 con 0x55d29e959ea0 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d29e86fe80 con 0x55d29e9512c0 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d29e96de20 con 0x55d29e9509a0 2024-04-02T13:52:47.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.149+0000 7f85b2fca740 10 client.0 _ll_register_callbacks cb 0x55d29e8abaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:47.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.152+0000 7f85ad89f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.152+0000 7f85ad89f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33116/0 (socket says 172.21.15.122:33116) 2024-04-02T13:52:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.152+0000 7f85ad89f640 1 -- 172.21.15.122:0/3444234762 learned_addr learned my addr 172.21.15.122:0/3444234762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.154+0000 7f85ad89f640 1 -- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 msgr2=0x55d29e956d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:52:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.154+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e956d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.154+0000 7f85ad89f640 1 -- 172.21.15.122:0/3444234762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 msgr2=0x55d29e957800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:52:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.154+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e957800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.154+0000 7f85ad89f640 1 -- 172.21.15.122:0/3444234762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29e6927d0 con 0x55d29e9512c0 2024-04-02T13:52:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.155+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ce440 tx=0x7f85a009c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105d3e83f1f1f098 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.155+0000 7f8591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.157+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00a58a0 con 0x55d29e9512c0 2024-04-02T13:52:47.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.157+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f85a00212f0 con 0x55d29e9512c0 2024-04-02T13:52:47.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.157+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a4011920 con 0x55d29e9512c0 2024-04-02T13:52:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.163+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f85a00214f0 con 0x55d29e9512c0 2024-04-02T13:52:47.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.166+0000 7f85b2fca740 10 client.29489 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:47.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.166+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d29e979e20 con 0x55d29e9512c0 2024-04-02T13:52:47.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a4011920 con 0x55d29e9512c0 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f85a0022030 con 0x55d29e9512c0 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 10 client.29489 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 10 client.29489 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 20 client.29489 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e97b6b0 0x55d29e97db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 4 client.29489 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:47.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.173+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d29e8c1cf0 con 0x55d29e97b6b0 2024-04-02T13:52:47.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.177+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e97b6b0 0x55d29e97db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.185+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e97b6b0 0x55d29e97db30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45f622312ca8d41 server_cookie=2a84150660f1164a in_seq=0 out_seq=0 2024-04-02T13:52:47.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.186+0000 7f8591ffb640 10 client.29489 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:47.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.188+0000 7f8591ffb640 1 -- 172.21.15.122:0/3444234762 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d29e8c1cf0 con 0x55d29e97b6b0 2024-04-02T13:52:47.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.188+0000 7f8591ffb640 10 client.29489 handle_command_reply: tid=0 2024-04-02T13:52:47.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.188+0000 7f85b2fca740 1 client.29489 shutdown 2024-04-02T13:52:47.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 msgr2=0x55d29e9572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85a00ce440 tx=0x7f85a009c650 comp rx=0 tx=0).stop 2024-04-02T13:52:47.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 shutdown_connections 2024-04-02T13:52:47.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e97b6b0 0x55d29e97db30 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e957800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9572c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3444234762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e956d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 >> 172.21.15.122:0/3444234762 conn(0x55d29e94b0d0 msgr2=0x55d29e94ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.196+0000 7f8591ffb640 0 client.29489 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:47.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.199+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 shutdown_connections 2024-04-02T13:52:47.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.201+0000 7f85b2fca740 1 -- 172.21.15.122:0/3444234762 wait complete. 2024-04-02T13:52:47.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.201+0000 7f85b2fca740 20 client.29489 trim_cache size 0 max 16384 2024-04-02T13:52:47.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.232+0000 7f85b2fca740 1 Processor -- start 2024-04-02T13:52:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.234+0000 7f85b2fca740 1 -- start start 2024-04-02T13:52:47.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.235+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 0x55d29e9594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.235+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 0x55d29e976b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.235+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e959ea0 0x55d29e983440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.235+0000 7f85b2fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8109a60 con 0x55d29e9512c0 2024-04-02T13:52:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.235+0000 7f85b2fca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810b660 con 0x55d29e9509a0 2024-04-02T13:52:47.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.236+0000 7f85b2fca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810a130 con 0x55d29e959ea0 2024-04-02T13:52:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.236+0000 7f85ae0a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 0x55d29e9594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.236+0000 7f85ae0a0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 0x55d29e9594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33118/0 (socket says 172.21.15.122:33118) 2024-04-02T13:52:47.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.236+0000 7f85ae0a0640 1 -- 172.21.15.122:0/3842981299 learned_addr learned my addr 172.21.15.122:0/3842981299 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ae8a1640 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e959ea0 0x55d29e983440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 0x55d29e976b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 -- 172.21.15.122:0/3842981299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e959ea0 msgr2=0x55d29e983440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e959ea0 0x55d29e983440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 -- 172.21.15.122:0/3842981299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 msgr2=0x55d29e9594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 0x55d29e9594b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.237+0000 7f85ad89f640 1 -- 172.21.15.122:0/3842981299 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d29e955310 con 0x55d29e9512c0 2024-04-02T13:52:47.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.238+0000 7f85ad89f640 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 0x55d29e976b90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f85a000cfd0 tx=0x7f85a0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c5f2811453dd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.240+0000 7f8593fff640 1 -- 172.21.15.122:0/3842981299 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00992a0 con 0x55d29e9512c0 2024-04-02T13:52:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.241+0000 7f8593fff640 1 -- 172.21.15.122:0/3842981299 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a001ac40 con 0x55d29e9512c0 2024-04-02T13:52:47.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.241+0000 7f8593fff640 1 -- 172.21.15.122:0/3842981299 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0098be0 con 0x55d29e9512c0 2024-04-02T13:52:47.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.242+0000 7f85b2fca740 1 -- 172.21.15.122:0/3842981299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 msgr2=0x55d29e976b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.242+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 0x55d29e976b90 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f85a000cfd0 tx=0x7f85a0000f40 comp rx=0 tx=0).stop 2024-04-02T13:52:47.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.244+0000 7f85b2fca740 1 -- 172.21.15.122:0/3842981299 shutdown_connections 2024-04-02T13:52:47.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.244+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e959ea0 0x55d29e983440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.244+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e9512c0 0x55d29e976b90 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.244+0000 7f85b2fca740 1 --2- 172.21.15.122:0/3842981299 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9509a0 0x55d29e9594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.244+0000 7f85b2fca740 1 -- 172.21.15.122:0/3842981299 >> 172.21.15.122:0/3842981299 conn(0x55d29e94b0d0 msgr2=0x55d29e955690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.248+0000 7f85b2fca740 1 -- 172.21.15.122:0/3842981299 shutdown_connections 2024-04-02T13:52:47.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.248+0000 7f85b2fca740 1 -- 172.21.15.122:0/3842981299 wait complete. 2024-04-02T13:52:47.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.248+0000 7f85b2fca740 1 -- start start 2024-04-02T13:52:47.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.248+0000 7f85b2fca740 1 Processor -- start 2024-04-02T13:52:47.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e9730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e953530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810af90 con 0x55d29e959ea0 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f85a810a800 con 0x55d29e9512c0 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85a8067f80 con 0x55d29e9509a0 2024-04-02T13:52:47.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.249+0000 7f85b2fca740 10 client.0 _ll_register_callbacks cb 0x7f85a000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.252+0000 7f85ae8a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e953530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.252+0000 7f85ae8a1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e953530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50756/0 (socket says 172.21.15.122:50756) 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.252+0000 7f85ae8a1640 1 -- 172.21.15.122:0/759277981 learned_addr learned my addr 172.21.15.122:0/759277981 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.252+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e9730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.253+0000 7f85ad89f640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.253+0000 7f85ae0a0640 1 -- 172.21.15.122:0/759277981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 msgr2=0x55d29e9735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.253+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9735e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.253+0000 7f85ae0a0640 1 -- 172.21.15.122:0/759277981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 msgr2=0x55d29e953530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:47.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.253+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e953530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.254+0000 7f85ae0a0640 1 -- 172.21.15.122:0/759277981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d29e953d30 con 0x55d29e9509a0 2024-04-02T13:52:47.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.255+0000 7f85ae0a0640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e9730a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8598000c00 tx=0x7f859802bc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad007a2d40c1247 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.257+0000 7f8591ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:47.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.257+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8598039040 con 0x55d29e9509a0 2024-04-02T13:52:47.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.257+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85a001ac40 con 0x55d29e9509a0 2024-04-02T13:52:47.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.257+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f85980043c0 con 0x55d29e9509a0 2024-04-02T13:52:47.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.261+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f859805c050 con 0x55d29e9509a0 2024-04-02T13:52:47.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.263+0000 7f85b2fca740 10 client.29233 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:47.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.263+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d29e9e1890 con 0x55d29e9509a0 2024-04-02T13:52:47.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a001ac40 con 0x55d29e9509a0 2024-04-02T13:52:47.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f859803c030 con 0x55d29e9509a0 2024-04-02T13:52:47.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 10 client.29233 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:47.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 10 client.29233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:52:47.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 20 client.29233 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:47.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e9e1fc0 0x55d29e9e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 4 client.29233 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:47.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.265+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d29e8bc040 con 0x55d29e9e1fc0 2024-04-02T13:52:47.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.267+0000 7f85ad89f640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e9e1fc0 0x55d29e9e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.270+0000 7f85ad89f640 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e9e1fc0 0x55d29e9e43b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9435f30e73251fa server_cookie=936f609c9cd5a3 in_seq=0 out_seq=0 2024-04-02T13:52:47.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.270+0000 7f8591ffb640 10 client.29233 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:47.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.271+0000 7f8591ffb640 1 -- 172.21.15.122:0/759277981 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d29e8bc040 con 0x55d29e9e1fc0 2024-04-02T13:52:47.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.271+0000 7f8591ffb640 10 client.29233 handle_command_reply: tid=0 2024-04-02T13:52:47.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.271+0000 7f85b2fca740 1 client.29233 shutdown 2024-04-02T13:52:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.276+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 msgr2=0x55d29e9730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.276+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e9730a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8598000c00 tx=0x7f859802bc70 comp rx=0 tx=0).stop 2024-04-02T13:52:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 shutdown_connections 2024-04-02T13:52:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55d29e9e1fc0 0x55d29e9e43b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d29e959ea0 0x55d29e953530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d29e9512c0 0x55d29e9735e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 --2- 172.21.15.122:0/759277981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d29e9509a0 0x55d29e9730a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 >> 172.21.15.122:0/759277981 conn(0x55d29e94b0d0 msgr2=0x55d29e94c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.277+0000 7f8591ffb640 0 client.29233 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:52:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.282+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 shutdown_connections 2024-04-02T13:52:47.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.282+0000 7f85b2fca740 1 -- 172.21.15.122:0/759277981 wait complete. 2024-04-02T13:52:47.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.284+0000 7f85b2fca740 20 client.29233 trim_cache size 0 max 16384 2024-04-02T13:52:47.291 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:47.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.291+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 msgr2=0x7f85940a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.292+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 0x7f85940a6360 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85980095f0 tx=0x7f8598005760 comp rx=0 tx=0).stop 2024-04-02T13:52:47.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.292+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 msgr2=0x7f85a81a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.292+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f85a000ea00 tx=0x7f85a000eec0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 shutdown_connections 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85940a3ea0 0x7f85940a6360 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f85a8105c70 0x7f85a819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a8101190 0x7f85a81a4ac0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 --2- 172.21.15.122:0/2523695337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a8100870 0x7f85a81a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.293+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 >> 172.21.15.122:0/2523695337 conn(0x7f85a80fc2a0 msgr2=0x7f85a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.297+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 shutdown_connections 2024-04-02T13:52:47.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.297+0000 7f85af0a2640 1 -- 172.21.15.122:0/2523695337 wait complete. 2024-04-02T13:52:47.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:52:47.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 135657 29.67 MB/sec execute 127 sec latency 211.119 ms 2024-04-02T13:52:47.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:47 smithi187 ceph-mon[36064]: pgmap v553: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- 172.21.15.122:0/3502780392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1041059d0 msgr2=0x7fb104105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 --2- 172.21.15.122:0/3502780392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1041059d0 0x7fb104105db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009930 tx=0x7fb0f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- 172.21.15.122:0/3502780392 shutdown_connections 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 --2- 172.21.15.122:0/3502780392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb104101a70 0x7fb10410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 --2- 172.21.15.122:0/3502780392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb104101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 --2- 172.21.15.122:0/3502780392 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1041059d0 0x7fb104105db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- 172.21.15.122:0/3502780392 >> 172.21.15.122:0/3502780392 conn(0x7fb104076f20 msgr2=0x7fb1040fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:47.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- 172.21.15.122:0/3502780392 shutdown_connections 2024-04-02T13:52:47.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- 172.21.15.122:0/3502780392 wait complete. 2024-04-02T13:52:47.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 Processor -- start 2024-04-02T13:52:47.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.991+0000 7fb10a682640 1 -- start start 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb1041983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1041059d0 0x7fb10419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb104110db0 con 0x7fb1041010b0 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb104110c30 con 0x7fb104101a70 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb10a682640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb104110f30 con 0x7fb1041059d0 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb1041983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33164/0 (socket says 172.21.15.122:33164) 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb1037fe640 1 -- 172.21.15.122:0/256735043 learned_addr learned my addr 172.21.15.122:0/256735043 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.992+0000 7fb108c1f640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1041059d0 0x7fb10419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 -- 172.21.15.122:0/256735043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1041059d0 msgr2=0x7fb10419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1041059d0 0x7fb10419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 -- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 msgr2=0x7fb1041983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb1041983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 -- 172.21.15.122:0/256735043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb10419d3c0 con 0x7fb104101a70 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb103fff640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb1041983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1037fe640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9f0 tx=0x7fb0f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1017fa640 1 -- 172.21.15.122:0/256735043 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800cd80 con 0x7fb104101a70 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb10419d6b0 con 0x7fb104101a70 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1017fa640 1 -- 172.21.15.122:0/256735043 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f8010910 con 0x7fb104101a70 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb1017fa640 1 -- 172.21.15.122:0/256735043 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8018990 con 0x7fb104101a70 2024-04-02T13:52:47.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.993+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1041a4f40 con 0x7fb104101a70 2024-04-02T13:52:47.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.997+0000 7fb1017fa640 1 -- 172.21.15.122:0/256735043 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb0f8018b30 con 0x7fb104101a70 2024-04-02T13:52:48.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.998+0000 7fb1017fa640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 0x7fb0ec0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.998+0000 7fb1017fa640 1 -- 172.21.15.122:0/256735043 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fb0f80d1f10 con 0x7fb104101a70 2024-04-02T13:52:48.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:47.998+0000 7fb103fff640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 0x7fb0ec0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:47 smithi188 ceph-mon[36117]: pgmap v553: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-04-02T13:52:48.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.029+0000 7fb103fff640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 0x7fb0ec0a62e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005ab0 tx=0x7fb0f4005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.030+0000 7fb10d25f740 1 Processor -- start 2024-04-02T13:52:48.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.030+0000 7fb10d25f740 1 -- start start 2024-04-02T13:52:48.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 0x5566270e5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 0x5566270d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55662703b7b0 con 0x5566270d1fa0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556626fda570 con 0x5566270e57f0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb10d25f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556627037760 con 0x5566270e4e40 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50786/0 (socket says 172.21.15.122:50786) 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb103fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 0x5566270e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 -- 172.21.15.122:0/2339846155 learned_addr learned my addr 172.21.15.122:0/2339846155 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb1037fe640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 0x5566270d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 -- 172.21.15.122:0/2339846155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 msgr2=0x5566270e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 0x5566270e5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 -- 172.21.15.122:0/2339846155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 msgr2=0x5566270d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 0x5566270d1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 -- 172.21.15.122:0/2339846155 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556626d4f3f0 con 0x5566270d1fa0 2024-04-02T13:52:48.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb103fff640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 0x5566270e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:48.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb1037fe640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 0x5566270d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:48.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.031+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00a430 tx=0x7fb0fc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b11c5fdab74f550 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.032+0000 7fb0ea7fc640 1 -- 172.21.15.122:0/2339846155 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc013790 con 0x5566270d1fa0 2024-04-02T13:52:48.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.032+0000 7fb0ea7fc640 1 -- 172.21.15.122:0/2339846155 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0fc019070 con 0x5566270d1fa0 2024-04-02T13:52:48.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.032+0000 7fb0ea7fc640 1 -- 172.21.15.122:0/2339846155 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc014de0 con 0x5566270d1fa0 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- 172.21.15.122:0/2339846155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 msgr2=0x5566270d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc00a430 tx=0x7fb0fc009650 comp rx=0 tx=0).stop 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- 172.21.15.122:0/2339846155 shutdown_connections 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270d2400 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e57f0 0x5566270d1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2339846155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e4e40 0x5566270e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- 172.21.15.122:0/2339846155 >> 172.21.15.122:0/2339846155 conn(0x5566270d4440 msgr2=0x5566270d4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- 172.21.15.122:0/2339846155 shutdown_connections 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- 172.21.15.122:0/2339846155 wait complete. 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 -- start start 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.033+0000 7fb10d25f740 1 Processor -- start 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270d1fa0 0x5566270eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5566270ec280 con 0x5566270e4e40 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556626ff6e80 con 0x5566270d1fa0 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5566270ec400 con 0x5566270e57f0 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb10d25f740 10 client.0 _ll_register_callbacks cb 0x556627032aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50794/0 (socket says 172.21.15.122:50794) 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb1037fe640 1 -- 172.21.15.122:0/2747772835 learned_addr learned my addr 172.21.15.122:0/2747772835 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57556/0 (socket says 172.21.15.122:57556) 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 -- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270d1fa0 msgr2=0x5566270eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270d1fa0 0x5566270eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 -- 172.21.15.122:0/2747772835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 msgr2=0x5566270e8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 -- 172.21.15.122:0/2747772835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556626e197d0 con 0x5566270e57f0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb1037fe640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb108c1f640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004a80 tx=0x7fb0fc014650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6acd0d6f4d2764e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.034+0000 7fb0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc013590 con 0x5566270e57f0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fb0fc013730 con 0x5566270e57f0 2024-04-02T13:52:48.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0fc019070 con 0x5566270e57f0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb0fc031050 con 0x5566270e57f0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 10 client.29242 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5566270f8350 con 0x5566270e57f0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0fc019070 con 0x5566270e57f0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb0fc028900 con 0x5566270e57f0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 10 client.29242 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 10 client.29242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 20 client.29242 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x5566270fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 4 client.29242 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.035+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556627048cf0 con 0x5566270f97b0 2024-04-02T13:52:48.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.036+0000 7fb103fff640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x5566270fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.036+0000 7fb0e37fe640 10 client.29242 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:48.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.036+0000 7fb103fff640 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x5566270fbba0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9373354a93bceb0 server_cookie=d2733ec94e74ab1b in_seq=0 out_seq=0 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.037+0000 7fb0e37fe640 1 -- 172.21.15.122:0/2747772835 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556627048cf0 con 0x5566270f97b0 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.037+0000 7fb0e37fe640 10 client.29242 handle_command_reply: tid=0 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.037+0000 7fb10d25f740 1 client.29242 shutdown 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 msgr2=0x5566270e92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc004a80 tx=0x7fb0fc014650 comp rx=0 tx=0).stop 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 shutdown_connections 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270e92d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e4e40 0x5566270e8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x5566270fbba0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb0e37fe640 0 client.29242 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 --2- 172.21.15.122:0/2747772835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270d1fa0 0x5566270eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 >> 172.21.15.122:0/2747772835 conn(0x5566270d4440 msgr2=0x5566270d3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 shutdown_connections 2024-04-02T13:52:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 1 -- 172.21.15.122:0/2747772835 wait complete. 2024-04-02T13:52:48.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.038+0000 7fb10d25f740 20 client.29242 trim_cache size 0 max 16384 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.059+0000 7fb10d25f740 1 Processor -- start 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.059+0000 7fb10d25f740 1 -- start start 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 0x5566270f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 0x5566270eb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041acf00 con 0x5566270e57f0 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041aeb00 con 0x5566270e4e40 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb10d25f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041ad5d0 con 0x5566270d1fa0 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 0x5566270eb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 0x5566270f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb1037fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33186/0 (socket says 172.21.15.122:33186) 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 0x5566270eb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50796/0 (socket says 172.21.15.122:50796) 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb1037fe640 1 -- 172.21.15.122:0/90449669 learned_addr learned my addr 172.21.15.122:0/90449669 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 -- 172.21.15.122:0/90449669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 msgr2=0x5566270eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270eb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 -- 172.21.15.122:0/90449669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 msgr2=0x5566270eb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 0x5566270eb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.060+0000 7fb103fff640 1 -- 172.21.15.122:0/90449669 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566270e80f0 con 0x5566270d1fa0 2024-04-02T13:52:48.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.061+0000 7fb103fff640 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 0x5566270f5180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009230 tx=0x7fb0f403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf99d2c91973d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.061+0000 7fb0e9ffb640 1 -- 172.21.15.122:0/90449669 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4057070 con 0x5566270d1fa0 2024-04-02T13:52:48.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.061+0000 7fb0e9ffb640 1 -- 172.21.15.122:0/90449669 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f4052b50 con 0x5566270d1fa0 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.061+0000 7fb0e9ffb640 1 -- 172.21.15.122:0/90449669 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4005000 con 0x5566270d1fa0 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 -- 172.21.15.122:0/90449669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 msgr2=0x5566270f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 0x5566270f5180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009230 tx=0x7fb0f403f000 comp rx=0 tx=0).stop 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 -- 172.21.15.122:0/90449669 shutdown_connections 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270e57f0 0x5566270eb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 --2- 172.21.15.122:0/90449669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270d1fa0 0x5566270f5180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.062+0000 7fb10d25f740 1 -- 172.21.15.122:0/90449669 >> 172.21.15.122:0/90449669 conn(0x5566270d4440 msgr2=0x5566270d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.063+0000 7fb10d25f740 1 -- 172.21.15.122:0/90449669 shutdown_connections 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.063+0000 7fb10d25f740 1 -- 172.21.15.122:0/90449669 wait complete. 2024-04-02T13:52:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.063+0000 7fb10d25f740 1 -- start start 2024-04-02T13:52:48.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.063+0000 7fb10d25f740 1 Processor -- start 2024-04-02T13:52:48.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041ae430 con 0x5566270d1fa0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041adca0 con 0x5566270e4e40 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1041aaf00 con 0x5566270e57f0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb10d25f740 10 client.0 _ll_register_callbacks cb 0x7fb0f4042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50812/0 (socket says 172.21.15.122:50812) 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 -- 172.21.15.122:0/3466336976 learned_addr learned my addr 172.21.15.122:0/3466336976 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb108c1f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb1037fe640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 -- 172.21.15.122:0/3466336976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 msgr2=0x5566270ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 -- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 msgr2=0x5566270f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270f1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 -- 172.21.15.122:0/3466336976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566270eab50 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb103fff640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40096c0 tx=0x7fb0f4004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71f7e136ca124c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.064+0000 7fb0e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4061060 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fb0f40524f0 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0f4052b50 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb0f4057070 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 10 client.20550 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55662715f590 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f4052b50 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb0f4003680 con 0x5566270d1fa0 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 10 client.20550 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 10 client.20550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.065+0000 7fb10d25f740 20 client.20550 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x556627161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb10d25f740 4 client.20550 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:48.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556627043040 con 0x5566270f97b0 2024-04-02T13:52:48.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb1037fe640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x556627161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb0e37fe640 10 client.20550 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:48.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.066+0000 7fb1037fe640 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x556627161c00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa0bb9452734155 server_cookie=190293888c3dda43 in_seq=0 out_seq=0 2024-04-02T13:52:48.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:47 smithi122 ceph-mon[32114]: pgmap v553: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 569 op/s 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb0e37fe640 1 -- 172.21.15.122:0/3466336976 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556627043040 con 0x5566270f97b0 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb0e37fe640 10 client.20550 handle_command_reply: tid=0 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 client.20550 shutdown 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 msgr2=0x5566270f1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb0f40096c0 tx=0x7fb0f4004190 comp rx=0 tx=0).stop 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 shutdown_connections 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5566270e57f0 0x5566270ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x5566270f97b0 0x556627161c00 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb0e37fe640 0 client.20550 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5566270e4e40 0x5566270f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 --2- 172.21.15.122:0/3466336976 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5566270d1fa0 0x5566270f1690 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 >> 172.21.15.122:0/3466336976 conn(0x5566270d4440 msgr2=0x5566270e5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 shutdown_connections 2024-04-02T13:52:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.072+0000 7fb10d25f740 1 -- 172.21.15.122:0/3466336976 wait complete. 2024-04-02T13:52:48.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.073+0000 7fb10d25f740 20 client.20550 trim_cache size 0 max 16384 2024-04-02T13:52:48.076 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.073+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 msgr2=0x7fb0ec0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.073+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 0x7fb0ec0a62e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4005ab0 tx=0x7fb0f4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 msgr2=0x7fb104198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb0f800e9f0 tx=0x7fb0f800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 shutdown_connections 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0ec0a3e20 0x7fb0ec0a62e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1041059d0 0x7fb10419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb104101a70 0x7fb104198910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 --2- 172.21.15.122:0/256735043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1041010b0 0x7fb1041983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.074+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 >> 172.21.15.122:0/256735043 conn(0x7fb104076f20 msgr2=0x7fb104100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.075+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 shutdown_connections 2024-04-02T13:52:48.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.075+0000 7fb10a682640 1 -- 172.21.15.122:0/256735043 wait complete. 2024-04-02T13:52:48.092 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:52:48.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 -- 172.21.15.122:0/3708768851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c106990 msgr2=0x7fda2c106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 --2- 172.21.15.122:0/3708768851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c106990 0x7fda2c106d70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda20009890 tx=0x7fda2002bb70 comp rx=0 tx=0).stop 2024-04-02T13:52:48.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 -- 172.21.15.122:0/3708768851 shutdown_connections 2024-04-02T13:52:48.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 --2- 172.21.15.122:0/3708768851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 --2- 172.21.15.122:0/3708768851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c100990 0x7fda2c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 --2- 172.21.15.122:0/3708768851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c106990 0x7fda2c106d70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 -- 172.21.15.122:0/3708768851 >> 172.21.15.122:0/3708768851 conn(0x7fda2c077ad0 msgr2=0x7fda2c077ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 -- 172.21.15.122:0/3708768851 shutdown_connections 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.276+0000 7fda32f72640 1 -- 172.21.15.122:0/3708768851 wait complete. 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 Processor -- start 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 -- start start 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c100990 0x7fda2c0fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c106990 0x7fda2c0fe420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c110d70 con 0x7fda2c101350 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c110bf0 con 0x7fda2c106990 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda32f72640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c110ef0 con 0x7fda2c100990 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c100990 0x7fda2c0fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50830/0 (socket says 172.21.15.122:50830) 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda2bfff640 1 -- 172.21.15.122:0/2829354648 learned_addr learned my addr 172.21.15.122:0/2829354648 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.277+0000 7fda3150f640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c106990 0x7fda2c0fe420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 -- 172.21.15.122:0/2829354648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c100990 msgr2=0x7fda2c0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c100990 0x7fda2c0fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 -- 172.21.15.122:0/2829354648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c106990 msgr2=0x7fda2c0fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c106990 0x7fda2c0fe420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 -- 172.21.15.122:0/2829354648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2c1a68b0 con 0x7fda2c101350 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda2bfff640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ea60 tx=0x7fda1800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda29ffb640 1 -- 172.21.15.122:0/2829354648 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1800cde0 con 0x7fda2c101350 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2c1a6ba0 con 0x7fda2c101350 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda29ffb640 1 -- 172.21.15.122:0/2829354648 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda180035f0 con 0x7fda2c101350 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda29ffb640 1 -- 172.21.15.122:0/2829354648 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18010670 con 0x7fda2c101350 2024-04-02T13:52:48.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.278+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda2c1a7140 con 0x7fda2c101350 2024-04-02T13:52:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.280+0000 7fda29ffb640 1 -- 172.21.15.122:0/2829354648 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fda180037a0 con 0x7fda2c101350 2024-04-02T13:52:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.281+0000 7fda29ffb640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 0x7fda1c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.281+0000 7fda29ffb640 1 -- 172.21.15.122:0/2829354648 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fda180d0e50 con 0x7fda2c101350 2024-04-02T13:52:48.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.281+0000 7fda30d0e640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 0x7fda1c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.281+0000 7fda35c32740 1 Processor -- start 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 -- start start 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650ae7a0 0x5557650aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650b7ca0 0x5557650ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557650127b0 con 0x5557650ae7a0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555764fb1570 con 0x5557650b7ca0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda35c32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55576500e760 con 0x5557650af0c0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda30d0e640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 0x7fda1c0a6210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda200097b0 tx=0x7fda200058d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650ae7a0 0x5557650aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57610/0 (socket says 172.21.15.122:57610) 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda3150f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650b7ca0 0x5557650ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 -- 172.21.15.122:0/3986473318 learned_addr learned my addr 172.21.15.122:0/3986473318 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 -- 172.21.15.122:0/3986473318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650b7ca0 msgr2=0x5557650ba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650b7ca0 0x5557650ba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 -- 172.21.15.122:0/3986473318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650ae7a0 msgr2=0x5557650aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650ae7a0 0x5557650aeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 -- 172.21.15.122:0/3986473318 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555764d263f0 con 0x5557650af0c0 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.282+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda180993d0 tx=0x7fda1809c5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e61287a5fccb350 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.283+0000 7fda16ffd640 1 -- 172.21.15.122:0/3986473318 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1809ca00 con 0x5557650af0c0 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.283+0000 7fda16ffd640 1 -- 172.21.15.122:0/3986473318 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda180b7070 con 0x5557650af0c0 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.283+0000 7fda16ffd640 1 -- 172.21.15.122:0/3986473318 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18099070 con 0x5557650af0c0 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.283+0000 7fda35c32740 1 -- 172.21.15.122:0/3986473318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 msgr2=0x5557650b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.283+0000 7fda35c32740 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda180993d0 tx=0x7fda1809c5c0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 -- 172.21.15.122:0/3986473318 shutdown_connections 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650b7ca0 0x5557650ba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650af0c0 0x5557650b7760 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 --2- 172.21.15.122:0/3986473318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650ae7a0 0x5557650aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 -- 172.21.15.122:0/3986473318 >> 172.21.15.122:0/3986473318 conn(0x5557650a8ed0 msgr2=0x5557650a92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 -- 172.21.15.122:0/3986473318 shutdown_connections 2024-04-02T13:52:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 -- 172.21.15.122:0/3986473318 wait complete. 2024-04-02T13:52:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 -- start start 2024-04-02T13:52:48.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 Processor -- start 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 0x5557650b7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.284+0000 7fda35c32740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda35c32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 0x5557650b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda35c32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5557650baa30 con 0x5557650b7ca0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda35c32740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555764fcde80 con 0x5557650af0c0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda35c32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5557650babb0 con 0x5557650ae7a0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda35c32740 10 client.0 _ll_register_callbacks cb 0x555765009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 0x5557650b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 0x5557650b7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:57618/0 (socket says 172.21.15.122:57618) 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33244/0 (socket says 172.21.15.122:33244) 2024-04-02T13:52:48.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda30d0e640 1 -- 172.21.15.122:0/3296845500 learned_addr learned my addr 172.21.15.122:0/3296845500 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda3150f640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 0x5557650b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 -- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 msgr2=0x5557650b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 0x5557650b7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 -- 172.21.15.122:0/3296845500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 msgr2=0x5557650b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 0x5557650b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 -- 172.21.15.122:0/3296845500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555764df07d0 con 0x5557650af0c0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda3150f640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 0x5557650b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda2bfff640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda180993d0 tx=0x7fda180ce150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8117d994853e3263 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.285+0000 7fda0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1809ca00 con 0x5557650af0c0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fda18099070 con 0x5557650af0c0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda180b7070 con 0x5557650af0c0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fda180bbbc0 con 0x5557650af0c0 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda35c32740 10 client.29522 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5557650c6b20 con 0x5557650af0c0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.286+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda180bbd60 con 0x5557650af0c0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fda18087080 con 0x5557650af0c0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 10 client.29522 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 10 client.29522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 20 client.29522 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x5557650ca260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 4 client.29522 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fda180b7070 con 0x5557650c7e70 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.287+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x5557650ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.288+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x5557650ca260 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de59b7d02f024856 server_cookie=4d4e28e357859228 in_seq=0 out_seq=0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.288+0000 7fda0bfff640 10 client.29522 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda0bfff640 1 -- 172.21.15.122:0/3296845500 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fda180b7070 con 0x5557650c7e70 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda0bfff640 10 client.29522 handle_command_reply: tid=0 2024-04-02T13:52:48.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 client.29522 shutdown 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 msgr2=0x5557650b4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda180993d0 tx=0x7fda180ce150 comp rx=0 tx=0).stop 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 shutdown_connections 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x5557650ca260 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda0bfff640 0 client.29522 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650b7ca0 0x5557650b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650af0c0 0x5557650b4ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 --2- 172.21.15.122:0/3296845500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650ae7a0 0x5557650b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 >> 172.21.15.122:0/3296845500 conn(0x5557650a8ed0 msgr2=0x5557650aac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 shutdown_connections 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 1 -- 172.21.15.122:0/3296845500 wait complete. 2024-04-02T13:52:48.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.289+0000 7fda35c32740 20 client.29522 trim_cache size 0 max 16384 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 Processor -- start 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 -- start start 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557650cd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557650cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557650cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1b13d0 con 0x5557650af0c0 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1b2fd0 con 0x5557650ae7a0 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda35c32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1b1aa0 con 0x5557650b7ca0 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda2bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557650cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50852/0 (socket says 172.21.15.122:50852) 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557650cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda2bfff640 1 -- 172.21.15.122:0/3744212424 learned_addr learned my addr 172.21.15.122:0/3744212424 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda3150f640 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda30d0e640 1 -- 172.21.15.122:0/3744212424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 msgr2=0x5557650b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.308+0000 7fda30d0e640 1 -- 172.21.15.122:0/3744212424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 msgr2=0x5557650cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557650cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda30d0e640 1 -- 172.21.15.122:0/3744212424 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557650c46f0 con 0x5557650ae7a0 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda30d0e640 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557650cd280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5557650bace0 tx=0x7fda2003e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f653f63c2b49f1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda167fc640 1 -- 172.21.15.122:0/3744212424 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2005a070 con 0x5557650ae7a0 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda167fc640 1 -- 172.21.15.122:0/3744212424 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda20044040 con 0x5557650ae7a0 2024-04-02T13:52:48.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.309+0000 7fda167fc640 1 -- 172.21.15.122:0/3744212424 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda20039040 con 0x5557650ae7a0 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 -- 172.21.15.122:0/3744212424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 msgr2=0x5557650cd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557650cd280 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5557650bace0 tx=0x7fda2003e000 comp rx=0 tx=0).stop 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 -- 172.21.15.122:0/3744212424 shutdown_connections 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557650cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 --2- 172.21.15.122:0/3744212424 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557650cd280 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 -- 172.21.15.122:0/3744212424 >> 172.21.15.122:0/3744212424 conn(0x5557650a8ed0 msgr2=0x5557650adf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 -- 172.21.15.122:0/3744212424 shutdown_connections 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.310+0000 7fda35c32740 1 -- 172.21.15.122:0/3744212424 wait complete. 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.311+0000 7fda35c32740 1 -- start start 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.311+0000 7fda35c32740 1 Processor -- start 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.311+0000 7fda35c32740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557651258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1b2900 con 0x5557650af0c0 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1b2170 con 0x5557650ae7a0 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fda2c1af3d0 con 0x5557650b7ca0 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:33262/0 (socket says 172.21.15.122:33262) 2024-04-02T13:52:48.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 -- 172.21.15.122:0/1485001736 learned_addr learned my addr 172.21.15.122:0/1485001736 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda35c32740 10 client.0 _ll_register_callbacks cb 0x7fda180b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda2bfff640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557651258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 -- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 msgr2=0x5557650ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 -- 172.21.15.122:0/1485001736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 msgr2=0x5557651258f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557651258f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 -- 172.21.15.122:0/1485001736 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557650cacc0 con 0x5557650ae7a0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda30d0e640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda2002c0b0 tx=0x7fda20054640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c826650d817139 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.312+0000 7fda0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2005a070 con 0x5557650ae7a0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fda20054740 con 0x5557650ae7a0 2024-04-02T13:52:48.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda20044040 con 0x5557650ae7a0 2024-04-02T13:52:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fda20039040 con 0x5557650ae7a0 2024-04-02T13:52:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda35c32740 10 client.29528 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.313+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55576512db40 con 0x5557650ae7a0 2024-04-02T13:52:48.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda20044040 con 0x5557650ae7a0 2024-04-02T13:52:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fda20054030 con 0x5557650ae7a0 2024-04-02T13:52:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 10 client.29528 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 10 client.29528 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:52:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 20 client.29528 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x555765130320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 4 client.29528 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557650af950 con 0x5557650c7e70 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.314+0000 7fda2bfff640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x555765130320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.315+0000 7fda2bfff640 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x555765130320 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2dd54d59e63e1c54 server_cookie=5132e956061fcebf in_seq=0 out_seq=0 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.315+0000 7fda0bfff640 10 client.29528 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda0bfff640 1 -- 172.21.15.122:0/1485001736 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557650af950 con 0x5557650c7e70 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda0bfff640 10 client.29528 handle_command_reply: tid=0 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 client.29528 shutdown 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 msgr2=0x5557651253b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda2002c0b0 tx=0x7fda20054640 comp rx=0 tx=0).stop 2024-04-02T13:52:48.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 shutdown_connections 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5557650c7e70 0x555765130320 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda0bfff640 0 client.29528 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5557650b7ca0 0x5557650ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5557650af0c0 0x5557651258f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 --2- 172.21.15.122:0/1485001736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5557650ae7a0 0x5557651253b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 >> 172.21.15.122:0/1485001736 conn(0x5557650a8ed0 msgr2=0x5557650aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 shutdown_connections 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.316+0000 7fda35c32740 1 -- 172.21.15.122:0/1485001736 wait complete. 2024-04-02T13:52:48.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda35c32740 20 client.29528 trim_cache size 0 max 16384 2024-04-02T13:52:48.324 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 msgr2=0x7fda1c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 0x7fda1c0a6210 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fda200097b0 tx=0x7fda200058d0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 msgr2=0x7fda2c0fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ea60 tx=0x7fda1800ef20 comp rx=0 tx=0).stop 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 shutdown_connections 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fda1c0a3d50 0x7fda1c0a6210 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda2c106990 0x7fda2c0fe420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda2c101350 0x7fda2c0fdee0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 --2- 172.21.15.122:0/2829354648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fda2c100990 0x7fda2c0fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.317+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 >> 172.21.15.122:0/2829354648 conn(0x7fda2c077ad0 msgr2=0x7fda2c06a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.318+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 shutdown_connections 2024-04-02T13:52:48.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.318+0000 7fda32f72640 1 -- 172.21.15.122:0/2829354648 wait complete. 2024-04-02T13:52:48.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 136185 29.69 MB/sec execute 128 sec latency 112.869 ms 2024-04-02T13:52:48.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:52:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- 172.21.15.122:0/640035786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c107b50 msgr2=0x7fba1c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 --2- 172.21.15.122:0/640035786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c107b50 0x7fba1c107f30 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fba18009930 tx=0x7fba1802bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:48.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- 172.21.15.122:0/640035786 shutdown_connections 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 --2- 172.21.15.122:0/640035786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c0696c0 0x7fba1c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 --2- 172.21.15.122:0/640035786 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 0x7fba1c069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 --2- 172.21.15.122:0/640035786 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c107b50 0x7fba1c107f30 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- 172.21.15.122:0/640035786 >> 172.21.15.122:0/640035786 conn(0x7fba1c077b00 msgr2=0x7fba1c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- 172.21.15.122:0/640035786 shutdown_connections 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- 172.21.15.122:0/640035786 wait complete. 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 Processor -- start 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.453+0000 7fba23d40640 1 -- start start 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 0x7fba1c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c107b50 0x7fba1c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111ea0 con 0x7fba1c0696c0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c111d20 con 0x7fba1c068d00 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba23d40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c112020 con 0x7fba1c107b50 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50876/0 (socket says 172.21.15.122:50876) 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 -- 172.21.15.122:0/3261715688 learned_addr learned my addr 172.21.15.122:0/3261715688 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba222dd640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c107b50 0x7fba1c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba21adc640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 0x7fba1c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 -- 172.21.15.122:0/3261715688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c107b50 msgr2=0x7fba1c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c107b50 0x7fba1c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 -- 172.21.15.122:0/3261715688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 msgr2=0x7fba1c198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 0x7fba1c198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 -- 172.21.15.122:0/3261715688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c19d330 con 0x7fba1c0696c0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba212db640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e9f0 tx=0x7fba0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.454+0000 7fba12ffd640 1 -- 172.21.15.122:0/3261715688 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00cd80 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.455+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c19d620 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.455+0000 7fba12ffd640 1 -- 172.21.15.122:0/3261715688 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c010910 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.455+0000 7fba12ffd640 1 -- 172.21.15.122:0/3261715688 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0188f0 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.455+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a4eb0 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.458+0000 7fba12ffd640 1 -- 172.21.15.122:0/3261715688 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fba0c010ac0 con 0x7fba1c0696c0 2024-04-02T13:52:48.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.458+0000 7fba26a09740 1 Processor -- start 2024-04-02T13:52:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.458+0000 7fba26a09740 1 -- start start 2024-04-02T13:52:48.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bc0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc51b2a7b0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cc51ac9570 con 0x55cc51bd47f0 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba26a09740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc51b26760 con 0x55cc51bd3e40 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38580/0 (socket says 172.21.15.122:38580) 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 -- 172.21.15.122:0/1437964185 learned_addr learned my addr 172.21.15.122:0/1437964185 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba222dd640 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba12ffd640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 -- 172.21.15.122:0/1437964185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 msgr2=0x55cc51bd4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 -- 172.21.15.122:0/1437964185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 msgr2=0x55cc51bc0980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bc0980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.459+0000 7fba212db640 1 -- 172.21.15.122:0/1437964185 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5183e3f0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba222dd640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba12ffd640 1 -- 172.21.15.122:0/3261715688 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fba0c0d0fe0 con 0x7fba1c0696c0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba212db640 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0ce220 tx=0x7fba0c0c7c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad57ea301877513 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba07fff640 1 -- 172.21.15.122:0/1437964185 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c09c9b0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba222dd640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 0x7fba080a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba14004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba07fff640 1 -- 172.21.15.122:0/1437964185 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba0c064e00 con 0x55cc51bc0fa0 2024-04-02T13:52:48.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.460+0000 7fba07fff640 1 -- 172.21.15.122:0/1437964185 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c0d4b80 con 0x55cc51bc0fa0 2024-04-02T13:52:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 -- 172.21.15.122:0/1437964185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 msgr2=0x55cc51bc1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0ce220 tx=0x7fba0c0c7c40 comp rx=0 tx=0).stop 2024-04-02T13:52:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 -- 172.21.15.122:0/1437964185 shutdown_connections 2024-04-02T13:52:48.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bc1400 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bc0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 --2- 172.21.15.122:0/1437964185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 -- 172.21.15.122:0/1437964185 >> 172.21.15.122:0/1437964185 conn(0x55cc51bc3440 msgr2=0x55cc51bc3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 -- 172.21.15.122:0/1437964185 shutdown_connections 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.461+0000 7fba26a09740 1 -- 172.21.15.122:0/1437964185 wait complete. 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 -- start start 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 Processor -- start 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bd8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cc51bdb220 con 0x55cc51bc0fa0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cc51ae5e80 con 0x55cc51bd47f0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cc51bdb3a0 con 0x55cc51bd3e40 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.462+0000 7fba26a09740 10 client.0 _ll_register_callbacks cb 0x55cc51b21aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba212db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38596/0 (socket says 172.21.15.122:38596) 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 -- 172.21.15.122:0/1719345683 learned_addr learned my addr 172.21.15.122:0/1719345683 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba222dd640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bd8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 -- 172.21.15.122:0/1719345683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 msgr2=0x55cc51bd7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 -- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 msgr2=0x55cc51bd8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bd8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 -- 172.21.15.122:0/1719345683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc519087d0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba212db640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba21adc640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba1802c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c6ccbbae65498 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba057fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.463+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18042070 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fba1802ccc0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba0c064e00 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fba18050050 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 10 client.20580 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55cc51be7310 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18004ba0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fba180035f0 con 0x55cc51bc0fa0 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 10 client.20580 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 10 client.20580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 20 client.20580 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51beac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 4 client.20580 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.464+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fba0c064e00 con 0x55cc51be87d0 2024-04-02T13:52:48.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.467+0000 7fba212db640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51beac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.470+0000 7fba057fa640 10 client.20580 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:48.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.470+0000 7fba212db640 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51beac00 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c806efdd5ed647e server_cookie=99ec443fe53ae3ee in_seq=0 out_seq=0 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.476+0000 7fba057fa640 1 -- 172.21.15.122:0/1719345683 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fba0c064e00 con 0x55cc51be87d0 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.476+0000 7fba057fa640 10 client.20580 handle_command_reply: tid=0 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.476+0000 7fba26a09740 1 client.20580 shutdown 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 msgr2=0x55cc51bda790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fba18009900 tx=0x7fba1802c800 comp rx=0 tx=0).stop 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 shutdown_connections 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51beac00 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba057fa640 0 client.20580 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51bd8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 --2- 172.21.15.122:0/1719345683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bc0fa0 0x55cc51bda790 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 >> 172.21.15.122:0/1719345683 conn(0x55cc51bc3440 msgr2=0x55cc51bc2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 shutdown_connections 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 1 -- 172.21.15.122:0/1719345683 wait complete. 2024-04-02T13:52:48.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.477+0000 7fba26a09740 20 client.20580 trim_cache size 0 max 16384 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 Processor -- start 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 -- start start 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 0x55cc51be4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.496+0000 7fba26a09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1acd60 con 0x55cc51bd47f0 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba26a09740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ae960 con 0x55cc51bc0fa0 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba26a09740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ad430 con 0x55cc51bd3e40 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba21adc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 0x55cc51be4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38604/0 (socket says 172.21.15.122:38604) 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba21adc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 0x55cc51be4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:45178/0 (socket says 172.21.15.122:45178) 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 -- 172.21.15.122:0/2059605572 learned_addr learned my addr 172.21.15.122:0/2059605572 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 -- 172.21.15.122:0/2059605572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 msgr2=0x55cc51bd6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 -- 172.21.15.122:0/2059605572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 msgr2=0x55cc51be4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 0x55cc51be4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 -- 172.21.15.122:0/2059605572 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc51bd4590 con 0x55cc51bd47f0 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba222dd640 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fba1400fa30 tx=0x7fba14013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe97a7f332f16ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba077fe640 1 -- 172.21.15.122:0/2059605572 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1402e070 con 0x55cc51bd47f0 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba077fe640 1 -- 172.21.15.122:0/2059605572 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1401ee20 con 0x55cc51bd47f0 2024-04-02T13:52:48.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.497+0000 7fba077fe640 1 -- 172.21.15.122:0/2059605572 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1401c420 con 0x55cc51bd47f0 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.498+0000 7fba26a09740 1 -- 172.21.15.122:0/2059605572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 msgr2=0x55cc51bd69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.498+0000 7fba26a09740 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fba1400fa30 tx=0x7fba14013030 comp rx=0 tx=0).stop 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 -- 172.21.15.122:0/2059605572 shutdown_connections 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd47f0 0x55cc51bd69a0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bd3e40 0x55cc51bd6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 --2- 172.21.15.122:0/2059605572 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bc0fa0 0x55cc51be4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 -- 172.21.15.122:0/2059605572 >> 172.21.15.122:0/2059605572 conn(0x55cc51bc3440 msgr2=0x55cc51bd1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 -- 172.21.15.122:0/2059605572 shutdown_connections 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.499+0000 7fba26a09740 1 -- 172.21.15.122:0/2059605572 wait complete. 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 -- start start 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 Processor -- start 2024-04-02T13:52:48.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bc0fa0 0x55cc51be0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51befb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1ae290 con 0x55cc51bd3e40 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1adb00 con 0x55cc51bd47f0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c1aad60 con 0x55cc51bc0fa0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba26a09740 10 client.0 _ll_register_callbacks cb 0x7fba140093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba212db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38614/0 (socket says 172.21.15.122:38614) 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba212db640 1 -- 172.21.15.122:0/3299626773 learned_addr learned my addr 172.21.15.122:0/3299626773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba21adc640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bc0fa0 0x55cc51be0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.500+0000 7fba222dd640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51befb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 -- 172.21.15.122:0/3299626773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bc0fa0 msgr2=0x55cc51be0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bc0fa0 0x55cc51be0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 -- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 msgr2=0x55cc51befb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51befb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 -- 172.21.15.122:0/3299626773 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc51bf01e0 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba212db640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0a5270 tx=0x7fba0c098e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9ce90d3a9e5af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba057fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c09de60 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7fba0c0d4b80 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba1401ee20 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fba0c09c540 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba26a09740 10 client.20586 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55cc51c4ec90 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.501+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba1401ee20 con 0x55cc51bd3e40 2024-04-02T13:52:48.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fba0c099250 con 0x55cc51bd3e40 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 10 client.20586 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 10 client.20586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 20 client.20586 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51c510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 4 client.20586 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cc51b37cf0 con 0x55cc51be87d0 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba21adc640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51c510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba057fa640 10 client.20586 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:48.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.502+0000 7fba21adc640 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51c510f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd002fafba7abbc server_cookie=7b1ca0eb3ca1d3c in_seq=0 out_seq=0 2024-04-02T13:52:48.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba057fa640 1 -- 172.21.15.122:0/3299626773 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cc51b37cf0 con 0x55cc51be87d0 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba057fa640 10 client.20586 handle_command_reply: tid=0 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 client.20586 shutdown 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 msgr2=0x55cc51bef5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0a5270 tx=0x7fba0c098e80 comp rx=0 tx=0).stop 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 shutdown_connections 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55cc51be87d0 0x55cc51c510f0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cc51bd47f0 0x55cc51befb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cc51bd3e40 0x55cc51bef5e0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 --2- 172.21.15.122:0/3299626773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cc51bc0fa0 0x55cc51be0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba057fa640 0 client.20586 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 >> 172.21.15.122:0/3299626773 conn(0x55cc51bc3440 msgr2=0x55cc51bd6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 shutdown_connections 2024-04-02T13:52:48.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.505+0000 7fba26a09740 1 -- 172.21.15.122:0/3299626773 wait complete. 2024-04-02T13:52:48.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba26a09740 20 client.20586 trim_cache size 0 max 16384 2024-04-02T13:52:48.511 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 msgr2=0x7fba080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 0x7fba080a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fba14009fd0 tx=0x7fba14004a40 comp rx=0 tx=0).stop 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 msgr2=0x7fba1c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00e9f0 tx=0x7fba0c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 shutdown_connections 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fba080a3d20 0x7fba080a61e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba1c107b50 0x7fba1c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba1c0696c0 0x7fba1c198880 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 --2- 172.21.15.122:0/3261715688 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba1c068d00 0x7fba1c198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.506+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 >> 172.21.15.122:0/3261715688 conn(0x7fba1c077b00 msgr2=0x7fba1c0fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.507+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 shutdown_connections 2024-04-02T13:52:48.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:48.507+0000 7fba23d40640 1 -- 172.21.15.122:0/3261715688 wait complete. 2024-04-02T13:52:49.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 136816 29.72 MB/sec execute 129 sec latency 74.921 ms 2024-04-02T13:52:49.522 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:52:49.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:52:49.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:52:49.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- 172.21.15.122:0/4113598317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08102460 msgr2=0x7eff0810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 --2- 172.21.15.122:0/4113598317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08102460 0x7eff0810e970 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7eff00009cd0 tx=0x7eff0002c0f0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- 172.21.15.122:0/4113598317 shutdown_connections 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 --2- 172.21.15.122:0/4113598317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08102460 0x7eff0810e970 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 --2- 172.21.15.122:0/4113598317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08101aa0 0x7eff08101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 --2- 172.21.15.122:0/4113598317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- 172.21.15.122:0/4113598317 >> 172.21.15.122:0/4113598317 conn(0x7eff080fd530 msgr2=0x7eff080ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- 172.21.15.122:0/4113598317 shutdown_connections 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- 172.21.15.122:0/4113598317 wait complete. 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 Processor -- start 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.658+0000 7eff10772640 1 -- start start 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08102460 0x7eff08074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff08111e90 con 0x7eff08101aa0 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff08111d10 con 0x7eff08102460 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff08112010 con 0x7eff08107ab0 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38636/0 (socket says 172.21.15.122:38636) 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 -- 172.21.15.122:0/829085212 learned_addr learned my addr 172.21.15.122:0/829085212 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08102460 0x7eff08074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 -- 172.21.15.122:0/829085212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 msgr2=0x7eff08074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 -- 172.21.15.122:0/829085212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08102460 msgr2=0x7eff08074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08102460 0x7eff08074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 -- 172.21.15.122:0/829085212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff081a6810 con 0x7eff08101aa0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff0e50e640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7eff0400d950 tx=0x7eff0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7efeff7fe640 1 -- 172.21.15.122:0/829085212 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff04017070 con 0x7eff08101aa0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.659+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff081a6b00 con 0x7eff08101aa0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.660+0000 7efeff7fe640 1 -- 172.21.15.122:0/829085212 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff04009c30 con 0x7eff08101aa0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.660+0000 7efeff7fe640 1 -- 172.21.15.122:0/829085212 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0400f6b0 con 0x7eff08101aa0 2024-04-02T13:52:49.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.660+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a70a0 con 0x7eff08101aa0 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 Processor -- start 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 -- start start 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 0x55bebf2ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf300e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bebf248770 con 0x55bebf2fea40 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bebf1f0960 con 0x55bebf2ed0a0 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.663+0000 7eff13432740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bebf2448c0 con 0x55bebf2ec780 2024-04-02T13:52:49.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0dd0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf300e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0dd0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf300e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38648/0 (socket says 172.21.15.122:38648) 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0dd0d640 1 -- 172.21.15.122:0/2620796858 learned_addr learned my addr 172.21.15.122:0/2620796858 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 0x55bebf2ecb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7efeff7fe640 1 -- 172.21.15.122:0/829085212 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7eff040109d0 con 0x7eff08101aa0 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0e50e640 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0ed0f640 1 -- 172.21.15.122:0/2620796858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ed0a0 msgr2=0x55bebf2e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0ed0f640 1 -- 172.21.15.122:0/2620796858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 msgr2=0x55bebf300e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.664+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf300e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.665+0000 7eff0ed0f640 1 -- 172.21.15.122:0/2620796858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebf0247e0 con 0x55bebf2ec780 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.665+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 0x55bebf2ecb60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff00037040 tx=0x7eff000323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9caecdbfbdd325c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.665+0000 7efeff7fe640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 0x7efef40a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.666+0000 7efefcff9640 1 -- 172.21.15.122:0/2620796858 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00039050 con 0x55bebf2ec780 2024-04-02T13:52:49.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.666+0000 7efefcff9640 1 -- 172.21.15.122:0/2620796858 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff00045070 con 0x55bebf2ec780 2024-04-02T13:52:49.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.666+0000 7efefcff9640 1 -- 172.21.15.122:0/2620796858 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003e60 con 0x55bebf2ec780 2024-04-02T13:52:49.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.666+0000 7eff0e50e640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 0x7efef40a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.666+0000 7efeff7fe640 1 -- 172.21.15.122:0/829085212 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7eff040d07a0 con 0x7eff08101aa0 2024-04-02T13:52:49.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 -- 172.21.15.122:0/2620796858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 msgr2=0x55bebf2ecb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff0e50e640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 0x7efef40a6290 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff040d67e0 tx=0x7eff040d6770 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 0x55bebf2ecb60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff00037040 tx=0x7eff000323c0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 -- 172.21.15.122:0/2620796858 shutdown_connections 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf300e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 --2- 172.21.15.122:0/2620796858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ec780 0x55bebf2ecb60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 -- 172.21.15.122:0/2620796858 >> 172.21.15.122:0/2620796858 conn(0x55bebf2e7200 msgr2=0x55bebf2e75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 -- 172.21.15.122:0/2620796858 shutdown_connections 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.667+0000 7eff13432740 1 -- 172.21.15.122:0/2620796858 wait complete. 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 -- start start 2024-04-02T13:52:49.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 Processor -- start 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff0dd0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff0dd0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38650/0 (socket says 172.21.15.122:38650) 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff0dd0d640 1 -- 172.21.15.122:0/1638935504 learned_addr learned my addr 172.21.15.122:0/1638935504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bebf301610 con 0x55bebf2ed0a0 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bebf203d20 con 0x55bebf2ec780 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.668+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bebf301790 con 0x55bebf2fea40 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff13432740 10 client.0 _ll_register_callbacks cb 0x55bebf1d0a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 -- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 msgr2=0x55bebf2e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 -- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 msgr2=0x55bebf2e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebf084f70 con 0x55bebf2ed0a0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7efef8012770 tx=0x7efef8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8c63cd83a8657 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7efef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef8004050 con 0x55bebf2ed0a0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.669+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7efef80041f0 con 0x55bebf2ed0a0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00045070 con 0x55bebf2ed0a0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7efef80043f0 con 0x55bebf2ed0a0 2024-04-02T13:52:49.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 10 client.20598 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55bebef1c170 con 0x55bebf2ed0a0 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efef800fe00 con 0x55bebf2ed0a0 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7efef8010430 con 0x55bebf2ed0a0 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 10 client.20598 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 10 client.20598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 20 client.20598 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf3111b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 4 client.20598 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7eff00045070 con 0x55bebf30edc0 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.670+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf3111b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.671+0000 7efef1ffb640 10 client.20598 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.671+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf3111b0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daacdcb82aeda7a1 server_cookie=8fffbcd07d8da505 in_seq=0 out_seq=0 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7efef1ffb640 1 -- 172.21.15.122:0/1638935504 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7eff00045070 con 0x55bebf30edc0 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7efef1ffb640 10 client.20598 handle_command_reply: tid=0 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 client.20598 shutdown 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 msgr2=0x55bebf2e3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7efef8012770 tx=0x7efef8012c30 comp rx=0 tx=0).stop 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 shutdown_connections 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf3111b0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7efef1ffb640 0 client.20598 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf2e3e20 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 --2- 172.21.15.122:0/1638935504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 >> 172.21.15.122:0/1638935504 conn(0x55bebf2e7200 msgr2=0x55bebf2e8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 shutdown_connections 2024-04-02T13:52:49.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 1 -- 172.21.15.122:0/1638935504 wait complete. 2024-04-02T13:52:49.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.672+0000 7eff13432740 20 client.20598 trim_cache size 0 max 16384 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 Processor -- start 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 -- start start 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf30a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff081ace40 con 0x55bebf2ed0a0 2024-04-02T13:52:49.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff081aea40 con 0x55bebf2ec780 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff13432740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff081ad510 con 0x55bebf2fea40 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff0ed0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff0ed0f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:45238/0 (socket says 172.21.15.122:45238) 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff0ed0f640 1 -- 172.21.15.122:0/526435487 learned_addr learned my addr 172.21.15.122:0/526435487 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.693+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf30a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 -- 172.21.15.122:0/526435487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 msgr2=0x55bebf2e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 -- 172.21.15.122:0/526435487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 msgr2=0x55bebf30a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf30a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 -- 172.21.15.122:0/526435487 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebf301760 con 0x55bebf2ec780 2024-04-02T13:52:49.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff00009a60 tx=0x7eff000095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b595b6b819cbcde server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7efef3fff640 1 -- 172.21.15.122:0/526435487 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003970 con 0x55bebf2ec780 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7efef3fff640 1 -- 172.21.15.122:0/526435487 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff00045040 con 0x55bebf2ec780 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.694+0000 7efef3fff640 1 -- 172.21.15.122:0/526435487 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00040b60 con 0x55bebf2ec780 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 -- 172.21.15.122:0/526435487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 msgr2=0x55bebf2ddab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7eff00009a60 tx=0x7eff000095a0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 -- 172.21.15.122:0/526435487 shutdown_connections 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2fea40 0x55bebf2e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2ed0a0 0x55bebf30a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 --2- 172.21.15.122:0/526435487 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf2ddab0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 -- 172.21.15.122:0/526435487 >> 172.21.15.122:0/526435487 conn(0x55bebf2e7200 msgr2=0x55bebf2e8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 -- 172.21.15.122:0/526435487 shutdown_connections 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.695+0000 7eff13432740 1 -- 172.21.15.122:0/526435487 wait complete. 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 -- start start 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 Processor -- start 2024-04-02T13:52:49.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf306f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ed0a0 0x55bebf3032d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff081ae370 con 0x55bebf2fea40 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7eff081adbe0 con 0x55bebf2ec780 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff081aae60 con 0x55bebf2ed0a0 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff13432740 10 client.0 _ll_register_callbacks cb 0x7efef8009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff0ed0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff0ed0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38664/0 (socket says 172.21.15.122:38664) 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff0ed0f640 1 -- 172.21.15.122:0/903840955 learned_addr learned my addr 172.21.15.122:0/903840955 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf306f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.696+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ed0a0 0x55bebf3032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 -- 172.21.15.122:0/903840955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ed0a0 msgr2=0x55bebf3032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ed0a0 0x55bebf3032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 -- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 msgr2=0x55bebf306f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf306f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 -- 172.21.15.122:0/903840955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebf304110 con 0x55bebf2fea40 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7eff0ed0f640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7eff00040680 tx=0x7eff00040740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f89dfddcb90bd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.697+0000 7efef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003970 con 0x55bebf2fea40 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7eff00003b10 con 0x55bebf2fea40 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00045040 con 0x55bebf2fea40 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7eff000408e0 con 0x55bebf2fea40 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7eff13432740 10 client.20604 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:49.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.698+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55bebf376230 con 0x55bebf2fea40 2024-04-02T13:52:49.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff0003a040 con 0x55bebf2fea40 2024-04-02T13:52:49.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7eff00034bd0 con 0x55bebf2fea40 2024-04-02T13:52:49.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 10 client.20604 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 10 client.20604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 20 client.20604 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf378410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 4 client.20604 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7eff00045040 con 0x55bebf30edc0 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.699+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf378410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.700+0000 7efef1ffb640 10 client.20604 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.700+0000 7eff0dd0d640 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf378410 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d698a126a298b53d server_cookie=caffef61257d35c1 in_seq=0 out_seq=0 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7efef1ffb640 1 -- 172.21.15.122:0/903840955 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7eff00045040 con 0x55bebf30edc0 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7efef1ffb640 10 client.20604 handle_command_reply: tid=0 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 client.20604 shutdown 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 msgr2=0x55bebf3038e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7eff00040680 tx=0x7eff00040740 comp rx=0 tx=0).stop 2024-04-02T13:52:49.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 shutdown_connections 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bebf30edc0 0x55bebf378410 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bebf2fea40 0x55bebf3038e0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bebf2ed0a0 0x55bebf3032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 --2- 172.21.15.122:0/903840955 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bebf2ec780 0x55bebf306f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 >> 172.21.15.122:0/903840955 conn(0x55bebf2e7200 msgr2=0x55bebf2dd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7efef1ffb640 0 client.20604 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 shutdown_connections 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.701+0000 7eff13432740 1 -- 172.21.15.122:0/903840955 wait complete. 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.702+0000 7eff13432740 20 client.20604 trim_cache size 0 max 16384 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:49.707 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:52:49.708 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "b20de7f7-c050-4dd4-b8b5-9f683b6a1621", 2024-04-02T13:52:49.708 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:52:49.708 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.702+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 msgr2=0x7efef40a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.702+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 0x7efef40a6290 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7eff040d67e0 tx=0x7eff040d6770 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.702+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 msgr2=0x7eff08073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.702+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7eff0400d950 tx=0x7eff0400de10 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 shutdown_connections 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7efef40a3dd0 0x7efef40a6290 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff08107ab0 0x7eff08074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7eff08102460 0x7eff08074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 --2- 172.21.15.122:0/829085212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff08101aa0 0x7eff08073e00 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 >> 172.21.15.122:0/829085212 conn(0x7eff080fd530 msgr2=0x7eff080fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 shutdown_connections 2024-04-02T13:52:49.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.703+0000 7eff10772640 1 -- 172.21.15.122:0/829085212 wait complete. 2024-04-02T13:52:49.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:52:49.720 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:52:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:49 smithi122 ceph-mon[32114]: pgmap v554: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-04-02T13:52:49.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 -- 172.21.15.122:0/3718450425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18100ce0 msgr2=0x7f4f18101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3718450425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18100ce0 0x7f4f18101160 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009900 tx=0x7f4f0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 -- 172.21.15.122:0/3718450425 shutdown_connections 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3718450425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 0x7f4f1810d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3718450425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18100ce0 0x7f4f18101160 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3718450425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18106a30 0x7f4f18106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 -- 172.21.15.122:0/3718450425 >> 172.21.15.122:0/3718450425 conn(0x7f4f180779e0 msgr2=0x7f4f18077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 -- 172.21.15.122:0/3718450425 shutdown_connections 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.857+0000 7f4f1da81640 1 -- 172.21.15.122:0/3718450425 wait complete. 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.858+0000 7f4f1da81640 1 Processor -- start 2024-04-02T13:52:49.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.858+0000 7f4f1da81640 1 -- start start 2024-04-02T13:52:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.858+0000 7f4f1da81640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18100ce0 0x7f4f18199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.858+0000 7f4f1da81640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 0x7f4f18199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.858+0000 7f4f1da81640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f1da81640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110dd0 con 0x7f4f18106a30 2024-04-02T13:52:49.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f1da81640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110c50 con 0x7f4f18100ce0 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f1da81640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f18110f50 con 0x7f4f181016a0 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38676/0 (socket says 172.21.15.122:38676) 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 -- 172.21.15.122:0/3266144236 learned_addr learned my addr 172.21.15.122:0/3266144236 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f177fe640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18100ce0 0x7f4f18199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 0x7f4f18199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 -- 172.21.15.122:0/3266144236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 msgr2=0x7f4f18199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 0x7f4f18199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 -- 172.21.15.122:0/3266144236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18100ce0 msgr2=0x7f4f18199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18100ce0 0x7f4f18199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 -- 172.21.15.122:0/3266144236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f181a46f0 con 0x7f4f18106a30 2024-04-02T13:52:49.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f17fff640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000eb10 tx=0x7f4f1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.859+0000 7f4f14ff9640 1 -- 172.21.15.122:0/3266144236 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10004b20 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.860+0000 7f4f14ff9640 1 -- 172.21.15.122:0/3266144236 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f10004cc0 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.860+0000 7f4f14ff9640 1 -- 172.21.15.122:0/3266144236 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10010760 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.860+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181a48c0 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.860+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f181a4e30 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.862+0000 7f4f14ff9640 1 -- 172.21.15.122:0/3266144236 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4f10005020 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.862+0000 7f4f20662740 1 Processor -- start 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f14ff9640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 0x7f4f000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 -- start start 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f14ff9640 1 -- 172.21.15.122:0/3266144236 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f4f100d1220 con 0x7f4f18106a30 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 0x7f4f000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bc8d0 0x557a699bccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557a69920830 con 0x557a699c5dd0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557a698c8820 con 0x557a699bd1f0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f20662740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557a6991c7e0 con 0x557a699bc8d0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38684/0 (socket says 172.21.15.122:38684) 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f17fff640 1 -- 172.21.15.122:0/2488117168 learned_addr learned my addr 172.21.15.122:0/2488117168 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f177fe640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bc8d0 0x557a699bccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.863+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 -- 172.21.15.122:0/2488117168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bc8d0 msgr2=0x557a699bccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bc8d0 0x557a699bccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 -- 172.21.15.122:0/2488117168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 msgr2=0x557a699c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 -- 172.21.15.122:0/2488117168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a696fe7d0 con 0x557a699c5dd0 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 0x7f4f000a6260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000c00 tx=0x7f4f0c02c5d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4f17fff640 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4f100a5510 tx=0x7f4f1009c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7938ccf91cb6af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4efdffb640 1 -- 172.21.15.122:0/2488117168 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009cdd0 con 0x557a699c5dd0 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4efdffb640 1 -- 172.21.15.122:0/2488117168 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f100a1180 con 0x557a699c5dd0 2024-04-02T13:52:49.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.864+0000 7f4efdffb640 1 -- 172.21.15.122:0/2488117168 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10005310 con 0x557a699c5dd0 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 -- 172.21.15.122:0/2488117168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 msgr2=0x557a699c81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f4f100a5510 tx=0x7f4f1009c910 comp rx=0 tx=0).stop 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 -- 172.21.15.122:0/2488117168 shutdown_connections 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c81c0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 --2- 172.21.15.122:0/2488117168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bc8d0 0x557a699bccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.865+0000 7f4f20662740 1 -- 172.21.15.122:0/2488117168 >> 172.21.15.122:0/2488117168 conn(0x557a699b7000 msgr2=0x557a699b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.866+0000 7f4f20662740 1 -- 172.21.15.122:0/2488117168 shutdown_connections 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.866+0000 7f4f20662740 1 -- 172.21.15.122:0/2488117168 wait complete. 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.866+0000 7f4f20662740 1 -- start start 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.866+0000 7f4f20662740 1 Processor -- start 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 0x557a699c29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557a699ccfd0 con 0x557a699c5dd0 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557a698dbd30 con 0x557a699bc8d0 2024-04-02T13:52:49.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557a699cd150 con 0x557a699bd1f0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f20662740 10 client.0 _ll_register_callbacks cb 0x557a698db720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38688/0 (socket says 172.21.15.122:38688) 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 -- 172.21.15.122:0/199094327 learned_addr learned my addr 172.21.15.122:0/199094327 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 0x557a699c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 -- 172.21.15.122:0/199094327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 msgr2=0x557a699c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 0x557a699c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 -- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 msgr2=0x557a699c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f17fff640 1 -- 172.21.15.122:0/199094327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a696343f0 con 0x557a699c5dd0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.867+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 0x557a699c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4f17fff640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4f100bbfd0 tx=0x7f4f100d11b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf61225d46e879f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4ef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1009b0d0 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f4f10005310 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f100a1180 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4f10005510 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4f20662740 10 client.20616 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.868+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557a695f40f0 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f100a1180 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f4f1009bd90 con 0x557a699c5dd0 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 10 client.20616 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 10 client.20616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 20 client.20616 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699dc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 4 client.20616 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:52:49.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a6992dd70 con 0x557a699d9f80 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.869+0000 7f4f177fe640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.870+0000 7f4ef6ffd640 10 client.20616 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.870+0000 7f4f177fe640 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699dc370 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4012a908148f5e server_cookie=c317b505867f7cb2 in_seq=0 out_seq=0 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/199094327 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a6992dd70 con 0x557a699d9f80 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4ef6ffd640 10 client.20616 handle_command_reply: tid=0 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 client.20616 shutdown 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 msgr2=0x557a699c2ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f4f100bbfd0 tx=0x7f4f100d11b0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 shutdown_connections 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699dc370 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699c5dd0 0x557a699c2ee0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4ef6ffd640 0 client.20616 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699bd1f0 0x557a699c29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 --2- 172.21.15.122:0/199094327 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 >> 172.21.15.122:0/199094327 conn(0x557a699b7000 msgr2=0x557a699b8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 shutdown_connections 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.873+0000 7f4f20662740 1 -- 172.21.15.122:0/199094327 wait complete. 2024-04-02T13:52:49.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.874+0000 7f4f20662740 20 client.20616 trim_cache size 0 max 16384 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 Processor -- start 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 -- start start 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699bd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181acf60 con 0x557a699bc8d0 2024-04-02T13:52:49.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181aeb60 con 0x557a699bd1f0 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f20662740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181ad630 con 0x557a699c5dd0 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38704/0 (socket says 172.21.15.122:38704) 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 -- 172.21.15.122:0/2838998501 learned_addr learned my addr 172.21.15.122:0/2838998501 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f17fff640 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699bd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 -- 172.21.15.122:0/2838998501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 msgr2=0x557a699c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 -- 172.21.15.122:0/2838998501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 msgr2=0x557a699bd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699bd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.897+0000 7f4f177fe640 1 -- 172.21.15.122:0/2838998501 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a699bd020 con 0x557a699bc8d0 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.898+0000 7f4f177fe640 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800ba90 tx=0x7f4f0800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269d42eda2b6511 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.898+0000 7f4efd7fa640 1 -- 172.21.15.122:0/2838998501 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08003d10 con 0x557a699bc8d0 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.898+0000 7f4efd7fa640 1 -- 172.21.15.122:0/2838998501 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f08024020 con 0x557a699bc8d0 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.898+0000 7f4efd7fa640 1 -- 172.21.15.122:0/2838998501 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08011920 con 0x557a699bc8d0 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- 172.21.15.122:0/2838998501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 msgr2=0x557a699d5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800ba90 tx=0x7f4f0800b390 comp rx=0 tx=0).stop 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- 172.21.15.122:0/2838998501 shutdown_connections 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bd1f0 0x557a699bd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 --2- 172.21.15.122:0/2838998501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bc8d0 0x557a699d5e90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- 172.21.15.122:0/2838998501 >> 172.21.15.122:0/2838998501 conn(0x557a699b7000 msgr2=0x557a699b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- 172.21.15.122:0/2838998501 shutdown_connections 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- 172.21.15.122:0/2838998501 wait complete. 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 -- start start 2024-04-02T13:52:49.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.899+0000 7f4f20662740 1 Processor -- start 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699bf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181ae490 con 0x557a699bd1f0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181add00 con 0x557a699bc8d0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f181aaf60 con 0x557a699c5dd0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f20662740 10 client.0 _ll_register_callbacks cb 0x7f4f10099120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f177fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38706/0 (socket says 172.21.15.122:38706) 2024-04-02T13:52:49.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 -- 172.21.15.122:0/3295571205 learned_addr learned my addr 172.21.15.122:0/3295571205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f17fff640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 -- 172.21.15.122:0/3295571205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 msgr2=0x557a699bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 -- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 msgr2=0x557a699d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.900+0000 7f4f16ffd640 1 -- 172.21.15.122:0/3295571205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a699bfca0 con 0x557a699bd1f0 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4f16ffd640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c035000 tx=0x7f4f0c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4692f9999c6a8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4ef6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c048070 con 0x557a699bd1f0 2024-04-02T13:52:49.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f08024020 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9525+0+0 (secure 0 0 0) 0x7f4f0c004ec0 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c05c050 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4f20662740 10 client.20622 fetch_fsmap learned FSMap version 73 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.901+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557a699c9580 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f08024020 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f4f0c003ba0 con 0x557a699bd1f0 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 10 client.20622 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 10 client.20622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 20 client.20622 populate_metadata read hostname 'smithi122' 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 4 client.20622 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.902+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a699be5c0 con 0x557a699d9f80 2024-04-02T13:52:49.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4f177fe640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4ef6ffd640 10 client.20622 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4f177fe640 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699cbd40 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a830911d785841c server_cookie=baa6bf0147f3c795 in_seq=0 out_seq=0 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4ef6ffd640 1 -- 172.21.15.122:0/3295571205 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557a699be5c0 con 0x557a699d9f80 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4ef6ffd640 10 client.20622 handle_command_reply: tid=0 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.903+0000 7f4f20662740 1 client.20622 shutdown 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 msgr2=0x557a699bf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c035000 tx=0x7f4f0c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 shutdown_connections 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557a699d9f80 0x557a699cbd40 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557a699c5dd0 0x557a699bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557a699bd1f0 0x557a699bf1f0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 --2- 172.21.15.122:0/3295571205 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557a699bc8d0 0x557a699d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4ef6ffd640 0 client.20622 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 >> 172.21.15.122:0/3295571205 conn(0x557a699b7000 msgr2=0x557a699c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 shutdown_connections 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 1 -- 172.21.15.122:0/3295571205 wait complete. 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.904+0000 7f4f20662740 20 client.20622 trim_cache size 0 max 16384 2024-04-02T13:52:49.913 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "b20de7f7-c050-4dd4-b8b5-9f683b6a1621": { 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "b20de7f7-c050-4dd4-b8b5-9f683b6a1621", 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:52:49.914 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:52:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 msgr2=0x7f4f000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 0x7f4f000a6260 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c000c00 tx=0x7f4f0c02c5d0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 msgr2=0x7f4f181a41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:52:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000eb10 tx=0x7f4f1000c2d0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 shutdown_connections 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f000a3da0 0x7f4f000a6260 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.905+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f18106a30 0x7f4f181a41b0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.906+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f181016a0 0x7f4f18199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.906+0000 7f4f1da81640 1 --2- 172.21.15.122:0/3266144236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f18100ce0 0x7f4f18199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.906+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 >> 172.21.15.122:0/3266144236 conn(0x7f4f180779e0 msgr2=0x7f4f180ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.906+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 shutdown_connections 2024-04-02T13:52:49.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:52:49.906+0000 7f4f1da81640 1 -- 172.21.15.122:0/3266144236 wait complete. 2024-04-02T13:52:49.925 INFO:tasks.cephfs.filesystem:scrub status for tag:b20de7f7-c050-4dd4-b8b5-9f683b6a1621 - {'path': '/', 'tag': 'b20de7f7-c050-4dd4-b8b5-9f683b6a1621', 'options': 'recursive,force'} 2024-04-02T13:52:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:49 smithi187 ceph-mon[36064]: pgmap v554: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-04-02T13:52:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:49 smithi188 ceph-mon[36117]: pgmap v554: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 719 op/s 2024-04-02T13:52:50.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 137428 29.73 MB/sec execute 130 sec latency 105.438 ms 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:52:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:50.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:50 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:52:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:50 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:52:51.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:50 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:51.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 138091 29.80 MB/sec execute 131 sec latency 80.648 ms 2024-04-02T13:52:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:51 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:51 smithi188 ceph-mon[36117]: pgmap v555: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-04-02T13:52:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:51 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:51 smithi122 ceph-mon[32114]: pgmap v555: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-04-02T13:52:52.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:51 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:52:52.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:51 smithi187 ceph-mon[36064]: pgmap v555: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 513 op/s 2024-04-02T13:52:52.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 138665 29.79 MB/sec execute 132 sec latency 104.899 ms 2024-04-02T13:52:53.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 139363 29.88 MB/sec execute 133 sec latency 95.378 ms 2024-04-02T13:52:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:53 smithi188 ceph-mon[36117]: pgmap v556: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 731 op/s 2024-04-02T13:52:54.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:53 smithi122 ceph-mon[32114]: pgmap v556: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 731 op/s 2024-04-02T13:52:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:53 smithi187 ceph-mon[36064]: pgmap v556: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 731 op/s 2024-04-02T13:52:54.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 140006 29.90 MB/sec execute 134 sec latency 69.000 ms 2024-04-02T13:52:55.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 140662 29.95 MB/sec execute 135 sec latency 71.592 ms 2024-04-02T13:52:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:55 smithi188 ceph-mon[36117]: pgmap v557: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 580 op/s 2024-04-02T13:52:56.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:55 smithi122 ceph-mon[32114]: pgmap v557: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 580 op/s 2024-04-02T13:52:56.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:55 smithi187 ceph-mon[36064]: pgmap v557: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 580 op/s 2024-04-02T13:52:56.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 141303 29.97 MB/sec execute 136 sec latency 119.225 ms 2024-04-02T13:52:57.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 141908 29.95 MB/sec execute 137 sec latency 94.701 ms 2024-04-02T13:52:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:57 smithi188 ceph-mon[36117]: pgmap v558: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 602 op/s 2024-04-02T13:52:58.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:57 smithi122 ceph-mon[32114]: pgmap v558: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 602 op/s 2024-04-02T13:52:58.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:57 smithi187 ceph-mon[36064]: pgmap v558: 129 pgs: 129 active+clean; 1.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 602 op/s 2024-04-02T13:52:58.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 142565 30.04 MB/sec execute 138 sec latency 77.341 ms 2024-04-02T13:52:59.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143184 30.06 MB/sec execute 139 sec latency 143.282 ms 2024-04-02T13:53:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:52:59 smithi188 ceph-mon[36117]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 698 op/s 2024-04-02T13:53:00.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:52:59 smithi122 ceph-mon[32114]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 698 op/s 2024-04-02T13:53:00.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:52:59 smithi187 ceph-mon[36064]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 698 op/s 2024-04-02T13:53:00.330 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143682 30.06 MB/sec execute 140 sec latency 190.433 ms 2024-04-02T13:53:01.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143682 29.84 MB/sec execute 141 sec latency 1190.514 ms 2024-04-02T13:53:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:01 smithi188 ceph-mon[36117]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 475 op/s 2024-04-02T13:53:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:01 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:02.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:01 smithi122 ceph-mon[32114]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 475 op/s 2024-04-02T13:53:02.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:02.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:01 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:02.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:01 smithi187 ceph-mon[36064]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.8 MiB/s wr, 475 op/s 2024-04-02T13:53:02.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:02.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:01 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:02.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143682 29.63 MB/sec execute 142 sec latency 2190.602 ms 2024-04-02T13:53:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:03.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:02 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:03.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:03.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:02 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:03.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:03.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:02 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:03.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143779 29.45 MB/sec execute 143 sec latency 2598.734 ms 2024-04-02T13:53:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:03 smithi188 ceph-mon[36117]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-04-02T13:53:04.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:03 smithi122 ceph-mon[32114]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-04-02T13:53:04.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:03 smithi187 ceph-mon[36064]: pgmap v561: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-04-02T13:53:04.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143779 29.24 MB/sec execute 144 sec latency 1428.849 ms 2024-04-02T13:53:05.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143779 29.04 MB/sec execute 145 sec latency 2428.944 ms 2024-04-02T13:53:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:05 smithi188 ceph-mon[36117]: pgmap v562: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-04-02T13:53:06.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:05 smithi122 ceph-mon[32114]: pgmap v562: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-04-02T13:53:06.238 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:05 smithi187 ceph-mon[36064]: pgmap v562: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.9 MiB/s wr, 427 op/s 2024-04-02T13:53:06.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 143779 28.84 MB/sec execute 146 sec latency 3429.053 ms 2024-04-02T13:53:07.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144016 28.72 MB/sec execute 147 sec latency 3901.065 ms 2024-04-02T13:53:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:07 smithi122 ceph-mon[32114]: pgmap v563: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.5 MiB/s wr, 396 op/s 2024-04-02T13:53:08.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:07 smithi187 ceph-mon[36064]: pgmap v563: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.5 MiB/s wr, 396 op/s 2024-04-02T13:53:08.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:07 smithi188 ceph-mon[36117]: pgmap v563: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 5.5 MiB/s wr, 396 op/s 2024-04-02T13:53:08.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144209 28.56 MB/sec execute 148 sec latency 306.081 ms 2024-04-02T13:53:09.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144258 28.39 MB/sec execute 149 sec latency 344.794 ms 2024-04-02T13:53:10.069 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:09 smithi188 ceph-mon[36117]: pgmap v564: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.4 MiB/s wr, 398 op/s 2024-04-02T13:53:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:09 smithi122 ceph-mon[32114]: pgmap v564: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.4 MiB/s wr, 398 op/s 2024-04-02T13:53:10.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:09 smithi187 ceph-mon[36064]: pgmap v564: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.4 MiB/s wr, 398 op/s 2024-04-02T13:53:10.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144385 28.28 MB/sec execute 150 sec latency 341.309 ms 2024-04-02T13:53:11.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 144788 28.26 MB/sec execute 151 sec latency 255.803 ms 2024-04-02T13:53:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:11 smithi122 ceph-mon[32114]: pgmap v565: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-04-02T13:53:12.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:11 smithi187 ceph-mon[36064]: pgmap v565: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-04-02T13:53:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:11 smithi188 ceph-mon[36117]: pgmap v565: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 218 op/s 2024-04-02T13:53:12.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 145308 28.22 MB/sec execute 152 sec latency 255.906 ms 2024-04-02T13:53:13.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:12 smithi122 ceph-mon[32114]: osdmap e118: 12 total, 12 up, 12 in 2024-04-02T13:53:13.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:12 smithi187 ceph-mon[36064]: osdmap e118: 12 total, 12 up, 12 in 2024-04-02T13:53:13.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:12 smithi188 ceph-mon[36117]: osdmap e118: 12 total, 12 up, 12 in 2024-04-02T13:53:13.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 145966 28.25 MB/sec execute 153 sec latency 63.400 ms 2024-04-02T13:53:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:13 smithi122 ceph-mon[32114]: pgmap v567: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-04-02T13:53:14.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:13 smithi187 ceph-mon[36064]: pgmap v567: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-04-02T13:53:14.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:13 smithi188 ceph-mon[36117]: pgmap v567: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 326 op/s 2024-04-02T13:53:14.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 146563 28.28 MB/sec execute 154 sec latency 156.976 ms 2024-04-02T13:53:15.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147244 28.40 MB/sec execute 155 sec latency 99.145 ms 2024-04-02T13:53:16.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:15 smithi122 ceph-mon[32114]: pgmap v568: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 390 op/s 2024-04-02T13:53:16.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:15 smithi187 ceph-mon[36064]: pgmap v568: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 390 op/s 2024-04-02T13:53:16.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:15 smithi188 ceph-mon[36117]: pgmap v568: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 390 op/s 2024-04-02T13:53:16.331 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147311 28.23 MB/sec execute 156 sec latency 973.051 ms 2024-04-02T13:53:17.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147397 28.07 MB/sec execute 157 sec latency 1531.192 ms 2024-04-02T13:53:18.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:17 smithi122 ceph-mon[32114]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2024-04-02T13:53:18.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:17 smithi187 ceph-mon[36064]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2024-04-02T13:53:18.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:17 smithi188 ceph-mon[36117]: pgmap v569: 129 pgs: 129 active+clean; 1.9 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 413 op/s 2024-04-02T13:53:18.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147634 27.97 MB/sec execute 158 sec latency 157.940 ms 2024-04-02T13:53:19.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 147926 27.90 MB/sec execute 159 sec latency 154.302 ms 2024-04-02T13:53:19.926 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:53:19.926 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:53:20.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:19 smithi122 ceph-mon[32114]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:20.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.095+0000 7f52b5ffb640 1 -- 172.21.15.122:0/4148515236 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0043920 con 0x7f52b8106a00 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.095+0000 7f52bd6d9640 1 -- 172.21.15.122:0/4148515236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 msgr2=0x7f52b8106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.095+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/4148515236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b8106de0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f52b001acb0 tx=0x7f52b003d1d0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 -- 172.21.15.122:0/4148515236 shutdown_connections 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/4148515236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 0x7f52b81056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/4148515236 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/4148515236 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b8106de0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 -- 172.21.15.122:0/4148515236 >> 172.21.15.122:0/4148515236 conn(0x7f52b8076f20 msgr2=0x7f52b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 -- 172.21.15.122:0/4148515236 shutdown_connections 2024-04-02T13:53:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.096+0000 7f52bd6d9640 1 -- 172.21.15.122:0/4148515236 wait complete. 2024-04-02T13:53:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.097+0000 7f52bd6d9640 1 Processor -- start 2024-04-02T13:53:20.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 -- start start 2024-04-02T13:53:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 0x7f52b8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b819a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8109ca0 con 0x7f52b80fffa0 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52450/0 (socket says 172.21.15.122:52450) 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52b6ffd640 1 -- 172.21.15.122:0/418124687 learned_addr learned my addr 172.21.15.122:0/418124687 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8109b20 con 0x7f52b8106a00 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.098+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b8109e20 con 0x7f52b8100940 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b67fc640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 0x7f52b8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b77fe640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b819a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 -- 172.21.15.122:0/418124687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 msgr2=0x7f52b8196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 0x7f52b8196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 -- 172.21.15.122:0/418124687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 msgr2=0x7f52b819a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b819a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 -- 172.21.15.122:0/418124687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b819afc0 con 0x7f52b80fffa0 2024-04-02T13:53:20.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b77fe640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:20.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.099+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f52b003ffd0 tx=0x7f52b001a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.100+0000 7f529ffff640 1 -- 172.21.15.122:0/418124687 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b004a040 con 0x7f52b80fffa0 2024-04-02T13:53:20.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.100+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b819b250 con 0x7f52b80fffa0 2024-04-02T13:53:20.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.100+0000 7f529ffff640 1 -- 172.21.15.122:0/418124687 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b0053070 con 0x7f52b80fffa0 2024-04-02T13:53:20.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.100+0000 7f529ffff640 1 -- 172.21.15.122:0/418124687 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00586c0 con 0x7f52b80fffa0 2024-04-02T13:53:20.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.100+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b819b680 con 0x7f52b80fffa0 2024-04-02T13:53:20.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.103+0000 7f529ffff640 1 -- 172.21.15.122:0/418124687 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f52b0043520 con 0x7f52b80fffa0 2024-04-02T13:53:20.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.104+0000 7f529ffff640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 0x7f52a00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.104+0000 7f529ffff640 1 -- 172.21.15.122:0/418124687 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f52b0103730 con 0x7f52b80fffa0 2024-04-02T13:53:20.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.107+0000 7f52b67fc640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 0x7f52a00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.109+0000 7f52b67fc640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 0x7f52a00a6260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52b8104c00 tx=0x7f52a800b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.111+0000 7f52c039e740 1 Processor -- start 2024-04-02T13:53:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.111+0000 7f52c039e740 1 -- start start 2024-04-02T13:53:20.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52c039e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 0x5564d32bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52c039e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 0x5564d32bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 0x5564d32bacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47452/0 (socket says 172.21.15.122:47452) 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52b6ffd640 1 -- 172.21.15.122:0/2213003548 learned_addr learned my addr 172.21.15.122:0/2213003548 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52c039e740 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32c3dd0 0x5564d32c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564d321e830 con 0x5564d32bb1f0 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52b67fc640 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.112+0000 7f52b77fe640 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32c3dd0 0x5564d32c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5564d31c6820 con 0x5564d32c3dd0 2024-04-02T13:53:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564d321a7e0 con 0x5564d32ba8d0 2024-04-02T13:53:20.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 -- 172.21.15.122:0/2213003548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 msgr2=0x5564d32bacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 0x5564d32bacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 -- 172.21.15.122:0/2213003548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32c3dd0 msgr2=0x5564d32c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32c3dd0 0x5564d32c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 -- 172.21.15.122:0/2213003548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d2ffc7d0 con 0x5564d32bb1f0 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.113+0000 7f52b67fc640 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c3890 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f52a8010f20 tx=0x7f52a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c578a7d252c453bf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.114+0000 7f529d7fa640 1 -- 172.21.15.122:0/2213003548 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8005220 con 0x5564d32bb1f0 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.114+0000 7f529d7fa640 1 -- 172.21.15.122:0/2213003548 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a8011e50 con 0x5564d32bb1f0 2024-04-02T13:53:20.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.114+0000 7f529d7fa640 1 -- 172.21.15.122:0/2213003548 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801e840 con 0x5564d32bb1f0 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 msgr2=0x5564d32c3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c3890 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f52a8010f20 tx=0x7f52a8013000 comp rx=0 tx=0).stop 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 shutdown_connections 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32c3dd0 0x5564d32c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c3890 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.115+0000 7f52c039e740 1 --2- 172.21.15.122:0/2213003548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32ba8d0 0x5564d32bacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.116+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 >> 172.21.15.122:0/2213003548 conn(0x5564d32b5000 msgr2=0x5564d32b53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.116+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 shutdown_connections 2024-04-02T13:53:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.116+0000 7f52c039e740 1 -- 172.21.15.122:0/2213003548 wait complete. 2024-04-02T13:53:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 -- start start 2024-04-02T13:53:20.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 Processor -- start 2024-04-02T13:53:20.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5564d32c6c10 con 0x5564d32bb1f0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5564d31d9d30 con 0x5564d32ba8d0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.117+0000 7f52c039e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5564d32c6d90 con 0x5564d32c3dd0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52c039e740 10 client.0 _ll_register_callbacks cb 0x5564d31d9720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47466/0 (socket says 172.21.15.122:47466) 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 -- 172.21.15.122:0/3518599814 learned_addr learned my addr 172.21.15.122:0/3518599814 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b67fc640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 -- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 msgr2=0x5564d32c3600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:53:20.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 -- 172.21.15.122:0/3518599814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 msgr2=0x5564d32c0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.118+0000 7f52b77fe640 1 -- 172.21.15.122:0/3518599814 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d2f323f0 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.119+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bf40 tx=0x7f52ac0128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f3fe4727a82c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.119+0000 7f529aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.119+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac01f060 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.119+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f52ac0108a0 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.119+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a8011e50 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.120+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f52ac010aa0 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.120+0000 7f52c039e740 10 client.29317 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.120+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5564d2ef20f0 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.120+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac010c40 con 0x5564d32c3dd0 2024-04-02T13:53:20.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.120+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f52ac019030 con 0x5564d32c3dd0 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 10 client.29317 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 10 client.29317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 20 client.29317 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d32d3b10 0x5564d32d5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 4 client.29317 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f52a8011e50 con 0x5564d32d3b10 2024-04-02T13:53:20.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.121+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d32d3b10 0x5564d32d5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.122+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d32d3b10 0x5564d32d5f30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32292951900e2f0 server_cookie=a9a7035b01014484 in_seq=0 out_seq=0 2024-04-02T13:53:20.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.122+0000 7f529aff5640 10 client.29317 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f529aff5640 1 -- 172.21.15.122:0/3518599814 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f52a8011e50 con 0x5564d32d3b10 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f529aff5640 10 client.29317 handle_command_reply: tid=0 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 client.29317 shutdown 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 msgr2=0x5564d32c0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bf40 tx=0x7f52ac0128e0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 shutdown_connections 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d32d3b10 0x5564d32d5f30 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c0ed0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 --2- 172.21.15.122:0/3518599814 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 >> 172.21.15.122:0/3518599814 conn(0x5564d32b5000 msgr2=0x5564d32b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.125+0000 7f529aff5640 0 client.29317 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:20.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.127+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 shutdown_connections 2024-04-02T13:53:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.128+0000 7f52c039e740 1 -- 172.21.15.122:0/3518599814 wait complete. 2024-04-02T13:53:20.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.129+0000 7f52c039e740 20 client.29317 trim_cache size 0 max 16384 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 Processor -- start 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 -- start start 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 0x5564d32cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32bb1f0 0x5564d32c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81acf30 con 0x5564d32ba8d0 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aeb30 con 0x5564d32bb1f0 2024-04-02T13:53:20.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.159+0000 7f52c039e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81ad600 con 0x5564d32c3dd0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 0x5564d32cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 0x5564d32cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52474/0 (socket says 172.21.15.122:52474) 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b6ffd640 1 -- 172.21.15.122:0/3338004509 learned_addr learned my addr 172.21.15.122:0/3338004509 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b67fc640 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32bb1f0 0x5564d32c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 -- 172.21.15.122:0/3338004509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32bb1f0 msgr2=0x5564d32c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32bb1f0 0x5564d32c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 -- 172.21.15.122:0/3338004509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 msgr2=0x5564d32cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 0x5564d32cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 -- 172.21.15.122:0/3338004509 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d32b4e30 con 0x5564d32c3dd0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.160+0000 7f52b77fe640 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c3600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bf10 tx=0x7f52ac00b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f6a066f95f68e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.161+0000 7f529cff9640 1 -- 172.21.15.122:0/3338004509 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac034070 con 0x5564d32c3dd0 2024-04-02T13:53:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.161+0000 7f529cff9640 1 -- 172.21.15.122:0/3338004509 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac010e40 con 0x5564d32c3dd0 2024-04-02T13:53:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.161+0000 7f529cff9640 1 -- 172.21.15.122:0/3338004509 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac029080 con 0x5564d32c3dd0 2024-04-02T13:53:20.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 -- 172.21.15.122:0/3338004509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 msgr2=0x5564d32c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c3600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00bf10 tx=0x7f52ac00b6b0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 -- 172.21.15.122:0/3338004509 shutdown_connections 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c3600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32bb1f0 0x5564d32c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 --2- 172.21.15.122:0/3338004509 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32ba8d0 0x5564d32cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 -- 172.21.15.122:0/3338004509 >> 172.21.15.122:0/3338004509 conn(0x5564d32b5000 msgr2=0x5564d32bbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 -- 172.21.15.122:0/3338004509 shutdown_connections 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.162+0000 7f52c039e740 1 -- 172.21.15.122:0/3338004509 wait complete. 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 -- start start 2024-04-02T13:53:20.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 Processor -- start 2024-04-02T13:53:20.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81ae460 con 0x5564d32bb1f0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81adcd0 con 0x5564d32ba8d0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b81aaf30 con 0x5564d32c3dd0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.163+0000 7f52c039e740 10 client.0 _ll_register_callbacks cb 0x7f52ac00ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.164+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.164+0000 7f52b77fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47486/0 (socket says 172.21.15.122:47486) 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.164+0000 7f52b77fe640 1 -- 172.21.15.122:0/2624614654 learned_addr learned my addr 172.21.15.122:0/2624614654 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b67fc640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b77fe640 1 -- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 msgr2=0x5564d32c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b77fe640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b77fe640 1 -- 172.21.15.122:0/2624614654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 msgr2=0x5564d32c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b77fe640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.165+0000 7f52b77fe640 1 -- 172.21.15.122:0/2624614654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d32c3c00 con 0x5564d32c3dd0 2024-04-02T13:53:20.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f52b77fe640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52ac023820 tx=0x7f52ac00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9fc0f607ebe437b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f529aff5640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac034040 con 0x5564d32c3dd0 2024-04-02T13:53:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52ac010e40 con 0x5564d32c3dd0 2024-04-02T13:53:20.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f52ac005030 con 0x5564d32c3dd0 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.167+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f52ac029040 con 0x5564d32c3dd0 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.168+0000 7f52c039e740 10 client.29323 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.168+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5564d334b230 con 0x5564d32c3dd0 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.168+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52ac010e40 con 0x5564d32c3dd0 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f52ac019040 con 0x5564d32c3dd0 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f52c039e740 10 client.29323 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f52c039e740 10 client.29323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f52c039e740 20 client.29323 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d334b830 0x5564d334dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.169+0000 7f52c039e740 4 client.29323 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:53:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.170+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5564d322bd70 con 0x5564d334b830 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.171+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d334b830 0x5564d334dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.172+0000 7f52b6ffd640 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d334b830 0x5564d334dc20 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe07a04b553ecf08 server_cookie=8def63e98755bb21 in_seq=0 out_seq=0 2024-04-02T13:53:20.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.172+0000 7f529aff5640 10 client.29323 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f529aff5640 1 -- 172.21.15.122:0/2624614654 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5564d322bd70 con 0x5564d334b830 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f529aff5640 10 client.29323 handle_command_reply: tid=0 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 client.29323 shutdown 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 msgr2=0x5564d32c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f52ac023820 tx=0x7f52ac00bea0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 shutdown_connections 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5564d334b830 0x5564d334dc20 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5564d32c3dd0 0x5564d32c8580 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f529aff5640 0 client.29323 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5564d32bb1f0 0x5564d32c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 --2- 172.21.15.122:0/2624614654 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5564d32ba8d0 0x5564d32c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.174+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 >> 172.21.15.122:0/2624614654 conn(0x5564d32b5000 msgr2=0x5564d32b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 shutdown_connections 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52c039e740 1 -- 172.21.15.122:0/2624614654 wait complete. 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52c039e740 20 client.29323 trim_cache size 0 max 16384 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:53:20.180 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 msgr2=0x7f52a00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 0x7f52a00a6260 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f52b8104c00 tx=0x7f52a800b390 comp rx=0 tx=0).stop 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 msgr2=0x7f52b8195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.175+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f52b003ffd0 tx=0x7f52b001a370 comp rx=0 tx=0).stop 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 shutdown_connections 2024-04-02T13:53:20.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f52a00a3da0 0x7f52a00a6260 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52b8106a00 0x7f52b819a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b8100940 0x7f52b8196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 --2- 172.21.15.122:0/418124687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52b80fffa0 0x7f52b8195fd0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 >> 172.21.15.122:0/418124687 conn(0x7f52b8076f20 msgr2=0x7f52b80fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 shutdown_connections 2024-04-02T13:53:20.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.176+0000 7f52bd6d9640 1 -- 172.21.15.122:0/418124687 wait complete. 2024-04-02T13:53:20.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:53:20.197 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:20.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:19 smithi187 ceph-mon[36064]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:20.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:19 smithi188 ceph-mon[36117]: pgmap v570: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:20.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 148332 27.84 MB/sec execute 160 sec latency 111.220 ms 2024-04-02T13:53:20.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.357+0000 7f8b137fe640 1 -- 172.21.15.122:0/2759427759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c032a00 con 0x7f8b1411b1a0 2024-04-02T13:53:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.358+0000 7f8b1aa77640 1 -- 172.21.15.122:0/2759427759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1411b1a0 msgr2=0x7f8b1411d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.358+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/2759427759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1411b1a0 0x7f8b1411d590 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009ca0 tx=0x7f8b0c02c120 comp rx=0 tx=0).stop 2024-04-02T13:53:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 -- 172.21.15.122:0/2759427759 shutdown_connections 2024-04-02T13:53:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/2759427759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1411b1a0 0x7f8b1411d590 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/2759427759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1406de00 0x7f8b1411ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/2759427759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406d4e0 0x7f8b1406d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 -- 172.21.15.122:0/2759427759 >> 172.21.15.122:0/2759427759 conn(0x7f8b1406cb50 msgr2=0x7f8b1410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 -- 172.21.15.122:0/2759427759 shutdown_connections 2024-04-02T13:53:20.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.359+0000 7f8b1aa77640 1 -- 172.21.15.122:0/2759427759 wait complete. 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 Processor -- start 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 -- start start 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1406d4e0 0x7f8b141a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1411b1a0 0x7f8b141abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14120d60 con 0x7f8b1406d4e0 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14120be0 con 0x7f8b1411b1a0 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b1aa77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b14120ee0 con 0x7f8b1406de00 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b13fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b13fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47516/0 (socket says 172.21.15.122:47516) 2024-04-02T13:53:20.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.360+0000 7f8b13fff640 1 -- 172.21.15.122:0/1164240545 learned_addr learned my addr 172.21.15.122:0/1164240545 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b19014640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1411b1a0 0x7f8b141abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 -- 172.21.15.122:0/1164240545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1411b1a0 msgr2=0x7f8b141abba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1411b1a0 0x7f8b141abba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 -- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1406d4e0 msgr2=0x7f8b141a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1406d4e0 0x7f8b141a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 -- 172.21.15.122:0/1164240545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b141ac2d0 con 0x7f8b1406de00 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.361+0000 7f8b13fff640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b450 tx=0x7f8afc00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.362+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc015020 con 0x7f8b1406de00 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.362+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc011d70 con 0x7f8b1406de00 2024-04-02T13:53:20.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.362+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f340 con 0x7f8b1406de00 2024-04-02T13:53:20.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.363+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b141ac5c0 con 0x7f8b1406de00 2024-04-02T13:53:20.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.364+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b141b3e50 con 0x7f8b1406de00 2024-04-02T13:53:20.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.365+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b140688f0 con 0x7f8b1406de00 2024-04-02T13:53:20.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.372+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8afc018020 con 0x7f8b1406de00 2024-04-02T13:53:20.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.373+0000 7f8b11ffb640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 0x7f8af40a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.374+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7f8afc0d05c0 con 0x7f8b1406de00 2024-04-02T13:53:20.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.374+0000 7f8b18813640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 0x7f8af40a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.375+0000 7f8b18813640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 0x7f8af40a63b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009710 tx=0x7f8b04006c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.376+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8afc00d040 con 0x7f8b1406de00 2024-04-02T13:53:20.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.774+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b141a8630 con 0x7f8b1406de00 2024-04-02T13:53:20.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.775+0000 7f8b11ffb640 1 -- 172.21.15.122:0/1164240545 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11949 (secure 0 0 0) 0x7f8afc09c9d0 con 0x7f8b1406de00 2024-04-02T13:53:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:53:20.779 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:52:42.709142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:53:20.779 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 73 2024-04-02T13:53:20.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 msgr2=0x7f8af40a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 0x7f8af40a63b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009710 tx=0x7f8b04006c90 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 msgr2=0x7f8b141a7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b450 tx=0x7f8afc00b910 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 shutdown_connections 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8af40a3ef0 0x7f8af40a63b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b1411b1a0 0x7f8b141abba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b1406de00 0x7f8b141a7820 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 --2- 172.21.15.122:0/1164240545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8b1406d4e0 0x7f8b141a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 >> 172.21.15.122:0/1164240545 conn(0x7f8b1406cb50 msgr2=0x7f8b1410ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 shutdown_connections 2024-04-02T13:53:20.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.779+0000 7f8b1aa77640 1 -- 172.21.15.122:0/1164240545 wait complete. 2024-04-02T13:53:20.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 -- 172.21.15.122:0/257335569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5601059d0 msgr2=0x7fe560105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 --2- 172.21.15.122:0/257335569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5601059d0 0x7fe560105db0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fe550009900 tx=0x7fe55002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 -- 172.21.15.122:0/257335569 shutdown_connections 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 --2- 172.21.15.122:0/257335569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe560101a70 0x7fe56010d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 --2- 172.21.15.122:0/257335569 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 --2- 172.21.15.122:0/257335569 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5601059d0 0x7fe560105db0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 -- 172.21.15.122:0/257335569 >> 172.21.15.122:0/257335569 conn(0x7fe560076f20 msgr2=0x7fe5600fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 -- 172.21.15.122:0/257335569 shutdown_connections 2024-04-02T13:53:20.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.948+0000 7fe564c55640 1 -- 172.21.15.122:0/257335569 wait complete. 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.949+0000 7fe564c55640 1 Processor -- start 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.949+0000 7fe564c55640 1 -- start start 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe560101a70 0x7fe560198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 0x7fe56019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe560110d80 con 0x7fe560101a70 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe560110c00 con 0x7fe5601010b0 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe564c55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe560110f00 con 0x7fe5601059d0 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54698/0 (socket says 172.21.15.122:54698) 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55e59c640 1 -- 172.21.15.122:0/3754790539 learned_addr learned my addr 172.21.15.122:0/3754790539 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 0x7fe56019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 -- 172.21.15.122:0/3754790539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 msgr2=0x7fe560198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 -- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe560101a70 msgr2=0x7fe560198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe560101a70 0x7fe560198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.950+0000 7fe55ed9d640 1 -- 172.21.15.122:0/3754790539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56019d320 con 0x7fe5601059d0 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 0x7fe56019cbf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe558009fd0 tx=0x7fe55800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe557fff640 1 -- 172.21.15.122:0/3754790539 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55800ce20 con 0x7fe5601059d0 2024-04-02T13:53:20.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56019d610 con 0x7fe5601059d0 2024-04-02T13:53:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe557fff640 1 -- 172.21.15.122:0/3754790539 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe558005020 con 0x7fe5601059d0 2024-04-02T13:53:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe557fff640 1 -- 172.21.15.122:0/3754790539 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558010720 con 0x7fe5601059d0 2024-04-02T13:53:20.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.951+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5600fc2a0 con 0x7fe5601059d0 2024-04-02T13:53:20.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.955+0000 7fe557fff640 1 -- 172.21.15.122:0/3754790539 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe558005540 con 0x7fe5601059d0 2024-04-02T13:53:20.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.955+0000 7fe56783c740 1 Processor -- start 2024-04-02T13:53:20.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.955+0000 7fe56783c740 1 -- start start 2024-04-02T13:53:20.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.955+0000 7fe56783c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.955+0000 7fe56783c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d9f60 0x5592898eaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe56783c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe56783c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55928983d7b0 con 0x5592898d95b0 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe56783c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592897dc570 con 0x5592898d9f60 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe56783c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559289839760 con 0x5592898eb4b0 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52522/0 (socket says 172.21.15.122:52522) 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 -- 172.21.15.122:0/978656650 learned_addr learned my addr 172.21.15.122:0/978656650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 -- 172.21.15.122:0/978656650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 msgr2=0x5592898ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 -- 172.21.15.122:0/978656650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d9f60 msgr2=0x5592898eaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d9f60 0x5592898eaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 -- 172.21.15.122:0/978656650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592895513f0 con 0x5592898d95b0 2024-04-02T13:53:20.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe557fff640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 0x7fe54c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe555d9b640 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe548009f90 tx=0x7fe548009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01e805764804f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.956+0000 7fe557fff640 1 -- 172.21.15.122:0/3754790539 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fe5580d1260 con 0x7fe5601059d0 2024-04-02T13:53:20.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.957+0000 7fe55e59c640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 0x7fe54c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.957+0000 7fe554d99640 1 -- 172.21.15.122:0/978656650 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548004030 con 0x5592898d95b0 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.957+0000 7fe554d99640 1 -- 172.21.15.122:0/978656650 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5480041d0 con 0x5592898d95b0 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.957+0000 7fe554d99640 1 -- 172.21.15.122:0/978656650 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480126b0 con 0x5592898d95b0 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 -- 172.21.15.122:0/978656650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 msgr2=0x5592898d9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fe548009f90 tx=0x7fe548009770 comp rx=0 tx=0).stop 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe55e59c640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 0x7fe54c0a6260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe550000c00 tx=0x7fe550005b50 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 -- 172.21.15.122:0/978656650 shutdown_connections 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d9f60 0x5592898eaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 --2- 172.21.15.122:0/978656650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898d9990 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 -- 172.21.15.122:0/978656650 >> 172.21.15.122:0/978656650 conn(0x5592898d3ed0 msgr2=0x5592898d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 -- 172.21.15.122:0/978656650 shutdown_connections 2024-04-02T13:53:20.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.958+0000 7fe56783c740 1 -- 172.21.15.122:0/978656650 wait complete. 2024-04-02T13:53:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 -- start start 2024-04-02T13:53:20.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 Processor -- start 2024-04-02T13:53:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d95b0 0x5592898eaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592898ee270 con 0x5592898d9f60 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592897f8e80 con 0x5592898d95b0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592898ee3f0 con 0x5592898eb4b0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe56783c740 10 client.0 _ll_register_callbacks cb 0x559289834aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52536/0 (socket says 172.21.15.122:52536) 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.959+0000 7fe555d9b640 1 -- 172.21.15.122:0/3930649317 learned_addr learned my addr 172.21.15.122:0/3930649317 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe55e59c640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d95b0 0x5592898eaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 -- 172.21.15.122:0/3930649317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 msgr2=0x5592898e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898e89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 -- 172.21.15.122:0/3930649317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d95b0 msgr2=0x5592898eaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d95b0 0x5592898eaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 -- 172.21.15.122:0/3930649317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928961b7d0 con 0x5592898d9f60 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe555d9b640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe548010c90 tx=0x7fe5480094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e08e9329e1c5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548012450 con 0x5592898d9f60 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.960+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5480041d0 con 0x5592898d9f60 2024-04-02T13:53:20.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fe5480125f0 con 0x5592898d9f60 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe548012830 con 0x5592898d9f60 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 10 client.20652 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5592898fa350 con 0x5592898d9f60 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5480041d0 con 0x5592898d9f60 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fe548031770 con 0x5592898d9f60 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 10 client.20652 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 10 client.20652 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 20 client.20652 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592898fb810 0x5592898fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 4 client.20652 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.961+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928984acf0 con 0x5592898fb810 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.962+0000 7fe55e59c640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592898fb810 0x5592898fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.962+0000 7fe545ffb640 10 client.20652 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.963+0000 7fe55e59c640 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592898fb810 0x5592898fdc00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=93775ef4361af2fc server_cookie=94b630b63799dc57 in_seq=0 out_seq=0 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe545ffb640 1 -- 172.21.15.122:0/3930649317 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55928984acf0 con 0x5592898fb810 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe545ffb640 10 client.20652 handle_command_reply: tid=0 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 client.20652 shutdown 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 msgr2=0x5592898e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fe548010c90 tx=0x7fe5480094f0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 shutdown_connections 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x5592898fb810 0x5592898fdc00 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898eb4b0 0x5592898e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592898e82a0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 --2- 172.21.15.122:0/3930649317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898d95b0 0x5592898eaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 >> 172.21.15.122:0/3930649317 conn(0x5592898d3ed0 msgr2=0x5592898d5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.964+0000 7fe545ffb640 0 client.20652 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.965+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 shutdown_connections 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.965+0000 7fe56783c740 1 -- 172.21.15.122:0/3930649317 wait complete. 2024-04-02T13:53:20.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.965+0000 7fe56783c740 20 client.20652 trim_cache size 0 max 16384 2024-04-02T13:53:20.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.986+0000 7fe56783c740 1 Processor -- start 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 -- start start 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d95b0 0x5592898f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x559289903920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601acff0 con 0x5592898d9f60 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601aebf0 con 0x5592898eb4b0 2024-04-02T13:53:20.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe56783c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601ad6c0 con 0x5592898d95b0 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe555d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52552/0 (socket says 172.21.15.122:52552) 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.987+0000 7fe555d9b640 1 -- 172.21.15.122:0/2879037556 learned_addr learned my addr 172.21.15.122:0/2879037556 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x559289903920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe55e59c640 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d95b0 0x5592898f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 -- 172.21.15.122:0/2879037556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d95b0 msgr2=0x5592898f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d95b0 0x5592898f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 -- 172.21.15.122:0/2879037556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 msgr2=0x559289903920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x559289903920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 -- 172.21.15.122:0/2879037556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592898d8890 con 0x5592898d9f60 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe555d9b640 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe548004700 tx=0x7fe548009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa84a23b295e558 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe547fff640 1 -- 172.21.15.122:0/2879037556 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548035030 con 0x5592898d9f60 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe547fff640 1 -- 172.21.15.122:0/2879037556 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe548004180 con 0x5592898d9f60 2024-04-02T13:53:20.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.988+0000 7fe547fff640 1 -- 172.21.15.122:0/2879037556 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe548009c50 con 0x5592898d9f60 2024-04-02T13:53:20.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.989+0000 7fe56783c740 1 -- 172.21.15.122:0/2879037556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 msgr2=0x5592899033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:20.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.989+0000 7fe56783c740 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fe548004700 tx=0x7fe548009560 comp rx=0 tx=0).stop 2024-04-02T13:53:20.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 -- 172.21.15.122:0/2879037556 shutdown_connections 2024-04-02T13:53:20.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x559289903920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d9f60 0x5592899033e0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 --2- 172.21.15.122:0/2879037556 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d95b0 0x5592898f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 -- 172.21.15.122:0/2879037556 >> 172.21.15.122:0/2879037556 conn(0x5592898d3ed0 msgr2=0x5592898fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:20.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 -- 172.21.15.122:0/2879037556 shutdown_connections 2024-04-02T13:53:20.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 -- 172.21.15.122:0/2879037556 wait complete. 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 -- start start 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.992+0000 7fe56783c740 1 Processor -- start 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d9f60 0x5592898ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x5592898ec370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601ae520 con 0x5592898d95b0 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601add90 con 0x5592898eb4b0 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe55e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52566/0 (socket says 172.21.15.122:52566) 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe55e59c640 1 -- 172.21.15.122:0/1147843989 learned_addr learned my addr 172.21.15.122:0/1147843989 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5601aaff0 con 0x5592898d9f60 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe55ed9d640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x5592898ec370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe555d9b640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d9f60 0x5592898ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.993+0000 7fe56783c740 10 client.0 _ll_register_callbacks cb 0x7fe548009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 -- 172.21.15.122:0/1147843989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d9f60 msgr2=0x5592898ebe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d9f60 0x5592898ebe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 -- 172.21.15.122:0/1147843989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 msgr2=0x5592898ec370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x5592898ec370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 -- 172.21.15.122:0/1147843989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592898f2810 con 0x5592898d95b0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe55e59c640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe550006010 tx=0x7fe5500058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563b6429d16dd131 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55002cab0 con 0x5592898d95b0 2024-04-02T13:53:20.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fe55002cc50 con 0x5592898d95b0 2024-04-02T13:53:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe548004180 con 0x5592898d95b0 2024-04-02T13:53:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe550067050 con 0x5592898d95b0 2024-04-02T13:53:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe56783c740 10 client.20658 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.994+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5592898f3b30 con 0x5592898d95b0 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe548004180 con 0x5592898d95b0 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fe550004ec0 con 0x5592898d95b0 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 10 client.20658 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 10 client.20658 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 20 client.20658 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559289961de0 0x559289964200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 4 client.20658 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559289845040 con 0x559289961de0 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.995+0000 7fe555d9b640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559289961de0 0x559289964200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.996+0000 7fe545ffb640 10 client.20658 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:21.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.996+0000 7fe555d9b640 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559289961de0 0x559289964200 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af04f76ca20ecb94 server_cookie=c2e414930c7883f in_seq=0 out_seq=0 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe545ffb640 1 -- 172.21.15.122:0/1147843989 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559289845040 con 0x559289961de0 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe545ffb640 10 client.20658 handle_command_reply: tid=0 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 client.20658 shutdown 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 msgr2=0x5592898eb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe550006010 tx=0x7fe5500058d0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 shutdown_connections 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559289961de0 0x559289964200 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe545ffb640 0 client.20658 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:20.999+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592898eb4b0 0x5592898ec370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.000+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592898d9f60 0x5592898ebe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.000+0000 7fe56783c740 1 --2- 172.21.15.122:0/1147843989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592898d95b0 0x5592898eb8f0 secure :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fe550006010 tx=0x7fe5500058d0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.000+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 >> 172.21.15.122:0/1147843989 conn(0x5592898d3ed0 msgr2=0x5592898e6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.000+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 shutdown_connections 2024-04-02T13:53:21.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.000+0000 7fe56783c740 1 -- 172.21.15.122:0/1147843989 wait complete. 2024-04-02T13:53:21.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.001+0000 7fe56783c740 20 client.20658 trim_cache size 0 max 16384 2024-04-02T13:53:21.005 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:21.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 msgr2=0x7fe54c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 0x7fe54c0a6260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe550000c00 tx=0x7fe550005b50 comp rx=0 tx=0).stop 2024-04-02T13:53:21.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 msgr2=0x7fe56019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 0x7fe56019cbf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe558009fd0 tx=0x7fe55800ef10 comp rx=0 tx=0).stop 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 shutdown_connections 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe54c0a3da0 0x7fe54c0a6260 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe550000c00 tx=0x7fe550005b50 comp rx=0 tx=0).stop 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.002+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5601059d0 0x7fe56019cbf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.003+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe560101a70 0x7fe560198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.003+0000 7fe564c55640 1 --2- 172.21.15.122:0/3754790539 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5601010b0 0x7fe560198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.003+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 >> 172.21.15.122:0/3754790539 conn(0x7fe560076f20 msgr2=0x7fe5601001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.004+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 shutdown_connections 2024-04-02T13:53:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.004+0000 7fe564c55640 1 -- 172.21.15.122:0/3754790539 wait complete. 2024-04-02T13:53:21.028 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:53:21.065 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:20 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1164240545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:21.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 -- 172.21.15.122:0/2892099980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4109040 msgr2=0x7fcbe4110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/2892099980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4109040 0x7fcbe4110c10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009ca0 tx=0x7fcbdc02c100 comp rx=0 tx=0).stop 2024-04-02T13:53:21.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 -- 172.21.15.122:0/2892099980 shutdown_connections 2024-04-02T13:53:21.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/2892099980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4109040 0x7fcbe4110c10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/2892099980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4104920 0x7fcbe4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/2892099980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe4103f70 0x7fcbe4104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 -- 172.21.15.122:0/2892099980 >> 172.21.15.122:0/2892099980 conn(0x7fcbe40778e0 msgr2=0x7fcbe4102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 -- 172.21.15.122:0/2892099980 shutdown_connections 2024-04-02T13:53:21.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.188+0000 7fcbebfe9640 1 -- 172.21.15.122:0/2892099980 wait complete. 2024-04-02T13:53:21.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.189+0000 7fcbebfe9640 1 Processor -- start 2024-04-02T13:53:21.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.189+0000 7fcbebfe9640 1 -- start start 2024-04-02T13:53:21.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4109040 0x7fcbe419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe419ee30 0x7fcbe419f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe41140d0 con 0x7fcbe4109040 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4113f50 con 0x7fcbe419ee30 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbebfe9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcbe4114250 con 0x7fcbe4103f70 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47586/0 (socket says 172.21.15.122:47586) 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 -- 172.21.15.122:0/1973348283 learned_addr learned my addr 172.21.15.122:0/1973348283 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4109040 0x7fcbe419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 -- 172.21.15.122:0/1973348283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe419ee30 msgr2=0x7fcbe419f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe419ee30 0x7fcbe419f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 -- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4109040 msgr2=0x7fcbe419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4109040 0x7fcbe419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.190+0000 7fcbe9d85640 1 -- 172.21.15.122:0/1973348283 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbe419f920 con 0x7fcbe4103f70 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e9f0 tx=0x7fcbe000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbdaffd640 1 -- 172.21.15.122:0/1973348283 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000cd80 con 0x7fcbe4103f70 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbdaffd640 1 -- 172.21.15.122:0/1973348283 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe0010910 con 0x7fcbe4103f70 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbdaffd640 1 -- 172.21.15.122:0/1973348283 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0018980 con 0x7fcbe4103f70 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbe419fc10 con 0x7fcbe4103f70 2024-04-02T13:53:21.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.191+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbe41a7400 con 0x7fcbe4103f70 2024-04-02T13:53:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.195+0000 7fcbdaffd640 1 -- 172.21.15.122:0/1973348283 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcbe0018b20 con 0x7fcbe4103f70 2024-04-02T13:53:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.196+0000 7fcbdaffd640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 0x7fcbd00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.196+0000 7fcbdaffd640 1 -- 172.21.15.122:0/1973348283 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fcbe00d1f90 con 0x7fcbe4103f70 2024-04-02T13:53:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.196+0000 7fcbeebd4740 1 Processor -- start 2024-04-02T13:53:21.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 -- start start 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 0x559cde34d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde355f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 0x559cde3588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cde2a87b0 con 0x559cde3564b0 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559cde247570 con 0x559cde34d8d0 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559cde2a4760 con 0x559cde34cfb0 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 0x559cde34d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 0x559cde34d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47592/0 (socket says 172.21.15.122:47592) 2024-04-02T13:53:21.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.197+0000 7fcbe9d85640 1 -- 172.21.15.122:0/3989503295 learned_addr learned my addr 172.21.15.122:0/3989503295 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.198+0000 7fcbe9584640 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde355f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 0x7fcbd00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 0x559cde3588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 -- 172.21.15.122:0/3989503295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 msgr2=0x559cde34d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 0x559cde34d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 -- 172.21.15.122:0/3989503295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 msgr2=0x559cde355f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde355f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.199+0000 7fcbea586640 1 -- 172.21.15.122:0/3989503295 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cddfbc3f0 con 0x559cde3564b0 2024-04-02T13:53:21.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.200+0000 7fcbea586640 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 0x559cde3588a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009bf0 tx=0x7fcbdc032340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c46aac6b64008 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.201+0000 7fcbcffff640 1 -- 172.21.15.122:0/3989503295 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc041020 con 0x559cde3564b0 2024-04-02T13:53:21.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.201+0000 7fcbcffff640 1 -- 172.21.15.122:0/3989503295 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x559cde3564b0 2024-04-02T13:53:21.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.201+0000 7fcbcffff640 1 -- 172.21.15.122:0/3989503295 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc039040 con 0x559cde3564b0 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 -- 172.21.15.122:0/3989503295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 msgr2=0x559cde3588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 0x559cde3588a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009bf0 tx=0x7fcbdc032340 comp rx=0 tx=0).stop 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 -- 172.21.15.122:0/3989503295 shutdown_connections 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde3564b0 0x559cde3588a0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde355f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/3989503295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34cfb0 0x559cde34d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbeebd4740 1 -- 172.21.15.122:0/3989503295 >> 172.21.15.122:0/3989503295 conn(0x559cde3476e0 msgr2=0x559cde347ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.202+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 0x7fcbd00a63b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005d80 tx=0x7fcbd4005cf0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.203+0000 7fcbeebd4740 1 -- 172.21.15.122:0/3989503295 shutdown_connections 2024-04-02T13:53:21.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.204+0000 7fcbeebd4740 1 -- 172.21.15.122:0/3989503295 wait complete. 2024-04-02T13:53:21.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.204+0000 7fcbeebd4740 1 -- start start 2024-04-02T13:53:21.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.204+0000 7fcbeebd4740 1 Processor -- start 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34d8d0 0x559cde353870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde353db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559cde36a2a0 con 0x559cde34d8d0 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559cde263e80 con 0x559cde34cfb0 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559cde36a420 con 0x559cde3564b0 2024-04-02T13:53:21.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.205+0000 7fcbeebd4740 10 client.0 _ll_register_callbacks cb 0x559cde29faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54794/0 (socket says 172.21.15.122:54794) 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 -- 172.21.15.122:0/2713252752 learned_addr learned my addr 172.21.15.122:0/2713252752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbea586640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde353db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 -- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 msgr2=0x559cde353db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde353db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 -- 172.21.15.122:0/2713252752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34d8d0 msgr2=0x559cde353870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34d8d0 0x559cde353870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.206+0000 7fcbe9d85640 1 -- 172.21.15.122:0/2713252752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cde0867d0 con 0x559cde34cfb0 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c6a0 tx=0x7fcbe009e190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b670da967c7cfb55 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00cf6c0 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbdc032c90 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fcbe009cee0 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.207+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fcbe00a1e40 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.208+0000 7fcbeebd4740 10 client.29618 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.208+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559cde3763f0 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fcbe0010ac0 con 0x559cde34cfb0 2024-04-02T13:53:21.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 10 client.29618 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 10 client.29618 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 20 client.29618 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde377900 0x559cde379d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 4 client.29618 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.209+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cde2b5cf0 con 0x559cde377900 2024-04-02T13:53:21.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.210+0000 7fcbe9584640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde377900 0x559cde379d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.213+0000 7fcbe9584640 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde377900 0x559cde379d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b5042f2ef75d7774 server_cookie=8a94fc4dcc2ef098 in_seq=0 out_seq=0 2024-04-02T13:53:21.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.213+0000 7fcbcd7fa640 10 client.29618 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/2713252752 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559cde2b5cf0 con 0x559cde377900 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbcd7fa640 10 client.29618 handle_command_reply: tid=0 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 client.29618 shutdown 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 msgr2=0x559cde353330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c6a0 tx=0x7fcbe009e190 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 shutdown_connections 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde353db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde377900 0x559cde379d30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34d8d0 0x559cde353870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/2713252752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34cfb0 0x559cde353330 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000c6a0 tx=0x7fcbe009e190 comp rx=0 tx=0).stop 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 >> 172.21.15.122:0/2713252752 conn(0x559cde3476e0 msgr2=0x559cde349430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbcd7fa640 0 client.29618 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 shutdown_connections 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 1 -- 172.21.15.122:0/2713252752 wait complete. 2024-04-02T13:53:21.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.216+0000 7fcbeebd4740 20 client.29618 trim_cache size 0 max 16384 2024-04-02T13:53:21.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:20 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1164240545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 Processor -- start 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 -- start start 2024-04-02T13:53:21.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde3731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde352490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8010220 con 0x559cde34ec60 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8011e20 con 0x559cde34d8d0 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbeebd4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc80108f0 con 0x559cde3564b0 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbea586640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbea586640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52598/0 (socket says 172.21.15.122:52598) 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.242+0000 7fcbea586640 1 -- 172.21.15.122:0/725610354 learned_addr learned my addr 172.21.15.122:0/725610354 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 -- 172.21.15.122:0/725610354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 msgr2=0x559cde352490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde352490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 -- 172.21.15.122:0/725610354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 msgr2=0x559cde3731c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde3731c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 -- 172.21.15.122:0/725610354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cde348910 con 0x559cde34ec60 2024-04-02T13:53:21.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.243+0000 7fcbea586640 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009bf0 tx=0x7fcbdc009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b324586361d60ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.245+0000 7fcbcf7fe640 1 -- 172.21.15.122:0/725610354 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc06c070 con 0x559cde34ec60 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.245+0000 7fcbcf7fe640 1 -- 172.21.15.122:0/725610354 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x559cde34ec60 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.245+0000 7fcbcf7fe640 1 -- 172.21.15.122:0/725610354 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc041070 con 0x559cde34ec60 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 -- 172.21.15.122:0/725610354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 msgr2=0x559cde353c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc009bf0 tx=0x7fcbdc009c20 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 -- 172.21.15.122:0/725610354 shutdown_connections 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde353c00 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde3564b0 0x559cde352490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/725610354 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde34d8d0 0x559cde3731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.246+0000 7fcbeebd4740 1 -- 172.21.15.122:0/725610354 >> 172.21.15.122:0/725610354 conn(0x559cde3476e0 msgr2=0x559cde33d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.247+0000 7fcbeebd4740 1 -- 172.21.15.122:0/725610354 shutdown_connections 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.247+0000 7fcbeebd4740 1 -- 172.21.15.122:0/725610354 wait complete. 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.248+0000 7fcbeebd4740 1 -- start start 2024-04-02T13:53:21.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.248+0000 7fcbeebd4740 1 Processor -- start 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.248+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34d8d0 0x559cde36afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde3564b0 0x559cde36ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52610/0 (socket says 172.21.15.122:52610) 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 -- 172.21.15.122:0/1216800307 learned_addr learned my addr 172.21.15.122:0/1216800307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8011750 con 0x559cde34ec60 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8010fc0 con 0x559cde3564b0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc800e220 con 0x559cde34d8d0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbea586640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde3564b0 0x559cde36ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbeebd4740 10 client.0 _ll_register_callbacks cb 0x7fcbe00cf1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34d8d0 0x559cde36afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 -- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34d8d0 msgr2=0x559cde36afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34d8d0 0x559cde36afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 -- 172.21.15.122:0/1216800307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde3564b0 msgr2=0x559cde36ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde3564b0 0x559cde36ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.249+0000 7fcbe9584640 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cde35feb0 con 0x559cde34ec60 2024-04-02T13:53:21.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.250+0000 7fcbe9584640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcbe410a430 tx=0x7fcbd401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924ae9bf1f46afd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.250+0000 7fcbcd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:21.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.250+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbd40295b0 con 0x559cde34ec60 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.252+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbdc032c90 con 0x559cde34ec60 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.252+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9613+0+0 (secure 0 0 0) 0x7fcbd4029750 con 0x559cde34ec60 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.252+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fcbd4005020 con 0x559cde34ec60 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.253+0000 7fcbeebd4740 10 client.20673 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.253+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559cde361200 con 0x559cde34ec60 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbdc032c90 con 0x559cde34ec60 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fcbd401e070 con 0x559cde34ec60 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 10 client.20673 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 10 client.20673 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 20 client.20673 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde3618f0 0x559cde363d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 4 client.20673 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:21.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.254+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cde2b0040 con 0x559cde3618f0 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.256+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde3618f0 0x559cde363d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.256+0000 7fcbcd7fa640 10 client.20673 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.256+0000 7fcbe9d85640 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde3618f0 0x559cde363d10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76d50a9d5e056c2a server_cookie=386de6090c7bd6c8 in_seq=0 out_seq=0 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.256+0000 7fcbcd7fa640 1 -- 172.21.15.122:0/1216800307 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cde2b0040 con 0x559cde3618f0 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbcd7fa640 10 client.20673 handle_command_reply: tid=0 2024-04-02T13:53:21.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 client.20673 shutdown 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 msgr2=0x559cde36b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fcbe410a430 tx=0x7fcbd401a000 comp rx=0 tx=0).stop 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 shutdown_connections 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559cde3564b0 0x559cde36ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x559cde3618f0 0x559cde363d10 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559cde34ec60 0x559cde36b520 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbcd7fa640 0 client.20673 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 --2- 172.21.15.122:0/1216800307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559cde34d8d0 0x559cde36afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.257+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 >> 172.21.15.122:0/1216800307 conn(0x559cde3476e0 msgr2=0x559cde33df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.258+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 shutdown_connections 2024-04-02T13:53:21.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.258+0000 7fcbeebd4740 1 -- 172.21.15.122:0/1216800307 wait complete. 2024-04-02T13:53:21.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.258+0000 7fcbeebd4740 20 client.20673 trim_cache size 0 max 16384 2024-04-02T13:53:21.263 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:21.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 msgr2=0x7fcbd00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 0x7fcbd00a63b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4005d80 tx=0x7fcbd4005cf0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 msgr2=0x7fcbe419a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcbe000e9f0 tx=0x7fcbe000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 shutdown_connections 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcbd00a3ef0 0x7fcbd00a63b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcbe419ee30 0x7fcbe419f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcbe4109040 0x7fcbe419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 --2- 172.21.15.122:0/1973348283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbe4103f70 0x7fcbe419a610 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.260+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 >> 172.21.15.122:0/1973348283 conn(0x7fcbe40778e0 msgr2=0x7fcbe4079560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.261+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 shutdown_connections 2024-04-02T13:53:21.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.261+0000 7fcbebfe9640 1 -- 172.21.15.122:0/1973348283 wait complete. 2024-04-02T13:53:21.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:20 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1164240545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:21.280 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:53:21.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 148885 27.83 MB/sec execute 161 sec latency 76.784 ms 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/1470357855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38107a90 msgr2=0x7f5a38107e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/1470357855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38107a90 0x7f5a38107e70 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009900 tx=0x7f5a3002bc40 comp rx=0 tx=0).stop 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/1470357855 shutdown_connections 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/1470357855 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a3810e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/1470357855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38101a80 0x7f5a38101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/1470357855 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38107a90 0x7f5a38107e70 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/1470357855 >> 172.21.15.122:0/1470357855 conn(0x7f5a380fd570 msgr2=0x7f5a380ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/1470357855 shutdown_connections 2024-04-02T13:53:21.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.481+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/1470357855 wait complete. 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.482+0000 7f5a3f1c8640 1 Processor -- start 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.482+0000 7f5a3f1c8640 1 -- start start 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3f1c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 0x7f5a38198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3f1c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a381988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 0x7f5a38198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 0x7f5a38198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54816/0 (socket says 172.21.15.122:54816) 2024-04-02T13:53:21.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a37fff640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a381988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3cf64640 1 -- 172.21.15.122:0/457885884 learned_addr learned my addr 172.21.15.122:0/457885884 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38107a90 0x7f5a3819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.483+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38111e70 con 0x7f5a38102440 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38111cf0 con 0x7f5a38101a80 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38111ff0 con 0x7f5a38107a90 2024-04-02T13:53:21.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a3d765640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38107a90 0x7f5a3819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 -- 172.21.15.122:0/457885884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38107a90 msgr2=0x7f5a3819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38107a90 0x7f5a3819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 -- 172.21.15.122:0/457885884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 msgr2=0x7f5a38198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 0x7f5a38198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3819d380 con 0x7f5a38102440 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.484+0000 7f5a37fff640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a381988d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a2000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.485+0000 7f5a35ffb640 1 -- 172.21.15.122:0/457885884 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20009c30 con 0x7f5a38102440 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.485+0000 7f5a35ffb640 1 -- 172.21.15.122:0/457885884 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a20009dd0 con 0x7f5a38102440 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.485+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3819d670 con 0x7f5a38102440 2024-04-02T13:53:21.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.485+0000 7f5a35ffb640 1 -- 172.21.15.122:0/457885884 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20010460 con 0x7f5a38102440 2024-04-02T13:53:21.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.485+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a381a4f00 con 0x7f5a38102440 2024-04-02T13:53:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.489+0000 7f5a41dab740 1 Processor -- start 2024-04-02T13:53:21.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.489+0000 7f5a41dab740 1 -- start start 2024-04-02T13:53:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.489+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a35ffb640 1 -- 172.21.15.122:0/457885884 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5a20003ef0 con 0x7f5a38102440 2024-04-02T13:53:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af8f90 0x562112af9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a41dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562112a587b0 con 0x562112af8f90 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a41dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621129f7570 con 0x562112b063f0 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a41dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562112a54760 con 0x562112af84f0 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a3d765640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af8f90 0x562112af9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a3d765640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47646/0 (socket says 172.21.15.122:47646) 2024-04-02T13:53:21.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a3d765640 1 -- 172.21.15.122:0/2112562453 learned_addr learned my addr 172.21.15.122:0/2112562453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 -- 172.21.15.122:0/2112562453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 msgr2=0x562112af89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 -- 172.21.15.122:0/2112562453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af8f90 msgr2=0x562112af9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af8f90 0x562112af9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 -- 172.21.15.122:0/2112562453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56211276c3f0 con 0x562112b063f0 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a35ffb640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 0x7f5a240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.490+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b087e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a200cda60 tx=0x7f5a200d8690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0f2724fd6af6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.491+0000 7f5a3d765640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 0x7f5a240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.491+0000 7f5a35ffb640 1 -- 172.21.15.122:0/457885884 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f5a200d0700 con 0x7f5a38102440 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.491+0000 7f5a2effd640 1 -- 172.21.15.122:0/2112562453 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20098600 con 0x562112b063f0 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.491+0000 7f5a2effd640 1 -- 172.21.15.122:0/2112562453 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2009c360 con 0x562112b063f0 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.491+0000 7f5a2effd640 1 -- 172.21.15.122:0/2112562453 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2009bcc0 con 0x562112b063f0 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- 172.21.15.122:0/2112562453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 msgr2=0x562112b087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b087e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a200cda60 tx=0x7f5a200d8690 comp rx=0 tx=0).stop 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- 172.21.15.122:0/2112562453 shutdown_connections 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a3d765640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 0x7f5a240a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5a28011630 tx=0x7f5a28009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b087e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af8f90 0x562112af9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2112562453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- 172.21.15.122:0/2112562453 >> 172.21.15.122:0/2112562453 conn(0x562112af3220 msgr2=0x562112af3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- 172.21.15.122:0/2112562453 shutdown_connections 2024-04-02T13:53:21.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- 172.21.15.122:0/2112562453 wait complete. 2024-04-02T13:53:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.492+0000 7f5a41dab740 1 -- start start 2024-04-02T13:53:21.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.493+0000 7f5a41dab740 1 Processor -- start 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.493+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112aef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52644/0 (socket says 172.21.15.122:52644) 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3cf64640 1 -- 172.21.15.122:0/955596080 learned_addr learned my addr 172.21.15.122:0/955596080 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 0x562112aefd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562112b09260 con 0x562112af84f0 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562112a13e80 con 0x562112af8f90 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562112b093e0 con 0x562112b063f0 2024-04-02T13:53:21.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112aef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a41dab740 10 client.0 _ll_register_callbacks cb 0x562112a4faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3d765640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 0x562112aefd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 -- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 msgr2=0x562112aefd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 0x562112aefd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 -- 172.21.15.122:0/955596080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 msgr2=0x562112af2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621128367d0 con 0x562112af8f90 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a3d765640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 0x562112aefd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.494+0000 7f5a37fff640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112aef5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bfd0 tx=0x7f5a20004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c453cdf5a6636b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:21.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20020070 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a2009c360 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f5a200d6730 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5a2009bea0 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 10 client.29633 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562112b152f0 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a2009c360 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f5a200cdce0 con 0x562112af8f90 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 10 client.29633 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 10 client.29633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.495+0000 7f5a41dab740 20 client.29633 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b16640 0x562112b18a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a41dab740 4 client.29633 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562112a65cf0 con 0x562112b16640 2024-04-02T13:53:21.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b16640 0x562112b18a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a17fff640 10 client.29633 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.496+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b16640 0x562112b18a30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d11bfdcfc4fc351c server_cookie=1823ed3bdcc8a1c2 in_seq=0 out_seq=0 2024-04-02T13:53:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.497+0000 7f5a17fff640 1 -- 172.21.15.122:0/955596080 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562112a65cf0 con 0x562112b16640 2024-04-02T13:53:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.497+0000 7f5a17fff640 10 client.29633 handle_command_reply: tid=0 2024-04-02T13:53:21.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.498+0000 7f5a41dab740 1 client.29633 shutdown 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 msgr2=0x562112aef5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112aef5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bfd0 tx=0x7f5a20004610 comp rx=0 tx=0).stop 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 shutdown_connections 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112b063f0 0x562112aefd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b16640 0x562112b18a30 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112aef5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 --2- 172.21.15.122:0/955596080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112af2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.499+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 >> 172.21.15.122:0/955596080 conn(0x562112af3220 msgr2=0x562112af5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.500+0000 7f5a17fff640 0 client.29633 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:21.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.500+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 shutdown_connections 2024-04-02T13:53:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.500+0000 7f5a41dab740 1 -- 172.21.15.122:0/955596080 wait complete. 2024-04-02T13:53:21.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.500+0000 7f5a41dab740 20 client.29633 trim_cache size 0 max 16384 2024-04-02T13:53:21.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 Processor -- start 2024-04-02T13:53:21.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 -- start start 2024-04-02T13:53:21.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112b063f0 0x562112af2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381acec0 con 0x562112b063f0 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381aeac0 con 0x562112af8f90 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a41dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ad590 con 0x562112af84f0 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a37fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54848/0 (socket says 172.21.15.122:54848) 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a37fff640 1 -- 172.21.15.122:0/2873840445 learned_addr learned my addr 172.21.15.122:0/2873840445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.519+0000 7f5a3d765640 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112b063f0 0x562112af2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 -- 172.21.15.122:0/2873840445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 msgr2=0x562112af3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 -- 172.21.15.122:0/2873840445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112b063f0 msgr2=0x562112af2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112b063f0 0x562112af2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 -- 172.21.15.122:0/2873840445 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562112af7f20 con 0x562112af8f90 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a37fff640 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5a200987c0 tx=0x7f5a2009a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4245baffd6da16ee server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a2e7fc640 1 -- 172.21.15.122:0/2873840445 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20020070 con 0x562112af8f90 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a2e7fc640 1 -- 172.21.15.122:0/2873840445 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a200d8e90 con 0x562112af8f90 2024-04-02T13:53:21.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.520+0000 7f5a2e7fc640 1 -- 172.21.15.122:0/2873840445 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2009c130 con 0x562112af8f90 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- 172.21.15.122:0/2873840445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 msgr2=0x562112af1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5a200987c0 tx=0x7f5a2009a870 comp rx=0 tx=0).stop 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- 172.21.15.122:0/2873840445 shutdown_connections 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112b063f0 0x562112af2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112af8f90 0x562112af1d00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 --2- 172.21.15.122:0/2873840445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af84f0 0x562112af3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- 172.21.15.122:0/2873840445 >> 172.21.15.122:0/2873840445 conn(0x562112af3220 msgr2=0x562112b02980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- 172.21.15.122:0/2873840445 shutdown_connections 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- 172.21.15.122:0/2873840445 wait complete. 2024-04-02T13:53:21.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.521+0000 7f5a41dab740 1 -- start start 2024-04-02T13:53:21.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 Processor -- start 2024-04-02T13:53:21.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af8f90 0x562112afc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b189a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ae3f0 con 0x562112af84f0 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a3cf64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52662/0 (socket says 172.21.15.122:52662) 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381adc60 con 0x562112b063f0 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a3cf64640 1 -- 172.21.15.122:0/149645526 learned_addr learned my addr 172.21.15.122:0/149645526 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381aaec0 con 0x562112af8f90 2024-04-02T13:53:21.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a3d765640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b189a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.522+0000 7f5a41dab740 10 client.0 _ll_register_callbacks cb 0x7f5a28009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a37fff640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af8f90 0x562112afc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 -- 172.21.15.122:0/149645526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af8f90 msgr2=0x562112afc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af8f90 0x562112afc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 -- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 msgr2=0x562112b189a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b189a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562112b19150 con 0x562112af84f0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a3cf64640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009230 tx=0x7f5a300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab28a09fc3f2c95c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a17fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.523+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002c8c0 con 0x562112af84f0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.524+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a200d8e90 con 0x562112af84f0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.524+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f5a3002ca60 con 0x562112af84f0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.524+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5a3002cc60 con 0x562112af84f0 2024-04-02T13:53:21.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.524+0000 7f5a41dab740 10 client.20691 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.525+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562112b01750 con 0x562112af84f0 2024-04-02T13:53:21.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.525+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a30004ba0 con 0x562112af84f0 2024-04-02T13:53:21.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.525+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f5a30004ec0 con 0x562112af84f0 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 10 client.20691 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 10 client.20691 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 20 client.20691 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b01ea0 0x562112b767e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 4 client.20691 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.526+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5a200d8e90 con 0x562112b01ea0 2024-04-02T13:53:21.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.527+0000 7f5a37fff640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b01ea0 0x562112b767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.527+0000 7f5a17fff640 10 client.20691 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.527+0000 7f5a37fff640 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b01ea0 0x562112b767e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6e66bb0c7b97cf7 server_cookie=b845ae986c7f4bca in_seq=0 out_seq=0 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.528+0000 7f5a17fff640 1 -- 172.21.15.122:0/149645526 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5a200d8e90 con 0x562112b01ea0 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.528+0000 7f5a17fff640 10 client.20691 handle_command_reply: tid=0 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.528+0000 7f5a41dab740 1 client.20691 shutdown 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.528+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 msgr2=0x562112afc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009230 tx=0x7f5a300058d0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 shutdown_connections 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562112b01ea0 0x562112b767e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562112b063f0 0x562112b189a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562112af8f90 0x562112afc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 --2- 172.21.15.122:0/149645526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562112af84f0 0x562112afc040 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 >> 172.21.15.122:0/149645526 conn(0x562112af3220 msgr2=0x562112af3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a17fff640 0 client.20691 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:21.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 shutdown_connections 2024-04-02T13:53:21.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 1 -- 172.21.15.122:0/149645526 wait complete. 2024-04-02T13:53:21.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.529+0000 7f5a41dab740 20 client.20691 trim_cache size 0 max 16384 2024-04-02T13:53:21.534 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 msgr2=0x7f5a240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 0x7f5a240a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5a28011630 tx=0x7f5a28009500 comp rx=0 tx=0).stop 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 msgr2=0x7f5a381988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a381988d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a2000dde0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 shutdown_connections 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5a240a3d20 0x7f5a240a61e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a38107a90 0x7f5a3819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a38102440 0x7f5a381988d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 --2- 172.21.15.122:0/457885884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a38101a80 0x7f5a38198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.530+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 >> 172.21.15.122:0/457885884 conn(0x7f5a380fd570 msgr2=0x7f5a380fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.531+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 shutdown_connections 2024-04-02T13:53:21.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.531+0000 7f5a3f1c8640 1 -- 172.21.15.122:0/457885884 wait complete. 2024-04-02T13:53:21.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:53:21.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 -- 172.21.15.122:0/1717061618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c109040 msgr2=0x7f750c1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 --2- 172.21.15.122:0/1717061618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c109040 0x7f750c1094c0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f7508009ca0 tx=0x7f750802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 -- 172.21.15.122:0/1717061618 shutdown_connections 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 --2- 172.21.15.122:0/1717061618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 0x7f750c115730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 --2- 172.21.15.122:0/1717061618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c109040 0x7f750c1094c0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 --2- 172.21.15.122:0/1717061618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c0695f0 0x7f750c108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 -- 172.21.15.122:0/1717061618 >> 172.21.15.122:0/1717061618 conn(0x7f750c06d040 msgr2=0x7f750c06d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 -- 172.21.15.122:0/1717061618 shutdown_connections 2024-04-02T13:53:21.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.731+0000 7f7514462640 1 -- 172.21.15.122:0/1717061618 wait complete. 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 Processor -- start 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 -- start start 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c109040 0x7f750c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 0x7f750c1acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1182a0 con 0x7f750c0695f0 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f750c118120 con 0x7f750c109040 2024-04-02T13:53:21.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.732+0000 7f7514462640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f750c118420 con 0x7f750c113340 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52670/0 (socket says 172.21.15.122:52670) 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 -- 172.21.15.122:0/193333028 learned_addr learned my addr 172.21.15.122:0/193333028 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75119fd640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c109040 0x7f750c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75129ff640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 0x7f750c1acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 -- 172.21.15.122:0/193333028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 msgr2=0x7f750c1acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 0x7f750c1acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 -- 172.21.15.122:0/193333028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c109040 msgr2=0x7f750c11a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c109040 0x7f750c11a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 -- 172.21.15.122:0/193333028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c1ad3d0 con 0x7f750c0695f0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.733+0000 7f75121fe640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.734+0000 7f74ff7fe640 1 -- 172.21.15.122:0/193333028 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750400cd80 con 0x7f750c0695f0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.734+0000 7f74ff7fe640 1 -- 172.21.15.122:0/193333028 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504010910 con 0x7f750c0695f0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.734+0000 7f74ff7fe640 1 -- 172.21.15.122:0/193333028 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750400cd80 con 0x7f750c0695f0 2024-04-02T13:53:21.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.734+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c1ad6c0 con 0x7f750c0695f0 2024-04-02T13:53:21.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.734+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c1b3d50 con 0x7f750c0695f0 2024-04-02T13:53:21.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.737+0000 7f74ff7fe640 1 -- 172.21.15.122:0/193333028 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7504010ac0 con 0x7f750c0695f0 2024-04-02T13:53:21.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.737+0000 7f7517045740 1 Processor -- start 2024-04-02T13:53:21.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.738+0000 7f7517045740 1 -- start start 2024-04-02T13:53:21.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.738+0000 7f74ff7fe640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 0x7f74f80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.738+0000 7f74ff7fe640 1 -- 172.21.15.122:0/193333028 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f75040d0db0 con 0x7f750c0695f0 2024-04-02T13:53:21.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.738+0000 7f75129ff640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 0x7f74f80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613e953d7b0 con 0x5613e95e2ca0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5613e94dc570 con 0x5613e95d97a0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f7517045740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5613e9539760 con 0x5613e95da0c0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54884/0 (socket says 172.21.15.122:54884) 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 -- 172.21.15.122:0/1276812123 learned_addr learned my addr 172.21.15.122:0/1276812123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75129ff640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 0x7f74f80a63e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f750c06b1b0 tx=0x7f7500005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75119fd640 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 -- 172.21.15.122:0/1276812123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 msgr2=0x5613e95e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 -- 172.21.15.122:0/1276812123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 msgr2=0x5613e95e5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.739+0000 7f75121fe640 1 -- 172.21.15.122:0/1276812123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e92513f0 con 0x5613e95d97a0 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.740+0000 7f75121fe640 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7504099330 tx=0x7f750409d1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=630f306b682c6dc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.740+0000 7f74fcbfe640 1 -- 172.21.15.122:0/1276812123 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75040ce390 con 0x5613e95d97a0 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.740+0000 7f74fcbfe640 1 -- 172.21.15.122:0/1276812123 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504098fd0 con 0x5613e95d97a0 2024-04-02T13:53:21.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.740+0000 7f74fcbfe640 1 -- 172.21.15.122:0/1276812123 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750409bfd0 con 0x5613e95d97a0 2024-04-02T13:53:21.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 -- 172.21.15.122:0/1276812123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 msgr2=0x5613e95d9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7504099330 tx=0x7f750409d1a0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 -- 172.21.15.122:0/1276812123 shutdown_connections 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 --2- 172.21.15.122:0/1276812123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95d9b80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 -- 172.21.15.122:0/1276812123 >> 172.21.15.122:0/1276812123 conn(0x5613e95d3ed0 msgr2=0x5613e95d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 -- 172.21.15.122:0/1276812123 shutdown_connections 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.741+0000 7f7517045740 1 -- 172.21.15.122:0/1276812123 wait complete. 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 -- start start 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 Processor -- start 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613e95e5a90 con 0x5613e95e2ca0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5613e94f8e80 con 0x5613e95d97a0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5613e95e5c10 con 0x5613e95da0c0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f75121fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f75129ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f75121fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54898/0 (socket says 172.21.15.122:54898) 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f75121fe640 1 -- 172.21.15.122:0/3825723869 learned_addr learned my addr 172.21.15.122:0/3825723869 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f7517045740 10 client.0 _ll_register_callbacks cb 0x5613e9534aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.742+0000 7f75119fd640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 -- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 msgr2=0x5613e95dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95dfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 -- 172.21.15.122:0/3825723869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 msgr2=0x5613e95e0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 -- 172.21.15.122:0/3825723869 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e931b7d0 con 0x5613e95d97a0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f75121fe640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f75040d10d0 tx=0x7f7504087320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff8db96364defdd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f74f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75040b5030 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f750409bfc0 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.743+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7504098fd0 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f75040bbb20 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 10 client.29651 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5613e95f1b20 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504098fd0 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f75040986b0 con 0x5613e95d97a0 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 10 client.29651 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 10 client.29651 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:53:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 20 client.29651 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e95f5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 4 client.29651 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.744+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613e954acf0 con 0x5613e95f2e70 2024-04-02T13:53:21.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.745+0000 7f75119fd640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e95f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.746+0000 7f74f5ffb640 10 client.29651 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.746+0000 7f75119fd640 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e95f5260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f952a4f05c4e90e6 server_cookie=54319f073aba2ba1 in_seq=0 out_seq=0 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f74f5ffb640 1 -- 172.21.15.122:0/3825723869 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5613e954acf0 con 0x5613e95f2e70 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f74f5ffb640 10 client.29651 handle_command_reply: tid=0 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 client.29651 shutdown 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 msgr2=0x5613e95e2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f75040d10d0 tx=0x7f7504087320 comp rx=0 tx=0).stop 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 shutdown_connections 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e95f5260 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.748+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95e2ca0 0x5613e95e0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f74f5ffb640 0 client.29651 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95da0c0 0x5613e95dfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 1 --2- 172.21.15.122:0/3825723869 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95d97a0 0x5613e95e2760 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 >> 172.21.15.122:0/3825723869 conn(0x5613e95d3ed0 msgr2=0x5613e95d5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 shutdown_connections 2024-04-02T13:53:21.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 1 -- 172.21.15.122:0/3825723869 wait complete. 2024-04-02T13:53:21.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.749+0000 7f7517045740 20 client.29651 trim_cache size 0 max 16384 2024-04-02T13:53:21.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.771+0000 7f7517045740 1 Processor -- start 2024-04-02T13:53:21.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.771+0000 7f7517045740 1 -- start start 2024-04-02T13:53:21.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 0x5613e95e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 0x5613e95e2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1bdfb0 con 0x5613e95da0c0 2024-04-02T13:53:21.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1bfbb0 con 0x5613e95e2ca0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f7517045740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1be680 con 0x5613e95d97a0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f75129ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 0x5613e95e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f75121fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f75129ff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 0x5613e95e2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54904/0 (socket says 172.21.15.122:54904) 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f75129ff640 1 -- 172.21.15.122:0/1235423595 learned_addr learned my addr 172.21.15.122:0/1235423595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.772+0000 7f75121fe640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47720/0 (socket says 172.21.15.122:47720) 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75119fd640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 0x5613e95e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 -- 172.21.15.122:0/1235423595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 msgr2=0x5613e95e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 0x5613e95e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 -- 172.21.15.122:0/1235423595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 msgr2=0x5613e95e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 0x5613e95e2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 -- 172.21.15.122:0/1235423595 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613e95d9ef0 con 0x5613e95d97a0 2024-04-02T13:53:21.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75119fd640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 0x5613e95e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f75121fe640 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75040ce710 tx=0x7f7504009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a146394e0da22d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f74f7fff640 1 -- 172.21.15.122:0/1235423595 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75040b5050 con 0x5613e95d97a0 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f74f7fff640 1 -- 172.21.15.122:0/1235423595 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75040b8030 con 0x5613e95d97a0 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.773+0000 7f74f7fff640 1 -- 172.21.15.122:0/1235423595 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750409c0d0 con 0x5613e95d97a0 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 -- 172.21.15.122:0/1235423595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 msgr2=0x5613e95ee950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f75040ce710 tx=0x7f7504009770 comp rx=0 tx=0).stop 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 -- 172.21.15.122:0/1235423595 shutdown_connections 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95e2ca0 0x5613e95e2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95da0c0 0x5613e95e2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 --2- 172.21.15.122:0/1235423595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95d97a0 0x5613e95ee950 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.774+0000 7f7517045740 1 -- 172.21.15.122:0/1235423595 >> 172.21.15.122:0/1235423595 conn(0x5613e95d3ed0 msgr2=0x5613e95d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 -- 172.21.15.122:0/1235423595 shutdown_connections 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 -- 172.21.15.122:0/1235423595 wait complete. 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 -- start start 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 Processor -- start 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 0x5613e95e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.775+0000 7f7517045740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95e2ca0 0x5613e95ea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52708/0 (socket says 172.21.15.122:52708) 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 -- 172.21.15.122:0/608208756 learned_addr learned my addr 172.21.15.122:0/608208756 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1bf4e0 con 0x5613e95d97a0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1bed50 con 0x5613e95da0c0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f750c1bbfb0 con 0x5613e95e2ca0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75129ff640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95e2ca0 0x5613e95ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75119fd640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 0x5613e95e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f7517045740 10 client.0 _ll_register_callbacks cb 0x7f750409c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 -- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95e2ca0 msgr2=0x5613e95ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95e2ca0 0x5613e95ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 -- 172.21.15.122:0/608208756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 msgr2=0x5613e95e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 0x5613e95e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613e95eaed0 con 0x5613e95d97a0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75040b8030 con 0x5613e95d97a0 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75119fd640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 0x5613e95e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:21.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f75121fe640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f7504098aa0 tx=0x7f750409c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee89b84b6d5db0e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.776+0000 7f74f5ffb640 10 client.20703 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f74f5ffb640 1 -- 172.21.15.122:0/608208756 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75040b5040 con 0x5613e95d97a0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f74f5ffb640 1 -- 172.21.15.122:0/608208756 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f7504087300 con 0x5613e95d97a0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f74f5ffb640 1 -- 172.21.15.122:0/608208756 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f7504098fd0 con 0x5613e95d97a0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 10 client.20703 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=73}) -- 0x5613e95eaed0 con 0x5613e95d97a0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f74f5ffb640 1 -- 172.21.15.122:0/608208756 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f7504010430 con 0x5613e95d97a0 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 10 client.20703 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 10 client.20703 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:53:21.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 20 client.20703 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e965acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 4 client.20703 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.777+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613e9545040 con 0x5613e95f2e70 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.778+0000 7f75119fd640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e965acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.779+0000 7f74f5ffb640 10 client.20703 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.779+0000 7f75119fd640 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e965acd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a565ca4765f1845e server_cookie=7452b8f4749053bd in_seq=0 out_seq=0 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.780+0000 7f74f5ffb640 1 -- 172.21.15.122:0/608208756 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613e9545040 con 0x5613e95f2e70 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.780+0000 7f74f5ffb640 10 client.20703 handle_command_reply: tid=0 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.780+0000 7f7517045740 1 client.20703 shutdown 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.780+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 msgr2=0x5613e95e6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.780+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f7504098aa0 tx=0x7f750409c1f0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 shutdown_connections 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x5613e95f2e70 0x5613e965acd0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5613e95e2ca0 0x5613e95ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f74f5ffb640 0 client.20703 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5613e95da0c0 0x5613e95e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 --2- 172.21.15.122:0/608208756 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613e95d97a0 0x5613e95e6460 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 >> 172.21.15.122:0/608208756 conn(0x5613e95d3ed0 msgr2=0x5613e95de380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 shutdown_connections 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 1 -- 172.21.15.122:0/608208756 wait complete. 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.781+0000 7f7517045740 20 client.20703 trim_cache size 0 max 16384 2024-04-02T13:53:21.787 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 msgr2=0x7f74f80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 0x7f74f80a63e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f750c06b1b0 tx=0x7f7500005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 msgr2=0x7f750c119c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9f0 tx=0x7f750400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 shutdown_connections 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f74f80a3f20 0x7f74f80a63e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f750c113340 0x7f750c1acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f750c109040 0x7f750c11a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 --2- 172.21.15.122:0/193333028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f750c0695f0 0x7f750c119c60 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 >> 172.21.15.122:0/193333028 conn(0x7f750c06d040 msgr2=0x7f750c072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 shutdown_connections 2024-04-02T13:53:21.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.782+0000 7f7514462640 1 -- 172.21.15.122:0/193333028 wait complete. 2024-04-02T13:53:21.803 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- 172.21.15.122:0/1972025456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 msgr2=0x7f270010e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 --2- 172.21.15.122:0/1972025456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f270010e9a0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009900 tx=0x7f26f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- 172.21.15.122:0/1972025456 shutdown_connections 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 --2- 172.21.15.122:0/1972025456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f270010e9a0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 --2- 172.21.15.122:0/1972025456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100bc0 0x7f27001066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 --2- 172.21.15.122:0/1972025456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2700100210 0x7f27001005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- 172.21.15.122:0/1972025456 >> 172.21.15.122:0/1972025456 conn(0x7f2700076f20 msgr2=0x7f27000fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- 172.21.15.122:0/1972025456 shutdown_connections 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- 172.21.15.122:0/1972025456 wait complete. 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 Processor -- start 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.941+0000 7f2705911640 1 -- start start 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 0x7f2700198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270019ccb0 0x7f270019d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2700111e80 con 0x7f2700106dd0 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2700111d00 con 0x7f2700100210 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f2705911640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2700112000 con 0x7f270019ccb0 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52720/0 (socket says 172.21.15.122:52720) 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26feffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 0x7f2700198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 -- 172.21.15.122:0/3845866950 learned_addr learned my addr 172.21.15.122:0/3845866950 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270019ccb0 0x7f270019d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 -- 172.21.15.122:0/3845866950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270019ccb0 msgr2=0x7f270019d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270019ccb0 0x7f270019d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 -- 172.21.15.122:0/3845866950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 msgr2=0x7f2700198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 0x7f2700198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 -- 172.21.15.122:0/3845866950 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270019d7a0 con 0x7f2700106dd0 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26feffd640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 0x7f2700198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.942+0000 7f26f7fff640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f26e800e9a0 tx=0x7f26e800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.943+0000 7f270490f640 1 -- 172.21.15.122:0/3845866950 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800cd80 con 0x7f2700106dd0 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.943+0000 7f270490f640 1 -- 172.21.15.122:0/3845866950 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e80035f0 con 0x7f2700106dd0 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.943+0000 7f270490f640 1 -- 172.21.15.122:0/3845866950 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8010670 con 0x7f2700106dd0 2024-04-02T13:53:21.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.943+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270019da90 con 0x7f2700106dd0 2024-04-02T13:53:21.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.943+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2700101380 con 0x7f2700106dd0 2024-04-02T13:53:21.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.946+0000 7f270490f640 1 -- 172.21.15.122:0/3845866950 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f26e80037a0 con 0x7f2700106dd0 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.947+0000 7f270490f640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 0x7f26ec0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.947+0000 7f27085ce740 1 Processor -- start 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.947+0000 7f26feffd640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 0x7f26ec0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.947+0000 7f270490f640 1 -- 172.21.15.122:0/3845866950 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f26e80d0df0 con 0x7f2700106dd0 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.947+0000 7f27085ce740 1 -- start start 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d15db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d169090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e04d0c17b0 con 0x55e04d15d7a0 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e04d060570 con 0x55e04d15e0c0 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f27085ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e04d0bd760 con 0x55e04d166ca0 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d15db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54956/0 (socket says 172.21.15.122:54956) 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d15db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52722/0 (socket says 172.21.15.122:52722) 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 -- 172.21.15.122:0/942484176 learned_addr learned my addr 172.21.15.122:0/942484176 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26feffd640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 0x7f26ec0a61d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005fd0 tx=0x7f26f0009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 -- 172.21.15.122:0/942484176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 msgr2=0x55e04d169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 -- 172.21.15.122:0/942484176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 msgr2=0x55e04d15db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d15db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.948+0000 7f26f7fff640 1 -- 172.21.15.122:0/942484176 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04cdd53f0 con 0x55e04d15e0c0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.949+0000 7f26f7fff640 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26e80a5080 tx=0x7f26e809c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9233734c96e1db1f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.949+0000 7f26f5ffb640 1 -- 172.21.15.122:0/942484176 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e809c940 con 0x55e04d15e0c0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.949+0000 7f26f5ffb640 1 -- 172.21.15.122:0/942484176 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e80a0cf0 con 0x55e04d15e0c0 2024-04-02T13:53:21.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.949+0000 7f26f5ffb640 1 -- 172.21.15.122:0/942484176 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80870c0 con 0x55e04d15e0c0 2024-04-02T13:53:21.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 -- 172.21.15.122:0/942484176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 msgr2=0x55e04d166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26e80a5080 tx=0x7f26e809c480 comp rx=0 tx=0).stop 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 -- 172.21.15.122:0/942484176 shutdown_connections 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15e0c0 0x55e04d166760 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f26e80a5080 tx=0x7f26e809c480 comp rx=0 tx=0).stop 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 --2- 172.21.15.122:0/942484176 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d15db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 -- 172.21.15.122:0/942484176 >> 172.21.15.122:0/942484176 conn(0x55e04d157ed0 msgr2=0x55e04d1582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 -- 172.21.15.122:0/942484176 shutdown_connections 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.950+0000 7f27085ce740 1 -- 172.21.15.122:0/942484176 wait complete. 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.951+0000 7f27085ce740 1 -- start start 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.951+0000 7f27085ce740 1 Processor -- start 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.951+0000 7f27085ce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d163c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.951+0000 7f27085ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d163c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52736/0 (socket says 172.21.15.122:52736) 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f27085ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d172ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f27085ce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e04d169a30 con 0x55e04d15d7a0 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54972/0 (socket says 172.21.15.122:54972) 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26feffd640 1 -- 172.21.15.122:0/4062450429 learned_addr learned my addr 172.21.15.122:0/4062450429 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e04d07ce80 con 0x55e04d166ca0 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e04d169bb0 con 0x55e04d164680 2024-04-02T13:53:21.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f27085ce740 10 client.0 _ll_register_callbacks cb 0x55e04d0b8aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d172ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 -- 172.21.15.122:0/4062450429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 msgr2=0x55e04d172ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d172ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 -- 172.21.15.122:0/4062450429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 msgr2=0x55e04d163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26f7fff640 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04ce9f7d0 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.952+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d172ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e04d158dd0 tx=0x7f26e8003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c474c022883ac12 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26deffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e809c940 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f26e8087260 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e80a0cf0 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f26e809bd90 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f27085ce740 10 client.29669 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55e04d169a30 con 0x55e04d166ca0 2024-04-02T13:53:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.953+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26e809bf30 con 0x55e04d166ca0 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f26e8098820 con 0x55e04d166ca0 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 10 client.29669 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 10 client.29669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 20 client.29669 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1771b0 0x55e04d1795a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 4 client.29669 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f26e80a0cf0 con 0x55e04d1771b0 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f26feffd640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1771b0 0x55e04d1795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.954+0000 7f26deffd640 10 client.29669 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:21.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.955+0000 7f26feffd640 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1771b0 0x55e04d1795a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c17cb2e2bda99f2 server_cookie=20b926c1b45add4c in_seq=0 out_seq=0 2024-04-02T13:53:21.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f26deffd640 1 -- 172.21.15.122:0/4062450429 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f26e80a0cf0 con 0x55e04d1771b0 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f26deffd640 10 client.29669 handle_command_reply: tid=0 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 client.29669 shutdown 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 msgr2=0x55e04d164140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e04d158dd0 tx=0x7f26e8003d40 comp rx=0 tx=0).stop 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 shutdown_connections 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1771b0 0x55e04d1795a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d172ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d166ca0 0x55e04d164140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f26deffd640 0 client.29669 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 --2- 172.21.15.122:0/4062450429 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d15d7a0 0x55e04d163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 >> 172.21.15.122:0/4062450429 conn(0x55e04d157ed0 msgr2=0x55e04d159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 shutdown_connections 2024-04-02T13:53:21.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 1 -- 172.21.15.122:0/4062450429 wait complete. 2024-04-02T13:53:21.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.957+0000 7f27085ce740 20 client.29669 trim_cache size 0 max 16384 2024-04-02T13:53:21.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.976+0000 7f27085ce740 1 Processor -- start 2024-04-02T13:53:21.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.976+0000 7f27085ce740 1 -- start start 2024-04-02T13:53:21.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d17f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 0x55e04d163ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b5810 con 0x55e04d166ca0 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b7410 con 0x55e04d15d7a0 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f27085ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b5ee0 con 0x55e04d164680 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54978/0 (socket says 172.21.15.122:54978) 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 -- 172.21.15.122:0/512458519 learned_addr learned my addr 172.21.15.122:0/512458519 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26f7fff640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d17f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 0x55e04d163ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 -- 172.21.15.122:0/512458519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 msgr2=0x55e04d17f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d17f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 -- 172.21.15.122:0/512458519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 msgr2=0x55e04d163ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 0x55e04d163ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 -- 172.21.15.122:0/512458519 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e04d162180 con 0x55e04d15d7a0 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 0x55e04d163ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.977+0000 7f26feffd640 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f26f0013010 tx=0x7f26f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c966e518defd634e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f26f57fa640 1 -- 172.21.15.122:0/512458519 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0004700 con 0x55e04d15d7a0 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f26f57fa640 1 -- 172.21.15.122:0/512458519 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f002a890 con 0x55e04d15d7a0 2024-04-02T13:53:21.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f26f57fa640 1 -- 172.21.15.122:0/512458519 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0029c50 con 0x55e04d15d7a0 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 -- 172.21.15.122:0/512458519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 msgr2=0x55e04d17ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f26f0013010 tx=0x7f26f0014000 comp rx=0 tx=0).stop 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 -- 172.21.15.122:0/512458519 shutdown_connections 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d166ca0 0x55e04d163ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d164680 0x55e04d17f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.978+0000 7f27085ce740 1 --2- 172.21.15.122:0/512458519 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d17ec90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- 172.21.15.122:0/512458519 >> 172.21.15.122:0/512458519 conn(0x55e04d157ed0 msgr2=0x55e04d163490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- 172.21.15.122:0/512458519 shutdown_connections 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- 172.21.15.122:0/512458519 wait complete. 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- start start 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 Processor -- start 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d164680 0x55e04d16ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d16b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b6d40 con 0x55e04d164680 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b65b0 con 0x55e04d15d7a0 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.979+0000 7f27085ce740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27001b3810 con 0x55e04d166ca0 2024-04-02T13:53:21.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54982/0 (socket says 172.21.15.122:54982) 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 -- 172.21.15.122:0/4041033343 learned_addr learned my addr 172.21.15.122:0/4041033343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d164680 0x55e04d16ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f27085ce740 10 client.0 _ll_register_callbacks cb 0x7f26e809c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26ff7fe640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d16b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 -- 172.21.15.122:0/4041033343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 msgr2=0x55e04d16b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d16b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 -- 172.21.15.122:0/4041033343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d164680 msgr2=0x55e04d16ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d164680 0x55e04d16ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 -- 172.21.15.122:0/4041033343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e04d1dc500 con 0x55e04d15d7a0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26feffd640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005c40 tx=0x7f26f000fd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4bd69d48bfb330 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26deffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:21.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f002a3b0 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f26f002a550 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.980+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f002a890 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.981+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f26f0040050 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.981+0000 7f27085ce740 10 client.29675 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.981+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55e04d1dd780 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.981+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26f002a890 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f26f003b030 con 0x55e04d15d7a0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 10 client.29675 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 10 client.29675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 20 client.29675 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1ddde0 0x55e04d1e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 4 client.29675 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e04d0cecf0 con 0x55e04d1ddde0 2024-04-02T13:53:21.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1ddde0 0x55e04d1e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f26f7fff640 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1ddde0 0x55e04d1e01d0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9eb95336b75daff server_cookie=60bfdad32470e57a in_seq=0 out_seq=0 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.982+0000 7f26deffd640 10 client.29675 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f26deffd640 1 -- 172.21.15.122:0/4041033343 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e04d0cecf0 con 0x55e04d1ddde0 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f26deffd640 10 client.29675 handle_command_reply: tid=0 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 client.29675 shutdown 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 msgr2=0x55e04d16a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005c40 tx=0x7f26f000fd60 comp rx=0 tx=0).stop 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 shutdown_connections 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55e04d1ddde0 0x55e04d1e01d0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e04d166ca0 0x55e04d16b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e04d164680 0x55e04d16ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f26deffd640 0 client.29675 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 --2- 172.21.15.122:0/4041033343 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e04d15d7a0 0x55e04d16a7d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 >> 172.21.15.122:0/4041033343 conn(0x55e04d157ed0 msgr2=0x55e04d156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 shutdown_connections 2024-04-02T13:53:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.983+0000 7f27085ce740 1 -- 172.21.15.122:0/4041033343 wait complete. 2024-04-02T13:53:21.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f27085ce740 20 client.29675 trim_cache size 0 max 16384 2024-04-02T13:53:21.993 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 msgr2=0x7f26ec0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 0x7f26ec0a61d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f26f0005fd0 tx=0x7f26f0009500 comp rx=0 tx=0).stop 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 msgr2=0x7f2700198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f26e800e9a0 tx=0x7f26e800ee60 comp rx=0 tx=0).stop 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 shutdown_connections 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f26ec0a3d10 0x7f26ec0a61d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f270019ccb0 0x7f270019d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2700106dd0 0x7f2700198980 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 --2- 172.21.15.122:0/3845866950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2700100210 0x7f2700198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 >> 172.21.15.122:0/3845866950 conn(0x7f2700076f20 msgr2=0x7f2700104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 shutdown_connections 2024-04-02T13:53:21.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:21.984+0000 7f2705911640 1 -- 172.21.15.122:0/3845866950 wait complete. 2024-04-02T13:53:22.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:21 smithi122 ceph-mon[32114]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:22.000 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:21 smithi122 ceph-mon[32114]: osdmap e119: 12 total, 12 up, 12 in 2024-04-02T13:53:22.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:21 smithi187 ceph-mon[36064]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:22.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:21 smithi187 ceph-mon[36064]: osdmap e119: 12 total, 12 up, 12 in 2024-04-02T13:53:22.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:21 smithi188 ceph-mon[36117]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 13 MiB/s wr, 683 op/s 2024-04-02T13:53:22.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:21 smithi188 ceph-mon[36117]: osdmap e119: 12 total, 12 up, 12 in 2024-04-02T13:53:22.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 149626 27.91 MB/sec execute 162 sec latency 81.771 ms 2024-04-02T13:53:23.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:53:23.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:23.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3962971713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4100970 msgr2=0x7f04f4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3962971713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4100970 0x7f04f4100df0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009930 tx=0x7f04f002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3962971713 shutdown_connections 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3962971713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4101330 0x7f04f410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3962971713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4100970 0x7f04f4100df0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3962971713 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4106980 0x7f04f4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3962971713 >> 172.21.15.122:0/3962971713 conn(0x7f04f4077b00 msgr2=0x7f04f4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3962971713 shutdown_connections 2024-04-02T13:53:23.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.150+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3962971713 wait complete. 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.151+0000 7f04fbfc8640 1 Processor -- start 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.151+0000 7f04fbfc8640 1 -- start start 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4100970 0x7f04f410e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 0x7f04f410ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4114060 con 0x7f04f4106980 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04f4113ee0 con 0x7f04f4101330 2024-04-02T13:53:23.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fbfc8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41141e0 con 0x7f04f4100970 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04f9d64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4100970 0x7f04f410e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04f9563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 0x7f04f410ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04f9563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:55002/0 (socket says 172.21.15.122:55002) 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04f9563640 1 -- 172.21.15.122:0/3754255150 learned_addr learned my addr 172.21.15.122:0/3754255150 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 -- 172.21.15.122:0/3754255150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4100970 msgr2=0x7f04f410e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4100970 0x7f04f410e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 -- 172.21.15.122:0/3754255150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 msgr2=0x7f04f410ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04fa565640 1 -- 172.21.15.122:0/3754255150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f41aac50 con 0x7f04f4106980 2024-04-02T13:53:23.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.152+0000 7f04f9563640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.153+0000 7f04fa565640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 0x7f04f410ef80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ea80 tx=0x7f04ec00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.154+0000 7f04e6ffd640 1 -- 172.21.15.122:0/3754255150 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00cea0 con 0x7f04f4106980 2024-04-02T13:53:23.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.154+0000 7f04e6ffd640 1 -- 172.21.15.122:0/3754255150 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04ec005020 con 0x7f04f4106980 2024-04-02T13:53:23.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.154+0000 7f04e6ffd640 1 -- 172.21.15.122:0/3754255150 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0106d0 con 0x7f04f4106980 2024-04-02T13:53:23.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.154+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f41aaf40 con 0x7f04f4106980 2024-04-02T13:53:23.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.154+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f41ab3b0 con 0x7f04f4106980 2024-04-02T13:53:23.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.158+0000 7f04e6ffd640 1 -- 172.21.15.122:0/3754255150 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f04ec005540 con 0x7f04f4106980 2024-04-02T13:53:23.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.158+0000 7f04e6ffd640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 0x7f04e00a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.159+0000 7f04e6ffd640 1 -- 172.21.15.122:0/3754255150 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f04ec0d11b0 con 0x7f04f4106980 2024-04-02T13:53:23.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.159+0000 7f04f9d64640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 0x7f04e00a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 Processor -- start 2024-04-02T13:53:23.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04f9d64640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 0x7f04e00a6250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009b50 tx=0x7f04e8005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 -- start start 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba06edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557ba0646770 con 0x557ba06e2be0 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557ba05ee960 con 0x557ba06eb7c0 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04febaa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557ba06428c0 con 0x557ba06e22c0 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04f9563640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04f9563640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52784/0 (socket says 172.21.15.122:52784) 2024-04-02T13:53:23.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04f9563640 1 -- 172.21.15.122:0/501176087 learned_addr learned my addr 172.21.15.122:0/501176087 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.160+0000 7f04f9d64640 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04fa565640 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba06edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 -- 172.21.15.122:0/501176087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 msgr2=0x557ba06e26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06e26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 -- 172.21.15.122:0/501176087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 msgr2=0x557ba06edbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba06edbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 -- 172.21.15.122:0/501176087 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba04227e0 con 0x557ba06e2be0 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04f9563640 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009230 tx=0x7f04f0009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2bfee9522c8ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.161+0000 7f04dffff640 1 -- 172.21.15.122:0/501176087 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0042070 con 0x557ba06e2be0 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.162+0000 7f04dffff640 1 -- 172.21.15.122:0/501176087 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0004a70 con 0x557ba06e2be0 2024-04-02T13:53:23.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.162+0000 7f04dffff640 1 -- 172.21.15.122:0/501176087 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00051e0 con 0x557ba06e2be0 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- 172.21.15.122:0/501176087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 msgr2=0x557ba06eb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009230 tx=0x7f04f0009670 comp rx=0 tx=0).stop 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- 172.21.15.122:0/501176087 shutdown_connections 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba06edbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06eb280 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 --2- 172.21.15.122:0/501176087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06e26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- 172.21.15.122:0/501176087 >> 172.21.15.122:0/501176087 conn(0x557ba06dc9f0 msgr2=0x557ba06dcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- 172.21.15.122:0/501176087 shutdown_connections 2024-04-02T13:53:23.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- 172.21.15.122:0/501176087 wait complete. 2024-04-02T13:53:23.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 -- start start 2024-04-02T13:53:23.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.163+0000 7f04febaa740 1 Processor -- start 2024-04-02T13:53:23.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba06e8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06e9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04f9d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba06e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04f9563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04f9563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:55032/0 (socket says 172.21.15.122:55032) 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04f9d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba06e8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52790/0 (socket says 172.21.15.122:52790) 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04f9563640 1 -- 172.21.15.122:0/2429722168 learned_addr learned my addr 172.21.15.122:0/2429722168 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557ba06ee550 con 0x557ba06e22c0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557ba0601d20 con 0x557ba06e2be0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.164+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557ba06ee6d0 con 0x557ba06eb7c0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04febaa740 10 client.0 _ll_register_callbacks cb 0x557ba05cea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04fa565640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 -- 172.21.15.122:0/2429722168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 msgr2=0x557ba06e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 -- 172.21.15.122:0/2429722168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 msgr2=0x557ba06e8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba06e8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba0482f70 con 0x557ba06e2be0 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.165+0000 7f04f9563640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009430 tx=0x7f04f0004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933e83220d2eb068 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0042050 con 0x557ba06e2be0 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f0004a70 con 0x557ba06e2be0 2024-04-02T13:53:23.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f04f0005190 con 0x557ba06e2be0 2024-04-02T13:53:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f04f0005390 con 0x557ba06e2be0 2024-04-02T13:53:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04febaa740 10 client.29687 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.166+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557ba031a170 con 0x557ba06e2be0 2024-04-02T13:53:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0004a70 con 0x557ba06e2be0 2024-04-02T13:53:23.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f04f0046400 con 0x557ba06e2be0 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 10 client.29687 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 10 client.29687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 20 client.29687 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba06fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 4 client.29687 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ba0653cb0 con 0x557ba06fba00 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.167+0000 7f04f9d64640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba06fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.168+0000 7f04f9d64640 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba06fddf0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e8e3d5e9ea2327 server_cookie=b1612a057e99b93a in_seq=0 out_seq=0 2024-04-02T13:53:23.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.168+0000 7f04dd7fa640 10 client.29687 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2429722168 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ba0653cb0 con 0x557ba06fba00 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04dd7fa640 10 client.29687 handle_command_reply: tid=0 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 client.29687 shutdown 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 msgr2=0x557ba06e8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009430 tx=0x7f04f0004ec0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 shutdown_connections 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba06fddf0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04dd7fa640 0 client.29687 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba06e8dd0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 --2- 172.21.15.122:0/2429722168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba06e8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.169+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 >> 172.21.15.122:0/2429722168 conn(0x557ba06dc9f0 msgr2=0x557ba06de740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.170+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 shutdown_connections 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.170+0000 7f04febaa740 1 -- 172.21.15.122:0/2429722168 wait complete. 2024-04-02T13:53:23.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.170+0000 7f04febaa740 20 client.29687 trim_cache size 0 max 16384 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 Processor -- start 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 -- start start 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06eafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba07033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b55f0 con 0x557ba06e2be0 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b71f0 con 0x557ba06eb7c0 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04febaa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b5cc0 con 0x557ba06e22c0 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52800/0 (socket says 172.21.15.122:52800) 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04fa565640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba07033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 -- 172.21.15.122:0/1129930305 learned_addr learned my addr 172.21.15.122:0/1129930305 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9d64640 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06eafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 -- 172.21.15.122:0/1129930305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 msgr2=0x557ba06eafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06eafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 -- 172.21.15.122:0/1129930305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 msgr2=0x557ba07033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba07033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.198+0000 7f04f9563640 1 -- 172.21.15.122:0/1129930305 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ba06e2a10 con 0x557ba06e2be0 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.199+0000 7f04fa565640 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba07033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.199+0000 7f04f9563640 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04f0046a10 tx=0x7f04f0005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a2f3947bc0c0c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.199+0000 7f04df7fe640 1 -- 172.21.15.122:0/1129930305 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0042070 con 0x557ba06e2be0 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.199+0000 7f04df7fe640 1 -- 172.21.15.122:0/1129930305 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0004a70 con 0x557ba06e2be0 2024-04-02T13:53:23.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.199+0000 7f04df7fe640 1 -- 172.21.15.122:0/1129930305 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0046400 con 0x557ba06e2be0 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 -- 172.21.15.122:0/1129930305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 msgr2=0x557ba06f7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f04f0046a10 tx=0x7f04f0005b90 comp rx=0 tx=0).stop 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 -- 172.21.15.122:0/1129930305 shutdown_connections 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06eb7c0 0x557ba07033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e2be0 0x557ba06f7410 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 --2- 172.21.15.122:0/1129930305 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06e22c0 0x557ba06eafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 -- 172.21.15.122:0/1129930305 >> 172.21.15.122:0/1129930305 conn(0x557ba06dc9f0 msgr2=0x557ba06e90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 -- 172.21.15.122:0/1129930305 shutdown_connections 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.200+0000 7f04febaa740 1 -- 172.21.15.122:0/1129930305 wait complete. 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 -- start start 2024-04-02T13:53:23.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 Processor -- start 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba075a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b6b20 con 0x557ba06e22c0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b6390 con 0x557ba06e2be0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04f41b35f0 con 0x557ba06eb7c0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04febaa740 10 client.0 _ll_register_callbacks cb 0x7f04ec099000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.201+0000 7f04f9563640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba075a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52802/0 (socket says 172.21.15.122:52802) 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 -- 172.21.15.122:0/2313690490 learned_addr learned my addr 172.21.15.122:0/2313690490 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 -- 172.21.15.122:0/2313690490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 msgr2=0x557ba06efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04fa565640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 -- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 msgr2=0x557ba075a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba075a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 -- 172.21.15.122:0/2313690490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ba06f0360 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04fa565640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9563640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba075a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04f9d64640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f04e80118d0 tx=0x7f04e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e19f761bd196b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e802a910 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f04e801e420 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.202+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f0004a70 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f04e803b050 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 10 client.20739 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557ba07628d0 con 0x557ba06e22c0 2024-04-02T13:53:23.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04f0004a70 con 0x557ba06e22c0 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f04e8032740 con 0x557ba06e22c0 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 10 client.20739 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 10 client.20739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 20 client.20739 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba07650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 4 client.20739 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.203+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557ba064e000 con 0x557ba06fba00 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.204+0000 7f04f9563640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba07650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.205+0000 7f04dd7fa640 10 client.20739 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.205+0000 7f04f9563640 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba07650c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=664a09607434c890 server_cookie=466cec049843d5a3 in_seq=0 out_seq=0 2024-04-02T13:53:23.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.206+0000 7f04dd7fa640 1 -- 172.21.15.122:0/2313690490 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557ba064e000 con 0x557ba06fba00 2024-04-02T13:53:23.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.206+0000 7f04dd7fa640 10 client.20739 handle_command_reply: tid=0 2024-04-02T13:53:23.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.206+0000 7f04febaa740 1 client.20739 shutdown 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 msgr2=0x557ba0759f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f04e80118d0 tx=0x7f04e8004730 comp rx=0 tx=0).stop 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 shutdown_connections 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x557ba06fba00 0x557ba07650c0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557ba06eb7c0 0x557ba06efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x557ba06e2be0 0x557ba075a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 --2- 172.21.15.122:0/2313690490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557ba06e22c0 0x557ba0759f60 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 >> 172.21.15.122:0/2313690490 conn(0x557ba06dc9f0 msgr2=0x557ba06db220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04dd7fa640 0 client.20739 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 shutdown_connections 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 1 -- 172.21.15.122:0/2313690490 wait complete. 2024-04-02T13:53:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.207+0000 7f04febaa740 20 client.20739 trim_cache size 0 max 16384 2024-04-02T13:53:23.214 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:23.214 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:53:23.214 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "5453e50e-98a1-4cfe-a857-5ff350b656fb", 2024-04-02T13:53:23.214 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:53:23.214 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 msgr2=0x7f04e00a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 0x7f04e00a6250 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009b50 tx=0x7f04e8005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 msgr2=0x7f04f410ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 0x7f04f410ef80 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00ea80 tx=0x7f04ec00ef40 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 shutdown_connections 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f04e00a3d90 0x7f04e00a6250 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04f4106980 0x7f04f410ef80 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04f4101330 0x7f04f410ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 --2- 172.21.15.122:0/3754255150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f4100970 0x7f04f410e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 >> 172.21.15.122:0/3754255150 conn(0x7f04f4077b00 msgr2=0x7f04f4103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 shutdown_connections 2024-04-02T13:53:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.208+0000 7f04fbfc8640 1 -- 172.21.15.122:0/3754255150 wait complete. 2024-04-02T13:53:23.225 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:53:23.225 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:53:23.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 150240 27.96 MB/sec execute 163 sec latency 113.278 ms 2024-04-02T13:53:23.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.376+0000 7f005e960640 1 -- 172.21.15.122:0/2912577069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0058110320 msgr2=0x7f005810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.376+0000 7f005e960640 1 --2- 172.21.15.122:0/2912577069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0058110320 0x7f005810de10 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f004c009900 tx=0x7f004c02bc40 comp rx=0 tx=0).stop 2024-04-02T13:53:23.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.377+0000 7f005e960640 1 -- 172.21.15.122:0/2912577069 shutdown_connections 2024-04-02T13:53:23.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.377+0000 7f005e960640 1 --2- 172.21.15.122:0/2912577069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f005811b2e0 0x7f005811d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.377+0000 7f005e960640 1 --2- 172.21.15.122:0/2912577069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.377+0000 7f005e960640 1 --2- 172.21.15.122:0/2912577069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0058110320 0x7f005810de10 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.377+0000 7f005e960640 1 -- 172.21.15.122:0/2912577069 >> 172.21.15.122:0/2912577069 conn(0x7f005806cbe0 msgr2=0x7f005806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.379+0000 7f005e960640 1 -- 172.21.15.122:0/2912577069 shutdown_connections 2024-04-02T13:53:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.380+0000 7f005e960640 1 -- 172.21.15.122:0/2912577069 wait complete. 2024-04-02T13:53:23.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.381+0000 7f005e960640 1 Processor -- start 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.381+0000 7f005e960640 1 -- start start 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0058110320 0x7f0058114a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 0x7f0058114fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00581114d0 con 0x7f005811b2e0 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0058111350 con 0x7f0058110320 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005e960640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0058111650 con 0x7f005810e350 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f0057fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f0057fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47854/0 (socket says 172.21.15.122:47854) 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f0057fff640 1 -- 172.21.15.122:0/800570228 learned_addr learned my addr 172.21.15.122:0/800570228 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f005cefd640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 0x7f0058114fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.382+0000 7f00577fe640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0058110320 0x7f0058114a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 -- 172.21.15.122:0/800570228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0058110320 msgr2=0x7f0058114a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0058110320 0x7f0058114a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 -- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 msgr2=0x7f0058114fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 0x7f0058114fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 -- 172.21.15.122:0/800570228 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00581156f0 con 0x7f005810e350 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f005cefd640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 0x7f0058114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.383+0000 7f0057fff640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.384+0000 7f00557fa640 1 -- 172.21.15.122:0/800570228 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c041070 con 0x7f005810e350 2024-04-02T13:53:23.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.384+0000 7f00557fa640 1 -- 172.21.15.122:0/800570228 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c02cca0 con 0x7f005810e350 2024-04-02T13:53:23.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.384+0000 7f00557fa640 1 -- 172.21.15.122:0/800570228 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c034440 con 0x7f005810e350 2024-04-02T13:53:23.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.384+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00581bbfd0 con 0x7f005810e350 2024-04-02T13:53:23.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.384+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581bc380 con 0x7f005810e350 2024-04-02T13:53:23.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.387+0000 7f00557fa640 1 -- 172.21.15.122:0/800570228 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f004c044070 con 0x7f005810e350 2024-04-02T13:53:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.388+0000 7f00557fa640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 0x7f00400a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.389+0000 7f00577fe640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 0x7f00400a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.389+0000 7f00557fa640 1 -- 172.21.15.122:0/800570228 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f004c0f2540 con 0x7f005810e350 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.389+0000 7f00577fe640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 0x7f00400a6330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0058116240 tx=0x7f004800a4c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.389+0000 7f0061629740 1 Processor -- start 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.389+0000 7f0061629740 1 -- start start 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55adcb24e830 con 0x55adcb2ea8d0 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55adcb1f6820 con 0x55adcb2f3dd0 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0061629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55adcb24a7e0 con 0x55adcb2eb1f0 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0057fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0057fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52824/0 (socket says 172.21.15.122:52824) 2024-04-02T13:53:23.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f0057fff640 1 -- 172.21.15.122:0/32100152 learned_addr learned my addr 172.21.15.122:0/32100152 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f00577fe640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.390+0000 7f005cefd640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 -- 172.21.15.122:0/32100152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 msgr2=0x55adcb2f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 -- 172.21.15.122:0/32100152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 msgr2=0x55adcb2eacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 -- 172.21.15.122:0/32100152 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcb02c7d0 con 0x55adcb2eb1f0 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f005cefd640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f0057fff640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.391+0000 7f00577fe640 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f3890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00581162d0 tx=0x7f0048011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab470668dad2523b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.392+0000 7f003e7fc640 1 -- 172.21.15.122:0/32100152 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048021070 con 0x55adcb2eb1f0 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.392+0000 7f003e7fc640 1 -- 172.21.15.122:0/32100152 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004801db30 con 0x55adcb2eb1f0 2024-04-02T13:53:23.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.392+0000 7f003e7fc640 1 -- 172.21.15.122:0/32100152 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004800fac0 con 0x55adcb2eb1f0 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 -- 172.21.15.122:0/32100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 msgr2=0x55adcb2f3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f3890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00581162d0 tx=0x7f0048011000 comp rx=0 tx=0).stop 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 -- 172.21.15.122:0/32100152 shutdown_connections 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f3890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 --2- 172.21.15.122:0/32100152 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2eacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.393+0000 7f0061629740 1 -- 172.21.15.122:0/32100152 >> 172.21.15.122:0/32100152 conn(0x55adcb2e5000 msgr2=0x55adcb2e53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.395+0000 7f0061629740 1 -- 172.21.15.122:0/32100152 shutdown_connections 2024-04-02T13:53:23.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.395+0000 7f0061629740 1 -- 172.21.15.122:0/32100152 wait complete. 2024-04-02T13:53:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.396+0000 7f0061629740 1 -- start start 2024-04-02T13:53:23.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.396+0000 7f0061629740 1 Processor -- start 2024-04-02T13:53:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55adcb2f6bc0 con 0x55adcb2ea8d0 2024-04-02T13:53:23.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55adcb209d30 con 0x55adcb2eb1f0 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55adcb2f6d40 con 0x55adcb2f3dd0 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0057fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0057fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52836/0 (socket says 172.21.15.122:52836) 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0057fff640 1 -- 172.21.15.122:0/4163679214 learned_addr learned my addr 172.21.15.122:0/4163679214 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.397+0000 7f0061629740 10 client.0 _ll_register_callbacks cb 0x55adcb209720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 -- 172.21.15.122:0/4163679214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 msgr2=0x55adcb2f0fe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f0fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 -- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 msgr2=0x55adcb2f0aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f0aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 -- 172.21.15.122:0/4163679214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcaf623f0 con 0x55adcb2ea8d0 2024-04-02T13:53:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.398+0000 7f0057fff640 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f004c0babd0 tx=0x7f004c0bde00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761ceb06f0a2466c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.399+0000 7f00377fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.399+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c04b040 con 0x55adcb2ea8d0 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.399+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f004c0c69e0 con 0x55adcb2ea8d0 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.400+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004801db30 con 0x55adcb2ea8d0 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.400+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f004c0be200 con 0x55adcb2ea8d0 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.400+0000 7f0061629740 10 client.20751 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:23.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.400+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55adcaf220f0 con 0x55adcb2ea8d0 2024-04-02T13:53:23.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.400+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c0d7070 con 0x55adcb2ea8d0 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f004c0f2100 con 0x55adcb2ea8d0 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 10 client.20751 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 10 client.20751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 20 client.20751 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb303b40 0x55adcb305fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 4 client.20751 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f004801db30 con 0x55adcb303b40 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.401+0000 7f00577fe640 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb303b40 0x55adcb305fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.402+0000 7f00577fe640 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb303b40 0x55adcb305fb0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0dbba07efb84edb server_cookie=accc38d4f4202b02 in_seq=0 out_seq=0 2024-04-02T13:53:23.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.402+0000 7f00377fe640 10 client.20751 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.410+0000 7f00377fe640 1 -- 172.21.15.122:0/4163679214 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f004801db30 con 0x55adcb303b40 2024-04-02T13:53:23.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.411+0000 7f00377fe640 10 client.20751 handle_command_reply: tid=0 2024-04-02T13:53:23.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.411+0000 7f0061629740 1 client.20751 shutdown 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.411+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 msgr2=0x55adcb2f3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.411+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f004c0babd0 tx=0x7f004c0bde00 comp rx=0 tx=0).stop 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 shutdown_connections 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb303b40 0x55adcb305fb0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2f0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 --2- 172.21.15.122:0/4163679214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2ea8d0 0x55adcb2f3710 secure :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f004c0babd0 tx=0x7f004c0bde00 comp rx=0 tx=0).stop 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 >> 172.21.15.122:0/4163679214 conn(0x55adcb2e5000 msgr2=0x55adcb2e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f00377fe640 0 client.20751 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 shutdown_connections 2024-04-02T13:53:23.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.412+0000 7f0061629740 1 -- 172.21.15.122:0/4163679214 wait complete. 2024-04-02T13:53:23.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.413+0000 7f0061629740 20 client.20751 trim_cache size 0 max 16384 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0061629740 1 Processor -- start 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0061629740 1 -- start start 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0061629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0061629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0061629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.444+0000 7f0057fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0057fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47872/0 (socket says 172.21.15.122:47872) 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0057fff640 1 -- 172.21.15.122:0/4174641289 learned_addr learned my addr 172.21.15.122:0/4174641289 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c6720 con 0x55adcb2f3dd0 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c8320 con 0x55adcb2eb1f0 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c6df0 con 0x55adcb2e5000 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f00577fe640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 -- 172.21.15.122:0/4174641289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 msgr2=0x55adcb2ffa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 -- 172.21.15.122:0/4174641289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 msgr2=0x55adcb2ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2ef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f005cefd640 1 -- 172.21.15.122:0/4174641289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcb2e4b30 con 0x55adcb2f3dd0 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.445+0000 7f0057fff640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.446+0000 7f005cefd640 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f31d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x55adcb2e5d60 tx=0x7f0050013b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624e31d56e1c13a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.446+0000 7f003dffb640 1 -- 172.21.15.122:0/4174641289 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050013220 con 0x55adcb2f3dd0 2024-04-02T13:53:23.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.446+0000 7f003dffb640 1 -- 172.21.15.122:0/4174641289 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0050012450 con 0x55adcb2f3dd0 2024-04-02T13:53:23.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.446+0000 7f003dffb640 1 -- 172.21.15.122:0/4174641289 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005001a5d0 con 0x55adcb2f3dd0 2024-04-02T13:53:23.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.447+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 msgr2=0x55adcb2f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.447+0000 7f0061629740 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f31d0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x55adcb2e5d60 tx=0x7f0050013b90 comp rx=0 tx=0).stop 2024-04-02T13:53:23.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 shutdown_connections 2024-04-02T13:53:23.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2f3dd0 0x55adcb2f31d0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2eb1f0 0x55adcb2ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 --2- 172.21.15.122:0/4174641289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2e5000 0x55adcb2ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 >> 172.21.15.122:0/4174641289 conn(0x55adcb2f1140 msgr2=0x55adcb2e4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 shutdown_connections 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 -- 172.21.15.122:0/4174641289 wait complete. 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.448+0000 7f0061629740 1 -- start start 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.449+0000 7f0061629740 1 Processor -- start 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 0x55adcb2edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2ee210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0057fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0057fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:55106/0 (socket says 172.21.15.122:55106) 2024-04-02T13:53:23.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f00577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 0x55adcb2edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0057fff640 1 -- 172.21.15.122:0/2579369533 learned_addr learned my addr 172.21.15.122:0/2579369533 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c7c50 con 0x55adcb2eb1f0 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c74c0 con 0x55adcb2e5000 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00581c4720 con 0x55adcb2f3dd0 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f0061629740 10 client.0 _ll_register_callbacks cb 0x7f004c0ba550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.450+0000 7f005cefd640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2ee210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 -- 172.21.15.122:0/2579369533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 msgr2=0x55adcb2ee210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2ee210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 -- 172.21.15.122:0/2579369533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 msgr2=0x55adcb2edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 0x55adcb2edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcb369130 con 0x55adcb2e5000 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f00577fe640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 0x55adcb2edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f005cefd640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:23.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0057fff640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f004c0f2510 tx=0x7f004c02cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be561d5f15ef2c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f00377fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c04b040 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f004c0c6be0 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.451+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0050012450 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f004c0c23b0 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 10 client.29708 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55adcb36a3a0 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0050012450 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f004c0bd650 con 0x55adcb2e5000 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 10 client.29708 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 10 client.29708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 20 client.29708 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb36a890 0x55adcb36cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:23.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 4 client.29708 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:53:23.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.452+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55adcb25bd70 con 0x55adcb36a890 2024-04-02T13:53:23.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.453+0000 7f00577fe640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb36a890 0x55adcb36cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:23.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.455+0000 7f00577fe640 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb36a890 0x55adcb36cc80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d98b0a8f08e68c server_cookie=b4fe1bdfd9ba7598 in_seq=0 out_seq=0 2024-04-02T13:53:23.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.455+0000 7f00377fe640 10 client.29708 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f00377fe640 1 -- 172.21.15.122:0/2579369533 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55adcb25bd70 con 0x55adcb36a890 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f00377fe640 10 client.29708 handle_command_reply: tid=0 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 client.29708 shutdown 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 msgr2=0x55adcb2ed790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f004c0f2510 tx=0x7f004c02cf90 comp rx=0 tx=0).stop 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 shutdown_connections 2024-04-02T13:53:23.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55adcb36a890 0x55adcb36cc80 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f00377fe640 0 client.29708 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55adcb2f3dd0 0x55adcb2ee210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55adcb2eb1f0 0x55adcb2edcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 --2- 172.21.15.122:0/2579369533 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55adcb2e5000 0x55adcb2ed790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 >> 172.21.15.122:0/2579369533 conn(0x55adcb2f1140 msgr2=0x55adcb2e63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 shutdown_connections 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 1 -- 172.21.15.122:0/2579369533 wait complete. 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.458+0000 7f0061629740 20 client.29708 trim_cache size 0 max 16384 2024-04-02T13:53:23.463 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "5453e50e-98a1-4cfe-a857-5ff350b656fb": { 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "5453e50e-98a1-4cfe-a857-5ff350b656fb", 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 msgr2=0x7f00400a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 0x7f00400a6330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0058116240 tx=0x7f004800a4c0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 msgr2=0x7f005811aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c0043c0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 shutdown_connections 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f00400a3e70 0x7f00400a6330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f005811b2e0 0x7f0058114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0058110320 0x7f0058114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 --2- 172.21.15.122:0/800570228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f005810e350 0x7f005811aa10 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f004c005ab0 tx=0x7f004c0043c0 comp rx=0 tx=0).stop 2024-04-02T13:53:23.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.459+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 >> 172.21.15.122:0/800570228 conn(0x7f005806cbe0 msgr2=0x7f005811c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:23.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.462+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 shutdown_connections 2024-04-02T13:53:23.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:23.462+0000 7f005e960640 1 -- 172.21.15.122:0/800570228 wait complete. 2024-04-02T13:53:23.484 INFO:tasks.cephfs.filesystem:scrub status for tag:5453e50e-98a1-4cfe-a857-5ff350b656fb - {'path': '/', 'tag': '5453e50e-98a1-4cfe-a857-5ff350b656fb', 'options': 'recursive,force'} 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 781 op/s 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:24.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:23 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 781 op/s 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:24.237 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.237 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.237 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.237 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:23 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 781 op/s 2024-04-02T13:53:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:53:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:23 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:24.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 150813 28.03 MB/sec execute 164 sec latency 78.174 ms 2024-04-02T13:53:25.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:24 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:25.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:24 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:25.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:24 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:25.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:24 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:25.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:24 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:25.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:24 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:25.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 151385 28.02 MB/sec execute 165 sec latency 73.678 ms 2024-04-02T13:53:26.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:25 smithi122 ceph-mon[32114]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 740 op/s 2024-04-02T13:53:26.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:25 smithi187 ceph-mon[36064]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 740 op/s 2024-04-02T13:53:26.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:25 smithi188 ceph-mon[36117]: pgmap v574: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 740 op/s 2024-04-02T13:53:26.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 151964 28.02 MB/sec execute 166 sec latency 97.944 ms 2024-04-02T13:53:27.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 152495 27.99 MB/sec execute 167 sec latency 204.036 ms 2024-04-02T13:53:28.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:27 smithi122 ceph-mon[32114]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 790 op/s 2024-04-02T13:53:28.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:27 smithi187 ceph-mon[36064]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 790 op/s 2024-04-02T13:53:28.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:27 smithi188 ceph-mon[36117]: pgmap v575: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 790 op/s 2024-04-02T13:53:28.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 153248 28.07 MB/sec execute 168 sec latency 89.848 ms 2024-04-02T13:53:29.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 153801 28.09 MB/sec execute 169 sec latency 261.242 ms 2024-04-02T13:53:30.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:29 smithi187 ceph-mon[36064]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:30.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:29 smithi188 ceph-mon[36117]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:30.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:29 smithi122 ceph-mon[32114]: pgmap v576: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:30.332 INFO:tasks.workunit.client.0.smithi122.stdout: 10 154469 28.21 MB/sec execute 170 sec latency 110.735 ms 2024-04-02T13:53:31.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 154967 28.17 MB/sec execute 171 sec latency 236.072 ms 2024-04-02T13:53:32.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:31 smithi187 ceph-mon[36064]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:32.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:31 smithi188 ceph-mon[36117]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:32.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:31 smithi122 ceph-mon[32114]: pgmap v577: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-02T13:53:32.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 155480 28.11 MB/sec execute 172 sec latency 240.563 ms 2024-04-02T13:53:33.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 156098 28.16 MB/sec execute 173 sec latency 75.250 ms 2024-04-02T13:53:34.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:33 smithi187 ceph-mon[36064]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 556 op/s 2024-04-02T13:53:34.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:33 smithi188 ceph-mon[36117]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 556 op/s 2024-04-02T13:53:34.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:33 smithi122 ceph-mon[32114]: pgmap v578: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 556 op/s 2024-04-02T13:53:34.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 156792 28.21 MB/sec execute 174 sec latency 59.209 ms 2024-04-02T13:53:35.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 157376 28.22 MB/sec execute 175 sec latency 168.084 ms 2024-04-02T13:53:36.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:35 smithi187 ceph-mon[36064]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2024-04-02T13:53:36.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:35 smithi188 ceph-mon[36117]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2024-04-02T13:53:36.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:35 smithi122 ceph-mon[32114]: pgmap v579: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 475 op/s 2024-04-02T13:53:36.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 157995 28.31 MB/sec execute 176 sec latency 60.508 ms 2024-04-02T13:53:37.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:37.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:37.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:37.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:36 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:37.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:37.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:36 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:37.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:37.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:36 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:37.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 158550 28.29 MB/sec execute 177 sec latency 136.786 ms 2024-04-02T13:53:38.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:37 smithi187 ceph-mon[36064]: pgmap v580: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-04-02T13:53:38.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:37 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:53:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:37 smithi188 ceph-mon[36117]: pgmap v580: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-04-02T13:53:38.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:37 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:53:38.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:37 smithi122 ceph-mon[32114]: pgmap v580: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-04-02T13:53:38.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:37 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:53:38.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 159155 28.31 MB/sec execute 178 sec latency 94.204 ms 2024-04-02T13:53:39.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:38 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:38 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:38 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:53:39.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 159663 28.30 MB/sec execute 179 sec latency 104.593 ms 2024-04-02T13:53:40.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:39 smithi188 ceph-mon[36117]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-02T13:53:40.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:39 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:40.168 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:39 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:40.169 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:39 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:53:40.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:39 smithi187 ceph-mon[36064]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-02T13:53:40.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:39 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:40.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:39 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:40.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:39 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:53:40.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:39 smithi122 ceph-mon[32114]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.5 MiB/s wr, 588 op/s 2024-04-02T13:53:40.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:39 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:53:40.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:39 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:53:40.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:39 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:53:40.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160206 28.31 MB/sec execute 180 sec latency 96.367 ms 2024-04-02T13:53:41.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160552 28.28 MB/sec execute 181 sec latency 418.754 ms 2024-04-02T13:53:42.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:41 smithi187 ceph-mon[36064]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2024-04-02T13:53:42.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:41 smithi188 ceph-mon[36117]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2024-04-02T13:53:42.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:41 smithi122 ceph-mon[32114]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 436 op/s 2024-04-02T13:53:42.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160566 28.13 MB/sec execute 182 sec latency 1418.809 ms 2024-04-02T13:53:43.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160685 28.00 MB/sec execute 183 sec latency 1785.082 ms 2024-04-02T13:53:44.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:43 smithi187 ceph-mon[36064]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-02T13:53:44.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:43 smithi188 ceph-mon[36117]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-02T13:53:44.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:43 smithi122 ceph-mon[32114]: pgmap v583: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-02T13:53:44.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 160937 27.90 MB/sec execute 184 sec latency 190.576 ms 2024-04-02T13:53:45.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 161213 27.80 MB/sec execute 185 sec latency 333.438 ms 2024-04-02T13:53:46.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:45 smithi187 ceph-mon[36064]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 487 op/s 2024-04-02T13:53:46.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:45 smithi188 ceph-mon[36117]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 487 op/s 2024-04-02T13:53:46.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:45 smithi122 ceph-mon[32114]: pgmap v584: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 487 op/s 2024-04-02T13:53:46.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 161447 27.72 MB/sec execute 186 sec latency 467.293 ms 2024-04-02T13:53:47.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 162093 27.79 MB/sec execute 187 sec latency 111.920 ms 2024-04-02T13:53:48.122 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:47 smithi122 ceph-mon[32114]: pgmap v585: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-04-02T13:53:48.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:47 smithi187 ceph-mon[36064]: pgmap v585: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-04-02T13:53:48.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:47 smithi188 ceph-mon[36117]: pgmap v585: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 485 op/s 2024-04-02T13:53:48.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 162817 27.86 MB/sec execute 188 sec latency 91.396 ms 2024-04-02T13:53:49.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 163427 27.91 MB/sec execute 189 sec latency 76.728 ms 2024-04-02T13:53:50.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:49 smithi187 ceph-mon[36064]: pgmap v586: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 771 op/s 2024-04-02T13:53:50.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:49 smithi188 ceph-mon[36117]: pgmap v586: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 771 op/s 2024-04-02T13:53:50.320 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:49 smithi122 ceph-mon[32114]: pgmap v586: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 771 op/s 2024-04-02T13:53:50.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 163971 27.91 MB/sec execute 190 sec latency 76.599 ms 2024-04-02T13:53:51.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 164455 27.87 MB/sec execute 191 sec latency 145.603 ms 2024-04-02T13:53:52.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:51 smithi187 ceph-mon[36064]: pgmap v587: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 600 op/s 2024-04-02T13:53:52.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:51 smithi188 ceph-mon[36117]: pgmap v587: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 600 op/s 2024-04-02T13:53:52.323 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:51 smithi122 ceph-mon[32114]: pgmap v587: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 600 op/s 2024-04-02T13:53:52.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 165069 27.87 MB/sec execute 192 sec latency 93.760 ms 2024-04-02T13:53:53.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 165818 27.95 MB/sec execute 193 sec latency 68.631 ms 2024-04-02T13:53:53.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:53:53.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:53:53.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.617+0000 7fb435399640 1 -- 172.21.15.122:0/821070316 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c02cc90 con 0x7fb430101220 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 -- 172.21.15.122:0/821070316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430101220 msgr2=0x7fb43010d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 --2- 172.21.15.122:0/821070316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430101220 0x7fb43010d730 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009930 tx=0x7fb42c02bc40 comp rx=0 tx=0).stop 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 -- 172.21.15.122:0/821070316 shutdown_connections 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 --2- 172.21.15.122:0/821070316 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430101220 0x7fb43010d730 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 --2- 172.21.15.122:0/821070316 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430100860 0x7fb430100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 --2- 172.21.15.122:0/821070316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430106870 0x7fb430106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 -- 172.21.15.122:0/821070316 >> 172.21.15.122:0/821070316 conn(0x7fb4300fc350 msgr2=0x7fb4300fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 -- 172.21.15.122:0/821070316 shutdown_connections 2024-04-02T13:53:53.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.618+0000 7fb43739d640 1 -- 172.21.15.122:0/821070316 wait complete. 2024-04-02T13:53:53.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.619+0000 7fb43739d640 1 Processor -- start 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.619+0000 7fb43739d640 1 -- start start 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430101220 0x7fb430198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430106870 0x7fb43019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111e10 con 0x7fb430100860 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111c90 con 0x7fb430106870 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43739d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb430111f90 con 0x7fb430101220 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53448/0 (socket says 172.21.15.122:53448) 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 -- 172.21.15.122:0/2479800523 learned_addr learned my addr 172.21.15.122:0/2479800523 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430106870 0x7fb43019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430101220 0x7fb430198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 -- 172.21.15.122:0/2479800523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430101220 msgr2=0x7fb430198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430101220 0x7fb430198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 -- 172.21.15.122:0/2479800523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430106870 msgr2=0x7fb43019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430106870 0x7fb43019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 -- 172.21.15.122:0/2479800523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43019d2a0 con 0x7fb430100860 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.620+0000 7fb43639b640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb42800eb90 tx=0x7fb42800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.621+0000 7fb4237fe640 1 -- 172.21.15.122:0/2479800523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428004d20 con 0x7fb430100860 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.621+0000 7fb4237fe640 1 -- 172.21.15.122:0/2479800523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42800ee90 con 0x7fb430100860 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.621+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43019d590 con 0x7fb430100860 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.621+0000 7fb4237fe640 1 -- 172.21.15.122:0/2479800523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428010830 con 0x7fb430100860 2024-04-02T13:53:53.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.621+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb43010e320 con 0x7fb430100860 2024-04-02T13:53:53.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.623+0000 7fb4237fe640 1 -- 172.21.15.122:0/2479800523 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb4280109d0 con 0x7fb430100860 2024-04-02T13:53:53.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.624+0000 7fb43b1e9740 1 Processor -- start 2024-04-02T13:53:53.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.624+0000 7fb43b1e9740 1 -- start start 2024-04-02T13:53:53.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 0x55bceda12aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda12fe0 0x55bceda153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bced96d830 con 0x55bceda09ae0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bced915820 con 0x55bceda0a400 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bced9697e0 con 0x55bceda12fe0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb4237fe640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 0x7fb41c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53460/0 (socket says 172.21.15.122:53460) 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 -- 172.21.15.122:0/2704792438 learned_addr learned my addr 172.21.15.122:0/2704792438 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda12fe0 0x55bceda153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43639b640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 0x55bceda12aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 0x7fb41c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb4237fe640 1 -- 172.21.15.122:0/2479800523 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fb4280d11f0 con 0x7fb430100860 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 -- 172.21.15.122:0/2704792438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda12fe0 msgr2=0x55bceda153d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda12fe0 0x55bceda153d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 -- 172.21.15.122:0/2704792438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 msgr2=0x55bceda12aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 0x55bceda12aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 -- 172.21.15.122:0/2704792438 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bced74b7d0 con 0x55bceda09ae0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb43639b640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 0x55bceda12aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.625+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009230 tx=0x7fb42c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cde27d5a69b7c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 0x7fb41c0a61d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005ab0 tx=0x7fb42c043000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb420bfe640 1 -- 172.21.15.122:0/2704792438 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0044d0 con 0x55bceda09ae0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb420bfe640 1 -- 172.21.15.122:0/2704792438 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c050070 con 0x55bceda09ae0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb420bfe640 1 -- 172.21.15.122:0/2704792438 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c0035f0 con 0x55bceda09ae0 2024-04-02T13:53:53.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2704792438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 msgr2=0x55bceda09ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.626+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fb42c009230 tx=0x7fb42c004040 comp rx=0 tx=0).stop 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2704792438 shutdown_connections 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda12fe0 0x55bceda153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda0a400 0x55bceda12aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2704792438 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda09ae0 0x55bceda09ec0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2704792438 >> 172.21.15.122:0/2704792438 conn(0x55bceda04210 msgr2=0x55bceda045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2704792438 shutdown_connections 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2704792438 wait complete. 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 -- start start 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.627+0000 7fb43b1e9740 1 Processor -- start 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 0x55bceda12850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 0x55bceda10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bceda15d70 con 0x55bceda0a400 2024-04-02T13:53:53.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55bced928d30 con 0x55bceda12fe0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55bceda15ef0 con 0x55bceda09ae0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43b1e9740 10 client.0 _ll_register_callbacks cb 0x55bced928720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43639b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 0x55bceda12850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb436b9c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 0x55bceda10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43639b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 0x55bceda12850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54616/0 (socket says 172.21.15.122:54616) 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53464/0 (socket says 172.21.15.122:53464) 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb43639b640 1 -- 172.21.15.122:0/2162956116 learned_addr learned my addr 172.21.15.122:0/2162956116 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 -- 172.21.15.122:0/2162956116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 msgr2=0x55bceda12850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 0x55bceda12850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 -- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 msgr2=0x55bceda10120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 0x55bceda10120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 -- 172.21.15.122:0/2162956116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bced6813f0 con 0x55bceda0a400 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 0x55bceda10120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:53.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb4300fe920 tx=0x7fb4240119d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec44c1e4fb1bffaf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.628+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424013730 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fb4240138d0 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c050070 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb424026050 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 10 client.20769 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55bced6410f0 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c050070 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb424014c90 con 0x55bceda0a400 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 10 client.20769 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 10 client.20769 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 20 client.20769 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda23490 0x55bceda258f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 4 client.20769 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:53.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.629+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bced97ad70 con 0x55bceda23490 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.630+0000 7fb43639b640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda23490 0x55bceda258f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.630+0000 7fb43639b640 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda23490 0x55bceda258f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59fb3f77941be1f0 server_cookie=9ee120df20e84d9f in_seq=0 out_seq=0 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.630+0000 7fb419ffb640 10 client.20769 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb419ffb640 1 -- 172.21.15.122:0/2162956116 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bced97ad70 con 0x55bceda23490 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb419ffb640 10 client.20769 handle_command_reply: tid=0 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 client.20769 shutdown 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 msgr2=0x55bceda0fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fb4300fe920 tx=0x7fb4240119d0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 shutdown_connections 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda23490 0x55bceda258f0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda12fe0 0x55bceda10120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb419ffb640 0 client.20769 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0fbe0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2162956116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda09ae0 0x55bceda12850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 >> 172.21.15.122:0/2162956116 conn(0x55bceda04210 msgr2=0x55bceda05f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.631+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 shutdown_connections 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.632+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2162956116 wait complete. 2024-04-02T13:53:53.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.632+0000 7fb43b1e9740 20 client.20769 trim_cache size 0 max 16384 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 Processor -- start 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 -- start start 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 0x55bceda12310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb414010010 con 0x55bceda04210 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb414011c10 con 0x55bceda09ae0 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43b1e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4140106e0 con 0x55bceda0a400 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53468/0 (socket says 172.21.15.122:53468) 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 -- 172.21.15.122:0/2889274222 learned_addr learned my addr 172.21.15.122:0/2889274222 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 0x55bceda12310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 -- 172.21.15.122:0/2889274222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 msgr2=0x55bceda12310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 0x55bceda12310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 -- 172.21.15.122:0/2889274222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 msgr2=0x55bceda0eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb43639b640 1 -- 172.21.15.122:0/2889274222 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bceda052b0 con 0x55bceda04210 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb435b9a640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.662+0000 7fb436b9c640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 0x55bceda12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.663+0000 7fb43639b640 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb42809a7b0 tx=0x7fb428013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae3d55b5c0ceff7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.663+0000 7fb41bfff640 1 -- 172.21.15.122:0/2889274222 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428081070 con 0x55bceda04210 2024-04-02T13:53:53.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.663+0000 7fb41bfff640 1 -- 172.21.15.122:0/2889274222 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4280a1040 con 0x55bceda04210 2024-04-02T13:53:53.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.663+0000 7fb41bfff640 1 -- 172.21.15.122:0/2889274222 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4280a2190 con 0x55bceda04210 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2889274222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 msgr2=0x55bceda1ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fb42809a7b0 tx=0x7fb428013000 comp rx=0 tx=0).stop 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2889274222 shutdown_connections 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda0a400 0x55bceda12310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/2889274222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda04210 0x55bceda1ec90 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2889274222 >> 172.21.15.122:0/2889274222 conn(0x55bceda10280 msgr2=0x55bceda0b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2889274222 shutdown_connections 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- 172.21.15.122:0/2889274222 wait complete. 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.664+0000 7fb43b1e9740 1 -- start start 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43b1e9740 1 Processor -- start 2024-04-02T13:53:53.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43b1e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43639b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43639b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54630/0 (socket says 172.21.15.122:54630) 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43639b640 1 -- 172.21.15.122:0/337794879 learned_addr learned my addr 172.21.15.122:0/337794879 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.665+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb414011540 con 0x55bceda0a400 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb414010db0 con 0x55bceda09ae0 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb41400e010 con 0x55bceda04210 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43b1e9740 10 client.0 _ll_register_callbacks cb 0x7fb424009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43639b640 1 -- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 msgr2=0x55bceda0cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43639b640 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43639b640 1 -- 172.21.15.122:0/337794879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 msgr2=0x55bceda0d2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43639b640 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0d2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.666+0000 7fb43639b640 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bceda88a70 con 0x55bceda04210 2024-04-02T13:53:53.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.667+0000 7fb43639b640 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb428004af0 tx=0x7fb4280a33a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f795a4e924a97b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.667+0000 7fb419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.667+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428081070 con 0x55bceda04210 2024-04-02T13:53:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.667+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fb42809a450 con 0x55bceda04210 2024-04-02T13:53:53.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.667+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4280a1040 con 0x55bceda04210 2024-04-02T13:53:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.670+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb42809e4e0 con 0x55bceda04210 2024-04-02T13:53:53.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.670+0000 7fb43b1e9740 10 client.29452 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.670+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55bceda89d30 con 0x55bceda04210 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.670+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4280a30c0 con 0x55bceda04210 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb4280ac020 con 0x55bceda04210 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 10 client.29452 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 10 client.29452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 20 client.29452 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda12fe0 0x55bceda8c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 4 client.29452 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:53:53.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb4280a1040 con 0x55bceda12fe0 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.671+0000 7fb435b9a640 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda12fe0 0x55bceda8c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.672+0000 7fb419ffb640 10 client.29452 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.672+0000 7fb435b9a640 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda12fe0 0x55bceda8c3c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ba088c622929d0 server_cookie=493f397c86b2a538 in_seq=0 out_seq=0 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb419ffb640 1 -- 172.21.15.122:0/337794879 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fb4280a1040 con 0x55bceda12fe0 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb419ffb640 10 client.29452 handle_command_reply: tid=0 2024-04-02T13:53:53.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 client.29452 shutdown 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 msgr2=0x55bceda0c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb428004af0 tx=0x7fb4280a33a0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 shutdown_connections 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55bceda12fe0 0x55bceda8c3c0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb419ffb640 0 client.29452 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.673+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bceda0a400 0x55bceda0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.674+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55bceda09ae0 0x55bceda0cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.674+0000 7fb43b1e9740 1 --2- 172.21.15.122:0/337794879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55bceda04210 0x55bceda0c840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.674+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 >> 172.21.15.122:0/337794879 conn(0x55bceda10280 msgr2=0x55bceda02c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.674+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 shutdown_connections 2024-04-02T13:53:53.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.674+0000 7fb43b1e9740 1 -- 172.21.15.122:0/337794879 wait complete. 2024-04-02T13:53:53.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.678+0000 7fb43b1e9740 20 client.29452 trim_cache size 0 max 16384 2024-04-02T13:53:53.684 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:53:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:53:53.685 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.682+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 msgr2=0x7fb41c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.682+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 0x7fb41c0a61d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb42c005ab0 tx=0x7fb42c043000 comp rx=0 tx=0).stop 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.682+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 msgr2=0x7fb430198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.682+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fb42800eb90 tx=0x7fb42800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 shutdown_connections 2024-04-02T13:53:53.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb41c0a3d10 0x7fb41c0a61d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb430106870 0x7fb43019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb430101220 0x7fb430198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 --2- 172.21.15.122:0/2479800523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb430100860 0x7fb430198300 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 >> 172.21.15.122:0/2479800523 conn(0x7fb4300fc350 msgr2=0x7fb4300fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.683+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 shutdown_connections 2024-04-02T13:53:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.684+0000 7fb43739d640 1 -- 172.21.15.122:0/2479800523 wait complete. 2024-04-02T13:53:53.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:53:53.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 -- 172.21.15.122:0/529628387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc190106a10 msgr2=0x7fc190106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 --2- 172.21.15.122:0/529628387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc190106a10 0x7fc190106df0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fc180009930 tx=0x7fc18002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 -- 172.21.15.122:0/529628387 shutdown_connections 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 --2- 172.21.15.122:0/529628387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1901013c0 0x7fc19010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 --2- 172.21.15.122:0/529628387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190100a00 0x7fc190100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 --2- 172.21.15.122:0/529628387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc190106a10 0x7fc190106df0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 -- 172.21.15.122:0/529628387 >> 172.21.15.122:0/529628387 conn(0x7fc190077b00 msgr2=0x7fc190077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:53.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 -- 172.21.15.122:0/529628387 shutdown_connections 2024-04-02T13:53:53.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.829+0000 7fc19502a640 1 -- 172.21.15.122:0/529628387 wait complete. 2024-04-02T13:53:53.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 Processor -- start 2024-04-02T13:53:53.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 -- start start 2024-04-02T13:53:53.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc190100a00 0x7fc190198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190106a10 0x7fc19019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc190110e70 con 0x7fc1901013c0 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc190110cf0 con 0x7fc190106a10 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc19502a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc190110ff0 con 0x7fc190100a00 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc18e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc18e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53484/0 (socket says 172.21.15.122:53484) 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc18e59c640 1 -- 172.21.15.122:0/1532301512 learned_addr learned my addr 172.21.15.122:0/1532301512 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc18f59e640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190106a10 0x7fc19019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.830+0000 7fc18ed9d640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc190100a00 0x7fc190198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 -- 172.21.15.122:0/1532301512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc190100a00 msgr2=0x7fc190198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc190100a00 0x7fc190198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 -- 172.21.15.122:0/1532301512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190106a10 msgr2=0x7fc19019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190106a10 0x7fc19019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 -- 172.21.15.122:0/1532301512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019d3c0 con 0x7fc1901013c0 2024-04-02T13:53:53.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc18e59c640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400cd80 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019d6b0 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.831+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a4f40 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.832+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc184010910 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.832+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184018980 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.833+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc17c005290 con 0x7fc1901013c0 2024-04-02T13:53:53.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.834+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc184018b20 con 0x7fc1901013c0 2024-04-02T13:53:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.834+0000 7fc177fff640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 0x7fc1780a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.835+0000 7fc18ed9d640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 0x7fc1780a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.835+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fc1840d0db0 con 0x7fc1901013c0 2024-04-02T13:53:53.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.835+0000 7fc18ed9d640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 0x7fc1780a6260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002df90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:53.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:53.839+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc18409ca10 con 0x7fc1901013c0 2024-04-02T13:53:54.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.192+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc17c005850 con 0x7fc1901013c0 2024-04-02T13:53:54.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.192+0000 7fc177fff640 1 -- 172.21.15.122:0/1532301512 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11949 (secure 0 0 0) 0x7fc1840a0d10 con 0x7fc1901013c0 2024-04-02T13:53:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:53:54.197 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:52:42.709142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:53:54.197 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 73 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 msgr2=0x7fc1780a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 0x7fc1780a6260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002df90 comp rx=0 tx=0).stop 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 msgr2=0x7fc190198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc18400e9f0 tx=0x7fc18400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 shutdown_connections 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc1780a3da0 0x7fc1780a6260 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc190106a10 0x7fc19019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1901013c0 0x7fc190198960 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 --2- 172.21.15.122:0/1532301512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc190100a00 0x7fc190198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 >> 172.21.15.122:0/1532301512 conn(0x7fc190077b00 msgr2=0x7fc1900ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.195+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 shutdown_connections 2024-04-02T13:53:54.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.196+0000 7fc19502a640 1 -- 172.21.15.122:0/1532301512 wait complete. 2024-04-02T13:53:54.211 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:53 smithi122 ceph-mon[32114]: pgmap v588: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-02T13:53:54.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:53:54.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:53 smithi187 ceph-mon[36064]: pgmap v588: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-02T13:53:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:53 smithi188 ceph-mon[36117]: pgmap v588: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 743 op/s 2024-04-02T13:53:54.333 INFO:tasks.workunit.client.0.smithi122.stdout: 10 166494 28.02 MB/sec execute 194 sec latency 102.403 ms 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 -- 172.21.15.122:0/2905274291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 msgr2=0x7f2a88105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/2905274291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a88105c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2a78009900 tx=0x7f2a7802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 -- 172.21.15.122:0/2905274291 shutdown_connections 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/2905274291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/2905274291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a88100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/2905274291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a88105c80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 -- 172.21.15.122:0/2905274291 >> 172.21.15.122:0/2905274291 conn(0x7f2a88077a90 msgr2=0x7f2a88077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 -- 172.21.15.122:0/2905274291 shutdown_connections 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.351+0000 7f2a8da6a640 1 -- 172.21.15.122:0/2905274291 wait complete. 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.352+0000 7f2a8da6a640 1 Processor -- start 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.352+0000 7f2a8da6a640 1 -- start start 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.352+0000 7f2a8da6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a8810e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.352+0000 7f2a8da6a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.352+0000 7f2a8da6a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a8810ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a8810e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a8810e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53512/0 (socket says 172.21.15.122:53512) 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a8da6a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88111ee0 con 0x7f2a881009c0 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a8da6a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88111d60 con 0x7f2a881058a0 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a8da6a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a88112060 con 0x7f2a88101360 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54660/0 (socket says 172.21.15.122:54660) 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 -- 172.21.15.122:0/295337453 learned_addr learned my addr 172.21.15.122:0/295337453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a87fff640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a8810ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 -- 172.21.15.122:0/295337453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 msgr2=0x7f2a8810ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a8810ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 -- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 msgr2=0x7f2a8810e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a8810e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.353+0000 7f2a86ffd640 1 -- 172.21.15.122:0/295337453 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a881aad80 con 0x7f2a88101360 2024-04-02T13:53:54.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9f0 tx=0x7f2a7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a84ff9640 1 -- 172.21.15.122:0/295337453 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00cd80 con 0x7f2a88101360 2024-04-02T13:53:54.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a84ff9640 1 -- 172.21.15.122:0/295337453 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a7c010910 con 0x7f2a88101360 2024-04-02T13:53:54.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a84ff9640 1 -- 172.21.15.122:0/295337453 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c018990 con 0x7f2a88101360 2024-04-02T13:53:54.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a881aafb0 con 0x7f2a88101360 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.354+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a881ab420 con 0x7f2a88101360 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.357+0000 7f2a90658740 1 Processor -- start 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.357+0000 7f2a90658740 1 -- start start 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.357+0000 7f2a84ff9640 1 -- 172.21.15.122:0/295337453 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2a7c010ac0 con 0x7f2a88101360 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.357+0000 7f2a90658740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.357+0000 7f2a90658740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a90658740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a90658740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559abde787b0 con 0x559abdf1dca0 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a90658740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559abde17570 con 0x559abdf147a0 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a90658740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559abde74760 con 0x559abdf150c0 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39422/0 (socket says 172.21.15.122:39422) 2024-04-02T13:53:54.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 -- 172.21.15.122:0/1942918423 learned_addr learned my addr 172.21.15.122:0/1942918423 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a877fe640 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 -- 172.21.15.122:0/1942918423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 msgr2=0x559abdf1d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 -- 172.21.15.122:0/1942918423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 msgr2=0x559abdf20090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf20090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 -- 172.21.15.122:0/1942918423 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559abdb8c3f0 con 0x559abdf147a0 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a84ff9640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 0x7f2a700a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a877fe640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 0x7f2a700a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a84ff9640 1 -- 172.21.15.122:0/295337453 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f2a7c0d0ed0 con 0x7f2a88101360 2024-04-02T13:53:54.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.358+0000 7f2a87fff640 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009f50 tx=0x7f2a800118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad47333a85ce7d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.359+0000 7f2a6dffb640 1 -- 172.21.15.122:0/1942918423 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004b20 con 0x559abdf147a0 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.359+0000 7f2a6dffb640 1 -- 172.21.15.122:0/1942918423 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80004cc0 con 0x559abdf147a0 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.359+0000 7f2a6dffb640 1 -- 172.21.15.122:0/1942918423 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80005020 con 0x559abdf147a0 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- 172.21.15.122:0/1942918423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 msgr2=0x559abdf14b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009f50 tx=0x7f2a800118d0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a877fe640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 0x7f2a700a6260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2a78005ab0 tx=0x7f2a7802e040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- 172.21.15.122:0/1942918423 shutdown_connections 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 --2- 172.21.15.122:0/1942918423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf14b80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- 172.21.15.122:0/1942918423 >> 172.21.15.122:0/1942918423 conn(0x559abdf0eed0 msgr2=0x559abdf0f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- 172.21.15.122:0/1942918423 shutdown_connections 2024-04-02T13:53:54.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- 172.21.15.122:0/1942918423 wait complete. 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.360+0000 7f2a90658740 1 -- start start 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 Processor -- start 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf1b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x559abdf20a90 con 0x559abdf1dca0 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x559abde33e80 con 0x559abdf147a0 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a90658740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559abdf20c10 con 0x559abdf150c0 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39432/0 (socket says 172.21.15.122:39432) 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a877fe640 1 -- 172.21.15.122:0/925793828 learned_addr learned my addr 172.21.15.122:0/925793828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.361+0000 7f2a87fff640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf1b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a90658740 10 client.0 _ll_register_callbacks cb 0x559abde6faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 -- 172.21.15.122:0/925793828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 msgr2=0x559abdf1ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 -- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 msgr2=0x559abdf1b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf1b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 -- 172.21.15.122:0/925793828 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559abdc567d0 con 0x559abdf150c0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a87fff640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf1b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0ce1d0 tx=0x7f2a7c09c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c584ec0f9d3b8613 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c025790 con 0x559abdf150c0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f2a7c09cd30 con 0x559abdf150c0 2024-04-02T13:53:54.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.362+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a80004cc0 con 0x559abdf150c0 2024-04-02T13:53:54.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.363+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0259d0 con 0x559abdf150c0 2024-04-02T13:53:54.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.363+0000 7f2a90658740 10 client.29470 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.363+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559abdf2cba0 con 0x559abdf150c0 2024-04-02T13:53:54.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.363+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a80004cc0 con 0x559abdf150c0 2024-04-02T13:53:54.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f2a7c0ba020 con 0x559abdf150c0 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 10 client.29470 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 10 client.29470 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 20 client.29470 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf2e000 0x559abdf30410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 4 client.29470 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559abde85cf0 con 0x559abdf2e000 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.364+0000 7f2a877fe640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf2e000 0x559abdf30410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.365+0000 7f2a877fe640 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf2e000 0x559abdf30410 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2593f21191ddcda server_cookie=abd164d351221d12 in_seq=0 out_seq=0 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.365+0000 7f2a66ffd640 10 client.29470 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a66ffd640 1 -- 172.21.15.122:0/925793828 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559abde85cf0 con 0x559abdf2e000 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a66ffd640 10 client.29470 handle_command_reply: tid=0 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 client.29470 shutdown 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 msgr2=0x559abdf1b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0ce1d0 tx=0x7f2a7c09c700 comp rx=0 tx=0).stop 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 shutdown_connections 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf2e000 0x559abdf30410 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf1b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf1b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 --2- 172.21.15.122:0/925793828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a66ffd640 0 client.29470 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 >> 172.21.15.122:0/925793828 conn(0x559abdf0eed0 msgr2=0x559abdf10c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 shutdown_connections 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 1 -- 172.21.15.122:0/925793828 wait complete. 2024-04-02T13:53:54.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.366+0000 7f2a90658740 20 client.29470 trim_cache size 0 max 16384 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 Processor -- start 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 -- start start 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf29950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b5850 con 0x559abdf1dca0 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b7450 con 0x559abdf147a0 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a90658740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b5f20 con 0x559abdf150c0 2024-04-02T13:53:54.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a86ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf29950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a87fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53534/0 (socket says 172.21.15.122:53534) 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.384+0000 7f2a87fff640 1 -- 172.21.15.122:0/2011135307 learned_addr learned my addr 172.21.15.122:0/2011135307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 -- 172.21.15.122:0/2011135307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 msgr2=0x559abdf29950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf29950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 -- 172.21.15.122:0/2011135307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 msgr2=0x559abdf1d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 -- 172.21.15.122:0/2011135307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559abdf13890 con 0x559abdf1dca0 2024-04-02T13:53:54.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a87fff640 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009f50 tx=0x7f2a80004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ab1c8521121672 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a6d7fa640 1 -- 172.21.15.122:0/2011135307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80026070 con 0x559abdf1dca0 2024-04-02T13:53:54.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a6d7fa640 1 -- 172.21.15.122:0/2011135307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8000f960 con 0x559abdf1dca0 2024-04-02T13:53:54.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.385+0000 7f2a6d7fa640 1 -- 172.21.15.122:0/2011135307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80022ae0 con 0x559abdf1dca0 2024-04-02T13:53:54.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- 172.21.15.122:0/2011135307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 msgr2=0x559abdf35ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009f50 tx=0x7f2a80004a00 comp rx=0 tx=0).stop 2024-04-02T13:53:54.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- 172.21.15.122:0/2011135307 shutdown_connections 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf1dca0 0x559abdf35ce0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf150c0 0x559abdf29950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 --2- 172.21.15.122:0/2011135307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf147a0 0x559abdf1d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- 172.21.15.122:0/2011135307 >> 172.21.15.122:0/2011135307 conn(0x559abdf0eed0 msgr2=0x559abdf0df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- 172.21.15.122:0/2011135307 shutdown_connections 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- 172.21.15.122:0/2011135307 wait complete. 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.386+0000 7f2a90658740 1 -- start start 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a90658740 1 Processor -- start 2024-04-02T13:53:54.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a90658740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53538/0 (socket says 172.21.15.122:53538) 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a877fe640 1 -- 172.21.15.122:0/144931186 learned_addr learned my addr 172.21.15.122:0/144931186 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.387+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf150c0 0x559abdf215e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf1dca0 0x559abdf21b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b6d80 con 0x559abdf147a0 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b65f0 con 0x559abdf150c0 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a881b3850 con 0x559abdf1dca0 2024-04-02T13:53:54.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf150c0 0x559abdf215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a90658740 10 client.0 _ll_register_callbacks cb 0x7f2a7c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a87fff640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf1dca0 0x559abdf21b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 -- 172.21.15.122:0/144931186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf1dca0 msgr2=0x559abdf21b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf1dca0 0x559abdf21b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 -- 172.21.15.122:0/144931186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf150c0 msgr2=0x559abdf215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf150c0 0x559abdf215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559abdf22350 con 0x559abdf147a0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a877fe640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2a7802c2e0 tx=0x7f2a78009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5197a3082e77e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.388+0000 7f2a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78056c10 con 0x559abdf147a0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8000f960 con 0x559abdf147a0 2024-04-02T13:53:54.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f2a7803a040 con 0x559abdf147a0 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f2a78065050 con 0x559abdf147a0 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 10 client.20793 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559abdf945d0 con 0x559abdf147a0 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a8000f960 con 0x559abdf147a0 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f2a780051e0 con 0x559abdf147a0 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 10 client.20793 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 10 client.20793 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 20 client.20793 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.389+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf94ca0 0x559abdf970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.390+0000 7f2a90658740 4 client.20793 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.390+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559abde80040 con 0x559abdf94ca0 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.390+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf94ca0 0x559abdf970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.390+0000 7f2a66ffd640 10 client.20793 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.390+0000 7f2a86ffd640 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf94ca0 0x559abdf970d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1debeea06153136 server_cookie=3c1f5daf6e02a433 in_seq=0 out_seq=0 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.391+0000 7f2a66ffd640 1 -- 172.21.15.122:0/144931186 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559abde80040 con 0x559abdf94ca0 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.391+0000 7f2a66ffd640 10 client.20793 handle_command_reply: tid=0 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.391+0000 7f2a90658740 1 client.20793 shutdown 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 msgr2=0x559abdf26270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f2a7802c2e0 tx=0x7f2a78009260 comp rx=0 tx=0).stop 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 shutdown_connections 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x559abdf94ca0 0x559abdf970d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559abdf1dca0 0x559abdf21b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x559abdf150c0 0x559abdf215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 --2- 172.21.15.122:0/144931186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x559abdf147a0 0x559abdf26270 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 >> 172.21.15.122:0/144931186 conn(0x559abdf0eed0 msgr2=0x559abdf10ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a66ffd640 0 client.20793 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 shutdown_connections 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 1 -- 172.21.15.122:0/144931186 wait complete. 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.392+0000 7f2a90658740 20 client.20793 trim_cache size 0 max 16384 2024-04-02T13:53:54.402 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 msgr2=0x7f2a700a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 0x7f2a700a6260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2a78005ab0 tx=0x7f2a7802e040 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 msgr2=0x7f2a8810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00e9f0 tx=0x7f2a7c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 shutdown_connections 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2a700a3da0 0x7f2a700a6260 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2a881058a0 0x7f2a8810ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a88101360 0x7f2a8810e7f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 --2- 172.21.15.122:0/295337453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2a881009c0 0x7f2a8810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 >> 172.21.15.122:0/295337453 conn(0x7f2a88077a90 msgr2=0x7f2a880fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 shutdown_connections 2024-04-02T13:53:54.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.393+0000 7f2a8da6a640 1 -- 172.21.15.122:0/295337453 wait complete. 2024-04-02T13:53:54.410 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:53:54.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 -- 172.21.15.122:0/1332671750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010102450 msgr2=0x7fa01010e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 --2- 172.21.15.122:0/1332671750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010102450 0x7fa01010e960 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fa008009cd0 tx=0x7fa00802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 -- 172.21.15.122:0/1332671750 shutdown_connections 2024-04-02T13:53:54.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 --2- 172.21.15.122:0/1332671750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010102450 0x7fa01010e960 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 --2- 172.21.15.122:0/1332671750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010101a90 0x7fa010101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 --2- 172.21.15.122:0/1332671750 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010107aa0 0x7fa010107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 -- 172.21.15.122:0/1332671750 >> 172.21.15.122:0/1332671750 conn(0x7fa010078ec0 msgr2=0x7fa0100792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 -- 172.21.15.122:0/1332671750 shutdown_connections 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.545+0000 7fa0154d9640 1 -- 172.21.15.122:0/1332671750 wait complete. 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 Processor -- start 2024-04-02T13:53:54.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 -- start start 2024-04-02T13:53:54.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010102450 0x7fa0101988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010107aa0 0x7fa01019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa010111eb0 con 0x7fa010101a90 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa010111d30 con 0x7fa010102450 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa0154d9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa010112030 con 0x7fa010107aa0 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53550/0 (socket says 172.21.15.122:53550) 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa00effd640 1 -- 172.21.15.122:0/2063527435 learned_addr learned my addr 172.21.15.122:0/2063527435 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010102450 0x7fa0101988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.546+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010107aa0 0x7fa01019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 -- 172.21.15.122:0/2063527435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010107aa0 msgr2=0x7fa01019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010107aa0 0x7fa01019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 -- 172.21.15.122:0/2063527435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010102450 msgr2=0x7fa0101988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010102450 0x7fa0101988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 -- 172.21.15.122:0/2063527435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01019d360 con 0x7fa010101a90 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa00effd640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9f0 tx=0x7fa00000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7f9ff7fff640 1 -- 172.21.15.122:0/2063527435 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000cd80 con 0x7fa010101a90 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01019d650 con 0x7fa010101a90 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7f9ff7fff640 1 -- 172.21.15.122:0/2063527435 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa000010910 con 0x7fa010101a90 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7f9ff7fff640 1 -- 172.21.15.122:0/2063527435 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0000188f0 con 0x7fa010101a90 2024-04-02T13:53:54.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.547+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101a4ee0 con 0x7fa010101a90 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.550+0000 7f9ff7fff640 1 -- 172.21.15.122:0/2063527435 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa000010ac0 con 0x7fa010101a90 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.550+0000 7fa01819f740 1 Processor -- start 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.550+0000 7fa01819f740 1 -- start start 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.550+0000 7fa01819f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa01819f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65bc0 0x558529e66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa01819f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e797c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa01819f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558529dc97b0 con 0x558529e65210 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa01819f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558529d68570 con 0x558529e773d0 2024-04-02T13:53:54.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa01819f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558529dc5760 con 0x558529e65bc0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53558/0 (socket says 172.21.15.122:53558) 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 -- 172.21.15.122:0/2525260570 learned_addr learned my addr 172.21.15.122:0/2525260570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e797c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00effd640 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65bc0 0x558529e66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7f9ff7fff640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 0x7f9ff80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 -- 172.21.15.122:0/2525260570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65bc0 msgr2=0x558529e66040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65bc0 0x558529e66040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 -- 172.21.15.122:0/2525260570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 msgr2=0x558529e797c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e797c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 -- 172.21.15.122:0/2525260570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558529add3f0 con 0x558529e65210 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 0x7f9ff80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.551+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c700 tx=0x7fa008009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c46ce48c80715016 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.552+0000 7f9ff7fff640 1 -- 172.21.15.122:0/2063527435 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fa0000d0ec0 con 0x7fa010101a90 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.552+0000 7f9ff57fa640 1 -- 172.21.15.122:0/2525260570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008051070 con 0x558529e65210 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.552+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 0x7f9ff80a6260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa00804a000 tx=0x7fa00804b000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.552+0000 7f9ff57fa640 1 -- 172.21.15.122:0/2525260570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008043050 con 0x558529e65210 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.552+0000 7f9ff57fa640 1 -- 172.21.15.122:0/2525260570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008039040 con 0x558529e65210 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- 172.21.15.122:0/2525260570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 msgr2=0x558529e655f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa00802c700 tx=0x7fa008009900 comp rx=0 tx=0).stop 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- 172.21.15.122:0/2525260570 shutdown_connections 2024-04-02T13:53:54.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e797c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65bc0 0x558529e66040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 --2- 172.21.15.122:0/2525260570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e655f0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- 172.21.15.122:0/2525260570 >> 172.21.15.122:0/2525260570 conn(0x558529e5fed0 msgr2=0x558529e602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- 172.21.15.122:0/2525260570 shutdown_connections 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- 172.21.15.122:0/2525260570 wait complete. 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.553+0000 7fa01819f740 1 -- start start 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 Processor -- start 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e67530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e67b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x558529e7a320 con 0x558529e65210 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x558529d84e80 con 0x558529e65bc0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558529e7a4a0 con 0x558529e773d0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa01819f740 10 client.0 _ll_register_callbacks cb 0x558529dc0aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53564/0 (socket says 172.21.15.122:53564) 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 -- 172.21.15.122:0/188186839 learned_addr learned my addr 172.21.15.122:0/188186839 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e67530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e67b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 -- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 msgr2=0x558529e67b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e67b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 -- 172.21.15.122:0/188186839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 msgr2=0x558529e67530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e67530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.554+0000 7fa00effd640 1 -- 172.21.15.122:0/188186839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558529ba77d0 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7fa00effd640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa0000ce220 tx=0x7fa00009c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92a7278fedc7139 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7f9ff2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0000ce650 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fa0000a53f0 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa008043050 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa00009cac0 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7fa01819f740 10 client.20805 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.555+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x558529e86410 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008043050 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fa00009c020 con 0x558529e65210 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 10 client.20805 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 10 client.20805 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:53:54.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 20 client.20805 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529e878d0 0x558529e89cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 4 client.20805 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558529dd6cf0 con 0x558529e878d0 2024-04-02T13:53:54.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.556+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529e878d0 0x558529e89cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.557+0000 7f9ff2ff5640 10 client.20805 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.557+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529e878d0 0x558529e89cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bcb01c052941b8d4 server_cookie=c53a443bd3c5867e in_seq=0 out_seq=0 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.559+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/188186839 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558529dd6cf0 con 0x558529e878d0 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.559+0000 7f9ff2ff5640 10 client.20805 handle_command_reply: tid=0 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.559+0000 7fa01819f740 1 client.20805 shutdown 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 msgr2=0x558529e6a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa0000ce220 tx=0x7fa00009c6f0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 shutdown_connections 2024-04-02T13:53:54.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e67b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e67530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529e878d0 0x558529e89cc0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7f9ff2ff5640 0 client.20805 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 --2- 172.21.15.122:0/188186839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e6a1a0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 >> 172.21.15.122:0/188186839 conn(0x558529e5fed0 msgr2=0x558529e61c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 shutdown_connections 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 1 -- 172.21.15.122:0/188186839 wait complete. 2024-04-02T13:53:54.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.560+0000 7fa01819f740 20 client.20805 trim_cache size 0 max 16384 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 Processor -- start 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 -- start start 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e5ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e69c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101ad010 con 0x558529e65210 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101aec10 con 0x558529e65bc0 2024-04-02T13:53:54.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa01819f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101ad6e0 con 0x558529e773d0 2024-04-02T13:53:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa00effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53574/0 (socket says 172.21.15.122:53574) 2024-04-02T13:53:54.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.581+0000 7fa00effd640 1 -- 172.21.15.122:0/2920265364 learned_addr learned my addr 172.21.15.122:0/2920265364 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e69c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 -- 172.21.15.122:0/2920265364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 msgr2=0x558529e69c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e69c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 -- 172.21.15.122:0/2920265364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 msgr2=0x558529e5ad90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e5ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 -- 172.21.15.122:0/2920265364 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558529e7a160 con 0x558529e65210 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.582+0000 7fa00effd640 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa00009ca50 tx=0x7fa000009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84722d64b63de00 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.583+0000 7f9ff4ff9640 1 -- 172.21.15.122:0/2920265364 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00009c110 con 0x558529e65210 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.583+0000 7f9ff4ff9640 1 -- 172.21.15.122:0/2920265364 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0000a0dc0 con 0x558529e65210 2024-04-02T13:53:54.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.583+0000 7f9ff4ff9640 1 -- 172.21.15.122:0/2920265364 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000098810 con 0x558529e65210 2024-04-02T13:53:54.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.584+0000 7fa01819f740 1 -- 172.21.15.122:0/2920265364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 msgr2=0x558529e83250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.584+0000 7fa01819f740 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa00009ca50 tx=0x7fa000009770 comp rx=0 tx=0).stop 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 -- 172.21.15.122:0/2920265364 shutdown_connections 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e773d0 0x558529e69c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e65bc0 0x558529e5ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 --2- 172.21.15.122:0/2920265364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65210 0x558529e83250 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 -- 172.21.15.122:0/2920265364 >> 172.21.15.122:0/2920265364 conn(0x558529e5fed0 msgr2=0x558529e602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 -- 172.21.15.122:0/2920265364 shutdown_connections 2024-04-02T13:53:54.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 -- 172.21.15.122:0/2920265364 wait complete. 2024-04-02T13:53:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.585+0000 7fa01819f740 1 -- start start 2024-04-02T13:53:54.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 Processor -- start 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 0x558529e74e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 0x558529e753a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e758e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101ae540 con 0x558529e65bc0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101addb0 con 0x558529e773d0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0101ab010 con 0x558529e65210 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa01819f740 10 client.0 _ll_register_callbacks cb 0x7fa00000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa00effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 0x558529e74e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa00e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 0x558529e753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa00effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 0x558529e74e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54762/0 (socket says 172.21.15.122:54762) 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa00effd640 1 -- 172.21.15.122:0/3410563493 learned_addr learned my addr 172.21.15.122:0/3410563493 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.586+0000 7fa00f7fe640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 -- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 msgr2=0x558529e74e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 0x558529e74e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 -- 172.21.15.122:0/3410563493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 msgr2=0x558529e758e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e758e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 -- 172.21.15.122:0/3410563493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558529eecde0 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa00e7fc640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 0x558529e753a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x558529e8a4e0 tx=0x7fa004005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4f9b89d59e5973 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7f9ff2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0040202f0 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0000a0dc0 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fa004020490 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa00402d050 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa01819f740 10 client.20811 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.587+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x558529eee110 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0000a0dc0 con 0x558529e65bc0 2024-04-02T13:53:54.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fa004028030 con 0x558529e65bc0 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 10 client.20811 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 10 client.20811 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 20 client.20811 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529eee830 0x558529ef0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 4 client.20811 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558529dd1040 con 0x558529eee830 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.588+0000 7fa00effd640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529eee830 0x558529ef0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.589+0000 7f9ff2ff5640 10 client.20811 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:54.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.589+0000 7fa00effd640 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529eee830 0x558529ef0c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f54d369d6b482fc8 server_cookie=6361677d121e8f51 in_seq=0 out_seq=0 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.592+0000 7f9ff2ff5640 1 -- 172.21.15.122:0/3410563493 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558529dd1040 con 0x558529eee830 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.592+0000 7f9ff2ff5640 10 client.20811 handle_command_reply: tid=0 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.592+0000 7fa01819f740 1 client.20811 shutdown 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 msgr2=0x558529e753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 0x558529e753a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x558529e8a4e0 tx=0x7fa004005850 comp rx=0 tx=0).stop 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 shutdown_connections 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x558529e773d0 0x558529e758e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x558529eee830 0x558529ef0c20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x558529e65bc0 0x558529e753a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 --2- 172.21.15.122:0/3410563493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558529e65210 0x558529e74e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 >> 172.21.15.122:0/3410563493 conn(0x558529e5fed0 msgr2=0x558529e5ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7f9ff2ff5640 0 client.20811 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:53:54.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 shutdown_connections 2024-04-02T13:53:54.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 1 -- 172.21.15.122:0/3410563493 wait complete. 2024-04-02T13:53:54.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa01819f740 20 client.20811 trim_cache size 0 max 16384 2024-04-02T13:53:54.598 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 msgr2=0x7f9ff80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 0x7f9ff80a6260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa00804a000 tx=0x7fa00804b000 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 msgr2=0x7fa010198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9f0 tx=0x7fa00000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 shutdown_connections 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ff80a3da0 0x7f9ff80a6260 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa010107aa0 0x7fa01019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa010102450 0x7fa0101988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 --2- 172.21.15.122:0/2063527435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa010101a90 0x7fa010198370 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.593+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 >> 172.21.15.122:0/2063527435 conn(0x7fa010078ec0 msgr2=0x7fa0101043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.594+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 shutdown_connections 2024-04-02T13:53:54.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.594+0000 7fa0154d9640 1 -- 172.21.15.122:0/2063527435 wait complete. 2024-04-02T13:53:54.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:53:54.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 -- 172.21.15.122:0/1304138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 msgr2=0x7f3d08106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 --2- 172.21.15.122:0/1304138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d08106d60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009860 tx=0x7f3cf802bb70 comp rx=0 tx=0).stop 2024-04-02T13:53:54.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 -- 172.21.15.122:0/1304138657 shutdown_connections 2024-04-02T13:53:54.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 --2- 172.21.15.122:0/1304138657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 --2- 172.21.15.122:0/1304138657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d08100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 --2- 172.21.15.122:0/1304138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d08106d60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.757+0000 7f3d0e650640 1 -- 172.21.15.122:0/1304138657 >> 172.21.15.122:0/1304138657 conn(0x7f3d08077b00 msgr2=0x7f3d08077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.758+0000 7f3d0e650640 1 -- 172.21.15.122:0/1304138657 shutdown_connections 2024-04-02T13:53:54.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.759+0000 7f3d0e650640 1 -- 172.21.15.122:0/1304138657 wait complete. 2024-04-02T13:53:54.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.760+0000 7f3d0e650640 1 Processor -- start 2024-04-02T13:53:54.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.760+0000 7f3d0e650640 1 -- start start 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d081961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d0819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d081961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d081961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53612/0 (socket says 172.21.15.122:53612) 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d07fff640 1 -- 172.21.15.122:0/3384114246 learned_addr learned my addr 172.21.15.122:0/3384114246 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810fcc0 con 0x7f3d08100970 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810fb40 con 0x7f3d08106980 2024-04-02T13:53:54.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d0819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0810fe40 con 0x7f3d08101330 2024-04-02T13:53:54.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.761+0000 7f3d077fe640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3384114246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 msgr2=0x7f3d08196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 msgr2=0x7f3d081961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d081961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0819b180 con 0x7f3d08106980 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d077fe640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d0819aa50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e940 tx=0x7f3d0000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.762+0000 7f3d057fa640 1 -- 172.21.15.122:0/3384114246 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d00004d20 con 0x7f3d08106980 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.763+0000 7f3d057fa640 1 -- 172.21.15.122:0/3384114246 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00005020 con 0x7f3d08106980 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.764+0000 7f3d057fa640 1 -- 172.21.15.122:0/3384114246 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000116b0 con 0x7f3d08106980 2024-04-02T13:53:54.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.764+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0819b470 con 0x7f3d08106980 2024-04-02T13:53:54.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.764+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d081a2d00 con 0x7f3d08106980 2024-04-02T13:53:54.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.767+0000 7f3d057fa640 1 -- 172.21.15.122:0/3384114246 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3d000054d0 con 0x7f3d08106980 2024-04-02T13:53:54.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.768+0000 7f3d057fa640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 0x7f3cf00a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.768+0000 7f3d057fa640 1 -- 172.21.15.122:0/3384114246 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f3d000d0910 con 0x7f3d08106980 2024-04-02T13:53:54.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.768+0000 7f3d11230740 1 Processor -- start 2024-04-02T13:53:54.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.768+0000 7f3d11230740 1 -- start start 2024-04-02T13:53:54.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f80390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 0x561515f88f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 0x561515f8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561515edb7b0 con 0x561515f7ffb0 2024-04-02T13:53:54.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561515e7a570 con 0x561515f894b0 2024-04-02T13:53:54.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d11230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561515ed7760 con 0x561515f808d0 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 0x561515f88f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 0x561515f88f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54790/0 (socket says 172.21.15.122:54790) 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d077fe640 1 -- 172.21.15.122:0/2408580847 learned_addr learned my addr 172.21.15.122:0/2408580847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.769+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 0x561515f8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 -- 172.21.15.122:0/2408580847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 msgr2=0x561515f88f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 0x561515f88f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 -- 172.21.15.122:0/2408580847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 msgr2=0x561515f80390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f80390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 -- 172.21.15.122:0/2408580847 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561515bef3f0 con 0x561515f894b0 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d07fff640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 0x7f3cf00a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 0x561515f8b8a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3d000cdc10 tx=0x7f3d0009be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d0f107e1bcbdf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.770+0000 7f3cee7fc640 1 -- 172.21.15.122:0/2408580847 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000b4020 con 0x561515f894b0 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.771+0000 7f3cee7fc640 1 -- 172.21.15.122:0/2408580847 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d000a0890 con 0x561515f894b0 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.771+0000 7f3cee7fc640 1 -- 172.21.15.122:0/2408580847 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001f070 con 0x561515f894b0 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- 172.21.15.122:0/2408580847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 msgr2=0x561515f8b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 0x561515f8b8a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3d000cdc10 tx=0x7f3d0009be40 comp rx=0 tx=0).stop 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d07fff640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 0x7f3cf00a63b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80095f0 tx=0x7f3cf8005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- 172.21.15.122:0/2408580847 shutdown_connections 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f894b0 0x561515f8b8a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f808d0 0x561515f88f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 --2- 172.21.15.122:0/2408580847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f80390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- 172.21.15.122:0/2408580847 >> 172.21.15.122:0/2408580847 conn(0x561515f7a6e0 msgr2=0x561515f7aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- 172.21.15.122:0/2408580847 shutdown_connections 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- 172.21.15.122:0/2408580847 wait complete. 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.772+0000 7f3d11230740 1 -- start start 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.773+0000 7f3d11230740 1 Processor -- start 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.773+0000 7f3d11230740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f7ffb0 0x561515f863c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.773+0000 7f3d11230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.773+0000 7f3d11230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f86e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.773+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d11230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561515f8c240 con 0x561515f808d0 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53614/0 (socket says 172.21.15.122:53614) 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d077fe640 1 -- 172.21.15.122:0/3523985831 learned_addr learned my addr 172.21.15.122:0/3523985831 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561515e96e80 con 0x561515f7ffb0 2024-04-02T13:53:54.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561515f8c3c0 con 0x561515f894b0 2024-04-02T13:53:54.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d11230740 10 client.0 _ll_register_callbacks cb 0x561515ed2aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f86e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f7ffb0 msgr2=0x561515f863c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f7ffb0 0x561515f863c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3523985831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 msgr2=0x561515f86900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d0cbed640 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561515cb97d0 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.774+0000 7f3d077fe640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f86e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000efd0 tx=0x7f3d00005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdbf3489b16201d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3ce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000ab060 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d000a0890 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f3d000d0450 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3d0001f040 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3d11230740 10 client.29506 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.776+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x561515f8c290 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d000a0890 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f3d00022060 con 0x561515f894b0 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 10 client.29506 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 10 client.29506 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:53:54.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 20 client.29506 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515f99530 0x561515f9b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 4 client.29506 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.777+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561515ee8cf0 con 0x561515f99530 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.778+0000 7f3d07fff640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515f99530 0x561515f9b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.778+0000 7f3d07fff640 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515f99530 0x561515f9b920 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3c0c652fb3988bf server_cookie=917e581035980568 in_seq=0 out_seq=0 2024-04-02T13:53:54.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.778+0000 7f3ce77fe640 10 client.29506 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:54.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.781+0000 7f3ce77fe640 1 -- 172.21.15.122:0/3523985831 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561515ee8cf0 con 0x561515f99530 2024-04-02T13:53:54.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.781+0000 7f3ce77fe640 10 client.29506 handle_command_reply: tid=0 2024-04-02T13:53:54.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.781+0000 7f3d11230740 1 client.29506 shutdown 2024-04-02T13:53:54.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 msgr2=0x561515f86e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f86e40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000efd0 tx=0x7f3d00005440 comp rx=0 tx=0).stop 2024-04-02T13:53:54.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 shutdown_connections 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f86e40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f808d0 0x561515f86900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515f99530 0x561515f9b920 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3ce77fe640 0 client.29506 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 --2- 172.21.15.122:0/3523985831 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f7ffb0 0x561515f863c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.782+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 >> 172.21.15.122:0/3523985831 conn(0x561515f7a6e0 msgr2=0x561515f7c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.783+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 shutdown_connections 2024-04-02T13:53:54.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.783+0000 7f3d11230740 1 -- 172.21.15.122:0/3523985831 wait complete. 2024-04-02T13:53:54.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.783+0000 7f3d11230740 20 client.29506 trim_cache size 0 max 16384 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 Processor -- start 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 -- start start 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 0x561515f88af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f95100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ace70 con 0x561515f894b0 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081aea70 con 0x561515f808d0 2024-04-02T13:53:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.804+0000 7f3d11230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ad540 con 0x561515f7ffb0 2024-04-02T13:53:54.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 0x561515f88af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53626/0 (socket says 172.21.15.122:53626) 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d07fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 0x561515f88af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54812/0 (socket says 172.21.15.122:54812) 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 -- 172.21.15.122:0/4148569842 learned_addr learned my addr 172.21.15.122:0/4148569842 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d077fe640 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f95100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 -- 172.21.15.122:0/4148569842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 msgr2=0x561515f88af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 0x561515f88af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 -- 172.21.15.122:0/4148569842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 msgr2=0x561515f95100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f95100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 -- 172.21.15.122:0/4148569842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561515f7b910 con 0x561515f894b0 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.805+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3d000984b0 tx=0x7f3d0009a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe643d4ed74154 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.806+0000 7f3cedffb640 1 -- 172.21.15.122:0/4148569842 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000ab040 con 0x561515f894b0 2024-04-02T13:53:54.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.806+0000 7f3cedffb640 1 -- 172.21.15.122:0/4148569842 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00098230 con 0x561515f894b0 2024-04-02T13:53:54.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.806+0000 7f3cedffb640 1 -- 172.21.15.122:0/4148569842 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000b4040 con 0x561515f894b0 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- 172.21.15.122:0/4148569842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 msgr2=0x561515fa10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f3d000984b0 tx=0x7f3d0009a9f0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- 172.21.15.122:0/4148569842 shutdown_connections 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f894b0 0x561515fa10b0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f95100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 --2- 172.21.15.122:0/4148569842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f7ffb0 0x561515f88af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- 172.21.15.122:0/4148569842 >> 172.21.15.122:0/4148569842 conn(0x561515f7a6e0 msgr2=0x561515f7c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- 172.21.15.122:0/4148569842 shutdown_connections 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- 172.21.15.122:0/4148569842 wait complete. 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.807+0000 7f3d11230740 1 -- start start 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.808+0000 7f3d11230740 1 Processor -- start 2024-04-02T13:53:54.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.808+0000 7f3d11230740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f91b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.808+0000 7f3d11230740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f82e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.808+0000 7f3d11230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.808+0000 7f3d11230740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081ae3a0 con 0x561515f7ffb0 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d11230740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081adc10 con 0x561515f808d0 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d11230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d081aae70 con 0x561515f894b0 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f82e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d077fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f82e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39560/0 (socket says 172.21.15.122:39560) 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d077fe640 1 -- 172.21.15.122:0/1672735693 learned_addr learned my addr 172.21.15.122:0/1672735693 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d07fff640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f91b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d11230740 10 client.0 _ll_register_callbacks cb 0x7f3cfc00da90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 -- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 msgr2=0x561515f82e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f82e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 -- 172.21.15.122:0/1672735693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 msgr2=0x561515f91b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f91b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.809+0000 7f3d0cbed640 1 -- 172.21.15.122:0/1672735693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561515ffe920 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.810+0000 7f3d0cbed640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f835d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d0009c1e0 tx=0x7f3d00005440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7508df0c5eb59f68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.810+0000 7f3ce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.810+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000b4040 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.810+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f3d000a1a20 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.810+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d00098230 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f3d000ab040 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 10 client.29512 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x561515ffe920 con 0x561515f894b0 2024-04-02T13:53:54.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d00098230 con 0x561515f894b0 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f3d0000f040 con 0x561515f894b0 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 10 client.29512 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 10 client.29512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 20 client.29512 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515fffd20 0x561516002110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 4 client.29512 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.811+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561515ee3040 con 0x561515fffd20 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.812+0000 7f3d07fff640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515fffd20 0x561516002110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.812+0000 7f3d07fff640 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515fffd20 0x561516002110 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b95e1fdabf8ab5f server_cookie=2b25cbb05028a06e in_seq=0 out_seq=0 2024-04-02T13:53:54.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.812+0000 7f3ce77fe640 10 client.29512 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3ce77fe640 1 -- 172.21.15.122:0/1672735693 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561515ee3040 con 0x561515fffd20 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3ce77fe640 10 client.29512 handle_command_reply: tid=0 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 client.29512 shutdown 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 msgr2=0x561515f835d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f835d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3d0009c1e0 tx=0x7f3d00005440 comp rx=0 tx=0).stop 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 shutdown_connections 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561515f894b0 0x561515f835d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.815+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x561515fffd20 0x561516002110 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561515f808d0 0x561515f82e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 1 --2- 172.21.15.122:0/1672735693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561515f7ffb0 0x561515f91b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3ce77fe640 0 client.29512 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 >> 172.21.15.122:0/1672735693 conn(0x561515f7a6e0 msgr2=0x561515f706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 shutdown_connections 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 1 -- 172.21.15.122:0/1672735693 wait complete. 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.816+0000 7f3d11230740 20 client.29512 trim_cache size 0 max 16384 2024-04-02T13:53:54.821 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:54.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 msgr2=0x7f3cf00a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 0x7f3cf00a63b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3cf80095f0 tx=0x7f3cf8005850 comp rx=0 tx=0).stop 2024-04-02T13:53:54.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 msgr2=0x7f3d0819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:54.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d0819aa50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000e940 tx=0x7f3d0000ee00 comp rx=0 tx=0).stop 2024-04-02T13:53:54.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 shutdown_connections 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3cf00a3ef0 0x7f3cf00a63b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d08106980 0x7f3d0819aa50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d08101330 0x7f3d08196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 --2- 172.21.15.122:0/3384114246 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d08100970 0x7f3d081961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 >> 172.21.15.122:0/3384114246 conn(0x7f3d08077b00 msgr2=0x7f3d08104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 shutdown_connections 2024-04-02T13:53:54.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.817+0000 7f3d0e650640 1 -- 172.21.15.122:0/3384114246 wait complete. 2024-04-02T13:53:54.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:53:55.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 -- 172.21.15.122:0/2272905096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20100380 msgr2=0x7fcd20100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2272905096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20100380 0x7fcd20100760 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009900 tx=0x7fcd1002bc30 comp rx=0 tx=0).stop 2024-04-02T13:53:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 -- 172.21.15.122:0/2272905096 shutdown_connections 2024-04-02T13:53:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2272905096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20105450 0x7fcd2010d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2272905096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20100d30 0x7fcd20104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2272905096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20100380 0x7fcd20100760 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 -- 172.21.15.122:0/2272905096 >> 172.21.15.122:0/2272905096 conn(0x7fcd20076f20 msgr2=0x7fcd200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.998+0000 7fcd2562c640 1 -- 172.21.15.122:0/2272905096 shutdown_connections 2024-04-02T13:53:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.999+0000 7fcd2562c640 1 -- 172.21.15.122:0/2272905096 wait complete. 2024-04-02T13:53:55.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:54.999+0000 7fcd2562c640 1 Processor -- start 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 -- start start 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20105450 0x7fcd2019d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 0x7fcd201962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20105450 0x7fcd2019d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2010fcd0 con 0x7fcd20105450 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2010fb50 con 0x7fcd20195e10 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd2562c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2010fe50 con 0x7fcd20100380 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd1effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd1effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54842/0 (socket says 172.21.15.122:54842) 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd1effd640 1 -- 172.21.15.122:0/2346288397 learned_addr learned my addr 172.21.15.122:0/2346288397 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.000+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 0x7fcd201962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 -- 172.21.15.122:0/2346288397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 msgr2=0x7fcd201962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 0x7fcd201962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 -- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20105450 msgr2=0x7fcd2019d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20105450 0x7fcd2019d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 -- 172.21.15.122:0/2346288397 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd20196af0 con 0x7fcd20100380 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 0x7fcd201962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd07fff640 1 -- 172.21.15.122:0/2346288397 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1002cc50 con 0x7fcd20100380 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd07fff640 1 -- 172.21.15.122:0/2346288397 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd10044070 con 0x7fcd20100380 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd07fff640 1 -- 172.21.15.122:0/2346288397 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10034440 con 0x7fcd20100380 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019a8b0 con 0x7fcd20100380 2024-04-02T13:53:55.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.001+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2019ad30 con 0x7fcd20100380 2024-04-02T13:53:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.005+0000 7fcd2820e740 1 Processor -- start 2024-04-02T13:53:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.005+0000 7fcd2820e740 1 -- start start 2024-04-02T13:53:55.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.005+0000 7fcd07fff640 1 -- 172.21.15.122:0/2346288397 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcd100345e0 con 0x7fcd20100380 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.005+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182082760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39598/0 (socket says 172.21.15.122:39598) 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd07fff640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 0x7fcd080a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd1f7fe640 1 -- 172.21.15.122:0/2821522341 learned_addr learned my addr 172.21.15.122:0/2821522341 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182085090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d181fdd7b0 con 0x55d182082ca0 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d181f7c570 con 0x55d1820797a0 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d181fd9760 con 0x55d18207a0c0 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182082760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd07fff640 1 -- 172.21.15.122:0/2346288397 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fcd100f2a50 con 0x7fcd20100380 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.006+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 0x7fcd080a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182085090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1effd640 1 -- 172.21.15.122:0/2821522341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 msgr2=0x55d182079b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1effd640 1 -- 172.21.15.122:0/2821522341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 msgr2=0x55d182085090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182085090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1effd640 1 -- 172.21.15.122:0/2821522341 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d181cf13f0 con 0x55d18207a0c0 2024-04-02T13:53:55.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:53:55.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.007+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 0x7fcd080a62c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcd201975d0 tx=0x7fcd14005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.009+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182082760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd100efe60 tx=0x7fcd10044850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33dbd51efb9b3074 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.009+0000 7fcd057fa640 1 -- 172.21.15.122:0/2821522341 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100be1f0 con 0x55d18207a0c0 2024-04-02T13:53:55.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.009+0000 7fcd057fa640 1 -- 172.21.15.122:0/2821522341 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd100bedb0 con 0x55d18207a0c0 2024-04-02T13:53:55.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.010+0000 7fcd057fa640 1 -- 172.21.15.122:0/2821522341 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100be010 con 0x55d18207a0c0 2024-04-02T13:53:55.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 msgr2=0x55d182082760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182082760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcd100efe60 tx=0x7fcd10044850 comp rx=0 tx=0).stop 2024-04-02T13:53:55.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 shutdown_connections 2024-04-02T13:53:55.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182085090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182082760 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2821522341 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182079b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 >> 172.21.15.122:0/2821522341 conn(0x55d182073ed0 msgr2=0x55d1820742b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 shutdown_connections 2024-04-02T13:53:55.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- 172.21.15.122:0/2821522341 wait complete. 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 -- start start 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.011+0000 7fcd2820e740 1 Processor -- start 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182080040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182080580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d182085850 con 0x55d182082ca0 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d181f98e80 con 0x55d1820797a0 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d1820859d0 con 0x55d18207a0c0 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd2820e740 10 client.0 _ll_register_callbacks cb 0x55d181fd4aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:55.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182080580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39608/0 (socket says 172.21.15.122:39608) 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 -- 172.21.15.122:0/3284293041 learned_addr learned my addr 172.21.15.122:0/3284293041 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182080040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 -- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 msgr2=0x55d182080040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182080040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 -- 172.21.15.122:0/3284293041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 msgr2=0x55d182080580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182080580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.012+0000 7fcd1effd640 1 -- 172.21.15.122:0/3284293041 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d181dbb7d0 con 0x55d1820797a0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd1effd640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd100be3b0 tx=0x7fcd10003980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81569e8c362dc8ae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100dc060 con 0x55d1820797a0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd100bedb0 con 0x55d1820797a0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fcd100bc9b0 con 0x55d1820797a0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fcd100be010 con 0x55d1820797a0 2024-04-02T13:53:55.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd2820e740 10 client.29786 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.013+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d182085850 con 0x55d1820797a0 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd100bedb0 con 0x55d1820797a0 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fcd100bdb00 con 0x55d1820797a0 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 10 client.29786 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 10 client.29786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 20 client.29786 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d182092a10 0x55d182094e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 4 client.29786 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d181feacf0 con 0x55d182092a10 2024-04-02T13:53:55.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.014+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d182092a10 0x55d182094e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.015+0000 7fcd02ff5640 10 client.29786 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:55.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.015+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d182092a10 0x55d182094e00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8f475c80ef33550 server_cookie=9edab36cd50b12ff in_seq=0 out_seq=0 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd02ff5640 1 -- 172.21.15.122:0/3284293041 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d181feacf0 con 0x55d182092a10 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd02ff5640 10 client.29786 handle_command_reply: tid=0 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 client.29786 shutdown 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 msgr2=0x55d18207fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcd100be3b0 tx=0x7fcd10003980 comp rx=0 tx=0).stop 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 shutdown_connections 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d182092a10 0x55d182094e00 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d182082ca0 0x55d182080580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d18207a0c0 0x55d182080040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 --2- 172.21.15.122:0/3284293041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18207fb00 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 >> 172.21.15.122:0/3284293041 conn(0x55d182073ed0 msgr2=0x55d182075c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd02ff5640 0 client.29786 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 shutdown_connections 2024-04-02T13:53:55.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 1 -- 172.21.15.122:0/3284293041 wait complete. 2024-04-02T13:53:55.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.016+0000 7fcd2820e740 20 client.29786 trim_cache size 0 max 16384 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 Processor -- start 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 -- start start 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182082230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182094a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201bdd00 con 0x55d18207a0c0 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201bf900 con 0x55d1820797a0 2024-04-02T13:53:55.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.037+0000 7fcd2820e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201be3d0 con 0x55d182082ca0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53672/0 (socket says 172.21.15.122:53672) 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 -- 172.21.15.122:0/2396703280 learned_addr learned my addr 172.21.15.122:0/2396703280 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1effd640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182082230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182094a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 -- 172.21.15.122:0/2396703280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 msgr2=0x55d182094a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182094a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 -- 172.21.15.122:0/2396703280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 msgr2=0x55d182082230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182082230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 -- 172.21.15.122:0/2396703280 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d182082a10 con 0x55d18207a0c0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182094a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fcd20197660 tx=0x7fcd14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d865352e5794795 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd04ff9640 1 -- 172.21.15.122:0/2396703280 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1402f070 con 0x55d18207a0c0 2024-04-02T13:53:55.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd04ff9640 1 -- 172.21.15.122:0/2396703280 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd14003d40 con 0x55d18207a0c0 2024-04-02T13:53:55.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.038+0000 7fcd04ff9640 1 -- 172.21.15.122:0/2396703280 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14028800 con 0x55d18207a0c0 2024-04-02T13:53:55.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 -- 172.21.15.122:0/2396703280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 msgr2=0x55d18208e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fcd20197660 tx=0x7fcd14014000 comp rx=0 tx=0).stop 2024-04-02T13:53:55.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 -- 172.21.15.122:0/2396703280 shutdown_connections 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182094a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d18208e680 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 --2- 172.21.15.122:0/2396703280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d182082230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 -- 172.21.15.122:0/2396703280 >> 172.21.15.122:0/2396703280 conn(0x55d182073ed0 msgr2=0x55d1820726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.039+0000 7fcd2820e740 1 -- 172.21.15.122:0/2396703280 shutdown_connections 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.040+0000 7fcd2820e740 1 -- 172.21.15.122:0/2396703280 wait complete. 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.040+0000 7fcd2820e740 1 -- start start 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.040+0000 7fcd2820e740 1 Processor -- start 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.040+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18208ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.040+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d1820860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd2820e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182086610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd2820e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201bf230 con 0x55d18207a0c0 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd2820e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201beaa0 con 0x55d1820797a0 2024-04-02T13:53:55.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d1820860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d1820860d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53678/0 (socket says 172.21.15.122:53678) 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1e7fc640 1 -- 172.21.15.122:0/4130777911 learned_addr learned my addr 172.21.15.122:0/4130777911 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18208ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201bbd00 con 0x55d182082ca0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd2820e740 10 client.0 _ll_register_callbacks cb 0x7fcd18009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1f7fe640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182086610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 -- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 msgr2=0x55d182086610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182086610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 -- 172.21.15.122:0/4130777911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 msgr2=0x55d1820860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d1820860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.041+0000 7fcd1effd640 1 -- 172.21.15.122:0/4130777911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d182085620 con 0x55d1820797a0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd1effd640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18208ac60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd100bb030 tx=0x7fcd100be540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30256fa912f98a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd100dc070 con 0x55d1820797a0 2024-04-02T13:53:55.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7fcd100c7090 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd14003d40 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fcd100be1f0 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd2820e740 10 client.29792 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.042+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d182085620 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd14003d40 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fcd100c3bf0 con 0x55d1820797a0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 10 client.29792 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 10 client.29792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 20 client.29792 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d1820f8ce0 0x55d1820fb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 4 client.29792 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d181fe5040 con 0x55d1820f8ce0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.043+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d1820f8ce0 0x55d1820fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.044+0000 7fcd02ff5640 10 client.29792 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:55.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.044+0000 7fcd1e7fc640 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d1820f8ce0 0x55d1820fb100 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19e42985bd33eef2 server_cookie=4e0dc5979c1c507 in_seq=0 out_seq=0 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.046+0000 7fcd02ff5640 1 -- 172.21.15.122:0/4130777911 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d181fe5040 con 0x55d1820f8ce0 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.046+0000 7fcd02ff5640 10 client.29792 handle_command_reply: tid=0 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.046+0000 7fcd2820e740 1 client.29792 shutdown 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 msgr2=0x55d18208ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18208ac60 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcd100bb030 tx=0x7fcd100be540 comp rx=0 tx=0).stop 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 shutdown_connections 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55d1820f8ce0 0x55d1820fb100 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d182082ca0 0x55d182086610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d18207a0c0 0x55d1820860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 --2- 172.21.15.122:0/4130777911 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d1820797a0 0x55d18208ac60 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 >> 172.21.15.122:0/4130777911 conn(0x55d182073ed0 msgr2=0x55d1820726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd02ff5640 0 client.29792 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 shutdown_connections 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 1 -- 172.21.15.122:0/4130777911 wait complete. 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.047+0000 7fcd2820e740 20 client.29792 trim_cache size 0 max 16384 2024-04-02T13:53:55.053 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 msgr2=0x7fcd080a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 0x7fcd080a62c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcd201975d0 tx=0x7fcd14005f90 comp rx=0 tx=0).stop 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 msgr2=0x7fcd2019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009850 tx=0x7fcd10004b00 comp rx=0 tx=0).stop 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 shutdown_connections 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcd080a3e00 0x7fcd080a62c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcd20195e10 0x7fcd201962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd20105450 0x7fcd2019d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.054 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 --2- 172.21.15.122:0/2346288397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd20100380 0x7fcd2019cd90 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 >> 172.21.15.122:0/2346288397 conn(0x7fcd20076f20 msgr2=0x7fcd20103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 shutdown_connections 2024-04-02T13:53:55.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.048+0000 7fcd2562c640 1 -- 172.21.15.122:0/2346288397 wait complete. 2024-04-02T13:53:55.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:53:55.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 -- 172.21.15.122:0/3720392506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594096bb0 msgr2=0x7ff59409a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/3720392506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594096bb0 0x7ff59409a9a0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff588009930 tx=0x7ff58802bc80 comp rx=0 tx=0).stop 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 -- 172.21.15.122:0/3720392506 shutdown_connections 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/3720392506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 0x7ff5940a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/3720392506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594096bb0 0x7ff59409a9a0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/3720392506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 -- 172.21.15.122:0/3720392506 >> 172.21.15.122:0/3720392506 conn(0x7ff59400b860 msgr2=0x7ff59400bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 -- 172.21.15.122:0/3720392506 shutdown_connections 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.212+0000 7ff5a0b19640 1 -- 172.21.15.122:0/3720392506 wait complete. 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.213+0000 7ff5a0b19640 1 Processor -- start 2024-04-02T13:53:55.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.213+0000 7ff5a0b19640 1 -- start start 2024-04-02T13:53:55.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.213+0000 7ff5a0b19640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940077a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.213+0000 7ff5a0b19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 0x7ff594007ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.213+0000 7ff5a0b19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff5a0b19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940a60e0 con 0x7ff594008220 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff5a0b19640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940a5f60 con 0x7ff59409b0d0 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff5a0b19640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5940a6260 con 0x7ff594096200 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53710/0 (socket says 172.21.15.122:53710) 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 -- 172.21.15.122:0/1930329746 learned_addr learned my addr 172.21.15.122:0/1930329746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59affd640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 0x7ff594007ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 -- 172.21.15.122:0/1930329746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 msgr2=0x7ff5940077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940077a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 -- 172.21.15.122:0/1930329746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 msgr2=0x7ff594007ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 0x7ff594007ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 -- 172.21.15.122:0/1930329746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59413ae90 con 0x7ff594008220 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.214+0000 7ff59bfff640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff59c051160 tx=0x7ff59c078490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.215+0000 7ff598ff9640 1 -- 172.21.15.122:0/1930329746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c085070 con 0x7ff594008220 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.215+0000 7ff598ff9640 1 -- 172.21.15.122:0/1930329746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59c078e80 con 0x7ff594008220 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.215+0000 7ff598ff9640 1 -- 172.21.15.122:0/1930329746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0753a0 con 0x7ff594008220 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.215+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59413b180 con 0x7ff594008220 2024-04-02T13:53:55.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.215+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59413b650 con 0x7ff594008220 2024-04-02T13:53:55.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 Processor -- start 2024-04-02T13:53:55.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff598ff9640 1 -- 172.21.15.122:0/1930329746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff59c088070 con 0x7ff594008220 2024-04-02T13:53:55.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 -- start start 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 0x562ef3b3bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b47270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ef3a9f7b0 con 0x562ef3b3c2a0 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562ef3a3e570 con 0x562ef3b44e80 2024-04-02T13:53:55.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.219+0000 7ff5a4964740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562ef3a9b760 con 0x562ef3b3b980 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53714/0 (socket says 172.21.15.122:53714) 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59b7fe640 1 -- 172.21.15.122:0/3775937863 learned_addr learned my addr 172.21.15.122:0/3775937863 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 0x562ef3b3bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b47270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff598ff9640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 0x7ff58c0a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59bfff640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 0x7ff58c0a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 -- 172.21.15.122:0/3775937863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 msgr2=0x562ef3b47270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b47270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 -- 172.21.15.122:0/3775937863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 msgr2=0x562ef3b44940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 -- 172.21.15.122:0/3775937863 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef37b33f0 con 0x562ef3b3b980 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff598ff9640 1 -- 172.21.15.122:0/1930329746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7ff59c13c310 con 0x7ff594008220 2024-04-02T13:53:55.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.220+0000 7ff59affd640 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 0x562ef3b3bd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c1c0 tx=0x7ff588005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a88c45da8053a0b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.221+0000 7ff59bfff640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 0x7ff58c0a6290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff59c139720 tx=0x7ff59c132fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.221+0000 7ff585ffb640 1 -- 172.21.15.122:0/3775937863 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588004020 con 0x562ef3b3b980 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.221+0000 7ff585ffb640 1 -- 172.21.15.122:0/3775937863 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588004ba0 con 0x562ef3b3b980 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.221+0000 7ff585ffb640 1 -- 172.21.15.122:0/3775937863 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588005130 con 0x562ef3b3b980 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- 172.21.15.122:0/3775937863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 msgr2=0x562ef3b3bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 0x562ef3b3bd60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c1c0 tx=0x7ff588005b50 comp rx=0 tx=0).stop 2024-04-02T13:53:55.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- 172.21.15.122:0/3775937863 shutdown_connections 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b47270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 --2- 172.21.15.122:0/3775937863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3b980 0x562ef3b3bd60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- 172.21.15.122:0/3775937863 >> 172.21.15.122:0/3775937863 conn(0x562ef3b360b0 msgr2=0x562ef3b36490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- 172.21.15.122:0/3775937863 shutdown_connections 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- 172.21.15.122:0/3775937863 wait complete. 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.222+0000 7ff5a4964740 1 -- start start 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 Processor -- start 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b41c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff59affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff59affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53724/0 (socket says 172.21.15.122:53724) 2024-04-02T13:53:55.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff59affd640 1 -- 172.21.15.122:0/630768065 learned_addr learned my addr 172.21.15.122:0/630768065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b42230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562ef3b47bb0 con 0x562ef3b3b980 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562ef3a5ae80 con 0x562ef3b44e80 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562ef3b47d30 con 0x562ef3b3c2a0 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.223+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b41c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b42230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff5a4964740 10 client.0 _ll_register_callbacks cb 0x562ef3a96aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 -- 172.21.15.122:0/630768065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 msgr2=0x562ef3b41c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b41c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 -- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 msgr2=0x562ef3b42230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b42230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef387d7d0 con 0x562ef3b3b980 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff588004ba0 con 0x562ef3b3b980 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff59affd640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff5880096c0 tx=0x7ff58802bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2606f1188cfb6e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff57affd640 10 client.20856 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff57affd640 1 -- 172.21.15.122:0/630768065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880050d0 con 0x562ef3b3b980 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.224+0000 7ff57affd640 1 -- 172.21.15.122:0/630768065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7ff588005270 con 0x562ef3b3b980 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff57affd640 1 -- 172.21.15.122:0/630768065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7ff588005470 con 0x562ef3b3b980 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 10 client.20856 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=73}) -- 0x562ef3b5c0b0 con 0x562ef3b3b980 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff57affd640 1 -- 172.21.15.122:0/630768065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7ff5880035f0 con 0x562ef3b3b980 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 10 client.20856 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:55.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 10 client.20856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 20 client.20856 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.225+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3b5d400 0x562ef3b5f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.226+0000 7ff5a4964740 4 client.20856 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.226+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef3aaccf0 con 0x562ef3b5d400 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.226+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3b5d400 0x562ef3b5f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.228+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3b5d400 0x562ef3b5f7f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796dc9aac7f0913b server_cookie=11fec0a6903937bd in_seq=0 out_seq=0 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.228+0000 7ff57affd640 10 client.20856 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.228+0000 7ff57affd640 1 -- 172.21.15.122:0/630768065 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ef3aaccf0 con 0x562ef3b5d400 2024-04-02T13:53:55.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.228+0000 7ff57affd640 10 client.20856 handle_command_reply: tid=0 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.228+0000 7ff5a4964740 1 client.20856 shutdown 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.229+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 msgr2=0x562ef3b44890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.229+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff5880096c0 tx=0x7ff58802bc80 comp rx=0 tx=0).stop 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 shutdown_connections 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3b5d400 0x562ef3b5f7f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b44e80 0x562ef3b42230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b41c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 --2- 172.21.15.122:0/630768065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3b980 0x562ef3b44890 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 >> 172.21.15.122:0/630768065 conn(0x562ef3b360b0 msgr2=0x562ef3b37e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff57affd640 0 client.20856 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 shutdown_connections 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 1 -- 172.21.15.122:0/630768065 wait complete. 2024-04-02T13:53:55.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.230+0000 7ff5a4964740 20 client.20856 trim_cache size 0 max 16384 2024-04-02T13:53:55.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:54 smithi187 ceph-mon[36064]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 661 op/s 2024-04-02T13:53:55.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:54 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1532301512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.252+0000 7ff5a4964740 1 Processor -- start 2024-04-02T13:53:55.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.252+0000 7ff5a4964740 1 -- start start 2024-04-02T13:53:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39680/0 (socket says 172.21.15.122:39680) 2024-04-02T13:53:55.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 -- 172.21.15.122:0/968581115 learned_addr learned my addr 172.21.15.122:0/968581115 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b44e80 0x562ef3b3cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff594141010 con 0x562ef3b44e80 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff594142c10 con 0x562ef3b3b980 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941416e0 con 0x562ef3b3c2a0 2024-04-02T13:53:55.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b44e80 0x562ef3b3cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 -- 172.21.15.122:0/968581115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 msgr2=0x562ef3b44890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 -- 172.21.15.122:0/968581115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b44e80 msgr2=0x562ef3b3cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b44e80 0x562ef3b3cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.253+0000 7ff59affd640 1 -- 172.21.15.122:0/968581115 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef3b3c0d0 con 0x562ef3b3b980 2024-04-02T13:53:55.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.254+0000 7ff59affd640 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c810 tx=0x7ff58804b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff830a02ea044618 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.254+0000 7ff5857fa640 1 -- 172.21.15.122:0/968581115 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880035f0 con 0x562ef3b3b980 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.254+0000 7ff5857fa640 1 -- 172.21.15.122:0/968581115 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588004ba0 con 0x562ef3b3b980 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.254+0000 7ff5857fa640 1 -- 172.21.15.122:0/968581115 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880346a0 con 0x562ef3b3b980 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.255+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 msgr2=0x562ef3b44350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.255+0000 7ff5a4964740 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff58802c810 tx=0x7ff58804b000 comp rx=0 tx=0).stop 2024-04-02T13:53:55.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.255+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 shutdown_connections 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b44e80 0x562ef3b3cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b3c2a0 0x562ef3b44890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 --2- 172.21.15.122:0/968581115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b44350 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 >> 172.21.15.122:0/968581115 conn(0x562ef3b360b0 msgr2=0x562ef3b36490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 shutdown_connections 2024-04-02T13:53:55.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 -- 172.21.15.122:0/968581115 wait complete. 2024-04-02T13:53:55.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.256+0000 7ff5a4964740 1 -- start start 2024-04-02T13:53:55.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.257+0000 7ff5a4964740 1 Processor -- start 2024-04-02T13:53:55.272 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:54 smithi188 ceph-mon[36117]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 661 op/s 2024-04-02T13:53:55.272 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:54 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1532301512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.257+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.257+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b3e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff5a4964740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b44e80 0x562ef3b3ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff5a4964740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff594142540 con 0x562ef3b3c2a0 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff5a4964740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff594141db0 con 0x562ef3b3b980 2024-04-02T13:53:55.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff5a4964740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff59413f030 con 0x562ef3b44e80 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff5a4964740 10 client.0 _ll_register_callbacks cb 0x7ff5900096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39690/0 (socket says 172.21.15.122:39690) 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 -- 172.21.15.122:0/2062419443 learned_addr learned my addr 172.21.15.122:0/2062419443 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b44e80 0x562ef3b3ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 -- 172.21.15.122:0/2062419443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b44e80 msgr2=0x562ef3b3ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b44e80 0x562ef3b3ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 -- 172.21.15.122:0/2062419443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 msgr2=0x562ef3b3e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b3e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b3e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59affd640 1 -- 172.21.15.122:0/2062419443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef3b3f0b0 con 0x562ef3b3b980 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.258+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b3e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff59affd640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff588005c40 tx=0x7ff58802c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcc62b8ff9961673 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff57affd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5880050e0 con 0x562ef3b3b980 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7ff588034440 con 0x562ef3b3b980 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff588004ba0 con 0x562ef3b3b980 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7ff588005280 con 0x562ef3b3b980 2024-04-02T13:53:55.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff5a4964740 10 client.29810 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x562ef3b3f0b0 con 0x562ef3b3b980 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.259+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff588034b20 con 0x562ef3b3b980 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7ff58805e6e0 con 0x562ef3b3b980 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 10 client.29810 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 10 client.29810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 20 client.29810 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3bc3680 0x562ef3bc5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 4 client.29810 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff588004ba0 con 0x562ef3bc3680 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.260+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3bc3680 0x562ef3bc5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff59b7fe640 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3bc3680 0x562ef3bc5a70 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8579c3dbe4a0044 server_cookie=cf5b648a34e11313 in_seq=0 out_seq=0 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff57affd640 10 client.29810 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff57affd640 1 -- 172.21.15.122:0/2062419443 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff588004ba0 con 0x562ef3bc3680 2024-04-02T13:53:55.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff57affd640 10 client.29810 handle_command_reply: tid=0 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff5a4964740 1 client.29810 shutdown 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 msgr2=0x562ef3b626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff588005c40 tx=0x7ff58802c640 comp rx=0 tx=0).stop 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 shutdown_connections 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.261+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562ef3bc3680 0x562ef3bc5a70 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562ef3b44e80 0x562ef3b3ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562ef3b3c2a0 0x562ef3b3e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 --2- 172.21.15.122:0/2062419443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562ef3b3b980 0x562ef3b626a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 >> 172.21.15.122:0/2062419443 conn(0x562ef3b360b0 msgr2=0x562ef3b3cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff57affd640 0 client.29810 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 shutdown_connections 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 1 -- 172.21.15.122:0/2062419443 wait complete. 2024-04-02T13:53:55.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.262+0000 7ff5a4964740 20 client.29810 trim_cache size 0 max 16384 2024-04-02T13:53:55.278 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:53:55.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 msgr2=0x7ff58c0a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 0x7ff58c0a6290 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff59c139720 tx=0x7ff59c132fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 msgr2=0x7ff59413a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:55.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff59c051160 tx=0x7ff59c078490 comp rx=0 tx=0).stop 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 shutdown_connections 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff58c0a3dd0 0x7ff58c0a6290 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff594008220 0x7ff59413a790 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff59409b0d0 0x7ff594007ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 --2- 172.21.15.122:0/1930329746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff594096200 0x7ff5940077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 >> 172.21.15.122:0/1930329746 conn(0x7ff59400b860 msgr2=0x7ff594094230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 shutdown_connections 2024-04-02T13:53:55.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:55.263+0000 7ff5a0b19640 1 -- 172.21.15.122:0/1930329746 wait complete. 2024-04-02T13:53:55.280 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:54 smithi122 ceph-mon[32114]: pgmap v589: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 661 op/s 2024-04-02T13:53:55.280 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:54 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1532301512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:53:55.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167126 28.08 MB/sec execute 195 sec latency 66.941 ms 2024-04-02T13:53:56.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:53:56.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:53:56.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:53:56.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167614 28.06 MB/sec execute 196 sec latency 82.915 ms 2024-04-02T13:53:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.449+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/2759268731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 msgr2=0x7f9bbc114cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.449+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/2759268731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc114cb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009ca0 tx=0x7f9bb402c190 comp rx=0 tx=0).stop 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/2759268731 shutdown_connections 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/2759268731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc114cb0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/2759268731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/2759268731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 0x7f9bbc100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/2759268731 >> 172.21.15.122:0/2759268731 conn(0x7f9bbc075db0 msgr2=0x7f9bbc0761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/2759268731 shutdown_connections 2024-04-02T13:53:56.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.450+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/2759268731 wait complete. 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 Processor -- start 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 -- start start 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 0x7f9bbc110920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.451+0000 7f9bc2cf2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc1184d0 con 0x7f9bbc1128c0 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bc2cf2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc118350 con 0x7f9bbc100430 2024-04-02T13:53:56.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bc2cf2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9bbc118650 con 0x7f9bbc100d50 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bbbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54936/0 (socket says 172.21.15.122:54936) 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bbbfff640 1 -- 172.21.15.122:0/1046928728 learned_addr learned my addr 172.21.15.122:0/1046928728 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 0x7f9bbc110920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bc128f640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.452+0000 7f9bbbfff640 1 -- 172.21.15.122:0/1046928728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 msgr2=0x7f9bbc110920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 0x7f9bbc110920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bbbfff640 1 -- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 msgr2=0x7f9bbc109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bbbfff640 1 -- 172.21.15.122:0/1046928728 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bbc10a4f0 con 0x7f9bbc100d50 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bc128f640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:53:56.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.453+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000d950 tx=0x7f9bb000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.456+0000 7f9bb9ffb640 1 -- 172.21.15.122:0/1046928728 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0017070 con 0x7f9bbc100d50 2024-04-02T13:53:56.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.456+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bbc10a7e0 con 0x7f9bbc100d50 2024-04-02T13:53:56.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.456+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bbc10e770 con 0x7f9bbc100d50 2024-04-02T13:53:56.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.458+0000 7f9bb9ffb640 1 -- 172.21.15.122:0/1046928728 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb0009c30 con 0x7f9bbc100d50 2024-04-02T13:53:56.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.458+0000 7f9bb9ffb640 1 -- 172.21.15.122:0/1046928728 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000f6b0 con 0x7f9bbc100d50 2024-04-02T13:53:56.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.461+0000 7f9bc59b5740 1 Processor -- start 2024-04-02T13:53:56.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 -- start start 2024-04-02T13:53:56.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899d07c0 0x564d899d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d8992b770 con 0x564d899c7be0 2024-04-02T13:53:56.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d898d3960 con 0x564d899c72c0 2024-04-02T13:53:56.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d899278c0 con 0x564d899d07c0 2024-04-02T13:53:56.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.462+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53772/0 (socket says 172.21.15.122:53772) 2024-04-02T13:53:56.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2226731762 learned_addr learned my addr 172.21.15.122:0/2226731762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2226731762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899d07c0 msgr2=0x564d899d2bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899d07c0 0x564d899d2bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2226731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 msgr2=0x564d899c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2226731762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d897077e0 con 0x564d899c7be0 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.463+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00bcb0 tx=0x7f9bac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e22fd43fdbf44f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.464+0000 7f9ba2ffd640 1 -- 172.21.15.122:0/2226731762 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac015070 con 0x564d899c7be0 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.464+0000 7f9ba2ffd640 1 -- 172.21.15.122:0/2226731762 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac01a940 con 0x564d899c7be0 2024-04-02T13:53:56.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.464+0000 7f9ba2ffd640 1 -- 172.21.15.122:0/2226731762 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bac011aa0 con 0x564d899c7be0 2024-04-02T13:53:56.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2226731762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 msgr2=0x564d899d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00bcb0 tx=0x7f9bac00f730 comp rx=0 tx=0).stop 2024-04-02T13:53:56.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2226731762 shutdown_connections 2024-04-02T13:53:56.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899d07c0 0x564d899d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899c7be0 0x564d899d0280 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2226731762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.465+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2226731762 >> 172.21.15.122:0/2226731762 conn(0x564d899c19f0 msgr2=0x564d899c1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.466+0000 7f9bb9ffb640 1 -- 172.21.15.122:0/1046928728 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9bb000f850 con 0x7f9bbc100d50 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.467+0000 7f9bb9ffb640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 0x7f9ba40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.467+0000 7f9bb9ffb640 1 -- 172.21.15.122:0/1046928728 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f9bb00d06a0 con 0x7f9bbc100d50 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.468+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2226731762 shutdown_connections 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.468+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 0x7f9ba40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.469+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 0x7f9ba40a63e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ba40 tx=0x7f9bac011680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.471+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2226731762 wait complete. 2024-04-02T13:53:56.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.472+0000 7f9bc59b5740 1 -- start start 2024-04-02T13:53:56.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.472+0000 7f9bc59b5740 1 Processor -- start 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564d899d35b0 con 0x564d899ce3b0 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x564d898e6d20 con 0x564d899c7be0 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564d899d3730 con 0x564d899c72c0 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc59b5740 10 client.0 _ll_register_callbacks cb 0x564d898b3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54940/0 (socket says 172.21.15.122:54940) 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.473+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2054526398 learned_addr learned my addr 172.21.15.122:0/2054526398 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 -- 172.21.15.122:0/2054526398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 msgr2=0x564d899cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 -- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 msgr2=0x564d899cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.474+0000 7f9bc128f640 1 -- 172.21.15.122:0/2054526398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d89767f70 con 0x564d899ce3b0 2024-04-02T13:53:56.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.477+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dcab0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009bf0 tx=0x7f9bb4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db167bd755e28403 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.477+0000 7f9b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:56.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.477+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4039050 con 0x564d899ce3b0 2024-04-02T13:53:56.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.477+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f9bb40328c0 con 0x564d899ce3b0 2024-04-02T13:53:56.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.478+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bac01a940 con 0x564d899ce3b0 2024-04-02T13:53:56.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.478+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f9bb4032ac0 con 0x564d899ce3b0 2024-04-02T13:53:56.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.479+0000 7f9bc59b5740 10 client.20868 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:56.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.479+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x564d895ff170 con 0x564d899ce3b0 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.485+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bac01a940 con 0x564d899ce3b0 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.485+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f9bb4040450 con 0x564d899ce3b0 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.485+0000 7f9bc59b5740 10 client.20868 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc59b5740 10 client.20868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc59b5740 20 client.20868 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d899e13e0 0x564d899e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc59b5740 4 client.20868 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:56.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d89938cb0 con 0x564d899e13e0 2024-04-02T13:53:56.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.486+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d899e13e0 0x564d899e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.490+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d899e13e0 0x564d899e3840 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa1fda5c9a31c56 server_cookie=aef694f2c182ab4 in_seq=0 out_seq=0 2024-04-02T13:53:56.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.490+0000 7f9b9bfff640 10 client.20868 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.495+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2054526398 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d89938cb0 con 0x564d899e13e0 2024-04-02T13:53:56.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.495+0000 7f9b9bfff640 10 client.20868 handle_command_reply: tid=0 2024-04-02T13:53:56.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.496+0000 7f9bc59b5740 1 client.20868 shutdown 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 msgr2=0x564d899dcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dcab0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009bf0 tx=0x7f9bb4003ba0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 shutdown_connections 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d899e13e0 0x564d899e3840 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dcab0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2054526398 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 >> 172.21.15.122:0/2054526398 conn(0x564d899c19f0 msgr2=0x564d899c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.497+0000 7f9b9bfff640 0 client.20868 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.498+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 shutdown_connections 2024-04-02T13:53:56.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.499+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2054526398 wait complete. 2024-04-02T13:53:56.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.499+0000 7f9bc59b5740 20 client.20868 trim_cache size 0 max 16384 2024-04-02T13:53:56.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.537+0000 7f9bc59b5740 1 Processor -- start 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 -- start start 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899e9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 0x564d899e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8010130 con 0x564d899ce3b0 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8011d30 con 0x564d899c72c0 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.538+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8010800 con 0x564d899c7be0 2024-04-02T13:53:56.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899e9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899e9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39736/0 (socket says 172.21.15.122:39736) 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/731253751 learned_addr learned my addr 172.21.15.122:0/731253751 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 0x564d899e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 -- 172.21.15.122:0/731253751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 msgr2=0x564d899e9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899e9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 -- 172.21.15.122:0/731253751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 msgr2=0x564d899dc500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dc500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.539+0000 7f9bbbfff640 1 -- 172.21.15.122:0/731253751 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d899cbe80 con 0x564d899c7be0 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.540+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 0x564d899e9690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00cd9a0 tx=0x7f9bb009bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbda77b118800e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.541+0000 7f9ba27fc640 1 -- 172.21.15.122:0/731253751 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0020050 con 0x564d899c7be0 2024-04-02T13:53:56.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.541+0000 7f9ba27fc640 1 -- 172.21.15.122:0/731253751 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb009c1f0 con 0x564d899c7be0 2024-04-02T13:53:56.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.541+0000 7f9ba27fc640 1 -- 172.21.15.122:0/731253751 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb00a0170 con 0x564d899c7be0 2024-04-02T13:53:56.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.542+0000 7f9bc59b5740 1 -- 172.21.15.122:0/731253751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 msgr2=0x564d899e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.542+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 0x564d899e9690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00cd9a0 tx=0x7f9bb009bdf0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.545+0000 7f9bc59b5740 1 -- 172.21.15.122:0/731253751 shutdown_connections 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.545+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899dc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.545+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c7be0 0x564d899e9690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.545+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/731253751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c72c0 0x564d899e9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.545+0000 7f9bc59b5740 1 -- 172.21.15.122:0/731253751 >> 172.21.15.122:0/731253751 conn(0x564d899c19f0 msgr2=0x564d899c36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.546+0000 7f9bc59b5740 1 -- 172.21.15.122:0/731253751 shutdown_connections 2024-04-02T13:53:56.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.546+0000 7f9bc59b5740 1 -- 172.21.15.122:0/731253751 wait complete. 2024-04-02T13:53:56.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.547+0000 7f9bc59b5740 1 -- start start 2024-04-02T13:53:56.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.547+0000 7f9bc59b5740 1 Processor -- start 2024-04-02T13:53:56.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8011660 con 0x564d899ce3b0 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8010ed0 con 0x564d899c7be0 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba800e130 con 0x564d899c72c0 2024-04-02T13:53:56.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.548+0000 7f9bc59b5740 10 client.0 _ll_register_callbacks cb 0x7f9bb4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc0a8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54966/0 (socket says 172.21.15.122:54966) 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc0a8e640 1 -- 172.21.15.122:0/2856655135 learned_addr learned my addr 172.21.15.122:0/2856655135 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bbbfff640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 -- 172.21.15.122:0/2856655135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 msgr2=0x564d899ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 -- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 msgr2=0x564d899cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.549+0000 7f9bc128f640 1 -- 172.21.15.122:0/2856655135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d89a46b90 con 0x564d899ce3b0 2024-04-02T13:53:56.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.550+0000 7f9bc128f640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899cb250 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4040850 tx=0x7f9bb4005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442e2fe0d7db04ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.551+0000 7f9b9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.551+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4039040 con 0x564d899ce3b0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.551+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f9bb40326c0 con 0x564d899ce3b0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.551+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb009c1f0 con 0x564d899ce3b0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.552+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f9bb4032e60 con 0x564d899ce3b0 2024-04-02T13:53:56.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.552+0000 7f9bc59b5740 10 client.20871 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:56.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.552+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x564d89a47ef0 con 0x564d899ce3b0 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.552+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bb009c1f0 con 0x564d899ce3b0 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.552+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f9bb4053460 con 0x564d899ce3b0 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 10 client.20871 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 10 client.20871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 20 client.20871 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d89a485f0 0x564d89a4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 4 client.20871 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:53:56.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.553+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564d89933000 con 0x564d89a485f0 2024-04-02T13:53:56.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.554+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d89a485f0 0x564d89a4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.555+0000 7f9bc0a8e640 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d89a485f0 0x564d89a4aa10 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3968e2bc4458be2d server_cookie=3fc96e6a03bb2f04 in_seq=0 out_seq=0 2024-04-02T13:53:56.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.556+0000 7f9b9bfff640 10 client.20871 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.559+0000 7f9b9bfff640 1 -- 172.21.15.122:0/2856655135 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564d89933000 con 0x564d89a485f0 2024-04-02T13:53:56.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.559+0000 7f9b9bfff640 10 client.20871 handle_command_reply: tid=0 2024-04-02T13:53:56.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.559+0000 7f9bc59b5740 1 client.20871 shutdown 2024-04-02T13:53:56.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.562+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 msgr2=0x564d899cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.562+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899cb250 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4040850 tx=0x7f9bb4005c60 comp rx=0 tx=0).stop 2024-04-02T13:53:56.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 shutdown_connections 2024-04-02T13:53:56.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564d89a485f0 0x564d89a4aa10 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564d899ce3b0 0x564d899cb250 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x564d899c7be0 0x564d899cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 --2- 172.21.15.122:0/2856655135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564d899c72c0 0x564d899ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 >> 172.21.15.122:0/2856655135 conn(0x564d899c19f0 msgr2=0x564d899c2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9b9bfff640 0 client.20871 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 shutdown_connections 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.563+0000 7f9bc59b5740 1 -- 172.21.15.122:0/2856655135 wait complete. 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.564+0000 7f9bc59b5740 20 client.20871 trim_cache size 0 max 16384 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "1cae2aaf-41a9-42a1-94b3-ce104bbd99ff", 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:53:56.568 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 msgr2=0x7f9ba40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 0x7f9ba40a63e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00ba40 tx=0x7f9bac011680 comp rx=0 tx=0).stop 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 msgr2=0x7f9bbc1099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000d950 tx=0x7f9bb000de10 comp rx=0 tx=0).stop 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 shutdown_connections 2024-04-02T13:53:56.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9ba40a3f20 0x7f9ba40a63e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bbc1128c0 0x7f9bbc109f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9bbc100d50 0x7f9bbc1099f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 --2- 172.21.15.122:0/1046928728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9bbc100430 0x7f9bbc110920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 >> 172.21.15.122:0/1046928728 conn(0x7f9bbc075db0 msgr2=0x7f9bbc0fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 shutdown_connections 2024-04-02T13:53:56.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.565+0000 7f9bc2cf2640 1 -- 172.21.15.122:0/1046928728 wait complete. 2024-04-02T13:53:56.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:53:56.585 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:53:56.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.751+0000 7f08b02e4640 1 -- 172.21.15.122:0/673350791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a8105470 msgr2=0x7f08a810d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.751+0000 7f08b02e4640 1 --2- 172.21.15.122:0/673350791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a8105470 0x7f08a810d040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f08a000acf0 tx=0x7f08a002d260 comp rx=0 tx=0).stop 2024-04-02T13:53:56.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.753+0000 7f08b02e4640 1 -- 172.21.15.122:0/673350791 shutdown_connections 2024-04-02T13:53:56.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.753+0000 7f08b02e4640 1 --2- 172.21.15.122:0/673350791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a8105470 0x7f08a810d040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.753+0000 7f08b02e4640 1 --2- 172.21.15.122:0/673350791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8100d50 0x7f08a8104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.753+0000 7f08b02e4640 1 --2- 172.21.15.122:0/673350791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a81003a0 0x7f08a8100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.753+0000 7f08b02e4640 1 -- 172.21.15.122:0/673350791 >> 172.21.15.122:0/673350791 conn(0x7f08a8076fc0 msgr2=0x7f08a80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.754+0000 7f08b02e4640 1 -- 172.21.15.122:0/673350791 shutdown_connections 2024-04-02T13:53:56.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.754+0000 7f08b02e4640 1 -- 172.21.15.122:0/673350791 wait complete. 2024-04-02T13:53:56.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.755+0000 7f08b02e4640 1 Processor -- start 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 -- start start 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a81003a0 0x7f08a80732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a806c480 0x7f08a806c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810fd00 con 0x7f08a806c480 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810fb80 con 0x7f08a81003a0 2024-04-02T13:53:56.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.757+0000 7f08b02e4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f08a810fe80 con 0x7f08a8105470 2024-04-02T13:53:56.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.758+0000 7f08ad87f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.758+0000 7f08ad87f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:54994/0 (socket says 172.21.15.122:54994) 2024-04-02T13:53:56.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.758+0000 7f08ad87f640 1 -- 172.21.15.122:0/1252815303 learned_addr learned my addr 172.21.15.122:0/1252815303 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.758+0000 7f08ae080640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a81003a0 0x7f08a80732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.758+0000 7f08ae881640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a806c480 0x7f08a806c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 -- 172.21.15.122:0/1252815303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a81003a0 msgr2=0x7f08a80732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a81003a0 0x7f08a80732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 -- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a806c480 msgr2=0x7f08a806c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a806c480 0x7f08a806c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 -- 172.21.15.122:0/1252815303 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a8070dc0 con 0x7f08a8105470 2024-04-02T13:53:56.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.759+0000 7f08ad87f640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f089800e9f0 tx=0x7f089800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.762+0000 7f089f7fe640 1 -- 172.21.15.122:0/1252815303 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089800ccc0 con 0x7f08a8105470 2024-04-02T13:53:56.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.762+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a80710b0 con 0x7f08a8105470 2024-04-02T13:53:56.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.762+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a8071600 con 0x7f08a8105470 2024-04-02T13:53:56.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.762+0000 7f089f7fe640 1 -- 172.21.15.122:0/1252815303 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0898010910 con 0x7f08a8105470 2024-04-02T13:53:56.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.762+0000 7f089f7fe640 1 -- 172.21.15.122:0/1252815303 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898018960 con 0x7f08a8105470 2024-04-02T13:53:56.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.766+0000 7f08b2fa4740 1 Processor -- start 2024-04-02T13:53:56.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.766+0000 7f08b2fa4740 1 -- start start 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f089f7fe640 1 -- 172.21.15.122:0/1252815303 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0898010ac0 con 0x7f08a8105470 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d09661c8d0 0x55d09661ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d096625dd0 0x55d0966281c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d096580830 con 0x55d096625dd0 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d096528820 con 0x55d09661c8d0 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d09657c7e0 con 0x55d09661d1f0 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08ae080640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08ae080640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:55004/0 (socket says 172.21.15.122:55004) 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08ae080640 1 -- 172.21.15.122:0/244142323 learned_addr learned my addr 172.21.15.122:0/244142323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08ae881640 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d09661c8d0 0x55d09661ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.767+0000 7f08ad87f640 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d096625dd0 0x55d0966281c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 -- 172.21.15.122:0/244142323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d09661c8d0 msgr2=0x55d09661ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d09661c8d0 0x55d09661ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 -- 172.21.15.122:0/244142323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d096625dd0 msgr2=0x55d0966281c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d096625dd0 0x55d0966281c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 -- 172.21.15.122:0/244142323 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09635e7d0 con 0x55d09661d1f0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f08ae080640 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b820 tx=0x7f08a400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96811a3f91c50b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f089f7fe640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 0x7f08940a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f089cbfe640 1 -- 172.21.15.122:0/244142323 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400d950 con 0x55d09661d1f0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f089cbfe640 1 -- 172.21.15.122:0/244142323 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4019070 con 0x55d09661d1f0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.768+0000 7f089cbfe640 1 -- 172.21.15.122:0/244142323 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40123e0 con 0x55d09661d1f0 2024-04-02T13:53:56.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08ae881640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 0x7f08940a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f089f7fe640 1 -- 172.21.15.122:0/1252815303 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f08980d0dd0 con 0x7f08a8105470 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 -- 172.21.15.122:0/244142323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 msgr2=0x55d096625890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b820 tx=0x7f08a400bce0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08ae881640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 0x7f08940a6360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f08a000aab0 tx=0x7f08a0005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 -- 172.21.15.122:0/244142323 shutdown_connections 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d096625dd0 0x55d0966281c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661d1f0 0x55d096625890 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/244142323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d09661c8d0 0x55d09661ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 -- 172.21.15.122:0/244142323 >> 172.21.15.122:0/244142323 conn(0x55d096617000 msgr2=0x55d0966173e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 -- 172.21.15.122:0/244142323 shutdown_connections 2024-04-02T13:53:56.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.769+0000 7f08b2fa4740 1 -- 172.21.15.122:0/244142323 wait complete. 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.770+0000 7f08b2fa4740 1 -- start start 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.770+0000 7f08b2fa4740 1 Processor -- start 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.770+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096625640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.770+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096625dd0 0x55d096622f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d096628bc0 con 0x55d09661d1f0 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d09653bd30 con 0x55d096625dd0 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08ad87f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08ad87f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53834/0 (socket says 172.21.15.122:53834) 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08ad87f640 1 -- 172.21.15.122:0/259090107 learned_addr learned my addr 172.21.15.122:0/259090107 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08ae080640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096625640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08ae881640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096625dd0 0x55d096622f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d096628d40 con 0x55d09661c8d0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.771+0000 7f08b2fa4740 10 client.0 _ll_register_callbacks cb 0x55d09653b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 -- 172.21.15.122:0/259090107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 msgr2=0x55d096625640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096625640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 -- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096625dd0 msgr2=0x55d096622f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096625dd0 0x55d096622f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 -- 172.21.15.122:0/259090107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0962943f0 con 0x55d09661d1f0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.772+0000 7f08ad87f640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f0898009730 tx=0x7f089809c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69f5331e9a0b173f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f0891ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980ce560 con 0x55d09661d1f0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f089809bf70 con 0x55d09661d1f0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a4019070 con 0x55d09661d1f0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f0898010db0 con 0x55d09661d1f0 2024-04-02T13:53:56.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f08b2fa4740 10 client.20880 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.773+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55d0962540f0 con 0x55d09661d1f0 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4019070 con 0x55d09661d1f0 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f0898018b00 con 0x55d09661d1f0 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 10 client.20880 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 10 client.20880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 20 client.20880 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:56.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096635da0 0x55d096638200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 4 client.20880 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:53:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09658dd70 con 0x55d096635da0 2024-04-02T13:53:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.774+0000 7f08ae080640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096635da0 0x55d096638200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.777+0000 7f0891ffb640 10 client.20880 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.777+0000 7f08ae080640 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096635da0 0x55d096638200 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f698a7876600efb server_cookie=f4c83447b4a3c209 in_seq=0 out_seq=0 2024-04-02T13:53:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f0891ffb640 1 -- 172.21.15.122:0/259090107 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d09658dd70 con 0x55d096635da0 2024-04-02T13:53:56.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f0891ffb640 10 client.20880 handle_command_reply: tid=0 2024-04-02T13:53:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 client.20880 shutdown 2024-04-02T13:53:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 msgr2=0x55d0966229d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f0898009730 tx=0x7f089809c7b0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 shutdown_connections 2024-04-02T13:53:56.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096635da0 0x55d096638200 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096625dd0 0x55d096622f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d0966229d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f0891ffb640 0 client.20880 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/259090107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096625640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.778+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 >> 172.21.15.122:0/259090107 conn(0x55d096617000 msgr2=0x55d096618d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.779+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 shutdown_connections 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.779+0000 7f08b2fa4740 1 -- 172.21.15.122:0/259090107 wait complete. 2024-04-02T13:53:56.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.779+0000 7f08b2fa4740 20 client.20880 trim_cache size 0 max 16384 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.808+0000 7f08b2fa4740 1 Processor -- start 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 -- start start 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096621910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096625100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f088c0100e0 con 0x55d09661d1f0 2024-04-02T13:53:56.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f088c011ce0 con 0x55d096617000 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08b2fa4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f088c0107b0 con 0x55d09661c8d0 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae881640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096625100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39792/0 (socket says 172.21.15.122:39792) 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 -- 172.21.15.122:0/4197066633 learned_addr learned my addr 172.21.15.122:0/4197066633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae881640 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096625100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53836/0 (socket says 172.21.15.122:53836) 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 -- 172.21.15.122:0/4197066633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 msgr2=0x55d096621910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096621910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 -- 172.21.15.122:0/4197066633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 msgr2=0x55d096625100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096625100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.809+0000 7f08ae080640 1 -- 172.21.15.122:0/4197066633 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d096616e30 con 0x55d096617000 2024-04-02T13:53:56.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.810+0000 7f08ae080640 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08a400f010 tx=0x7f08a400f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b08d3cfb4f659c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.810+0000 7f0893fff640 1 -- 172.21.15.122:0/4197066633 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4024070 con 0x55d096617000 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.810+0000 7f0893fff640 1 -- 172.21.15.122:0/4197066633 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a402b070 con 0x55d096617000 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.810+0000 7f0893fff640 1 -- 172.21.15.122:0/4197066633 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40254d0 con 0x55d096617000 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- 172.21.15.122:0/4197066633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 msgr2=0x55d096631b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f08a400f010 tx=0x7f08a400f040 comp rx=0 tx=0).stop 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- 172.21.15.122:0/4197066633 shutdown_connections 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096625100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d096621910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/4197066633 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d096631b10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- 172.21.15.122:0/4197066633 >> 172.21.15.122:0/4197066633 conn(0x55d096623070 msgr2=0x55d096616240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- 172.21.15.122:0/4197066633 shutdown_connections 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- 172.21.15.122:0/4197066633 wait complete. 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 -- start start 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.811+0000 7f08b2fa4740 1 Processor -- start 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d09661fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096620320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08ae080640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08ae080640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:39798/0 (socket says 172.21.15.122:39798) 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08ae080640 1 -- 172.21.15.122:0/2806108600 learned_addr learned my addr 172.21.15.122:0/2806108600 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f088c011610 con 0x55d09661d1f0 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f088c010e80 con 0x55d096617000 2024-04-02T13:53:56.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f088c00e0e0 con 0x55d09661c8d0 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.812+0000 7f08ad87f640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d09661fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 -- 172.21.15.122:0/2806108600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 msgr2=0x55d09661fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d09661fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 -- 172.21.15.122:0/2806108600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 msgr2=0x55d096620320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096620320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0966281c0 con 0x55d096617000 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08b2fa4740 10 client.0 _ll_register_callbacks cb 0x7f0898098bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a402b070 con 0x55d096617000 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08ae080640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f08a4028040 tx=0x7f08a400b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10da8642b21a24d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f0891ffb640 10 client.29840 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f0891ffb640 1 -- 172.21.15.122:0/2806108600 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4024040 con 0x55d096617000 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f0891ffb640 1 -- 172.21.15.122:0/2806108600 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9525+0+0 (secure 0 0 0) 0x7f08a40146f0 con 0x55d096617000 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f0891ffb640 1 -- 172.21.15.122:0/2806108600 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f08a4014970 con 0x55d096617000 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08b2fa4740 10 client.29840 fetch_fsmap learned FSMap version 73 2024-04-02T13:53:56.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.813+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=73}) -- 0x55d09669c4a0 con 0x55d096617000 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f0891ffb640 1 -- 172.21.15.122:0/2806108600 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f08a4013070 con 0x55d096617000 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 10 client.29840 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 10 client.29840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 20 client.29840 populate_metadata read hostname 'smithi122' 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096625dd0 0x55d09669e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 4 client.29840 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:53:56.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d0965880c0 con 0x55d096625dd0 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.814+0000 7f08ad87f640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096625dd0 0x55d09669e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.815+0000 7f08ad87f640 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096625dd0 0x55d09669e9c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd974f033b5fef6 server_cookie=64ce537ab85f0a8 in_seq=0 out_seq=0 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.815+0000 7f0891ffb640 10 client.29840 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.815+0000 7f0891ffb640 1 -- 172.21.15.122:0/2806108600 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d0965880c0 con 0x55d096625dd0 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.815+0000 7f0891ffb640 10 client.29840 handle_command_reply: tid=0 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.815+0000 7f08b2fa4740 1 client.29840 shutdown 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 msgr2=0x55d09661f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f08a4028040 tx=0x7f08a400b280 comp rx=0 tx=0).stop 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 shutdown_connections 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d096625dd0 0x55d09669e9c0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d09661d1f0 0x55d096620320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d09661c8d0 0x55d09661fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f0891ffb640 0 client.29840 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 --2- 172.21.15.122:0/2806108600 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d096617000 0x55d09661f8a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 >> 172.21.15.122:0/2806108600 conn(0x55d096623070 msgr2=0x55d0966159b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.816+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 shutdown_connections 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.817+0000 7f08b2fa4740 1 -- 172.21.15.122:0/2806108600 wait complete. 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.817+0000 7f08b2fa4740 20 client.29840 trim_cache size 0 max 16384 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "1cae2aaf-41a9-42a1-94b3-ce104bbd99ff": { 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "1cae2aaf-41a9-42a1-94b3-ce104bbd99ff", 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:53:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 msgr2=0x7f08940a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 0x7f08940a6360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f08a000aab0 tx=0x7f08a0005fd0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 msgr2=0x7f08a80737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f089800e9f0 tx=0x7f089800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 shutdown_connections 2024-04-02T13:53:56.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f08940a3ea0 0x7f08940a6360 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08a806c480 0x7f08a806c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08a8105470 0x7f08a80737e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 --2- 172.21.15.122:0/1252815303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a81003a0 0x7f08a80732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 >> 172.21.15.122:0/1252815303 conn(0x7f08a8076fc0 msgr2=0x7f08a8103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 shutdown_connections 2024-04-02T13:53:56.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:53:56.818+0000 7f08b02e4640 1 -- 172.21.15.122:0/1252815303 wait complete. 2024-04-02T13:53:56.839 INFO:tasks.cephfs.filesystem:scrub status for tag:1cae2aaf-41a9-42a1-94b3-ce104bbd99ff - {'path': '/', 'tag': '1cae2aaf-41a9-42a1-94b3-ce104bbd99ff', 'options': 'recursive,force'} 2024-04-02T13:53:57.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 167990 27.99 MB/sec execute 197 sec latency 136.196 ms 2024-04-02T13:53:57.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-04-02T13:53:57.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:53:57.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:57.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:57.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:57.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:57.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:57.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:57.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:57.487 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:57 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:57.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-04-02T13:53:59.628 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:53:59.628 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.628 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.628 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.628 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.629 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.629 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.629 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.629 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:57 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: pgmap v590: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 10 MiB/s wr, 566 op/s 2024-04-02T13:53:59.630 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.631 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:57 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:58 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:53:59 smithi187 ceph-mon[36064]: pgmap v591: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-02T13:53:59.632 INFO:tasks.workunit.client.0.smithi122.stdout: 10 168547 27.99 MB/sec execute 198 sec latency 104.954 ms 2024-04-02T13:53:59.632 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:58 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.633 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:53:59 smithi188 ceph-mon[36117]: pgmap v591: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.634 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:58 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:53:59.635 INFO:tasks.workunit.client.0.smithi122.stdout: 10 169318 28.04 MB/sec execute 199 sec latency 88.789 ms 2024-04-02T13:53:59.635 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:53:59 smithi122 ceph-mon[32114]: pgmap v591: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-02T13:54:00.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 169962 28.10 MB/sec execute 200 sec latency 131.515 ms 2024-04-02T13:54:01.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170521 28.13 MB/sec execute 201 sec latency 239.212 ms 2024-04-02T13:54:01.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:01 smithi187 ceph-mon[36064]: pgmap v592: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 412 op/s 2024-04-02T13:54:01.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:01 smithi188 ceph-mon[36117]: pgmap v592: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 412 op/s 2024-04-02T13:54:01.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:01 smithi122 ceph-mon[32114]: pgmap v592: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.1 MiB/s wr, 412 op/s 2024-04-02T13:54:02.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170741 28.04 MB/sec execute 202 sec latency 358.728 ms 2024-04-02T13:54:03.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170849 27.92 MB/sec execute 203 sec latency 365.215 ms 2024-04-02T13:54:03.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:03 smithi187 ceph-mon[36064]: pgmap v593: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 627 op/s 2024-04-02T13:54:03.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:03 smithi187 ceph-mon[36064]: osdmap e120: 12 total, 12 up, 12 in 2024-04-02T13:54:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:03 smithi188 ceph-mon[36117]: pgmap v593: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 627 op/s 2024-04-02T13:54:03.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:03 smithi188 ceph-mon[36117]: osdmap e120: 12 total, 12 up, 12 in 2024-04-02T13:54:03.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:03 smithi122 ceph-mon[32114]: pgmap v593: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 627 op/s 2024-04-02T13:54:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:03 smithi122 ceph-mon[32114]: osdmap e120: 12 total, 12 up, 12 in 2024-04-02T13:54:04.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170869 27.80 MB/sec execute 204 sec latency 331.712 ms 2024-04-02T13:54:05.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 170974 27.71 MB/sec execute 205 sec latency 357.320 ms 2024-04-02T13:54:05.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:05 smithi187 ceph-mon[36064]: pgmap v595: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 535 op/s 2024-04-02T13:54:05.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:05 smithi188 ceph-mon[36117]: pgmap v595: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 535 op/s 2024-04-02T13:54:05.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:05 smithi122 ceph-mon[32114]: pgmap v595: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 535 op/s 2024-04-02T13:54:06.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 171292 27.66 MB/sec execute 206 sec latency 323.755 ms 2024-04-02T13:54:07.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 171727 27.61 MB/sec execute 207 sec latency 182.563 ms 2024-04-02T13:54:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:07 smithi187 ceph-mon[36064]: pgmap v596: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2024-04-02T13:54:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:07 smithi188 ceph-mon[36117]: pgmap v596: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2024-04-02T13:54:07.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:07 smithi122 ceph-mon[32114]: pgmap v596: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.2 MiB/s wr, 533 op/s 2024-04-02T13:54:08.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 172233 27.60 MB/sec execute 208 sec latency 201.050 ms 2024-04-02T13:54:09.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 172881 27.63 MB/sec execute 209 sec latency 216.954 ms 2024-04-02T13:54:09.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:09 smithi187 ceph-mon[36064]: pgmap v597: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:09.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:09 smithi188 ceph-mon[36117]: pgmap v597: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:09.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:09 smithi122 ceph-mon[32114]: pgmap v597: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:10.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173393 27.66 MB/sec execute 210 sec latency 188.942 ms 2024-04-02T13:54:11.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 173938 27.65 MB/sec execute 211 sec latency 179.396 ms 2024-04-02T13:54:11.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:11 smithi188 ceph-mon[36117]: pgmap v598: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:11.570 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:11 smithi122 ceph-mon[32114]: pgmap v598: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:11.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:11 smithi187 ceph-mon[36064]: pgmap v598: 129 pgs: 129 active+clean; 2.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 607 op/s 2024-04-02T13:54:12.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174111 27.57 MB/sec execute 212 sec latency 700.502 ms 2024-04-02T13:54:12.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: osdmap e121: 12 total, 12 up, 12 in 2024-04-02T13:54:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:12 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: osdmap e121: 12 total, 12 up, 12 in 2024-04-02T13:54:12.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:12 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: osdmap e121: 12 total, 12 up, 12 in 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:12 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:13.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174158 27.45 MB/sec execute 213 sec latency 1273.206 ms 2024-04-02T13:54:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:13 smithi187 ceph-mon[36064]: pgmap v600: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-04-02T13:54:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:13 smithi188 ceph-mon[36117]: pgmap v600: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-04-02T13:54:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:13 smithi122 ceph-mon[32114]: pgmap v600: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.9 MiB/s wr, 490 op/s 2024-04-02T13:54:14.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174313 27.35 MB/sec execute 214 sec latency 890.865 ms 2024-04-02T13:54:15.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174511 27.27 MB/sec execute 215 sec latency 155.564 ms 2024-04-02T13:54:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:15 smithi187 ceph-mon[36064]: pgmap v601: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 572 op/s 2024-04-02T13:54:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:15 smithi188 ceph-mon[36117]: pgmap v601: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 572 op/s 2024-04-02T13:54:15.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:15 smithi122 ceph-mon[32114]: pgmap v601: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.0 MiB/s wr, 572 op/s 2024-04-02T13:54:16.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 174845 27.25 MB/sec execute 216 sec latency 163.556 ms 2024-04-02T13:54:17.334 INFO:tasks.workunit.client.0.smithi122.stdout: 10 175189 27.24 MB/sec execute 217 sec latency 155.468 ms 2024-04-02T13:54:17.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:17 smithi187 ceph-mon[36064]: pgmap v602: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 582 op/s 2024-04-02T13:54:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:17 smithi188 ceph-mon[36117]: pgmap v602: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 582 op/s 2024-04-02T13:54:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:17 smithi122 ceph-mon[32114]: pgmap v602: 129 pgs: 129 active+clean; 2.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.3 MiB/s wr, 582 op/s 2024-04-02T13:54:18.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 175714 27.20 MB/sec execute 218 sec latency 168.699 ms 2024-04-02T13:54:19.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 176338 27.20 MB/sec execute 219 sec latency 80.679 ms 2024-04-02T13:54:19.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:19 smithi187 ceph-mon[36064]: pgmap v603: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:19 smithi188 ceph-mon[36117]: pgmap v603: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:19 smithi122 ceph-mon[32114]: pgmap v603: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:20.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 177024 27.31 MB/sec execute 220 sec latency 143.622 ms 2024-04-02T13:54:21.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 177595 27.28 MB/sec execute 221 sec latency 78.913 ms 2024-04-02T13:54:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:21 smithi187 ceph-mon[36064]: pgmap v604: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:21 smithi188 ceph-mon[36117]: pgmap v604: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:21.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:21 smithi122 ceph-mon[32114]: pgmap v604: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 726 op/s 2024-04-02T13:54:22.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 178099 27.26 MB/sec execute 222 sec latency 101.444 ms 2024-04-02T13:54:23.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 178797 27.38 MB/sec execute 223 sec latency 83.945 ms 2024-04-02T13:54:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:23 smithi187 ceph-mon[36064]: pgmap v605: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 785 op/s 2024-04-02T13:54:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:23 smithi188 ceph-mon[36117]: pgmap v605: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 785 op/s 2024-04-02T13:54:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:23 smithi122 ceph-mon[32114]: pgmap v605: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 19 MiB/s wr, 785 op/s 2024-04-02T13:54:24.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 179423 27.35 MB/sec execute 224 sec latency 129.668 ms 2024-04-02T13:54:25.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 179931 27.36 MB/sec execute 225 sec latency 107.084 ms 2024-04-02T13:54:25.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:25 smithi187 ceph-mon[36064]: pgmap v606: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-04-02T13:54:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:25 smithi188 ceph-mon[36117]: pgmap v606: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-04-02T13:54:25.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:25 smithi122 ceph-mon[32114]: pgmap v606: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-04-02T13:54:26.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 180553 27.44 MB/sec execute 226 sec latency 103.947 ms 2024-04-02T13:54:26.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:26.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.000+0000 7f568ca86640 1 -- 172.21.15.122:0/2319675278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56881048c0 msgr2=0x7f5688108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.000+0000 7f568ca86640 1 --2- 172.21.15.122:0/2319675278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56881048c0 0x7f5688108900 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f567c009860 tx=0x7f567c02bb90 comp rx=0 tx=0).stop 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 -- 172.21.15.122:0/2319675278 shutdown_connections 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 --2- 172.21.15.122:0/2319675278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 0x7f5688110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 --2- 172.21.15.122:0/2319675278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56881048c0 0x7f5688108900 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 --2- 172.21.15.122:0/2319675278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 0x7f56881042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 -- 172.21.15.122:0/2319675278 >> 172.21.15.122:0/2319675278 conn(0x7f56880ff770 msgr2=0x7f5688101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 -- 172.21.15.122:0/2319675278 shutdown_connections 2024-04-02T13:54:27.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.001+0000 7f568ca86640 1 -- 172.21.15.122:0/2319675278 wait complete. 2024-04-02T13:54:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 Processor -- start 2024-04-02T13:54:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 -- start start 2024-04-02T13:54:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 0x7f568819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 0x7f568819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5688114050 con 0x7f568819ee20 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5688113ed0 con 0x7f5688108fe0 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f568ca86640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56881141d0 con 0x7f5688103f10 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f5686d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f5686d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43946/0 (socket says 172.21.15.122:43946) 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.002+0000 7f5686d9d640 1 -- 172.21.15.122:0/1153449514 learned_addr learned my addr 172.21.15.122:0/1153449514 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5685d9b640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 0x7f568819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f568659c640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 0x7f568819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 -- 172.21.15.122:0/1153449514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 msgr2=0x7f568819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 0x7f568819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 -- 172.21.15.122:0/1153449514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 msgr2=0x7f568819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 0x7f568819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 -- 172.21.15.122:0/1153449514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568819f910 con 0x7f568819ee20 2024-04-02T13:54:27.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.003+0000 7f5686d9d640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f568000efd0 tx=0x7f568000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.004+0000 7f56777fe640 1 -- 172.21.15.122:0/1153449514 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680003b70 con 0x7f568819ee20 2024-04-02T13:54:27.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.004+0000 7f56777fe640 1 -- 172.21.15.122:0/1153449514 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5680003d10 con 0x7f568819ee20 2024-04-02T13:54:27.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.004+0000 7f56777fe640 1 -- 172.21.15.122:0/1153449514 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56800109f0 con 0x7f568819ee20 2024-04-02T13:54:27.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.004+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568819fc00 con 0x7f568819ee20 2024-04-02T13:54:27.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.004+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56881a7360 con 0x7f568819ee20 2024-04-02T13:54:27.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.007+0000 7f568f66d740 1 Processor -- start 2024-04-02T13:54:27.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.007+0000 7f56777fe640 1 -- 172.21.15.122:0/1153449514 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5680004020 con 0x7f568819ee20 2024-04-02T13:54:27.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.008+0000 7f568f66d740 1 -- start start 2024-04-02T13:54:27.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.008+0000 7f568f66d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f568f66d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f568f66d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1cebdd0 0x5585c1cee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f568f66d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585c1c46830 con 0x5585c1ce31f0 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f568f66d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5585c1bee820 con 0x5585c1cebdd0 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f568f66d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5585c1c427e0 con 0x5585c1ce28d0 2024-04-02T13:54:27.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f56777fe640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 0x7f56700a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.009+0000 7f5686d9d640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 0x7f56700a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1cebdd0 0x5585c1cee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46330/0 (socket says 172.21.15.122:46330) 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f5685d9b640 1 -- 172.21.15.122:0/3245157564 learned_addr learned my addr 172.21.15.122:0/3245157564 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f56777fe640 1 -- 172.21.15.122:0/1153449514 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f56800d0f10 con 0x7f568819ee20 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.010+0000 7f568659c640 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5686d9d640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 0x7f56700a6330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f568005d310 tx=0x7f5680067190 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 -- 172.21.15.122:0/3245157564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1cebdd0 msgr2=0x5585c1cee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1cebdd0 0x5585c1cee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 -- 172.21.15.122:0/3245157564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 msgr2=0x5585c1ceb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 -- 172.21.15.122:0/3245157564 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c1a247d0 con 0x5585c1ce28d0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f567c005e00 tx=0x7f567c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18f96d7cdb6f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.011+0000 7f5674ff9640 1 -- 172.21.15.122:0/3245157564 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039040 con 0x5585c1ce28d0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.012+0000 7f5674ff9640 1 -- 172.21.15.122:0/3245157564 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c02ca10 con 0x5585c1ce28d0 2024-04-02T13:54:27.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.012+0000 7f5674ff9640 1 -- 172.21.15.122:0/3245157564 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c0329e0 con 0x5585c1ce28d0 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.012+0000 7f568f66d740 1 -- 172.21.15.122:0/3245157564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 msgr2=0x5585c1ce2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.012+0000 7f568f66d740 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f567c005e00 tx=0x7f567c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 -- 172.21.15.122:0/3245157564 shutdown_connections 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1cebdd0 0x5585c1cee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 --2- 172.21.15.122:0/3245157564 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce2cb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 -- 172.21.15.122:0/3245157564 >> 172.21.15.122:0/3245157564 conn(0x5585c1cdd000 msgr2=0x5585c1cdd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 -- 172.21.15.122:0/3245157564 shutdown_connections 2024-04-02T13:54:27.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.013+0000 7f568f66d740 1 -- 172.21.15.122:0/3245157564 wait complete. 2024-04-02T13:54:27.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.014+0000 7f568f66d740 1 -- start start 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.015+0000 7f568f66d740 1 Processor -- start 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.015+0000 7f568f66d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.015+0000 7f568f66d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ce8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37500/0 (socket says 172.21.15.122:37500) 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ce8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ce8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43958/0 (socket says 172.21.15.122:43958) 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568f66d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1ce9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 -- 172.21.15.122:0/1891382071 learned_addr learned my addr 172.21.15.122:0/1891382071 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5585c1ceeb60 con 0x5585c1ce31f0 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5585c1c01d30 con 0x5585c1ce28d0 2024-04-02T13:54:27.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5585c1ceece0 con 0x5585c1cebdd0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1ce9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f568f66d740 10 client.0 _ll_register_callbacks cb 0x5585c1c01720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 -- 172.21.15.122:0/1891382071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 msgr2=0x5585c1ce9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1ce9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 -- 172.21.15.122:0/1891382071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 msgr2=0x5585c1ce8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ce8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.016+0000 7f5685d9b640 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c195a3f0 con 0x5585c1ce28d0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c02ca10 con 0x5585c1ce28d0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f5685d9b640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f567c0095f0 tx=0x7f567c034d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca2ee4ac70cc5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f566dffb640 10 client.29852 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f566dffb640 1 -- 172.21.15.122:0/1891382071 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039040 con 0x5585c1ce28d0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f566dffb640 1 -- 172.21.15.122:0/1891382071 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f567c004030 con 0x5585c1ce28d0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f566dffb640 1 -- 172.21.15.122:0/1891382071 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f567c004270 con 0x5585c1ce28d0 2024-04-02T13:54:27.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f568f66d740 10 client.29852 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:27.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.017+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=73}) -- 0x5585c191a0f0 con 0x5585c1ce28d0 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f566dffb640 1 -- 172.21.15.122:0/1891382071 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f567c050040 con 0x5585c1ce28d0 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 10 client.29852 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 10 client.29852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 20 client.29852 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1cfb8c0 0x5585c1cfdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 4 client.29852 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c1c53d70 con 0x5585c1cfb8c0 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.018+0000 7f568659c640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1cfb8c0 0x5585c1cfdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.019+0000 7f568659c640 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1cfb8c0 0x5585c1cfdcb0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb88f791ec5b945 server_cookie=edfabaa69800eddf in_seq=0 out_seq=0 2024-04-02T13:54:27.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.019+0000 7f566dffb640 10 client.29852 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:27.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f566dffb640 1 -- 172.21.15.122:0/1891382071 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585c1c53d70 con 0x5585c1cfb8c0 2024-04-02T13:54:27.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f566dffb640 10 client.29852 handle_command_reply: tid=0 2024-04-02T13:54:27.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 client.29852 shutdown 2024-04-02T13:54:27.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 msgr2=0x5585c1ceb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f567c0095f0 tx=0x7f567c034d90 comp rx=0 tx=0).stop 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 shutdown_connections 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1cfb8c0 0x5585c1cfdcb0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1ce9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1ce8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 --2- 172.21.15.122:0/1891382071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1ceb720 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 >> 172.21.15.122:0/1891382071 conn(0x5585c1cdd000 msgr2=0x5585c1cded50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.021+0000 7f566dffb640 0 client.29852 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.022+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 shutdown_connections 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.022+0000 7f568f66d740 1 -- 172.21.15.122:0/1891382071 wait complete. 2024-04-02T13:54:27.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.022+0000 7f568f66d740 20 client.29852 trim_cache size 0 max 16384 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.051+0000 7f568f66d740 1 Processor -- start 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 -- start start 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1cebdd0 0x5585c1ceb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b11c0 con 0x5585c1cebdd0 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f5685d9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46352/0 (socket says 172.21.15.122:46352) 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37506/0 (socket says 172.21.15.122:37506) 2024-04-02T13:54:27.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f5685d9b640 1 -- 172.21.15.122:0/802810143 learned_addr learned my addr 172.21.15.122:0/802810143 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b2dc0 con 0x5585c1ce31f0 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.052+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b1890 con 0x5585c1ce28d0 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1cebdd0 0x5585c1ceb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 -- 172.21.15.122:0/802810143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 msgr2=0x5585c1ceb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 -- 172.21.15.122:0/802810143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1cebdd0 msgr2=0x5585c1ceb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1cebdd0 0x5585c1ceb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 -- 172.21.15.122:0/802810143 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c1ceeaa0 con 0x5585c1ce28d0 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f568659c640 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:27.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f5685d9b640 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f567c02c9e0 tx=0x7f567c04a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3dd24cf87c68e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.053+0000 7f566ffff640 1 -- 172.21.15.122:0/802810143 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039070 con 0x5585c1ce28d0 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.054+0000 7f566ffff640 1 -- 172.21.15.122:0/802810143 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c02ca10 con 0x5585c1ce28d0 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.054+0000 7f566ffff640 1 -- 172.21.15.122:0/802810143 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c062040 con 0x5585c1ce28d0 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 msgr2=0x5585c1ce79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f567c02c9e0 tx=0x7f567c04a000 comp rx=0 tx=0).stop 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 shutdown_connections 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1cebdd0 0x5585c1ceb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce31f0 0x5585c1ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 --2- 172.21.15.122:0/802810143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1ce28d0 0x5585c1ce79f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 >> 172.21.15.122:0/802810143 conn(0x5585c1cdd000 msgr2=0x5585c1cdc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.055+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 shutdown_connections 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.056+0000 7f568f66d740 1 -- 172.21.15.122:0/802810143 wait complete. 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.056+0000 7f568f66d740 1 -- start start 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.056+0000 7f568f66d740 1 Processor -- start 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1d008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1d00de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1cfdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b26f0 con 0x5585c1ce31f0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f56881b1f60 con 0x5585c1ce28d0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f56881af1c0 con 0x5585c1cebdd0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568f66d740 10 client.0 _ll_register_callbacks cb 0x7f567c0344d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1d00de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1d00de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:43974/0 (socket says 172.21.15.122:43974) 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f568659c640 1 -- 172.21.15.122:0/3941513779 learned_addr learned my addr 172.21.15.122:0/3941513779 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1cfdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.057+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1d008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 -- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 msgr2=0x5585c1d008a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1d008a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 -- 172.21.15.122:0/3941513779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 msgr2=0x5585c1d00de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1d00de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 -- 172.21.15.122:0/3941513779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c1cfe580 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f5685d9b640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1cfdd50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f567c042420 tx=0x7f567c04a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfdd3832e9b7cddc server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f566dffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c039070 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c02ca10 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.058+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f567c032a80 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f567c032c80 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 10 client.29593 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5585c1d69ec0 con 0x5585c1cebdd0 2024-04-02T13:54:27.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c02ca10 con 0x5585c1cebdd0 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f567c03f030 con 0x5585c1cebdd0 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 10 client.29593 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 10 client.29593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 20 client.29593 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1d6a2a0 0x5585c1d6c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 4 client.29593 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.059+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585c1c4e0c0 con 0x5585c1d6a2a0 2024-04-02T13:54:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.060+0000 7f568659c640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1d6a2a0 0x5585c1d6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.061+0000 7f568659c640 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1d6a2a0 0x5585c1d6c690 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc46b21319a8504 server_cookie=13b955a10a64796c in_seq=0 out_seq=0 2024-04-02T13:54:27.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.061+0000 7f566dffb640 10 client.29593 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:27.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.061+0000 7f566dffb640 1 -- 172.21.15.122:0/3941513779 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585c1c4e0c0 con 0x5585c1d6a2a0 2024-04-02T13:54:27.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.061+0000 7f566dffb640 10 client.29593 handle_command_reply: tid=0 2024-04-02T13:54:27.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.061+0000 7f568f66d740 1 client.29593 shutdown 2024-04-02T13:54:27.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.063+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 msgr2=0x5585c1cfdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.063+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1cfdd50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f567c042420 tx=0x7f567c04a000 comp rx=0 tx=0).stop 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 shutdown_connections 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5585c1d6a2a0 0x5585c1d6c690 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585c1cebdd0 0x5585c1cfdd50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5585c1ce31f0 0x5585c1d00de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 --2- 172.21.15.122:0/3941513779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5585c1ce28d0 0x5585c1d008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 >> 172.21.15.122:0/3941513779 conn(0x5585c1cdd000 msgr2=0x5585c1cdc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f566dffb640 0 client.29593 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:27.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 shutdown_connections 2024-04-02T13:54:27.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.065+0000 7f568f66d740 1 -- 172.21.15.122:0/3941513779 wait complete. 2024-04-02T13:54:27.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.066+0000 7f568f66d740 20 client.29593 trim_cache size 0 max 16384 2024-04-02T13:54:27.070 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:54:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:54:27.070 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:54:27.070 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.066+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 msgr2=0x7f56700a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.066+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 0x7f56700a6330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f568005d310 tx=0x7f5680067190 comp rx=0 tx=0).stop 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.066+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 msgr2=0x7f568819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.066+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f568000efd0 tx=0x7f568000c6a0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 shutdown_connections 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f56700a3e70 0x7f56700a6330 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568819ee20 0x7f568819f2d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5688108fe0 0x7f568819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 --2- 172.21.15.122:0/1153449514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5688103f10 0x7f568819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 >> 172.21.15.122:0/1153449514 conn(0x7f56880ff770 msgr2=0x7f5688101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.067+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 shutdown_connections 2024-04-02T13:54:27.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.068+0000 7f568ca86640 1 -- 172.21.15.122:0/1153449514 wait complete. 2024-04-02T13:54:27.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:54:27.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:54:27.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 -- 172.21.15.122:0/2265292411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 msgr2=0x7fb070106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 --2- 172.21.15.122:0/2265292411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb070106db0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fb060009930 tx=0x7fb06002bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:27.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 -- 172.21.15.122:0/2265292411 shutdown_connections 2024-04-02T13:54:27.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 --2- 172.21.15.122:0/2265292411 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb070101380 0x7fb07010d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 --2- 172.21.15.122:0/2265292411 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0701009c0 0x7fb070100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 --2- 172.21.15.122:0/2265292411 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb070106db0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 -- 172.21.15.122:0/2265292411 >> 172.21.15.122:0/2265292411 conn(0x7fb070077aa0 msgr2=0x7fb070077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 -- 172.21.15.122:0/2265292411 shutdown_connections 2024-04-02T13:54:27.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.238+0000 7fb074a0d640 1 -- 172.21.15.122:0/2265292411 wait complete. 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 Processor -- start 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 -- start start 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0701009c0 0x7fb07019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb0701a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb07006ac60 con 0x7fb0701069d0 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb07006aae0 con 0x7fb0701009c0 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.239+0000 7fb074a0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb07006ade0 con 0x7fb070101380 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb0701a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46388/0 (socket says 172.21.15.122:46388) 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0701009c0 0x7fb07019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 -- 172.21.15.122:0/1745480445 learned_addr learned my addr 172.21.15.122:0/1745480445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 -- 172.21.15.122:0/1745480445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0701009c0 msgr2=0x7fb07019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0701009c0 0x7fb07019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 -- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 msgr2=0x7fb0701a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb0701a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 -- 172.21.15.122:0/1745480445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0701a6950 con 0x7fb070101380 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06ed9d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb0701a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.240+0000 7fb06dd9b640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb06400e9f0 tx=0x7fb06400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.241+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06400cd80 con 0x7fb070101380 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.241+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb064010910 con 0x7fb070101380 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.241+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064018990 con 0x7fb070101380 2024-04-02T13:54:27.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.241+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0701a6b20 con 0x7fb070101380 2024-04-02T13:54:27.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.241+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0701a6ff0 con 0x7fb070101380 2024-04-02T13:54:27.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.242+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb070068060 con 0x7fb070101380 2024-04-02T13:54:27.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.243+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb064010ac0 con 0x7fb070101380 2024-04-02T13:54:27.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.244+0000 7fb05f7fe640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 0x7fb0580a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.244+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fb0640d1170 con 0x7fb070101380 2024-04-02T13:54:27.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.244+0000 7fb06e59c640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 0x7fb0580a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.245+0000 7fb06e59c640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 0x7fb0580a62a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb060009900 tx=0x7fb06002df90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.249+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb06409cd70 con 0x7fb070101380 2024-04-02T13:54:27.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 180908 27.38 MB/sec execute 227 sec latency 118.714 ms 2024-04-02T13:54:27.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.591+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb07010d2c0 con 0x7fb070101380 2024-04-02T13:54:27.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.593+0000 7fb05f7fe640 1 -- 172.21.15.122:0/1745480445 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+11949 (secure 0 0 0) 0x7fb0640a1070 con 0x7fb070101380 2024-04-02T13:54:27.597 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:54:27.597 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:52:42.709142+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24487,24485,24460,14628,24512]},"id":1}]} 2024-04-02T13:54:27.597 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 73 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 msgr2=0x7fb0580a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 0x7fb0580a62a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb060009900 tx=0x7fb06002df90 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 msgr2=0x7fb07019b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb06400e9f0 tx=0x7fb06400eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 shutdown_connections 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0580a3de0 0x7fb0580a62a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0701069d0 0x7fb0701a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb070101380 0x7fb07019b8d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 --2- 172.21.15.122:0/1745480445 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb0701009c0 0x7fb07019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 >> 172.21.15.122:0/1745480445 conn(0x7fb070077aa0 msgr2=0x7fb070111310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 shutdown_connections 2024-04-02T13:54:27.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.596+0000 7fb074a0d640 1 -- 172.21.15.122:0/1745480445 wait complete. 2024-04-02T13:54:27.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:54:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:27 smithi187 ceph-mon[36064]: pgmap v607: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-04-02T13:54:27.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.759+0000 7fa952cfb640 1 -- 172.21.15.122:0/2498548165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c1058a0 msgr2=0x7fa94c105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.759+0000 7fa952cfb640 1 --2- 172.21.15.122:0/2498548165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c1058a0 0x7fa94c105c80 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa940009900 tx=0x7fa94002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 -- 172.21.15.122:0/2498548165 shutdown_connections 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 --2- 172.21.15.122:0/2498548165 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c101360 0x7fa94c10d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 --2- 172.21.15.122:0/2498548165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 --2- 172.21.15.122:0/2498548165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c1058a0 0x7fa94c105c80 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 -- 172.21.15.122:0/2498548165 >> 172.21.15.122:0/2498548165 conn(0x7fa94c077a90 msgr2=0x7fa94c077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.760+0000 7fa952cfb640 1 -- 172.21.15.122:0/2498548165 shutdown_connections 2024-04-02T13:54:27.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.761+0000 7fa952cfb640 1 -- 172.21.15.122:0/2498548165 wait complete. 2024-04-02T13:54:27.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.761+0000 7fa952cfb640 1 Processor -- start 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.761+0000 7fa952cfb640 1 -- start start 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 0x7fa94c199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c1058a0 0x7fa94c1a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c110de0 con 0x7fa94c101360 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c110c60 con 0x7fa94c1058a0 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa952cfb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c110f60 con 0x7fa94c1009c0 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 0x7fa94c199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 0x7fa94c199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44014/0 (socket says 172.21.15.122:44014) 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa94bfff640 1 -- 172.21.15.122:0/3831138657 learned_addr learned my addr 172.21.15.122:0/3831138657 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa951298640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c1058a0 0x7fa94c1a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 -- 172.21.15.122:0/3831138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c1058a0 msgr2=0x7fa94c1a42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c1058a0 0x7fa94c1a42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 -- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 msgr2=0x7fa94c199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 0x7fa94c199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.762+0000 7fa950a97640 1 -- 172.21.15.122:0/3831138657 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1a47e0 con 0x7fa94c1009c0 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa950a97640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa94002c210 tx=0x7fa940005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa949ffb640 1 -- 172.21.15.122:0/3831138657 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940042020 con 0x7fa94c1009c0 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1a49b0 con 0x7fa94c1009c0 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa949ffb640 1 -- 172.21.15.122:0/3831138657 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa94002ce00 con 0x7fa94c1009c0 2024-04-02T13:54:27.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa949ffb640 1 -- 172.21.15.122:0/3831138657 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940003a50 con 0x7fa94c1009c0 2024-04-02T13:54:27.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:27 smithi188 ceph-mon[36117]: pgmap v607: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-04-02T13:54:27.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.763+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1a4ec0 con 0x7fa94c1009c0 2024-04-02T13:54:27.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.767+0000 7fa9559b7740 1 Processor -- start 2024-04-02T13:54:27.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.767+0000 7fa9559b7740 1 -- start start 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f48a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f48d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa951298640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa949ffb640 1 -- 172.21.15.122:0/3831138657 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa940045020 con 0x7fa94c1009c0 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa951298640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46410/0 (socket says 172.21.15.122:46410) 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa951298640 1 -- 172.21.15.122:0/2843530413 learned_addr learned my addr 172.21.15.122:0/2843530413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa950a97640 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f48a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561c9f3e57b0 con 0x561c9f4820c0 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561c9f384570 con 0x561c9f48aca0 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.768+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561c9f3e1760 con 0x561c9f4817a0 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 -- 172.21.15.122:0/2843530413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 msgr2=0x561c9f48d090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f48d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 -- 172.21.15.122:0/2843530413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 msgr2=0x561c9f48a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f48a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 -- 172.21.15.122:0/2843530413 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9f0f93f0 con 0x561c9f4817a0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa949ffb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 0x7fa93c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa951298640 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa944009f50 tx=0x7fa9440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ab103008bb5e5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa949ffb640 1 -- 172.21.15.122:0/3831138657 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fa9400f1f80 con 0x7fa94c1009c0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa936ffd640 1 -- 172.21.15.122:0/2843530413 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa944004b20 con 0x561c9f4817a0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa936ffd640 1 -- 172.21.15.122:0/2843530413 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa944004cc0 con 0x561c9f4817a0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.769+0000 7fa936ffd640 1 -- 172.21.15.122:0/2843530413 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440054c0 con 0x561c9f4817a0 2024-04-02T13:54:27.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa950a97640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 0x7fa93c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 msgr2=0x561c9f481b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa944009f50 tx=0x7fa9440118d0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa950a97640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 0x7fa93c0a6260 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x561c9f481270 tx=0x7fa940041fb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 shutdown_connections 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f48d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f48a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 --2- 172.21.15.122:0/2843530413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4817a0 0x561c9f481b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 >> 172.21.15.122:0/2843530413 conn(0x561c9f47bed0 msgr2=0x561c9f47c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.770+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 shutdown_connections 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 -- 172.21.15.122:0/2843530413 wait complete. 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 -- start start 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 Processor -- start 2024-04-02T13:54:27.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f487a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f48aca0 0x561c9f4881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.771+0000 7fa9559b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561c9f48da90 con 0x561c9f4820c0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa9559b7740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561c9f3a0e80 con 0x561c9f4817a0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa9559b7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561c9f48dc10 con 0x561c9f48aca0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa9559b7740 10 client.0 _ll_register_callbacks cb 0x561c9f3dcaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f487a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f487a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44032/0 (socket says 172.21.15.122:44032) 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa94bfff640 1 -- 172.21.15.122:0/3477688567 learned_addr learned my addr 172.21.15.122:0/3477688567 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f48aca0 0x561c9f4881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 -- 172.21.15.122:0/3477688567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f48aca0 msgr2=0x561c9f4881a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f48aca0 0x561c9f4881a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 -- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 msgr2=0x561c9f487a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f487a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 -- 172.21.15.122:0/3477688567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9f1c37d0 con 0x561c9f4817a0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa951298640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48a6d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa94400ff30 tx=0x7fa94400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f732fd988c862748 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.772+0000 7fa92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94400fbd0 con 0x561c9f4817a0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa944004cc0 con 0x561c9f4817a0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fa944016040 con 0x561c9f4817a0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa9440053d0 con 0x561c9f4817a0 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa9559b7740 10 client.29873 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:27.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x561c9f4a22f0 con 0x561c9f4817a0 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.773+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa944004cc0 con 0x561c9f4817a0 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fa944022050 con 0x561c9f4817a0 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 10 client.29873 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 10 client.29873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 20 client.29873 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f4a36a0 0x561c9f4a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 4 client.29873 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c9f3f2cf0 con 0x561c9f4a36a0 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f4a36a0 0x561c9f4a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa92bfff640 10 client.29873 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:54:27.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.774+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f4a36a0 0x561c9f4a5a90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5988ee1b736e48e3 server_cookie=60224ce657459756 in_seq=0 out_seq=0 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa92bfff640 1 -- 172.21.15.122:0/3477688567 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c9f3f2cf0 con 0x561c9f4a36a0 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa92bfff640 10 client.29873 handle_command_reply: tid=0 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 client.29873 shutdown 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 msgr2=0x561c9f48a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48a6d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa94400ff30 tx=0x7fa94400ff90 comp rx=0 tx=0).stop 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 shutdown_connections 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f4a36a0 0x561c9f4a5a90 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f48aca0 0x561c9f4881a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4820c0 0x561c9f487a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa92bfff640 0 client.29873 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3477688567 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48a6d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 >> 172.21.15.122:0/3477688567 conn(0x561c9f47bed0 msgr2=0x561c9f47dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 shutdown_connections 2024-04-02T13:54:27.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.777+0000 7fa9559b7740 1 -- 172.21.15.122:0/3477688567 wait complete. 2024-04-02T13:54:27.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.778+0000 7fa9559b7740 20 client.29873 trim_cache size 0 max 16384 2024-04-02T13:54:27.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 Processor -- start 2024-04-02T13:54:27.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 -- start start 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 0x561c9f486a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f4ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa951298640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 0x561c9f486a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa951298640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 0x561c9f486a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44034/0 (socket says 172.21.15.122:44034) 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa951298640 1 -- 172.21.15.122:0/3631620132 learned_addr learned my addr 172.21.15.122:0/3631620132 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f4ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1acf30 con 0x561c9f4817a0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aeb30 con 0x561c9f48aca0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.794+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ad600 con 0x561c9f4820c0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f4ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa951298640 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f4ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 -- 172.21.15.122:0/3631620132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 msgr2=0x561c9f4ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f4ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 -- 172.21.15.122:0/3631620132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 msgr2=0x561c9f486a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 0x561c9f486a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 -- 172.21.15.122:0/3631620132 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9f47cc60 con 0x561c9f4820c0 2024-04-02T13:54:27.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa94bfff640 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f4ab220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x561c9f4a6160 tx=0x7fa938010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f00ae58f76a29257 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.795+0000 7fa9367fc640 1 -- 172.21.15.122:0/3631620132 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938021020 con 0x561c9f4820c0 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9367fc640 1 -- 172.21.15.122:0/3631620132 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa93801d620 con 0x561c9f4820c0 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9367fc640 1 -- 172.21.15.122:0/3631620132 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93800f3c0 con 0x561c9f4820c0 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 msgr2=0x561c9f4ab220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f4ab220 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x561c9f4a6160 tx=0x7fa938010800 comp rx=0 tx=0).stop 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 shutdown_connections 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f48aca0 0x561c9f4ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f4ab220 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 --2- 172.21.15.122:0/3631620132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f4817a0 0x561c9f486a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.796+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 >> 172.21.15.122:0/3631620132 conn(0x561c9f47bed0 msgr2=0x561c9f486330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.797+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 shutdown_connections 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.797+0000 7fa9559b7740 1 -- 172.21.15.122:0/3631620132 wait complete. 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.797+0000 7fa9559b7740 1 -- start start 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.797+0000 7fa9559b7740 1 Processor -- start 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f48c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f48aca0 0x561c9f48c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1ae460 con 0x561c9f48aca0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa951298640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa951298640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37572/0 (socket says 172.21.15.122:37572) 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f48c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa951298640 1 -- 172.21.15.122:0/1375660520 learned_addr learned my addr 172.21.15.122:0/1375660520 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1adcd0 con 0x561c9f4817a0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aaf30 con 0x561c9f4820c0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa951298640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f48aca0 0x561c9f48c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa9559b7740 10 client.0 _ll_register_callbacks cb 0x7fa944009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 -- 172.21.15.122:0/1375660520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 msgr2=0x561c9f48bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 -- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f48aca0 msgr2=0x561c9f48c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f48aca0 0x561c9f48c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa94bfff640 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9f4999f0 con 0x561c9f4820c0 2024-04-02T13:54:27.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.798+0000 7fa951298640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa94bfff640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f48c020 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x561c9f384640 tx=0x7fa93801d420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c1fdf4e707e23e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa938021030 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fa938005040 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93801d620 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fa93801ce90 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa9559b7740 10 client.29617 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x561c9f509d80 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.799+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa93801c3a0 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fa9380047c0 con 0x561c9f4820c0 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 10 client.29617 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 10 client.29617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 20 client.29617 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f50a330 0x561c9f50c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:27.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 4 client.29617 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:27.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa93801d620 con 0x561c9f50a330 2024-04-02T13:54:27.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa951298640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f50a330 0x561c9f50c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa92bfff640 10 client.29617 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.800+0000 7fa951298640 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f50a330 0x561c9f50c750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=992de30fc2773b4b server_cookie=bb969398c710c2d6 in_seq=0 out_seq=0 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa92bfff640 1 -- 172.21.15.122:0/1375660520 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa93801d620 con 0x561c9f50a330 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa92bfff640 10 client.29617 handle_command_reply: tid=0 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 client.29617 shutdown 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 msgr2=0x561c9f48c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f48c020 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x561c9f384640 tx=0x7fa93801d420 comp rx=0 tx=0).stop 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 shutdown_connections 2024-04-02T13:54:27.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x561c9f50a330 0x561c9f50c750 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa92bfff640 0 client.29617 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561c9f48aca0 0x561c9f48c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561c9f4820c0 0x561c9f48c020 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 --2- 172.21.15.122:0/1375660520 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561c9f4817a0 0x561c9f48bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 >> 172.21.15.122:0/1375660520 conn(0x561c9f47bed0 msgr2=0x561c9f47d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 shutdown_connections 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.802+0000 7fa9559b7740 1 -- 172.21.15.122:0/1375660520 wait complete. 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.803+0000 7fa9559b7740 20 client.29617 trim_cache size 0 max 16384 2024-04-02T13:54:27.809 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 msgr2=0x7fa93c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 0x7fa93c0a6260 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x561c9f481270 tx=0x7fa940041fb0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 msgr2=0x7fa94c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c199220 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa94002c210 tx=0x7fa940005e80 comp rx=0 tx=0).stop 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 shutdown_connections 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa93c0a3da0 0x7fa93c0a6260 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa94c1058a0 0x7fa94c1a42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa94c101360 0x7fa94c199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 --2- 172.21.15.122:0/3831138657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa94c1009c0 0x7fa94c199220 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:27.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 >> 172.21.15.122:0/3831138657 conn(0x7fa94c077a90 msgr2=0x7fa94c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:27.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 shutdown_connections 2024-04-02T13:54:27.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.804+0000 7fa952cfb640 1 -- 172.21.15.122:0/3831138657 wait complete. 2024-04-02T13:54:27.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:27 smithi122 ceph-mon[32114]: pgmap v607: 129 pgs: 129 active+clean; 2.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 611 op/s 2024-04-02T13:54:27.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:54:27.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.992+0000 7fb48eef2640 1 -- 172.21.15.122:0/803059507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 msgr2=0x7fb488068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:27.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.992+0000 7fb48eef2640 1 --2- 172.21.15.122:0/803059507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb488068c60 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fb480009be0 tx=0x7fb48002bef0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.995+0000 7fb48eef2640 1 -- 172.21.15.122:0/803059507 shutdown_connections 2024-04-02T13:54:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.995+0000 7fb48eef2640 1 --2- 172.21.15.122:0/803059507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb488068c60 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.995+0000 7fb48eef2640 1 --2- 172.21.15.122:0/803059507 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488100970 0x7fb488100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.995+0000 7fb48eef2640 1 --2- 172.21.15.122:0/803059507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488106980 0x7fb488106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.996+0000 7fb48eef2640 1 -- 172.21.15.122:0/803059507 >> 172.21.15.122:0/803059507 conn(0x7fb488077b00 msgr2=0x7fb488077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.997+0000 7fb48eef2640 1 -- 172.21.15.122:0/803059507 shutdown_connections 2024-04-02T13:54:28.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.998+0000 7fb48eef2640 1 -- 172.21.15.122:0/803059507 wait complete. 2024-04-02T13:54:28.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.999+0000 7fb48eef2640 1 Processor -- start 2024-04-02T13:54:28.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:27.999+0000 7fb48eef2640 1 -- start start 2024-04-02T13:54:28.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 0x7fb48819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 0x7fb48819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb488114190 con 0x7fb488101330 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb488114010 con 0x7fb488106980 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48eef2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb488114310 con 0x7fb488100970 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 0x7fb48819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48d48f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 0x7fb48819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44060/0 (socket says 172.21.15.122:44060) 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb47ffff640 1 -- 172.21.15.122:0/487615021 learned_addr learned my addr 172.21.15.122:0/487615021 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.000+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 0x7fb48819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46452/0 (socket says 172.21.15.122:46452) 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 -- 172.21.15.122:0/487615021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 msgr2=0x7fb48819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 0x7fb48819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 -- 172.21.15.122:0/487615021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 msgr2=0x7fb48819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 0x7fb48819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 -- 172.21.15.122:0/487615021 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48819f470 con 0x7fb488101330 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb48d48f640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 0x7fb48819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.001+0000 7fb47ffff640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb4840092c0 tx=0x7fb484009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.002+0000 7fb47dffb640 1 -- 172.21.15.122:0/487615021 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401f620 con 0x7fb488101330 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.002+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48819f760 con 0x7fb488101330 2024-04-02T13:54:28.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.002+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48810fa50 con 0x7fb488101330 2024-04-02T13:54:28.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.002+0000 7fb47dffb640 1 -- 172.21.15.122:0/487615021 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb48401fca0 con 0x7fb488101330 2024-04-02T13:54:28.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.003+0000 7fb47dffb640 1 -- 172.21.15.122:0/487615021 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48401e380 con 0x7fb488101330 2024-04-02T13:54:28.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.008+0000 7fb491bba740 1 Processor -- start 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 -- start start 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb47dffb640 1 -- 172.21.15.122:0/487615021 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb48401f7c0 con 0x7fb488101330 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9629b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c9635090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5587c958d7b0 con 0x5587c9632ca0 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587c952c570 con 0x5587c96297a0 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb491bba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587c9589760 con 0x5587c962a0c0 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46462/0 (socket says 172.21.15.122:46462) 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb48cc8e640 1 -- 172.21.15.122:0/169638273 learned_addr learned my addr 172.21.15.122:0/169638273 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb47ffff640 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c9635090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb48cc8e640 1 -- 172.21.15.122:0/169638273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 msgr2=0x5587c9629b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.009+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9629b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb48cc8e640 1 -- 172.21.15.122:0/169638273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 msgr2=0x5587c9635090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c9635090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb48cc8e640 1 -- 172.21.15.122:0/169638273 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c92a13f0 con 0x5587c962a0c0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb47dffb640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 0x7fb4700a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb478009d00 tx=0x7fb4780107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3c808a2f441416e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb47dffb640 1 -- 172.21.15.122:0/487615021 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fb4840d8a90 con 0x7fb488101330 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb46effd640 1 -- 172.21.15.122:0/169638273 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47801c480 con 0x5587c962a0c0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.010+0000 7fb46effd640 1 -- 172.21.15.122:0/169638273 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478011920 con 0x5587c962a0c0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.011+0000 7fb46effd640 1 -- 172.21.15.122:0/169638273 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47801aa20 con 0x5587c962a0c0 2024-04-02T13:54:28.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.011+0000 7fb48d48f640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 0x7fb4700a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.012+0000 7fb491bba740 1 -- 172.21.15.122:0/169638273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 msgr2=0x5587c9632760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.012+0000 7fb491bba740 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb478009d00 tx=0x7fb4780107a0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 -- 172.21.15.122:0/169638273 shutdown_connections 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c9635090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 --2- 172.21.15.122:0/169638273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9629b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 -- 172.21.15.122:0/169638273 >> 172.21.15.122:0/169638273 conn(0x5587c9623ed0 msgr2=0x5587c96242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 -- 172.21.15.122:0/169638273 shutdown_connections 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 -- 172.21.15.122:0/169638273 wait complete. 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 -- start start 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.013+0000 7fb491bba740 1 Processor -- start 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9632650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c962f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5587c9635a90 con 0x5587c9632ca0 2024-04-02T13:54:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5587c9548e80 con 0x5587c96297a0 2024-04-02T13:54:28.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587c9635c10 con 0x5587c962a0c0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb491bba740 10 client.0 _ll_register_callbacks cb 0x5587c9584aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb48d48f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb48d48f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44082/0 (socket says 172.21.15.122:44082) 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.014+0000 7fb48d48f640 1 -- 172.21.15.122:0/2466916256 learned_addr learned my addr 172.21.15.122:0/2466916256 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9632650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb47ffff640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c962f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48d48f640 1 -- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 msgr2=0x5587c962f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48d48f640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c962f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48d48f640 1 -- 172.21.15.122:0/2466916256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 msgr2=0x5587c9632650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48d48f640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9632650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.015+0000 7fb48d48f640 1 -- 172.21.15.122:0/2466916256 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c936b7d0 con 0x5587c9632ca0 2024-04-02T13:54:28.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb48d48f640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 0x7fb4700a63f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb480009be0 tx=0x7fb48003e000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb48d48f640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb48002c500 tx=0x7fb48002c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e4f12e65c06a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb467fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480004620 con 0x5587c9632ca0 2024-04-02T13:54:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fb48003a040 con 0x5587c9632ca0 2024-04-02T13:54:28.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.016+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb478011920 con 0x5587c9632ca0 2024-04-02T13:54:28.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.017+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb4800047c0 con 0x5587c9632ca0 2024-04-02T13:54:28.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.017+0000 7fb491bba740 10 client.20937 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.017+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5587c96528d0 con 0x5587c9632ca0 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb478011920 con 0x5587c9632ca0 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb480042480 con 0x5587c9632ca0 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 10 client.20937 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 10 client.20937 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 20 client.20937 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c96542d0 0x5587c9656750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 4 client.20937 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587c959acf0 con 0x5587c96542d0 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.018+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c96542d0 0x5587c9656750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.019+0000 7fb467fff640 10 client.20937 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:54:28.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.020+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c96542d0 0x5587c9656750 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a3acd2b2bfceda2 server_cookie=c6568e229fc41968 in_seq=0 out_seq=0 2024-04-02T13:54:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.023+0000 7fb467fff640 1 -- 172.21.15.122:0/2466916256 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587c959acf0 con 0x5587c96542d0 2024-04-02T13:54:28.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.023+0000 7fb467fff640 10 client.20937 handle_command_reply: tid=0 2024-04-02T13:54:28.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.023+0000 7fb491bba740 1 client.20937 shutdown 2024-04-02T13:54:28.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.027+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 msgr2=0x5587c962ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.027+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb48002c500 tx=0x7fb48002c890 comp rx=0 tx=0).stop 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.027+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 shutdown_connections 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.027+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c96542d0 0x5587c9656750 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c9632ca0 0x5587c962ff80 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c962f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb491bba740 1 --2- 172.21.15.122:0/2466916256 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c9632650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 >> 172.21.15.122:0/2466916256 conn(0x5587c9623ed0 msgr2=0x5587c9625c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb467fff640 0 client.20937 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:54:28.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.028+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 shutdown_connections 2024-04-02T13:54:28.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.029+0000 7fb491bba740 1 -- 172.21.15.122:0/2466916256 wait complete. 2024-04-02T13:54:28.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.029+0000 7fb491bba740 20 client.20937 trim_cache size 0 max 16384 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.053+0000 7fb491bba740 1 Processor -- start 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.053+0000 7fb491bba740 1 -- start start 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c9632ca0 0x5587c965c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb474010050 con 0x5587c96297a0 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb474011c50 con 0x5587c9632ca0 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb491bba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb474010720 con 0x5587c962a0c0 2024-04-02T13:54:28.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46490/0 (socket says 172.21.15.122:46490) 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb47ffff640 1 -- 172.21.15.122:0/2429634070 learned_addr learned my addr 172.21.15.122:0/2429634070 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44098/0 (socket says 172.21.15.122:44098) 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48d48f640 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c9632ca0 0x5587c965c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 -- 172.21.15.122:0/2429634070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 msgr2=0x5587c9632650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 -- 172.21.15.122:0/2429634070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c9632ca0 msgr2=0x5587c965c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c9632ca0 0x5587c965c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.054+0000 7fb48cc8e640 1 -- 172.21.15.122:0/2429634070 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587c9628b20 con 0x5587c96297a0 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.055+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb478010ca0 tx=0x7fb478005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56e06d2bf4eb949 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.055+0000 7fb46e7fc640 1 -- 172.21.15.122:0/2429634070 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478005040 con 0x5587c96297a0 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.055+0000 7fb46e7fc640 1 -- 172.21.15.122:0/2429634070 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb47802a200 con 0x5587c96297a0 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.055+0000 7fb46e7fc640 1 -- 172.21.15.122:0/2429634070 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47801a890 con 0x5587c96297a0 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.056+0000 7fb491bba740 1 -- 172.21.15.122:0/2429634070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 msgr2=0x5587c9632110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb478010ca0 tx=0x7fb478005fd0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 -- 172.21.15.122:0/2429634070 shutdown_connections 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c9632ca0 0x5587c965c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c962a0c0 0x5587c9632650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 --2- 172.21.15.122:0/2429634070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c96297a0 0x5587c9632110 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 -- 172.21.15.122:0/2429634070 >> 172.21.15.122:0/2429634070 conn(0x5587c9623ed0 msgr2=0x5587c962aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 -- 172.21.15.122:0/2429634070 shutdown_connections 2024-04-02T13:54:28.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.057+0000 7fb491bba740 1 -- 172.21.15.122:0/2429634070 wait complete. 2024-04-02T13:54:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.058+0000 7fb491bba740 1 -- start start 2024-04-02T13:54:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.058+0000 7fb491bba740 1 Processor -- start 2024-04-02T13:54:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.059+0000 7fb491bba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c964c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.059+0000 7fb491bba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c962a0c0 0x5587c9634120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.059+0000 7fb491bba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.059+0000 7fb491bba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb474011580 con 0x5587c962a0c0 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.059+0000 7fb491bba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb474010df0 con 0x5587c96297a0 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46496/0 (socket says 172.21.15.122:46496) 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 -- 172.21.15.122:0/3746817763 learned_addr learned my addr 172.21.15.122:0/3746817763 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb47400e050 con 0x5587c9632ca0 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb491bba740 10 client.0 _ll_register_callbacks cb 0x7fb480009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 -- 172.21.15.122:0/3746817763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 msgr2=0x5587c964c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c964c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 -- 172.21.15.122:0/3746817763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c962a0c0 msgr2=0x5587c9634120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c962a0c0 0x5587c9634120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.060+0000 7fb48d48f640 1 -- 172.21.15.122:0/3746817763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587c9633080 con 0x5587c9632ca0 2024-04-02T13:54:28.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.061+0000 7fb48d48f640 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb480042970 tx=0x7fb4800426f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13160aea4bf57c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.061+0000 7fb467fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.061+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480004620 con 0x5587c9632ca0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.061+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fb48004c660 con 0x5587c9632ca0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.062+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb47802a200 con 0x5587c9632ca0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.062+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fb48006c050 con 0x5587c9632ca0 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.062+0000 7fb491bba740 10 client.29632 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.062+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5587c964b0c0 con 0x5587c9632ca0 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb480063070 con 0x5587c9632ca0 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fb48005d070 con 0x5587c9632ca0 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 10 client.29632 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 10 client.29632 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 20 client.29632 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c964b7e0 0x5587c96cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 4 client.29632 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb47802a200 con 0x5587c964b7e0 2024-04-02T13:54:28.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.063+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c964b7e0 0x5587c96cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb467fff640 10 client.29632 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb48cc8e640 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c964b7e0 0x5587c96cd8f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1493160a36cc154b server_cookie=f4ccf18922dab9d7 in_seq=0 out_seq=0 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb467fff640 1 -- 172.21.15.122:0/3746817763 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb47802a200 con 0x5587c964b7e0 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb467fff640 10 client.29632 handle_command_reply: tid=0 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb491bba740 1 client.29632 shutdown 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.064+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 msgr2=0x5587c9634660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb480042970 tx=0x7fb4800426f0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 shutdown_connections 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587c9632ca0 0x5587c9634660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5587c964b7e0 0x5587c96cd8f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5587c962a0c0 0x5587c9634120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 --2- 172.21.15.122:0/3746817763 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5587c96297a0 0x5587c964c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb467fff640 0 client.29632 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:54:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 >> 172.21.15.122:0/3746817763 conn(0x5587c9623ed0 msgr2=0x5587c962f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 shutdown_connections 2024-04-02T13:54:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 1 -- 172.21.15.122:0/3746817763 wait complete. 2024-04-02T13:54:28.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.065+0000 7fb491bba740 20 client.29632 trim_cache size 0 max 16384 2024-04-02T13:54:28.070 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 msgr2=0x7fb4700a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 0x7fb4700a63f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb480009be0 tx=0x7fb48003e000 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 msgr2=0x7fb48819a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fb4840092c0 tx=0x7fb484009780 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 shutdown_connections 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb4700a3f30 0x7fb4700a63f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb488106980 0x7fb48819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb488101330 0x7fb48819a9c0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 --2- 172.21.15.122:0/487615021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb488100970 0x7fb48819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 >> 172.21.15.122:0/487615021 conn(0x7fb488077b00 msgr2=0x7fb488102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 shutdown_connections 2024-04-02T13:54:28.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.066+0000 7fb48eef2640 1 -- 172.21.15.122:0/487615021 wait complete. 2024-04-02T13:54:28.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 -- 172.21.15.122:0/3729921906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 msgr2=0x7fd320100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 --2- 172.21.15.122:0/3729921906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd320100c60 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fd310009900 tx=0x7fd31002bc30 comp rx=0 tx=0).stop 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 -- 172.21.15.122:0/3729921906 shutdown_connections 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 --2- 172.21.15.122:0/3729921906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd32010e430 0x7fd320110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 --2- 172.21.15.122:0/3729921906 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3201011a0 0x7fd32010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 --2- 172.21.15.122:0/3729921906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd320100c60 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.229+0000 7fd324de2640 1 -- 172.21.15.122:0/3729921906 >> 172.21.15.122:0/3729921906 conn(0x7fd3200777b0 msgr2=0x7fd3200feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.230+0000 7fd324de2640 1 -- 172.21.15.122:0/3729921906 shutdown_connections 2024-04-02T13:54:28.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.230+0000 7fd324de2640 1 -- 172.21.15.122:0/3729921906 wait complete. 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.231+0000 7fd324de2640 1 Processor -- start 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.231+0000 7fd324de2640 1 -- start start 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.231+0000 7fd324de2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.231+0000 7fd324de2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3201011a0 0x7fd32019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd324de2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32010e430 0x7fd32019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd324de2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd320114130 con 0x7fd320100880 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd324de2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd320113fb0 con 0x7fd32010e430 2024-04-02T13:54:28.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd324de2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3201142b0 con 0x7fd3201011a0 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44118/0 (socket says 172.21.15.122:44118) 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 -- 172.21.15.122:0/502047091 learned_addr learned my addr 172.21.15.122:0/502047091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32010e430 0x7fd32019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd317fff640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3201011a0 0x7fd32019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 -- 172.21.15.122:0/502047091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3201011a0 msgr2=0x7fd32019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3201011a0 0x7fd32019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 -- 172.21.15.122:0/502047091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32010e430 msgr2=0x7fd32019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32010e430 0x7fd32019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.232+0000 7fd31e59c640 1 -- 172.21.15.122:0/502047091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32019f6a0 con 0x7fd320100880 2024-04-02T13:54:28.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd31e59c640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd310000c00 tx=0x7fd310004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd3177fe640 1 -- 172.21.15.122:0/502047091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310043020 con 0x7fd320100880 2024-04-02T13:54:28.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd3177fe640 1 -- 172.21.15.122:0/502047091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd310004250 con 0x7fd320100880 2024-04-02T13:54:28.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32019f930 con 0x7fd320100880 2024-04-02T13:54:28.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3201a7220 con 0x7fd320100880 2024-04-02T13:54:28.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.233+0000 7fd3177fe640 1 -- 172.21.15.122:0/502047091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310005220 con 0x7fd320100880 2024-04-02T13:54:28.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.235+0000 7fd3177fe640 1 -- 172.21.15.122:0/502047091 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd310046020 con 0x7fd320100880 2024-04-02T13:54:28.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.236+0000 7fd3177fe640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 0x7fd30c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.236+0000 7fd3279ca740 1 Processor -- start 2024-04-02T13:54:28.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.236+0000 7fd3177fe640 1 -- 172.21.15.122:0/502047091 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fd3100f2df0 con 0x7fd320100880 2024-04-02T13:54:28.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd317fff640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 0x7fd30c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 -- start start 2024-04-02T13:54:28.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc02b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e5fb667b0 con 0x563e5fc030c0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e5fb05570 con 0x563e5fc027a0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd3279ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e5fb62760 con 0x563e5fc0bca0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46516/0 (socket says 172.21.15.122:46516) 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc02b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3022521604 learned_addr learned my addr 172.21.15.122:0/3022521604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd317fff640 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3022521604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 msgr2=0x563e5fc02b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc02b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3022521604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 msgr2=0x563e5fc0b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc0b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd317fff640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 0x7fd30c0a6260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd32010f6e0 tx=0x7fd308009cb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.237+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3022521604 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5f87a3f0 con 0x563e5fc0bca0 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.238+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd318009f50 tx=0x7fd318011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c498facf9c9408a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.238+0000 7fd314ff9640 1 -- 172.21.15.122:0/3022521604 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318004d20 con 0x563e5fc0bca0 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.238+0000 7fd314ff9640 1 -- 172.21.15.122:0/3022521604 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd318004050 con 0x563e5fc0bca0 2024-04-02T13:54:28.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.238+0000 7fd314ff9640 1 -- 172.21.15.122:0/3022521604 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318005280 con 0x563e5fc0bca0 2024-04-02T13:54:28.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 -- 172.21.15.122:0/3022521604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 msgr2=0x563e5fc0e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd318009f50 tx=0x7fd318011810 comp rx=0 tx=0).stop 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 -- 172.21.15.122:0/3022521604 shutdown_connections 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc0e090 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc0b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3022521604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc02b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 -- 172.21.15.122:0/3022521604 >> 172.21.15.122:0/3022521604 conn(0x563e5fbfced0 msgr2=0x563e5fbfd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 -- 172.21.15.122:0/3022521604 shutdown_connections 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.239+0000 7fd3279ca740 1 -- 172.21.15.122:0/3022521604 wait complete. 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.240+0000 7fd3279ca740 1 -- start start 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.240+0000 7fd3279ca740 1 Processor -- start 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.240+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc0b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.240+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 0x563e5fc08980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.240+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd317fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 0x563e5fc08980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd317fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 0x563e5fc08980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:46526/0 (socket says 172.21.15.122:46526) 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44140/0 (socket says 172.21.15.122:44140) 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc0b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31e59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc0b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37680/0 (socket says 172.21.15.122:37680) 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd3279ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e5fc172a0 con 0x563e5fc0bca0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd3279ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e5fb21e80 con 0x563e5fc027a0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd317fff640 1 -- 172.21.15.122:0/2501931551 learned_addr learned my addr 172.21.15.122:0/2501931551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e5fc17420 con 0x563e5fc030c0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd3279ca740 10 client.0 _ll_register_callbacks cb 0x563e5fb5daa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 -- 172.21.15.122:0/2501931551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 msgr2=0x563e5fc08980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 0x563e5fc08980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 -- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 msgr2=0x563e5fc0b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc0b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd31ed9d640 1 -- 172.21.15.122:0/2501931551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5f9447d0 con 0x563e5fc0bca0 2024-04-02T13:54:28.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.241+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd318004050 con 0x563e5fc0bca0 2024-04-02T13:54:28.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd318012fd0 tx=0x7fd318004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6723e91330a7ca88 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd305ffb640 10 client.20952 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd305ffb640 1 -- 172.21.15.122:0/2501931551 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318029020 con 0x563e5fc0bca0 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd305ffb640 1 -- 172.21.15.122:0/2501931551 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fd318019430 con 0x563e5fc0bca0 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd305ffb640 1 -- 172.21.15.122:0/2501931551 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd318019670 con 0x563e5fc0bca0 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd3279ca740 10 client.20952 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.242+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=73}) -- 0x563e5fc231c0 con 0x563e5fc0bca0 2024-04-02T13:54:28.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd305ffb640 1 -- 172.21.15.122:0/2501931551 <== mon.0 v2:172.21.15.122:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fd318019810 con 0x563e5fc0bca0 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 10 client.20952 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 10 client.20952 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 20 client.20952 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc24570 0x563e5fc26960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 4 client.20952 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5fb73cf0 con 0x563e5fc24570 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.243+0000 7fd31e59c640 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc24570 0x563e5fc26960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.244+0000 7fd31e59c640 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc24570 0x563e5fc26960 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85daf655bd5b4a0f server_cookie=1405d81a47465d5a in_seq=0 out_seq=0 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.244+0000 7fd305ffb640 10 client.20952 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd305ffb640 1 -- 172.21.15.122:0/2501931551 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e5fb73cf0 con 0x563e5fc24570 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd305ffb640 10 client.20952 handle_command_reply: tid=0 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 client.20952 shutdown 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 msgr2=0x563e5fc08ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fd318012fd0 tx=0x7fd318004430 comp rx=0 tx=0).stop 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 shutdown_connections 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc0bca0 0x563e5fc08ee0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc030c0 0x563e5fc08980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.245+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc24570 0x563e5fc26960 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2501931551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc027a0 0x563e5fc0b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 >> 172.21.15.122:0/2501931551 conn(0x563e5fbfced0 msgr2=0x563e5fbfec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd305ffb640 0 client.20952 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 shutdown_connections 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd3279ca740 1 -- 172.21.15.122:0/2501931551 wait complete. 2024-04-02T13:54:28.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.246+0000 7fd3279ca740 20 client.20952 trim_cache size 0 max 16384 2024-04-02T13:54:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.272+0000 7fd3279ca740 1 Processor -- start 2024-04-02T13:54:28.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.272+0000 7fd3279ca740 1 -- start start 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc027a0 0x563e5fbf81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 0x563e5fc201f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd30000fff0 con 0x563e5fc027a0 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd300011bf0 con 0x563e5fc030c0 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd3279ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3000106c0 con 0x563e5fc0bca0 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.273+0000 7fd317fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 0x563e5fc201f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd317fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 0x563e5fc201f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37692/0 (socket says 172.21.15.122:37692) 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd317fff640 1 -- 172.21.15.122:0/3036621232 learned_addr learned my addr 172.21.15.122:0/3036621232 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3036621232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 msgr2=0x563e5fc201f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 0x563e5fc201f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3036621232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc027a0 msgr2=0x563e5fbf81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc027a0 0x563e5fbf81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 -- 172.21.15.122:0/3036621232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5fc02ef0 con 0x563e5fc0bca0 2024-04-02T13:54:28.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.274+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc07b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x563e5fc01910 tx=0x7fd318023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461efda1a0c043b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.275+0000 7fd307fff640 1 -- 172.21.15.122:0/3036621232 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318029040 con 0x563e5fc0bca0 2024-04-02T13:54:28.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.275+0000 7fd307fff640 1 -- 172.21.15.122:0/3036621232 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd31803c070 con 0x563e5fc0bca0 2024-04-02T13:54:28.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.275+0000 7fd307fff640 1 -- 172.21.15.122:0/3036621232 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd318037bb0 con 0x563e5fc0bca0 2024-04-02T13:54:28.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.276+0000 7fd3279ca740 1 -- 172.21.15.122:0/3036621232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 msgr2=0x563e5fc07b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.276+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc07b40 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x563e5fc01910 tx=0x7fd318023000 comp rx=0 tx=0).stop 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 -- 172.21.15.122:0/3036621232 shutdown_connections 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc0bca0 0x563e5fc07b40 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc030c0 0x563e5fc201f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 --2- 172.21.15.122:0/3036621232 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc027a0 0x563e5fbf81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 -- 172.21.15.122:0/3036621232 >> 172.21.15.122:0/3036621232 conn(0x563e5fbfced0 msgr2=0x563e5fbfb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 -- 172.21.15.122:0/3036621232 shutdown_connections 2024-04-02T13:54:28.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.277+0000 7fd3279ca740 1 -- 172.21.15.122:0/3036621232 wait complete. 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.278+0000 7fd3279ca740 1 -- start start 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.278+0000 7fd3279ca740 1 Processor -- start 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.278+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc027a0 0x563e5fc1c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.278+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.278+0000 7fd3279ca740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc0bca0 0x563e5fc061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd3279ca740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd300011520 con 0x563e5fc030c0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd3279ca740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd300010d90 con 0x563e5fc0bca0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd3279ca740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd30000dff0 con 0x563e5fc027a0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd3279ca740 10 client.0 _ll_register_callbacks cb 0x7fd318009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd317fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd317fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:44154/0 (socket says 172.21.15.122:44154) 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd317fff640 1 -- 172.21.15.122:0/2411243561 learned_addr learned my addr 172.21.15.122:0/2411243561 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd31ed9d640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc0bca0 0x563e5fc061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.279+0000 7fd31e59c640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc027a0 0x563e5fc1c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 -- 172.21.15.122:0/2411243561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc027a0 msgr2=0x563e5fc1c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc027a0 0x563e5fc1c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 -- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc0bca0 msgr2=0x563e5fc061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc0bca0 0x563e5fc061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 -- 172.21.15.122:0/2411243561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5fc067e0 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd317fff640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fd308016950 tx=0x7fd3080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad873e405e08fb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.280+0000 7fd305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3080179a0 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd31803c070 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fd308017b40 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fd308017d40 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd3279ca740 10 client.20955 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.281+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x563e5fc0cfa0 con 0x563e5fc030c0 2024-04-02T13:54:28.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.282+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd30801b040 con 0x563e5fc030c0 2024-04-02T13:54:28.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fd308020040 con 0x563e5fc030c0 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 10 client.20955 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 10 client.20955 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 20 client.20955 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc0d730 0x563e5fc8ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 4 client.20955 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd31803c070 con 0x563e5fc0d730 2024-04-02T13:54:28.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.283+0000 7fd31e59c640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc0d730 0x563e5fc8ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd31e59c640 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc0d730 0x563e5fc8ca60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=683d507601e75747 server_cookie=1094705b40679f9c in_seq=0 out_seq=0 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd305ffb640 10 client.20955 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd305ffb640 1 -- 172.21.15.122:0/2411243561 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd31803c070 con 0x563e5fc0d730 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd305ffb640 10 client.20955 handle_command_reply: tid=0 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd3279ca740 1 client.20955 shutdown 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 msgr2=0x563e5fc05c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.284+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fd308016950 tx=0x7fd3080046c0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 shutdown_connections 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x563e5fc0d730 0x563e5fc8ca60 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e5fc0bca0 0x563e5fc061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e5fc030c0 0x563e5fc05c60 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 --2- 172.21.15.122:0/2411243561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e5fc027a0 0x563e5fc1c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 >> 172.21.15.122:0/2411243561 conn(0x563e5fbfced0 msgr2=0x563e5fbfbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd305ffb640 0 client.20955 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:54:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 shutdown_connections 2024-04-02T13:54:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 1 -- 172.21.15.122:0/2411243561 wait complete. 2024-04-02T13:54:28.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.285+0000 7fd3279ca740 20 client.20955 trim_cache size 0 max 16384 2024-04-02T13:54:28.294 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 msgr2=0x7fd30c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 0x7fd30c0a6260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd32010f6e0 tx=0x7fd308009cb0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 msgr2=0x7fd32019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fd310000c00 tx=0x7fd310004a30 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 shutdown_connections 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd30c0a3da0 0x7fd30c0a6260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd32010e430 0x7fd32019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3201011a0 0x7fd32019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 --2- 172.21.15.122:0/502047091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd320100880 0x7fd32019a6b0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 >> 172.21.15.122:0/502047091 conn(0x7fd3200777b0 msgr2=0x7fd3200ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 shutdown_connections 2024-04-02T13:54:28.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.286+0000 7fd324de2640 1 -- 172.21.15.122:0/502047091 wait complete. 2024-04-02T13:54:28.307 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:54:28.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 181515 27.35 MB/sec execute 228 sec latency 103.666 ms 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 -- 172.21.15.122:0/871904549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863810e350 msgr2=0x7f863810e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 --2- 172.21.15.122:0/871904549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863810e350 0x7f863810e7d0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f862c009900 tx=0x7f862c02bc60 comp rx=0 tx=0).stop 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 -- 172.21.15.122:0/871904549 shutdown_connections 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 --2- 172.21.15.122:0/871904549 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f863811b2e0 0x7f863811d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 --2- 172.21.15.122:0/871904549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863810e350 0x7f863810e7d0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 --2- 172.21.15.122:0/871904549 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8638110320 0x7f863810de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.468+0000 7f863ea42640 1 -- 172.21.15.122:0/871904549 >> 172.21.15.122:0/871904549 conn(0x7f863806cbe0 msgr2=0x7f863806cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.469+0000 7f863ea42640 1 -- 172.21.15.122:0/871904549 shutdown_connections 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.469+0000 7f863ea42640 1 -- 172.21.15.122:0/871904549 wait complete. 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.470+0000 7f863ea42640 1 Processor -- start 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.470+0000 7f863ea42640 1 -- start start 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f863810e350 0x7f86381186e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863811b2e0 0x7f86381117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8638120c90 con 0x7f863811b2e0 2024-04-02T13:54:28.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8638120b10 con 0x7f863810e350 2024-04-02T13:54:28.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f863ea42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8638120e10 con 0x7f8638110320 2024-04-02T13:54:28.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f86377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f86377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53212/0 (socket says 172.21.15.122:53212) 2024-04-02T13:54:28.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.471+0000 7f86377fe640 1 -- 172.21.15.122:0/2869854859 learned_addr learned my addr 172.21.15.122:0/2869854859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.472+0000 7f86377fe640 1 -- 172.21.15.122:0/2869854859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f863810e350 msgr2=0x7f86381186e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.472+0000 7f86377fe640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f863810e350 0x7f86381186e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.472+0000 7f86377fe640 1 -- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863811b2e0 msgr2=0x7f86381117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.472+0000 7f86377fe640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863811b2e0 0x7f86381117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.472+0000 7f86377fe640 1 -- 172.21.15.122:0/2869854859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86381120a0 con 0x7f8638110320 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f86377fe640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f862c005ab0 tx=0x7f862c02c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f86357fa640 1 -- 172.21.15.122:0/2869854859 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c041070 con 0x7f8638110320 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f86357fa640 1 -- 172.21.15.122:0/2869854859 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c0054e0 con 0x7f8638110320 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8638112330 con 0x7f8638110320 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f86357fa640 1 -- 172.21.15.122:0/2869854859 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c034440 con 0x7f8638110320 2024-04-02T13:54:28.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.473+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86381163e0 con 0x7f8638110320 2024-04-02T13:54:28.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.476+0000 7f86357fa640 1 -- 172.21.15.122:0/2869854859 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f862c044070 con 0x7f8638110320 2024-04-02T13:54:28.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.476+0000 7f8641620740 1 Processor -- start 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 -- start start 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 0x560e575b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 0x560e575b8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e575107b0 con 0x560e575ad0c0 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560e574af570 con 0x560e575ac7a0 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8641620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560e5750c760 con 0x560e575b5ca0 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8637fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 0x560e575b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8637fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 0x560e575b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52580/0 (socket says 172.21.15.122:52580) 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f8637fff640 1 -- 172.21.15.122:0/1240295441 learned_addr learned my addr 172.21.15.122:0/1240295441 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f86377fe640 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 0x560e575b8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.477+0000 7f863cfdf640 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 -- 172.21.15.122:0/1240295441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 msgr2=0x560e575acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575acb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 -- 172.21.15.122:0/1240295441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 msgr2=0x560e575b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 0x560e575b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 -- 172.21.15.122:0/1240295441 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e572243f0 con 0x560e575b5ca0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86357fa640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 0x7f86200a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86357fa640 1 -- 172.21.15.122:0/2869854859 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f862c0f2d70 con 0x7f8638110320 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f86377fe640 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 0x560e575b8090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f862c0f0180 tx=0x7f862c0bac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bd1e36db20ecf0d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 0x7f86200a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.478+0000 7f861e7fc640 1 -- 172.21.15.122:0/1240295441 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c04b070 con 0x560e575b5ca0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.479+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 0x7f86200a6320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8630005c40 tx=0x7f8630009440 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.479+0000 7f861e7fc640 1 -- 172.21.15.122:0/1240295441 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c0bf0d0 con 0x560e575b5ca0 2024-04-02T13:54:28.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.479+0000 7f861e7fc640 1 -- 172.21.15.122:0/1240295441 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c0be7a0 con 0x560e575b5ca0 2024-04-02T13:54:28.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.480+0000 7f8641620740 1 -- 172.21.15.122:0/1240295441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 msgr2=0x560e575b8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.480+0000 7f8641620740 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 0x560e575b8090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f862c0f0180 tx=0x7f862c0bac70 comp rx=0 tx=0).stop 2024-04-02T13:54:28.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.481+0000 7f8641620740 1 -- 172.21.15.122:0/1240295441 shutdown_connections 2024-04-02T13:54:28.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.481+0000 7f8641620740 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575b5ca0 0x560e575b8090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.481+0000 7f8641620740 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575ad0c0 0x560e575b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.481+0000 7f8641620740 1 --2- 172.21.15.122:0/1240295441 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575acb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.481+0000 7f8641620740 1 -- 172.21.15.122:0/1240295441 >> 172.21.15.122:0/1240295441 conn(0x560e575a6ed0 msgr2=0x560e575a72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.482+0000 7f8641620740 1 -- 172.21.15.122:0/1240295441 shutdown_connections 2024-04-02T13:54:28.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.483+0000 7f8641620740 1 -- 172.21.15.122:0/1240295441 wait complete. 2024-04-02T13:54:28.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.483+0000 7f8641620740 1 -- start start 2024-04-02T13:54:28.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 Processor -- start 2024-04-02T13:54:28.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e575b8850 con 0x560e575b5ca0 2024-04-02T13:54:28.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560e574cbe80 con 0x560e575ac7a0 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560e575b89d0 con 0x560e575ad0c0 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f8641620740 10 client.0 _ll_register_callbacks cb 0x560e57507aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.484+0000 7f863cfdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52582/0 (socket says 172.21.15.122:52582) 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 -- 172.21.15.122:0/467586224 learned_addr learned my addr 172.21.15.122:0/467586224 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f86377fe640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 -- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 msgr2=0x560e575b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 -- 172.21.15.122:0/467586224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 msgr2=0x560e575b56e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575b56e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 -- 172.21.15.122:0/467586224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e572ee7d0 con 0x560e575b5ca0 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f863cfdf640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f86300126d0 tx=0x7f8630005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a388971caad1f8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.485+0000 7f86177fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:28.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.486+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630023070 con 0x560e575b5ca0 2024-04-02T13:54:28.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.486+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862c0bf0d0 con 0x560e575b5ca0 2024-04-02T13:54:28.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.486+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f8630011b60 con 0x560e575b5ca0 2024-04-02T13:54:28.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.487+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f863002f050 con 0x560e575b5ca0 2024-04-02T13:54:28.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.487+0000 7f8641620740 10 client.20964 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.487+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x560e575c48a0 con 0x560e575b5ca0 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.489+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c0bf0d0 con 0x560e575b5ca0 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.489+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f863000f040 con 0x560e575b5ca0 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 10 client.20964 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 10 client.20964 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 20 client.20964 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e575c5d50 0x560e575c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 4 client.20964 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e5751dcf0 con 0x560e575c5d50 2024-04-02T13:54:28.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.490+0000 7f8637fff640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e575c5d50 0x560e575c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.493+0000 7f8637fff640 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e575c5d50 0x560e575c81d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff1c2deea9aa6be7 server_cookie=beace599177abba6 in_seq=0 out_seq=0 2024-04-02T13:54:28.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.493+0000 7f86177fe640 10 client.20964 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:54:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.494+0000 7f86177fe640 1 -- 172.21.15.122:0/467586224 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e5751dcf0 con 0x560e575c5d50 2024-04-02T13:54:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.494+0000 7f86177fe640 10 client.20964 handle_command_reply: tid=0 2024-04-02T13:54:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.494+0000 7f8641620740 1 client.20964 shutdown 2024-04-02T13:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 msgr2=0x560e575b31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f86300126d0 tx=0x7f8630005bd0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 shutdown_connections 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e575c5d50 0x560e575c81d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b31b0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 --2- 172.21.15.122:0/467586224 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 >> 172.21.15.122:0/467586224 conn(0x560e575a6ed0 msgr2=0x560e575a8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.496+0000 7f86177fe640 0 client.20964 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:54:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.497+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 shutdown_connections 2024-04-02T13:54:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.497+0000 7f8641620740 1 -- 172.21.15.122:0/467586224 wait complete. 2024-04-02T13:54:28.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.497+0000 7f8641620740 20 client.20964 trim_cache size 0 max 16384 2024-04-02T13:54:28.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.523+0000 7f8641620740 1 Processor -- start 2024-04-02T13:54:28.762 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:28 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1745480445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.523+0000 7f8641620740 1 -- start start 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c65c0 con 0x560e575b5ca0 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c81c0 con 0x560e575ac7a0 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f8641620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c6c90 con 0x560e575ad0c0 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f86377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f86377fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53248/0 (socket says 172.21.15.122:53248) 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.524+0000 7f86377fe640 1 -- 172.21.15.122:0/4044626023 learned_addr learned my addr 172.21.15.122:0/4044626023 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 -- 172.21.15.122:0/4044626023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 msgr2=0x560e575c1680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:54:28.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575c1680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 -- 172.21.15.122:0/4044626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 msgr2=0x560e575cdfe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575cdfe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 -- 172.21.15.122:0/4044626023 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e575acef0 con 0x560e575ad0c0 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.526+0000 7f86377fe640 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f862c0be6d0 tx=0x7f862c0f62d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7395f0ec23a22706 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.527+0000 7f861dffb640 1 -- 172.21.15.122:0/4044626023 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c04b040 con 0x560e575ad0c0 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.527+0000 7f861dffb640 1 -- 172.21.15.122:0/4044626023 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c0f5030 con 0x560e575ad0c0 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.527+0000 7f861dffb640 1 -- 172.21.15.122:0/4044626023 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c0be330 con 0x560e575ad0c0 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 -- 172.21.15.122:0/4044626023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 msgr2=0x560e575cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f862c0be6d0 tx=0x7f862c0f62d0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 -- 172.21.15.122:0/4044626023 shutdown_connections 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575ad0c0 0x560e575cdaa0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.528+0000 7f8641620740 1 --2- 172.21.15.122:0/4044626023 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ac7a0 0x560e575c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.529+0000 7f8641620740 1 -- 172.21.15.122:0/4044626023 >> 172.21.15.122:0/4044626023 conn(0x560e575a6ed0 msgr2=0x560e575a72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.529+0000 7f8641620740 1 -- 172.21.15.122:0/4044626023 shutdown_connections 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.531+0000 7f8641620740 1 -- 172.21.15.122:0/4044626023 wait complete. 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.531+0000 7f8641620740 1 -- start start 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.531+0000 7f8641620740 1 Processor -- start 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 0x560e575b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 0x560e575bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c7af0 con 0x560e575b5ca0 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c7360 con 0x560e575ad0c0 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86381c45c0 con 0x560e575bcab0 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8641620740 10 client.0 _ll_register_callbacks cb 0x7f8630016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8637fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 0x560e575b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8637fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 0x560e575b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43512/0 (socket says 172.21.15.122:43512) 2024-04-02T13:54:28.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f8637fff640 1 -- 172.21.15.122:0/2052290699 learned_addr learned my addr 172.21.15.122:0/2052290699 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 0x560e575bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f86377fe640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f863cfdf640 1 -- 172.21.15.122:0/2052290699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 msgr2=0x560e575b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 0x560e575b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f863cfdf640 1 -- 172.21.15.122:0/2052290699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 msgr2=0x560e575b6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.532+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f863cfdf640 1 -- 172.21.15.122:0/2052290699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e575bd470 con 0x560e575bcab0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f86377fe640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f863cfdf640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 0x560e575bcf30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86300117b0 tx=0x7f8630012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff9615c3ae99139 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f86177fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630037020 con 0x560e575bcab0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f8630027a70 con 0x560e575bcab0 2024-04-02T13:54:28.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862c0f5030 con 0x560e575bcab0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.533+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f8630042050 con 0x560e575bcab0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 10 client.29668 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x560e5762c310 con 0x560e575bcab0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c0f5030 con 0x560e575bcab0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f863003e4c0 con 0x560e575bcab0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 10 client.29668 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 10 client.29668 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 20 client.29668 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e5762cb70 0x560e5762ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 4 client.29668 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.534+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e57518040 con 0x560e5762cb70 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.535+0000 7f8637fff640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e5762cb70 0x560e5762ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.536+0000 7f86177fe640 10 client.29668 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.536+0000 7f8637fff640 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e5762cb70 0x560e5762ef90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47791c4aa27d9cf2 server_cookie=4e7e3d61c9e83b71 in_seq=0 out_seq=0 2024-04-02T13:54:28.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.537+0000 7f86177fe640 1 -- 172.21.15.122:0/2052290699 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e57518040 con 0x560e5762cb70 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.537+0000 7f86177fe640 10 client.29668 handle_command_reply: tid=0 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.537+0000 7f8641620740 1 client.29668 shutdown 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 msgr2=0x560e575bcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 0x560e575bcf30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f86300117b0 tx=0x7f8630012f90 comp rx=0 tx=0).stop 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 shutdown_connections 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x560e5762cb70 0x560e5762ef90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e575bcab0 0x560e575bcf30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f86177fe640 0 client.29668 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e575b5ca0 0x560e575b6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 --2- 172.21.15.122:0/2052290699 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e575ad0c0 0x560e575b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 >> 172.21.15.122:0/2052290699 conn(0x560e575a6ed0 msgr2=0x560e575a8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.538+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 shutdown_connections 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f8641620740 1 -- 172.21.15.122:0/2052290699 wait complete. 2024-04-02T13:54:28.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f8641620740 20 client.29668 trim_cache size 0 max 16384 2024-04-02T13:54:28.773 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:54:28.774 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:28 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1745480445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 msgr2=0x7f86200a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 0x7f86200a6320 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8630005c40 tx=0x7f8630009440 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 msgr2=0x7f8638118c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f862c005ab0 tx=0x7f862c02c8b0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 shutdown_connections 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f86200a3e60 0x7f86200a6320 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f863811b2e0 0x7f86381117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8638110320 0x7f8638118c20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.539+0000 7f863ea42640 1 --2- 172.21.15.122:0/2869854859 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f863810e350 0x7f86381186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.540+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 >> 172.21.15.122:0/2869854859 conn(0x7f863806cbe0 msgr2=0x7f863811c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.540+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 shutdown_connections 2024-04-02T13:54:28.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.540+0000 7f863ea42640 1 -- 172.21.15.122:0/2869854859 wait complete. 2024-04-02T13:54:28.776 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:54:28.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:28 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1745480445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:54:28.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 -- 172.21.15.122:0/1167690061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 msgr2=0x7fe17410e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 --2- 172.21.15.122:0/1167690061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe17410e9a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009ca0 tx=0x7fe16c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 -- 172.21.15.122:0/1167690061 shutdown_connections 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 --2- 172.21.15.122:0/1167690061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe17410e9a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 --2- 172.21.15.122:0/1167690061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe174100bc0 0x7fe1741066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.973+0000 7fe179111640 1 --2- 172.21.15.122:0/1167690061 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe1741005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.974+0000 7fe179111640 1 -- 172.21.15.122:0/1167690061 >> 172.21.15.122:0/1167690061 conn(0x7fe174076f20 msgr2=0x7fe1740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.974+0000 7fe179111640 1 -- 172.21.15.122:0/1167690061 shutdown_connections 2024-04-02T13:54:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.974+0000 7fe179111640 1 -- 172.21.15.122:0/1167690061 wait complete. 2024-04-02T13:54:28.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.975+0000 7fe179111640 1 Processor -- start 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.975+0000 7fe179111640 1 -- start start 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.975+0000 7fe179111640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.975+0000 7fe179111640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe179111640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe17419cc40 0x7fe17419d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43520/0 (socket says 172.21.15.122:43520) 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe17259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe17259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52616/0 (socket says 172.21.15.122:52616) 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe172d9d640 1 -- 172.21.15.122:0/2121319558 learned_addr learned my addr 172.21.15.122:0/2121319558 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe174111e80 con 0x7fe174106dd0 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe17359e640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe17419cc40 0x7fe17419d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe174111d00 con 0x7fe174100210 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.976+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe174112000 con 0x7fe17419cc40 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 -- 172.21.15.122:0/2121319558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe17419cc40 msgr2=0x7fe17419d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe17419cc40 0x7fe17419d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 -- 172.21.15.122:0/2121319558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 msgr2=0x7fe174198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17419d730 con 0x7fe174106dd0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe172d9d640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.977+0000 7fe17259c640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9c0 tx=0x7fe16800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.978+0000 7fe15bfff640 1 -- 172.21.15.122:0/2121319558 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16800ccc0 con 0x7fe174106dd0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.978+0000 7fe15bfff640 1 -- 172.21.15.122:0/2121319558 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe168005020 con 0x7fe174106dd0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.978+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17419da20 con 0x7fe174106dd0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.978+0000 7fe15bfff640 1 -- 172.21.15.122:0/2121319558 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168004a00 con 0x7fe174106dd0 2024-04-02T13:54:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.978+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741a51c0 con 0x7fe174106dd0 2024-04-02T13:54:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.981+0000 7fe15bfff640 1 -- 172.21.15.122:0/2121319558 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fe1680051d0 con 0x7fe174106dd0 2024-04-02T13:54:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.982+0000 7fe17bdcc740 1 Processor -- start 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.982+0000 7fe15bfff640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 0x7fe15c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.982+0000 7fe15bfff640 1 -- 172.21.15.122:0/2121319558 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fe1680d0aa0 con 0x7fe174106dd0 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 -- start start 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17359e640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 0x7fe15c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829712b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297154a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558296697b0 con 0x555829709bb0 2024-04-02T13:54:28.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555829608570 con 0x55582970a4d0 2024-04-02T13:54:28.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555829665760 con 0x5558297130b0 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297154a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52626/0 (socket says 172.21.15.122:52626) 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe17359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297154a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53276/0 (socket says 172.21.15.122:53276) 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.983+0000 7fe172d9d640 1 -- 172.21.15.122:0/1289097047 learned_addr learned my addr 172.21.15.122:0/1289097047 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe17259c640 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829712b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 -- 172.21.15.122:0/1289097047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 msgr2=0x5558297154a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297154a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 -- 172.21.15.122:0/1289097047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 msgr2=0x555829712b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829712b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 -- 172.21.15.122:0/1289097047 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582937d3f0 con 0x555829709bb0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe17259c640 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829712b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe17359e640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 0x7fe15c0a6260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005e30 tx=0x7fe16c0099d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.984+0000 7fe172d9d640 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe164011650 tx=0x7fe164009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2356945b04f21ada server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.985+0000 7fe1597fa640 1 -- 172.21.15.122:0/1289097047 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16401e070 con 0x555829709bb0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.985+0000 7fe1597fa640 1 -- 172.21.15.122:0/1289097047 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16400fba0 con 0x555829709bb0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.985+0000 7fe1597fa640 1 -- 172.21.15.122:0/1289097047 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164019d50 con 0x555829709bb0 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- 172.21.15.122:0/1289097047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 msgr2=0x555829709f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe164011650 tx=0x7fe164009f90 comp rx=0 tx=0).stop 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- 172.21.15.122:0/1289097047 shutdown_connections 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297154a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829712b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/1289097047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829709f90 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- 172.21.15.122:0/1289097047 >> 172.21.15.122:0/1289097047 conn(0x5558297042e0 msgr2=0x5558297046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:28.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- 172.21.15.122:0/1289097047 shutdown_connections 2024-04-02T13:54:28.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- 172.21.15.122:0/1289097047 wait complete. 2024-04-02T13:54:28.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.986+0000 7fe17bdcc740 1 -- start start 2024-04-02T13:54:28.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 Processor -- start 2024-04-02T13:54:28.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55582970a4d0 0x55582970c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558297130b0 0x55582970cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5558296ffb00 con 0x555829709bb0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555829624e80 con 0x5558297130b0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5558296ffc80 con 0x55582970a4d0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52632/0 (socket says 172.21.15.122:52632) 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.987+0000 7fe172d9d640 1 -- 172.21.15.122:0/3685579523 learned_addr learned my addr 172.21.15.122:0/3685579523 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe17bdcc740 10 client.0 _ll_register_callbacks cb 0x555829660aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe17259c640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55582970a4d0 0x55582970c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe17359e640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558297130b0 0x55582970cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe172d9d640 1 -- 172.21.15.122:0/3685579523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55582970a4d0 msgr2=0x55582970c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe172d9d640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55582970a4d0 0x55582970c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe172d9d640 1 -- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558297130b0 msgr2=0x55582970cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe172d9d640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558297130b0 0x55582970cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.988+0000 7fe172d9d640 1 -- 172.21.15.122:0/3685579523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558294477d0 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.989+0000 7fe172d9d640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe164019850 tx=0x7fe164009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d167d6972ad6b3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.989+0000 7fe156ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.989+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16401e040 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.989+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fe16400fd50 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.989+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16400fba0 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe164019a50 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 10 client.20976 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x5558297373f0 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16400fba0 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fe16402a030 con 0x555829709bb0 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 10 client.20976 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:28.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 10 client.20976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 20 client.20976 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555829738850 0x55582973ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 4 client.20976 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.990+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555829676cf0 con 0x555829738850 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.991+0000 7fe17259c640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555829738850 0x55582973ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.991+0000 7fe156ff5640 10 client.20976 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:28.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.991+0000 7fe17259c640 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555829738850 0x55582973ac40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510e4f3e40b13d4f server_cookie=54bf8d3804a3d7b7 in_seq=0 out_seq=0 2024-04-02T13:54:28.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.992+0000 7fe156ff5640 1 -- 172.21.15.122:0/3685579523 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555829676cf0 con 0x555829738850 2024-04-02T13:54:28.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.992+0000 7fe156ff5640 10 client.20976 handle_command_reply: tid=0 2024-04-02T13:54:28.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.992+0000 7fe17bdcc740 1 client.20976 shutdown 2024-04-02T13:54:28.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 msgr2=0x55582970c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:28.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe164019850 tx=0x7fe164009ef0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 shutdown_connections 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555829738850 0x55582973ac40 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5558297130b0 0x55582970cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55582970a4d0 0x55582970c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3685579523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x55582970c260 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 >> 172.21.15.122:0/3685579523 conn(0x5558297042e0 msgr2=0x555829706030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe156ff5640 0 client.20976 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 shutdown_connections 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3685579523 wait complete. 2024-04-02T13:54:29.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:28.993+0000 7fe17bdcc740 20 client.20976 trim_cache size 0 max 16384 2024-04-02T13:54:29.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.010+0000 7fe17bdcc740 1 Processor -- start 2024-04-02T13:54:29.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 -- start start 2024-04-02T13:54:29.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 0x555829712860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x5558297403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe174073080 con 0x5558297130b0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741a63a0 con 0x55582970a4d0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17bdcc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe174073750 con 0x555829709bb0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 0x555829712860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe172d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 0x555829712860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53290/0 (socket says 172.21.15.122:53290) 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52644/0 (socket says 172.21.15.122:52644) 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe172d9d640 1 -- 172.21.15.122:0/3570394759 learned_addr learned my addr 172.21.15.122:0/3570394759 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17259c640 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x5558297403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 -- 172.21.15.122:0/3570394759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 msgr2=0x555829712860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 0x555829712860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 -- 172.21.15.122:0/3570394759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 msgr2=0x5558297403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x5558297403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.011+0000 7fe17359e640 1 -- 172.21.15.122:0/3570394759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55582970a300 con 0x5558297130b0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.012+0000 7fe17359e640 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009bf0 tx=0x7fe16c02e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a0fbea977497c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:29.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.012+0000 7fe158ff9640 1 -- 172.21.15.122:0/3570394759 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c039040 con 0x5558297130b0 2024-04-02T13:54:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.012+0000 7fe158ff9640 1 -- 172.21.15.122:0/3570394759 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe16c02cd60 con 0x5558297130b0 2024-04-02T13:54:29.018 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.012+0000 7fe158ff9640 1 -- 172.21.15.122:0/3570394759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16c034440 con 0x5558297130b0 2024-04-02T13:54:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.013+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3570394759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 msgr2=0x555829740910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:29.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.013+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fe16c009bf0 tx=0x7fe16c02e050 comp rx=0 tx=0).stop 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.013+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3570394759 shutdown_connections 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.013+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5558297130b0 0x555829740910 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.013+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x5558297403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.014+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/3570394759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555829709bb0 0x555829712860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.014+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3570394759 >> 172.21.15.122:0/3570394759 conn(0x5558297042e0 msgr2=0x5558297046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.014+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3570394759 shutdown_connections 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.014+0000 7fe17bdcc740 1 -- 172.21.15.122:0/3570394759 wait complete. 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.014+0000 7fe17bdcc740 1 -- start start 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.015+0000 7fe17bdcc740 1 Processor -- start 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.015+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829734be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.015+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.015+0000 7fe17bdcc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297314a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.015+0000 7fe17259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43550/0 (socket says 172.21.15.122:43550) 2024-04-02T13:54:29.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 -- 172.21.15.122:0/4232340325 learned_addr learned my addr 172.21.15.122:0/4232340325 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741a5cd0 con 0x555829709bb0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17359e640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297314a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe1741a5540 con 0x55582970a4d0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe174071080 con 0x5558297130b0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe172d9d640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829734be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17bdcc740 10 client.0 _ll_register_callbacks cb 0x7fe16401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:29.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 -- 172.21.15.122:0/4232340325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 msgr2=0x5558297314a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297314a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 -- 172.21.15.122:0/4232340325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 msgr2=0x555829734be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829734be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.016+0000 7fe17259c640 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555829731b40 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe16c02cd60 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe17259c640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe16809c2f0 tx=0x7fe16809c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ae23b2ca9e67da server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe156ff5640 10 client.29924 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe156ff5640 1 -- 172.21.15.122:0/4232340325 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16801c040 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe156ff5640 1 -- 172.21.15.122:0/4232340325 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fe168098c60 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.017+0000 7fe156ff5640 1 -- 172.21.15.122:0/4232340325 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7fe1680a5260 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 10 client.29924 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=73}) -- 0x55582979d9f0 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe156ff5640 1 -- 172.21.15.122:0/4232340325 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7fe1680ce020 con 0x55582970a4d0 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 10 client.29924 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:29.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 10 client.29924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 20 client.29924 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55582979e060 0x5558297a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 4 client.29924 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.018+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555829671040 con 0x55582979e060 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.019+0000 7fe172d9d640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55582979e060 0x5558297a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.019+0000 7fe156ff5640 10 client.29924 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:29.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.019+0000 7fe172d9d640 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55582979e060 0x5558297a0480 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae76b7492e40733 server_cookie=fdc0f45b16d54b60 in_seq=0 out_seq=0 2024-04-02T13:54:29.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.021+0000 7fe156ff5640 1 -- 172.21.15.122:0/4232340325 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555829671040 con 0x55582979e060 2024-04-02T13:54:29.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.021+0000 7fe156ff5640 10 client.29924 handle_command_reply: tid=0 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.021+0000 7fe17bdcc740 1 client.29924 shutdown 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.021+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 msgr2=0x555829730f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.021+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe16809c2f0 tx=0x7fe16809c6b0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 shutdown_connections 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55582979e060 0x5558297a0480 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5558297130b0 0x5558297314a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55582970a4d0 0x555829730f60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 --2- 172.21.15.122:0/4232340325 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555829709bb0 0x555829734be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 >> 172.21.15.122:0/4232340325 conn(0x5558297042e0 msgr2=0x5558296fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe156ff5640 0 client.29924 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 shutdown_connections 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 1 -- 172.21.15.122:0/4232340325 wait complete. 2024-04-02T13:54:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.022+0000 7fe17bdcc740 20 client.29924 trim_cache size 0 max 16384 2024-04-02T13:54:29.028 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:54:29.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 msgr2=0x7fe15c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:29.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 0x7fe15c0a6260 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe16c005e30 tx=0x7fe16c0099d0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 msgr2=0x7fe1741988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fe16800e9c0 tx=0x7fe16800ee80 comp rx=0 tx=0).stop 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 shutdown_connections 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fe15c0a3da0 0x7fe15c0a6260 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe17419cc40 0x7fe17419d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe174106dd0 0x7fe1741988c0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 --2- 172.21.15.122:0/2121319558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe174100210 0x7fe174198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 >> 172.21.15.122:0/2121319558 conn(0x7fe174076f20 msgr2=0x7fe174104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 shutdown_connections 2024-04-02T13:54:29.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:29.023+0000 7fe179111640 1 -- 172.21.15.122:0/2121319558 wait complete. 2024-04-02T13:54:29.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 182219 27.44 MB/sec execute 229 sec latency 65.278 ms 2024-04-02T13:54:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:29 smithi187 ceph-mon[36064]: pgmap v608: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 818 op/s 2024-04-02T13:54:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:29 smithi188 ceph-mon[36117]: pgmap v608: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 818 op/s 2024-04-02T13:54:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:29 smithi122 ceph-mon[32114]: pgmap v608: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 17 MiB/s wr, 818 op/s 2024-04-02T13:54:30.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:54:30.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:54:30.042 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:54:30.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 -- 172.21.15.122:0/2836902634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 msgr2=0x7f3174100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 --2- 172.21.15.122:0/2836902634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f3174100840 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f3164009930 tx=0x7f316402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 -- 172.21.15.122:0/2836902634 shutdown_connections 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 --2- 172.21.15.122:0/2836902634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 0x7f317410e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 --2- 172.21.15.122:0/2836902634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f3174100840 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 --2- 172.21.15.122:0/2836902634 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 0x7f31741080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 -- 172.21.15.122:0/2836902634 >> 172.21.15.122:0/2836902634 conn(0x7f3174075d40 msgr2=0x7f3174076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 -- 172.21.15.122:0/2836902634 shutdown_connections 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.175+0000 7f317b3d6640 1 -- 172.21.15.122:0/2836902634 wait complete. 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 Processor -- start 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 -- start start 2024-04-02T13:54:30.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 0x7f3174198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 0x7f317419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3174111f60 con 0x7f31741003c0 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3174111de0 con 0x7f3174107d00 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f317b3d6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31741120e0 con 0x7f3174100d80 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.176+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52676/0 (socket says 172.21.15.122:52676) 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 -- 172.21.15.122:0/283419839 learned_addr learned my addr 172.21.15.122:0/283419839 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179973640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 0x7f317419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3178971640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 0x7f3174198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 -- 172.21.15.122:0/283419839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 msgr2=0x7f3174198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 0x7f3174198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 -- 172.21.15.122:0/283419839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 msgr2=0x7f317419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 0x7f317419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 -- 172.21.15.122:0/283419839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317419d3b0 con 0x7f31741003c0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f3179172640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9f0 tx=0x7f317000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f316a7fc640 1 -- 172.21.15.122:0/283419839 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317000cd80 con 0x7f31741003c0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f316a7fc640 1 -- 172.21.15.122:0/283419839 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3170010910 con 0x7f31741003c0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317419d6a0 con 0x7f31741003c0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f316a7fc640 1 -- 172.21.15.122:0/283419839 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170018990 con 0x7f31741003c0 2024-04-02T13:54:30.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.177+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31741a4f30 con 0x7f31741003c0 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f316a7fc640 1 -- 172.21.15.122:0/283419839 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3170010ac0 con 0x7f31741003c0 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 Processor -- start 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 -- start start 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f316a7fc640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f5edea9770 con 0x55f5edf452c0 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.180+0000 7f317dfb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f5ede51960 con 0x55f5edf4e7c0 2024-04-02T13:54:30.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f317dfb6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f5edea58c0 con 0x55f5edf45be0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179973640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52680/0 (socket says 172.21.15.122:52680) 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179172640 1 -- 172.21.15.122:0/2302708192 learned_addr learned my addr 172.21.15.122:0/2302708192 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179973640 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f316a7fc640 1 -- 172.21.15.122:0/283419839 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f31700d0f10 con 0x7f31741003c0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 -- 172.21.15.122:0/2302708192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 msgr2=0x55f5edf50bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf50bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 -- 172.21.15.122:0/2302708192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 msgr2=0x55f5edf456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 -- 172.21.15.122:0/2302708192 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5edc857e0 con 0x55f5edf45be0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3179973640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f316c005fd0 tx=0x7f316c009f10 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f3178971640 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4e280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3164009230 tx=0x7f31640095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49dee37be8d78837 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f315f7fe640 1 -- 172.21.15.122:0/2302708192 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164043070 con 0x55f5edf45be0 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f315f7fe640 1 -- 172.21.15.122:0/2302708192 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316402c670 con 0x55f5edf45be0 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.181+0000 7f315f7fe640 1 -- 172.21.15.122:0/2302708192 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3164004940 con 0x55f5edf45be0 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- 172.21.15.122:0/2302708192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 msgr2=0x55f5edf4e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4e280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3164009230 tx=0x7f31640095f0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- 172.21.15.122:0/2302708192 shutdown_connections 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf50bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4e280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2302708192 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- 172.21.15.122:0/2302708192 >> 172.21.15.122:0/2302708192 conn(0x55f5edf3f9f0 msgr2=0x55f5edf3fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- 172.21.15.122:0/2302708192 shutdown_connections 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- 172.21.15.122:0/2302708192 wait complete. 2024-04-02T13:54:30.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.182+0000 7f317dfb6740 1 -- start start 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 Processor -- start 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf452c0 0x55f5edf4b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf45be0 0x55f5edf4bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f5edf51550 con 0x55f5edf4e7c0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f5ede64d20 con 0x55f5edf45be0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f5edf516d0 con 0x55f5edf452c0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f317dfb6740 10 client.0 _ll_register_callbacks cb 0x55f5ede31a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179172640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf452c0 0x55f5edf4b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52690/0 (socket says 172.21.15.122:52690) 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 -- 172.21.15.122:0/151368537 learned_addr learned my addr 172.21.15.122:0/151368537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3178971640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf45be0 0x55f5edf4bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 -- 172.21.15.122:0/151368537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf452c0 msgr2=0x55f5edf4b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.199 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf452c0 0x55f5edf4b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 -- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf45be0 msgr2=0x55f5edf4bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf45be0 0x55f5edf4bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 -- 172.21.15.122:0/151368537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5edce5f70 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f3179973640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f316c01e570 tx=0x7f316c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80eb84dfc972f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.183+0000 7f315cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c024070 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f316c003970 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316402c670 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f316c033050 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 10 client.20994 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55f5edb7d170 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316402c670 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f316c004020 con 0x55f5edf4e7c0 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 10 client.20994 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:30.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 10 client.20994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 20 client.20994 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edf61020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 4 client.20994 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5edeb6cb0 con 0x55f5edf5ec30 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.184+0000 7f3179172640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edf61020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.185+0000 7f315cff9640 10 client.20994 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.185+0000 7f3179172640 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edf61020 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3246807f9c231a39 server_cookie=4dd42a0790d75319 in_seq=0 out_seq=0 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.185+0000 7f315cff9640 1 -- 172.21.15.122:0/151368537 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5edeb6cb0 con 0x55f5edf5ec30 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.185+0000 7f315cff9640 10 client.20994 handle_command_reply: tid=0 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.185+0000 7f317dfb6740 1 client.20994 shutdown 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 msgr2=0x55f5edf4c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f316c01e570 tx=0x7f316c01d000 comp rx=0 tx=0).stop 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 shutdown_connections 2024-04-02T13:54:30.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edf61020 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f315cff9640 0 client.20994 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf4c280 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf45be0 0x55f5edf4bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 --2- 172.21.15.122:0/151368537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf452c0 0x55f5edf4b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 >> 172.21.15.122:0/151368537 conn(0x55f5edf3f9f0 msgr2=0x55f5edf41740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 shutdown_connections 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 1 -- 172.21.15.122:0/151368537 wait complete. 2024-04-02T13:54:30.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.186+0000 7f317dfb6740 20 client.20994 trim_cache size 0 max 16384 2024-04-02T13:54:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.205+0000 7f317dfb6740 1 Processor -- start 2024-04-02T13:54:30.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.205+0000 7f317dfb6740 1 -- start start 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf66910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31741aefc0 con 0x55f5edf452c0 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31741b0bc0 con 0x55f5edf4e7c0 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f317dfb6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31741af690 con 0x55f5edf45be0 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179172640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52698/0 (socket says 172.21.15.122:52698) 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf66910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179172640 1 -- 172.21.15.122:0/1623163679 learned_addr learned my addr 172.21.15.122:0/1623163679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3178971640 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 -- 172.21.15.122:0/1623163679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 msgr2=0x55f5edf4d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 -- 172.21.15.122:0/1623163679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 msgr2=0x55f5edf5a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 -- 172.21.15.122:0/1623163679 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5edf4a7b0 con 0x55f5edf4e7c0 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179172640 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f3179973640 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf66910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f316c000f80 tx=0x7f316c01e400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775d9bed67db5dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f315effd640 1 -- 172.21.15.122:0/1623163679 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c03c070 con 0x55f5edf4e7c0 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f315effd640 1 -- 172.21.15.122:0/1623163679 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c0115f0 con 0x55f5edf4e7c0 2024-04-02T13:54:30.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.206+0000 7f315effd640 1 -- 172.21.15.122:0/1623163679 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c024040 con 0x55f5edf4e7c0 2024-04-02T13:54:30.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 -- 172.21.15.122:0/1623163679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 msgr2=0x55f5edf66910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf66910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f316c000f80 tx=0x7f316c01e400 comp rx=0 tx=0).stop 2024-04-02T13:54:30.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 -- 172.21.15.122:0/1623163679 shutdown_connections 2024-04-02T13:54:30.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf4e7c0 0x55f5edf66910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf4d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.207+0000 7f317dfb6740 1 --2- 172.21.15.122:0/1623163679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf452c0 0x55f5edf5a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.208+0000 7f317dfb6740 1 -- 172.21.15.122:0/1623163679 >> 172.21.15.122:0/1623163679 conn(0x55f5edf3f9f0 msgr2=0x55f5edf4cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.208+0000 7f317dfb6740 1 -- 172.21.15.122:0/1623163679 shutdown_connections 2024-04-02T13:54:30.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.208+0000 7f317dfb6740 1 -- 172.21.15.122:0/1623163679 wait complete. 2024-04-02T13:54:30.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.208+0000 7f317dfb6740 1 -- start start 2024-04-02T13:54:30.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.208+0000 7f317dfb6740 1 Processor -- start 2024-04-02T13:54:30.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf52df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf53330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31741b04f0 con 0x55f5edf4e7c0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31741afd60 con 0x55f5edf452c0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31741acfc0 con 0x55f5edf45be0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f317dfb6740 10 client.0 _ll_register_callbacks cb 0x7f316c009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43600/0 (socket says 172.21.15.122:43600) 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 -- 172.21.15.122:0/2865627843 learned_addr learned my addr 172.21.15.122:0/2865627843 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3178971640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf52df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179973640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf53330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 -- 172.21.15.122:0/2865627843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 msgr2=0x55f5edf52df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf52df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 -- 172.21.15.122:0/2865627843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 msgr2=0x55f5edf53330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.209+0000 7f3179172640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf53330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f3179172640 1 -- 172.21.15.122:0/2865627843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5edf45a10 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f3179172640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f317000cfd0 tx=0x7f31700bd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12b9bdb675ed0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f315cff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3170026040 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f316c0115f0 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f317009ae70 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f31700bd0b0 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f317dfb6740 10 client.29942 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.210+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55f5edf61390 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f316c0115f0 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f3170018cc0 con 0x55f5edf452c0 2024-04-02T13:54:30.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 10 client.29942 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 10 client.29942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 20 client.29942 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edfc8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 4 client.29942 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5edeb1000 con 0x55f5edf5ec30 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.211+0000 7f3178971640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edfc8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.212+0000 7f315cff9640 10 client.29942 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.212+0000 7f3178971640 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edfc8360 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fbbce7fa70089b server_cookie=9caf51fd81a4d80a in_seq=0 out_seq=0 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.213+0000 7f315cff9640 1 -- 172.21.15.122:0/2865627843 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5edeb1000 con 0x55f5edf5ec30 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.213+0000 7f315cff9640 10 client.29942 handle_command_reply: tid=0 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.213+0000 7f317dfb6740 1 client.29942 shutdown 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 msgr2=0x55f5edf56a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f317000cfd0 tx=0x7f31700bd040 comp rx=0 tx=0).stop 2024-04-02T13:54:30.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 shutdown_connections 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f5edf5ec30 0x55f5edfc8360 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f5edf4e7c0 0x55f5edf53330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f315cff9640 0 client.29942 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f5edf45be0 0x55f5edf52df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 --2- 172.21.15.122:0/2865627843 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f5edf452c0 0x55f5edf56a70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.215+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 >> 172.21.15.122:0/2865627843 conn(0x55f5edf3f9f0 msgr2=0x55f5edf3eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 shutdown_connections 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317dfb6740 1 -- 172.21.15.122:0/2865627843 wait complete. 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317dfb6740 20 client.29942 trim_cache size 0 max 16384 2024-04-02T13:54:30.225 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:54:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:54:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "8aa2f7d3-7e8b-4328-a18a-4edb308aed90", 2024-04-02T13:54:30.226 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:54:30.226 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 msgr2=0x7f31600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f316c005fd0 tx=0x7f316c009f10 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 msgr2=0x7f31741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.216+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f317000e9f0 tx=0x7f317000eeb0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 shutdown_connections 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3174107d00 0x7f317419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3174100d80 0x7f3174198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 --2- 172.21.15.122:0/283419839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31741003c0 0x7f31741983c0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 >> 172.21.15.122:0/283419839 conn(0x7f3174075d40 msgr2=0x7f31740ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 shutdown_connections 2024-04-02T13:54:30.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.217+0000 7f317b3d6640 1 -- 172.21.15.122:0/283419839 wait complete. 2024-04-02T13:54:30.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:54:30.236 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:54:30.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 182827 27.44 MB/sec execute 230 sec latency 104.010 ms 2024-04-02T13:54:30.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.395+0000 7f1fb6588640 1 -- 172.21.15.122:0/92840931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0100970 msgr2=0x7f1fb0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.395+0000 7f1fb6588640 1 --2- 172.21.15.122:0/92840931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0100970 0x7f1fb0100df0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4009890 tx=0x7f1fa402bb70 comp rx=0 tx=0).stop 2024-04-02T13:54:30.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.396+0000 7f1fb6588640 1 -- 172.21.15.122:0/92840931 shutdown_connections 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.396+0000 7f1fb6588640 1 --2- 172.21.15.122:0/92840931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0101330 0x7f1fb010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.396+0000 7f1fb6588640 1 --2- 172.21.15.122:0/92840931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0100970 0x7f1fb0100df0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.396+0000 7f1fb6588640 1 --2- 172.21.15.122:0/92840931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.397+0000 7f1fb6588640 1 -- 172.21.15.122:0/92840931 >> 172.21.15.122:0/92840931 conn(0x7f1fb0077b00 msgr2=0x7f1fb0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.397+0000 7f1fb6588640 1 -- 172.21.15.122:0/92840931 shutdown_connections 2024-04-02T13:54:30.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.397+0000 7f1fb6588640 1 -- 172.21.15.122:0/92840931 wait complete. 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.397+0000 7f1fb6588640 1 Processor -- start 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 -- start start 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 0x7f1fb0116080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 0x7f1fb0110140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1faffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 0x7f1fb0116080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1faffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 0x7f1fb0116080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43612/0 (socket says 172.21.15.122:43612) 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 0x7f1fb0110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 0x7f1fb0110140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53378/0 (socket says 172.21.15.122:53378) 2024-04-02T13:54:30.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb010ec10 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1faffff640 1 -- 172.21.15.122:0/281407114 learned_addr learned my addr 172.21.15.122:0/281407114 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb010ea90 con 0x7f1fb0100970 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb010ed90 con 0x7f1fb0101330 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.398+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 -- 172.21.15.122:0/281407114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 msgr2=0x7f1fb0110140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 0x7f1fb0110140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 -- 172.21.15.122:0/281407114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 msgr2=0x7f1fb0116080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 0x7f1fb0116080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb0110cc0 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0110680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fad7fa640 1 -- 172.21.15.122:0/281407114 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800ce20 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fad7fa640 1 -- 172.21.15.122:0/281407114 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8005020 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fad7fa640 1 -- 172.21.15.122:0/281407114 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8010720 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb01b3820 con 0x7f1fb0106980 2024-04-02T13:54:30.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.399+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb01b3bd0 con 0x7f1fb0106980 2024-04-02T13:54:30.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.402+0000 7f1fad7fa640 1 -- 172.21.15.122:0/281407114 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1fa8005540 con 0x7f1fb0106980 2024-04-02T13:54:30.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.403+0000 7f1fb9168740 1 Processor -- start 2024-04-02T13:54:30.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.403+0000 7f1fad7fa640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 0x7f1f980a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.403+0000 7f1fb9168740 1 -- start start 2024-04-02T13:54:30.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.403+0000 7f1fad7fa640 1 -- 172.21.15.122:0/281407114 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f1fa80d1190 con 0x7f1fb0106980 2024-04-02T13:54:30.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 0x7f1f980a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd911c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b4bce9830 con 0x556b4bd8edd0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b4bc91820 con 0x556b4bd861f0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb9168740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b4bce57e0 con 0x556b4bd858d0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53390/0 (socket says 172.21.15.122:53390) 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 -- 172.21.15.122:0/839601659 learned_addr learned my addr 172.21.15.122:0/839601659 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd911c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 -- 172.21.15.122:0/839601659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 msgr2=0x556b4bd8e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 -- 172.21.15.122:0/839601659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 msgr2=0x556b4bd911c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd911c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 -- 172.21.15.122:0/839601659 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4bac77d0 con 0x556b4bd858d0 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 0x7f1f980a62b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0111940 tx=0x7f1fa4034c60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1faffff640 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009d00 tx=0x7f1fa0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71eb71c86973c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.404+0000 7f1f967fc640 1 -- 172.21.15.122:0/839601659 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0009df0 con 0x556b4bd858d0 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1f967fc640 1 -- 172.21.15.122:0/839601659 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa00035f0 con 0x556b4bd858d0 2024-04-02T13:54:30.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1f967fc640 1 -- 172.21.15.122:0/839601659 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0019670 con 0x556b4bd858d0 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 -- 172.21.15.122:0/839601659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 msgr2=0x556b4bd85cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009d00 tx=0x7f1fa0011810 comp rx=0 tx=0).stop 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 -- 172.21.15.122:0/839601659 shutdown_connections 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd911c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 --2- 172.21.15.122:0/839601659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd85cb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 -- 172.21.15.122:0/839601659 >> 172.21.15.122:0/839601659 conn(0x556b4bd80000 msgr2=0x556b4bd803e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.405+0000 7f1fb9168740 1 -- 172.21.15.122:0/839601659 shutdown_connections 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.406+0000 7f1fb9168740 1 -- 172.21.15.122:0/839601659 wait complete. 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.406+0000 7f1fb9168740 1 -- start start 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.406+0000 7f1fb9168740 1 Processor -- start 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd8e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd8bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b4bda2bd0 con 0x556b4bd8edd0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556b4bca4d30 con 0x556b4bd861f0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556b4bda2d50 con 0x556b4bd858d0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb9168740 10 client.0 _ll_register_callbacks cb 0x556b4bca4720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb4b25640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd8bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:43624/0 (socket says 172.21.15.122:43624) 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 -- 172.21.15.122:0/614508491 learned_addr learned my addr 172.21.15.122:0/614508491 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faffff640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd8e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 -- 172.21.15.122:0/614508491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 msgr2=0x556b4bd8e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd8e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 -- 172.21.15.122:0/614508491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 msgr2=0x556b4bd8bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd8bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1faf7fe640 1 -- 172.21.15.122:0/614508491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4b9fd3f0 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.407+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd8bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4036000 tx=0x7f1fa402bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11849a14eb2d9cb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1f8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4053060 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f1fa402c920 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa00035f0 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f1fa4039040 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1fb9168740 10 client.29954 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.408+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x556b4b9bd0f0 con 0x556b4bd861f0 2024-04-02T13:54:30.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa00035f0 con 0x556b4bd861f0 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f1fa403f030 con 0x556b4bd861f0 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 10 client.29954 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 10 client.29954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 20 client.29954 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4bdafcb0 0x556b4bdb20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 4 client.29954 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.409+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b4bcf6d70 con 0x556b4bdafcb0 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.410+0000 7f1faffff640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4bdafcb0 0x556b4bdb20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.410+0000 7f1faffff640 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4bdafcb0 0x556b4bdb20d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbaa76b7ba09eb00 server_cookie=d0d082908f1915f3 in_seq=0 out_seq=0 2024-04-02T13:54:30.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.410+0000 7f1f8f7fe640 10 client.29954 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.415+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/614508491 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b4bcf6d70 con 0x556b4bdafcb0 2024-04-02T13:54:30.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.415+0000 7f1f8f7fe640 10 client.29954 handle_command_reply: tid=0 2024-04-02T13:54:30.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.415+0000 7f1fb9168740 1 client.29954 shutdown 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 msgr2=0x556b4bd8b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4036000 tx=0x7f1fa402bb70 comp rx=0 tx=0).stop 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 shutdown_connections 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4bdafcb0 0x556b4bdb20d0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd8edd0 0x556b4bd8bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd861f0 0x556b4bd8b9b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 --2- 172.21.15.122:0/614508491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bd8e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 >> 172.21.15.122:0/614508491 conn(0x556b4bd80000 msgr2=0x556b4bd81d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1f8f7fe640 0 client.29954 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 shutdown_connections 2024-04-02T13:54:30.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 1 -- 172.21.15.122:0/614508491 wait complete. 2024-04-02T13:54:30.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.416+0000 7f1fb9168740 20 client.29954 trim_cache size 0 max 16384 2024-04-02T13:54:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.440+0000 7f1fb9168740 1 Processor -- start 2024-04-02T13:54:30.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.440+0000 7f1fb9168740 1 -- start start 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.440+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bd870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd7f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb9168740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b55b0 con 0x556b4bd861f0 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb9168740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b71b0 con 0x556b4bd8edd0 2024-04-02T13:54:30.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb9168740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b5c80 con 0x556b4bd858d0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53412/0 (socket says 172.21.15.122:53412) 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 -- 172.21.15.122:0/2415666332 learned_addr learned my addr 172.21.15.122:0/2415666332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1fb4b25640 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd7f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bd870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 -- 172.21.15.122:0/2415666332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 msgr2=0x556b4bd7f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd7f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 -- 172.21.15.122:0/2415666332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 msgr2=0x556b4bd870c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bd870c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 -- 172.21.15.122:0/2415666332 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4bd7fe30 con 0x556b4bd858d0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.441+0000 7f1faffff640 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009c50 tx=0x7f1fa00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad0c313b42c0d5a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.442+0000 7f1f95ffb640 1 -- 172.21.15.122:0/2415666332 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005040 con 0x556b4bd858d0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.442+0000 7f1f95ffb640 1 -- 172.21.15.122:0/2415666332 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa0004640 con 0x556b4bd858d0 2024-04-02T13:54:30.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.442+0000 7f1f95ffb640 1 -- 172.21.15.122:0/2415666332 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0003850 con 0x556b4bd858d0 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.442+0000 7f1fb9168740 1 -- 172.21.15.122:0/2415666332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 msgr2=0x556b4bdaba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.442+0000 7f1fb9168740 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009c50 tx=0x7f1fa00043b0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- 172.21.15.122:0/2415666332 shutdown_connections 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd7f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bd870c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- 172.21.15.122:0/2415666332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bdaba90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- 172.21.15.122:0/2415666332 >> 172.21.15.122:0/2415666332 conn(0x556b4bd80000 msgr2=0x556b4bd7e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- 172.21.15.122:0/2415666332 shutdown_connections 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- 172.21.15.122:0/2415666332 wait complete. 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- start start 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 Processor -- start 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bda7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b6ae0 con 0x556b4bd861f0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb01b6350 con 0x556b4bd8edd0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb00688f0 con 0x556b4bd858d0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.443+0000 7f1fb9168740 10 client.0 _ll_register_callbacks cb 0x7f1fa40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bda7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52784/0 (socket says 172.21.15.122:52784) 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 -- 172.21.15.122:0/1230341001 learned_addr learned my addr 172.21.15.122:0/1230341001 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1fb4b25640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bda7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:53418/0 (socket says 172.21.15.122:53418) 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 -- 172.21.15.122:0/1230341001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 msgr2=0x556b4bda7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bda7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 -- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 msgr2=0x556b4bd881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 -- 172.21.15.122:0/1230341001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4bd889e0 con 0x556b4bd861f0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1faf7fe640 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4003690 tx=0x7f1fa4003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b54d25d5e2ac3905 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1f8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa4053040 con 0x556b4bd861f0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f1fa402cb30 con 0x556b4bd861f0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fa0004640 con 0x556b4bd861f0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f1fa4067050 con 0x556b4bd861f0 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1fb9168740 10 client.21018 fetch_fsmap learned FSMap version 73 2024-04-02T13:54:30.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.444+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x556b4be15da0 con 0x556b4bd861f0 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fa0004640 con 0x556b4bd861f0 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 73) ==== 8378+0+0 (secure 0 0 0) 0x7f1fa403f070 con 0x556b4bd861f0 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 10 client.21018 fetch_fsmap finished waiting for FSMap version 73 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 10 client.21018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 20 client.21018 populate_metadata read hostname 'smithi122' 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4be164f0 0x556b4be18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 4 client.21018 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b4bcf10c0 con 0x556b4be164f0 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.445+0000 7f1faffff640 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4be164f0 0x556b4be18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.446+0000 7f1f8f7fe640 10 client.21018 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.446+0000 7f1faffff640 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4be164f0 0x556b4be18910 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8234c6cc24fe2c12 server_cookie=df21a8b8b914e5d8 in_seq=0 out_seq=0 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.452+0000 7f1f8f7fe640 1 -- 172.21.15.122:0/1230341001 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x556b4bcf10c0 con 0x556b4be164f0 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.452+0000 7f1f8f7fe640 10 client.21018 handle_command_reply: tid=0 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.452+0000 7f1fb9168740 1 client.21018 shutdown 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.452+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 msgr2=0x556b4bda84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.452+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4003690 tx=0x7f1fa4003e40 comp rx=0 tx=0).stop 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 shutdown_connections 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x556b4be164f0 0x556b4be18910 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x556b4bd8edd0 0x556b4bd881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b4bd861f0 0x556b4bda84e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 --2- 172.21.15.122:0/1230341001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556b4bd858d0 0x556b4bda7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 >> 172.21.15.122:0/1230341001 conn(0x556b4bd80000 msgr2=0x556b4bd7f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1f8f7fe640 0 client.21018 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 shutdown_connections 2024-04-02T13:54:30.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 1 -- 172.21.15.122:0/1230341001 wait complete. 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.453+0000 7f1fb9168740 20 client.21018 trim_cache size 0 max 16384 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "scrub active (59 inodes in the stack)", 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "8aa2f7d3-7e8b-4328-a18a-4edb308aed90": { 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "8aa2f7d3-7e8b-4328-a18a-4edb308aed90", 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:54:30.458 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:54:30.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:54:30.459 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 msgr2=0x7f1f980a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 0x7f1f980a62b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0111940 tx=0x7f1fa4034c60 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 msgr2=0x7f1fb0110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0110680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8009fd0 tx=0x7f1fa800ef10 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 shutdown_connections 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1f980a3df0 0x7f1f980a62b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fb0106980 0x7f1fb0110680 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb0101330 0x7f1fb0110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 --2- 172.21.15.122:0/281407114 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fb0100970 0x7f1fb0116080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 >> 172.21.15.122:0/281407114 conn(0x7f1fb0077b00 msgr2=0x7f1fb0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 shutdown_connections 2024-04-02T13:54:30.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:54:30.454+0000 7f1fb6588640 1 -- 172.21.15.122:0/281407114 wait complete. 2024-04-02T13:54:30.471 INFO:tasks.cephfs.filesystem:scrub status for tag:8aa2f7d3-7e8b-4328-a18a-4edb308aed90 - {'path': '/', 'tag': '8aa2f7d3-7e8b-4328-a18a-4edb308aed90', 'options': 'recursive,force'} 2024-04-02T13:54:31.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 183441 27.48 MB/sec execute 231 sec latency 169.368 ms 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: pgmap v609: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 445 op/s 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:31 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: pgmap v609: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 445 op/s 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:31 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: pgmap v609: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 445 op/s 2024-04-02T13:54:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:54:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:31.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:31 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:54:32.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 183924 27.49 MB/sec execute 232 sec latency 221.967 ms 2024-04-02T13:54:33.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 183950 27.38 MB/sec execute 233 sec latency 1215.934 ms 2024-04-02T13:54:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:33 smithi187 ceph-mon[36064]: pgmap v610: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 575 op/s 2024-04-02T13:54:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:33 smithi188 ceph-mon[36117]: pgmap v610: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 575 op/s 2024-04-02T13:54:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:33 smithi122 ceph-mon[32114]: pgmap v610: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.4 MiB/s wr, 575 op/s 2024-04-02T13:54:34.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 184133 27.32 MB/sec execute 234 sec latency 1283.501 ms 2024-04-02T13:54:35.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 184319 27.23 MB/sec execute 235 sec latency 197.512 ms 2024-04-02T13:54:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:35 smithi187 ceph-mon[36064]: pgmap v611: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-04-02T13:54:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:35 smithi188 ceph-mon[36117]: pgmap v611: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-04-02T13:54:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:35 smithi122 ceph-mon[32114]: pgmap v611: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 545 op/s 2024-04-02T13:54:36.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 184618 27.15 MB/sec execute 236 sec latency 201.940 ms 2024-04-02T13:54:37.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 185266 27.19 MB/sec execute 237 sec latency 94.569 ms 2024-04-02T13:54:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:37 smithi187 ceph-mon[36064]: pgmap v612: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 461 op/s 2024-04-02T13:54:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:37 smithi188 ceph-mon[36117]: pgmap v612: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 461 op/s 2024-04-02T13:54:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:37 smithi122 ceph-mon[32114]: pgmap v612: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 461 op/s 2024-04-02T13:54:38.335 INFO:tasks.workunit.client.0.smithi122.stdout: 10 185945 27.22 MB/sec execute 238 sec latency 97.905 ms 2024-04-02T13:54:39.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 186627 27.28 MB/sec execute 239 sec latency 78.096 ms 2024-04-02T13:54:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:39 smithi188 ceph-mon[36117]: pgmap v613: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 709 op/s 2024-04-02T13:54:39.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:39 smithi122 ceph-mon[32114]: pgmap v613: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 709 op/s 2024-04-02T13:54:39.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:39 smithi187 ceph-mon[36064]: pgmap v613: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 709 op/s 2024-04-02T13:54:40.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 187236 27.34 MB/sec execute 240 sec latency 148.136 ms 2024-04-02T13:54:40.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:40 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:40.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:40 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:40.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:40 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:54:41.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 187731 27.32 MB/sec execute 241 sec latency 97.304 ms 2024-04-02T13:54:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:41 smithi188 ceph-mon[36117]: pgmap v614: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 559 op/s 2024-04-02T13:54:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:41 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:41 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:41.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:41 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:54:41.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:41 smithi122 ceph-mon[32114]: pgmap v614: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 559 op/s 2024-04-02T13:54:41.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:41 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:41.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:41 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:41.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:41 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:54:41.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:41 smithi187 ceph-mon[36064]: pgmap v614: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 559 op/s 2024-04-02T13:54:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:41 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:54:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:41 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:54:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:41 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:54:42.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 188165 27.29 MB/sec execute 242 sec latency 122.203 ms 2024-04-02T13:54:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:42.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:42.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:42.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:42 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:54:42.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:42.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:42.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:42.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:42.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:42 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:54:42.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:42.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:42.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:42.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:42.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:42 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:54:43.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 188634 27.26 MB/sec execute 243 sec latency 100.282 ms 2024-04-02T13:54:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:43 smithi188 ceph-mon[36117]: pgmap v615: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-04-02T13:54:43.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:43 smithi122 ceph-mon[32114]: pgmap v615: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-04-02T13:54:43.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:43 smithi187 ceph-mon[36064]: pgmap v615: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 718 op/s 2024-04-02T13:54:44.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 189342 27.30 MB/sec execute 244 sec latency 67.743 ms 2024-04-02T13:54:44.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:44 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:54:44.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:44 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:54:45.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:44 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:54:45.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 190079 27.36 MB/sec execute 245 sec latency 119.716 ms 2024-04-02T13:54:45.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:45 smithi122 ceph-mon[32114]: pgmap v616: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:54:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:45 smithi187 ceph-mon[36064]: pgmap v616: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:54:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:45 smithi188 ceph-mon[36117]: pgmap v616: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:54:46.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 190619 27.38 MB/sec execute 246 sec latency 218.837 ms 2024-04-02T13:54:47.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 191278 27.45 MB/sec execute 247 sec latency 70.286 ms 2024-04-02T13:54:47.787 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:47 smithi187 ceph-mon[36064]: pgmap v617: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 540 op/s 2024-04-02T13:54:47.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:47 smithi122 ceph-mon[32114]: pgmap v617: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 540 op/s 2024-04-02T13:54:48.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:47 smithi188 ceph-mon[36117]: pgmap v617: 129 pgs: 129 active+clean; 2.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 540 op/s 2024-04-02T13:54:48.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 191771 27.44 MB/sec execute 248 sec latency 84.019 ms 2024-04-02T13:54:49.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 192188 27.37 MB/sec execute 249 sec latency 91.766 ms 2024-04-02T13:54:49.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:49 smithi122 ceph-mon[32114]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-02T13:54:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:49 smithi187 ceph-mon[36064]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-02T13:54:50.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:49 smithi188 ceph-mon[36117]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-02T13:54:50.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 192867 27.41 MB/sec execute 250 sec latency 96.830 ms 2024-04-02T13:54:51.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 193643 27.49 MB/sec execute 251 sec latency 95.130 ms 2024-04-02T13:54:51.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:51 smithi122 ceph-mon[32114]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 473 op/s 2024-04-02T13:54:51.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:51 smithi187 ceph-mon[36064]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 473 op/s 2024-04-02T13:54:52.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:51 smithi188 ceph-mon[36117]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.6 MiB/s wr, 473 op/s 2024-04-02T13:54:52.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194278 27.49 MB/sec execute 252 sec latency 64.450 ms 2024-04-02T13:54:53.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194471 27.44 MB/sec execute 253 sec latency 738.682 ms 2024-04-02T13:54:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:53 smithi122 ceph-mon[32114]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-04-02T13:54:53.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:53 smithi187 ceph-mon[36064]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-04-02T13:54:54.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:53 smithi188 ceph-mon[36117]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 571 op/s 2024-04-02T13:54:54.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194527 27.34 MB/sec execute 254 sec latency 1541.670 ms 2024-04-02T13:54:55.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 194763 27.30 MB/sec execute 255 sec latency 237.522 ms 2024-04-02T13:54:55.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:55 smithi122 ceph-mon[32114]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 499 op/s 2024-04-02T13:54:55.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:55 smithi187 ceph-mon[36064]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 499 op/s 2024-04-02T13:54:56.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:55 smithi188 ceph-mon[36117]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 499 op/s 2024-04-02T13:54:56.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 195020 27.26 MB/sec execute 256 sec latency 155.425 ms 2024-04-02T13:54:57.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 195340 27.21 MB/sec execute 257 sec latency 150.174 ms 2024-04-02T13:54:57.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:57 smithi122 ceph-mon[32114]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 449 op/s 2024-04-02T13:54:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:57 smithi187 ceph-mon[36064]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 449 op/s 2024-04-02T13:54:58.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:57 smithi188 ceph-mon[36117]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 449 op/s 2024-04-02T13:54:58.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 195902 27.17 MB/sec execute 258 sec latency 90.350 ms 2024-04-02T13:54:59.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 196524 27.24 MB/sec execute 259 sec latency 71.938 ms 2024-04-02T13:54:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:59 smithi122 ceph-mon[32114]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-04-02T13:54:59.843 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:54:59 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:54:59.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:59 smithi187 ceph-mon[36064]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-04-02T13:54:59.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:54:59 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:59 smithi188 ceph-mon[36117]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 687 op/s 2024-04-02T13:55:00.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:54:59 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:00.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197080 27.25 MB/sec execute 260 sec latency 188.433 ms 2024-04-02T13:55:00.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:00.473 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 -- 172.21.15.122:0/3306745277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 msgr2=0x7ff028101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 --2- 172.21.15.122:0/3306745277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff028101f10 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7ff024009930 tx=0x7ff02402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 -- 172.21.15.122:0/3306745277 shutdown_connections 2024-04-02T13:55:00.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 --2- 172.21.15.122:0/3306745277 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028102450 0x7ff02810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 --2- 172.21.15.122:0/3306745277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff028101f10 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 --2- 172.21.15.122:0/3306745277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028107aa0 0x7ff028107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 -- 172.21.15.122:0/3306745277 >> 172.21.15.122:0/3306745277 conn(0x7ff028078f00 msgr2=0x7ff028079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 -- 172.21.15.122:0/3306745277 shutdown_connections 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.587+0000 7ff0304ba640 1 -- 172.21.15.122:0/3306745277 wait complete. 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 Processor -- start 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 -- start start 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028102450 0x7ff028198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028107aa0 0x7ff02819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff028111eb0 con 0x7ff028101a90 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff028111d30 con 0x7ff028107aa0 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff0304ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff028112030 con 0x7ff028102450 2024-04-02T13:55:00.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51172/0 (socket says 172.21.15.122:51172) 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 -- 172.21.15.122:0/2432735746 learned_addr learned my addr 172.21.15.122:0/2432735746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02ea57640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028107aa0 0x7ff02819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02da55640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028102450 0x7ff028198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 -- 172.21.15.122:0/2432735746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028102450 msgr2=0x7ff028198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028102450 0x7ff028198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 -- 172.21.15.122:0/2432735746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028107aa0 msgr2=0x7ff02819cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028107aa0 0x7ff02819cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 -- 172.21.15.122:0/2432735746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02819d3c0 con 0x7ff028101a90 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff02e256640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2432735746 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800cd80 con 0x7ff028101a90 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.588+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2432735746 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff018005500 con 0x7ff028101a90 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.589+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02819d6b0 con 0x7ff028101a90 2024-04-02T13:55:00.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.589+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2432735746 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018010690 con 0x7ff028101a90 2024-04-02T13:55:00.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.589+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281a4f40 con 0x7ff028101a90 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2432735746 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff0180046c0 con 0x7ff028101a90 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 Processor -- start 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 -- start start 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff01f7fe640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 0x7ff0140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7342bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 0x560e73434890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e7338f830 con 0x560e73434dd0 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560e73337820 con 0x560e7342c1f0 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.591+0000 7ff03317c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560e7338b7e0 con 0x560e7342b8d0 2024-04-02T13:55:00.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 0x7ff0140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff01f7fe640 1 -- 172.21.15.122:0/2432735746 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff0180d0f90 con 0x7ff028101a90 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51182/0 (socket says 172.21.15.122:51182) 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 -- 172.21.15.122:0/4090277213 learned_addr learned my addr 172.21.15.122:0/4090277213 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02da55640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 0x560e73434890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02e256640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7342bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 -- 172.21.15.122:0/4090277213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 msgr2=0x560e7342bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7342bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 -- 172.21.15.122:0/4090277213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 msgr2=0x560e73434890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 0x560e73434890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 -- 172.21.15.122:0/4090277213 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e7316d7d0 con 0x560e73434dd0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02da55640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 0x560e73434890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 0x7ff0140a6260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff020012c80 tx=0x7ff020009e90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff02ea57640 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fc0 tx=0x7ff02001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8751b9686f82680e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff01cbfe640 1 -- 172.21.15.122:0/4090277213 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020024070 con 0x560e73434dd0 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff01cbfe640 1 -- 172.21.15.122:0/4090277213 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02000dbe0 con 0x560e73434dd0 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.592+0000 7ff01cbfe640 1 -- 172.21.15.122:0/4090277213 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02001e3f0 con 0x560e73434dd0 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- 172.21.15.122:0/4090277213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 msgr2=0x560e734371c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7ff020009fc0 tx=0x7ff02001c000 comp rx=0 tx=0).stop 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- 172.21.15.122:0/4090277213 shutdown_connections 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e73434dd0 0x560e734371c0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342c1f0 0x560e73434890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 --2- 172.21.15.122:0/4090277213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7342bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- 172.21.15.122:0/4090277213 >> 172.21.15.122:0/4090277213 conn(0x560e73426000 msgr2=0x560e734263e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- 172.21.15.122:0/4090277213 shutdown_connections 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- 172.21.15.122:0/4090277213 wait complete. 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 -- start start 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.593+0000 7ff03317c740 1 Processor -- start 2024-04-02T13:55:00.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342b8d0 0x560e734346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e73434dd0 0x560e73431f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560e73437b60 con 0x560e7342c1f0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560e7334ad30 con 0x560e7342b8d0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560e73437ce0 con 0x560e73434dd0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 10 client.0 _ll_register_callbacks cb 0x560e7334a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51196/0 (socket says 172.21.15.122:51196) 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 -- 172.21.15.122:0/132688127 learned_addr learned my addr 172.21.15.122:0/132688127 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02e256640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342b8d0 0x560e734346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 -- 172.21.15.122:0/132688127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e73434dd0 msgr2=0x560e73431f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e73434dd0 0x560e73431f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 -- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342b8d0 msgr2=0x560e734346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342b8d0 0x560e734346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 -- 172.21.15.122:0/132688127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e730a33f0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff02da55640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff024005ab0 tx=0x7ff02402ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5649b816aee30 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024005030 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff0240051d0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.594+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff02000dbe0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff02404d050 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 10 client.21030 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x560e730630f0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff02000dbe0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7ff0240035f0 con 0x560e7342c1f0 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 10 client.21030 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 10 client.21030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:00.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 20 client.21030 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e73444e00 0x560e734472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 4 client.21030 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e7339cd70 con 0x560e73444e00 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.595+0000 7ff02e256640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e73444e00 0x560e734472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff011ffb640 10 client.21030 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff02e256640 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e73444e00 0x560e734472a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fb74586f3cbda3 server_cookie=d9f60ee26ac1a509 in_seq=0 out_seq=0 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff011ffb640 1 -- 172.21.15.122:0/132688127 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e7339cd70 con 0x560e73444e00 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff011ffb640 10 client.21030 handle_command_reply: tid=0 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff03317c740 1 client.21030 shutdown 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 msgr2=0x560e73431a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.596+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7ff024005ab0 tx=0x7ff02402ce10 comp rx=0 tx=0).stop 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 shutdown_connections 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e73444e00 0x560e734472a0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e73434dd0 0x560e73431f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff011ffb640 0 client.21030 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73431a40 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 --2- 172.21.15.122:0/132688127 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e7342b8d0 0x560e734346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 >> 172.21.15.122:0/132688127 conn(0x560e73426000 msgr2=0x560e73427d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 shutdown_connections 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 1 -- 172.21.15.122:0/132688127 wait complete. 2024-04-02T13:55:00.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.597+0000 7ff03317c740 20 client.21030 trim_cache size 0 max 16384 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 Processor -- start 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 -- start start 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e73440aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e734346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281af060 con 0x560e7342c1f0 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b0bd0 con 0x560e73434dd0 2024-04-02T13:55:00.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff03317c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281af730 con 0x560e7342b8d0 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51208/0 (socket says 172.21.15.122:51208) 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02ea57640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e734346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 -- 172.21.15.122:0/2427561268 learned_addr learned my addr 172.21.15.122:0/2427561268 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02e256640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e73440aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 -- 172.21.15.122:0/2427561268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 msgr2=0x560e73440aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e73440aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 -- 172.21.15.122:0/2427561268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 msgr2=0x560e734346b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e734346b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 -- 172.21.15.122:0/2427561268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e73425e30 con 0x560e7342c1f0 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02ea57640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e734346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:00.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02e256640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e73440aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.615+0000 7ff02da55640 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7ff024003bb0 tx=0x7ff02402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842163d847c0b783 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff013fff640 1 -- 172.21.15.122:0/2427561268 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024005d50 con 0x560e7342c1f0 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff013fff640 1 -- 172.21.15.122:0/2427561268 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024009520 con 0x560e7342c1f0 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff013fff640 1 -- 172.21.15.122:0/2427561268 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024003e90 con 0x560e7342c1f0 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 -- 172.21.15.122:0/2427561268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 msgr2=0x560e73434170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7ff024003bb0 tx=0x7ff02402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 -- 172.21.15.122:0/2427561268 shutdown_connections 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e734346b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e73434170 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 --2- 172.21.15.122:0/2427561268 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e73440aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 -- 172.21.15.122:0/2427561268 >> 172.21.15.122:0/2427561268 conn(0x560e73426000 msgr2=0x560e73425240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 -- 172.21.15.122:0/2427561268 shutdown_connections 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.616+0000 7ff03317c740 1 -- 172.21.15.122:0/2427561268 wait complete. 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 -- start start 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 Processor -- start 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7343d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e7342e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281b0500 con 0x560e7342c1f0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281afe00 con 0x560e73434dd0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0281ad060 con 0x560e7342b8d0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff03317c740 10 client.0 _ll_register_callbacks cb 0x7ff02402ccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51218/0 (socket says 172.21.15.122:51218) 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 -- 172.21.15.122:0/4171274889 learned_addr learned my addr 172.21.15.122:0/4171274889 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02e256640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7343d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 -- 172.21.15.122:0/4171274889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 msgr2=0x560e7343d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7343d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02ea57640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e7342e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 -- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 msgr2=0x560e7342e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e7342e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02da55640 1 -- 172.21.15.122:0/4171274889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e7342f110 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.617+0000 7ff02e256640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7343d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff02da55640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff02402e000 tx=0x7ff024004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb02868ed9c2ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024060060 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff0240035f0 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff024009520 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff024067050 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 10 client.21036 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x560e734ab380 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024009520 con 0x560e7342c1f0 2024-04-02T13:55:00.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7ff024034b30 con 0x560e7342c1f0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 10 client.21036 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 10 client.21036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 20 client.21036 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e734ab9e0 0x560e734ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 4 client.21036 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e733970c0 con 0x560e734ab9e0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.618+0000 7ff02e256640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e734ab9e0 0x560e734ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff02e256640 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e734ab9e0 0x560e734ade00 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446a1a0591fda2dd server_cookie=e009eea0d7fed3d6 in_seq=0 out_seq=0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff011ffb640 10 client.21036 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff011ffb640 1 -- 172.21.15.122:0/4171274889 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e733970c0 con 0x560e734ab9e0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff011ffb640 10 client.21036 handle_command_reply: tid=0 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 client.21036 shutdown 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 msgr2=0x560e7342e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7ff02402e000 tx=0x7ff024004540 comp rx=0 tx=0).stop 2024-04-02T13:55:00.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 shutdown_connections 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560e734ab9e0 0x560e734ade00 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560e73434dd0 0x560e7342e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560e7342c1f0 0x560e7342e3a0 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 --2- 172.21.15.122:0/4171274889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560e7342b8d0 0x560e7343d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff011ffb640 0 client.21036 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 >> 172.21.15.122:0/4171274889 conn(0x560e73426000 msgr2=0x560e73427430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 shutdown_connections 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 1 -- 172.21.15.122:0/4171274889 wait complete. 2024-04-02T13:55:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.619+0000 7ff03317c740 20 client.21036 trim_cache size 0 max 16384 2024-04-02T13:55:00.631 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:00.631 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:55:00.631 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:55:00.631 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 msgr2=0x7ff0140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 0x7ff0140a6260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff020012c80 tx=0x7ff020009e90 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 msgr2=0x7ff0281983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e9f0 tx=0x7ff01800edf0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 shutdown_connections 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0140a3da0 0x7ff0140a6260 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff028107aa0 0x7ff02819cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff028102450 0x7ff028198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 --2- 172.21.15.122:0/2432735746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff028101a90 0x7ff0281983d0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 >> 172.21.15.122:0/2432735746 conn(0x7ff028078f00 msgr2=0x7ff0281044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 shutdown_connections 2024-04-02T13:55:00.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.620+0000 7ff0304ba640 1 -- 172.21.15.122:0/2432735746 wait complete. 2024-04-02T13:55:00.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:55:00.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:55:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 -- 172.21.15.122:0/3369255506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc073ef0 msgr2=0x7f3dfc0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 --2- 172.21.15.122:0/3369255506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc0742d0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009900 tx=0x7f3df802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 -- 172.21.15.122:0/3369255506 shutdown_connections 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 --2- 172.21.15.122:0/3369255506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc10e340 0x7f3dfc110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 --2- 172.21.15.122:0/3369255506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc10de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 --2- 172.21.15.122:0/3369255506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc0742d0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 -- 172.21.15.122:0/3369255506 >> 172.21.15.122:0/3369255506 conn(0x7f3dfc0fd570 msgr2=0x7f3dfc0ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 -- 172.21.15.122:0/3369255506 shutdown_connections 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.748+0000 7f3e03d00640 1 -- 172.21.15.122:0/3369255506 wait complete. 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 Processor -- start 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 -- start start 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc1140c0 con 0x7f3dfc10e340 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc113f40 con 0x7f3dfc074810 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e03d00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc114240 con 0x7f3dfc073ef0 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51238/0 (socket says 172.21.15.122:51238) 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 -- 172.21.15.122:0/1791945985 learned_addr learned my addr 172.21.15.122:0/1791945985 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e01a9c640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0129b640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 -- 172.21.15.122:0/1791945985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc073ef0 msgr2=0x7f3dfc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 -- 172.21.15.122:0/1791945985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 msgr2=0x7f3dfc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0229d640 1 -- 172.21.15.122:0/1791945985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc19f520 con 0x7f3dfc10e340 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.749+0000 7f3e0129b640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3e0229d640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9d0 tx=0x7f3df400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df400cd80 con 0x7f3dfc10e340 2024-04-02T13:55:00.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc19f810 con 0x7f3dfc10e340 2024-04-02T13:55:00.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3df4005020 con 0x7f3dfc10e340 2024-04-02T13:55:00.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40116d0 con 0x7f3dfc10e340 2024-04-02T13:55:00.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.750+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1a70a0 con 0x7f3dfc10e340 2024-04-02T13:55:00.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.751+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dfc068d40 con 0x7f3dfc10e340 2024-04-02T13:55:00.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.752+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3df40051d0 con 0x7f3dfc10e340 2024-04-02T13:55:00.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.752+0000 7f3df2ffd640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:00.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.752+0000 7f3e01a9c640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:00.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.752+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f3df40d1160 con 0x7f3dfc10e340 2024-04-02T13:55:00.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.753+0000 7f3e01a9c640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 0x7f3de80a61e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df802c7f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:00.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:00.757+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f3df4016030 con 0x7f3dfc10e340 2024-04-02T13:55:01.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.008+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dfc02d370 con 0x7f3dfc10e340 2024-04-02T13:55:01.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.009+0000 7f3df2ffd640 1 -- 172.21.15.122:0/1791945985 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+11943 (secure 0 0 0) 0x7f3df409d410 con 0x7f3dfc10e340 2024-04-02T13:55:01.012 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:55:01.012 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":75,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:54:57.724566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[24485,24487,24460,24512,14628]},"id":1}]} 2024-04-02T13:55:01.012 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 75 2024-04-02T13:55:01.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 msgr2=0x7f3de80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 0x7f3de80a61e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3df8000c00 tx=0x7f3df802c7f0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 msgr2=0x7f3dfc19edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f3df400e9d0 tx=0x7f3df400ee90 comp rx=0 tx=0).stop 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 shutdown_connections 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3de80a3d20 0x7f3de80a61e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dfc10e340 0x7f3dfc19edf0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc074810 0x7f3dfc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 --2- 172.21.15.122:0/1791945985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dfc073ef0 0x7f3dfc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 >> 172.21.15.122:0/1791945985 conn(0x7f3dfc0fd570 msgr2=0x7f3dfc0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 shutdown_connections 2024-04-02T13:55:01.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.011+0000 7f3e03d00640 1 -- 172.21.15.122:0/1791945985 wait complete. 2024-04-02T13:55:01.025 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 -- 172.21.15.122:0/1688215822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401069b0 msgr2=0x7fbe40106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/1688215822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401069b0 0x7fbe40106d90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009930 tx=0x7fbe3002bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 -- 172.21.15.122:0/1688215822 shutdown_connections 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/1688215822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401013b0 0x7fbe4010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/1688215822 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401009f0 0x7fbe40100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/1688215822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401069b0 0x7fbe40106d90 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 -- 172.21.15.122:0/1688215822 >> 172.21.15.122:0/1688215822 conn(0x7fbe40077b60 msgr2=0x7fbe40077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 -- 172.21.15.122:0/1688215822 shutdown_connections 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.127+0000 7fbe47eb7640 1 -- 172.21.15.122:0/1688215822 wait complete. 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 Processor -- start 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 -- start start 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 0x7fbe40198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401069b0 0x7fbe4019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40110de0 con 0x7fbe401009f0 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40110c60 con 0x7fbe401013b0 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe47eb7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe40110f60 con 0x7fbe401069b0 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51264/0 (socket says 172.21.15.122:51264) 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 -- 172.21.15.122:0/3304093335 learned_addr learned my addr 172.21.15.122:0/3304093335 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe46454640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401069b0 0x7fbe4019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45452640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 0x7fbe40198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 -- 172.21.15.122:0/3304093335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401069b0 msgr2=0x7fbe4019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401069b0 0x7fbe4019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 -- 172.21.15.122:0/3304093335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 msgr2=0x7fbe40198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 0x7fbe40198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 -- 172.21.15.122:0/3304093335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4019d420 con 0x7fbe401009f0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45452640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 0x7fbe40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe45c53640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009230 tx=0x7fbe3002cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.128+0000 7fbe36ffd640 1 -- 172.21.15.122:0/3304093335 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30041070 con 0x7fbe401009f0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.129+0000 7fbe36ffd640 1 -- 172.21.15.122:0/3304093335 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe30004450 con 0x7fbe401009f0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.129+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4019d6b0 con 0x7fbe401009f0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.129+0000 7fbe36ffd640 1 -- 172.21.15.122:0/3304093335 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30034440 con 0x7fbe401009f0 2024-04-02T13:55:01.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.129+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe401a4fa0 con 0x7fbe401009f0 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe36ffd640 1 -- 172.21.15.122:0/3304093335 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbe30044070 con 0x7fbe401009f0 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 Processor -- start 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 -- start start 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ed7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85ee0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb85e3b7b0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eb85dda570 con 0x55eb85ed80c0 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.131+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb85e37760 con 0x55eb85ed77a0 2024-04-02T13:55:01.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51278/0 (socket says 172.21.15.122:51278) 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 -- 172.21.15.122:0/3639177686 learned_addr learned my addr 172.21.15.122:0/3639177686 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe46454640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ed7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe36ffd640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 0x7fbe2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45c53640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85ee0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe46454640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 0x7fbe2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 -- 172.21.15.122:0/3639177686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 msgr2=0x55eb85ed7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ed7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 -- 172.21.15.122:0/3639177686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 msgr2=0x55eb85ee0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85ee0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 -- 172.21.15.122:0/3639177686 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb85b4f3f0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45c53640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85ee0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe36ffd640 1 -- 172.21.15.122:0/3304093335 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fbe300f2560 con 0x7fbe401009f0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe45452640 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007800 tx=0x7fbe3c007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ab73f141e47d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe46454640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 0x7fbe2c0a61e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098b0 tx=0x7fbe38004ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe2bfff640 1 -- 172.21.15.122:0/3639177686 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00d950 con 0x55eb85ee0ca0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe2bfff640 1 -- 172.21.15.122:0/3639177686 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3c018070 con 0x55eb85ee0ca0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.132+0000 7fbe2bfff640 1 -- 172.21.15.122:0/3639177686 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0123e0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/3639177686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 msgr2=0x55eb85ee3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007800 tx=0x7fbe3c007cc0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/3639177686 shutdown_connections 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee3090 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85ee0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/3639177686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ed7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/3639177686 >> 172.21.15.122:0/3639177686 conn(0x55eb85ed1ed0 msgr2=0x55eb85ed22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/3639177686 shutdown_connections 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/3639177686 wait complete. 2024-04-02T13:55:01.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.133+0000 7fbe4ab7a740 1 -- start start 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 Processor -- start 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ee0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ee0ca0 0x55eb85ede010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eb85ee3a90 con 0x55eb85ed80c0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eb85df6e80 con 0x55eb85ee0ca0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb85ee3c10 con 0x55eb85ed77a0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe4ab7a740 10 client.0 _ll_register_callbacks cb 0x55eb85e32aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51280/0 (socket says 172.21.15.122:51280) 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 -- 172.21.15.122:0/2002709269 learned_addr learned my addr 172.21.15.122:0/2002709269 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45c53640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ee0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe46454640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ee0ca0 0x55eb85ede010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 -- 172.21.15.122:0/2002709269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 msgr2=0x55eb85ee0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ee0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 -- 172.21.15.122:0/2002709269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ee0ca0 msgr2=0x55eb85ede010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ee0ca0 0x55eb85ede010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 -- 172.21.15.122:0/2002709269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb85c197d0 con 0x55eb85ed80c0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45c53640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ee0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe45452640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007370 tx=0x7fbe3c0136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833c774de088f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.134+0000 7fbe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00dd60 con 0x55eb85ed80c0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fbe3c01c3f0 con 0x55eb85ed80c0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe3c018070 con 0x55eb85ed80c0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbe3c01c630 con 0x55eb85ed80c0 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 10 client.21054 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55eb85eefa60 con 0x55eb85ed80c0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3c018070 con 0x55eb85ed80c0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fbe3c024800 con 0x55eb85ed80c0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 10 client.21054 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 10 client.21054 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 20 client.21054 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85ef0f20 0x55eb85ef3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 4 client.21054 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb85e48cf0 con 0x55eb85ef0f20 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.135+0000 7fbe45c53640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85ef0f20 0x55eb85ef3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe45c53640 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85ef0f20 0x55eb85ef3310 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97ee4850d9e4c1cd server_cookie=20c9b962342d6142 in_seq=0 out_seq=0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe297fa640 10 client.21054 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe297fa640 1 -- 172.21.15.122:0/2002709269 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb85e48cf0 con 0x55eb85ef0f20 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe297fa640 10 client.21054 handle_command_reply: tid=0 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 client.21054 shutdown 2024-04-02T13:55:01.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 msgr2=0x55eb85edda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007370 tx=0x7fbe3c0136b0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 shutdown_connections 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85ef0f20 0x55eb85ef3310 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ee0ca0 0x55eb85ede010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ed80c0 0x55eb85edda00 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2002709269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85ee0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe297fa640 0 client.21054 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.136+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 >> 172.21.15.122:0/2002709269 conn(0x55eb85ed1ed0 msgr2=0x55eb85ed3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.137+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 shutdown_connections 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.137+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2002709269 wait complete. 2024-04-02T13:55:01.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.137+0000 7fbe4ab7a740 20 client.21054 trim_cache size 0 max 16384 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 Processor -- start 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 -- start start 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85eec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85edd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401af0c0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401b0c30 con 0x55eb85ed80c0 2024-04-02T13:55:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401af790 con 0x55eb85ed77a0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51284/0 (socket says 172.21.15.122:51284) 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 -- 172.21.15.122:0/2478337705 learned_addr learned my addr 172.21.15.122:0/2478337705 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe45c53640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85eec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 -- 172.21.15.122:0/2478337705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 msgr2=0x55eb85eec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85eec950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe45452640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85edd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 -- 172.21.15.122:0/2478337705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 msgr2=0x55eb85edd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85edd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 -- 172.21.15.122:0/2478337705 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb85ede380 con 0x55eb85ee0ca0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe45452640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85edd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe46454640 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fbe38003c30 tx=0x7fbe38009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d391260a742e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe2b7fe640 1 -- 172.21.15.122:0/2478337705 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3801a040 con 0x55eb85ee0ca0 2024-04-02T13:55:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe2b7fe640 1 -- 172.21.15.122:0/2478337705 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe38003c60 con 0x55eb85ee0ca0 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.150+0000 7fbe2b7fe640 1 -- 172.21.15.122:0/2478337705 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe380052a0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2478337705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 msgr2=0x55eb85ee0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fbe38003c30 tx=0x7fbe38009ad0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2478337705 shutdown_connections 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee0130 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed80c0 0x55eb85edd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/2478337705 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed77a0 0x55eb85eec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2478337705 >> 172.21.15.122:0/2478337705 conn(0x55eb85ed1ed0 msgr2=0x55eb85ed22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2478337705 shutdown_connections 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/2478337705 wait complete. 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 -- start start 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.151+0000 7fbe4ab7a740 1 Processor -- start 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 0x55eb85ee1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed80c0 0x55eb85ee1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401b0560 con 0x55eb85ee0ca0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401afe60 con 0x55eb85ed77a0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe401ad0c0 con 0x55eb85ed80c0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe4ab7a740 10 client.0 _ll_register_callbacks cb 0x7fbe38009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51294/0 (socket says 172.21.15.122:51294) 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 -- 172.21.15.122:0/1252871107 learned_addr learned my addr 172.21.15.122:0/1252871107 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe45452640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed80c0 0x55eb85ee1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe45c53640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 0x55eb85ee1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 -- 172.21.15.122:0/1252871107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed80c0 msgr2=0x55eb85ee1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed80c0 0x55eb85ee1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 -- 172.21.15.122:0/1252871107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 msgr2=0x55eb85ee1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 0x55eb85ee1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 -- 172.21.15.122:0/1252871107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb85ef3680 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe45c53640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 0x55eb85ee1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe46454640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbe38004820 tx=0x7fbe38004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3617b3fac3f43257 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.152+0000 7fbe297fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38031070 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fbe38023690 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe38003c60 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fbe3801a030 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 10 client.21060 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55eb85ed7ef0 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe38003c60 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fbe38020070 con 0x55eb85ee0ca0 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 10 client.21060 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 10 client.21060 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 20 client.21060 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85f57360 0x55eb85f59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 4 client.21060 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb85e43040 con 0x55eb85f57360 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.153+0000 7fbe45c53640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85f57360 0x55eb85f59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe297fa640 10 client.21060 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe45c53640 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85f57360 0x55eb85f59750 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78c84fb26ab1253c server_cookie=b360cff7e46acae0 in_seq=0 out_seq=0 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe297fa640 1 -- 172.21.15.122:0/1252871107 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb85e43040 con 0x55eb85f57360 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe297fa640 10 client.21060 handle_command_reply: tid=0 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 client.21060 shutdown 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 msgr2=0x55eb85ee1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fbe38004820 tx=0x7fbe38004530 comp rx=0 tx=0).stop 2024-04-02T13:55:01.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 shutdown_connections 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55eb85f57360 0x55eb85f59750 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eb85ee0ca0 0x55eb85ee1ce0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe297fa640 0 client.21060 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb85ed80c0 0x55eb85ee1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 --2- 172.21.15.122:0/1252871107 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eb85ed77a0 0x55eb85ee1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 >> 172.21.15.122:0/1252871107 conn(0x55eb85ed1ed0 msgr2=0x55eb85ed3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 shutdown_connections 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 1 -- 172.21.15.122:0/1252871107 wait complete. 2024-04-02T13:55:01.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.154+0000 7fbe4ab7a740 20 client.21060 trim_cache size 0 max 16384 2024-04-02T13:55:01.165 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:01.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 msgr2=0x7fbe2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 0x7fbe2c0a61e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbe380098b0 tx=0x7fbe38004ab0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 msgr2=0x7fbe40198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009230 tx=0x7fbe3002cc70 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 shutdown_connections 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbe2c0a3d20 0x7fbe2c0a61e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe401069b0 0x7fbe4019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe401013b0 0x7fbe40198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 --2- 172.21.15.122:0/3304093335 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe401009f0 0x7fbe40198430 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 >> 172.21.15.122:0/3304093335 conn(0x7fbe40077b60 msgr2=0x7fbe4006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 shutdown_connections 2024-04-02T13:55:01.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.155+0000 7fbe47eb7640 1 -- 172.21.15.122:0/3304093335 wait complete. 2024-04-02T13:55:01.169 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:55:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- 172.21.15.122:0/524425198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633410e4a0 msgr2=0x7f6334110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 --2- 172.21.15.122:0/524425198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633410e4a0 0x7f6334110890 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f632c009ca0 tx=0x7f632c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- 172.21.15.122:0/524425198 shutdown_connections 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 --2- 172.21.15.122:0/524425198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f633410e4a0 0x7f6334110890 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 --2- 172.21.15.122:0/524425198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341012a0 0x7f633410df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 --2- 172.21.15.122:0/524425198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f63341008f0 0x7f6334100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- 172.21.15.122:0/524425198 >> 172.21.15.122:0/524425198 conn(0x7f6334077b60 msgr2=0x7f6334077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- 172.21.15.122:0/524425198 shutdown_connections 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- 172.21.15.122:0/524425198 wait complete. 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 Processor -- start 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 -- start start 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.270+0000 7f63399ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 0x7f633419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 0x7f633419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63341140e0 con 0x7f63341012a0 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6334113f60 con 0x7f633410e4a0 2024-04-02T13:55:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6334114260 con 0x7f63341008f0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51306/0 (socket says 172.21.15.122:51306) 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 -- 172.21.15.122:0/4083517891 learned_addr learned my addr 172.21.15.122:0/4083517891 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63337fe640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 0x7f633419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 -- 172.21.15.122:0/4083517891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 msgr2=0x7f633419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f6332ffd640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 0x7f633419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 0x7f633419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 -- 172.21.15.122:0/4083517891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 msgr2=0x7f633419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 0x7f633419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 -- 172.21.15.122:0/4083517891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f633419f5a0 con 0x7f63341012a0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63337fe640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 0x7f633419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f6332ffd640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 0x7f633419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f632bfff640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d950 tx=0x7f631c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63389b8640 1 -- 172.21.15.122:0/4083517891 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c017070 con 0x7f63341012a0 2024-04-02T13:55:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63389b8640 1 -- 172.21.15.122:0/4083517891 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c009c30 con 0x7f63341012a0 2024-04-02T13:55:01.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f633419f890 con 0x7f63341012a0 2024-04-02T13:55:01.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63389b8640 1 -- 172.21.15.122:0/4083517891 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c00f6b0 con 0x7f63341012a0 2024-04-02T13:55:01.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.271+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63341a7120 con 0x7f63341012a0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.273+0000 7f63389b8640 1 -- 172.21.15.122:0/4083517891 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f631c0109d0 con 0x7f63341012a0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 Processor -- start 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 -- start start 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b799ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b799b0c0 0x5590b79a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5590b78fe7b0 con 0x5590b79a3ca0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5590b789d570 con 0x5590b799a7a0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f633c5a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5590b78fa760 con 0x5590b799b0c0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51316/0 (socket says 172.21.15.122:51316) 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 -- 172.21.15.122:0/1655383129 learned_addr learned my addr 172.21.15.122:0/1655383129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f63337fe640 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b799ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f63389b8640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 0x7f63200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f6332ffd640 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b799b0c0 0x5590b79a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 -- 172.21.15.122:0/1655383129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b799b0c0 msgr2=0x5590b79a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b799b0c0 0x5590b79a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 -- 172.21.15.122:0/1655383129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 msgr2=0x5590b799ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b799ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f63337fe640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 0x7f63200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 -- 172.21.15.122:0/1655383129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b76123f0 con 0x5590b79a3ca0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f63389b8640 1 -- 172.21.15.122:0/4083517891 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f631c0d0850 con 0x7f63341012a0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.274+0000 7f632bfff640 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f631c09c100 tx=0x7f631c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55676ab29b7f8d4c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f63337fe640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 0x7f63200a61e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f632c02c630 tx=0x7f632c02c0f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f6329ffb640 1 -- 172.21.15.122:0/1655383129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c09cb00 con 0x5590b79a3ca0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f6329ffb640 1 -- 172.21.15.122:0/1655383129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c009de0 con 0x5590b79a3ca0 2024-04-02T13:55:01.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f6329ffb640 1 -- 172.21.15.122:0/1655383129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c0cde30 con 0x5590b79a3ca0 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 -- 172.21.15.122:0/1655383129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 msgr2=0x5590b79a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f631c09c100 tx=0x7f631c022000 comp rx=0 tx=0).stop 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 -- 172.21.15.122:0/1655383129 shutdown_connections 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b79a3ca0 0x5590b79a6090 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b799b0c0 0x5590b79a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1655383129 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b799ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 -- 172.21.15.122:0/1655383129 >> 172.21.15.122:0/1655383129 conn(0x5590b7994ed0 msgr2=0x5590b79952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 -- 172.21.15.122:0/1655383129 shutdown_connections 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.275+0000 7f633c5a5740 1 -- 172.21.15.122:0/1655383129 wait complete. 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 -- start start 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 Processor -- start 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5590b79a6a30 con 0x5590b799a7a0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5590b78b9e80 con 0x5590b799b0c0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5590b79a6bb0 con 0x5590b79a3ca0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f633c5a5740 10 client.0 _ll_register_callbacks cb 0x5590b78f5aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51332/0 (socket says 172.21.15.122:51332) 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 -- 172.21.15.122:0/1646415350 learned_addr learned my addr 172.21.15.122:0/1646415350 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f63337fe640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 -- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 msgr2=0x5590b79a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f632bfff640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 -- 172.21.15.122:0/1646415350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 msgr2=0x5590b79a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f6332ffd640 1 -- 172.21.15.122:0/1646415350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b76dc7d0 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.276+0000 7f632bfff640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6332ffd640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6324011660 tx=0x7f6324011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892ac57a096cdb82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324004220 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f63240043c0 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631c009de0 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f6324025050 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 10 client.21072 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5590b79b2b50 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c009de0 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f6324003ae0 con 0x5590b799a7a0 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 10 client.21072 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 10 client.21072 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 20 client.21072 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b79b4010 0x5590b79b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 4 client.21072 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.277+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590b790bcf0 con 0x5590b79b4010 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.278+0000 7f632bfff640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b79b4010 0x5590b79b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.278+0000 7f6312ffd640 10 client.21072 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.278+0000 7f632bfff640 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b79b4010 0x5590b79b64d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97ea5631d3dae7d3 server_cookie=49dafc87e887e9ab in_seq=0 out_seq=0 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f6312ffd640 1 -- 172.21.15.122:0/1646415350 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590b790bcf0 con 0x5590b79b4010 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f6312ffd640 10 client.21072 handle_command_reply: tid=0 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 client.21072 shutdown 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 msgr2=0x5590b79a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f6324011660 tx=0x7f6324011b20 comp rx=0 tx=0).stop 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 shutdown_connections 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b79b4010 0x5590b79b64d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 --2- 172.21.15.122:0/1646415350 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79a3710 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f6312ffd640 0 client.21072 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 >> 172.21.15.122:0/1646415350 conn(0x5590b7994ed0 msgr2=0x5590b7996c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 shutdown_connections 2024-04-02T13:55:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 1 -- 172.21.15.122:0/1646415350 wait complete. 2024-04-02T13:55:01.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.279+0000 7f633c5a5740 20 client.21072 trim_cache size 0 max 16384 2024-04-02T13:55:01.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 Processor -- start 2024-04-02T13:55:01.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 -- start start 2024-04-02T13:55:01.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b79af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63341b1380 con 0x5590b799b0c0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63341b2ef0 con 0x5590b799a7a0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f633c5a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63341b1a50 con 0x5590b79a3ca0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51336/0 (socket says 172.21.15.122:51336) 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 -- 172.21.15.122:0/912946604 learned_addr learned my addr 172.21.15.122:0/912946604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f6332ffd640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b79af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f63337fe640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 -- 172.21.15.122:0/912946604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 msgr2=0x5590b79a31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 -- 172.21.15.122:0/912946604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 msgr2=0x5590b79af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b79af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f632bfff640 1 -- 172.21.15.122:0/912946604 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590b799aef0 con 0x5590b799b0c0 2024-04-02T13:55:01.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.292+0000 7f63337fe640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f632bfff640 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ce1b0 tx=0x7f631c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd014e44cd71c49a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f63297fa640 1 -- 172.21.15.122:0/912946604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c09cb00 con 0x5590b799b0c0 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f63297fa640 1 -- 172.21.15.122:0/912946604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c009de0 con 0x5590b799b0c0 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f63297fa640 1 -- 172.21.15.122:0/912946604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c098a70 con 0x5590b799b0c0 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 -- 172.21.15.122:0/912946604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 msgr2=0x5590b7993db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f631c0ce1b0 tx=0x7f631c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 -- 172.21.15.122:0/912946604 shutdown_connections 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799b0c0 0x5590b7993db0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 --2- 172.21.15.122:0/912946604 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799a7a0 0x5590b79af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 -- 172.21.15.122:0/912946604 >> 172.21.15.122:0/912946604 conn(0x5590b7994ed0 msgr2=0x5590b79936a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 -- 172.21.15.122:0/912946604 shutdown_connections 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.293+0000 7f633c5a5740 1 -- 172.21.15.122:0/912946604 wait complete. 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 -- start start 2024-04-02T13:55:01.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 Processor -- start 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f63341b2820 con 0x5590b799a7a0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f63341b2120 con 0x5590b799b0c0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63341af380 con 0x5590b79a3ca0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f633c5a5740 10 client.0 _ll_register_callbacks cb 0x7f6324009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51348/0 (socket says 172.21.15.122:51348) 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 -- 172.21.15.122:0/705769380 learned_addr learned my addr 172.21.15.122:0/705769380 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f632bfff640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 -- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 msgr2=0x5590b79a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 -- 172.21.15.122:0/705769380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 msgr2=0x5590b79a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f6332ffd640 1 -- 172.21.15.122:0/705769380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590b79a52c0 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.294+0000 7f632bfff640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6332ffd640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x5590b79948b0 tx=0x7f6324005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce90dcad56cff707 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632400fd50 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f6324005090 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631c009de0 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f632403b050 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 10 client.21078 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5590b7a1a660 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f631c009de0 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f6324012480 con 0x5590b799a7a0 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 10 client.21078 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 10 client.21078 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 20 client.21078 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b7a1ac50 0x5590b7a1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 4 client.21078 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.295+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590b7906040 con 0x5590b7a1ac50 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f632bfff640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b7a1ac50 0x5590b7a1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f6312ffd640 10 client.21078 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f632bfff640 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b7a1ac50 0x5590b7a1d090 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=36ae7a2516f0f915 server_cookie=fac1012abf672ac7 in_seq=0 out_seq=0 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f6312ffd640 1 -- 172.21.15.122:0/705769380 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590b7906040 con 0x5590b7a1ac50 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f6312ffd640 10 client.21078 handle_command_reply: tid=0 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 client.21078 shutdown 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 msgr2=0x5590b79ac0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x5590b79948b0 tx=0x7f6324005cd0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 shutdown_connections 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5590b79a3ca0 0x5590b79a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5590b799b0c0 0x5590b79a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x5590b7a1ac50 0x5590b7a1d090 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 --2- 172.21.15.122:0/705769380 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5590b799a7a0 0x5590b79ac0a0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f6312ffd640 0 client.21078 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 >> 172.21.15.122:0/705769380 conn(0x5590b7994ed0 msgr2=0x5590b799fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 shutdown_connections 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 1 -- 172.21.15.122:0/705769380 wait complete. 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.296+0000 7f633c5a5740 20 client.21078 trim_cache size 0 max 16384 2024-04-02T13:55:01.306 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 msgr2=0x7f63200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 0x7f63200a61e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f632c02c630 tx=0x7f632c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 msgr2=0x7f633419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d950 tx=0x7f631c00de10 comp rx=0 tx=0).stop 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 shutdown_connections 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f63200a3d20 0x7f63200a61e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f633410e4a0 0x7f633419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f63341012a0 0x7f633419ab40 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 --2- 172.21.15.122:0/4083517891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63341008f0 0x7f633419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 >> 172.21.15.122:0/4083517891 conn(0x7f6334077b60 msgr2=0x7f633410f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 shutdown_connections 2024-04-02T13:55:01.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.297+0000 7f63399ba640 1 -- 172.21.15.122:0/4083517891 wait complete. 2024-04-02T13:55:01.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:55:01.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197080 27.14 MB/sec execute 261 sec latency 1188.464 ms 2024-04-02T13:55:01.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 -- 172.21.15.122:0/3303352771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 msgr2=0x7fcf30106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 --2- 172.21.15.122:0/3303352771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30106d60 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009930 tx=0x7fcf2c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:01.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 -- 172.21.15.122:0/3303352771 shutdown_connections 2024-04-02T13:55:01.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 --2- 172.21.15.122:0/3303352771 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30101330 0x7fcf3010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.416 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 --2- 172.21.15.122:0/3303352771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30100970 0x7fcf30100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 --2- 172.21.15.122:0/3303352771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30106d60 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 -- 172.21.15.122:0/3303352771 >> 172.21.15.122:0/3303352771 conn(0x7fcf30077b00 msgr2=0x7fcf30077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 -- 172.21.15.122:0/3303352771 shutdown_connections 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.411+0000 7fcf384f2640 1 -- 172.21.15.122:0/3303352771 wait complete. 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 Processor -- start 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 -- start start 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 0x7fcf30103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 0x7fcf30103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30110d60 con 0x7fcf30106980 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30110be0 con 0x7fcf30100970 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf384f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30110ee0 con 0x7fcf30101330 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51364/0 (socket says 172.21.15.122:51364) 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 -- 172.21.15.122:0/594501442 learned_addr learned my addr 172.21.15.122:0/594501442 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf3628e640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 0x7fcf30103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 0x7fcf30103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 -- 172.21.15.122:0/594501442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 msgr2=0x7fcf30103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 0x7fcf30103540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 -- 172.21.15.122:0/594501442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 msgr2=0x7fcf30103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 0x7fcf30103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf36a8f640 1 -- 172.21.15.122:0/594501442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf301a6890 con 0x7fcf30106980 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf3628e640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 0x7fcf30103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.412+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 0x7fcf30103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf36a8f640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800eaa0 tx=0x7fcf2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf277fe640 1 -- 172.21.15.122:0/594501442 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800cdb0 con 0x7fcf30106980 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf277fe640 1 -- 172.21.15.122:0/594501442 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28005020 con 0x7fcf30106980 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf277fe640 1 -- 172.21.15.122:0/594501442 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280116b0 con 0x7fcf30106980 2024-04-02T13:55:01.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf301a6b80 con 0x7fcf30106980 2024-04-02T13:55:01.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.413+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a70d0 con 0x7fcf30106980 2024-04-02T13:55:01.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.415+0000 7fcf277fe640 1 -- 172.21.15.122:0/594501442 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcf28005560 con 0x7fcf30106980 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.415+0000 7fcf3b1ba740 1 Processor -- start 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.415+0000 7fcf3b1ba740 1 -- start start 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef91b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a51eef57b0 con 0x55a51ef920c0 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a51ee94570 con 0x55a51ef917a0 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a51eef1760 con 0x55a51ef9aca0 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf277fe640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 0x7fcf1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51376/0 (socket says 172.21.15.122:51376) 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 -- 172.21.15.122:0/4111084373 learned_addr learned my addr 172.21.15.122:0/4111084373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf36a8f640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 0x7fcf1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef91b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf277fe640 1 -- 172.21.15.122:0/594501442 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fcf280d0e50 con 0x7fcf30106980 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 -- 172.21.15.122:0/4111084373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 msgr2=0x55a51ef9d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 -- 172.21.15.122:0/4111084373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 msgr2=0x55a51ef91b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef91b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 -- 172.21.15.122:0/4111084373 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51ec093f0 con 0x55a51ef920c0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef91b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e890d19958cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf36a8f640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 0x7fcf1c0a6260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcf2801c000 tx=0x7fcf2806f4f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.416+0000 7fcf24bfe640 1 -- 172.21.15.122:0/4111084373 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c02cb00 con 0x55a51ef920c0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf24bfe640 1 -- 172.21.15.122:0/4111084373 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c02cca0 con 0x55a51ef920c0 2024-04-02T13:55:01.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf24bfe640 1 -- 172.21.15.122:0/4111084373 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004920 con 0x55a51ef920c0 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4111084373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 msgr2=0x55a51ef9a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c02c650 comp rx=0 tx=0).stop 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4111084373 shutdown_connections 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9a760 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4111084373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef91b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4111084373 >> 172.21.15.122:0/4111084373 conn(0x55a51ef8bed0 msgr2=0x55a51ef8c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4111084373 shutdown_connections 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.417+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4111084373 wait complete. 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 -- start start 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 Processor -- start 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55a51ef9da30 con 0x55a51ef9aca0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55a51eeb0e80 con 0x55a51ef917a0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a51ef9dbb0 con 0x55a51ef920c0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3b1ba740 10 client.0 _ll_register_callbacks cb 0x55a51eeecaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51390/0 (socket says 172.21.15.122:51390) 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 -- 172.21.15.122:0/4171259151 learned_addr learned my addr 172.21.15.122:0/4171259151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 -- 172.21.15.122:0/4171259151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 msgr2=0x55a51ef97a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef97a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 -- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 msgr2=0x55a51ef9a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef9a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 -- 172.21.15.122:0/4171259151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51ecd37d0 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.418+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fcf20013e50 tx=0x7fcf20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6736e15b6b30a911 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000be20 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fcf20005020 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c02cca0 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fcf20005220 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 10 client.21090 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55a51efa9ae0 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c02cca0 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fcf2001d030 con 0x55a51ef9aca0 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 10 client.21090 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 10 client.21090 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 20 client.21090 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51efaafa0 0x55a51efad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 4 client.21090 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.419+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a51ef02cf0 con 0x55a51efaafa0 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.420+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51efaafa0 0x55a51efad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.420+0000 7fcf19ffb640 10 client.21090 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.420+0000 7fcf3628e640 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51efaafa0 0x55a51efad3b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed9841c197ad4fa server_cookie=e79c2cd65e1a3791 in_seq=0 out_seq=0 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf19ffb640 1 -- 172.21.15.122:0/4171259151 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a51ef02cf0 con 0x55a51efaafa0 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf19ffb640 10 client.21090 handle_command_reply: tid=0 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 client.21090 shutdown 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 msgr2=0x55a51ef98020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fcf20013e50 tx=0x7fcf20004730 comp rx=0 tx=0).stop 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 shutdown_connections 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef98020 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51efaafa0 0x55a51efad3b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/4171259151 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51ef9a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 >> 172.21.15.122:0/4171259151 conn(0x55a51ef8bed0 msgr2=0x55a51ef8dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf19ffb640 0 client.21090 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 shutdown_connections 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/4171259151 wait complete. 2024-04-02T13:55:01.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.421+0000 7fcf3b1ba740 20 client.21090 trim_cache size 0 max 16384 2024-04-02T13:55:01.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 Processor -- start 2024-04-02T13:55:01.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 -- start start 2024-04-02T13:55:01.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301b1340 con 0x55a51ef9aca0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301b2eb0 con 0x55a51ef917a0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301b1a10 con 0x55a51ef920c0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf35a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf35a8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51400/0 (socket says 172.21.15.122:51400) 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf35a8d640 1 -- 172.21.15.122:0/3662518721 learned_addr learned my addr 172.21.15.122:0/3662518721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.434+0000 7fcf3628e640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 -- 172.21.15.122:0/3662518721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 msgr2=0x55a51ef9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 -- 172.21.15.122:0/3662518721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 msgr2=0x55a51efa6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 -- 172.21.15.122:0/3662518721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51ef91ef0 con 0x55a51ef9aca0 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3628e640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000b440 tx=0x7fcf200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98778e7140a86fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf1bfff640 1 -- 172.21.15.122:0/3662518721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20011240 con 0x55a51ef9aca0 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf1bfff640 1 -- 172.21.15.122:0/3662518721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2001e3f0 con 0x55a51ef9aca0 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf1bfff640 1 -- 172.21.15.122:0/3662518721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20005620 con 0x55a51ef9aca0 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/3662518721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 msgr2=0x55a51ef9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000b440 tx=0x7fcf200046c0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/3662518721 shutdown_connections 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef9aca0 0x55a51ef9a680 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef920c0 0x55a51ef9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/3662518721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.435+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/3662518721 >> 172.21.15.122:0/3662518721 conn(0x55a51ef8bed0 msgr2=0x55a51ef979d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/3662518721 shutdown_connections 2024-04-02T13:55:01.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/3662518721 wait complete. 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- start start 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 Processor -- start 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301b27e0 con 0x55a51ef920c0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301b20e0 con 0x55a51ef917a0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf301af340 con 0x55a51ef9aca0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3b1ba740 10 client.0 _ll_register_callbacks cb 0x7fcf2000b650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51412/0 (socket says 172.21.15.122:51412) 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 -- 172.21.15.122:0/1964705882 learned_addr learned my addr 172.21.15.122:0/1964705882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 -- 172.21.15.122:0/1964705882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 msgr2=0x55a51ef9c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 -- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 msgr2=0x55a51efa3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf3628e640 1 -- 172.21.15.122:0/1964705882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51ef94e40 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.436+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3628e640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec094bd1fc8024f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c052060 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fcf2c04ddc0 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a51eefd040 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c059050 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 10 client.21096 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55a51ef9b480 con 0x55a51ef920c0 2024-04-02T13:55:01.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a51eefd040 con 0x55a51ef920c0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fcf2c005040 con 0x55a51ef920c0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 10 client.21096 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 10 client.21096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 20 client.21096 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51ef9b7c0 0x55a51f013500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 4 client.21096 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.437+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a51ee7a6b0 con 0x55a51ef9b7c0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51ef9b7c0 0x55a51f013500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf19ffb640 10 client.21096 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf35a8d640 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51ef9b7c0 0x55a51f013500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a20eda8bf223423e server_cookie=116f7bbf968d1764 in_seq=0 out_seq=0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf19ffb640 1 -- 172.21.15.122:0/1964705882 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a51ee7a6b0 con 0x55a51ef9b7c0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf19ffb640 10 client.21096 handle_command_reply: tid=0 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 client.21096 shutdown 2024-04-02T13:55:01.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 msgr2=0x55a51ef9c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c006010 tx=0x7fcf2c005bd0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 shutdown_connections 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a51ef9aca0 0x55a51ef9c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a51ef920c0 0x55a51ef9c390 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55a51ef9b7c0 0x55a51f013500 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 --2- 172.21.15.122:0/1964705882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55a51ef917a0 0x55a51efa3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf19ffb640 0 client.21096 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 >> 172.21.15.122:0/1964705882 conn(0x55a51ef8bed0 msgr2=0x55a51ef961b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 shutdown_connections 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 1 -- 172.21.15.122:0/1964705882 wait complete. 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.438+0000 7fcf3b1ba740 20 client.21096 trim_cache size 0 max 16384 2024-04-02T13:55:01.448 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:01.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 msgr2=0x7fcf1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 0x7fcf1c0a6260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcf2801c000 tx=0x7fcf2806f4f0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 msgr2=0x7fcf30103a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800eaa0 tx=0x7fcf2800ef60 comp rx=0 tx=0).stop 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 shutdown_connections 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf1c0a3da0 0x7fcf1c0a6260 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf30106980 0x7fcf30103a80 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf30101330 0x7fcf30103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 --2- 172.21.15.122:0/594501442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf30100970 0x7fcf30103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 >> 172.21.15.122:0/594501442 conn(0x7fcf30077b00 msgr2=0x7fcf30068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 shutdown_connections 2024-04-02T13:55:01.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.439+0000 7fcf384f2640 1 -- 172.21.15.122:0/594501442 wait complete. 2024-04-02T13:55:01.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:55:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.660+0000 7fc94a7fc640 1 -- 172.21.15.122:0/2331270885 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944032990 con 0x7fc94c11b2e0 2024-04-02T13:55:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.660+0000 7fc951c93640 1 -- 172.21.15.122:0/2331270885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c11b2e0 msgr2=0x7fc94c11d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.660+0000 7fc951c93640 1 --2- 172.21.15.122:0/2331270885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c11b2e0 0x7fc94c11d6d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc944009ca0 tx=0x7fc94402c100 comp rx=0 tx=0).stop 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 -- 172.21.15.122:0/2331270885 shutdown_connections 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 --2- 172.21.15.122:0/2331270885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c11b2e0 0x7fc94c11d6d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 --2- 172.21.15.122:0/2331270885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c10e350 0x7fc94c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 --2- 172.21.15.122:0/2331270885 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 0x7fc94c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 -- 172.21.15.122:0/2331270885 >> 172.21.15.122:0/2331270885 conn(0x7fc94c06cbe0 msgr2=0x7fc94c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 -- 172.21.15.122:0/2331270885 shutdown_connections 2024-04-02T13:55:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.661+0000 7fc951c93640 1 -- 172.21.15.122:0/2331270885 wait complete. 2024-04-02T13:55:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 Processor -- start 2024-04-02T13:55:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 -- start start 2024-04-02T13:55:01.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c10e350 0x7fc94c11a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 0x7fc94c11aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111090 con 0x7fc94c11b2e0 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c110f10 con 0x7fc94c110320 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.662+0000 7fc951c93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c111210 con 0x7fc94c10e350 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51416/0 (socket says 172.21.15.122:51416) 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 -- 172.21.15.122:0/2350721723 learned_addr learned my addr 172.21.15.122:0/2350721723 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94affd640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 0x7fc94c11aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c10e350 0x7fc94c11a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 -- 172.21.15.122:0/2350721723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c10e350 msgr2=0x7fc94c11a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c10e350 0x7fc94c11a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 -- 172.21.15.122:0/2350721723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 msgr2=0x7fc94c11aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 0x7fc94c11aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.663+0000 7fc94bfff640 1 -- 172.21.15.122:0/2350721723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c114ed0 con 0x7fc94c11b2e0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.664+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc9440348f0 tx=0x7fc944035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.664+0000 7fc948ff9640 1 -- 172.21.15.122:0/2350721723 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944039040 con 0x7fc94c11b2e0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.664+0000 7fc948ff9640 1 -- 172.21.15.122:0/2350721723 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc944044070 con 0x7fc94c11b2e0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.664+0000 7fc948ff9640 1 -- 172.21.15.122:0/2350721723 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440042a0 con 0x7fc94c11b2e0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.664+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c115160 con 0x7fc94c11b2e0 2024-04-02T13:55:01.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.665+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1bc410 con 0x7fc94c11b2e0 2024-04-02T13:55:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc948ff9640 1 -- 172.21.15.122:0/2350721723 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc944003970 con 0x7fc94c11b2e0 2024-04-02T13:55:01.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 Processor -- start 2024-04-02T13:55:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 -- start start 2024-04-02T13:55:01.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc948ff9640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 0x7fc9340a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f0cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f15760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 0x55eed9f18090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eed9e707b0 con 0x55eed9f15ca0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eed9e0f570 con 0x55eed9f0c7a0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc95487e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eed9e6c760 con 0x55eed9f0d0c0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc948ff9640 1 -- 172.21.15.122:0/2350721723 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fc9440f2950 con 0x7fc94c11b2e0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.669+0000 7fc94affd640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 0x7fc9340a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f15760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94affd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f15760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36914/0 (socket says 172.21.15.122:36914) 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94affd640 1 -- 172.21.15.122:0/2756085245 learned_addr learned my addr 172.21.15.122:0/2756085245 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 0x55eed9f18090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f0cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94bfff640 1 -- 172.21.15.122:0/2756085245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 msgr2=0x55eed9f15760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f15760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94bfff640 1 -- 172.21.15.122:0/2756085245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 msgr2=0x55eed9f0cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.670+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f0cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.671+0000 7fc94bfff640 1 -- 172.21.15.122:0/2756085245 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed9b843f0 con 0x55eed9f15ca0 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.671+0000 7fc94affd640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 0x7fc9340a6320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc93c013990 tx=0x7fc93c009b30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.671+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 0x55eed9f18090 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc9440efd60 tx=0x7fc944003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdff2baeff210442 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.671+0000 7fc931ffb640 1 -- 172.21.15.122:0/2756085245 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440becb0 con 0x55eed9f15ca0 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.672+0000 7fc931ffb640 1 -- 172.21.15.122:0/2756085245 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9440be5b0 con 0x55eed9f15ca0 2024-04-02T13:55:01.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc931ffb640 1 -- 172.21.15.122:0/2756085245 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc944004440 con 0x55eed9f15ca0 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 -- 172.21.15.122:0/2756085245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 msgr2=0x55eed9f18090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 0x55eed9f18090 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fc9440efd60 tx=0x7fc944003f90 comp rx=0 tx=0).stop 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 -- 172.21.15.122:0/2756085245 shutdown_connections 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f15ca0 0x55eed9f18090 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f15760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 --2- 172.21.15.122:0/2756085245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f0cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.673+0000 7fc95487e740 1 -- 172.21.15.122:0/2756085245 >> 172.21.15.122:0/2756085245 conn(0x55eed9f06ed0 msgr2=0x55eed9f072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.674+0000 7fc95487e740 1 -- 172.21.15.122:0/2756085245 shutdown_connections 2024-04-02T13:55:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.674+0000 7fc95487e740 1 -- 172.21.15.122:0/2756085245 wait complete. 2024-04-02T13:55:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.674+0000 7fc95487e740 1 -- start start 2024-04-02T13:55:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.674+0000 7fc95487e740 1 Processor -- start 2024-04-02T13:55:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f12a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 0x55eed9f131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55eed9f18850 con 0x55eed9f0d0c0 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55eed9e2be80 con 0x55eed9f0c7a0 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eed9f189d0 con 0x55eed9f15ca0 2024-04-02T13:55:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc95487e740 10 client.0 _ll_register_callbacks cb 0x55eed9e67aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f156d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:34972/0 (socket says 172.21.15.122:34972) 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc94b7fe640 1 -- 172.21.15.122:0/2076031682 learned_addr learned my addr 172.21.15.122:0/2076031682 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.675+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 0x55eed9f131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94affd640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f12a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 -- 172.21.15.122:0/2076031682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 msgr2=0x55eed9f156d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f156d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 -- 172.21.15.122:0/2076031682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 msgr2=0x55eed9f12a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f12a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 -- 172.21.15.122:0/2076031682 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed9c4e7d0 con 0x55eed9f15ca0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc94bfff640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 0x55eed9f131a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9440c73e0 tx=0x7fc9440f01e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f9a96125daa390 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.676+0000 7fc92affd640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440c7220 con 0x55eed9f15ca0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9440be5b0 con 0x55eed9f15ca0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fc9440be810 con 0x55eed9f15ca0 2024-04-02T13:55:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fc9440becb0 con 0x55eed9f15ca0 2024-04-02T13:55:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc95487e740 10 client.29776 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55eed9f18850 con 0x55eed9f15ca0 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9440be5b0 con 0x55eed9f15ca0 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.677+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fc94403c030 con 0x55eed9f15ca0 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 10 client.29776 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 10 client.29776 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 20 client.29776 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f25ce0 0x55eed9f28120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 4 client.29776 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eed9e7dcf0 con 0x55eed9f25ce0 2024-04-02T13:55:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.678+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f25ce0 0x55eed9f28120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.679+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f25ce0 0x55eed9f28120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c5ef5fb4b532bf4 server_cookie=b80c8a449e63cf47 in_seq=0 out_seq=0 2024-04-02T13:55:01.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.679+0000 7fc92affd640 10 client.29776 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc92affd640 1 -- 172.21.15.122:0/2076031682 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eed9e7dcf0 con 0x55eed9f25ce0 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc92affd640 10 client.29776 handle_command_reply: tid=0 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 client.29776 shutdown 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 msgr2=0x55eed9f131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 0x55eed9f131a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9440c73e0 tx=0x7fc9440f01e0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 shutdown_connections 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f25ce0 0x55eed9f28120 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f15ca0 0x55eed9f131a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f12a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 --2- 172.21.15.122:0/2076031682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f0c7a0 0x55eed9f156d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 >> 172.21.15.122:0/2076031682 conn(0x55eed9f06ed0 msgr2=0x55eed9f08c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc92affd640 0 client.29776 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:01.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 shutdown_connections 2024-04-02T13:55:01.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.680+0000 7fc95487e740 1 -- 172.21.15.122:0/2076031682 wait complete. 2024-04-02T13:55:01.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.681+0000 7fc95487e740 20 client.29776 trim_cache size 0 max 16384 2024-04-02T13:55:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.706+0000 7fc95487e740 1 Processor -- start 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.706+0000 7fc95487e740 1 -- start start 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f2da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f2df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c65d0 con 0x55eed9f0c7a0 2024-04-02T13:55:01.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c8140 con 0x55eed9f15ca0 2024-04-02T13:55:01.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc95487e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c6ca0 con 0x55eed9f0d0c0 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51454/0 (socket says 172.21.15.122:51454) 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 -- 172.21.15.122:0/1354261021 learned_addr learned my addr 172.21.15.122:0/1354261021 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94affd640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f2da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94bfff640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f2df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 -- 172.21.15.122:0/1354261021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 msgr2=0x55eed9f2da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f2da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 -- 172.21.15.122:0/1354261021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 msgr2=0x55eed9f2df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.707+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f2df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc94b7fe640 1 -- 172.21.15.122:0/1354261021 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eed9f10fd0 con 0x55eed9f0c7a0 2024-04-02T13:55:01.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc94affd640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f2da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b810 tx=0x7fc94000bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c70f9ba089c895 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc9317fa640 1 -- 172.21.15.122:0/1354261021 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940003d50 con 0x55eed9f0c7a0 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc9317fa640 1 -- 172.21.15.122:0/1354261021 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc940022070 con 0x55eed9f0c7a0 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.708+0000 7fc9317fa640 1 -- 172.21.15.122:0/1354261021 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc940011c00 con 0x55eed9f0c7a0 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.709+0000 7fc95487e740 1 -- 172.21.15.122:0/1354261021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 msgr2=0x55eed9f21710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.709+0000 7fc95487e740 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fc94000b810 tx=0x7fc94000bcf0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.712+0000 7fc95487e740 1 -- 172.21.15.122:0/1354261021 shutdown_connections 2024-04-02T13:55:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.712+0000 7fc95487e740 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f2df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.712+0000 7fc95487e740 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0d0c0 0x55eed9f2da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.712+0000 7fc95487e740 1 --2- 172.21.15.122:0/1354261021 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0c7a0 0x55eed9f21710 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.712+0000 7fc95487e740 1 -- 172.21.15.122:0/1354261021 >> 172.21.15.122:0/1354261021 conn(0x55eed9f06ed0 msgr2=0x55eed9f0dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.715+0000 7fc95487e740 1 -- 172.21.15.122:0/1354261021 shutdown_connections 2024-04-02T13:55:01.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.715+0000 7fc95487e740 1 -- 172.21.15.122:0/1354261021 wait complete. 2024-04-02T13:55:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.715+0000 7fc95487e740 1 -- start start 2024-04-02T13:55:01.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 Processor -- start 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0c7a0 0x55eed9f1dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f0fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c7a70 con 0x55eed9f0d0c0 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c7370 con 0x55eed9f15ca0 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c1c45d0 con 0x55eed9f0c7a0 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc95487e740 10 client.0 _ll_register_callbacks cb 0x7fc93c0134e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc94affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc94affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51462/0 (socket says 172.21.15.122:51462) 2024-04-02T13:55:01.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.716+0000 7fc94affd640 1 -- 172.21.15.122:0/4065388207 learned_addr learned my addr 172.21.15.122:0/4065388207 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0c7a0 0x55eed9f1dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 -- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0c7a0 msgr2=0x55eed9f1dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0c7a0 0x55eed9f1dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 -- 172.21.15.122:0/4065388207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 msgr2=0x55eed9f0fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f0fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 -- 172.21.15.122:0/4065388207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eed9f102c0 con 0x55eed9f0d0c0 2024-04-02T13:55:01.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.717+0000 7fc94affd640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00f790 tx=0x7fc93c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623140cd16674f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.718+0000 7fc92affd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.718+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0044b0 con 0x55eed9f0d0c0 2024-04-02T13:55:01.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.718+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7fc93c01c040 con 0x55eed9f0d0c0 2024-04-02T13:55:01.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.718+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc940022070 con 0x55eed9f0d0c0 2024-04-02T13:55:01.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7fc93c005020 con 0x55eed9f0d0c0 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 10 client.21114 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55eed9f16950 con 0x55eed9f0d0c0 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc940022070 con 0x55eed9f0d0c0 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7fc93c01f040 con 0x55eed9f0d0c0 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 10 client.21114 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 10 client.21114 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 20 client.21114 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f8c2f0 0x55eed9f8e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 4 client.21114 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.719+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eed9e78040 con 0x55eed9f8c2f0 2024-04-02T13:55:01.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.720+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f8c2f0 0x55eed9f8e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.720+0000 7fc92affd640 10 client.21114 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:01.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.721+0000 7fc94b7fe640 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f8c2f0 0x55eed9f8e710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=99fb3cdb12c611cb server_cookie=446eefa00cb1710b in_seq=0 out_seq=0 2024-04-02T13:55:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.722+0000 7fc92affd640 1 -- 172.21.15.122:0/4065388207 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eed9e78040 con 0x55eed9f8c2f0 2024-04-02T13:55:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.722+0000 7fc92affd640 10 client.21114 handle_command_reply: tid=0 2024-04-02T13:55:01.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.722+0000 7fc95487e740 1 client.21114 shutdown 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.722+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 msgr2=0x55eed9f1e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.722+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00f790 tx=0x7fc93c004730 comp rx=0 tx=0).stop 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 shutdown_connections 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55eed9f8c2f0 0x55eed9f8e710 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55eed9f15ca0 0x55eed9f0fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55eed9f0d0c0 0x55eed9f1e160 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 --2- 172.21.15.122:0/4065388207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eed9f0c7a0 0x55eed9f1dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 >> 172.21.15.122:0/4065388207 conn(0x55eed9f06ed0 msgr2=0x55eed9f11350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc92affd640 0 client.21114 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:01.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 shutdown_connections 2024-04-02T13:55:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 1 -- 172.21.15.122:0/4065388207 wait complete. 2024-04-02T13:55:01.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.723+0000 7fc95487e740 20 client.21114 trim_cache size 0 max 16384 2024-04-02T13:55:01.727 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.724+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 msgr2=0x7fc9340a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.724+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 0x7fc9340a6320 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc93c013990 tx=0x7fc93c009b30 comp rx=0 tx=0).stop 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.724+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 msgr2=0x7fc94c114640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.724+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fc9440348f0 tx=0x7fc944035d10 comp rx=0 tx=0).stop 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 shutdown_connections 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc9340a3e60 0x7fc9340a6320 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc94c11b2e0 0x7fc94c114640 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc94c110320 0x7fc94c11aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 --2- 172.21.15.122:0/2350721723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc94c10e350 0x7fc94c11a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 >> 172.21.15.122:0/2350721723 conn(0x7fc94c06cbe0 msgr2=0x7fc94c11c920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 shutdown_connections 2024-04-02T13:55:01.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.725+0000 7fc951c93640 1 -- 172.21.15.122:0/2350721723 wait complete. 2024-04-02T13:55:01.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:55:01.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:01 smithi122 ceph-mon[32114]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 590 op/s 2024-04-02T13:55:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1791945985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:01.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.895+0000 7ff060d85640 1 -- 172.21.15.122:0/1986101512 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058032340 con 0x7ff05c10e350 2024-04-02T13:55:01.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 -- 172.21.15.122:0/1986101512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 msgr2=0x7ff05c10e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 --2- 172.21.15.122:0/1986101512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c10e7d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff058009860 tx=0x7ff05802bb70 comp rx=0 tx=0).stop 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 -- 172.21.15.122:0/1986101512 shutdown_connections 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 --2- 172.21.15.122:0/1986101512 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05c11b2e0 0x7ff05c11d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 --2- 172.21.15.122:0/1986101512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c10e7d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 --2- 172.21.15.122:0/1986101512 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c10de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.896+0000 7ff063feb640 1 -- 172.21.15.122:0/1986101512 >> 172.21.15.122:0/1986101512 conn(0x7ff05c06cbe0 msgr2=0x7ff05c06cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.897+0000 7ff063feb640 1 -- 172.21.15.122:0/1986101512 shutdown_connections 2024-04-02T13:55:01.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.897+0000 7ff063feb640 1 -- 172.21.15.122:0/1986101512 wait complete. 2024-04-02T13:55:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.898+0000 7ff063feb640 1 Processor -- start 2024-04-02T13:55:01.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.898+0000 7ff063feb640 1 -- start start 2024-04-02T13:55:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05c11b2e0 0x7ff05c1aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c120cd0 con 0x7ff05c110320 2024-04-02T13:55:01.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c120b50 con 0x7ff05c11b2e0 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff063feb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c120e50 con 0x7ff05c10e350 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51492/0 (socket says 172.21.15.122:51492) 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 -- 172.21.15.122:0/662129646 learned_addr learned my addr 172.21.15.122:0/662129646 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061d87640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 -- 172.21.15.122:0/662129646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 msgr2=0x7ff05c1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c1a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 -- 172.21.15.122:0/662129646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05c11b2e0 msgr2=0x7ff05c1aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05c11b2e0 0x7ff05c1aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061586640 1 -- 172.21.15.122:0/662129646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c1ac150 con 0x7ff05c110320 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.899+0000 7ff061d87640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:01.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff061586640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff058035fd0 tx=0x7ff0580346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff052ffd640 1 -- 172.21.15.122:0/662129646 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058041030 con 0x7ff05c110320 2024-04-02T13:55:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff052ffd640 1 -- 172.21.15.122:0/662129646 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0580044b0 con 0x7ff05c110320 2024-04-02T13:55:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c1ac3e0 con 0x7ff05c110320 2024-04-02T13:55:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff052ffd640 1 -- 172.21.15.122:0/662129646 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff058039040 con 0x7ff05c110320 2024-04-02T13:55:01.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.900+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1b3cd0 con 0x7ff05c110320 2024-04-02T13:55:01.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.904+0000 7ff066bcb740 1 Processor -- start 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 -- start start 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cac6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29cad2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c29ca2a7b0 con 0x55c29cac67a0 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c29c9c9570 con 0x55c29cacfca0 2024-04-02T13:55:01.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff066bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c29ca26760 con 0x55c29cac70c0 2024-04-02T13:55:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.905+0000 7ff061d87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36984/0 (socket says 172.21.15.122:36984) 2024-04-02T13:55:01.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 -- 172.21.15.122:0/925257865 learned_addr learned my addr 172.21.15.122:0/925257865 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 -- 172.21.15.122:0/925257865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 msgr2=0x55c29cad2090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29cad2090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 -- 172.21.15.122:0/925257865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 msgr2=0x55c29cac6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cac6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 -- 172.21.15.122:0/925257865 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29c73e3f0 con 0x55c29cac70c0 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff061d87640 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff04c011510 tx=0x7ff04c0119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c31f063c472bd9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.906+0000 7ff052ffd640 1 -- 172.21.15.122:0/662129646 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff058044070 con 0x7ff05c110320 2024-04-02T13:55:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.907+0000 7ff047fff640 1 -- 172.21.15.122:0/925257865 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c004020 con 0x55c29cac70c0 2024-04-02T13:55:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.907+0000 7ff047fff640 1 -- 172.21.15.122:0/925257865 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c0041c0 con 0x55c29cac70c0 2024-04-02T13:55:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.907+0000 7ff047fff640 1 -- 172.21.15.122:0/925257865 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00f8f0 con 0x55c29cac70c0 2024-04-02T13:55:01.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.907+0000 7ff052ffd640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 0x7ff0480a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.908+0000 7ff052ffd640 1 -- 172.21.15.122:0/662129646 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff0580f1fb0 con 0x7ff05c110320 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.908+0000 7ff062588640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 0x7ff0480a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 -- 172.21.15.122:0/925257865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 msgr2=0x55c29cacf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff04c011510 tx=0x7ff04c0119d0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 -- 172.21.15.122:0/925257865 shutdown_connections 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29cad2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29cacf760 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 --2- 172.21.15.122:0/925257865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cac6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff066bcb740 1 -- 172.21.15.122:0/925257865 >> 172.21.15.122:0/925257865 conn(0x55c29cac0ed0 msgr2=0x55c29cac12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.909+0000 7ff062588640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 0x7ff0480a62e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff054005c40 tx=0x7ff054005bd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.910+0000 7ff066bcb740 1 -- 172.21.15.122:0/925257865 shutdown_connections 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.910+0000 7ff066bcb740 1 -- 172.21.15.122:0/925257865 wait complete. 2024-04-02T13:55:01.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.910+0000 7ff066bcb740 1 -- start start 2024-04-02T13:55:01.914 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.910+0000 7ff066bcb740 1 Processor -- start 2024-04-02T13:55:01.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cacca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cacd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55c29cad2ae0 con 0x55c29cacfca0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55c29c9e5e80 con 0x55c29cac70c0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c29cad2c60 con 0x55c29cac67a0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff061d87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff061d87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:36994/0 (socket says 172.21.15.122:36994) 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff061d87640 1 -- 172.21.15.122:0/3046821777 learned_addr learned my addr 172.21.15.122:0/3046821777 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff062588640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cacd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.911+0000 7ff066bcb740 10 client.0 _ll_register_callbacks cb 0x55c29ca21aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 -- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 msgr2=0x55c29cacca70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cacca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 -- 172.21.15.122:0/3046821777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 msgr2=0x55c29cacd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cacd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 -- 172.21.15.122:0/3046821777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29c8087d0 con 0x55c29cac67a0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff061d87640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01b040 tx=0x7ff04c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87cf7917ecf29fca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c028020 con 0x55c29cac67a0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff04c0041c0 con 0x55c29cac67a0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff04c016040 con 0x55c29cac67a0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff04c033050 con 0x55c29cac67a0 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.912+0000 7ff066bcb740 10 client.29794 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55c29cade9d0 con 0x55c29cac67a0 2024-04-02T13:55:01.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff04c0041c0 con 0x55c29cac67a0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7ff04c00f5c0 con 0x55c29cac67a0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 10 client.29794 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 10 client.29794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 20 client.29794 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cae21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 4 client.29794 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c29ca37cf0 con 0x55c29cadfde0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.913+0000 7ff061586640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cae21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.914+0000 7ff061586640 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cae21d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f193faf491f8a2 server_cookie=96b5c26b93099bb5 in_seq=0 out_seq=0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.914+0000 7ff0457fa640 10 client.29794 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.915+0000 7ff0457fa640 1 -- 172.21.15.122:0/3046821777 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c29ca37cf0 con 0x55c29cadfde0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.915+0000 7ff0457fa640 10 client.29794 handle_command_reply: tid=0 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.915+0000 7ff066bcb740 1 client.29794 shutdown 2024-04-02T13:55:01.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 msgr2=0x55c29cacf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01b040 tx=0x7ff04c009620 comp rx=0 tx=0).stop 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 shutdown_connections 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cae21d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cacd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cacca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff0457fa640 0 client.29794 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3046821777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cacf6e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.916+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 >> 172.21.15.122:0/3046821777 conn(0x55c29cac0ed0 msgr2=0x55c29cac2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.917+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 shutdown_connections 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.917+0000 7ff066bcb740 1 -- 172.21.15.122:0/3046821777 wait complete. 2024-04-02T13:55:01.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.917+0000 7ff066bcb740 20 client.29794 trim_cache size 0 max 16384 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.944+0000 7ff066bcb740 1 Processor -- start 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 -- start start 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cadba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cae7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1bdf00 con 0x55c29cacfca0 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1bfa70 con 0x55c29cac70c0 2024-04-02T13:55:01.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.945+0000 7ff066bcb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1be5d0 con 0x55c29cac67a0 2024-04-02T13:55:01.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.946+0000 7ff062588640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cae7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.946+0000 7ff061586640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.946+0000 7ff062588640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cae7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51514/0 (socket says 172.21.15.122:51514) 2024-04-02T13:55:01.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.946+0000 7ff061586640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35042/0 (socket says 172.21.15.122:35042) 2024-04-02T13:55:01.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.946+0000 7ff062588640 1 -- 172.21.15.122:0/363504854 learned_addr learned my addr 172.21.15.122:0/363504854 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061d87640 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cadba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061586640 1 -- 172.21.15.122:0/363504854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 msgr2=0x55c29cadba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061586640 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cadba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061586640 1 -- 172.21.15.122:0/363504854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 msgr2=0x55c29cae7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061586640 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cae7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.947+0000 7ff061586640 1 -- 172.21.15.122:0/363504854 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c29cac6ef0 con 0x55c29cac70c0 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.948+0000 7ff061586640 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ef3c0 tx=0x7ff0580de040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc85f37c4a8b295 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.948+0000 7ff0477fe640 1 -- 172.21.15.122:0/363504854 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580348e0 con 0x55c29cac70c0 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.948+0000 7ff0477fe640 1 -- 172.21.15.122:0/363504854 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff05802cbf0 con 0x55c29cac70c0 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.948+0000 7ff0477fe640 1 -- 172.21.15.122:0/363504854 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0580bbfe0 con 0x55c29cac70c0 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.949+0000 7ff066bcb740 1 -- 172.21.15.122:0/363504854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 msgr2=0x55c29cae7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.949+0000 7ff066bcb740 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff0580ef3c0 tx=0x7ff0580de040 comp rx=0 tx=0).stop 2024-04-02T13:55:01.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.950+0000 7ff066bcb740 1 -- 172.21.15.122:0/363504854 shutdown_connections 2024-04-02T13:55:01.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.950+0000 7ff066bcb740 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cacfca0 0x55c29cae7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.950+0000 7ff066bcb740 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cac70c0 0x55c29cae7910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.950+0000 7ff066bcb740 1 --2- 172.21.15.122:0/363504854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac67a0 0x55c29cadba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.950+0000 7ff066bcb740 1 -- 172.21.15.122:0/363504854 >> 172.21.15.122:0/363504854 conn(0x55c29cac0ed0 msgr2=0x55c29cac12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.951+0000 7ff066bcb740 1 -- 172.21.15.122:0/363504854 shutdown_connections 2024-04-02T13:55:01.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.951+0000 7ff066bcb740 1 -- 172.21.15.122:0/363504854 wait complete. 2024-04-02T13:55:01.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.952+0000 7ff066bcb740 1 -- start start 2024-04-02T13:55:01.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.952+0000 7ff066bcb740 1 Processor -- start 2024-04-02T13:55:01.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cad8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff066bcb740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29caca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff061586640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff061586640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37010/0 (socket says 172.21.15.122:37010) 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff061586640 1 -- 172.21.15.122:0/3801207757 learned_addr learned my addr 172.21.15.122:0/3801207757 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.953+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1bf3a0 con 0x55c29cac67a0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.954+0000 7ff062588640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29caca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.954+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1beca0 con 0x55c29cacfca0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.954+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1bbf00 con 0x55c29cac70c0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.954+0000 7ff066bcb740 10 client.0 _ll_register_callbacks cb 0x7ff04c009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:01.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.954+0000 7ff061d87640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cad8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.955+0000 7ff061d87640 1 -- 172.21.15.122:0/3801207757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 msgr2=0x55c29caca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061d87640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061d87640 1 -- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 msgr2=0x55c29caca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061d87640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29caca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061d87640 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29cac8f10 con 0x55c29cac67a0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061586640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff062588640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29caca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff061d87640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cad8400 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00fb80 tx=0x7ff04c00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d99b0d3f3ea5bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c028070 con 0x55c29cac67a0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.956+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7ff04c011240 con 0x55c29cac67a0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.957+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05802cbf0 con 0x55c29cac67a0 2024-04-02T13:55:01.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.957+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff04c005c50 con 0x55c29cac67a0 2024-04-02T13:55:01.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.957+0000 7ff066bcb740 10 client.21129 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:01.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.957+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55c29cb456b0 con 0x55c29cac67a0 2024-04-02T13:55:01.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff05802cbf0 con 0x55c29cac67a0 2024-04-02T13:55:01.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7ff04c004680 con 0x55c29cac67a0 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 10 client.21129 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 10 client.21129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 20 client.21129 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cb47e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 4 client.21129 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.961+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c29ca32040 con 0x55c29cadfde0 2024-04-02T13:55:01.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.963+0000 7ff061586640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cb47e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:01.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.967+0000 7ff061586640 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cb47e00 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171c33b41c9c6763 server_cookie=adedf9458d5b5e30 in_seq=0 out_seq=0 2024-04-02T13:55:01.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.967+0000 7ff0457fa640 10 client.21129 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:01.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.968+0000 7ff0457fa640 1 -- 172.21.15.122:0/3801207757 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c29ca32040 con 0x55c29cadfde0 2024-04-02T13:55:01.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.968+0000 7ff0457fa640 10 client.21129 handle_command_reply: tid=0 2024-04-02T13:55:01.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.968+0000 7ff066bcb740 1 client.21129 shutdown 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 msgr2=0x55c29cad8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cad8400 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00fb80 tx=0x7ff04c00ffd0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 shutdown_connections 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55c29cadfde0 0x55c29cb47e00 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55c29cacfca0 0x55c29caca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c29cac70c0 0x55c29caca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 --2- 172.21.15.122:0/3801207757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55c29cac67a0 0x55c29cad8400 secure :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00fb80 tx=0x7ff04c00ffd0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 >> 172.21.15.122:0/3801207757 conn(0x55c29cac0ed0 msgr2=0x55c29cacb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.972+0000 7ff0457fa640 0 client.21129 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:01.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.973+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 shutdown_connections 2024-04-02T13:55:01.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.973+0000 7ff066bcb740 1 -- 172.21.15.122:0/3801207757 wait complete. 2024-04-02T13:55:01.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.973+0000 7ff066bcb740 20 client.21129 trim_cache size 0 max 16384 2024-04-02T13:55:01.976 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:01.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.976+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 msgr2=0x7ff0480a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 0x7ff0480a62e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff054005c40 tx=0x7ff054005bd0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 msgr2=0x7ff05c1a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7ff058035fd0 tx=0x7ff0580346f0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 shutdown_connections 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff0480a3e20 0x7ff0480a62e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff05c11b2e0 0x7ff05c1aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c110320 0x7ff05c1a76a0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 --2- 172.21.15.122:0/662129646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff05c10e350 0x7ff05c1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 >> 172.21.15.122:0/662129646 conn(0x7ff05c06cbe0 msgr2=0x7ff05c11ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:01.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 shutdown_connections 2024-04-02T13:55:01.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:01.977+0000 7ff063feb640 1 -- 172.21.15.122:0/662129646 wait complete. 2024-04-02T13:55:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:01 smithi187 ceph-mon[36064]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 590 op/s 2024-04-02T13:55:01.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1791945985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:01 smithi188 ceph-mon[36117]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 590 op/s 2024-04-02T13:55:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1791945985' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:02.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197459 27.10 MB/sec execute 262 sec latency 1354.465 ms 2024-04-02T13:55:03.002 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:55:03.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:55:03.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 -- 172.21.15.122:0/440993989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e20073ea0 msgr2=0x7f3e20074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 --2- 172.21.15.122:0/440993989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e20073ea0 0x7f3e20074280 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009900 tx=0x7f3e1802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 -- 172.21.15.122:0/440993989 shutdown_connections 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 --2- 172.21.15.122:0/440993989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e20112a80 0x7f3e20114e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 --2- 172.21.15.122:0/440993989 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e200747c0 0x7f3e20112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 --2- 172.21.15.122:0/440993989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e20073ea0 0x7f3e20074280 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 -- 172.21.15.122:0/440993989 >> 172.21.15.122:0/440993989 conn(0x7f3e200fd560 msgr2=0x7f3e200ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 -- 172.21.15.122:0/440993989 shutdown_connections 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.160+0000 7f3e2562a640 1 -- 172.21.15.122:0/440993989 wait complete. 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 Processor -- start 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 -- start start 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e20073ea0 0x7f3e201a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e20112a80 0x7f3e2019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e201184c0 con 0x7f3e200747c0 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20118340 con 0x7f3e20112a80 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.161+0000 7f3e2562a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e20118640 con 0x7f3e20073ea0 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.162+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.162+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51540/0 (socket says 172.21.15.122:51540) 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.162+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3507285964 learned_addr learned my addr 172.21.15.122:0/3507285964 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.162+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e20112a80 0x7f3e2019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.163+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3507285964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e20073ea0 msgr2=0x7f3e201a46e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:55:03.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.163+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e20073ea0 0x7f3e201a46e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.163+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3507285964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e20112a80 msgr2=0x7f3e2019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.163+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e20112a80 0x7f3e2019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.163+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3507285964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2019f070 con 0x7f3e200747c0 2024-04-02T13:55:03.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.165+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000eab0 tx=0x7f3e1000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.166+0000 7f3e17fff640 1 -- 172.21.15.122:0/3507285964 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1000ce40 con 0x7f3e200747c0 2024-04-02T13:55:03.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.166+0000 7f3e17fff640 1 -- 172.21.15.122:0/3507285964 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e10011950 con 0x7f3e200747c0 2024-04-02T13:55:03.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.166+0000 7f3e17fff640 1 -- 172.21.15.122:0/3507285964 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e10010aa0 con 0x7f3e200747c0 2024-04-02T13:55:03.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.167+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2019f360 con 0x7f3e200747c0 2024-04-02T13:55:03.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.167+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e20105d10 con 0x7f3e200747c0 2024-04-02T13:55:03.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.170+0000 7f3e28214740 1 Processor -- start 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 -- start start 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c21280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593b6b7c770 con 0x5593b6c182c0 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593b6b24960 con 0x5593b6c18be0 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e28214740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593b6b788c0 con 0x5593b6c217c0 2024-04-02T13:55:03.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.172+0000 7f3e17fff640 1 -- 172.21.15.122:0/3507285964 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3e10010c40 con 0x7f3e200747c0 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37050/0 (socket says 172.21.15.122:37050) 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3376842067 learned_addr learned my addr 172.21.15.122:0/3376842067 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3376842067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 msgr2=0x5593b6c21280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c21280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3376842067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 msgr2=0x5593b6c186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 -- 172.21.15.122:0/3376842067 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b69587e0 con 0x5593b6c217c0 2024-04-02T13:55:03.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.173+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e100ce020 tx=0x7f3e100d9830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5a9f9abddf32f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.174+0000 7f3e157fa640 1 -- 172.21.15.122:0/3376842067 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100d6850 con 0x5593b6c217c0 2024-04-02T13:55:03.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.174+0000 7f3e157fa640 1 -- 172.21.15.122:0/3376842067 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e100e13e0 con 0x5593b6c217c0 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.174+0000 7f3e157fa640 1 -- 172.21.15.122:0/3376842067 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e100d5600 con 0x5593b6c217c0 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e17fff640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 0x7f3e0c0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 0x7f3e0c0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e17fff640 1 -- 172.21.15.122:0/3507285964 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f3e100d0de0 con 0x7f3e200747c0 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e28214740 1 -- 172.21.15.122:0/3376842067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 msgr2=0x5593b6c23bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e28214740 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e100ce020 tx=0x7f3e100d9830 comp rx=0 tx=0).stop 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e28214740 1 -- 172.21.15.122:0/3376842067 shutdown_connections 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.175+0000 7f3e28214740 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c23bb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c21280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 --2- 172.21.15.122:0/3376842067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 -- 172.21.15.122:0/3376842067 >> 172.21.15.122:0/3376842067 conn(0x5593b6c129f0 msgr2=0x5593b6c12dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 0x7f3e0c0a63b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e2006b1b0 tx=0x7f3e080097c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 -- 172.21.15.122:0/3376842067 shutdown_connections 2024-04-02T13:55:03.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 -- 172.21.15.122:0/3376842067 wait complete. 2024-04-02T13:55:03.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 -- start start 2024-04-02T13:55:03.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.176+0000 7f3e28214740 1 Processor -- start 2024-04-02T13:55:03.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c182c0 0x5593b6c1e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 0x5593b6c1ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593b6c355c0 con 0x5593b6c217c0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5593b6b37d20 con 0x5593b6c182c0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5593b6c35740 con 0x5593b6c18be0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e28214740 10 client.0 _ll_register_callbacks cb 0x5593b6b04a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e1e7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 0x5593b6c1ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51554/0 (socket says 172.21.15.122:51554) 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.177+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/1038133497 learned_addr learned my addr 172.21.15.122:0/1038133497 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/1038133497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 msgr2=0x5593b6c1ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 0x5593b6c1ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c182c0 msgr2=0x5593b6c1e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c182c0 0x5593b6c1e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/1038133497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b69b8f70 con 0x5593b6c217c0 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 0x5593b6c1ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f3e08004700 tx=0x7f3e08005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121c90c65a57c11c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e067fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.178+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0801d820 con 0x5593b6c217c0 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.179+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e100e13e0 con 0x5593b6c217c0 2024-04-02T13:55:03.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.179+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f3e0801d9c0 con 0x5593b6c217c0 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.179+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f3e0801dbc0 con 0x5593b6c217c0 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.179+0000 7f3e28214740 10 client.21138 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.179+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5593b6850170 con 0x5593b6c217c0 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e100e13e0 con 0x5593b6c217c0 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f3e08004be0 con 0x5593b6c217c0 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 10 client.21138 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 10 client.21138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 20 client.21138 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:03.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6c42bc0 0x5593b6c45040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 4 client.21138 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:03.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.180+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593b6b89cb0 con 0x5593b6c42bc0 2024-04-02T13:55:03.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.181+0000 7f3e1effd640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6c42bc0 0x5593b6c45040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.183+0000 7f3e1effd640 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6c42bc0 0x5593b6c45040 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22c9f72d681793 server_cookie=c9c79cb078e4fbfc in_seq=0 out_seq=0 2024-04-02T13:55:03.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.183+0000 7f3e067fc640 10 client.21138 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.184+0000 7f3e067fc640 1 -- 172.21.15.122:0/1038133497 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593b6b89cb0 con 0x5593b6c42bc0 2024-04-02T13:55:03.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.184+0000 7f3e067fc640 10 client.21138 handle_command_reply: tid=0 2024-04-02T13:55:03.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 client.21138 shutdown 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 msgr2=0x5593b6c1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f3e08004700 tx=0x7f3e08005f90 comp rx=0 tx=0).stop 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 shutdown_connections 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6c42bc0 0x5593b6c45040 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c1f260 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c18be0 0x5593b6c1ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 --2- 172.21.15.122:0/1038133497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c182c0 0x5593b6c1e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 >> 172.21.15.122:0/1038133497 conn(0x5593b6c129f0 msgr2=0x5593b6c14740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e067fc640 0 client.21138 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 shutdown_connections 2024-04-02T13:55:03.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.185+0000 7f3e28214740 1 -- 172.21.15.122:0/1038133497 wait complete. 2024-04-02T13:55:03.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.186+0000 7f3e28214740 20 client.21138 trim_cache size 0 max 16384 2024-04-02T13:55:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.217+0000 7f3e28214740 1 Processor -- start 2024-04-02T13:55:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.217+0000 7f3e28214740 1 -- start start 2024-04-02T13:55:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e28214740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 0x5593b6c20f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e28214740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c3e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 0x5593b6c20f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e1effd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 0x5593b6c20f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37072/0 (socket says 172.21.15.122:37072) 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e1effd640 1 -- 172.21.15.122:0/2085058271 learned_addr learned my addr 172.21.15.122:0/2085058271 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.218+0000 7f3e28214740 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c4a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.219+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006d020 con 0x5593b6c217c0 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.219+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c4a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.219+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006eb90 con 0x5593b6c18be0 2024-04-02T13:55:03.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.219+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006d6f0 con 0x5593b6c182c0 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.220+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/2085058271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 msgr2=0x5593b6c20f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.220+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 0x5593b6c20f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.220+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/2085058271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 msgr2=0x5593b6c3e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.220+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c3e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.220+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/2085058271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593b6c1d790 con 0x5593b6c217c0 2024-04-02T13:55:03.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.221+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c4a910 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f3e08016040 tx=0x7f3e08005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bd3e37978d15a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.221+0000 7f3e14ff9640 1 -- 172.21.15.122:0/2085058271 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0803a070 con 0x5593b6c217c0 2024-04-02T13:55:03.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.221+0000 7f3e14ff9640 1 -- 172.21.15.122:0/2085058271 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e08005000 con 0x5593b6c217c0 2024-04-02T13:55:03.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.221+0000 7f3e14ff9640 1 -- 172.21.15.122:0/2085058271 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08004a10 con 0x5593b6c217c0 2024-04-02T13:55:03.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.222+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 msgr2=0x5593b6c4a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.222+0000 7f3e28214740 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c4a910 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f3e08016040 tx=0x7f3e08005cc0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.223+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 shutdown_connections 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.223+0000 7f3e28214740 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c217c0 0x5593b6c4a910 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.223+0000 7f3e28214740 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c3e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.223+0000 7f3e28214740 1 --2- 172.21.15.122:0/2085058271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c182c0 0x5593b6c20f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.223+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 >> 172.21.15.122:0/2085058271 conn(0x5593b6c129f0 msgr2=0x5593b6c146b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.224+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 shutdown_connections 2024-04-02T13:55:03.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.224+0000 7f3e28214740 1 -- 172.21.15.122:0/2085058271 wait complete. 2024-04-02T13:55:03.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.224+0000 7f3e28214740 1 -- start start 2024-04-02T13:55:03.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.225+0000 7f3e28214740 1 Processor -- start 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.225+0000 7f3e28214740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c3ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.225+0000 7f3e28214740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c379f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.225+0000 7f3e28214740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e28214740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006e4c0 con 0x5593b6c182c0 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e28214740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2006ddc0 con 0x5593b6c18be0 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:37080/0 (socket says 172.21.15.122:37080) 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/784124994 learned_addr learned my addr 172.21.15.122:0/784124994 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1e7fc640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2010e010 con 0x5593b6c217c0 2024-04-02T13:55:03.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1effd640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c3ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 msgr2=0x5593b6c379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c379f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/784124994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 msgr2=0x5593b6c3ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c3ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e1f7fe640 1 -- 172.21.15.122:0/784124994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593b6c385a0 con 0x5593b6c217c0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.226+0000 7f3e28214740 10 client.0 _ll_register_callbacks cb 0x7f3e08009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e08005000 con 0x5593b6c217c0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e1f7fe640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e0801a480 tx=0x7f3e08009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f9e587c4b2b399 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e067fc640 10 client.29809 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e067fc640 1 -- 172.21.15.122:0/784124994 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08044060 con 0x5593b6c217c0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e067fc640 1 -- 172.21.15.122:0/784124994 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f3e08004730 con 0x5593b6c217c0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e067fc640 1 -- 172.21.15.122:0/784124994 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f3e0803a070 con 0x5593b6c217c0 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e28214740 10 client.29809 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:03.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.227+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=75}) -- 0x5593b6ca9ec0 con 0x5593b6c217c0 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e067fc640 1 -- 172.21.15.122:0/784124994 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f3e08004970 con 0x5593b6c217c0 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 10 client.29809 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 10 client.29809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 20 client.29809 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6caa420 0x5593b6cac810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 4 client.29809 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5593b6b84000 con 0x5593b6caa420 2024-04-02T13:55:03.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.228+0000 7f3e1effd640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6caa420 0x5593b6cac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.232+0000 7f3e1effd640 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6caa420 0x5593b6cac810 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6525dc479cabf32a server_cookie=95b3893aa520ff82 in_seq=0 out_seq=0 2024-04-02T13:55:03.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.232+0000 7f3e067fc640 10 client.29809 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.233+0000 7f3e067fc640 1 -- 172.21.15.122:0/784124994 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5593b6b84000 con 0x5593b6caa420 2024-04-02T13:55:03.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.233+0000 7f3e067fc640 10 client.29809 handle_command_reply: tid=0 2024-04-02T13:55:03.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.233+0000 7f3e28214740 1 client.29809 shutdown 2024-04-02T13:55:03.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 msgr2=0x5593b6c37f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3e0801a480 tx=0x7f3e08009280 comp rx=0 tx=0).stop 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 shutdown_connections 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5593b6caa420 0x5593b6cac810 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5593b6c217c0 0x5593b6c37f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5593b6c18be0 0x5593b6c379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 --2- 172.21.15.122:0/784124994 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593b6c182c0 0x5593b6c3ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e067fc640 0 client.29809 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 >> 172.21.15.122:0/784124994 conn(0x5593b6c129f0 msgr2=0x5593b6c11220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.234+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 shutdown_connections 2024-04-02T13:55:03.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.235+0000 7f3e28214740 1 -- 172.21.15.122:0/784124994 wait complete. 2024-04-02T13:55:03.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.235+0000 7f3e28214740 20 client.29809 trim_cache size 0 max 16384 2024-04-02T13:55:03.239 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:03.239 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:55:03.239 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "a97c3c78-f9dd-4650-b0a5-700a79a1ebb0", 2024-04-02T13:55:03.239 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:55:03.239 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 msgr2=0x7f3e0c0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 0x7f3e0c0a63b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e2006b1b0 tx=0x7f3e080097c0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 msgr2=0x7f3e201a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000eab0 tx=0x7f3e1000ef70 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 shutdown_connections 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f3e0c0a3ef0 0x7f3e0c0a63b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e20112a80 0x7f3e2019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e200747c0 0x7f3e201a4c20 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 --2- 172.21.15.122:0/3507285964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3e20073ea0 0x7f3e201a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 >> 172.21.15.122:0/3507285964 conn(0x7f3e200fd560 msgr2=0x7f3e200fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 shutdown_connections 2024-04-02T13:55:03.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.237+0000 7f3e2562a640 1 -- 172.21.15.122:0/3507285964 wait complete. 2024-04-02T13:55:03.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:03.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:03.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197652 27.02 MB/sec execute 263 sec latency 432.462 ms 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/2267473172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 msgr2=0x7f8a54105c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/2267473172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 0x7f8a54105c80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009900 tx=0x7f8a5002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/2267473172 shutdown_connections 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/2267473172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 0x7f8a5410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/2267473172 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a54100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/2267473172 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 0x7f8a54105c80 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/2267473172 >> 172.21.15.122:0/2267473172 conn(0x7f8a54077a90 msgr2=0x7f8a54077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/2267473172 shutdown_connections 2024-04-02T13:55:03.419 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.415+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/2267473172 wait complete. 2024-04-02T13:55:03.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.416+0000 7f8a5b6a5640 1 Processor -- start 2024-04-02T13:55:03.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.416+0000 7f8a5b6a5640 1 -- start start 2024-04-02T13:55:03.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.416+0000 7f8a5b6a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a5b6a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 0x7f8a54197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a59441640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a59441640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35132/0 (socket says 172.21.15.122:35132) 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a5b6a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 0x7f8a5419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a59441640 1 -- 172.21.15.122:0/3024551927 learned_addr learned my addr 172.21.15.122:0/3024551927 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54110de0 con 0x7f8a541058a0 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54110c60 con 0x7f8a541009c0 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.417+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54110f60 con 0x7f8a54101360 2024-04-02T13:55:03.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a58c40640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 0x7f8a54197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 -- 172.21.15.122:0/3024551927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 msgr2=0x7f8a54197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 0x7f8a54197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 -- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 msgr2=0x7f8a5419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 0x7f8a5419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5419c2d0 con 0x7f8a541009c0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.418+0000 7f8a59441640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a500043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.419+0000 7f8a4a7fc640 1 -- 172.21.15.122:0/3024551927 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50042020 con 0x7f8a541009c0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.419+0000 7f8a4a7fc640 1 -- 172.21.15.122:0/3024551927 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a500054e0 con 0x7f8a541009c0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.419+0000 7f8a4a7fc640 1 -- 172.21.15.122:0/3024551927 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50003af0 con 0x7f8a541009c0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.419+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5419c560 con 0x7f8a541009c0 2024-04-02T13:55:03.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.419+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a541a3e50 con 0x7f8a541009c0 2024-04-02T13:55:03.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.423+0000 7f8a5e369740 1 Processor -- start 2024-04-02T13:55:03.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.423+0000 7f8a4a7fc640 1 -- 172.21.15.122:0/3024551927 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f8a50045020 con 0x7f8a541009c0 2024-04-02T13:55:03.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.423+0000 7f8a5e369740 1 -- start start 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18c0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18d5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560aa1825830 con 0x560aa18cdf70 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560aa17cd820 con 0x560aa18c1420 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a5e369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560aa18217e0 con 0x560aa18c0a80 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51588/0 (socket says 172.21.15.122:51588) 2024-04-02T13:55:03.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a59441640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18c0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a59c42640 1 -- 172.21.15.122:0/1127076007 learned_addr learned my addr 172.21.15.122:0/1127076007 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.424+0000 7f8a58c40640 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18d5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 -- 172.21.15.122:0/1127076007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 msgr2=0x560aa18c0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18c0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 -- 172.21.15.122:0/1127076007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 msgr2=0x560aa18d5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18d5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 -- 172.21.15.122:0/1127076007 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa16037d0 con 0x560aa18cdf70 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a4a7fc640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 0x7f8a400a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009d40 tx=0x7f8a4c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cc534d59b12867 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a59441640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 0x7f8a400a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a4a7fc640 1 -- 172.21.15.122:0/3024551927 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f8a500f27d0 con 0x7f8a541009c0 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a3f7fe640 1 -- 172.21.15.122:0/1127076007 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c016070 con 0x560aa18cdf70 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a3f7fe640 1 -- 172.21.15.122:0/1127076007 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c005020 con 0x560aa18cdf70 2024-04-02T13:55:03.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.425+0000 7f8a3f7fe640 1 -- 172.21.15.122:0/1127076007 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c0116b0 con 0x560aa18cdf70 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- 172.21.15.122:0/1127076007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 msgr2=0x560aa18ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a59441640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 0x7f8a400a63b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a500be130 tx=0x7f8a500f8530 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009d40 tx=0x7f8a4c00e6c0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- 172.21.15.122:0/1127076007 shutdown_connections 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18d5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18c0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1127076007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18ce350 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- 172.21.15.122:0/1127076007 >> 172.21.15.122:0/1127076007 conn(0x560aa18bba70 msgr2=0x560aa18bbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- 172.21.15.122:0/1127076007 shutdown_connections 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- 172.21.15.122:0/1127076007 wait complete. 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.426+0000 7f8a5e369740 1 -- start start 2024-04-02T13:55:03.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 Processor -- start 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c1420 0x560aa18d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa18d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560aa18d6440 con 0x560aa18c0a80 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x560aa17e0d30 con 0x560aa18cdf70 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560aa18d65c0 con 0x560aa18c1420 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a5e369740 10 client.0 _ll_register_callbacks cb 0x560aa17e0720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51604/0 (socket says 172.21.15.122:51604) 2024-04-02T13:55:03.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 -- 172.21.15.122:0/872890323 learned_addr learned my addr 172.21.15.122:0/872890323 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a58c40640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c1420 0x560aa18d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa18d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 -- 172.21.15.122:0/872890323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c1420 msgr2=0x560aa18d2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c1420 0x560aa18d2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 -- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 msgr2=0x560aa18d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa18d3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 -- 172.21.15.122:0/872890323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa15393f0 con 0x560aa18c0a80 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.427+0000 7f8a59c42640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005cc0 tx=0x7f8a4c005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdc830eeed057a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c016040 con 0x560aa18c0a80 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f8a4c025030 con 0x560aa18c0a80 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c005020 con 0x560aa18c0a80 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c016040 con 0x560aa18c0a80 2024-04-02T13:55:03.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 10 client.21153 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x560aa14f90f0 con 0x560aa18c0a80 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c005020 con 0x560aa18c0a80 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f8a4c00f040 con 0x560aa18c0a80 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 10 client.21153 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 10 client.21153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 20 client.21153 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa18e3560 0x560aa18e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 4 client.21153 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.428+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560aa1832d70 con 0x560aa18e3560 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.429+0000 7f8a58c40640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa18e3560 0x560aa18e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.431+0000 7f8a58c40640 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa18e3560 0x560aa18e5950 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93812e91be2e8f0d server_cookie=11d47551df7333a5 in_seq=0 out_seq=0 2024-04-02T13:55:03.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.431+0000 7f8a3cff9640 10 client.21153 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a3cff9640 1 -- 172.21.15.122:0/872890323 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560aa1832d70 con 0x560aa18e3560 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a3cff9640 10 client.21153 handle_command_reply: tid=0 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 client.21153 shutdown 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 msgr2=0x560aa18d5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c005cc0 tx=0x7f8a4c005a40 comp rx=0 tx=0).stop 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 shutdown_connections 2024-04-02T13:55:03.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa18e3560 0x560aa18e5950 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a3cff9640 0 client.21153 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa18d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c1420 0x560aa18d2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 --2- 172.21.15.122:0/872890323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c0a80 0x560aa18d5bb0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 >> 172.21.15.122:0/872890323 conn(0x560aa18bba70 msgr2=0x560aa18ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 shutdown_connections 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 1 -- 172.21.15.122:0/872890323 wait complete. 2024-04-02T13:55:03.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.433+0000 7f8a5e369740 20 client.21153 trim_cache size 0 max 16384 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 Processor -- start 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 -- start start 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18df370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18eb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ad150 con 0x560aa18cdf70 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541aecc0 con 0x560aa18c1420 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a5e369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ad820 con 0x560aa18c0a80 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a59c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51612/0 (socket says 172.21.15.122:51612) 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a59c42640 1 -- 172.21.15.122:0/467528003 learned_addr learned my addr 172.21.15.122:0/467528003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.459+0000 7f8a59c42640 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a58c40640 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18eb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 -- 172.21.15.122:0/467528003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 msgr2=0x560aa18df370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18df370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 -- 172.21.15.122:0/467528003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 msgr2=0x560aa18eb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18eb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 -- 172.21.15.122:0/467528003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560aa18b7c50 con 0x560aa18cdf70 2024-04-02T13:55:03.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.460+0000 7f8a59c42640 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c028000 tx=0x7f8a4c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da3ab4ed1fa587c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.461+0000 7f8a3effd640 1 -- 172.21.15.122:0/467528003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c016040 con 0x560aa18cdf70 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.461+0000 7f8a3effd640 1 -- 172.21.15.122:0/467528003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c01e070 con 0x560aa18cdf70 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.461+0000 7f8a3effd640 1 -- 172.21.15.122:0/467528003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c004e30 con 0x560aa18cdf70 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 -- 172.21.15.122:0/467528003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 msgr2=0x560aa18eb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c028000 tx=0x7f8a4c000f40 comp rx=0 tx=0).stop 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 -- 172.21.15.122:0/467528003 shutdown_connections 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18cdf70 0x560aa18eb580 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18c1420 0x560aa18eb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 --2- 172.21.15.122:0/467528003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18df370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.462+0000 7f8a5e369740 1 -- 172.21.15.122:0/467528003 >> 172.21.15.122:0/467528003 conn(0x560aa18bba70 msgr2=0x560aa18cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.463+0000 7f8a5e369740 1 -- 172.21.15.122:0/467528003 shutdown_connections 2024-04-02T13:55:03.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.463+0000 7f8a5e369740 1 -- 172.21.15.122:0/467528003 wait complete. 2024-04-02T13:55:03.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.463+0000 7f8a5e369740 1 -- start start 2024-04-02T13:55:03.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.463+0000 7f8a5e369740 1 Processor -- start 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18ea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 0x560aa18eadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa19488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ae5f0 con 0x560aa18c1420 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541adef0 con 0x560aa18cdf70 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8a541ab0d0 con 0x560aa18c0a80 2024-04-02T13:55:03.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a5e369740 10 client.0 _ll_register_callbacks cb 0x7f8a4c009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a58c40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 0x560aa18eadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a58c40640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 0x560aa18eadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51622/0 (socket says 172.21.15.122:51622) 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a58c40640 1 -- 172.21.15.122:0/1211156847 learned_addr learned my addr 172.21.15.122:0/1211156847 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa19488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 -- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 msgr2=0x560aa19488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa19488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 -- 172.21.15.122:0/1211156847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 msgr2=0x560aa18eadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 0x560aa18eadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.464+0000 7f8a59c42640 1 -- 172.21.15.122:0/1211156847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560aa1948df0 con 0x560aa18c0a80 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a59c42640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18ea890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00ebc0 tx=0x7f8a4c01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1c89393678194a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c016070 con 0x560aa18c0a80 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9525+0+0 (secure 0 0 0) 0x7f8a4c005710 con 0x560aa18c0a80 2024-04-02T13:55:03.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a4c01e070 con 0x560aa18c0a80 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8a4c04b050 con 0x560aa18c0a80 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a5e369740 10 client.29827 fetch_fsmap learned FSMap version 75 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x560aa1949f40 con 0x560aa18c0a80 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a4c005010 con 0x560aa18c0a80 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 75) ==== 8366+0+0 (secure 0 0 0) 0x7f8a4c03e070 con 0x560aa18c0a80 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.465+0000 7f8a5e369740 10 client.29827 fetch_fsmap finished waiting for FSMap version 75 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a5e369740 10 client.29827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a5e369740 20 client.29827 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa194a590 0x560aa194c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a5e369740 4 client.29827 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8a4c01e070 con 0x560aa194a590 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a58c40640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa194a590 0x560aa194c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a3cff9640 10 client.29827 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.466+0000 7f8a58c40640 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa194a590 0x560aa194c980 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dadb5c01839823f server_cookie=f75d4eb84044fab4 in_seq=0 out_seq=0 2024-04-02T13:55:03.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a3cff9640 1 -- 172.21.15.122:0/1211156847 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f8a4c01e070 con 0x560aa194a590 2024-04-02T13:55:03.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a3cff9640 10 client.29827 handle_command_reply: tid=0 2024-04-02T13:55:03.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 client.29827 shutdown 2024-04-02T13:55:03.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 msgr2=0x560aa18ea890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18ea890 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00ebc0 tx=0x7f8a4c01c000 comp rx=0 tx=0).stop 2024-04-02T13:55:03.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 shutdown_connections 2024-04-02T13:55:03.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x560aa194a590 0x560aa194c980 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x560aa18cdf70 0x560aa19488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.467+0000 7f8a3cff9640 0 client.29827 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560aa18c1420 0x560aa18eadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 1 --2- 172.21.15.122:0/1211156847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560aa18c0a80 0x560aa18ea890 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 >> 172.21.15.122:0/1211156847 conn(0x560aa18bba70 msgr2=0x560aa18d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 shutdown_connections 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 1 -- 172.21.15.122:0/1211156847 wait complete. 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.468+0000 7f8a5e369740 20 client.29827 trim_cache size 0 max 16384 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:03.474 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: "a97c3c78-f9dd-4650-b0a5-700a79a1ebb0": { 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "a97c3c78-f9dd-4650-b0a5-700a79a1ebb0", 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:55:03.475 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.470+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 msgr2=0x7f8a400a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.470+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 0x7f8a400a63b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f8a500be130 tx=0x7f8a500f8530 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.470+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 msgr2=0x7f8a541972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.470+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8a50000c00 tx=0x7f8a500043c0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 shutdown_connections 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f8a400a3ef0 0x7f8a400a63b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a541058a0 0x7f8a5419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a54101360 0x7f8a54197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 --2- 172.21.15.122:0/3024551927 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8a541009c0 0x7f8a541972e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 >> 172.21.15.122:0/3024551927 conn(0x7f8a54077a90 msgr2=0x7f8a540feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 shutdown_connections 2024-04-02T13:55:03.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:03.471+0000 7f8a5b6a5640 1 -- 172.21.15.122:0/3024551927 wait complete. 2024-04-02T13:55:03.490 INFO:tasks.cephfs.filesystem:scrub status for tag:a97c3c78-f9dd-4650-b0a5-700a79a1ebb0 - {'path': '/', 'tag': 'a97c3c78-f9dd-4650-b0a5-700a79a1ebb0', 'options': 'recursive,force'} 2024-04-02T13:55:03.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:03 smithi122 ceph-mon[32114]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 743 op/s 2024-04-02T13:55:03.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:03 smithi187 ceph-mon[36064]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 743 op/s 2024-04-02T13:55:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:03 smithi188 ceph-mon[36117]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 16 MiB/s wr, 743 op/s 2024-04-02T13:55:04.336 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197671 26.93 MB/sec execute 264 sec latency 403.082 ms 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:04 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:55:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:04 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:04 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:05.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197683 26.84 MB/sec execute 265 sec latency 420.888 ms 2024-04-02T13:55:05.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:05 smithi187 ceph-mon[36064]: pgmap v626: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 628 op/s 2024-04-02T13:55:06.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:05 smithi188 ceph-mon[36117]: pgmap v626: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 628 op/s 2024-04-02T13:55:06.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:05 smithi122 ceph-mon[32114]: pgmap v626: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 628 op/s 2024-04-02T13:55:06.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197794 26.76 MB/sec execute 266 sec latency 454.792 ms 2024-04-02T13:55:07.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 197956 26.70 MB/sec execute 267 sec latency 715.778 ms 2024-04-02T13:55:07.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:07 smithi187 ceph-mon[36064]: pgmap v627: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 543 op/s 2024-04-02T13:55:07.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:07 smithi187 ceph-mon[36064]: osdmap e122: 12 total, 12 up, 12 in 2024-04-02T13:55:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:07 smithi188 ceph-mon[36117]: pgmap v627: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 543 op/s 2024-04-02T13:55:08.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:07 smithi188 ceph-mon[36117]: osdmap e122: 12 total, 12 up, 12 in 2024-04-02T13:55:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:07 smithi122 ceph-mon[32114]: pgmap v627: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 543 op/s 2024-04-02T13:55:08.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:07 smithi122 ceph-mon[32114]: osdmap e122: 12 total, 12 up, 12 in 2024-04-02T13:55:08.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 198519 26.74 MB/sec execute 268 sec latency 437.570 ms 2024-04-02T13:55:09.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 198988 26.69 MB/sec execute 269 sec latency 90.898 ms 2024-04-02T13:55:09.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:09 smithi187 ceph-mon[36064]: pgmap v629: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 540 op/s 2024-04-02T13:55:09.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:09 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:09 smithi188 ceph-mon[36117]: pgmap v629: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 540 op/s 2024-04-02T13:55:10.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:09 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:10.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:09 smithi122 ceph-mon[32114]: pgmap v629: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 6.5 MiB/s wr, 540 op/s 2024-04-02T13:55:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:09 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:55:10.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 199671 26.73 MB/sec execute 270 sec latency 78.648 ms 2024-04-02T13:55:11.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 200346 26.80 MB/sec execute 271 sec latency 124.147 ms 2024-04-02T13:55:11.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:11 smithi187 ceph-mon[36064]: pgmap v630: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-04-02T13:55:12.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:11 smithi188 ceph-mon[36117]: pgmap v630: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-04-02T13:55:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:11 smithi122 ceph-mon[32114]: pgmap v630: 129 pgs: 5 active+clean+snaptrim, 111 active+clean, 13 active+clean+snaptrim_wait; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.3 MiB/s wr, 530 op/s 2024-04-02T13:55:12.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 200926 26.81 MB/sec execute 272 sec latency 68.437 ms 2024-04-02T13:55:13.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 201467 26.82 MB/sec execute 273 sec latency 87.134 ms 2024-04-02T13:55:13.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:13 smithi187 ceph-mon[36064]: pgmap v631: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-02T13:55:14.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:13 smithi188 ceph-mon[36117]: pgmap v631: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-02T13:55:14.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:13 smithi122 ceph-mon[32114]: pgmap v631: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.6 MiB/s wr, 520 op/s 2024-04-02T13:55:14.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 201928 26.81 MB/sec execute 274 sec latency 80.565 ms 2024-04-02T13:55:15.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 202372 26.78 MB/sec execute 275 sec latency 104.761 ms 2024-04-02T13:55:15.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:15 smithi187 ceph-mon[36064]: pgmap v632: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:16.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:15 smithi188 ceph-mon[36117]: pgmap v632: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:16.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:15 smithi122 ceph-mon[32114]: pgmap v632: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:16.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 202982 26.78 MB/sec execute 276 sec latency 91.349 ms 2024-04-02T13:55:17.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 203561 26.84 MB/sec execute 277 sec latency 100.670 ms 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: pgmap v633: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: osdmap e123: 12 total, 12 up, 12 in 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:17 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: pgmap v633: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: osdmap e123: 12 total, 12 up, 12 in 2024-04-02T13:55:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:17 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:17.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: pgmap v633: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 536 op/s 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: osdmap e123: 12 total, 12 up, 12 in 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:17 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:18.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 204227 26.88 MB/sec execute 278 sec latency 81.727 ms 2024-04-02T13:55:19.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 204672 26.88 MB/sec execute 279 sec latency 126.984 ms 2024-04-02T13:55:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:19 smithi187 ceph-mon[36064]: pgmap v635: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2024-04-02T13:55:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:19 smithi188 ceph-mon[36117]: pgmap v635: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2024-04-02T13:55:19.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:19 smithi122 ceph-mon[32114]: pgmap v635: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 509 op/s 2024-04-02T13:55:20.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 205154 26.88 MB/sec execute 280 sec latency 93.299 ms 2024-04-02T13:55:21.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 205586 26.83 MB/sec execute 281 sec latency 110.213 ms 2024-04-02T13:55:21.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:21 smithi187 ceph-mon[36064]: pgmap v636: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 498 op/s 2024-04-02T13:55:21.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:21 smithi188 ceph-mon[36117]: pgmap v636: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 498 op/s 2024-04-02T13:55:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:21 smithi122 ceph-mon[32114]: pgmap v636: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 498 op/s 2024-04-02T13:55:22.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206105 26.82 MB/sec execute 282 sec latency 137.380 ms 2024-04-02T13:55:23.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206887 26.88 MB/sec execute 283 sec latency 78.371 ms 2024-04-02T13:55:23.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:23 smithi187 ceph-mon[36064]: pgmap v637: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 498 op/s 2024-04-02T13:55:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:23 smithi188 ceph-mon[36117]: pgmap v637: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 498 op/s 2024-04-02T13:55:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:23 smithi122 ceph-mon[32114]: pgmap v637: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 498 op/s 2024-04-02T13:55:24.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206904 26.79 MB/sec execute 284 sec latency 1000.996 ms 2024-04-02T13:55:25.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 206973 26.71 MB/sec execute 285 sec latency 1745.469 ms 2024-04-02T13:55:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:25 smithi187 ceph-mon[36064]: pgmap v638: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-04-02T13:55:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:25 smithi188 ceph-mon[36117]: pgmap v638: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-04-02T13:55:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:25 smithi122 ceph-mon[32114]: pgmap v638: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 502 op/s 2024-04-02T13:55:26.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 207215 26.68 MB/sec execute 286 sec latency 865.583 ms 2024-04-02T13:55:27.337 INFO:tasks.workunit.client.0.smithi122.stdout: 10 207496 26.63 MB/sec execute 287 sec latency 410.505 ms 2024-04-02T13:55:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:27 smithi187 ceph-mon[36064]: pgmap v639: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 516 op/s 2024-04-02T13:55:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:27 smithi188 ceph-mon[36117]: pgmap v639: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 516 op/s 2024-04-02T13:55:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:27 smithi122 ceph-mon[32114]: pgmap v639: 129 pgs: 129 active+clean; 2.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 516 op/s 2024-04-02T13:55:28.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 207815 26.61 MB/sec execute 288 sec latency 133.163 ms 2024-04-02T13:55:29.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 208401 26.66 MB/sec execute 289 sec latency 73.360 ms 2024-04-02T13:55:29.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:29 smithi187 ceph-mon[36064]: pgmap v640: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-04-02T13:55:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:29 smithi188 ceph-mon[36117]: pgmap v640: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-04-02T13:55:29.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:29 smithi122 ceph-mon[32114]: pgmap v640: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-04-02T13:55:30.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 208732 26.60 MB/sec execute 290 sec latency 171.636 ms 2024-04-02T13:55:31.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 209304 26.61 MB/sec execute 291 sec latency 70.967 ms 2024-04-02T13:55:31.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:31 smithi187 ceph-mon[36064]: pgmap v641: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:55:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:31 smithi188 ceph-mon[36117]: pgmap v641: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:55:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:31 smithi122 ceph-mon[32114]: pgmap v641: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 645 op/s 2024-04-02T13:55:32.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 209889 26.60 MB/sec execute 292 sec latency 121.020 ms 2024-04-02T13:55:33.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 210537 26.65 MB/sec execute 293 sec latency 89.239 ms 2024-04-02T13:55:33.491 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:33.491 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:33.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:33 smithi187 ceph-mon[36064]: pgmap v642: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-02T13:55:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:33 smithi188 ceph-mon[36117]: pgmap v642: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-02T13:55:33.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:33 smithi122 ceph-mon[32114]: pgmap v642: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-02T13:55:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.091+0000 7fa8f83af640 1 -- 172.21.15.122:0/2366777267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01009c0 msgr2=0x7fa8f0100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.091+0000 7fa8f83af640 1 --2- 172.21.15.122:0/2366777267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01009c0 0x7fa8f0100e20 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.092+0000 7fa8f83af640 1 -- 172.21.15.122:0/2366777267 shutdown_connections 2024-04-02T13:55:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.092+0000 7fa8f83af640 1 --2- 172.21.15.122:0/2366777267 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 0x7fa8f0069e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.092+0000 7fa8f83af640 1 --2- 172.21.15.122:0/2366777267 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01009c0 0x7fa8f0100e20 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.092+0000 7fa8f83af640 1 --2- 172.21.15.122:0/2366777267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01058a0 0x7fa8f0105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.092+0000 7fa8f83af640 1 -- 172.21.15.122:0/2366777267 >> 172.21.15.122:0/2366777267 conn(0x7fa8f0077a90 msgr2=0x7fa8f0077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.094+0000 7fa8f83af640 1 -- 172.21.15.122:0/2366777267 shutdown_connections 2024-04-02T13:55:34.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.095+0000 7fa8f83af640 1 -- 172.21.15.122:0/2366777267 wait complete. 2024-04-02T13:55:34.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.096+0000 7fa8f83af640 1 Processor -- start 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 -- start start 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01009c0 0x7fa8f0113700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 0x7fa8f01107a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0114170 con 0x7fa8f01058a0 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0113ff0 con 0x7fa8f0101360 2024-04-02T13:55:34.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.097+0000 7fa8f83af640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f01142f0 con 0x7fa8f01009c0 2024-04-02T13:55:34.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.098+0000 7fa8f694c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.098+0000 7fa8f694c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59258/0 (socket says 172.21.15.122:59258) 2024-04-02T13:55:34.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.098+0000 7fa8f694c640 1 -- 172.21.15.122:0/1754366736 learned_addr learned my addr 172.21.15.122:0/1754366736 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.098+0000 7fa8f594a640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 0x7fa8f01107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.098+0000 7fa8f614b640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01009c0 0x7fa8f0113700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.099+0000 7fa8f694c640 1 -- 172.21.15.122:0/1754366736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01009c0 msgr2=0x7fa8f0113700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.099+0000 7fa8f694c640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01009c0 0x7fa8f0113700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.099+0000 7fa8f694c640 1 -- 172.21.15.122:0/1754366736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 msgr2=0x7fa8f01107a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.099+0000 7fa8f694c640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 0x7fa8f01107a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.099+0000 7fa8f694c640 1 -- 172.21.15.122:0/1754366736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f0111570 con 0x7fa8f01058a0 2024-04-02T13:55:34.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.100+0000 7fa8f694c640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c970 tx=0x7fa8ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.100+0000 7fa8e77fe640 1 -- 172.21.15.122:0/1754366736 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec007bb0 con 0x7fa8f01058a0 2024-04-02T13:55:34.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.101+0000 7fa8e77fe640 1 -- 172.21.15.122:0/1754366736 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec007d50 con 0x7fa8f01058a0 2024-04-02T13:55:34.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.101+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f00725b0 con 0x7fa8f01058a0 2024-04-02T13:55:34.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.101+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f0072b00 con 0x7fa8f01058a0 2024-04-02T13:55:34.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.101+0000 7fa8e77fe640 1 -- 172.21.15.122:0/1754366736 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0188a0 con 0x7fa8f01058a0 2024-04-02T13:55:34.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.105+0000 7fa8e77fe640 1 -- 172.21.15.122:0/1754366736 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa8ec00f450 con 0x7fa8f01058a0 2024-04-02T13:55:34.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 Processor -- start 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 -- start start 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a038d0 0x561180a03cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 0x561180a0c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.106+0000 7fa8faf98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561180967830 con 0x561180a041f0 2024-04-02T13:55:34.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8faf98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56118090f820 con 0x561180a0cdd0 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8faf98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5611809637e0 con 0x561180a038d0 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8e77fe640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 0x7fa8dc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8e77fe640 1 -- 172.21.15.122:0/1754366736 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fa8ec0d0b40 con 0x7fa8f01058a0 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35172/0 (socket says 172.21.15.122:35172) 2024-04-02T13:55:34.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.107+0000 7fa8f594a640 1 -- 172.21.15.122:0/518038026 learned_addr learned my addr 172.21.15.122:0/518038026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.108+0000 7fa8f694c640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 0x7fa8dc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.108+0000 7fa8f614b640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 0x561180a0c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f594a640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a038d0 0x561180a03cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f614b640 1 -- 172.21.15.122:0/518038026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a038d0 msgr2=0x561180a03cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f614b640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a038d0 0x561180a03cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f614b640 1 -- 172.21.15.122:0/518038026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 msgr2=0x561180a0f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f614b640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f614b640 1 -- 172.21.15.122:0/518038026 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611807457d0 con 0x561180a041f0 2024-04-02T13:55:34.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.109+0000 7fa8f594a640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:34.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.110+0000 7fa8f694c640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 0x7fa8dc0a63e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec099060 tx=0x7fa8ec004730 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.110+0000 7fa8f614b640 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 0x561180a0c890 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007da0 tx=0x7fa8e800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3910946ffb2914 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.110+0000 7fa8e4bfe640 1 -- 172.21.15.122:0/518038026 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8014070 con 0x561180a041f0 2024-04-02T13:55:34.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.110+0000 7fa8e4bfe640 1 -- 172.21.15.122:0/518038026 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e8004d20 con 0x561180a041f0 2024-04-02T13:55:34.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.110+0000 7fa8e4bfe640 1 -- 172.21.15.122:0/518038026 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8010950 con 0x561180a041f0 2024-04-02T13:55:34.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 -- 172.21.15.122:0/518038026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 msgr2=0x561180a0c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 0x561180a0c890 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007da0 tx=0x7fa8e800e720 comp rx=0 tx=0).stop 2024-04-02T13:55:34.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 -- 172.21.15.122:0/518038026 shutdown_connections 2024-04-02T13:55:34.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a0cdd0 0x561180a0f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a041f0 0x561180a0c890 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 --2- 172.21.15.122:0/518038026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a038d0 0x561180a03cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.111+0000 7fa8faf98740 1 -- 172.21.15.122:0/518038026 >> 172.21.15.122:0/518038026 conn(0x5611809fe000 msgr2=0x5611809fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.112+0000 7fa8faf98740 1 -- 172.21.15.122:0/518038026 shutdown_connections 2024-04-02T13:55:34.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.112+0000 7fa8faf98740 1 -- 172.21.15.122:0/518038026 wait complete. 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.112+0000 7fa8faf98740 1 -- start start 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.112+0000 7fa8faf98740 1 Processor -- start 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a0cdd0 0x561180a09f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561180a0fc10 con 0x561180a038d0 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x561180922d30 con 0x561180a041f0 2024-04-02T13:55:34.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561180a0fd90 con 0x561180a0cdd0 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8faf98740 10 client.0 _ll_register_callbacks cb 0x561180922720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a0c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59272/0 (socket says 172.21.15.122:59272) 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f594a640 1 -- 172.21.15.122:0/3518316458 learned_addr learned my addr 172.21.15.122:0/3518316458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a0cdd0 0x561180a09f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 -- 172.21.15.122:0/3518316458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a0cdd0 msgr2=0x561180a09f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a0cdd0 0x561180a09f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 -- 172.21.15.122:0/3518316458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 msgr2=0x561180a0c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a0c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.113+0000 7fa8f614b640 1 -- 172.21.15.122:0/3518316458 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118067b3f0 con 0x561180a041f0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.114+0000 7fa8f614b640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a099f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007da0 tx=0x7fa8e8010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a20185b39be9bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.114+0000 7fa8d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.114+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8014070 con 0x561180a041f0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.114+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fa8e8004a40 con 0x561180a041f0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.114+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e8004d20 con 0x561180a041f0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.115+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa8e8007970 con 0x561180a041f0 2024-04-02T13:55:34.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.115+0000 7fa8faf98740 10 client.30065 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:34.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.115+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56118063b0f0 con 0x561180a041f0 2024-04-02T13:55:34.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.116+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e8004d20 con 0x561180a041f0 2024-04-02T13:55:34.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.116+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa8e8023860 con 0x561180a041f0 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 10 client.30065 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 10 client.30065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 20 client.30065 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a1cde0 0x561180a1f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 4 client.30065 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:34.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.118+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561180974d70 con 0x561180a1cde0 2024-04-02T13:55:34.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.119+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a1cde0 0x561180a1f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.120+0000 7fa8d9ffb640 10 client.30065 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:34.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.120+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a1cde0 0x561180a1f1d0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb6f43e01e1b55f server_cookie=63ba578a2eaa9a6e in_seq=0 out_seq=0 2024-04-02T13:55:34.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.122+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/3518316458 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561180974d70 con 0x561180a1cde0 2024-04-02T13:55:34.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.122+0000 7fa8d9ffb640 10 client.30065 handle_command_reply: tid=0 2024-04-02T13:55:34.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.122+0000 7fa8faf98740 1 client.30065 shutdown 2024-04-02T13:55:34.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.122+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 msgr2=0x561180a099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.122+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a099f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007da0 tx=0x7fa8e8010cf0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 shutdown_connections 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a1cde0 0x561180a1f1d0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a0cdd0 0x561180a09f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a099f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3518316458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a0c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.125+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 >> 172.21.15.122:0/3518316458 conn(0x5611809fe000 msgr2=0x5611809ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.126+0000 7fa8d9ffb640 0 client.30065 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:34.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.127+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 shutdown_connections 2024-04-02T13:55:34.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.127+0000 7fa8faf98740 1 -- 172.21.15.122:0/3518316458 wait complete. 2024-04-02T13:55:34.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.127+0000 7fa8faf98740 20 client.30065 trim_cache size 0 max 16384 2024-04-02T13:55:34.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 Processor -- start 2024-04-02T13:55:34.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 -- start start 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a038d0 0x561180a082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a041f0 0x561180a0c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4010040 con 0x5611809fe000 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4011b80 con 0x561180a038d0 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8faf98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4010710 con 0x561180a041f0 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.166+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59280/0 (socket says 172.21.15.122:59280) 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 -- 172.21.15.122:0/3836126311 learned_addr learned my addr 172.21.15.122:0/3836126311 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a041f0 0x561180a0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f614b640 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a038d0 0x561180a082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 -- 172.21.15.122:0/3836126311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a041f0 msgr2=0x561180a0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a041f0 0x561180a0c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 -- 172.21.15.122:0/3836126311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a038d0 msgr2=0x561180a082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a038d0 0x561180a082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 -- 172.21.15.122:0/3836126311 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611809fde30 con 0x5611809fe000 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8f594a640 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036040 tx=0x7fa8e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8ed5dd02c22b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.167+0000 7fa8dbfff640 1 -- 172.21.15.122:0/3836126311 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004aa0 con 0x5611809fe000 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.168+0000 7fa8dbfff640 1 -- 172.21.15.122:0/3836126311 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e0004c40 con 0x5611809fe000 2024-04-02T13:55:34.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.168+0000 7fa8dbfff640 1 -- 172.21.15.122:0/3836126311 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0004aa0 con 0x5611809fe000 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.168+0000 7fa8faf98740 1 -- 172.21.15.122:0/3836126311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 msgr2=0x561180a18b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.168+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0036040 tx=0x7fa8e0009260 comp rx=0 tx=0).stop 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- 172.21.15.122:0/3836126311 shutdown_connections 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561180a041f0 0x561180a0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a038d0 0x561180a082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- 172.21.15.122:0/3836126311 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5611809fe000 0x561180a18b60 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- 172.21.15.122:0/3836126311 >> 172.21.15.122:0/3836126311 conn(0x561180a1f4d0 msgr2=0x561180a05030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- 172.21.15.122:0/3836126311 shutdown_connections 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- 172.21.15.122:0/3836126311 wait complete. 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- start start 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 Processor -- start 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 0x561180a06640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a06b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a070c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d40114b0 con 0x561180a038d0 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4010de0 con 0x561180a041f0 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.169+0000 7fa8faf98740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d400e040 con 0x5611809fe000 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8faf98740 10 client.0 _ll_register_callbacks cb 0x7fa8e8010980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 0x561180a06640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a06b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f594a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 0x561180a06640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58396/0 (socket says 172.21.15.122:58396) 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f594a640 1 -- 172.21.15.122:0/2441120621 learned_addr learned my addr 172.21.15.122:0/2441120621 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f594a640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 -- 172.21.15.122:0/2441120621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 msgr2=0x561180a06640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 0x561180a06640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 -- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 msgr2=0x561180a070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 -- 172.21.15.122:0/2441120621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561180a821b0 con 0x561180a038d0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8f614b640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a06b80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x5611809f9ba0 tx=0x7fa8e8018640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab6ef6f15690ada server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.170+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8014030 con 0x561180a038d0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.171+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fa8e80050e0 con 0x561180a038d0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.171+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8e0004c40 con 0x561180a038d0 2024-04-02T13:55:34.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.171+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa8e8007bf0 con 0x561180a038d0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.171+0000 7fa8faf98740 10 client.21177 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.171+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x561180a833c0 con 0x561180a038d0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8e0004c40 con 0x561180a038d0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa8e8023480 con 0x561180a038d0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 10 client.21177 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 10 client.21177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 20 client.21177 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a0cdd0 0x561180a859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 4 client.21177 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56118096f0c0 con 0x561180a0cdd0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.172+0000 7fa8f594a640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a0cdd0 0x561180a859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.173+0000 7fa8d9ffb640 10 client.21177 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:34.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.173+0000 7fa8f594a640 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a0cdd0 0x561180a859c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a064cf23f33b88a server_cookie=8c90f8d2513e0014 in_seq=0 out_seq=0 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.173+0000 7fa8d9ffb640 1 -- 172.21.15.122:0/2441120621 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56118096f0c0 con 0x561180a0cdd0 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.173+0000 7fa8d9ffb640 10 client.21177 handle_command_reply: tid=0 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.173+0000 7fa8faf98740 1 client.21177 shutdown 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 msgr2=0x561180a06b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a06b80 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x5611809f9ba0 tx=0x7fa8e8018640 comp rx=0 tx=0).stop 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 shutdown_connections 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x561180a0cdd0 0x561180a859c0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x561180a041f0 0x561180a070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561180a038d0 0x561180a06b80 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 --2- 172.21.15.122:0/2441120621 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5611809fe000 0x561180a06640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 >> 172.21.15.122:0/2441120621 conn(0x561180a1f4d0 msgr2=0x5611809fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8d9ffb640 0 client.21177 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 shutdown_connections 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.174+0000 7fa8faf98740 1 -- 172.21.15.122:0/2441120621 wait complete. 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.175+0000 7fa8faf98740 20 client.21177 trim_cache size 0 max 16384 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:34.182 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:55:34.183 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:55:34.183 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:34.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.175+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 msgr2=0x7fa8dc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.175+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 0x7fa8dc0a63e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec099060 tx=0x7fa8ec004730 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.175+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 msgr2=0x7fa8f0110ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.175+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00c970 tx=0x7fa8ec00ce30 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 shutdown_connections 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa8dc0a3f20 0x7fa8dc0a63e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8f01058a0 0x7fa8f0110ce0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa8f0101360 0x7fa8f01107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 --2- 172.21.15.122:0/1754366736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8f01009c0 0x7fa8f0113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 >> 172.21.15.122:0/1754366736 conn(0x7fa8f0077a90 msgr2=0x7fa8f0102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 shutdown_connections 2024-04-02T13:55:34.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.176+0000 7fa8f83af640 1 -- 172.21.15.122:0/1754366736 wait complete. 2024-04-02T13:55:34.196 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:55:34.196 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 -- 172.21.15.122:0/866196243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 msgr2=0x7f3170107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 --2- 172.21.15.122:0/866196243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f3170107e80 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3164009930 tx=0x7f316402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 -- 172.21.15.122:0/866196243 shutdown_connections 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 --2- 172.21.15.122:0/866196243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170102450 0x7f317010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 --2- 172.21.15.122:0/866196243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 0x7f3170101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 --2- 172.21.15.122:0/866196243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f3170107e80 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 -- 172.21.15.122:0/866196243 >> 172.21.15.122:0/866196243 conn(0x7f3170078f00 msgr2=0x7f3170079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 -- 172.21.15.122:0/866196243 shutdown_connections 2024-04-02T13:55:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.331+0000 7f3176c86640 1 -- 172.21.15.122:0/866196243 wait complete. 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 Processor -- start 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 -- start start 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 0x7f3170198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170102450 0x7f31701988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3170111eb0 con 0x7f3170107aa0 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3170111d30 con 0x7f3170101a90 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3176c86640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3170112030 con 0x7f3170102450 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59302/0 (socket says 172.21.15.122:59302) 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 -- 172.21.15.122:0/3169029595 learned_addr learned my addr 172.21.15.122:0/3169029595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3174a22640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 0x7f3170198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 -- 172.21.15.122:0/3169029595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170102450 msgr2=0x7f31701988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170102450 0x7f31701988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 -- 172.21.15.122:0/3169029595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 msgr2=0x7f3170198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 0x7f3170198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 -- 172.21.15.122:0/3169029595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317019d360 con 0x7f3170107aa0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.332+0000 7f3175223640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f316800eb10 tx=0x7f316800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.333+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168004b20 con 0x7f3170107aa0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.333+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317019d650 con 0x7f3170107aa0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.333+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3168004cc0 con 0x7f3170107aa0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.333+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168010760 con 0x7f3170107aa0 2024-04-02T13:55:34.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.333+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31701a4ee0 con 0x7f3170107aa0 2024-04-02T13:55:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.334+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3170068060 con 0x7f3170107aa0 2024-04-02T13:55:34.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 211180 26.68 MB/sec execute 294 sec latency 150.185 ms 2024-04-02T13:55:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.336+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3168005000 con 0x7f3170107aa0 2024-04-02T13:55:34.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.337+0000 7f316dffb640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.337+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f31680d1650 con 0x7f3170107aa0 2024-04-02T13:55:34.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.337+0000 7f3174a22640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.338+0000 7f3174a22640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3164005ab0 tx=0x7f3164005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.341+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f316809d900 con 0x7f3170107aa0 2024-04-02T13:55:34.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.678+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31701996c0 con 0x7f3170107aa0 2024-04-02T13:55:34.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.680+0000 7f316dffb640 1 -- 172.21.15.122:0/3169029595 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11942 (secure 0 0 0) 0x7f31701996c0 con 0x7f3170107aa0 2024-04-02T13:55:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:55:34.684 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:55:07.726900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:55:34.684 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-04-02T13:55:34.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.684+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 msgr2=0x7f31600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.684+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3164005ab0 tx=0x7f3164005fd0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.684+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 msgr2=0x7f317019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.684+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f316800eb10 tx=0x7f316800c2d0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 shutdown_connections 2024-04-02T13:55:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f31600a3d20 0x7f31600a61e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3170107aa0 0x7f317019cc30 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3170102450 0x7f31701988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 --2- 172.21.15.122:0/3169029595 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3170101a90 0x7f3170198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.685+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 >> 172.21.15.122:0/3169029595 conn(0x7f3170078f00 msgr2=0x7f3170104430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.686+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 shutdown_connections 2024-04-02T13:55:34.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.686+0000 7f3176c86640 1 -- 172.21.15.122:0/3169029595 wait complete. 2024-04-02T13:55:34.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3927550731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 msgr2=0x7fbfe41042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3927550731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe41042e0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009900 tx=0x7fbfd402bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3927550731 shutdown_connections 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3927550731 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe4108fd0 0x7fbfe4110ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3927550731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe41048b0 0x7fbfe41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3927550731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe41042e0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3927550731 >> 172.21.15.122:0/3927550731 conn(0x7fbfe40ff780 msgr2=0x7fbfe4101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3927550731 shutdown_connections 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.852+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3927550731 wait complete. 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.853+0000 7fbfe9e42640 1 Processor -- start 2024-04-02T13:55:34.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.853+0000 7fbfe9e42640 1 -- start start 2024-04-02T13:55:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe4108fd0 0x7fbfe419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe419eea0 0x7fbfe419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4114070 con 0x7fbfe4103f00 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe4113ef0 con 0x7fbfe419eea0 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe9e42640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41141f0 con 0x7fbfe4108fd0 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59330/0 (socket says 172.21.15.122:59330) 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 -- 172.21.15.122:0/3102916571 learned_addr learned my addr 172.21.15.122:0/3102916571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe4108fd0 0x7fbfe419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe419eea0 0x7fbfe419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 -- 172.21.15.122:0/3102916571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe4108fd0 msgr2=0x7fbfe419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe4108fd0 0x7fbfe419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 -- 172.21.15.122:0/3102916571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe419eea0 msgr2=0x7fbfe419f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe419eea0 0x7fbfe419f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.854+0000 7fbfe37fe640 1 -- 172.21.15.122:0/3102916571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe419f990 con 0x7fbfe4103f00 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe37fe640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4006010 tx=0x7fbfd402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe0ff9640 1 -- 172.21.15.122:0/3102916571 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4041070 con 0x7fbfe4103f00 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe0ff9640 1 -- 172.21.15.122:0/3102916571 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd4004450 con 0x7fbfe4103f00 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe0ff9640 1 -- 172.21.15.122:0/3102916571 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4003e00 con 0x7fbfe4103f00 2024-04-02T13:55:34.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe419fc20 con 0x7fbfe4103f00 2024-04-02T13:55:34.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.855+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe41a73e0 con 0x7fbfe4103f00 2024-04-02T13:55:34.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.858+0000 7fbfe0ff9640 1 -- 172.21.15.122:0/3102916571 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbfd4044070 con 0x7fbfe4103f00 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 Processor -- start 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 -- start start 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d17a0 0x563e1a9d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe0ff9640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 0x7fbfcc0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e1a9357b0 con 0x563e1a9daca0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e1a8d4570 con 0x563e1a9d20c0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfeca20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e1a931760 con 0x563e1a9d17a0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe37fe640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 0x7fbfcc0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35242/0 (socket says 172.21.15.122:35242) 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe2ffd640 1 -- 172.21.15.122:0/1424852221 learned_addr learned my addr 172.21.15.122:0/1424852221 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d17a0 0x563e1a9d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.859+0000 7fbfe0ff9640 1 -- 172.21.15.122:0/3102916571 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fbfd40f2400 con 0x7fbfe4103f00 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 -- 172.21.15.122:0/1424852221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d17a0 msgr2=0x563e1a9d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d17a0 0x563e1a9d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 -- 172.21.15.122:0/1424852221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 msgr2=0x563e1a9dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 -- 172.21.15.122:0/1424852221 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1a6493f0 con 0x563e1a9d20c0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe37fe640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 0x7fbfcc0a6330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40be100 tx=0x7fbfd40e91d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009c60 tx=0x7fbfd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99afc6d75fcd252 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfc9ffb640 1 -- 172.21.15.122:0/1424852221 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8004250 con 0x563e1a9d20c0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.860+0000 7fbfc9ffb640 1 -- 172.21.15.122:0/1424852221 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd801e070 con 0x563e1a9d20c0 2024-04-02T13:55:34.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.861+0000 7fbfc9ffb640 1 -- 172.21.15.122:0/1424852221 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80054e0 con 0x563e1a9d20c0 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.861+0000 7fbfeca20740 1 -- 172.21.15.122:0/1424852221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 msgr2=0x563e1a9da760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.861+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009c60 tx=0x7fbfd8011810 comp rx=0 tx=0).stop 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 -- 172.21.15.122:0/1424852221 shutdown_connections 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9da760 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1424852221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d17a0 0x563e1a9d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 -- 172.21.15.122:0/1424852221 >> 172.21.15.122:0/1424852221 conn(0x563e1a9cbed0 msgr2=0x563e1a9cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 -- 172.21.15.122:0/1424852221 shutdown_connections 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 -- 172.21.15.122:0/1424852221 wait complete. 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.862+0000 7fbfeca20740 1 -- start start 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.863+0000 7fbfeca20740 1 Processor -- start 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.863+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.863+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9d79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.863+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfeca20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563e1a9ddb30 con 0x563e1a9d17a0 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59350/0 (socket says 172.21.15.122:59350) 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2710548978 learned_addr learned my addr 172.21.15.122:0/2710548978 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563e1a8f0e80 con 0x563e1a9d20c0 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563e1a9ddcb0 con 0x563e1a9daca0 2024-04-02T13:55:34.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9d79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfeca20740 10 client.0 _ll_register_callbacks cb 0x563e1a92caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2710548978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 msgr2=0x563e1a9d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2710548978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 msgr2=0x563e1a9d79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9d79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.864+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1a7137d0 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc013890 tx=0x7fbfdc009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b627172ca840894 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc013190 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fbfdc009040 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfd801e070 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc028050 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfeca20740 10 client.21195 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.865+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563e1a9e9910 con 0x563e1a9d17a0 2024-04-02T13:55:34.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfd801e070 con 0x563e1a9d17a0 2024-04-02T13:55:34.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fbfdc02b020 con 0x563e1a9d17a0 2024-04-02T13:55:34.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 10 client.21195 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 10 client.21195 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 20 client.21195 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1a9ed200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 4 client.21195 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e1a942cf0 con 0x563e1a9eadd0 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.866+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1a9ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.867+0000 7fbfc2ffd640 10 client.21195 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:34.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.867+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1a9ed200 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f1686cf037f21722 server_cookie=d7b9f3d28388e1d in_seq=0 out_seq=0 2024-04-02T13:55:34.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.869+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/2710548978 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e1a942cf0 con 0x563e1a9eadd0 2024-04-02T13:55:34.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.869+0000 7fbfc2ffd640 10 client.21195 handle_command_reply: tid=0 2024-04-02T13:55:34.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.869+0000 7fbfeca20740 1 client.21195 shutdown 2024-04-02T13:55:34.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 msgr2=0x563e1a9da630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc013890 tx=0x7fbfdc009dd0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 shutdown_connections 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1a9ed200 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfc2ffd640 0 client.21195 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9d79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2710548978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9da630 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 >> 172.21.15.122:0/2710548978 conn(0x563e1a9cbed0 msgr2=0x563e1a9cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 shutdown_connections 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 1 -- 172.21.15.122:0/2710548978 wait complete. 2024-04-02T13:55:34.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.870+0000 7fbfeca20740 20 client.21195 trim_cache size 0 max 16384 2024-04-02T13:55:34.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 Processor -- start 2024-04-02T13:55:34.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 -- start start 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d20c0 0x563e1a9d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9da0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41ace20 con 0x563e1a9daca0 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41ae960 con 0x563e1a9d17a0 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfeca20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41ad4f0 con 0x563e1a9d20c0 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35260/0 (socket says 172.21.15.122:35260) 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2772519101 learned_addr learned my addr 172.21.15.122:0/2772519101 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.890+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9da0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2772519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d20c0 msgr2=0x563e1a9d6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d20c0 0x563e1a9d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2772519101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 msgr2=0x563e1a9da0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9da0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 -- 172.21.15.122:0/2772519101 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e1a9d82b0 con 0x563e1a9d17a0 2024-04-02T13:55:34.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x563e1a9d8d50 tx=0x7fbfdc0135c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbbaef73ec9567e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.891+0000 7fbfc97fa640 1 -- 172.21.15.122:0/2772519101 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc013190 con 0x563e1a9d17a0 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.892+0000 7fbfc97fa640 1 -- 172.21.15.122:0/2772519101 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc017070 con 0x563e1a9d17a0 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.892+0000 7fbfc97fa640 1 -- 172.21.15.122:0/2772519101 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0052a0 con 0x563e1a9d17a0 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- 172.21.15.122:0/2772519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 msgr2=0x563e1a9e6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x563e1a9d8d50 tx=0x7fbfdc0135c0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- 172.21.15.122:0/2772519101 shutdown_connections 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9daca0 0x563e1a9da0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9d20c0 0x563e1a9d6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 --2- 172.21.15.122:0/2772519101 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d17a0 0x563e1a9e6a70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- 172.21.15.122:0/2772519101 >> 172.21.15.122:0/2772519101 conn(0x563e1a9cbed0 msgr2=0x563e1a9cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- 172.21.15.122:0/2772519101 shutdown_connections 2024-04-02T13:55:34.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- 172.21.15.122:0/2772519101 wait complete. 2024-04-02T13:55:34.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.893+0000 7fbfeca20740 1 -- start start 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 Processor -- start 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41ae290 con 0x563e1a9d17a0 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41adbc0 con 0x563e1a9d20c0 2024-04-02T13:55:34.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfeca20740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe41aae20 con 0x563e1a9daca0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.894+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59372/0 (socket says 172.21.15.122:59372) 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 -- 172.21.15.122:0/1150460687 learned_addr learned my addr 172.21.15.122:0/1150460687 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfeca20740 10 client.0 _ll_register_callbacks cb 0x7fbfdc0134f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 -- 172.21.15.122:0/1150460687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 msgr2=0x563e1a9dbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9dbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 -- 172.21.15.122:0/1150460687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 msgr2=0x563e1a9db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9db850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 -- 172.21.15.122:0/1150460687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e1a9dc600 con 0x563e1a9d17a0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfe3fff640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ff80 tx=0x7fbfdc00fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97869045a8f07f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00fbe0 con 0x563e1a9d17a0 2024-04-02T13:55:34.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.895+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fbfdc003970 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfdc017070 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbfdc003b70 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 10 client.21201 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563e1aa51400 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfdc017070 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fbfdc02b030 con 0x563e1a9d17a0 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 10 client.21201 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 10 client.21201 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 20 client.21201 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1aa53ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 4 client.21201 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:34.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.896+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e1a93d040 con 0x563e1a9eadd0 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.897+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1aa53ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.897+0000 7fbfc2ffd640 10 client.21201 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.897+0000 7fbfe2ffd640 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1aa53ac0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ee5df630fc52d8 server_cookie=ebeddd7aeb1bbab2 in_seq=0 out_seq=0 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.898+0000 7fbfc2ffd640 1 -- 172.21.15.122:0/1150460687 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e1a93d040 con 0x563e1a9eadd0 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.898+0000 7fbfc2ffd640 10 client.21201 handle_command_reply: tid=0 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.898+0000 7fbfeca20740 1 client.21201 shutdown 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 msgr2=0x563e1a9e3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ff80 tx=0x7fbfdc00fe80 comp rx=0 tx=0).stop 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 shutdown_connections 2024-04-02T13:55:34.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563e1a9eadd0 0x563e1aa53ac0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563e1a9daca0 0x563e1a9dbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfc2ffd640 0 client.21201 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563e1a9d20c0 0x563e1a9db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 --2- 172.21.15.122:0/1150460687 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563e1a9d17a0 0x563e1a9e3430 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 >> 172.21.15.122:0/1150460687 conn(0x563e1a9cbed0 msgr2=0x563e1a9caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 shutdown_connections 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 1 -- 172.21.15.122:0/1150460687 wait complete. 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfeca20740 20 client.21201 trim_cache size 0 max 16384 2024-04-02T13:55:34.905 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:34.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.899+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 msgr2=0x7fbfcc0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 0x7fbfcc0a6330 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40be100 tx=0x7fbfd40e91d0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 msgr2=0x7fbfe419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4006010 tx=0x7fbfd402cb30 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 shutdown_connections 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbfcc0a3e70 0x7fbfcc0a6330 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe419eea0 0x7fbfe419f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfe4108fd0 0x7fbfe419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 --2- 172.21.15.122:0/3102916571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfe4103f00 0x7fbfe419a630 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 >> 172.21.15.122:0/3102916571 conn(0x7fbfe40ff780 msgr2=0x7fbfe4100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 shutdown_connections 2024-04-02T13:55:34.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:34.900+0000 7fbfe9e42640 1 -- 172.21.15.122:0/3102916571 wait complete. 2024-04-02T13:55:34.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 -- 172.21.15.122:0/908317291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 msgr2=0x7f5918100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 --2- 172.21.15.122:0/908317291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918100df0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5908009900 tx=0x7f590802bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 -- 172.21.15.122:0/908317291 shutdown_connections 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 --2- 172.21.15.122:0/908317291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f591810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 --2- 172.21.15.122:0/908317291 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918100df0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 --2- 172.21.15.122:0/908317291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 0x7f5918106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 -- 172.21.15.122:0/908317291 >> 172.21.15.122:0/908317291 conn(0x7f5918077b00 msgr2=0x7f5918077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 -- 172.21.15.122:0/908317291 shutdown_connections 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.052+0000 7f591ecf4640 1 -- 172.21.15.122:0/908317291 wait complete. 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 Processor -- start 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 -- start start 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f5918199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 0x7f59181a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5918110de0 con 0x7f5918101330 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ca90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5918110c60 con 0x7f5918100970 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ca90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35288/0 (socket says 172.21.15.122:35288) 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5918110f60 con 0x7f5918106980 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591ca90640 1 -- 172.21.15.122:0/1691664830 learned_addr learned my addr 172.21.15.122:0/1691664830 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.053+0000 7f591d291640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 0x7f59181a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f590ffff640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f5918199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 -- 172.21.15.122:0/1691664830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 msgr2=0x7f59181a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 0x7f59181a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 -- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 msgr2=0x7f5918199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f5918199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 -- 172.21.15.122:0/1691664830 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59181a47c0 con 0x7f5918100970 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f590ffff640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f5918199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ca90640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f591400eb90 tx=0x7f591400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f590dffb640 1 -- 172.21.15.122:0/1691664830 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914004b20 con 0x7f5918100970 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f590dffb640 1 -- 172.21.15.122:0/1691664830 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5914004cc0 con 0x7f5918100970 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f590dffb640 1 -- 172.21.15.122:0/1691664830 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5914010720 con 0x7f5918100970 2024-04-02T13:55:35.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.054+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5918196ff0 con 0x7f5918100970 2024-04-02T13:55:35.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.055+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59181974f0 con 0x7f5918100970 2024-04-02T13:55:35.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.057+0000 7f590dffb640 1 -- 172.21.15.122:0/1691664830 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5914005020 con 0x7f5918100970 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f590dffb640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 0x7f59000a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 Processor -- start 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f590dffb640 1 -- 172.21.15.122:0/1691664830 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f59140d1190 con 0x7f5918100970 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f590ffff640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 0x7f59000a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 -- start start 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 0x55765a0ae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55765a0097b0 con 0x55765a0a57a0 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557659fa8570 con 0x55765a0aeca0 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f59219b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55765a005760 con 0x55765a0a60c0 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f591ca90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f591ca90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59400/0 (socket says 172.21.15.122:59400) 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f591ca90640 1 -- 172.21.15.122:0/3423543218 learned_addr learned my addr 172.21.15.122:0/3423543218 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f590ffff640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 0x7f59000a6260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f590802efd0 tx=0x7f5908005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.058+0000 7f591d291640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f590ffff640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 0x55765a0ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 -- 172.21.15.122:0/3423543218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 msgr2=0x55765a0ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 0x55765a0ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 -- 172.21.15.122:0/3423543218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 msgr2=0x55765a0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 -- 172.21.15.122:0/3423543218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557659d1d3f0 con 0x55765a0a57a0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f590ffff640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 0x55765a0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f591ca90640 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f5914099710 tx=0x7f591409c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1351b04c63ff74 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f58feffd640 1 -- 172.21.15.122:0/3423543218 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b6070 con 0x55765a0a57a0 2024-04-02T13:55:35.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.059+0000 7f58feffd640 1 -- 172.21.15.122:0/3423543218 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591409cd40 con 0x55765a0a57a0 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f58feffd640 1 -- 172.21.15.122:0/3423543218 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140993b0 con 0x55765a0a57a0 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 -- 172.21.15.122:0/3423543218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 msgr2=0x55765a0a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f5914099710 tx=0x7f591409c880 comp rx=0 tx=0).stop 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 -- 172.21.15.122:0/3423543218 shutdown_connections 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a60c0 0x55765a0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 --2- 172.21.15.122:0/3423543218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a57a0 0x55765a0a5b80 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 -- 172.21.15.122:0/3423543218 >> 172.21.15.122:0/3423543218 conn(0x55765a09fed0 msgr2=0x55765a0a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 -- 172.21.15.122:0/3423543218 shutdown_connections 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.060+0000 7f59219b0740 1 -- 172.21.15.122:0/3423543218 wait complete. 2024-04-02T13:55:35.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.061+0000 7f59219b0740 1 -- start start 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.061+0000 7f59219b0740 1 Processor -- start 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.061+0000 7f59219b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.061+0000 7f59219b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.061+0000 7f59219b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 0x55765a0ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f59219b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55765a0b1ae0 con 0x55765a0aeca0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f59219b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557659fc4e80 con 0x55765a0a60c0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f59219b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55765a0b1c60 con 0x55765a0a57a0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591d291640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 0x55765a0ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f590ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f590ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35306/0 (socket says 172.21.15.122:35306) 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f590ffff640 1 -- 172.21.15.122:0/2194343805 learned_addr learned my addr 172.21.15.122:0/2194343805 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591d291640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 0x55765a0ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59406/0 (socket says 172.21.15.122:59406) 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f59219b0740 10 client.0 _ll_register_callbacks cb 0x55765a000aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 -- 172.21.15.122:0/2194343805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 msgr2=0x55765a0aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 -- 172.21.15.122:0/2194343805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 msgr2=0x55765a0ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 0x55765a0ac1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 -- 172.21.15.122:0/2194343805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557659de77d0 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f590ffff640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f591ca90640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0ae6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59140a12b0 tx=0x7f591409b250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc41b3d40a074c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.062+0000 7f58f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59140b6040 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591409cd40 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f59140a5520 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f59140a59b0 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f59219b0740 10 client.29872 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55765a0b1b30 con 0x55765a0a57a0 2024-04-02T13:55:35.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f591409cd40 con 0x55765a0a57a0 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f5914005430 con 0x55765a0a57a0 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.063+0000 7f59219b0740 10 client.29872 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f59219b0740 10 client.29872 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f59219b0740 20 client.29872 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a0bf0d0 0x55765a0c1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f59219b0740 4 client.29872 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55765a016cf0 con 0x55765a0bf0d0 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f590ffff640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a0bf0d0 0x55765a0c1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f58f7fff640 10 client.29872 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:35.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.064+0000 7f590ffff640 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a0bf0d0 0x55765a0c1560 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2bb5a4a315c1b040 server_cookie=c18aab8867df8453 in_seq=0 out_seq=0 2024-04-02T13:55:35.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.065+0000 7f58f7fff640 1 -- 172.21.15.122:0/2194343805 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55765a016cf0 con 0x55765a0bf0d0 2024-04-02T13:55:35.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.065+0000 7f58f7fff640 10 client.29872 handle_command_reply: tid=0 2024-04-02T13:55:35.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.065+0000 7f59219b0740 1 client.29872 shutdown 2024-04-02T13:55:35.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 msgr2=0x55765a0ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0ae6d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59140a12b0 tx=0x7f591409b250 comp rx=0 tx=0).stop 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 shutdown_connections 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a0bf0d0 0x55765a0c1560 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0aeca0 0x55765a0ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f58f7fff640 0 client.29872 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a60c0 0x55765a0aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 --2- 172.21.15.122:0/2194343805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0ae6d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 >> 172.21.15.122:0/2194343805 conn(0x55765a09fed0 msgr2=0x55765a0a1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 shutdown_connections 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 1 -- 172.21.15.122:0/2194343805 wait complete. 2024-04-02T13:55:35.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.066+0000 7f59219b0740 20 client.29872 trim_cache size 0 max 16384 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.084+0000 7f59219b0740 1 Processor -- start 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.084+0000 7f59219b0740 1 -- start start 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 0x55765a0ba9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0aeca0 0x55765a0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59181acf00 con 0x55765a0a60c0 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59181aea40 con 0x55765a0a57a0 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f59219b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ad5d0 con 0x55765a0aeca0 2024-04-02T13:55:35.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f591ca90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 0x55765a0ba9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f591ca90640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 0x55765a0ba9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35316/0 (socket says 172.21.15.122:35316) 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59416/0 (socket says 172.21.15.122:59416) 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 -- 172.21.15.122:0/3604366672 learned_addr learned my addr 172.21.15.122:0/3604366672 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f591d291640 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0aeca0 0x55765a0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 -- 172.21.15.122:0/3604366672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0aeca0 msgr2=0x55765a0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0aeca0 0x55765a0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 -- 172.21.15.122:0/3604366672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 msgr2=0x55765a0ba9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 0x55765a0ba9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 -- 172.21.15.122:0/3604366672 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765a0ac510 con 0x55765a0a60c0 2024-04-02T13:55:35.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.085+0000 7f590ffff640 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5908005ab0 tx=0x7f590803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1ff696e86ce919 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.086+0000 7f58fe7fc640 1 -- 172.21.15.122:0/3604366672 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908005240 con 0x55765a0a60c0 2024-04-02T13:55:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.086+0000 7f58fe7fc640 1 -- 172.21.15.122:0/3604366672 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5908055070 con 0x55765a0a60c0 2024-04-02T13:55:35.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.086+0000 7f58fe7fc640 1 -- 172.21.15.122:0/3604366672 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5908004920 con 0x55765a0a60c0 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- 172.21.15.122:0/3604366672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 msgr2=0x55765a0c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f5908005ab0 tx=0x7f590803c000 comp rx=0 tx=0).stop 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- 172.21.15.122:0/3604366672 shutdown_connections 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0aeca0 0x55765a0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0c6e40 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 --2- 172.21.15.122:0/3604366672 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0a57a0 0x55765a0ba9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- 172.21.15.122:0/3604366672 >> 172.21.15.122:0/3604366672 conn(0x55765a09fed0 msgr2=0x55765a0a6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- 172.21.15.122:0/3604366672 shutdown_connections 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- 172.21.15.122:0/3604366672 wait complete. 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 -- start start 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.087+0000 7f59219b0740 1 Processor -- start 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59181ae370 con 0x55765a0a60c0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59181adca0 con 0x55765a0aeca0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59181aaf00 con 0x55765a0a57a0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f59219b0740 10 client.0 _ll_register_callbacks cb 0x7f59140b4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f591ca90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59418/0 (socket says 172.21.15.122:59418) 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 -- 172.21.15.122:0/941318932 learned_addr learned my addr 172.21.15.122:0/941318932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f591d291640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 -- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 msgr2=0x55765a0af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 -- 172.21.15.122:0/941318932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 msgr2=0x55765a0afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 -- 172.21.15.122:0/941318932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765a0b61a0 con 0x55765a0a60c0 2024-04-02T13:55:35.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f591ca90640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f590ffff640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5908036040 tx=0x7f590802ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a3b6b39bd31179 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.088+0000 7f58f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080501d0 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f5908050370 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5908055070 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5908069050 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f59219b0740 10 client.21216 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55765a0b74a0 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5908055070 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.089+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f5908048040 con 0x55765a0a60c0 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 10 client.21216 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 10 client.21216 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 20 client.21216 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a1258c0 0x55765a127ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 4 client.21216 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:35.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55765a011040 con 0x55765a1258c0 2024-04-02T13:55:35.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f591ca90640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a1258c0 0x55765a127ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f591ca90640 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a1258c0 0x55765a127ce0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=80ba9bfe33c66102 server_cookie=cdd6e0c4f5252b93 in_seq=0 out_seq=0 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.090+0000 7f58f7fff640 10 client.21216 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f58f7fff640 1 -- 172.21.15.122:0/941318932 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55765a011040 con 0x55765a1258c0 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f58f7fff640 10 client.21216 handle_command_reply: tid=0 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 client.21216 shutdown 2024-04-02T13:55:35.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 msgr2=0x55765a0af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5908036040 tx=0x7f590802ce70 comp rx=0 tx=0).stop 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 shutdown_connections 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55765a0aeca0 0x55765a0afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.092+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55765a1258c0 0x55765a127ce0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f58f7fff640 0 client.21216 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55765a0a60c0 0x55765a0af760 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 1 --2- 172.21.15.122:0/941318932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55765a0a57a0 0x55765a0af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 >> 172.21.15.122:0/941318932 conn(0x55765a09fed0 msgr2=0x55765a09e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 shutdown_connections 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 1 -- 172.21.15.122:0/941318932 wait complete. 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.093+0000 7f59219b0740 20 client.21216 trim_cache size 0 max 16384 2024-04-02T13:55:35.099 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:35.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 msgr2=0x7f59000a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 0x7f59000a6260 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f590802efd0 tx=0x7f5908005fd0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 msgr2=0x7f5918199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f591400eb90 tx=0x7f591400ef90 comp rx=0 tx=0).stop 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 shutdown_connections 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f59000a3da0 0x7f59000a6260 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5918106980 0x7f59181a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5918101330 0x7f5918199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 --2- 172.21.15.122:0/1691664830 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5918100970 0x7f5918199200 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 >> 172.21.15.122:0/1691664830 conn(0x7f5918077b00 msgr2=0x7f5918103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 shutdown_connections 2024-04-02T13:55:35.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.094+0000 7f591ecf4640 1 -- 172.21.15.122:0/1691664830 wait complete. 2024-04-02T13:55:35.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/301166239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 msgr2=0x7fd3b8100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/301166239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b8100760 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800aa10 tx=0x7fd3a802cf70 comp rx=0 tx=0).stop 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/301166239 shutdown_connections 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/301166239 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b8105450 0x7fd3b810d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/301166239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8100d30 0x7fd3b8104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/301166239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b8100760 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/301166239 >> 172.21.15.122:0/301166239 conn(0x7fd3b8076f20 msgr2=0x7fd3b80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/301166239 shutdown_connections 2024-04-02T13:55:35.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.262+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/301166239 wait complete. 2024-04-02T13:55:35.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 Processor -- start 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 -- start start 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8105450 0x7fd3b81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 0x7fd3b819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810fc80 con 0x7fd3b8100380 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810fb00 con 0x7fd3b819cc60 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3bcb7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b810fe00 con 0x7fd3b8105450 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59436/0 (socket says 172.21.15.122:59436) 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 0x7fd3b819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 -- 172.21.15.122:0/640500012 learned_addr learned my addr 172.21.15.122:0/640500012 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3affff640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8105450 0x7fd3b81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 -- 172.21.15.122:0/640500012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8105450 msgr2=0x7fd3b81988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8105450 0x7fd3b81988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 -- 172.21.15.122:0/640500012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 msgr2=0x7fd3b819d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 0x7fd3b819d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b659c640 1 -- 172.21.15.122:0/640500012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b819d750 con 0x7fd3b8100380 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.263+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 0x7fd3b819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3b659c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006010 tx=0x7fd3a80052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3af7fe640 1 -- 172.21.15.122:0/640500012 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8042020 con 0x7fd3b8100380 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b819d9e0 con 0x7fd3b8100380 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b81a51d0 con 0x7fd3b8100380 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3af7fe640 1 -- 172.21.15.122:0/640500012 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a8004450 con 0x7fd3b8100380 2024-04-02T13:55:35.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.264+0000 7fd3af7fe640 1 -- 172.21.15.122:0/640500012 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8004a40 con 0x7fd3b8100380 2024-04-02T13:55:35.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.267+0000 7fd3bf3fd740 1 Processor -- start 2024-04-02T13:55:35.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.267+0000 7fd3af7fe640 1 -- 172.21.15.122:0/640500012 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd3a8045020 con 0x7fd3b8100380 2024-04-02T13:55:35.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.267+0000 7fd3bf3fd740 1 -- start start 2024-04-02T13:55:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cfbb0 0x56394e0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56394e0337b0 con 0x56394e0e13d0 2024-04-02T13:55:35.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56394dfd2570 con 0x56394e0cf200 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56394e02f760 con 0x56394e0cfbb0 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35358/0 (socket says 172.21.15.122:35358) 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 -- 172.21.15.122:0/3731657804 learned_addr learned my addr 172.21.15.122:0/3731657804 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b659c640 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cfbb0 0x56394e0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3af7fe640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 0x7fd3a40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 -- 172.21.15.122:0/3731657804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cfbb0 msgr2=0x56394e0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cfbb0 0x56394e0d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 -- 172.21.15.122:0/3731657804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 msgr2=0x56394e0e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.268+0000 7fd3b6d9d640 1 -- 172.21.15.122:0/3731657804 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394dd473f0 con 0x56394e0cf200 2024-04-02T13:55:35.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3af7fe640 1 -- 172.21.15.122:0/640500012 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd3a80f28e0 con 0x7fd3b8100380 2024-04-02T13:55:35.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 0x7fd3a40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00118d0 tx=0x7fd3b0011d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fded55ffdc565e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3acff9640 1 -- 172.21.15.122:0/3731657804 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0017050 con 0x56394e0cf200 2024-04-02T13:55:35.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3acff9640 1 -- 172.21.15.122:0/3731657804 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3b001bb90 con 0x56394e0cf200 2024-04-02T13:55:35.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.269+0000 7fd3acff9640 1 -- 172.21.15.122:0/3731657804 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000e3b0 con 0x56394e0cf200 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 0x7fd3a40a6260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0023000 tx=0x7fd3b0024000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/3731657804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 msgr2=0x56394e0cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00118d0 tx=0x7fd3b0011d90 comp rx=0 tx=0).stop 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/3731657804 shutdown_connections 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cfbb0 0x56394e0d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/3731657804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0cf5e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/3731657804 >> 172.21.15.122:0/3731657804 conn(0x56394e0c9ed0 msgr2=0x56394e0ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/3731657804 shutdown_connections 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.270+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/3731657804 wait complete. 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.271+0000 7fd3bf3fd740 1 -- start start 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.271+0000 7fd3bf3fd740 1 Processor -- start 2024-04-02T13:55:35.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.271+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.271+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.271+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56394e0e4260 con 0x56394e0e13d0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35366/0 (socket says 172.21.15.122:35366) 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 -- 172.21.15.122:0/479629739 learned_addr learned my addr 172.21.15.122:0/479629739 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56394dfeee80 con 0x56394e0cfbb0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56394e0e43e0 con 0x56394e0cf200 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3bf3fd740 10 client.0 _ll_register_callbacks cb 0x56394e02aaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3b659c640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 -- 172.21.15.122:0/479629739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 msgr2=0x56394e0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 -- 172.21.15.122:0/479629739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 msgr2=0x56394e0d1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0d1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394de117d0 con 0x56394e0cfbb0 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd3affff640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000bed0 tx=0x7fd3a0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1541afcac22bbc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.272+0000 7fd39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.273+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0004020 con 0x56394e0cfbb0 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.273+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3b001bb90 con 0x56394e0cfbb0 2024-04-02T13:55:35.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.273+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd3a00041c0 con 0x56394e0cfbb0 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd3a0024050 con 0x56394e0cfbb0 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd3bf3fd740 10 client.30122 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56394e0f02f0 con 0x56394e0cfbb0 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3b001bb90 con 0x56394e0cfbb0 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd3a00038c0 con 0x56394e0cfbb0 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd3bf3fd740 10 client.30122 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.274+0000 7fd3bf3fd740 10 client.30122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3bf3fd740 20 client.30122 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0f17c0 0x56394e0f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3bf3fd740 4 client.30122 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:35.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56394e040cf0 con 0x56394e0f17c0 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3b659c640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0f17c0 0x56394e0f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd39dffb640 10 client.30122 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.275+0000 7fd3b659c640 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0f17c0 0x56394e0f3c80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=505bf48848b7e1fc server_cookie=b2118778f166d7d2 in_seq=0 out_seq=0 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.276+0000 7fd39dffb640 1 -- 172.21.15.122:0/479629739 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56394e040cf0 con 0x56394e0f17c0 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.276+0000 7fd39dffb640 10 client.30122 handle_command_reply: tid=0 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 client.30122 shutdown 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 msgr2=0x56394e0d14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd3a000bed0 tx=0x7fd3a0011810 comp rx=0 tx=0).stop 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 shutdown_connections 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0e13d0 0x56394e0d1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0f17c0 0x56394e0f3c80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cfbb0 0x56394e0d14c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/479629739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd39dffb640 0 client.30122 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:35.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 >> 172.21.15.122:0/479629739 conn(0x56394e0c9ed0 msgr2=0x56394e0cbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 shutdown_connections 2024-04-02T13:55:35.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/479629739 wait complete. 2024-04-02T13:55:35.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.277+0000 7fd3bf3fd740 20 client.30122 trim_cache size 0 max 16384 2024-04-02T13:55:35.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 Processor -- start 2024-04-02T13:55:35.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 -- start start 2024-04-02T13:55:35.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0e13d0 0x56394e0d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd398010000 con 0x56394e0cfbb0 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd398011b40 con 0x56394e0e13d0 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3bf3fd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd3980106d0 con 0x56394e0cf200 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0ed1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58568/0 (socket says 172.21.15.122:58568) 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59466/0 (socket says 172.21.15.122:59466) 2024-04-02T13:55:35.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.298+0000 7fd3b659c640 1 -- 172.21.15.122:0/4113207443 learned_addr learned my addr 172.21.15.122:0/4113207443 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0e13d0 0x56394e0d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 -- 172.21.15.122:0/4113207443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 msgr2=0x56394e0ed1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0ed1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 -- 172.21.15.122:0/4113207443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0e13d0 msgr2=0x56394e0d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0e13d0 0x56394e0d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 -- 172.21.15.122:0/4113207443 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56394e0e0690 con 0x56394e0cfbb0 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd3affff640 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0003d60 tx=0x7fd3a0005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b427922c533fcc59 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd39ffff640 1 -- 172.21.15.122:0/4113207443 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a002c020 con 0x56394e0cfbb0 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd39ffff640 1 -- 172.21.15.122:0/4113207443 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a000b480 con 0x56394e0cfbb0 2024-04-02T13:55:35.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.299+0000 7fd39ffff640 1 -- 172.21.15.122:0/4113207443 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0005060 con 0x56394e0cfbb0 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/4113207443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 msgr2=0x56394e0f37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0003d60 tx=0x7fd3a0005c50 comp rx=0 tx=0).stop 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/4113207443 shutdown_connections 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0e13d0 0x56394e0d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0f37d0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/4113207443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0cf200 0x56394e0ed1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/4113207443 >> 172.21.15.122:0/4113207443 conn(0x56394e0c9ed0 msgr2=0x56394e0c4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.300+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/4113207443 shutdown_connections 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.301+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/4113207443 wait complete. 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.301+0000 7fd3bf3fd740 1 -- start start 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.301+0000 7fd3bf3fd740 1 Processor -- start 2024-04-02T13:55:35.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0e13d0 0x56394e0de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3b659c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35392/0 (socket says 172.21.15.122:35392) 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3b659c640 1 -- 172.21.15.122:0/2532341820 learned_addr learned my addr 172.21.15.122:0/2532341820 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd398011470 con 0x56394e0cfbb0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd398010da0 con 0x56394e0cf200 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd39800e000 con 0x56394e0e13d0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3b6d9d640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0e13d0 0x56394e0de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3bf3fd740 10 client.0 _ll_register_callbacks cb 0x7fd3a000bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 -- 172.21.15.122:0/2532341820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0e13d0 msgr2=0x56394e0de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0e13d0 0x56394e0de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 -- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 msgr2=0x56394e0dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3affff640 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56394e0df0f0 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.302+0000 7fd3b659c640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd3affff640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0de3b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x56394e0e4600 tx=0x7fd3a0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fcd0a625d99823 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a002c040 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd3a00041e0 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a000b480 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd3a003e050 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd3bf3fd740 10 client.21234 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.303+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56394e0e96f0 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a000b480 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd3a00108a0 con 0x56394e0cfbb0 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 10 client.21234 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 10 client.21234 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 20 client.21234 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0e9da0 0x56394e159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 4 client.21234 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.304+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56394e03b040 con 0x56394e0e9da0 2024-04-02T13:55:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.306+0000 7fd3b659c640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0e9da0 0x56394e159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.307+0000 7fd3b659c640 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0e9da0 0x56394e159dd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f503d16c4748dc53 server_cookie=38cb80178d5fcad9 in_seq=0 out_seq=0 2024-04-02T13:55:35.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.308+0000 7fd39dffb640 10 client.21234 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:35.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.309+0000 7fd39dffb640 1 -- 172.21.15.122:0/2532341820 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56394e03b040 con 0x56394e0e9da0 2024-04-02T13:55:35.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.309+0000 7fd39dffb640 10 client.21234 handle_command_reply: tid=0 2024-04-02T13:55:35.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.309+0000 7fd3bf3fd740 1 client.21234 shutdown 2024-04-02T13:55:35.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.309+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 msgr2=0x56394e0de3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.309+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0de3b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x56394e0e4600 tx=0x7fd3a0011810 comp rx=0 tx=0).stop 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 shutdown_connections 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56394e0e13d0 0x56394e0de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56394e0cfbb0 0x56394e0de3b0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x56394e0e9da0 0x56394e159dd0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 --2- 172.21.15.122:0/2532341820 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56394e0cf200 0x56394e0dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.310+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 >> 172.21.15.122:0/2532341820 conn(0x56394e0c9ed0 msgr2=0x56394e0f37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.311+0000 7fd39dffb640 0 client.21234 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.311+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 shutdown_connections 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.311+0000 7fd3bf3fd740 1 -- 172.21.15.122:0/2532341820 wait complete. 2024-04-02T13:55:35.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.311+0000 7fd3bf3fd740 20 client.21234 trim_cache size 0 max 16384 2024-04-02T13:55:35.316 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 msgr2=0x7fd3a40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 0x7fd3a40a6260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0023000 tx=0x7fd3b0024000 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 msgr2=0x7fd3b81983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006010 tx=0x7fd3a80052b0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 shutdown_connections 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd3a40a3da0 0x7fd3a40a6260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd3b819cc60 0x7fd3b819d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3b8105450 0x7fd3b81988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 --2- 172.21.15.122:0/640500012 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3b8100380 0x7fd3b81983a0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 >> 172.21.15.122:0/640500012 conn(0x7fd3b8076f20 msgr2=0x7fd3b80feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 shutdown_connections 2024-04-02T13:55:35.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.315+0000 7fd3bcb7c640 1 -- 172.21.15.122:0/640500012 wait complete. 2024-04-02T13:55:35.330 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:55:35.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 211733 26.72 MB/sec execute 295 sec latency 133.147 ms 2024-04-02T13:55:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 -- 172.21.15.122:0/3650889455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3920108310 msgr2=0x7f39201086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 --2- 172.21.15.122:0/3650889455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3920108310 0x7f39201086f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3910009900 tx=0x7f391002bc60 comp rx=0 tx=0).stop 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 -- 172.21.15.122:0/3650889455 shutdown_connections 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 --2- 172.21.15.122:0/3650889455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f392010d3e0 0x7f3920114fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 --2- 172.21.15.122:0/3650889455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3920108cc0 0x7f392010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 --2- 172.21.15.122:0/3650889455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3920108310 0x7f39201086f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.491+0000 7f39269e2640 1 -- 172.21.15.122:0/3650889455 >> 172.21.15.122:0/3650889455 conn(0x7f3920076040 msgr2=0x7f3920078460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.492+0000 7f39269e2640 1 -- 172.21.15.122:0/3650889455 shutdown_connections 2024-04-02T13:55:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.492+0000 7f39269e2640 1 -- 172.21.15.122:0/3650889455 wait complete. 2024-04-02T13:55:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.493+0000 7f39269e2640 1 Processor -- start 2024-04-02T13:55:35.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.493+0000 7f39269e2640 1 -- start start 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.494+0000 7f39269e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.494+0000 7f39269e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 0x7f39201a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.494+0000 7f39269e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392019e7a0 0x7f392019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.494+0000 7f391ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.494+0000 7f391ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59504/0 (socket says 172.21.15.122:59504) 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 0x7f39201a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 0x7f39201a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58598/0 (socket says 172.21.15.122:58598) 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39201185c0 con 0x7f3920108310 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3920118440 con 0x7f392019e7a0 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3920118740 con 0x7f392010d3e0 2024-04-02T13:55:35.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 -- 172.21.15.122:0/3456685102 learned_addr learned my addr 172.21.15.122:0/3456685102 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f3924f7f640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392019e7a0 0x7f392019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 -- 172.21.15.122:0/3456685102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 msgr2=0x7f39201a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 0x7f39201a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 -- 172.21.15.122:0/3456685102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392019e7a0 msgr2=0x7f392019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392019e7a0 0x7f392019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392019f3a0 con 0x7f3920108310 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.495+0000 7f391ffff640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3910004f50 tx=0x7f391002c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.496+0000 7f391d7fa640 1 -- 172.21.15.122:0/3456685102 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3910041020 con 0x7f3920108310 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.496+0000 7f391d7fa640 1 -- 172.21.15.122:0/3456685102 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f391002cd30 con 0x7f3920108310 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.496+0000 7f391d7fa640 1 -- 172.21.15.122:0/3456685102 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39100346a0 con 0x7f3920108310 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.496+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392019f630 con 0x7f3920108310 2024-04-02T13:55:35.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.496+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39201ab730 con 0x7f3920108310 2024-04-02T13:55:35.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 Processor -- start 2024-04-02T13:55:35.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 -- start start 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f391d7fa640 1 -- 172.21.15.122:0/3456685102 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f3910044020 con 0x7f3920108310 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 0x5634772f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772fcca0 0x5634772ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634772577b0 con 0x5634772f40c0 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5634771f6570 con 0x5634772fcca0 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.499+0000 7f39295c6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563477253760 con 0x5634772f37a0 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 0x5634772f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59518/0 (socket says 172.21.15.122:59518) 2024-04-02T13:55:35.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391ffff640 1 -- 172.21.15.122:0/424210746 learned_addr learned my addr 172.21.15.122:0/424210746 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 -- 172.21.15.122:0/424210746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772fcca0 msgr2=0x5634772ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772fcca0 0x5634772ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 -- 172.21.15.122:0/424210746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 msgr2=0x5634772fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 -- 172.21.15.122:0/424210746 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563476f6b3f0 con 0x5634772f37a0 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391ffff640 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391d7fa640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 0x7f39080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f391ffff640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 0x7f39080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.500+0000 7f3924f7f640 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 0x5634772f3b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3918009f70 tx=0x7f3918009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=859ff529fab3b0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.501+0000 7f391d7fa640 1 -- 172.21.15.122:0/3456685102 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f39100f2f30 con 0x7f3920108310 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.501+0000 7f39067fc640 1 -- 172.21.15.122:0/424210746 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3918013850 con 0x5634772f37a0 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.501+0000 7f39067fc640 1 -- 172.21.15.122:0/424210746 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3918019070 con 0x5634772f37a0 2024-04-02T13:55:35.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.501+0000 7f39067fc640 1 -- 172.21.15.122:0/424210746 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39180122e0 con 0x5634772f37a0 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 -- 172.21.15.122:0/424210746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 msgr2=0x5634772f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 0x5634772f3b80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3918009f70 tx=0x7f3918009650 comp rx=0 tx=0).stop 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f391ffff640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 0x7f39080a6260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f39100bb560 tx=0x7f39100f6960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 -- 172.21.15.122:0/424210746 shutdown_connections 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772fcca0 0x5634772ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 --2- 172.21.15.122:0/424210746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f37a0 0x5634772f3b80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 -- 172.21.15.122:0/424210746 >> 172.21.15.122:0/424210746 conn(0x5634772eded0 msgr2=0x5634772ee2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 -- 172.21.15.122:0/424210746 shutdown_connections 2024-04-02T13:55:35.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.502+0000 7f39295c6740 1 -- 172.21.15.122:0/424210746 wait complete. 2024-04-02T13:55:35.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.503+0000 7f39295c6740 1 -- start start 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.503+0000 7f39295c6740 1 Processor -- start 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.503+0000 7f39295c6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634772fa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5634772ffa90 con 0x5634772f40c0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772f9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35430/0 (socket says 172.21.15.122:35430) 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f3924f7f640 1 -- 172.21.15.122:0/646395435 learned_addr learned my addr 172.21.15.122:0/646395435 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563477212e80 con 0x5634772f37a0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5634772ffc10 con 0x5634772fcca0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.504+0000 7f39295c6740 10 client.0 _ll_register_callbacks cb 0x56347724eaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f3924f7f640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634772fa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f391f7fe640 1 -- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 msgr2=0x5634772fa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f391f7fe640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634772fa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f391f7fe640 1 -- 172.21.15.122:0/646395435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 msgr2=0x5634772f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f391f7fe640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f391f7fe640 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634770357d0 con 0x5634772f40c0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.505+0000 7f3924f7f640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634772fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.506+0000 7f391f7fe640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fa0a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x5634772ef060 tx=0x7f391400d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09b19cb6b0c6043 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.506+0000 7f38ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.506+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914014430 con 0x5634772f40c0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.506+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f39140145d0 con 0x5634772f40c0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.507+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3918019070 con 0x5634772f40c0 2024-04-02T13:55:35.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.507+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3914022050 con 0x5634772f40c0 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.507+0000 7f39295c6740 10 client.21243 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.507+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56347730bc20 con 0x5634772f40c0 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914019070 con 0x5634772f40c0 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f3914013070 con 0x5634772f40c0 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 10 client.21243 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 10 client.21243 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 20 client.21243 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x56347730f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 4 client.21243 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.508+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3918019070 con 0x56347730d4b0 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.509+0000 7f3924f7f640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x56347730f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.509+0000 7f38ff7fe640 10 client.21243 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:35.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.510+0000 7f3924f7f640 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x56347730f8f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9bdeff642b4fa69 server_cookie=b8d5f0f2e8274d6 in_seq=0 out_seq=0 2024-04-02T13:55:35.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f38ff7fe640 1 -- 172.21.15.122:0/646395435 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f3918019070 con 0x56347730d4b0 2024-04-02T13:55:35.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f38ff7fe640 10 client.21243 handle_command_reply: tid=0 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 client.21243 shutdown 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 msgr2=0x5634772fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fa0a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x5634772ef060 tx=0x7f391400d5f0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 shutdown_connections 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x56347730f8f0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634772fa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f40c0 0x5634772fa0a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 --2- 172.21.15.122:0/646395435 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f38ff7fe640 0 client.21243 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.514+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 >> 172.21.15.122:0/646395435 conn(0x5634772eded0 msgr2=0x5634772efc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.515+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 shutdown_connections 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.515+0000 7f39295c6740 1 -- 172.21.15.122:0/646395435 wait complete. 2024-04-02T13:55:35.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.515+0000 7f39295c6740 20 client.21243 trim_cache size 0 max 16384 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.537+0000 7f39295c6740 1 Processor -- start 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 -- start start 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 0x563477308950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772fcca0 0x563477315200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b55b0 con 0x5634772fcca0 2024-04-02T13:55:35.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b70f0 con 0x5634772f37a0 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f39295c6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b5c80 con 0x5634772f40c0 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772fc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35432/0 (socket says 172.21.15.122:35432) 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f3924f7f640 1 -- 172.21.15.122:0/323297332 learned_addr learned my addr 172.21.15.122:0/323297332 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f3924f7f640 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772fcca0 0x563477315200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.538+0000 7f391f7fe640 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 0x563477308950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 -- 172.21.15.122:0/323297332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 msgr2=0x5634772fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772fc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 -- 172.21.15.122:0/323297332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772fcca0 msgr2=0x563477315200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772fcca0 0x563477315200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 -- 172.21.15.122:0/323297332 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634772f3ef0 con 0x5634772f40c0 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.539+0000 7f391f7fe640 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 0x563477308950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3914000ea0 tx=0x7f391400db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38bcf57218390673 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.540+0000 7f3905ffb640 1 -- 172.21.15.122:0/323297332 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400b280 con 0x5634772f40c0 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.540+0000 7f3905ffb640 1 -- 172.21.15.122:0/323297332 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914019040 con 0x5634772f40c0 2024-04-02T13:55:35.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.540+0000 7f3905ffb640 1 -- 172.21.15.122:0/323297332 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914003910 con 0x5634772f40c0 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 -- 172.21.15.122:0/323297332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 msgr2=0x563477308950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 0x563477308950 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3914000ea0 tx=0x7f391400db70 comp rx=0 tx=0).stop 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 -- 172.21.15.122:0/323297332 shutdown_connections 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772fcca0 0x563477315200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772f40c0 0x563477308950 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 --2- 172.21.15.122:0/323297332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f37a0 0x5634772fc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 -- 172.21.15.122:0/323297332 >> 172.21.15.122:0/323297332 conn(0x5634772eded0 msgr2=0x5634772f4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 -- 172.21.15.122:0/323297332 shutdown_connections 2024-04-02T13:55:35.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.541+0000 7f39295c6740 1 -- 172.21.15.122:0/323297332 wait complete. 2024-04-02T13:55:35.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.542+0000 7f39295c6740 1 -- start start 2024-04-02T13:55:35.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.542+0000 7f39295c6740 1 Processor -- start 2024-04-02T13:55:35.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 0x563477300360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634773008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b6a20 con 0x5634772f37a0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b6350 con 0x5634772f40c0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f39201b35b0 con 0x5634772fcca0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f39295c6740 10 client.0 _ll_register_callbacks cb 0x7f391400b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f3924f7f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59546/0 (socket says 172.21.15.122:59546) 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f3924f7f640 1 -- 172.21.15.122:0/856837571 learned_addr learned my addr 172.21.15.122:0/856837571 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.543+0000 7f391f7fe640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 0x563477300360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 -- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 msgr2=0x5634773008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634773008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 -- 172.21.15.122:0/856837571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 msgr2=0x5634772ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 -- 172.21.15.122:0/856837571 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563477371d30 con 0x5634772f40c0 2024-04-02T13:55:35.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f3924f7f640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f391f7fe640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 0x563477300360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3914012c60 tx=0x7f3914012690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dc20f5e67f62cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f38ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.544+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400b280 con 0x5634772f40c0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.545+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f3914014840 con 0x5634772f40c0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.545+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3914019040 con 0x5634772f40c0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.545+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3914038050 con 0x5634772f40c0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.545+0000 7f39295c6740 10 client.30140 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.545+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563477373030 con 0x5634772f40c0 2024-04-02T13:55:35.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.546+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3914019040 con 0x5634772f40c0 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f391402f410 con 0x5634772f40c0 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 10 client.30140 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 10 client.30140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 20 client.30140 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x563477375640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 4 client.30140 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:35.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.547+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563477264cf0 con 0x56347730d4b0 2024-04-02T13:55:35.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.548+0000 7f3924f7f640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x563477375640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.548+0000 7f3924f7f640 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x563477375640 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2991f5c0b4f33915 server_cookie=53831923b9c40cab in_seq=0 out_seq=0 2024-04-02T13:55:35.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f38ff7fe640 10 client.30140 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f38ff7fe640 1 -- 172.21.15.122:0/856837571 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563477264cf0 con 0x56347730d4b0 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f38ff7fe640 10 client.30140 handle_command_reply: tid=0 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f39295c6740 1 client.30140 shutdown 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 msgr2=0x563477300360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.549+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 0x563477300360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3914012c60 tx=0x7f3914012690 comp rx=0 tx=0).stop 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 shutdown_connections 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x56347730d4b0 0x563477375640 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634772fcca0 0x5634773008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f38ff7fe640 0 client.30140 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5634772f40c0 0x563477300360 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 --2- 172.21.15.122:0/856837571 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5634772f37a0 0x5634772ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 >> 172.21.15.122:0/856837571 conn(0x5634772eded0 msgr2=0x5634772f9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 shutdown_connections 2024-04-02T13:55:35.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 1 -- 172.21.15.122:0/856837571 wait complete. 2024-04-02T13:55:35.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39295c6740 20 client.30140 trim_cache size 0 max 16384 2024-04-02T13:55:35.555 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 msgr2=0x7f39080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 0x7f39080a6260 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f39100bb560 tx=0x7f39100f6960 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 msgr2=0x7f39201a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.550+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f3910004f50 tx=0x7f391002c930 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 shutdown_connections 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f39080a3da0 0x7f39080a6260 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f392019e7a0 0x7f392019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f392010d3e0 0x7f39201a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 --2- 172.21.15.122:0/3456685102 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3920108310 0x7f39201a4690 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 >> 172.21.15.122:0/3456685102 conn(0x7f3920076040 msgr2=0x7f39200769b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 shutdown_connections 2024-04-02T13:55:35.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.551+0000 7f39269e2640 1 -- 172.21.15.122:0/3456685102 wait complete. 2024-04-02T13:55:35.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:55:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:35 smithi187 ceph-mon[36064]: pgmap v643: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-04-02T13:55:35.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:35 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3169029595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 -- 172.21.15.122:0/4253165280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f501014d0 msgr2=0x7f4f501018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 --2- 172.21.15.122:0/4253165280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f501014d0 0x7f4f501018b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009900 tx=0x7f4f4002bc30 comp rx=0 tx=0).stop 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 -- 172.21.15.122:0/4253165280 shutdown_connections 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 --2- 172.21.15.122:0/4253165280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f5010e510 0x7f4f50110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 --2- 172.21.15.122:0/4253165280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f50101df0 0x7f4f5010dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 --2- 172.21.15.122:0/4253165280 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f501014d0 0x7f4f501018b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 -- 172.21.15.122:0/4253165280 >> 172.21.15.122:0/4253165280 conn(0x7f4f50077a90 msgr2=0x7f4f50077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 -- 172.21.15.122:0/4253165280 shutdown_connections 2024-04-02T13:55:35.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.753+0000 7f4f55214640 1 -- 172.21.15.122:0/4253165280 wait complete. 2024-04-02T13:55:35.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 Processor -- start 2024-04-02T13:55:35.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 -- start start 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f50101df0 0x7f4f5019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 0x7f4f5019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50114160 con 0x7f4f501014d0 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f50113fe0 con 0x7f4f5010e510 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f55214640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501142e0 con 0x7f4f50101df0 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59566/0 (socket says 172.21.15.122:59566) 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.754+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/3647041702 learned_addr learned my addr 172.21.15.122:0/3647041702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f50101df0 0x7f4f5019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 0x7f4f5019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/3647041702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f50101df0 msgr2=0x7f4f5019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f50101df0 0x7f4f5019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/3647041702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 msgr2=0x7f4f5019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 0x7f4f5019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/3647041702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5019f610 con 0x7f4f501014d0 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 0x7f4f5019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.755+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f4f40006010 tx=0x7f4f4002cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.756+0000 7f4f37fff640 1 -- 172.21.15.122:0/3647041702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40041070 con 0x7f4f501014d0 2024-04-02T13:55:35.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.756+0000 7f4f37fff640 1 -- 172.21.15.122:0/3647041702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f40004450 con 0x7f4f501014d0 2024-04-02T13:55:35.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.756+0000 7f4f37fff640 1 -- 172.21.15.122:0/3647041702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40003d00 con 0x7f4f501014d0 2024-04-02T13:55:35.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.756+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5019f8a0 con 0x7f4f501014d0 2024-04-02T13:55:35.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.758+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f501a7190 con 0x7f4f501014d0 2024-04-02T13:55:35.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.761+0000 7f4f37fff640 1 -- 172.21.15.122:0/3647041702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4f40044070 con 0x7f4f501014d0 2024-04-02T13:55:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.761+0000 7f4f57dff740 1 Processor -- start 2024-04-02T13:55:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.761+0000 7f4f57dff740 1 -- start start 2024-04-02T13:55:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd35b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621cd2b37b0 con 0x5621cd34f7a0 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621cd252570 con 0x5621cd3500c0 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f57dff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621cd2af760 con 0x5621cd358ca0 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59580/0 (socket says 172.21.15.122:59580) 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3033483317 learned_addr learned my addr 172.21.15.122:0/3033483317 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f37fff640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 0x7f4f380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3033483317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 msgr2=0x5621cd35b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd35b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3033483317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 msgr2=0x5621cd358760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3033483317 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ccfc73f0 con 0x5621cd34f7a0 2024-04-02T13:55:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 0x7f4f380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f37fff640 1 -- 172.21.15.122:0/3647041702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f4f400f4460 con 0x7f4f501014d0 2024-04-02T13:55:35.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.762+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f4f48011910 tx=0x7f4f48011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd57e564b5e2d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:35 smithi188 ceph-mon[36117]: pgmap v643: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-04-02T13:55:35.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:35 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3169029595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.763+0000 7f4f357fa640 1 -- 172.21.15.122:0/3033483317 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4801f040 con 0x5621cd34f7a0 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.763+0000 7f4f357fa640 1 -- 172.21.15.122:0/3033483317 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48020020 con 0x5621cd34f7a0 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.763+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 0x7f4f380a6260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f48021000 tx=0x7f4f48022000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.763+0000 7f4f357fa640 1 -- 172.21.15.122:0/3033483317 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48017050 con 0x5621cd34f7a0 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- 172.21.15.122:0/3033483317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 msgr2=0x5621cd34fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f4f48011910 tx=0x7f4f48011dd0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- 172.21.15.122:0/3033483317 shutdown_connections 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd35b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3033483317 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd34fb80 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- 172.21.15.122:0/3033483317 >> 172.21.15.122:0/3033483317 conn(0x5621cd349ed0 msgr2=0x5621cd34a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- 172.21.15.122:0/3033483317 shutdown_connections 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- 172.21.15.122:0/3033483317 wait complete. 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 -- start start 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.764+0000 7f4f57dff740 1 Processor -- start 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd355af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd356230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5621cd35ba90 con 0x5621cd34f7a0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5621cd26ee80 con 0x5621cd3500c0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59588/0 (socket says 172.21.15.122:59588) 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd356230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/1725255243 learned_addr learned my addr 172.21.15.122:0/1725255243 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd355af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5621cd35bc10 con 0x5621cd358ca0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f57dff740 10 client.0 _ll_register_callbacks cb 0x5621cd2aaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4e59c640 1 -- 172.21.15.122:0/1725255243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 msgr2=0x5621cd356230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.765+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd356230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f4e59c640 1 -- 172.21.15.122:0/1725255243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 msgr2=0x5621cd358760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f4e59c640 1 -- 172.21.15.122:0/1725255243 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621cd0917d0 con 0x5621cd3500c0 2024-04-02T13:55:35.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd355af0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f501003f0 tx=0x7f4f44011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a44c496b07a1afa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4401e060 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f4f440035f0 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f48020020 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4f44025050 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f57dff740 10 client.30152 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.766+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5621cd367be0 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f48020020 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f4f4400f950 con 0x5621cd3500c0 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 10 client.30152 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 10 client.30152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 20 client.30152 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd36b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 4 client.30152 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621cd2c0cf0 con 0x5621cd3690a0 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.767+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd36b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.768+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd36b4f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70b5590b613be75 server_cookie=994325172966280c in_seq=0 out_seq=0 2024-04-02T13:55:35.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.768+0000 7f4f32ff5640 10 client.30152 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:35.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f32ff5640 1 -- 172.21.15.122:0/1725255243 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621cd2c0cf0 con 0x5621cd3690a0 2024-04-02T13:55:35.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f32ff5640 10 client.30152 handle_command_reply: tid=0 2024-04-02T13:55:35.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 client.30152 shutdown 2024-04-02T13:55:35.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 msgr2=0x5621cd355af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd355af0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f501003f0 tx=0x7f4f44011a90 comp rx=0 tx=0).stop 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 shutdown_connections 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd36b4f0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f32ff5640 0 client.30152 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd358ca0 0x5621cd356230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd355af0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 --2- 172.21.15.122:0/1725255243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd358760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 >> 172.21.15.122:0/1725255243 conn(0x5621cd349ed0 msgr2=0x5621cd34bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 shutdown_connections 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 1 -- 172.21.15.122:0/1725255243 wait complete. 2024-04-02T13:55:35.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.771+0000 7f4f57dff740 20 client.30152 trim_cache size 0 max 16384 2024-04-02T13:55:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 Processor -- start 2024-04-02T13:55:35.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 -- start start 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd34f7a0 0x5621cd3649b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501b1270 con 0x5621cd358ca0 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501b2db0 con 0x5621cd3500c0 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f57dff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501b1940 con 0x5621cd34f7a0 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f4f59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f4f59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59600/0 (socket says 172.21.15.122:59600) 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3052874601 learned_addr learned my addr 172.21.15.122:0/3052874601 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.793+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd34f7a0 0x5621cd3649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3052874601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd34f7a0 msgr2=0x5621cd3649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd34f7a0 0x5621cd3649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3052874601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 msgr2=0x5621cd358220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 -- 172.21.15.122:0/3052874601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621cd3565a0 con 0x5621cd358ca0 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:55:35.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4f48011910 tx=0x7f4f48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a275babeb49a2a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.794+0000 7f4f34ff9640 1 -- 172.21.15.122:0/3052874601 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48032030 con 0x5621cd358ca0 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.795+0000 7f4f34ff9640 1 -- 172.21.15.122:0/3052874601 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4800fc60 con 0x5621cd358ca0 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.795+0000 7f4f34ff9640 1 -- 172.21.15.122:0/3052874601 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4801f040 con 0x5621cd358ca0 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.795+0000 7f4f57dff740 1 -- 172.21.15.122:0/3052874601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 msgr2=0x5621cd358760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.795+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f4f48011910 tx=0x7f4f48014000 comp rx=0 tx=0).stop 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- 172.21.15.122:0/3052874601 shutdown_connections 2024-04-02T13:55:35.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd358ca0 0x5621cd358760 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd3500c0 0x5621cd358220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- 172.21.15.122:0/3052874601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd34f7a0 0x5621cd3649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- 172.21.15.122:0/3052874601 >> 172.21.15.122:0/3052874601 conn(0x5621cd349ed0 msgr2=0x5621cd3543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- 172.21.15.122:0/3052874601 shutdown_connections 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- 172.21.15.122:0/3052874601 wait complete. 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- start start 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 Processor -- start 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd3500c0 0x5621cd352ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd358ca0 0x5621cd35fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501b26e0 con 0x5621cd34f7a0 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501b2010 con 0x5621cd358ca0 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.796+0000 7f4f57dff740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f501af270 con 0x5621cd3500c0 2024-04-02T13:55:35.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f57dff740 10 client.0 _ll_register_callbacks cb 0x7f4f44009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59616/0 (socket says 172.21.15.122:59616) 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/4147143115 learned_addr learned my addr 172.21.15.122:0/4147143115 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4f59e640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd358ca0 0x5621cd35fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd3500c0 0x5621cd352ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/4147143115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd3500c0 msgr2=0x5621cd352ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd3500c0 0x5621cd352ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd358ca0 msgr2=0x5621cd35fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd358ca0 0x5621cd35fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 -- 172.21.15.122:0/4147143115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621cd34fef0 con 0x5621cd34f7a0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f4ed9d640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4f400b9960 tx=0x7f4f40009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9230a3b1297e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f32ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.797+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f400bce60 con 0x5621cd34f7a0 2024-04-02T13:55:35.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621cd2bb040 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f4f400f4970 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4f40005570 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f57dff740 10 client.21267 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5621cd3615b0 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.798+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5621cd2bb040 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f4f400b9b70 con 0x5621cd34f7a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 10 client.21267 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 10 client.21267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 20 client.21267 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd3d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 4 client.21267 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621cd2386b0 con 0x5621cd3690a0 2024-04-02T13:55:35.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.799+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd3d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f4e59c640 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd3d19a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52b5612bb928f628 server_cookie=20f3e0ac52779ad1 in_seq=0 out_seq=0 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f32ff5640 10 client.21267 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f32ff5640 1 -- 172.21.15.122:0/4147143115 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621cd2386b0 con 0x5621cd3690a0 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f32ff5640 10 client.21267 handle_command_reply: tid=0 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 client.21267 shutdown 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 msgr2=0x5621cd352570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f4f400b9960 tx=0x7f4f40009620 comp rx=0 tx=0).stop 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 shutdown_connections 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5621cd3690a0 0x5621cd3d19a0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5621cd358ca0 0x5621cd35fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5621cd3500c0 0x5621cd352ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 --2- 172.21.15.122:0/4147143115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5621cd34f7a0 0x5621cd352570 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f32ff5640 0 client.21267 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:35.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.800+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 >> 172.21.15.122:0/4147143115 conn(0x5621cd349ed0 msgr2=0x5621cd34b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 shutdown_connections 2024-04-02T13:55:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f57dff740 1 -- 172.21.15.122:0/4147143115 wait complete. 2024-04-02T13:55:35.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f57dff740 20 client.21267 trim_cache size 0 max 16384 2024-04-02T13:55:35.809 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 msgr2=0x7f4f380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 0x7f4f380a6260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f48021000 tx=0x7f4f48022000 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 msgr2=0x7f4f5019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.801+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f4f40006010 tx=0x7f4f4002cb30 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 shutdown_connections 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f4f380a3da0 0x7f4f380a6260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f5010e510 0x7f4f5019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f50101df0 0x7f4f5019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 --2- 172.21.15.122:0/3647041702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f501014d0 0x7f4f5019a620 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 >> 172.21.15.122:0/3647041702 conn(0x7f4f50077a90 msgr2=0x7f4f5010f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 shutdown_connections 2024-04-02T13:55:35.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:35.802+0000 7f4f55214640 1 -- 172.21.15.122:0/3647041702 wait complete. 2024-04-02T13:55:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:35 smithi122 ceph-mon[32114]: pgmap v643: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 648 op/s 2024-04-02T13:55:35.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:35 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3169029595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:55:36.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 212231 26.71 MB/sec execute 296 sec latency 143.751 ms 2024-04-02T13:55:36.822 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:55:36.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:55:36.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 -- 172.21.15.122:0/929051356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0100970 msgr2=0x7fd5c0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/929051356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0100970 0x7fd5c0100df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009890 tx=0x7fd5b002bb70 comp rx=0 tx=0).stop 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 -- 172.21.15.122:0/929051356 shutdown_connections 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/929051356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0101330 0x7fd5c010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/929051356 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0100970 0x7fd5c0100df0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/929051356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0106980 0x7fd5c0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 -- 172.21.15.122:0/929051356 >> 172.21.15.122:0/929051356 conn(0x7fd5c0077b00 msgr2=0x7fd5c0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 -- 172.21.15.122:0/929051356 shutdown_connections 2024-04-02T13:55:36.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.972+0000 7fd5c7f59640 1 -- 172.21.15.122:0/929051356 wait complete. 2024-04-02T13:55:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 Processor -- start 2024-04-02T13:55:36.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 -- start start 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 0x7fd5c010e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0101330 0x7fd5c010e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 0x7fd5c010ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0111e90 con 0x7fd5c0100970 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0111d10 con 0x7fd5c0101330 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c7f59640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c0112010 con 0x7fd5c0106980 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 0x7fd5c010e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c54f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0101330 0x7fd5c010e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 0x7fd5c010e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59644/0 (socket says 172.21.15.122:59644) 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/3596824391 learned_addr learned my addr 172.21.15.122:0/3596824391 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.973+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 0x7fd5c010ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 -- 172.21.15.122:0/3596824391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0101330 msgr2=0x7fd5c010e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0101330 0x7fd5c010e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 -- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 msgr2=0x7fd5c010e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 0x7fd5c010e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 -- 172.21.15.122:0/3596824391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c01a68d0 con 0x7fd5c0106980 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 0x7fd5c010ed50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ec50 tx=0x7fd5b800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5b6ffd640 1 -- 172.21.15.122:0/3596824391 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003b70 con 0x7fd5c0106980 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c01a6bc0 con 0x7fd5c0106980 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5b6ffd640 1 -- 172.21.15.122:0/3596824391 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8003d10 con 0x7fd5c0106980 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5b6ffd640 1 -- 172.21.15.122:0/3596824391 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8010890 con 0x7fd5c0106980 2024-04-02T13:55:36.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.974+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c01a70e0 con 0x7fd5c0106980 2024-04-02T13:55:36.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.977+0000 7fd5b6ffd640 1 -- 172.21.15.122:0/3596824391 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd5b8003fc0 con 0x7fd5c0106980 2024-04-02T13:55:36.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 Processor -- start 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 -- start start 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5b6ffd640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 0x7fd5ac0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9359280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5638e92b4770 con 0x5638e93597c0 2024-04-02T13:55:36.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5638e925c960 con 0x5638e9350be0 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.978+0000 7fd5cac21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5638e92b08c0 con 0x5638e93502c0 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58698/0 (socket says 172.21.15.122:58698) 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/74348765 learned_addr learned my addr 172.21.15.122:0/74348765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5b6ffd640 1 -- 172.21.15.122:0/3596824391 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd5b80d1240 con 0x7fd5c0106980 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/74348765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 msgr2=0x5638e9359280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9359280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/74348765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 msgr2=0x5638e935bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/74348765 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e90907e0 con 0x5638e93502c0 2024-04-02T13:55:36.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 0x7fd5ac0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:55:36.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.979+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009d00 tx=0x7fd5bc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=806d0e02edb3a37f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:36.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.980+0000 7fd5abfff640 1 -- 172.21.15.122:0/74348765 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc004230 con 0x5638e93502c0 2024-04-02T13:55:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.980+0000 7fd5abfff640 1 -- 172.21.15.122:0/74348765 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc0043d0 con 0x5638e93502c0 2024-04-02T13:55:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.980+0000 7fd5abfff640 1 -- 172.21.15.122:0/74348765 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0198e0 con 0x5638e93502c0 2024-04-02T13:55:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.980+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 0x7fd5ac0a62e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0000c00 tx=0x7fd5b0005ab0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.980+0000 7fd5cac21740 1 -- 172.21.15.122:0/74348765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 msgr2=0x5638e93506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:36.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009d00 tx=0x7fd5bc011810 comp rx=0 tx=0).stop 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 -- 172.21.15.122:0/74348765 shutdown_connections 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9359280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 --2- 172.21.15.122:0/74348765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93506a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 -- 172.21.15.122:0/74348765 >> 172.21.15.122:0/74348765 conn(0x5638e934a9f0 msgr2=0x5638e934add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.981+0000 7fd5cac21740 1 -- 172.21.15.122:0/74348765 shutdown_connections 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.982+0000 7fd5cac21740 1 -- 172.21.15.122:0/74348765 wait complete. 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.982+0000 7fd5cac21740 1 -- start start 2024-04-02T13:55:36.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.982+0000 7fd5cac21740 1 Processor -- start 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9357220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5638e935c5b0 con 0x5638e93597c0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5638e926fd20 con 0x5638e9350be0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5cac21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5638e935c730 con 0x5638e93502c0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5c54f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.983+0000 7fd5c54f4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35530/0 (socket says 172.21.15.122:35530) 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.984+0000 7fd5c54f4640 1 -- 172.21.15.122:0/4244661616 learned_addr learned my addr 172.21.15.122:0/4244661616 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.984+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.984+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9357220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.984+0000 7fd5cac21740 10 client.0 _ll_register_callbacks cb 0x5638e923ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.985+0000 7fd5c54f4640 1 -- 172.21.15.122:0/4244661616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 msgr2=0x5638e93567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.985+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.985+0000 7fd5c54f4640 1 -- 172.21.15.122:0/4244661616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 msgr2=0x5638e9357220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.985+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9357220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.985+0000 7fd5c54f4640 1 -- 172.21.15.122:0/4244661616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e90f0f70 con 0x5638e9350be0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.986+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0035fd0 tx=0x7fd5b0034d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24cf75a9cd2e688d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.986+0000 7fd5a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.987+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b004e060 con 0x5638e9350be0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.987+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd5b00043d0 con 0x5638e9350be0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.987+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5bc0043d0 con 0x5638e9350be0 2024-04-02T13:55:36.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.988+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd5b0039070 con 0x5638e9350be0 2024-04-02T13:55:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.988+0000 7fd5cac21740 10 client.30167 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5638e8f88170 con 0x5638e9350be0 2024-04-02T13:55:36.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5bc0043d0 con 0x5638e9350be0 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd5b002c5a0 con 0x5638e9350be0 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5cac21740 10 client.30167 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5cac21740 10 client.30167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5cac21740 20 client.30167 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.989+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e9369ab0 0x5638e936bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.990+0000 7fd5cac21740 4 client.30167 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.990+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638e92c1cb0 con 0x5638e9369ab0 2024-04-02T13:55:36.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.990+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e9369ab0 0x5638e936bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:36.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.991+0000 7fd5a97fa640 10 client.30167 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:36.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.991+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e9369ab0 0x5638e936bee0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283f3280f91d48df server_cookie=de3ee04aeaf6ab23 in_seq=0 out_seq=0 2024-04-02T13:55:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.993+0000 7fd5a97fa640 1 -- 172.21.15.122:0/4244661616 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638e92c1cb0 con 0x5638e9369ab0 2024-04-02T13:55:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.993+0000 7fd5a97fa640 10 client.30167 handle_command_reply: tid=0 2024-04-02T13:55:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.993+0000 7fd5cac21740 1 client.30167 shutdown 2024-04-02T13:55:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 msgr2=0x5638e9356ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:36.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0035fd0 tx=0x7fd5b0034d90 comp rx=0 tx=0).stop 2024-04-02T13:55:36.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 shutdown_connections 2024-04-02T13:55:36.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e9369ab0 0x5638e936bee0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9357220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e9350be0 0x5638e9356ce0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4244661616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e93502c0 0x5638e93567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 >> 172.21.15.122:0/4244661616 conn(0x5638e934a9f0 msgr2=0x5638e934c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5a97fa640 0 client.30167 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 shutdown_connections 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.994+0000 7fd5cac21740 1 -- 172.21.15.122:0/4244661616 wait complete. 2024-04-02T13:55:36.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:36.995+0000 7fd5cac21740 20 client.30167 trim_cache size 0 max 16384 2024-04-02T13:55:37.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.028+0000 7fd5cac21740 1 Processor -- start 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.028+0000 7fd5cac21740 1 -- start start 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e9358ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e9365470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9349930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e9358ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5c64f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9349930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5c5cf5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e9358ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35540/0 (socket says 172.21.15.122:35540) 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5c5cf5640 1 -- 172.21.15.122:0/4078729736 learned_addr learned my addr 172.21.15.122:0/4078729736 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5c54f4640 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e9365470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b1300 con 0x5638e93597c0 2024-04-02T13:55:37.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b2e40 con 0x5638e93502c0 2024-04-02T13:55:37.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.029+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b19d0 con 0x5638e9350be0 2024-04-02T13:55:37.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 -- 172.21.15.122:0/4078729736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 msgr2=0x5638e9365470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e9365470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 -- 172.21.15.122:0/4078729736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 msgr2=0x5638e9358ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e9358ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 -- 172.21.15.122:0/4078729736 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638e9355750 con 0x5638e93597c0 2024-04-02T13:55:37.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.030+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9349930 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ce540 tx=0x7fd5b800acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5863dca0a86cab server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.031+0000 7fd5ab7fe640 1 -- 172.21.15.122:0/4078729736 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b809d490 con 0x5638e93597c0 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.031+0000 7fd5ab7fe640 1 -- 172.21.15.122:0/4078729736 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b809cd90 con 0x5638e93597c0 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.031+0000 7fd5ab7fe640 1 -- 172.21.15.122:0/4078729736 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80ce7c0 con 0x5638e93597c0 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 msgr2=0x5638e9349930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9349930 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ce540 tx=0x7fd5b800acd0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 shutdown_connections 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e9349930 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e9365470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 --2- 172.21.15.122:0/4078729736 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e9358ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 >> 172.21.15.122:0/4078729736 conn(0x5638e934a9f0 msgr2=0x5638e9349220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.032+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 shutdown_connections 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.033+0000 7fd5cac21740 1 -- 172.21.15.122:0/4078729736 wait complete. 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.033+0000 7fd5cac21740 1 -- start start 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.033+0000 7fd5cac21740 1 Processor -- start 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e935d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e935daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b2770 con 0x5638e93597c0 2024-04-02T13:55:37.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01b20a0 con 0x5638e93502c0 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c01af300 con 0x5638e9350be0 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5cac21740 10 client.0 _ll_register_callbacks cb 0x7fd5b0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5c54f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e935daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5c54f4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e935daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58716/0 (socket says 172.21.15.122:58716) 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5c64f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.034+0000 7fd5c54f4640 1 -- 172.21.15.122:0/2254700752 learned_addr learned my addr 172.21.15.122:0/2254700752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e935d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 -- 172.21.15.122:0/2254700752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 msgr2=0x5638e935daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e935daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 -- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 msgr2=0x5638e935d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e935d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 -- 172.21.15.122:0/2254700752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638e9350a10 con 0x5638e93597c0 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5c64f6640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935e030 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8007380 tx=0x7fd5b8098990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d2acff3171f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:37.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.035+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80b5020 con 0x5638e93597c0 2024-04-02T13:55:37.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.036+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7fd5b800ad40 con 0x5638e93597c0 2024-04-02T13:55:37.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.038+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5b809cd90 con 0x5638e93597c0 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd5b80a5a00 con 0x5638e93597c0 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 10 client.21285 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5638e93e13b0 con 0x5638e93597c0 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5b809cd90 con 0x5638e93597c0 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd5b807bbb0 con 0x5638e93597c0 2024-04-02T13:55:37.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 10 client.21285 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:37.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 10 client.21285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:37.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 20 client.21285 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:37.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.039+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e93e1970 0x5638e93e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.040+0000 7fd5cac21740 4 client.21285 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:55:37.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.040+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5638e92bc000 con 0x5638e93e1970 2024-04-02T13:55:37.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.040+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e93e1970 0x5638e93e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.041+0000 7fd5a97fa640 10 client.21285 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.041+0000 7fd5c5cf5640 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e93e1970 0x5638e93e3d60 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23e2457d2924d76 server_cookie=631ff60c0226f164 in_seq=0 out_seq=0 2024-04-02T13:55:37.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.043+0000 7fd5a97fa640 1 -- 172.21.15.122:0/2254700752 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5638e92bc000 con 0x5638e93e1970 2024-04-02T13:55:37.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.043+0000 7fd5a97fa640 10 client.21285 handle_command_reply: tid=0 2024-04-02T13:55:37.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.043+0000 7fd5cac21740 1 client.21285 shutdown 2024-04-02T13:55:37.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 msgr2=0x5638e935e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935e030 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8007380 tx=0x7fd5b8098990 comp rx=0 tx=0).stop 2024-04-02T13:55:37.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 shutdown_connections 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5638e93e1970 0x5638e93e3d60 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5638e93597c0 0x5638e935e030 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5a97fa640 0 client.21285 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5638e9350be0 0x5638e935daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 --2- 172.21.15.122:0/2254700752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5638e93502c0 0x5638e935d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.046+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 >> 172.21.15.122:0/2254700752 conn(0x5638e934a9f0 msgr2=0x5638e9349ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.048+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 shutdown_connections 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.048+0000 7fd5cac21740 1 -- 172.21.15.122:0/2254700752 wait complete. 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.049+0000 7fd5cac21740 20 client.21285 trim_cache size 0 max 16384 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:55:37.053 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "160548b1-2039-465d-bfd4-a6e047fda2d8", 2024-04-02T13:55:37.054 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:55:37.054 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 msgr2=0x7fd5ac0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 0x7fd5ac0a62e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0000c00 tx=0x7fd5b0005ab0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 msgr2=0x7fd5c010ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 0x7fd5c010ed50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ec50 tx=0x7fd5b800c6a0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 shutdown_connections 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.052+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd5ac0a3e20 0x7fd5ac0a62e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.053+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5c0106980 0x7fd5c010ed50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.053+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c0101330 0x7fd5c010e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.053+0000 7fd5c7f59640 1 --2- 172.21.15.122:0/3596824391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5c0100970 0x7fd5c010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.053+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 >> 172.21.15.122:0/3596824391 conn(0x7fd5c0077b00 msgr2=0x7fd5c01031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.054+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 shutdown_connections 2024-04-02T13:55:37.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.054+0000 7fd5c7f59640 1 -- 172.21.15.122:0/3596824391 wait complete. 2024-04-02T13:55:37.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:55:37.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:55:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.251+0000 7f94d22fb640 1 -- 172.21.15.122:0/661589558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc105980 msgr2=0x7f94cc105d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.251+0000 7f94d22fb640 1 --2- 172.21.15.122:0/661589558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc105980 0x7f94cc105d60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009900 tx=0x7f94bc02bc50 comp rx=0 tx=0).stop 2024-04-02T13:55:37.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.255+0000 7f94d22fb640 1 -- 172.21.15.122:0/661589558 shutdown_connections 2024-04-02T13:55:37.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.255+0000 7f94d22fb640 1 --2- 172.21.15.122:0/661589558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc10d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.255+0000 7f94d22fb640 1 --2- 172.21.15.122:0/661589558 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc100c70 0x7f94cc1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.255+0000 7f94d22fb640 1 --2- 172.21.15.122:0/661589558 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc105980 0x7f94cc105d60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.255+0000 7f94d22fb640 1 -- 172.21.15.122:0/661589558 >> 172.21.15.122:0/661589558 conn(0x7f94cc0778f0 msgr2=0x7f94cc077d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.257+0000 7f94d22fb640 1 -- 172.21.15.122:0/661589558 shutdown_connections 2024-04-02T13:55:37.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.259+0000 7f94d22fb640 1 -- 172.21.15.122:0/661589558 wait complete. 2024-04-02T13:55:37.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.260+0000 7f94d22fb640 1 Processor -- start 2024-04-02T13:55:37.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.260+0000 7f94d22fb640 1 -- start start 2024-04-02T13:55:37.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc100c70 0x7f94cc1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc105980 0x7f94cc19e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc10fcf0 con 0x7f94cc105980 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc10fb70 con 0x7f94cc100c70 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94d22fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc10fe70 con 0x7f94cc101630 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58724/0 (socket says 172.21.15.122:58724) 2024-04-02T13:55:37.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.261+0000 7f94c3fff640 1 -- 172.21.15.122:0/3387454452 learned_addr learned my addr 172.21.15.122:0/3387454452 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 -- 172.21.15.122:0/3387454452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc100c70 msgr2=0x7f94cc1a4400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc100c70 0x7f94cc1a4400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 -- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc105980 msgr2=0x7f94cc19e600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc105980 0x7f94cc19e600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 -- 172.21.15.122:0/3387454452 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc19ed80 con 0x7f94cc101630 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.262+0000 7f94c3fff640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94b400d950 tx=0x7f94b400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.263+0000 7f94c9ffb640 1 -- 172.21.15.122:0/3387454452 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4017030 con 0x7f94cc101630 2024-04-02T13:55:37.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.263+0000 7f94c9ffb640 1 -- 172.21.15.122:0/3387454452 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b4004020 con 0x7f94cc101630 2024-04-02T13:55:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.263+0000 7f94c9ffb640 1 -- 172.21.15.122:0/3387454452 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b4003d60 con 0x7f94cc101630 2024-04-02T13:55:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.263+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc19f070 con 0x7f94cc101630 2024-04-02T13:55:37.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.263+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc1ab2c0 con 0x7f94cc101630 2024-04-02T13:55:37.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.267+0000 7f94d4fbd740 1 Processor -- start 2024-04-02T13:55:37.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.267+0000 7f94d4fbd740 1 -- start start 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.267+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.267+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928781f0 0x55f092880890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092880dd0 0x55f0928831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f0927db830 con 0x55f0928781f0 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f092783820 con 0x55f092880dd0 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f0927d77e0 con 0x55f0928778d0 2024-04-02T13:55:37.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d0898640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d0898640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58732/0 (socket says 172.21.15.122:58732) 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94d0898640 1 -- 172.21.15.122:0/1570003551 learned_addr learned my addr 172.21.15.122:0/1570003551 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.268+0000 7f94cbfff640 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928781f0 0x55f092880890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 -- 172.21.15.122:0/1570003551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092880dd0 msgr2=0x55f0928831c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092880dd0 0x55f0928831c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 -- 172.21.15.122:0/1570003551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928781f0 msgr2=0x55f092880890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928781f0 0x55f092880890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 -- 172.21.15.122:0/1570003551 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0925b97d0 con 0x55f0928778d0 2024-04-02T13:55:37.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.269+0000 7f94d0898640 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d20 tx=0x7f94c400b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e60373c6426b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.270+0000 7f94c27fc640 1 -- 172.21.15.122:0/1570003551 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c400bd40 con 0x55f0928778d0 2024-04-02T13:55:37.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.270+0000 7f94c27fc640 1 -- 172.21.15.122:0/1570003551 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c4017070 con 0x55f0928778d0 2024-04-02T13:55:37.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.271+0000 7f94d4fbd740 1 -- 172.21.15.122:0/1570003551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 msgr2=0x55f092877cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.271+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f94c4009d20 tx=0x7f94c400b420 comp rx=0 tx=0).stop 2024-04-02T13:55:37.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.272+0000 7f94c9ffb640 1 -- 172.21.15.122:0/3387454452 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f94b401a070 con 0x7f94cc101630 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94c9ffb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 0x7f94b80a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94c9ffb640 1 -- 172.21.15.122:0/3387454452 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f94b40d07f0 con 0x7f94cc101630 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94d4fbd740 1 -- 172.21.15.122:0/1570003551 shutdown_connections 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092880dd0 0x55f0928831c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928781f0 0x55f092880890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/1570003551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928778d0 0x55f092877cb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.273+0000 7f94d4fbd740 1 -- 172.21.15.122:0/1570003551 >> 172.21.15.122:0/1570003551 conn(0x55f092872000 msgr2=0x55f0928723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.274+0000 7f94d0898640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 0x7f94b80a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.274+0000 7f94d4fbd740 1 -- 172.21.15.122:0/1570003551 shutdown_connections 2024-04-02T13:55:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.274+0000 7f94d4fbd740 1 -- 172.21.15.122:0/1570003551 wait complete. 2024-04-02T13:55:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 -- start start 2024-04-02T13:55:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d0898640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 0x7f94b80a63e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94c4012e70 tx=0x7f94c4011390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 Processor -- start 2024-04-02T13:55:37.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f092894c20 con 0x55f0928778d0 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55f092796d30 con 0x55f0928781f0 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f092894da0 con 0x55f092880dd0 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.275+0000 7f94d4fbd740 10 client.0 _ll_register_callbacks cb 0x55f092796720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:37.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59710/0 (socket says 172.21.15.122:59710) 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 -- 172.21.15.122:0/2718485284 learned_addr learned my addr 172.21.15.122:0/2718485284 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94d0898640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 -- 172.21.15.122:0/2718485284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 msgr2=0x55f09287dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 -- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 msgr2=0x55f09287d9a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287d9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 -- 172.21.15.122:0/2718485284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0924ef3f0 con 0x55f0928778d0 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94cbfff640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f94bc035000 tx=0x7f94bc02cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e7cb11dfecf753 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.276+0000 7f94ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.277+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc042030 con 0x55f0928778d0 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.277+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94c4017070 con 0x55f0928778d0 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.277+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f94bc005040 con 0x55f0928778d0 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.277+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f94bc04f050 con 0x55f0928778d0 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.279+0000 7f94d4fbd740 10 client.21294 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:37.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.279+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f0924af0f0 con 0x55f0928778d0 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94c4017070 con 0x55f0928778d0 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f94bc0035f0 con 0x55f0928778d0 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 10 client.21294 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 10 client.21294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 20 client.21294 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0928a1e30 0x55f0928a42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 4 client.21294 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:55:37.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.280+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0927e8d70 con 0x55f0928a1e30 2024-04-02T13:55:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.281+0000 7f94c3fff640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0928a1e30 0x55f0928a42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.281+0000 7f94c3fff640 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0928a1e30 0x55f0928a42a0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57337f250ef40fbd server_cookie=d6b54be6d6429280 in_seq=0 out_seq=0 2024-04-02T13:55:37.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.281+0000 7f94ab7fe640 10 client.21294 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.282+0000 7f94ab7fe640 1 -- 172.21.15.122:0/2718485284 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f0927e8d70 con 0x55f0928a1e30 2024-04-02T13:55:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.282+0000 7f94ab7fe640 10 client.21294 handle_command_reply: tid=0 2024-04-02T13:55:37.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.282+0000 7f94d4fbd740 1 client.21294 shutdown 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 msgr2=0x55f092880610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f94bc035000 tx=0x7f94bc02cad0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 shutdown_connections 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f0928a1e30 0x55f0928a42a0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2718485284 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f0928778d0 0x55f092880610 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 >> 172.21.15.122:0/2718485284 conn(0x55f092872000 msgr2=0x55f092873d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94ab7fe640 0 client.21294 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 shutdown_connections 2024-04-02T13:55:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2718485284 wait complete. 2024-04-02T13:55:37.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.283+0000 7f94d4fbd740 20 client.21294 trim_cache size 0 max 16384 2024-04-02T13:55:37.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.312+0000 7f94d4fbd740 1 Processor -- start 2024-04-02T13:55:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.312+0000 7f94d4fbd740 1 -- start start 2024-04-02T13:55:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 0x55f09289db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59724/0 (socket says 172.21.15.122:59724) 2024-04-02T13:55:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 0x55f09289db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94cbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 0x55f09289db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:58744/0 (socket says 172.21.15.122:58744) 2024-04-02T13:55:37.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94c3fff640 1 -- 172.21.15.122:0/2884549834 learned_addr learned my addr 172.21.15.122:0/2884549834 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.313+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092873b40 0x55f0928a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1aeec0 con 0x55f092880dd0 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1accb0 con 0x55f092873b40 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1ae7f0 con 0x55f0928781f0 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94d0898640 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092873b40 0x55f0928a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94c3fff640 1 -- 172.21.15.122:0/2884549834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 msgr2=0x55f09289db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94c3fff640 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 0x55f09289db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94c3fff640 1 -- 172.21.15.122:0/2884549834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092873b40 msgr2=0x55f0928a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94c3fff640 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092873b40 0x55f0928a9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.314+0000 7f94c3fff640 1 -- 172.21.15.122:0/2884549834 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0928a4610 con 0x55f092880dd0 2024-04-02T13:55:37.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.315+0000 7f94c3fff640 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f94b409c040 tx=0x7f94b40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a62a0d7d8ec2f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.315+0000 7f94c1ffb640 1 -- 172.21.15.122:0/2884549834 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b40b0070 con 0x55f092880dd0 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.315+0000 7f94c1ffb640 1 -- 172.21.15.122:0/2884549834 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b4014040 con 0x55f092880dd0 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.315+0000 7f94c1ffb640 1 -- 172.21.15.122:0/2884549834 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b40989b0 con 0x55f092880dd0 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 msgr2=0x55f0928790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f94b409c040 tx=0x7f94b40035f0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 shutdown_connections 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f092873b40 0x55f0928a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092880dd0 0x55f0928790c0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/2884549834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f0928781f0 0x55f09289db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.316+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 >> 172.21.15.122:0/2884549834 conn(0x55f092872000 msgr2=0x55f092873430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.317+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 shutdown_connections 2024-04-02T13:55:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.317+0000 7f94d4fbd740 1 -- 172.21.15.122:0/2884549834 wait complete. 2024-04-02T13:55:37.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.317+0000 7f94d4fbd740 1 -- start start 2024-04-02T13:55:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 Processor -- start 2024-04-02T13:55:37.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092873b40 0x55f09289a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1ada50 con 0x55f092873b40 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1ad380 con 0x55f0928781f0 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc1ae120 con 0x55f092880dd0 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94d4fbd740 10 client.0 _ll_register_callbacks cb 0x7f94bc0043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94c3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:35594/0 (socket says 172.21.15.122:35594) 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.318+0000 7f94c3fff640 1 -- 172.21.15.122:0/703417448 learned_addr learned my addr 172.21.15.122:0/703417448 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 -- 172.21.15.122:0/703417448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 msgr2=0x55f09287aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 -- 172.21.15.122:0/703417448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092873b40 msgr2=0x55f09289a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:55:37.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092873b40 0x55f09289a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 -- 172.21.15.122:0/703417448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f09287b290 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94c3fff640 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94b40987a0 tx=0x7f94b4097fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb4629ffa9fc4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.319+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b40b0040 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.320+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94b4014040 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.320+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9525+0+0 (secure 0 0 0) 0x7f94b40041d0 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.320+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f94b40a4fb0 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.320+0000 7f94d4fbd740 10 client.30179 fetch_fsmap learned FSMap version 76 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.320+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f092908540 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.321+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94b4014040 con 0x55f0928781f0 2024-04-02T13:55:37.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.321+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f94b4024060 con 0x55f0928781f0 2024-04-02T13:55:37.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.322+0000 7f94d4fbd740 10 client.30179 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:55:37.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.322+0000 7f94d4fbd740 10 client.30179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:55:37.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.322+0000 7f94d4fbd740 20 client.30179 populate_metadata read hostname 'smithi122' 2024-04-02T13:55:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.323+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f092908bb0 0x55f09290afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:55:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.323+0000 7f94d4fbd740 4 client.30179 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:55:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.323+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0927e30c0 con 0x55f092908bb0 2024-04-02T13:55:37.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.324+0000 7f94cbfff640 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f092908bb0 0x55f09290afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:55:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.325+0000 7f94cbfff640 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f092908bb0 0x55f09290afc0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82be2602186c543b server_cookie=369ffd1ea8c6c34b in_seq=0 out_seq=0 2024-04-02T13:55:37.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.325+0000 7f94ab7fe640 10 client.30179 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.325+0000 7f94ab7fe640 1 -- 172.21.15.122:0/703417448 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f94bc04c110 con 0x55f092908bb0 2024-04-02T13:55:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.325+0000 7f94ab7fe640 10 client.30179 handle_command_reply: tid=0 2024-04-02T13:55:37.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.326+0000 7f94d4fbd740 1 client.30179 shutdown 2024-04-02T13:55:37.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.329+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 msgr2=0x55f09287a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.329+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f94b40987a0 tx=0x7f94b4097fe0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 shutdown_connections 2024-04-02T13:55:37.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55f092908bb0 0x55f09290afc0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f092880dd0 0x55f09287aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55f0928781f0 0x55f09287a480 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 --2- 172.21.15.122:0/703417448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f092873b40 0x55f09289a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 >> 172.21.15.122:0/703417448 conn(0x55f092872000 msgr2=0x55f0928709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94ab7fe640 0 client.30179 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.332+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 shutdown_connections 2024-04-02T13:55:37.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.333+0000 7f94d4fbd740 1 -- 172.21.15.122:0/703417448 wait complete. 2024-04-02T13:55:37.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.333+0000 7f94d4fbd740 20 client.30179 trim_cache size 0 max 16384 2024-04-02T13:55:37.337 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:55:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:55:37.337 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "160548b1-2039-465d-bfd4-a6e047fda2d8": { 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "160548b1-2039-465d-bfd4-a6e047fda2d8", 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:55:37.338 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:55:37.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 212604 26.67 MB/sec execute 297 sec latency 144.837 ms 2024-04-02T13:55:37.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.335+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 msgr2=0x7f94b80a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.335+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 0x7f94b80a63e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f94c4012e70 tx=0x7f94c4011390 comp rx=0 tx=0).stop 2024-04-02T13:55:37.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.335+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 msgr2=0x7f94cc1a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:55:37.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.335+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f94b400d950 tx=0x7f94b400de10 comp rx=0 tx=0).stop 2024-04-02T13:55:37.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 shutdown_connections 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f94b80a3f20 0x7f94b80a63e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94cc105980 0x7f94cc19e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94cc101630 0x7f94cc1a4940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 --2- 172.21.15.122:0/3387454452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f94cc100c70 0x7f94cc1a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 >> 172.21.15.122:0/3387454452 conn(0x7f94cc0778f0 msgr2=0x7f94cc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:55:37.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 shutdown_connections 2024-04-02T13:55:37.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:55:37.337+0000 7f94d22fb640 1 -- 172.21.15.122:0/3387454452 wait complete. 2024-04-02T13:55:37.359 INFO:tasks.cephfs.filesystem:scrub status for tag:160548b1-2039-465d-bfd4-a6e047fda2d8 - {'path': '/', 'tag': '160548b1-2039-465d-bfd4-a6e047fda2d8', 'options': 'recursive,force'} 2024-04-02T13:55:37.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:37 smithi187 ceph-mon[36064]: pgmap v644: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-04-02T13:55:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:37 smithi188 ceph-mon[36117]: pgmap v644: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-04-02T13:55:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:37 smithi122 ceph-mon[32114]: pgmap v644: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 584 op/s 2024-04-02T13:55:38.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 213184 26.69 MB/sec execute 298 sec latency 99.392 ms 2024-04-02T13:55:38.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:55:38.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.738 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:55:38.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.739 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:38 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:55:38.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.770 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:38 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:55:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:38.824 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:38 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:55:39.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 213698 26.69 MB/sec execute 299 sec latency 127.889 ms 2024-04-02T13:55:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:39 smithi187 ceph-mon[36064]: pgmap v645: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 758 op/s 2024-04-02T13:55:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:39 smithi188 ceph-mon[36117]: pgmap v645: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 758 op/s 2024-04-02T13:55:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:39 smithi122 ceph-mon[32114]: pgmap v645: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 758 op/s 2024-04-02T13:55:40.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 214294 26.70 MB/sec execute 300 sec latency 74.533 ms 2024-04-02T13:55:41.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 214955 26.72 MB/sec execute 301 sec latency 106.043 ms 2024-04-02T13:55:41.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:41 smithi187 ceph-mon[36064]: pgmap v646: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 551 op/s 2024-04-02T13:55:41.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:41 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:41 smithi188 ceph-mon[36117]: pgmap v646: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 551 op/s 2024-04-02T13:55:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:41 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:41.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:41 smithi122 ceph-mon[32114]: pgmap v646: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.2 MiB/s wr, 551 op/s 2024-04-02T13:55:41.823 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:41 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:55:42.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 215448 26.74 MB/sec execute 302 sec latency 98.212 ms 2024-04-02T13:55:43.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216059 26.75 MB/sec execute 303 sec latency 81.091 ms 2024-04-02T13:55:43.479 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:43 smithi187 ceph-mon[36064]: pgmap v647: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 639 op/s 2024-04-02T13:55:43.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:43 smithi188 ceph-mon[36117]: pgmap v647: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 639 op/s 2024-04-02T13:55:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:43 smithi122 ceph-mon[32114]: pgmap v647: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 639 op/s 2024-04-02T13:55:44.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216325 26.71 MB/sec execute 304 sec latency 519.216 ms 2024-04-02T13:55:45.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216343 26.62 MB/sec execute 305 sec latency 1497.888 ms 2024-04-02T13:55:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:45 smithi187 ceph-mon[36064]: pgmap v648: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-04-02T13:55:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:45 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:45 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:45 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:45 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:45 smithi188 ceph-mon[36117]: pgmap v648: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-04-02T13:55:45.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:45 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:45 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:45 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:45 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.825 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:45 smithi122 ceph-mon[32114]: pgmap v648: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-04-02T13:55:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:45 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:45 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:45 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:45.826 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:45 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216550 26.58 MB/sec execute 306 sec latency 1719.681 ms 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:46 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:46 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi187 to 174.1M 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi187 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi188 to 174.1M 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi188 to 182619136: error parsing value: Value '182619136' is below minimum 939524096 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:46.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:46 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:47.338 INFO:tasks.workunit.client.0.smithi122.stdout: 10 216801 26.55 MB/sec execute 307 sec latency 157.437 ms 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: pgmap v649: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 461 op/s 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:47.637 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: pgmap v649: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 461 op/s 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:47.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: pgmap v649: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 461 op/s 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:55:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:55:48.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217064 26.51 MB/sec execute 308 sec latency 171.318 ms 2024-04-02T13:55:48.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:48 smithi187 ceph-mon[36064]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:55:48.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:48 smithi187 ceph-mon[36064]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:55:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:48 smithi188 ceph-mon[36117]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:55:48.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:48 smithi188 ceph-mon[36117]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:55:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:48 smithi122 ceph-mon[32114]: Adjusting osd_memory_target on smithi122 to 176.1M 2024-04-02T13:55:48.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:48 smithi122 ceph-mon[32114]: Unable to set osd_memory_target on smithi122 to 184728678: error parsing value: Value '184728678' is below minimum 939524096 2024-04-02T13:55:49.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217592 26.50 MB/sec execute 309 sec latency 104.747 ms 2024-04-02T13:55:49.722 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:49 smithi122 ceph-mon[32114]: pgmap v650: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 647 op/s 2024-04-02T13:55:49.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:49 smithi187 ceph-mon[36064]: pgmap v650: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 647 op/s 2024-04-02T13:55:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:49 smithi188 ceph-mon[36117]: pgmap v650: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 647 op/s 2024-04-02T13:55:50.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 217977 26.47 MB/sec execute 310 sec latency 168.618 ms 2024-04-02T13:55:51.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 218576 26.49 MB/sec execute 311 sec latency 180.030 ms 2024-04-02T13:55:51.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:51 smithi187 ceph-mon[36064]: pgmap v651: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 572 op/s 2024-04-02T13:55:51.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:51 smithi188 ceph-mon[36117]: pgmap v651: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 572 op/s 2024-04-02T13:55:51.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:51 smithi122 ceph-mon[32114]: pgmap v651: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 572 op/s 2024-04-02T13:55:52.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 219221 26.51 MB/sec execute 312 sec latency 115.320 ms 2024-04-02T13:55:52.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:52.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:52.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:52.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:52.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:52 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:55:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:52.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:52.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:52.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:52 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:55:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:52.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:52 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:55:53.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 219826 26.56 MB/sec execute 313 sec latency 107.956 ms 2024-04-02T13:55:53.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:53 smithi187 ceph-mon[36064]: pgmap v652: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 714 op/s 2024-04-02T13:55:53.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:53 smithi188 ceph-mon[36117]: pgmap v652: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 714 op/s 2024-04-02T13:55:53.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:53 smithi122 ceph-mon[32114]: pgmap v652: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 714 op/s 2024-04-02T13:55:54.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 220376 26.58 MB/sec execute 314 sec latency 107.452 ms 2024-04-02T13:55:55.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 220755 26.54 MB/sec execute 315 sec latency 110.013 ms 2024-04-02T13:55:55.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:55 smithi187 ceph-mon[36064]: pgmap v653: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-04-02T13:55:55.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:55 smithi188 ceph-mon[36117]: pgmap v653: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-04-02T13:55:55.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:55 smithi122 ceph-mon[32114]: pgmap v653: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 607 op/s 2024-04-02T13:55:56.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 221290 26.55 MB/sec execute 316 sec latency 70.969 ms 2024-04-02T13:55:57.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 221837 26.53 MB/sec execute 317 sec latency 117.071 ms 2024-04-02T13:55:57.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:57 smithi187 ceph-mon[36064]: pgmap v654: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 575 op/s 2024-04-02T13:55:57.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:57 smithi188 ceph-mon[36117]: pgmap v654: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 575 op/s 2024-04-02T13:55:57.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:57 smithi122 ceph-mon[32114]: pgmap v654: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 13 MiB/s wr, 575 op/s 2024-04-02T13:55:58.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 222505 26.56 MB/sec execute 318 sec latency 86.088 ms 2024-04-02T13:55:59.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223181 26.62 MB/sec execute 319 sec latency 168.996 ms 2024-04-02T13:55:59.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:55:59 smithi187 ceph-mon[36064]: pgmap v655: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 700 op/s 2024-04-02T13:55:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:55:59 smithi188 ceph-mon[36117]: pgmap v655: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 700 op/s 2024-04-02T13:55:59.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:55:59 smithi122 ceph-mon[32114]: pgmap v655: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 700 op/s 2024-04-02T13:56:00.339 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223227 26.55 MB/sec execute 320 sec latency 1144.126 ms 2024-04-02T13:56:01.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223344 26.48 MB/sec execute 321 sec latency 1142.034 ms 2024-04-02T13:56:01.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:01 smithi187 ceph-mon[36064]: pgmap v656: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 573 op/s 2024-04-02T13:56:01.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:01 smithi188 ceph-mon[36117]: pgmap v656: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 573 op/s 2024-04-02T13:56:01.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:01 smithi122 ceph-mon[32114]: pgmap v656: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 573 op/s 2024-04-02T13:56:02.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223615 26.45 MB/sec execute 322 sec latency 1087.071 ms 2024-04-02T13:56:03.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 223976 26.42 MB/sec execute 323 sec latency 389.980 ms 2024-04-02T13:56:03.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:03 smithi122 ceph-mon[32114]: pgmap v657: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 675 op/s 2024-04-02T13:56:03.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:03 smithi187 ceph-mon[36064]: pgmap v657: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 675 op/s 2024-04-02T13:56:03.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:03 smithi188 ceph-mon[36117]: pgmap v657: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 675 op/s 2024-04-02T13:56:04.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 224295 26.38 MB/sec execute 324 sec latency 337.096 ms 2024-04-02T13:56:05.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 224468 26.32 MB/sec execute 325 sec latency 390.348 ms 2024-04-02T13:56:05.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:05 smithi187 ceph-mon[36064]: pgmap v658: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 577 op/s 2024-04-02T13:56:05.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:05 smithi188 ceph-mon[36117]: pgmap v658: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 577 op/s 2024-04-02T13:56:05.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:05 smithi122 ceph-mon[32114]: pgmap v658: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 577 op/s 2024-04-02T13:56:06.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 224690 26.28 MB/sec execute 326 sec latency 417.603 ms 2024-04-02T13:56:07.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 224876 26.24 MB/sec execute 327 sec latency 445.445 ms 2024-04-02T13:56:07.360 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:07.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:07.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.536+0000 7f1ad972b640 1 -- 172.21.15.122:0/930304774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad41009c0 msgr2=0x7f1ad4100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.536+0000 7f1ad972b640 1 --2- 172.21.15.122:0/930304774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad41009c0 0x7f1ad4100e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009900 tx=0x7f1ac802bd10 comp rx=0 tx=0).stop 2024-04-02T13:56:07.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.540+0000 7f1ad972b640 1 -- 172.21.15.122:0/930304774 shutdown_connections 2024-04-02T13:56:07.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.540+0000 7f1ad972b640 1 --2- 172.21.15.122:0/930304774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad4101360 0x7f1ad410d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.540+0000 7f1ad972b640 1 --2- 172.21.15.122:0/930304774 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad41009c0 0x7f1ad4100e20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.540+0000 7f1ad972b640 1 --2- 172.21.15.122:0/930304774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad4105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.540+0000 7f1ad972b640 1 -- 172.21.15.122:0/930304774 >> 172.21.15.122:0/930304774 conn(0x7f1ad4077a90 msgr2=0x7f1ad4077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.541+0000 7f1ad972b640 1 -- 172.21.15.122:0/930304774 shutdown_connections 2024-04-02T13:56:07.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.541+0000 7f1ad972b640 1 -- 172.21.15.122:0/930304774 wait complete. 2024-04-02T13:56:07.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.542+0000 7f1ad972b640 1 Processor -- start 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 -- start start 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad41009c0 0x7f1ad40732b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad4101360 0x7f1ad40737f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad410fcd0 con 0x7f1ad41009c0 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad410fb50 con 0x7f1ad4101360 2024-04-02T13:56:07.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.543+0000 7f1ad972b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad410fe50 con 0x7f1ad41058a0 2024-04-02T13:56:07.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.546+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.546+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47068/0 (socket says 172.21.15.122:47068) 2024-04-02T13:56:07.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.546+0000 7f1ad37fe640 1 -- 172.21.15.122:0/145298872 learned_addr learned my addr 172.21.15.122:0/145298872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.546+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad41009c0 0x7f1ad40732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.547+0000 7f1ad37fe640 1 -- 172.21.15.122:0/145298872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad4101360 msgr2=0x7f1ad40737f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.547+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad4101360 0x7f1ad40737f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.547+0000 7f1ad37fe640 1 -- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad41009c0 msgr2=0x7f1ad40732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.547+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad41009c0 0x7f1ad40732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.547+0000 7f1ad37fe640 1 -- 172.21.15.122:0/145298872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad406cdd0 con 0x7f1ad41058a0 2024-04-02T13:56:07.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.548+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cce0 tx=0x7f1acc00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.549+0000 7f1abbfff640 1 -- 172.21.15.122:0/145298872 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00f450 con 0x7f1ad41058a0 2024-04-02T13:56:07.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.549+0000 7f1abbfff640 1 -- 172.21.15.122:0/145298872 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc00fad0 con 0x7f1ad41058a0 2024-04-02T13:56:07.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.549+0000 7f1abbfff640 1 -- 172.21.15.122:0/145298872 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc018c50 con 0x7f1ad41058a0 2024-04-02T13:56:07.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.549+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad406d0c0 con 0x7f1ad41058a0 2024-04-02T13:56:07.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.549+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad4071070 con 0x7f1ad41058a0 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.557+0000 7f1abbfff640 1 -- 172.21.15.122:0/145298872 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1acc00fc80 con 0x7f1ad41058a0 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.558+0000 7f1abbfff640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 0x7f1abc0a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.558+0000 7f1abbfff640 1 -- 172.21.15.122:0/145298872 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f1acc0d0f40 con 0x7f1ad41058a0 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 Processor -- start 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 -- start start 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b31f0 0x5555668bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555566816830 con 0x5555668b28d0 2024-04-02T13:56:07.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555667be820 con 0x5555668bbdd0 2024-04-02T13:56:07.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.559+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5555668127e0 con 0x5555668b31f0 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:46896/0 (socket says 172.21.15.122:46896) 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 -- 172.21.15.122:0/751455667 learned_addr learned my addr 172.21.15.122:0/751455667 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 0x7f1abc0a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 -- 172.21.15.122:0/751455667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b31f0 msgr2=0x5555668bb890 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b31f0 0x5555668bb890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 -- 172.21.15.122:0/751455667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 msgr2=0x5555668b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:07.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.560+0000 7f1ad37fe640 1 -- 172.21.15.122:0/751455667 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555665f47d0 con 0x5555668bbdd0 2024-04-02T13:56:07.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.562+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 0x7f1abc0a63b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4006fd0 tx=0x7f1ac400a0d0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.563+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1acc099410 tx=0x7f1acc09ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79ad08ae377792ff server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.568+0000 7f1ab97fa640 1 -- 172.21.15.122:0/751455667 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0bd070 con 0x5555668bbdd0 2024-04-02T13:56:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.568+0000 7f1ab97fa640 1 -- 172.21.15.122:0/751455667 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc09ad40 con 0x5555668bbdd0 2024-04-02T13:56:07.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.568+0000 7f1ab97fa640 1 -- 172.21.15.122:0/751455667 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc004b40 con 0x5555668bbdd0 2024-04-02T13:56:07.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.570+0000 7f1adc3ef740 1 -- 172.21.15.122:0/751455667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 msgr2=0x5555668be1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.571+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1acc099410 tx=0x7f1acc09ca40 comp rx=0 tx=0).stop 2024-04-02T13:56:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.574+0000 7f1adc3ef740 1 -- 172.21.15.122:0/751455667 shutdown_connections 2024-04-02T13:56:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.574+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668bbdd0 0x5555668be1c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.574+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b31f0 0x5555668bb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.574+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/751455667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.574+0000 7f1adc3ef740 1 -- 172.21.15.122:0/751455667 >> 172.21.15.122:0/751455667 conn(0x5555668ad000 msgr2=0x5555668ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.575+0000 7f1adc3ef740 1 -- 172.21.15.122:0/751455667 shutdown_connections 2024-04-02T13:56:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.579+0000 7f1adc3ef740 1 -- 172.21.15.122:0/751455667 wait complete. 2024-04-02T13:56:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.579+0000 7f1adc3ef740 1 -- start start 2024-04-02T13:56:07.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.579+0000 7f1adc3ef740 1 Processor -- start 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b28d0 0x5555668bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5555668e0bd0 con 0x5555668b31f0 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5555667d1d30 con 0x5555668b28d0 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5555668e0d50 con 0x5555668bbdd0 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1adc3ef740 10 client.0 _ll_register_callbacks cb 0x5555667d1720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1ad27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1ad27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53804/0 (socket says 172.21.15.122:53804) 2024-04-02T13:56:07.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1ad27fc640 1 -- 172.21.15.122:0/3437674620 learned_addr learned my addr 172.21.15.122:0/3437674620 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.581+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b28d0 0x5555668bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 -- 172.21.15.122:0/3437674620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 msgr2=0x5555668b8f90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668b8f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 -- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b28d0 msgr2=0x5555668bb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b28d0 0x5555668bb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 -- 172.21.15.122:0/3437674620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55556652a3f0 con 0x5555668b31f0 2024-04-02T13:56:07.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.582+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005940 tx=0x7f1ac80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e39045f09ddc24 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.588+0000 7f1ab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:07.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.588+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8064020 con 0x5555668b31f0 2024-04-02T13:56:07.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.588+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f1ac8004240 con 0x5555668b31f0 2024-04-02T13:56:07.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.590+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1acc09ad40 con 0x5555668b31f0 2024-04-02T13:56:07.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.592+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1ac8004440 con 0x5555668b31f0 2024-04-02T13:56:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.593+0000 7f1adc3ef740 10 client.21303 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.593+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5555664ea0f0 con 0x5555668b31f0 2024-04-02T13:56:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc09ad40 con 0x5555668b31f0 2024-04-02T13:56:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac805f8b0 con 0x5555668b31f0 2024-04-02T13:56:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 10 client.21303 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 10 client.21303 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 20 client.21303 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5555668edd80 0x5555668f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 4 client.21303 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.597+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555566823d70 con 0x5555668edd80 2024-04-02T13:56:07.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.599+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5555668edd80 0x5555668f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.602+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5555668edd80 0x5555668f0200 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea79fb08b2bbac9b server_cookie=26d57076723fd0e3 in_seq=0 out_seq=0 2024-04-02T13:56:07.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.603+0000 7f1ab6ff5640 10 client.21303 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:07.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.603+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/3437674620 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555566823d70 con 0x5555668edd80 2024-04-02T13:56:07.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.603+0000 7f1ab6ff5640 10 client.21303 handle_command_reply: tid=0 2024-04-02T13:56:07.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.603+0000 7f1adc3ef740 1 client.21303 shutdown 2024-04-02T13:56:07.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 msgr2=0x5555668b8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005940 tx=0x7f1ac80056d0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 shutdown_connections 2024-04-02T13:56:07.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5555668edd80 0x5555668f0200 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668b8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b31f0 0x5555668b8a50 secure :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005940 tx=0x7f1ac80056d0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/3437674620 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b28d0 0x5555668bb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 >> 172.21.15.122:0/3437674620 conn(0x5555668ad000 msgr2=0x5555668aed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.607+0000 7f1ab6ff5640 0 client.21303 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.613+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 shutdown_connections 2024-04-02T13:56:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.614+0000 7f1adc3ef740 1 -- 172.21.15.122:0/3437674620 wait complete. 2024-04-02T13:56:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.614+0000 7f1adc3ef740 20 client.21303 trim_cache size 0 max 16384 2024-04-02T13:56:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.659+0000 7f1adc3ef740 1 Processor -- start 2024-04-02T13:56:07.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.659+0000 7f1adc3ef740 1 -- start start 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b28d0 0x5555668e9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668bbdd0 0x5555668b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0010250 con 0x5555668bbdd0 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0011d90 con 0x5555668b4360 2024-04-02T13:56:07.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:46912/0 (socket says 172.21.15.122:46912) 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1ad37fe640 1 -- 172.21.15.122:0/672498425 learned_addr learned my addr 172.21.15.122:0/672498425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.660+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0010920 con 0x5555668b28d0 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668bbdd0 0x5555668b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad37fe640 1 -- 172.21.15.122:0/672498425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b28d0 msgr2=0x5555668e9af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:07.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b28d0 0x5555668e9af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad37fe640 1 -- 172.21.15.122:0/672498425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668bbdd0 msgr2=0x5555668b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668bbdd0 0x5555668b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.661+0000 7f1ad37fe640 1 -- 172.21.15.122:0/672498425 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555668ace30 con 0x5555668b4360 2024-04-02T13:56:07.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.662+0000 7f1ad37fe640 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1acc09c9d0 tx=0x7f1acc007670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d35fe23daa804795 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.663+0000 7f1ab8ff9640 1 -- 172.21.15.122:0/672498425 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1acc0aa070 con 0x5555668b4360 2024-04-02T13:56:07.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.663+0000 7f1ab8ff9640 1 -- 172.21.15.122:0/672498425 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0990b0 con 0x5555668b4360 2024-04-02T13:56:07.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.664+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 msgr2=0x5555668bb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.664+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1acc09c9d0 tx=0x7f1acc007670 comp rx=0 tx=0).stop 2024-04-02T13:56:07.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.666+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 shutdown_connections 2024-04-02T13:56:07.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.666+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668bb180 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.666+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668bbdd0 0x5555668b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.666+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/672498425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668b28d0 0x5555668e9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.666+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 >> 172.21.15.122:0/672498425 conn(0x5555668ad000 msgr2=0x5555668b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.668+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 shutdown_connections 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 -- 172.21.15.122:0/672498425 wait complete. 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 -- start start 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 Processor -- start 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668e1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668e2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac00116c0 con 0x5555668b28d0 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac0010ff0 con 0x5555668b4360 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac000e250 con 0x5555668bbdd0 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.669+0000 7f1adc3ef740 10 client.0 _ll_register_callbacks cb 0x7f1ac802cd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:47090/0 (socket says 172.21.15.122:47090) 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad37fe640 1 -- 172.21.15.122:0/734808923 learned_addr learned my addr 172.21.15.122:0/734808923 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668e1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 -- 172.21.15.122:0/734808923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 msgr2=0x5555668e2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:07.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668e2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 -- 172.21.15.122:0/734808923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 msgr2=0x5555668e19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:07.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.670+0000 7f1ad27fc640 1 -- 172.21.15.122:0/734808923 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555566953340 con 0x5555668b4360 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.671+0000 7f1ad27fc640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668e1ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5555668b9b10 tx=0x7f1ac8060000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=328b5e886da9e41a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.671+0000 7f1ab6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.671+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8076070 con 0x5555668b4360 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.671+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1acc0990b0 con 0x5555668b4360 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.672+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f1ac805f860 con 0x5555668b4360 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.672+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1ac807e050 con 0x5555668b4360 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.672+0000 7f1adc3ef740 10 client.30194 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:07.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.672+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5555669545b0 con 0x5555668b4360 2024-04-02T13:56:07.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.674+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1acc0990b0 con 0x5555668b4360 2024-04-02T13:56:07.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.674+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f1ac8079030 con 0x5555668b4360 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 10 client.30194 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 10 client.30194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 20 client.30194 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555566954c80 0x555566957070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 4 client.30194 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:07.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.677+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55556681e0c0 con 0x555566954c80 2024-04-02T13:56:07.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.680+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555566954c80 0x555566957070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.692+0000 7f1ad2ffd640 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555566954c80 0x555566957070 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7025f56bd791b58 server_cookie=e540ed357e2aaf42 in_seq=0 out_seq=0 2024-04-02T13:56:07.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.693+0000 7f1ab6ff5640 10 client.30194 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:07.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.696+0000 7f1ab6ff5640 1 -- 172.21.15.122:0/734808923 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55556681e0c0 con 0x555566954c80 2024-04-02T13:56:07.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.696+0000 7f1ab6ff5640 10 client.30194 handle_command_reply: tid=0 2024-04-02T13:56:07.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.697+0000 7f1adc3ef740 1 client.30194 shutdown 2024-04-02T13:56:07.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.697+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 msgr2=0x5555668e1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.697+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668e1ee0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5555668b9b10 tx=0x7f1ac8060000 comp rx=0 tx=0).stop 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 shutdown_connections 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x555566954c80 0x555566957070 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5555668bbdd0 0x5555668e2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5555668b4360 0x5555668e1ee0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 --2- 172.21.15.122:0/734808923 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5555668b28d0 0x5555668e19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 >> 172.21.15.122:0/734808923 conn(0x5555668ad000 msgr2=0x5555668ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1ab6ff5640 0 client.30194 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 shutdown_connections 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 1 -- 172.21.15.122:0/734808923 wait complete. 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.698+0000 7f1adc3ef740 20 client.30194 trim_cache size 0 max 16384 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:56:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:56:07.704 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 msgr2=0x7f1abc0a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 0x7f1abc0a63b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4006fd0 tx=0x7f1ac400a0d0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 msgr2=0x7f1ad406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00cce0 tx=0x7f1acc00a620 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 shutdown_connections 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f1abc0a3ef0 0x7f1abc0a63b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ad41058a0 0x7f1ad406c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.700+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1ad4101360 0x7f1ad40737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.701+0000 7f1ad972b640 1 --2- 172.21.15.122:0/145298872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1ad41009c0 0x7f1ad40732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.701+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 >> 172.21.15.122:0/145298872 conn(0x7f1ad4077a90 msgr2=0x7f1ad40fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.701+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 shutdown_connections 2024-04-02T13:56:07.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.701+0000 7f1ad972b640 1 -- 172.21.15.122:0/145298872 wait complete. 2024-04-02T13:56:07.727 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:07 smithi122 ceph-mon[32114]: pgmap v659: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-04-02T13:56:07.728 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:56:07.728 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:56:07.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:07 smithi187 ceph-mon[36064]: pgmap v659: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-04-02T13:56:07.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:07 smithi188 ceph-mon[36117]: pgmap v659: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 11 MiB/s wr, 496 op/s 2024-04-02T13:56:07.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.898+0000 7f9cc5a19640 1 -- 172.21.15.122:0/4049316819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0100790 msgr2=0x7f9cc0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.898+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/4049316819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0100790 0x7f9cc0100c10 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009930 tx=0x7f9cb802bc20 comp rx=0 tx=0).stop 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.899+0000 7f9cc5a19640 1 -- 172.21.15.122:0/4049316819 shutdown_connections 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.899+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/4049316819 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0101150 0x7f9cc010d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.899+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/4049316819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0100790 0x7f9cc0100c10 secure :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8009930 tx=0x7f9cb802bc20 comp rx=0 tx=0).stop 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.899+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/4049316819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 0x7f9cc0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.899+0000 7f9cc5a19640 1 -- 172.21.15.122:0/4049316819 >> 172.21.15.122:0/4049316819 conn(0x7f9cc00fc2a0 msgr2=0x7f9cc00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:07.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.903+0000 7f9cc5a19640 1 -- 172.21.15.122:0/4049316819 shutdown_connections 2024-04-02T13:56:07.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.903+0000 7f9cc5a19640 1 -- 172.21.15.122:0/4049316819 wait complete. 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.904+0000 7f9cc5a19640 1 Processor -- start 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.904+0000 7f9cc5a19640 1 -- start start 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 0x7f9cc0110440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 0x7f9cc0110e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc010ec60 con 0x7f9cc0101150 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc010eae0 con 0x7f9cc0110980 2024-04-02T13:56:07.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5a19640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc010ede0 con 0x7f9cc01067a0 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc4a17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc4a17640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:53834/0 (socket says 172.21.15.122:53834) 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc4a17640 1 -- 172.21.15.122:0/3581660754 learned_addr learned my addr 172.21.15.122:0/3581660754 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cc5218640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 0x7f9cc0110e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.905+0000 7f9cbffff640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 0x7f9cc0110440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 -- 172.21.15.122:0/3581660754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 msgr2=0x7f9cc0110440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.910 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 0x7f9cc0110440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 -- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 msgr2=0x7f9cc01163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 -- 172.21.15.122:0/3581660754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01b38a0 con 0x7f9cc0110980 2024-04-02T13:56:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc4a17640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.906+0000 7f9cc5218640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 0x7f9cc0110e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d680 tx=0x7f9ca800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.908+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8017070 con 0x7f9cc0110980 2024-04-02T13:56:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.908+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc0117950 con 0x7f9cc0110980 2024-04-02T13:56:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.908+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ca8009820 con 0x7f9cc0110980 2024-04-02T13:56:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.908+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc0117e70 con 0x7f9cc0110980 2024-04-02T13:56:07.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.908+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8004e00 con 0x7f9cc0110980 2024-04-02T13:56:07.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.909+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc0101ed0 con 0x7f9cc0110980 2024-04-02T13:56:07.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.911+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f9ca8005220 con 0x7f9cc0110980 2024-04-02T13:56:07.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.911+0000 7f9cbdffb640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 0x7f9cac0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.912+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f9ca809a030 con 0x7f9cc0110980 2024-04-02T13:56:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.912+0000 7f9cc4a17640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 0x7f9cac0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.913+0000 7f9cc4a17640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 0x7f9cac0a6360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009fd0 tx=0x7f9cb0009720 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:07.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:07.921+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f9ca80a0290 con 0x7f9cc0110980 2024-04-02T13:56:08.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.313+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc0108d00 con 0x7f9cc0110980 2024-04-02T13:56:08.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.313+0000 7f9cbdffb640 1 -- 172.21.15.122:0/3581660754 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11942 (secure 0 0 0) 0x7f9ca80a4b50 con 0x7f9cc0110980 2024-04-02T13:56:08.317 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:56:08.318 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:55:07.726900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:56:08.318 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-04-02T13:56:08.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.317+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 msgr2=0x7f9cac0a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.317+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 0x7f9cac0a6360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009fd0 tx=0x7f9cb0009720 comp rx=0 tx=0).stop 2024-04-02T13:56:08.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.317+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 msgr2=0x7f9cc0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.317+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 0x7f9cc0110e10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d680 tx=0x7f9ca800db40 comp rx=0 tx=0).stop 2024-04-02T13:56:08.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 shutdown_connections 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f9cac0a3ea0 0x7f9cac0a6360 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9cc0110980 0x7f9cc0110e10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc01067a0 0x7f9cc0110440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 --2- 172.21.15.122:0/3581660754 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9cc0101150 0x7f9cc01163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 >> 172.21.15.122:0/3581660754 conn(0x7f9cc00fc2a0 msgr2=0x7f9cc00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 shutdown_connections 2024-04-02T13:56:08.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.318+0000 7f9cc5a19640 1 -- 172.21.15.122:0/3581660754 wait complete. 2024-04-02T13:56:08.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:56:08.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 225108 26.21 MB/sec execute 328 sec latency 734.273 ms 2024-04-02T13:56:08.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 -- 172.21.15.122:0/1087472586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 msgr2=0x7fbac409c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 --2- 172.21.15.122:0/1087472586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac409c1f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009900 tx=0x7fbabc02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:08.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 -- 172.21.15.122:0/1087472586 shutdown_connections 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 --2- 172.21.15.122:0/1087472586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4005670 0x7fbac40a2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 --2- 172.21.15.122:0/1087472586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4004cb0 0x7fbac4005130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 --2- 172.21.15.122:0/1087472586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac409c1f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 -- 172.21.15.122:0/1087472586 >> 172.21.15.122:0/1087472586 conn(0x7fbac400a740 msgr2=0x7fbac400ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 -- 172.21.15.122:0/1087472586 shutdown_connections 2024-04-02T13:56:08.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.489+0000 7fbacb59e640 1 -- 172.21.15.122:0/1087472586 wait complete. 2024-04-02T13:56:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 Processor -- start 2024-04-02T13:56:08.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 -- start start 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4004cb0 0x7fbac412c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac4130df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40a60e0 con 0x7fbac4005670 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40a5f60 con 0x7fbac409be10 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbac9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbac9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36802/0 (socket says 172.21.15.122:36802) 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbac9d9b640 1 -- 172.21.15.122:0/309928000 learned_addr learned my addr 172.21.15.122:0/309928000 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacad9d640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac4130df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbaca59c640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4004cb0 0x7fbac412c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.491+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbac40a6260 con 0x7fbac4004cb0 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 -- 172.21.15.122:0/309928000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4004cb0 msgr2=0x7fbac412c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4004cb0 0x7fbac412c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 -- 172.21.15.122:0/309928000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 msgr2=0x7fbac4130df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac4130df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 -- 172.21.15.122:0/309928000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac4131520 con 0x7fbac4005670 2024-04-02T13:56:08.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbacad9d640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac4130df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.492+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fbac000ea60 tx=0x7fbac000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.493+0000 7fbabb7fe640 1 -- 172.21.15.122:0/309928000 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac000cd20 con 0x7fbac4005670 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.493+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac4131810 con 0x7fbac4005670 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.493+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac41390a0 con 0x7fbac4005670 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.493+0000 7fbabb7fe640 1 -- 172.21.15.122:0/309928000 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac00035f0 con 0x7fbac4005670 2024-04-02T13:56:08.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.493+0000 7fbabb7fe640 1 -- 172.21.15.122:0/309928000 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac0010690 con 0x7fbac4005670 2024-04-02T13:56:08.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.496+0000 7fbad39de740 1 Processor -- start 2024-04-02T13:56:08.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.496+0000 7fbad39de740 1 -- start start 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.496+0000 7fbabb7fe640 1 -- 172.21.15.122:0/309928000 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbac0010830 con 0x7fbac4005670 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0f2a0 0x55beb9a17940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55beb99727b0 con 0x55beb9a17e80 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55beb9911570 con 0x55beb9a0e980 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbad39de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55beb996e760 con 0x55beb9a0f2a0 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbaca59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0f2a0 0x55beb9a17940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36818/0 (socket says 172.21.15.122:36818) 2024-04-02T13:56:08.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 -- 172.21.15.122:0/867463457 learned_addr learned my addr 172.21.15.122:0/867463457 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbabb7fe640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 0x7fbab40a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbacad9d640 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 -- 172.21.15.122:0/867463457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0f2a0 msgr2=0x55beb9a17940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0f2a0 0x55beb9a17940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 -- 172.21.15.122:0/867463457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 msgr2=0x55beb9a0ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.497+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a0ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbac9d9b640 1 -- 172.21.15.122:0/867463457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb96863f0 con 0x55beb9a17e80 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbacad9d640 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a0ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbabb7fe640 1 -- 172.21.15.122:0/309928000 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fbac0010430 con 0x7fbac4005670 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbacad9d640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 0x7fbab40a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbac00cdea0 tx=0x7fbac0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5834fef09df1f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbab8ff9640 1 -- 172.21.15.122:0/867463457 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac009c550 con 0x55beb9a17e80 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.498+0000 7fbab8ff9640 1 -- 172.21.15.122:0/867463457 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac0010670 con 0x55beb9a17e80 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.499+0000 7fbab8ff9640 1 -- 172.21.15.122:0/867463457 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac009c0e0 con 0x55beb9a17e80 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.499+0000 7fbacad9d640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 0x7fbab40a63e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbacc067230 tx=0x7fbacc069040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 -- 172.21.15.122:0/867463457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 msgr2=0x55beb9a1a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbac00cdea0 tx=0x7fbac0020000 comp rx=0 tx=0).stop 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 -- 172.21.15.122:0/867463457 shutdown_connections 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a1a270 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0f2a0 0x55beb9a17940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 --2- 172.21.15.122:0/867463457 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a0ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 -- 172.21.15.122:0/867463457 >> 172.21.15.122:0/867463457 conn(0x55beb9a090b0 msgr2=0x55beb9a09490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 -- 172.21.15.122:0/867463457 shutdown_connections 2024-04-02T13:56:08.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.500+0000 7fbad39de740 1 -- 172.21.15.122:0/867463457 wait complete. 2024-04-02T13:56:08.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.501+0000 7fbad39de740 1 -- start start 2024-04-02T13:56:08.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.501+0000 7fbad39de740 1 Processor -- start 2024-04-02T13:56:08.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a14bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55beb9a1ac10 con 0x55beb9a17e80 2024-04-02T13:56:08.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55beb992de80 con 0x55beb9a0f2a0 2024-04-02T13:56:08.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55beb9a1ad90 con 0x55beb9a0e980 2024-04-02T13:56:08.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbad39de740 10 client.0 _ll_register_callbacks cb 0x55beb9969aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:08.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbaca59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbaca59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38282/0 (socket says 172.21.15.122:38282) 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbaca59c640 1 -- 172.21.15.122:0/1066898566 learned_addr learned my addr 172.21.15.122:0/1066898566 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.504+0000 7fbacad9d640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 -- 172.21.15.122:0/1066898566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 msgr2=0x55beb9a14bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a14bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a14bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 -- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 msgr2=0x55beb9a15130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a15130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 -- 172.21.15.122:0/1066898566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb97507d0 con 0x55beb9a0e980 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbacad9d640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbaca59c640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009230 tx=0x7fbabc02c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e035a13b4105842c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02ccc0 con 0x55beb9a0e980 2024-04-02T13:56:08.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55beb997fcf0 con 0x55beb9a0e980 2024-04-02T13:56:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.505+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fbabc0046c0 con 0x55beb9a0e980 2024-04-02T13:56:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.506+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbabc0048e0 con 0x55beb9a0e980 2024-04-02T13:56:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.506+0000 7fbad39de740 10 client.29980 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:08.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.506+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55beb9a2f460 con 0x55beb9a0e980 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.506+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55beb997fcf0 con 0x55beb9a0e980 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fbabc046070 con 0x55beb9a0e980 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 10 client.29980 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 10 client.29980 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 20 client.29980 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a30e20 0x55beb9a332c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 4 client.29980 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beb997a040 con 0x55beb9a30e20 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a30e20 0x55beb9a332c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a30e20 0x55beb9a332c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9cb062938da2254 server_cookie=46938ab5e6a2899a in_seq=0 out_seq=0 2024-04-02T13:56:08.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.507+0000 7fbab1ffb640 10 client.29980 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.511+0000 7fbab1ffb640 1 -- 172.21.15.122:0/1066898566 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55beb997a040 con 0x55beb9a30e20 2024-04-02T13:56:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.511+0000 7fbab1ffb640 10 client.29980 handle_command_reply: tid=0 2024-04-02T13:56:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.511+0000 7fbad39de740 1 client.29980 shutdown 2024-04-02T13:56:08.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 msgr2=0x55beb9a17600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbabc009230 tx=0x7fbabc02c7e0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 shutdown_connections 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a30e20 0x55beb9a332c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a17e80 0x55beb9a15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a14bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 --2- 172.21.15.122:0/1066898566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a0e980 0x55beb9a17600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.512+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 >> 172.21.15.122:0/1066898566 conn(0x55beb9a090b0 msgr2=0x55beb9a0ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.513+0000 7fbab1ffb640 0 client.29980 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.513+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 shutdown_connections 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.513+0000 7fbad39de740 1 -- 172.21.15.122:0/1066898566 wait complete. 2024-04-02T13:56:08.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.513+0000 7fbad39de740 20 client.29980 trim_cache size 0 max 16384 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.538+0000 7fbad39de740 1 Processor -- start 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.538+0000 7fbad39de740 1 -- start start 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.538+0000 7fbad39de740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a17320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.538+0000 7fbad39de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbad39de740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbad39de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac010120 con 0x55beb9a0f2a0 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbac9d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbad39de740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac011c60 con 0x55beb9a0e980 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbacad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbacad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38292/0 (socket says 172.21.15.122:38292) 2024-04-02T13:56:08.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbacad9d640 1 -- 172.21.15.122:0/2530201721 learned_addr learned my addr 172.21.15.122:0/2530201721 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.539+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac0107f0 con 0x55beb9a17e80 2024-04-02T13:56:08.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 -- 172.21.15.122:0/2530201721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 msgr2=0x55beb9a38bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 -- 172.21.15.122:0/2530201721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 msgr2=0x55beb9a17320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a17320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 -- 172.21.15.122:0/2530201721 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb9a0f0d0 con 0x55beb9a0f2a0 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbacad9d640 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13bd0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x55beb9a0a240 tx=0x7fbac00d44c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32337b0adb125a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbab3fff640 1 -- 172.21.15.122:0/2530201721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac009c2c0 con 0x55beb9a0f2a0 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbab3fff640 1 -- 172.21.15.122:0/2530201721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac00d9070 con 0x55beb9a0f2a0 2024-04-02T13:56:08.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.540+0000 7fbab3fff640 1 -- 172.21.15.122:0/2530201721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac00a0b30 con 0x55beb9a0f2a0 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.541+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 msgr2=0x55beb9a13bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.541+0000 7fbad39de740 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13bd0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x55beb9a0a240 tx=0x7fbac00d44c0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 shutdown_connections 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a38bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13bd0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 --2- 172.21.15.122:0/2530201721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0e980 0x55beb9a17320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 >> 172.21.15.122:0/2530201721 conn(0x55beb9a090b0 msgr2=0x55beb9a134c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 shutdown_connections 2024-04-02T13:56:08.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 -- 172.21.15.122:0/2530201721 wait complete. 2024-04-02T13:56:08.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 -- start start 2024-04-02T13:56:08.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.542+0000 7fbad39de740 1 Processor -- start 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36836/0 (socket says 172.21.15.122:36836) 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 -- 172.21.15.122:0/3419247698 learned_addr learned my addr 172.21.15.122:0/3419247698 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a14310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac011590 con 0x55beb9a0e980 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac010ec0 con 0x55beb9a0f2a0 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac00e120 con 0x55beb9a17e80 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbad39de740 10 client.0 _ll_register_callbacks cb 0x7fbabc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbacad9d640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a14310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 -- 172.21.15.122:0/3419247698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 msgr2=0x55beb9a14310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a14310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 -- 172.21.15.122:0/3419247698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 msgr2=0x55beb9a13dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.543+0000 7fbaca59c640 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb9a2bde0 con 0x55beb9a0e980 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbaca59c640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006010 tx=0x7fbabc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd1a460d1ba9902 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbab1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbabc02cac0 con 0x55beb9a0e980 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fbabc02cc60 con 0x55beb9a0e980 2024-04-02T13:56:08.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbac00d9070 con 0x55beb9a0e980 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fbabc004a80 con 0x55beb9a0e980 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbad39de740 10 client.21327 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.544+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55beb9a2d0c0 con 0x55beb9a0e980 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbac00d9070 con 0x55beb9a0e980 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fbabc046040 con 0x55beb9a0e980 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbad39de740 10 client.21327 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbad39de740 10 client.21327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbad39de740 20 client.21327 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.545+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a2d8c0 0x55beb9a99450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.546+0000 7fbad39de740 4 client.21327 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:08.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.546+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55beb98f76b0 con 0x55beb9a2d8c0 2024-04-02T13:56:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.546+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a2d8c0 0x55beb9a99450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.546+0000 7fbab1ffb640 10 client.21327 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.546+0000 7fbac9d9b640 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a2d8c0 0x55beb9a99450 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b102da1ca072647f server_cookie=9320243bf9d78fcd in_seq=0 out_seq=0 2024-04-02T13:56:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.548+0000 7fbab1ffb640 1 -- 172.21.15.122:0/3419247698 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55beb98f76b0 con 0x55beb9a2d8c0 2024-04-02T13:56:08.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.548+0000 7fbab1ffb640 10 client.21327 handle_command_reply: tid=0 2024-04-02T13:56:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.548+0000 7fbad39de740 1 client.21327 shutdown 2024-04-02T13:56:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.548+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 msgr2=0x55beb9a13890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.549+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006010 tx=0x7fbabc009520 comp rx=0 tx=0).stop 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 shutdown_connections 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55beb9a2d8c0 0x55beb9a99450 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55beb9a17e80 0x55beb9a14310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55beb9a0f2a0 0x55beb9a13dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 --2- 172.21.15.122:0/3419247698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55beb9a0e980 0x55beb9a13890 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 >> 172.21.15.122:0/3419247698 conn(0x55beb9a090b0 msgr2=0x55beb9a0ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbab1ffb640 0 client.21327 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:08.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 shutdown_connections 2024-04-02T13:56:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.550+0000 7fbad39de740 1 -- 172.21.15.122:0/3419247698 wait complete. 2024-04-02T13:56:08.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbad39de740 20 client.21327 trim_cache size 0 max 16384 2024-04-02T13:56:08.555 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 msgr2=0x7fbab40a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 0x7fbab40a63e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbacc067230 tx=0x7fbacc069040 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 msgr2=0x7fbac412cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fbac000ea60 tx=0x7fbac000ee60 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 shutdown_connections 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbab40a3f20 0x7fbab40a63e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbac409be10 0x7fbac4130df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbac4005670 0x7fbac412cac0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 --2- 172.21.15.122:0/309928000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbac4004cb0 0x7fbac412c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.551+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 >> 172.21.15.122:0/309928000 conn(0x7fbac400a740 msgr2=0x7fbac4093a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.552+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 shutdown_connections 2024-04-02T13:56:08.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.552+0000 7fbacb59e640 1 -- 172.21.15.122:0/309928000 wait complete. 2024-04-02T13:56:08.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:56:08.605 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:08 smithi122 ceph-mon[32114]: osdmap e124: 12 total, 12 up, 12 in 2024-04-02T13:56:08.605 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3581660754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:08.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.723+0000 7ff2b1104640 1 -- 172.21.15.122:0/894988072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac1048c0 msgr2=0x7ff2ac108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.723+0000 7ff2b1104640 1 --2- 172.21.15.122:0/894988072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac1048c0 0x7ff2ac108900 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009ca0 tx=0x7ff2a402c0f0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 -- 172.21.15.122:0/894988072 shutdown_connections 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 --2- 172.21.15.122:0/894988072 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac108fe0 0x7ff2ac110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 --2- 172.21.15.122:0/894988072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac1048c0 0x7ff2ac108900 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 --2- 172.21.15.122:0/894988072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac103f10 0x7ff2ac1042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 -- 172.21.15.122:0/894988072 >> 172.21.15.122:0/894988072 conn(0x7ff2ac0ff770 msgr2=0x7ff2ac101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 -- 172.21.15.122:0/894988072 shutdown_connections 2024-04-02T13:56:08.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.724+0000 7ff2b1104640 1 -- 172.21.15.122:0/894988072 wait complete. 2024-04-02T13:56:08.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.725+0000 7ff2b1104640 1 Processor -- start 2024-04-02T13:56:08.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 -- start start 2024-04-02T13:56:08.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac108fe0 0x7ff2ac19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 0x7ff2ac19f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac114090 con 0x7ff2ac19ed00 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac113f10 con 0x7ff2ac103f10 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2b1104640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ac114210 con 0x7ff2ac108fe0 2024-04-02T13:56:08.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2aad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2aad9d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37294/0 (socket says 172.21.15.122:37294) 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2aad9d640 1 -- 172.21.15.122:0/1607748582 learned_addr learned my addr 172.21.15.122:0/1607748582 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 0x7ff2ac19f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.726+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac108fe0 0x7ff2ac19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 -- 172.21.15.122:0/1607748582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac108fe0 msgr2=0x7ff2ac19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac108fe0 0x7ff2ac19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 -- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 msgr2=0x7ff2ac19f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 0x7ff2ac19f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 -- 172.21.15.122:0/1607748582 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ac19f7f0 con 0x7ff2ac103f10 2024-04-02T13:56:08.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 0x7ff2ac19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.727+0000 7ff293fff640 1 -- 172.21.15.122:0/1607748582 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00cd80 con 0x7ff2ac103f10 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.728+0000 7ff293fff640 1 -- 172.21.15.122:0/1607748582 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff29c010910 con 0x7ff2ac103f10 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.728+0000 7ff293fff640 1 -- 172.21.15.122:0/1607748582 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29c018990 con 0x7ff2ac103f10 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.728+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ac19fae0 con 0x7ff2ac103f10 2024-04-02T13:56:08.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.728+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ac1a71f0 con 0x7ff2ac103f10 2024-04-02T13:56:08.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:08 smithi187 ceph-mon[36064]: osdmap e124: 12 total, 12 up, 12 in 2024-04-02T13:56:08.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3581660754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:08.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.733+0000 7ff293fff640 1 -- 172.21.15.122:0/1607748582 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff29c018b30 con 0x7ff2ac103f10 2024-04-02T13:56:08.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.734+0000 7ff293fff640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 0x7ff2940a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.734+0000 7ff293fff640 1 -- 172.21.15.122:0/1607748582 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7ff29c0d0de0 con 0x7ff2ac103f10 2024-04-02T13:56:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.735+0000 7ff2b3dc6740 1 Processor -- start 2024-04-02T13:56:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.735+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 0x7ff2940a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.735+0000 7ff2b3dc6740 1 -- start start 2024-04-02T13:56:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf259bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 0x562cf25a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 0x562cf25a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cf24ff7b0 con 0x562cf25a4ca0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562cf249e570 con 0x562cf259c0c0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562cf24fb760 con 0x562cf259b7a0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf259bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aad9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf259bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38328/0 (socket says 172.21.15.122:38328) 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aad9d640 1 -- 172.21.15.122:0/1666142422 learned_addr learned my addr 172.21.15.122:0/1666142422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 0x562cf25a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 0x562cf25a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 0x7ff2940a62e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff2a402c700 tx=0x7ff2a4000f40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 -- 172.21.15.122:0/1666142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 msgr2=0x562cf259bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf259bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 -- 172.21.15.122:0/1666142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 msgr2=0x562cf25a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 0x562cf25a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2aa59c640 1 -- 172.21.15.122:0/1666142422 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf22133f0 con 0x562cf259c0c0 2024-04-02T13:56:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.736+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 0x562cf25a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.737+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 0x562cf25a4760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009bf0 tx=0x7ff2a403e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6226e6b24776388 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.737+0000 7ff2917fa640 1 -- 172.21.15.122:0/1666142422 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a404e070 con 0x562cf259c0c0 2024-04-02T13:56:08.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.738+0000 7ff2917fa640 1 -- 172.21.15.122:0/1666142422 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4044030 con 0x562cf259c0c0 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.738+0000 7ff2917fa640 1 -- 172.21.15.122:0/1666142422 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4039040 con 0x562cf259c0c0 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.738+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/1666142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 msgr2=0x562cf25a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.738+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 0x562cf25a4760 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009bf0 tx=0x7ff2a403e000 comp rx=0 tx=0).stop 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/1666142422 shutdown_connections 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf25a4ca0 0x562cf25a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259c0c0 0x562cf25a4760 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/1666142422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf259bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/1666142422 >> 172.21.15.122:0/1666142422 conn(0x562cf2595ed0 msgr2=0x562cf25962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/1666142422 shutdown_connections 2024-04-02T13:56:08.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.739+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/1666142422 wait complete. 2024-04-02T13:56:08.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.740+0000 7ff2b3dc6740 1 -- start start 2024-04-02T13:56:08.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.740+0000 7ff2b3dc6740 1 Processor -- start 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf25a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562cf25a7a30 con 0x562cf259c0c0 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562cf24bae80 con 0x562cf25a4ca0 2024-04-02T13:56:08.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562cf25a7bb0 con 0x562cf259b7a0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36872/0 (socket says 172.21.15.122:36872) 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 -- 172.21.15.122:0/711476477 learned_addr learned my addr 172.21.15.122:0/711476477 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2b3dc6740 10 client.0 _ll_register_callbacks cb 0x562cf24f6aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 -- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 msgr2=0x562cf25a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf25a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 -- 172.21.15.122:0/711476477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 msgr2=0x562cf25a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.741+0000 7ff2aa59c640 1 -- 172.21.15.122:0/711476477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf22dd7d0 con 0x562cf259c0c0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005e30 tx=0x7ff2a4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbeb3a6c30e83d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff28eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a404e040 con 0x562cf259c0c0 2024-04-02T13:56:08.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7ff2a4003b70 con 0x562cf259c0c0 2024-04-02T13:56:08.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a4044030 con 0x562cf259c0c0 2024-04-02T13:56:08.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.742+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff2a4067050 con 0x562cf259c0c0 2024-04-02T13:56:08.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 10 client.21339 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:08.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562cf25b3820 con 0x562cf259c0c0 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4044030 con 0x562cf259c0c0 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7ff2a4049070 con 0x562cf259c0c0 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 10 client.21339 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 10 client.21339 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 20 client.21339 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.743+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf25b4b70 0x562cf25b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.744+0000 7ff2b3dc6740 4 client.21339 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:08.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.744+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cf250ccf0 con 0x562cf25b4b70 2024-04-02T13:56:08.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.744+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf25b4b70 0x562cf25b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.744+0000 7ff28eff5640 10 client.21339 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:08.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.744+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf25b4b70 0x562cf25b6f60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52bad22183893da4 server_cookie=a61acaf8b99e8ce7 in_seq=0 out_seq=0 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.746+0000 7ff28eff5640 1 -- 172.21.15.122:0/711476477 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562cf250ccf0 con 0x562cf25b4b70 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.746+0000 7ff28eff5640 10 client.21339 handle_command_reply: tid=0 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.746+0000 7ff2b3dc6740 1 client.21339 shutdown 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.746+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 msgr2=0x562cf25a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.746+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4005e30 tx=0x7ff2a4004580 comp rx=0 tx=0).stop 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 shutdown_connections 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf25b4b70 0x562cf25b6f60 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a19f0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/711476477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259b7a0 0x562cf25a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 >> 172.21.15.122:0/711476477 conn(0x562cf2595ed0 msgr2=0x562cf2597c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff28eff5640 0 client.21339 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:08.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 shutdown_connections 2024-04-02T13:56:08.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/711476477 wait complete. 2024-04-02T13:56:08.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.747+0000 7ff2b3dc6740 20 client.21339 trim_cache size 0 max 16384 2024-04-02T13:56:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:08 smithi188 ceph-mon[36117]: osdmap e124: 12 total, 12 up, 12 in 2024-04-02T13:56:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3581660754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 Processor -- start 2024-04-02T13:56:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 -- start start 2024-04-02T13:56:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259b7a0 0x562cf25b0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259c0c0 0x562cf25a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff2980101f0 con 0x562cf259b7a0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff298011d30 con 0x562cf25a4ca0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2980108c0 con 0x562cf259c0c0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2aa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259c0c0 0x562cf25a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2ab59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2ab59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37310/0 (socket says 172.21.15.122:37310) 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2ab59e640 1 -- 172.21.15.122:0/2944105991 learned_addr learned my addr 172.21.15.122:0/2944105991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.767+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259b7a0 0x562cf25b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 -- 172.21.15.122:0/2944105991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259c0c0 msgr2=0x562cf25a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259c0c0 0x562cf25a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 -- 172.21.15.122:0/2944105991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259b7a0 msgr2=0x562cf25b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259b7a0 0x562cf25b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 -- 172.21.15.122:0/2944105991 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf259bef0 con 0x562cf25a4ca0 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00041d0 tx=0x7ff2a00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f787df8914ad88a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff290ff9640 1 -- 172.21.15.122:0/2944105991 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0020070 con 0x562cf25a4ca0 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff290ff9640 1 -- 172.21.15.122:0/2944105991 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0011920 con 0x562cf25a4ca0 2024-04-02T13:56:08.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.768+0000 7ff290ff9640 1 -- 172.21.15.122:0/2944105991 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a001b990 con 0x562cf25a4ca0 2024-04-02T13:56:08.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/2944105991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 msgr2=0x562cf25a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00041d0 tx=0x7ff2a00039c0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/2944105991 shutdown_connections 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf25a4ca0 0x562cf25a4660 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf259c0c0 0x562cf25a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/2944105991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259b7a0 0x562cf25b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/2944105991 >> 172.21.15.122:0/2944105991 conn(0x562cf2595ed0 msgr2=0x562cf25962b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/2944105991 shutdown_connections 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/2944105991 wait complete. 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.769+0000 7ff2b3dc6740 1 -- start start 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 Processor -- start 2024-04-02T13:56:08.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259b7a0 0x562cf25a5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 0x562cf25a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff298011660 con 0x562cf259c0c0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff298010f90 con 0x562cf259b7a0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800e1f0 con 0x562cf25a4ca0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36878/0 (socket says 172.21.15.122:36878) 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 -- 172.21.15.122:0/3948403655 learned_addr learned my addr 172.21.15.122:0/3948403655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2b3dc6740 10 client.0 _ll_register_callbacks cb 0x7ff2a000b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259b7a0 0x562cf25a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 0x562cf25a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 -- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 msgr2=0x562cf25a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 0x562cf25a60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 -- 172.21.15.122:0/3948403655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259b7a0 msgr2=0x562cf25a5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259b7a0 0x562cf25a5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2aa59c640 1 -- 172.21.15.122:0/3948403655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf25ac0d0 con 0x562cf259c0c0 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.770+0000 7ff2ab59e640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 0x562cf25a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2aa59c640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40049d0 tx=0x7ff2a4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c6e8cd9ea235fb server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a404e070 con 0x562cf259c0c0 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7ff2a4053ac0 con 0x562cf259c0c0 2024-04-02T13:56:08.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2a0011920 con 0x562cf259c0c0 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff2a4009b20 con 0x562cf259c0c0 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 10 client.21345 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562cf25ad390 con 0x562cf259c0c0 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a0011920 con 0x562cf259c0c0 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7ff2a4049070 con 0x562cf259c0c0 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 10 client.21345 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 10 client.21345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 20 client.21345 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf261ae70 0x562cf261d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 4 client.21345 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:08.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.771+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562cf2507040 con 0x562cf261ae70 2024-04-02T13:56:08.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.772+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf261ae70 0x562cf261d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.773+0000 7ff2aad9d640 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf261ae70 0x562cf261d290 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=105eb631329c0f9b server_cookie=588d3f41e0367dc in_seq=0 out_seq=0 2024-04-02T13:56:08.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.773+0000 7ff28eff5640 10 client.21345 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff28eff5640 1 -- 172.21.15.122:0/3948403655 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562cf2507040 con 0x562cf261ae70 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff28eff5640 10 client.21345 handle_command_reply: tid=0 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 client.21345 shutdown 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 msgr2=0x562cf25a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40049d0 tx=0x7ff2a4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 shutdown_connections 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562cf25a4ca0 0x562cf25a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x562cf261ae70 0x562cf261d290 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562cf259c0c0 0x562cf25a5b80 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 --2- 172.21.15.122:0/3948403655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562cf259b7a0 0x562cf25a5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 >> 172.21.15.122:0/3948403655 conn(0x562cf2595ed0 msgr2=0x562cf25a02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff28eff5640 0 client.21345 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 shutdown_connections 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 1 -- 172.21.15.122:0/3948403655 wait complete. 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.777+0000 7ff2b3dc6740 20 client.21345 trim_cache size 0 max 16384 2024-04-02T13:56:08.784 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 msgr2=0x7ff2940a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 0x7ff2940a62e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff2a402c700 tx=0x7ff2a4000f40 comp rx=0 tx=0).stop 2024-04-02T13:56:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 msgr2=0x7ff2ac19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff29c00e9f0 tx=0x7ff29c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 shutdown_connections 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff2940a3e20 0x7ff2940a62e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2ac19ed00 0x7ff2ac19f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2ac108fe0 0x7ff2ac19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 --2- 172.21.15.122:0/1607748582 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff2ac103f10 0x7ff2ac19a490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 >> 172.21.15.122:0/1607748582 conn(0x7ff2ac0ff770 msgr2=0x7ff2ac100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 shutdown_connections 2024-04-02T13:56:08.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.778+0000 7ff2b1104640 1 -- 172.21.15.122:0/1607748582 wait complete. 2024-04-02T13:56:08.797 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 -- 172.21.15.122:0/1417543289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 msgr2=0x7fa1e41061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/1417543289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e41061e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc01a910 tx=0x7fa1dc03ccf0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 -- 172.21.15.122:0/1417543289 shutdown_connections 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/1417543289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e41061e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/1417543289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4101270 0x7fa1e4105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/1417543289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4100950 0x7fa1e4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 -- 172.21.15.122:0/1417543289 >> 172.21.15.122:0/1417543289 conn(0x7fa1e40fc360 msgr2=0x7fa1e40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 -- 172.21.15.122:0/1417543289 shutdown_connections 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.980+0000 7fa1e9a06640 1 -- 172.21.15.122:0/1417543289 wait complete. 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 Processor -- start 2024-04-02T13:56:08.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 -- start start 2024-04-02T13:56:08.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4101270 0x7fa1e4110cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e4111230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e41095d0 con 0x7fa1e4101270 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4109450 con 0x7fa1e4105d50 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.981+0000 7fa1e9a06640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4109750 con 0x7fa1e4100950 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38376/0 (socket says 172.21.15.122:38376) 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/2312024217 learned_addr learned my addr 172.21.15.122:0/2312024217 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4101270 0x7fa1e4110cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e4111230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/2312024217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 msgr2=0x7fa1e4111230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e4111230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4101270 msgr2=0x7fa1e4110cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4101270 0x7fa1e4110cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.982+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/2312024217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e4111960 con 0x7fa1e4100950 2024-04-02T13:56:08.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9f0 tx=0x7fa1d800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e8a04640 1 -- 172.21.15.122:0/2312024217 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800cd80 con 0x7fa1e4100950 2024-04-02T13:56:08.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e41a2760 con 0x7fa1e4100950 2024-04-02T13:56:08.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e41a2b60 con 0x7fa1e4100950 2024-04-02T13:56:08.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e8a04640 1 -- 172.21.15.122:0/2312024217 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8010910 con 0x7fa1e4100950 2024-04-02T13:56:08.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.983+0000 7fa1e8a04640 1 -- 172.21.15.122:0/2312024217 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80188f0 con 0x7fa1e4100950 2024-04-02T13:56:08.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 Processor -- start 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 -- start start 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be12a0 0x565078be9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078bec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565078b447b0 con 0x565078be12a0 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x565078ae3570 con 0x565078be9e80 2024-04-02T13:56:08.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565078b40760 con 0x565078be0980 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38392/0 (socket says 172.21.15.122:38392) 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.987+0000 7fa1e37fe640 1 -- 172.21.15.122:0/343474574 learned_addr learned my addr 172.21.15.122:0/343474574 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078bec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 -- 172.21.15.122:0/343474574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 msgr2=0x565078bec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078bec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 -- 172.21.15.122:0/343474574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be12a0 msgr2=0x565078be9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be12a0 0x565078be9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 -- 172.21.15.122:0/343474574 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650788583f0 con 0x565078be0980 2024-04-02T13:56:08.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.988+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005d50 tx=0x7fa1dc005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b399d29fa15a25f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:08.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.989+0000 7fa1c9ffb640 1 -- 172.21.15.122:0/343474574 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc052020 con 0x565078be0980 2024-04-02T13:56:08.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.989+0000 7fa1c9ffb640 1 -- 172.21.15.122:0/343474574 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1dc004040 con 0x565078be0980 2024-04-02T13:56:08.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.989+0000 7fa1c9ffb640 1 -- 172.21.15.122:0/343474574 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc03dba0 con 0x565078be0980 2024-04-02T13:56:08.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/343474574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 msgr2=0x565078be0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:08.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc005d50 tx=0x7fa1dc005d80 comp rx=0 tx=0).stop 2024-04-02T13:56:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/343474574 shutdown_connections 2024-04-02T13:56:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078bec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be12a0 0x565078be9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/343474574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be0d60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.990+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/343474574 >> 172.21.15.122:0/343474574 conn(0x565078bdb0b0 msgr2=0x565078bdb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:08.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.992+0000 7fa1e8a04640 1 -- 172.21.15.122:0/2312024217 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa1d8010430 con 0x7fa1e4100950 2024-04-02T13:56:08.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.993+0000 7fa1e8a04640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 0x7fa1cc0a6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:08.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.993+0000 7fa1e8a04640 1 -- 172.21.15.122:0/2312024217 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fa1d8018690 con 0x7fa1e4100950 2024-04-02T13:56:08.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.994+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 0x7fa1cc0a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:08.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.994+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/343474574 shutdown_connections 2024-04-02T13:56:08.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.994+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/343474574 wait complete. 2024-04-02T13:56:08.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.994+0000 7fa1ec5ee740 1 -- start start 2024-04-02T13:56:09.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.996+0000 7fa1ec5ee740 1 Processor -- start 2024-04-02T13:56:09.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.997+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 0x7fa1cc0a6460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x565078bdc240 tx=0x7fa1d4009500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.997+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.997+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be9e80 0x565078be6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.997+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078bf6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.997+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.998+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37352/0 (socket says 172.21.15.122:37352) 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.998+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/1282087598 learned_addr learned my addr 172.21.15.122:0/1282087598 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be9e80 0x565078be6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078bf6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565078becc60 con 0x565078be7250 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x565078affe80 con 0x565078be0980 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x565078becde0 con 0x565078be9e80 2024-04-02T13:56:09.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1ec5ee740 10 client.0 _ll_register_callbacks cb 0x565078b3baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/1282087598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be9e80 msgr2=0x565078be6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be9e80 0x565078be6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/1282087598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 msgr2=0x565078bf6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078bf6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:08.999+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650789227d0 con 0x565078be0980 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.000+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1dc004040 con 0x565078be0980 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.000+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9c0 tx=0x7fa1d809f120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d56a0f7ef6b2816 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.001+0000 7fa1c2ffd640 10 client.30236 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.001+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1282087598 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80d0660 con 0x565078be0980 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.001+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1282087598 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fa1d80bdde0 con 0x565078be0980 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.001+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1282087598 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa1d80a2dd0 con 0x565078be0980 2024-04-02T13:56:09.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.002+0000 7fa1ec5ee740 10 client.30236 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.002+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({fsmap=76}) -- 0x565078bf9030 con 0x565078be0980 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1282087598 <== mon.2 v2:172.21.15.187:3300/0 4 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa1d809de70 con 0x565078be0980 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 10 client.30236 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 10 client.30236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 20 client.30236 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078bfa3e0 0x565078bfc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 4 client.30236 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:09.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.003+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa1dc004040 con 0x565078bfa3e0 2024-04-02T13:56:09.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.004+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078bfa3e0 0x565078bfc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.005+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078bfa3e0 0x565078bfc880 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3ce54e5ef222d1 server_cookie=74db3b4e76ba104d in_seq=0 out_seq=0 2024-04-02T13:56:09.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.005+0000 7fa1c2ffd640 10 client.30236 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:09.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1282087598 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa1dc004040 con 0x565078bfa3e0 2024-04-02T13:56:09.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1c2ffd640 10 client.30236 handle_command_reply: tid=0 2024-04-02T13:56:09.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1ec5ee740 1 client.30236 shutdown 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 msgr2=0x565078be98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9c0 tx=0x7fa1d809f120 comp rx=0 tx=0).stop 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 shutdown_connections 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.006+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078bf6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.007+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be9e80 0x565078be6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.007+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078bfa3e0 0x565078bfc880 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.007+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1282087598 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078be98e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.007+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 >> 172.21.15.122:0/1282087598 conn(0x565078bdb0b0 msgr2=0x565078bdce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.007+0000 7fa1c2ffd640 0 client.30236 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:09.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.009+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 shutdown_connections 2024-04-02T13:56:09.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.009+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1282087598 wait complete. 2024-04-02T13:56:09.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.009+0000 7fa1ec5ee740 20 client.30236 trim_cache size 0 max 16384 2024-04-02T13:56:09.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 Processor -- start 2024-04-02T13:56:09.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 -- start start 2024-04-02T13:56:09.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078c02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078c02690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0010190 con 0x565078be7250 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0011cd0 con 0x565078be9e80 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0010860 con 0x565078be0980 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38416/0 (socket says 172.21.15.122:38416) 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/3524727828 learned_addr learned my addr 172.21.15.122:0/3524727828 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078c02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.035+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078c02690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/3524727828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 msgr2=0x565078c02690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078c02690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/3524727828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 msgr2=0x565078c02150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078c02150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 -- 172.21.15.122:0/3524727828 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565078bec5e0 con 0x565078be0980 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.036+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1d809b3a0 tx=0x7fa1d8022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3492e2db5c8c9f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.037+0000 7fa1c97fa640 1 -- 172.21.15.122:0/3524727828 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80a76e0 con 0x565078be0980 2024-04-02T13:56:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.037+0000 7fa1c97fa640 1 -- 172.21.15.122:0/3524727828 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8010720 con 0x565078be0980 2024-04-02T13:56:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.037+0000 7fa1c97fa640 1 -- 172.21.15.122:0/3524727828 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d809a9d0 con 0x565078be0980 2024-04-02T13:56:09.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.039+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/3524727828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 msgr2=0x565078be9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.039+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa1d809b3a0 tx=0x7fa1d8022000 comp rx=0 tx=0).stop 2024-04-02T13:56:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.040+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/3524727828 shutdown_connections 2024-04-02T13:56:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.040+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be9e80 0x565078c02690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.040+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be7250 0x565078c02150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.040+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/3524727828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be0980 0x565078be9410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.040+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/3524727828 >> 172.21.15.122:0/3524727828 conn(0x565078bdb0b0 msgr2=0x565078be5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.041+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/3524727828 shutdown_connections 2024-04-02T13:56:09.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.041+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/3524727828 wait complete. 2024-04-02T13:56:09.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.041+0000 7fa1ec5ee740 1 -- start start 2024-04-02T13:56:09.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 Processor -- start 2024-04-02T13:56:09.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be7250 0x565078beb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0011600 con 0x565078be9e80 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0010f30 con 0x565078be0980 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d000e190 con 0x565078be7250 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1ec5ee740 10 client.0 _ll_register_callbacks cb 0x7fa1d8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1e37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36946/0 (socket says 172.21.15.122:36946) 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1e2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37366/0 (socket says 172.21.15.122:37366) 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.042+0000 7fa1e37fe640 1 -- 172.21.15.122:0/1691455666 learned_addr learned my addr 172.21.15.122:0/1691455666 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e27fc640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be7250 0x565078beb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 -- 172.21.15.122:0/1691455666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be7250 msgr2=0x565078beb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be7250 0x565078beb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 -- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 msgr2=0x565078beb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 -- 172.21.15.122:0/1691455666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565078bf0b70 con 0x565078be9e80 2024-04-02T13:56:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e37fe640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc03d910 tx=0x7fa1dc03d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ccd07d25e63ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:09.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc052070 con 0x565078be9e80 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.043+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7fa1dc004c50 con 0x565078be9e80 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8010720 con 0x565078be9e80 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa1dc065050 con 0x565078be9e80 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1ec5ee740 10 client.21360 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x565078c61090 con 0x565078be9e80 2024-04-02T13:56:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8010720 con 0x565078be9e80 2024-04-02T13:56:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.044+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa1dc0035f0 con 0x565078be9e80 2024-04-02T13:56:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 10 client.21360 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 10 client.21360 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:56:09.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 20 client.21360 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078c613d0 0x565078c637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 4 client.21360 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:09.051 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.045+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565078b51cf0 con 0x565078c613d0 2024-04-02T13:56:09.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.049+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078c613d0 0x565078c637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.054+0000 7fa1e2ffd640 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078c613d0 0x565078c637c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b18d6411ad16a4df server_cookie=b8e2cf5c81af1113 in_seq=0 out_seq=0 2024-04-02T13:56:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.054+0000 7fa1c2ffd640 10 client.21360 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.055+0000 7fa1c2ffd640 1 -- 172.21.15.122:0/1691455666 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565078b51cf0 con 0x565078c613d0 2024-04-02T13:56:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.055+0000 7fa1c2ffd640 10 client.21360 handle_command_reply: tid=0 2024-04-02T13:56:09.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.055+0000 7fa1ec5ee740 1 client.21360 shutdown 2024-04-02T13:56:09.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.056+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 msgr2=0x565078bf0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.056+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc03d910 tx=0x7fa1dc03d940 comp rx=0 tx=0).stop 2024-04-02T13:56:09.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 shutdown_connections 2024-04-02T13:56:09.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565078be9e80 0x565078bf0630 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x565078be7250 0x565078beb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x565078c613d0 0x565078c637c0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 --2- 172.21.15.122:0/1691455666 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x565078be0980 0x565078beb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 >> 172.21.15.122:0/1691455666 conn(0x565078bdb0b0 msgr2=0x565078be5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.059+0000 7fa1c2ffd640 0 client.21360 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:09.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.060+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 shutdown_connections 2024-04-02T13:56:09.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.060+0000 7fa1ec5ee740 1 -- 172.21.15.122:0/1691455666 wait complete. 2024-04-02T13:56:09.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.060+0000 7fa1ec5ee740 20 client.21360 trim_cache size 0 max 16384 2024-04-02T13:56:09.067 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 msgr2=0x7fa1cc0a6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 0x7fa1cc0a6460 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x565078bdc240 tx=0x7fa1d4009500 comp rx=0 tx=0).stop 2024-04-02T13:56:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 msgr2=0x7fa1e4114c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800e9f0 tx=0x7fa1d800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 shutdown_connections 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa1cc0a3fa0 0x7fa1cc0a6460 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1e4105d50 0x7fa1e4111230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa1e4101270 0x7fa1e4110cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 --2- 172.21.15.122:0/2312024217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e4100950 0x7fa1e4114c10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.065+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 >> 172.21.15.122:0/2312024217 conn(0x7fa1e40fc360 msgr2=0x7fa1e4103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.066+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 shutdown_connections 2024-04-02T13:56:09.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.066+0000 7fa1e9a06640 1 -- 172.21.15.122:0/2312024217 wait complete. 2024-04-02T13:56:09.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:56:09.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 -- 172.21.15.122:0/796031697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108106980 msgr2=0x7f2108106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 --2- 172.21.15.122:0/796031697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108106980 0x7f2108106d60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2100009c10 tx=0x7f210002bef0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 -- 172.21.15.122:0/796031697 shutdown_connections 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 --2- 172.21.15.122:0/796031697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108101330 0x7f210810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 --2- 172.21.15.122:0/796031697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 --2- 172.21.15.122:0/796031697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108106980 0x7f2108106d60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 -- 172.21.15.122:0/796031697 >> 172.21.15.122:0/796031697 conn(0x7f2108077b00 msgr2=0x7f2108077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 -- 172.21.15.122:0/796031697 shutdown_connections 2024-04-02T13:56:09.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.242+0000 7f210f752640 1 -- 172.21.15.122:0/796031697 wait complete. 2024-04-02T13:56:09.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 Processor -- start 2024-04-02T13:56:09.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 -- start start 2024-04-02T13:56:09.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108101330 0x7f2108196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f210810fcd0 con 0x7f2108106980 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f210810fb50 con 0x7f2108100970 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.243+0000 7f210f752640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f210810fe50 con 0x7f2108101330 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:36966/0 (socket says 172.21.15.122:36966) 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 -- 172.21.15.122:0/2910940140 learned_addr learned my addr 172.21.15.122:0/2910940140 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210cced640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108101330 0x7f2108196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210d4ee640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 -- 172.21.15.122:0/2910940140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108101330 msgr2=0x7f2108196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108101330 0x7f2108196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 -- 172.21.15.122:0/2910940140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 msgr2=0x7f2108195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 -- 172.21.15.122:0/2910940140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081aac50 con 0x7f2108106980 2024-04-02T13:56:09.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210d4ee640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:09.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.244+0000 7f210dcef640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9a0 tx=0x7f20f800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.245+0000 7f20fe7fc640 1 -- 172.21.15.122:0/2910940140 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f800ccc0 con 0x7f2108106980 2024-04-02T13:56:09.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.245+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081aaf40 con 0x7f2108106980 2024-04-02T13:56:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.245+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081ab460 con 0x7f2108106980 2024-04-02T13:56:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.245+0000 7f20fe7fc640 1 -- 172.21.15.122:0/2910940140 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f800ce60 con 0x7f2108106980 2024-04-02T13:56:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.246+0000 7f20fe7fc640 1 -- 172.21.15.122:0/2910940140 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8010690 con 0x7f2108106980 2024-04-02T13:56:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 Processor -- start 2024-04-02T13:56:09.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f20fe7fc640 1 -- 172.21.15.122:0/2910940140 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f20f8010830 con 0x7f2108106980 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 -- start start 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fdab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc42f3e7b0 con 0x55fc42fe3ca0 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fc42edd570 con 0x55fc42fdb0c0 2024-04-02T13:56:09.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.249+0000 7f211241c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc42f3a760 con 0x55fc42fda7a0 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37400/0 (socket says 172.21.15.122:37400) 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 -- 172.21.15.122:0/1402644333 learned_addr learned my addr 172.21.15.122:0/1402644333 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210cced640 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fdab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210cced640 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f20fe7fc640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 0x7f20f40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f20fe7fc640 1 -- 172.21.15.122:0/2910940140 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f20f80d0d10 con 0x7f2108106980 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 -- 172.21.15.122:0/1402644333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 msgr2=0x55fc42fdab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fdab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 -- 172.21.15.122:0/1402644333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 msgr2=0x55fc42fe6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 -- 172.21.15.122:0/1402644333 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc42c523f0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.250+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f210002c500 tx=0x7f2100009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e48addcb1fc22aa server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.251+0000 7f210dcef640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 0x7f20f40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.251+0000 7f20f37fe640 1 -- 172.21.15.122:0/1402644333 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100043070 con 0x55fc42fdb0c0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.251+0000 7f20f37fe640 1 -- 172.21.15.122:0/1402644333 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100033d00 con 0x55fc42fdb0c0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.251+0000 7f20f37fe640 1 -- 172.21.15.122:0/1402644333 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210002caf0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.251+0000 7f210dcef640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 0x7f20f40a6260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004cf0 tx=0x7f20f8009ef0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.252+0000 7f211241c740 1 -- 172.21.15.122:0/1402644333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 msgr2=0x55fc42fe3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.252+0000 7f211241c740 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f210002c500 tx=0x7f2100009320 comp rx=0 tx=0).stop 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 -- 172.21.15.122:0/1402644333 shutdown_connections 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3760 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 --2- 172.21.15.122:0/1402644333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fdab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 -- 172.21.15.122:0/1402644333 >> 172.21.15.122:0/1402644333 conn(0x55fc42fd4ed0 msgr2=0x55fc42fd52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 -- 172.21.15.122:0/1402644333 shutdown_connections 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 -- 172.21.15.122:0/1402644333 wait complete. 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 -- start start 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.253+0000 7f211241c740 1 Processor -- start 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fe36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fc42fe6a30 con 0x55fc42fe3ca0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55fc42ef9e80 con 0x55fc42fdb0c0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55fc42fe6bb0 con 0x55fc42fda7a0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f211241c740 10 client.0 _ll_register_callbacks cb 0x55fc42f35aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37410/0 (socket says 172.21.15.122:37410) 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 -- 172.21.15.122:0/1825025951 learned_addr learned my addr 172.21.15.122:0/1825025951 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210cced640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fe36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210cced640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 -- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 msgr2=0x55fc42fe36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fe36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 -- 172.21.15.122:0/1825025951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 msgr2=0x55fc42fe1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.254+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f210d4ee640 1 -- 172.21.15.122:0/1825025951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc42d1c7d0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2100036040 tx=0x7f2100003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1abf5fe3ae700 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f20f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100043040 con 0x55fc42fdb0c0 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100033d00 con 0x55fc42fdb0c0 2024-04-02T13:56:09.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f2100033890 con 0x55fc42fdb0c0 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f210005a050 con 0x55fc42fdb0c0 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.255+0000 7f211241c740 10 client.30254 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fc42ff2af0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100033d00 con 0x55fc42fdb0c0 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2100055030 con 0x55fc42fdb0c0 2024-04-02T13:56:09.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 10 client.30254 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 10 client.30254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 20 client.30254 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42ff41f0 0x55fc42ff6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 4 client.30254 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc42f4bcf0 con 0x55fc42ff41f0 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.256+0000 7f210cced640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42ff41f0 0x55fc42ff6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.257+0000 7f210cced640 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42ff41f0 0x55fc42ff6630 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4e6f7b3fb877cc8 server_cookie=184a0245e48ed4cd in_seq=0 out_seq=0 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.257+0000 7f20f0ff9640 10 client.30254 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1825025951 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc42f4bcf0 con 0x55fc42ff41f0 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f20f0ff9640 10 client.30254 handle_command_reply: tid=0 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 client.30254 shutdown 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 msgr2=0x55fc42fe0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2100036040 tx=0x7f2100003d00 comp rx=0 tx=0).stop 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 shutdown_connections 2024-04-02T13:56:09.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42ff41f0 0x55fc42ff6630 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe0a40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 --2- 172.21.15.122:0/1825025951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 >> 172.21.15.122:0/1825025951 conn(0x55fc42fd4ed0 msgr2=0x55fc42fd6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f20f0ff9640 0 client.30254 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 shutdown_connections 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 1 -- 172.21.15.122:0/1825025951 wait complete. 2024-04-02T13:56:09.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.259+0000 7f211241c740 20 client.30254 trim_cache size 0 max 16384 2024-04-02T13:56:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.282+0000 7f211241c740 1 Processor -- start 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.282+0000 7f211241c740 1 -- start start 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fef980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec010120 con 0x55fc42fdb0c0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210cced640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210cced640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fef980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:38462/0 (socket says 172.21.15.122:38462) 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210cced640 1 -- 172.21.15.122:0/3267364585 learned_addr learned my addr 172.21.15.122:0/3267364585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec011c60 con 0x55fc42fe3ca0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0107f0 con 0x55fc42fda7a0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210cced640 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 -- 172.21.15.122:0/3267364585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 msgr2=0x55fc42fef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 -- 172.21.15.122:0/3267364585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fe3ca0 msgr2=0x55fc42fe36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.283+0000 7f210d4ee640 1 -- 172.21.15.122:0/3267364585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc42fe13c0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.284+0000 7f210d4ee640 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3170 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f2100003d80 tx=0x7f21000333e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f5d7f78498cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.284+0000 7f20f2ffd640 1 -- 172.21.15.122:0/3267364585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100043070 con 0x55fc42fdb0c0 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.284+0000 7f20f2ffd640 1 -- 172.21.15.122:0/3267364585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100009480 con 0x55fc42fdb0c0 2024-04-02T13:56:09.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.284+0000 7f20f2ffd640 1 -- 172.21.15.122:0/3267364585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210004c040 con 0x55fc42fdb0c0 2024-04-02T13:56:09.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 msgr2=0x55fc42fe3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3170 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f2100003d80 tx=0x7f21000333e0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 shutdown_connections 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fe3ca0 0x55fc42fe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fdb0c0 0x55fc42fe3170 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 --2- 172.21.15.122:0/3267364585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 >> 172.21.15.122:0/3267364585 conn(0x55fc42fd4ed0 msgr2=0x55fc42fd52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 shutdown_connections 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- 172.21.15.122:0/3267364585 wait complete. 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 -- start start 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 Processor -- start 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.285+0000 7f211241c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fde6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fdccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec011590 con 0x55fc42fe3ca0 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec010ec0 con 0x55fc42fdb0c0 2024-04-02T13:56:09.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec00e120 con 0x55fc42fda7a0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f211241c740 10 client.0 _ll_register_callbacks cb 0x7f210400bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37028/0 (socket says 172.21.15.122:37028) 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 -- 172.21.15.122:0/1080773307 learned_addr learned my addr 172.21.15.122:0/1080773307 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fdccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fde6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 -- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 msgr2=0x55fc42fde6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fde6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 -- 172.21.15.122:0/1080773307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 msgr2=0x55fc42fdccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fdccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 -- 172.21.15.122:0/1080773307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc42fdaef0 con 0x55fc42fe3ca0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fdccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f20f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:09.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.286+0000 7f210cced640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f2108197790 tx=0x7f210402b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe11cd518d3a2674 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104004b00 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f2104027040 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100009480 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2104039050 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f211241c740 10 client.21378 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55fc42febde0 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.287+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100009480 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2104018620 con 0x55fc42fe3ca0 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 10 client.21378 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 10 client.21378 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:09.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 20 client.21378 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42fec4e0 0x55fc4305c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 4 client.21378 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc42f46040 con 0x55fc42fec4e0 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42fec4e0 0x55fc4305c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f210d4ee640 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42fec4e0 0x55fc4305c780 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35ce8abcbeb38c20 server_cookie=73a4cda642e1647f in_seq=0 out_seq=0 2024-04-02T13:56:09.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.288+0000 7f20f0ff9640 10 client.21378 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:09.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.290+0000 7f20f0ff9640 1 -- 172.21.15.122:0/1080773307 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc42f46040 con 0x55fc42fec4e0 2024-04-02T13:56:09.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.290+0000 7f20f0ff9640 10 client.21378 handle_command_reply: tid=0 2024-04-02T13:56:09.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.290+0000 7f211241c740 1 client.21378 shutdown 2024-04-02T13:56:09.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 msgr2=0x55fc42fdd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f2108197790 tx=0x7f210402b040 comp rx=0 tx=0).stop 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 shutdown_connections 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55fc42fec4e0 0x55fc4305c780 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fc42fe3ca0 0x55fc42fdd2e0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55fc42fdb0c0 0x55fc42fdccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f20f0ff9640 0 client.21378 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 --2- 172.21.15.122:0/1080773307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55fc42fda7a0 0x55fc42fde6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 >> 172.21.15.122:0/1080773307 conn(0x55fc42fd4ed0 msgr2=0x55fc42fd36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 shutdown_connections 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 1 -- 172.21.15.122:0/1080773307 wait complete. 2024-04-02T13:56:09.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.291+0000 7f211241c740 20 client.21378 trim_cache size 0 max 16384 2024-04-02T13:56:09.299 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:09.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 msgr2=0x7f20f40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 0x7f20f40a6260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004cf0 tx=0x7f20f8009ef0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 msgr2=0x7f2108196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f20f800e9a0 tx=0x7f20f800ee60 comp rx=0 tx=0).stop 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 shutdown_connections 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f20f40a3da0 0x7f20f40a6260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2108106980 0x7f2108196a50 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2108101330 0x7f2108196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 --2- 172.21.15.122:0/2910940140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2108100970 0x7f2108195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 >> 172.21.15.122:0/2910940140 conn(0x7f2108077b00 msgr2=0x7f2108103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 shutdown_connections 2024-04-02T13:56:09.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.292+0000 7f210f752640 1 -- 172.21.15.122:0/2910940140 wait complete. 2024-04-02T13:56:09.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:56:09.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 225582 26.22 MB/sec execute 329 sec latency 287.605 ms 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 -- 172.21.15.122:0/17530212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 msgr2=0x7f8608100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 --2- 172.21.15.122:0/17530212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f8608100e10 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009930 tx=0x7f85f402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 -- 172.21.15.122:0/17530212 shutdown_connections 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 --2- 172.21.15.122:0/17530212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 0x7f860810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 --2- 172.21.15.122:0/17530212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f8608100e10 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 --2- 172.21.15.122:0/17530212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 0x7f8608106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 -- 172.21.15.122:0/17530212 >> 172.21.15.122:0/17530212 conn(0x7f86080777b0 msgr2=0x7f86080feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 -- 172.21.15.122:0/17530212 shutdown_connections 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.444+0000 7f860f055640 1 -- 172.21.15.122:0/17530212 wait complete. 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 Processor -- start 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 -- start start 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 0x7f8608198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 0x7f860819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110db0 con 0x7f8608100990 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110c30 con 0x7f8608101350 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860f055640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8608110f30 con 0x7f86081069a0 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860cdf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860cdf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37036/0 (socket says 172.21.15.122:37036) 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860cdf1640 1 -- 172.21.15.122:0/4241702387 learned_addr learned my addr 172.21.15.122:0/4241702387 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f8607fff640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 0x7f8608198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.445+0000 7f860d5f2640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 0x7f860819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 -- 172.21.15.122:0/4241702387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 msgr2=0x7f860819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 0x7f860819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 -- 172.21.15.122:0/4241702387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 msgr2=0x7f8608198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 0x7f8608198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 -- 172.21.15.122:0/4241702387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860819d3d0 con 0x7f8608100990 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860cdf1640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00d770 tx=0x7f85fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f8605ffb640 1 -- 172.21.15.122:0/4241702387 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc017070 con 0x7f8608100990 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f8605ffb640 1 -- 172.21.15.122:0/4241702387 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85fc005500 con 0x7f8608100990 2024-04-02T13:56:09.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.446+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860819d6c0 con 0x7f8608100990 2024-04-02T13:56:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.447+0000 7f8605ffb640 1 -- 172.21.15.122:0/4241702387 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc004e00 con 0x7f8608100990 2024-04-02T13:56:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.447+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a4f50 con 0x7f8608100990 2024-04-02T13:56:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.449+0000 7f8605ffb640 1 -- 172.21.15.122:0/4241702387 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f85fc004fa0 con 0x7f8608100990 2024-04-02T13:56:09.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8605ffb640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 0x7f85f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8607fff640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 0x7f85f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8605ffb640 1 -- 172.21.15.122:0/4241702387 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f85fc0d0580 con 0x7f8608100990 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8611c39740 1 Processor -- start 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8607fff640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 0x7f85f80a61e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85f4005ab0 tx=0x7f85f4005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.450+0000 7f8611c39740 1 -- start start 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d43eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d447ca0 0x56503d44a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56503d3a27b0 con 0x56503d43f0c0 2024-04-02T13:56:09.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56503d341570 con 0x56503d43e7a0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8611c39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56503d39e760 con 0x56503d447ca0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37040/0 (socket says 172.21.15.122:37040) 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 -- 172.21.15.122:0/3314449779 learned_addr learned my addr 172.21.15.122:0/3314449779 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f860cdf1640 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d43eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f860d5f2640 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d447ca0 0x56503d44a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 -- 172.21.15.122:0/3314449779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d447ca0 msgr2=0x56503d44a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d447ca0 0x56503d44a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 -- 172.21.15.122:0/3314449779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 msgr2=0x56503d43eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d43eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.451+0000 7f8607fff640 1 -- 172.21.15.122:0/3314449779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503d0b63f0 con 0x56503d43f0c0 2024-04-02T13:56:09.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.452+0000 7f8607fff640 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f85f4035000 tx=0x7f85f403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145934e007d83f46 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.452+0000 7f85f2ffd640 1 -- 172.21.15.122:0/3314449779 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4004180 con 0x56503d43f0c0 2024-04-02T13:56:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.452+0000 7f85f2ffd640 1 -- 172.21.15.122:0/3314449779 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f4004ba0 con 0x56503d43f0c0 2024-04-02T13:56:09.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.452+0000 7f85f2ffd640 1 -- 172.21.15.122:0/3314449779 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40035f0 con 0x56503d43f0c0 2024-04-02T13:56:09.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- 172.21.15.122:0/3314449779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 msgr2=0x56503d447760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f85f4035000 tx=0x7f85f403f000 comp rx=0 tx=0).stop 2024-04-02T13:56:09.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- 172.21.15.122:0/3314449779 shutdown_connections 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d447ca0 0x56503d44a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d43f0c0 0x56503d447760 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 --2- 172.21.15.122:0/3314449779 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d43eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- 172.21.15.122:0/3314449779 >> 172.21.15.122:0/3314449779 conn(0x56503d438ed0 msgr2=0x56503d4392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- 172.21.15.122:0/3314449779 shutdown_connections 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- 172.21.15.122:0/3314449779 wait complete. 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 -- start start 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.453+0000 7f8611c39740 1 Processor -- start 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d4476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d444a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56503d44aa30 con 0x56503d447ca0 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56503d35de80 con 0x56503d43f0c0 2024-04-02T13:56:09.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56503d44abb0 con 0x56503d43e7a0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8611c39740 10 client.0 _ll_register_callbacks cb 0x56503d399aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860cdf1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d4476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37050/0 (socket says 172.21.15.122:37050) 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 -- 172.21.15.122:0/2281764727 learned_addr learned my addr 172.21.15.122:0/2281764727 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f8607fff640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d444a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 -- 172.21.15.122:0/2281764727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 msgr2=0x56503d4476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d4476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 -- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 msgr2=0x56503d444a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d444a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 -- 172.21.15.122:0/2281764727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503d1807d0 con 0x56503d447ca0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f860d5f2640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8600011d10 tx=0x7f860000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89d0f65bc4d83e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f85e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.454+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860000fe60 con 0x56503d447ca0 2024-04-02T13:56:09.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.455+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f8600004b20 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.455+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f4004ba0 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.455+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8600004d60 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.455+0000 7f8611c39740 10 client.21390 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.455+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56503d45aef0 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f4004ba0 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f8600019030 con 0x56503d447ca0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 10 client.21390 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 10 client.21390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 20 client.21390 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d45e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 4 client.21390 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56503d3afcf0 con 0x56503d45c400 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f860cdf1640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d45e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f860cdf1640 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d45e7f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375f4b434510081c server_cookie=95d36552fbf72838 in_seq=0 out_seq=0 2024-04-02T13:56:09.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.456+0000 7f85e7fff640 10 client.21390 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f85e7fff640 1 -- 172.21.15.122:0/2281764727 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56503d3afcf0 con 0x56503d45c400 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f85e7fff640 10 client.21390 handle_command_reply: tid=0 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 client.21390 shutdown 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 msgr2=0x56503d4451a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8600011d10 tx=0x7f860000f790 comp rx=0 tx=0).stop 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 shutdown_connections 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d45e7f0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f85e7fff640 0 client.21390 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d4451a0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d444a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 --2- 172.21.15.122:0/2281764727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d4476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.457+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 >> 172.21.15.122:0/2281764727 conn(0x56503d438ed0 msgr2=0x56503d43ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.458+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 shutdown_connections 2024-04-02T13:56:09.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.458+0000 7f8611c39740 1 -- 172.21.15.122:0/2281764727 wait complete. 2024-04-02T13:56:09.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.458+0000 7f8611c39740 20 client.21390 trim_cache size 0 max 16384 2024-04-02T13:56:09.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 Processor -- start 2024-04-02T13:56:09.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 -- start start 2024-04-02T13:56:09.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d453920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d464600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86081acfa0 con 0x56503d447ca0 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86081aeae0 con 0x56503d43f0c0 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8611c39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ad670 con 0x56503d43e7a0 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8607fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8607fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37478/0 (socket says 172.21.15.122:37478) 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f8607fff640 1 -- 172.21.15.122:0/920969373 learned_addr learned my addr 172.21.15.122:0/920969373 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.480+0000 7f860cdf1640 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d453920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f860d5f2640 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d464600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 -- 172.21.15.122:0/920969373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 msgr2=0x56503d453920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d453920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 -- 172.21.15.122:0/920969373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 msgr2=0x56503d464600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d464600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 -- 172.21.15.122:0/920969373 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503d43eef0 con 0x56503d43f0c0 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f8607fff640 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85f402cd10 tx=0x7f85f402c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfa26a32c163cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.481+0000 7f85f27fc640 1 -- 172.21.15.122:0/920969373 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4005520 con 0x56503d43f0c0 2024-04-02T13:56:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.482+0000 7f85f27fc640 1 -- 172.21.15.122:0/920969373 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f40046c0 con 0x56503d43f0c0 2024-04-02T13:56:09.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.482+0000 7f85f27fc640 1 -- 172.21.15.122:0/920969373 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f40038e0 con 0x56503d43f0c0 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- 172.21.15.122:0/920969373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 msgr2=0x56503d4640c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f85f402cd10 tx=0x7f85f402c7e0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- 172.21.15.122:0/920969373 shutdown_connections 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d464600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43f0c0 0x56503d4640c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 --2- 172.21.15.122:0/920969373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43e7a0 0x56503d453920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- 172.21.15.122:0/920969373 >> 172.21.15.122:0/920969373 conn(0x56503d438ed0 msgr2=0x56503d43fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- 172.21.15.122:0/920969373 shutdown_connections 2024-04-02T13:56:09.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- 172.21.15.122:0/920969373 wait complete. 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 -- start start 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.483+0000 7f8611c39740 1 Processor -- start 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d4503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43f0c0 0x56503d4488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ae410 con 0x56503d447ca0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f86081add40 con 0x56503d43e7a0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86081aafa0 con 0x56503d43f0c0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8611c39740 10 client.0 _ll_register_callbacks cb 0x7f8600009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37068/0 (socket says 172.21.15.122:37068) 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 -- 172.21.15.122:0/1037970980 learned_addr learned my addr 172.21.15.122:0/1037970980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f8607fff640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43f0c0 0x56503d4488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860cdf1640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d4503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 -- 172.21.15.122:0/1037970980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43f0c0 msgr2=0x56503d4488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43f0c0 0x56503d4488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 -- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 msgr2=0x56503d4503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d4503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.484+0000 7f860d5f2640 1 -- 172.21.15.122:0/1037970980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56503d4493e0 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f860d5f2640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f8600000f80 tx=0x7f8600011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8d86126086f916 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f85e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600031020 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f860001ac70 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f40046c0 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8600039050 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f8611c39740 10 client.21396 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56503d44dbc0 con 0x56503d447ca0 2024-04-02T13:56:09.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.485+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8600029040 con 0x56503d447ca0 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f8600019040 con 0x56503d447ca0 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 10 client.21396 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 10 client.21396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 20 client.21396 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d4c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 4 client.21396 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f85f40046c0 con 0x56503d45c400 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.486+0000 7f860cdf1640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d4c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.487+0000 7f85e7fff640 10 client.21396 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:09.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.487+0000 7f860cdf1640 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d4c96b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee49dd96adf627e server_cookie=9a520edc19558b14 in_seq=0 out_seq=0 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f85e7fff640 1 -- 172.21.15.122:0/1037970980 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f85f40046c0 con 0x56503d45c400 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f85e7fff640 10 client.21396 handle_command_reply: tid=0 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f8611c39740 1 client.21396 shutdown 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 msgr2=0x56503d448e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f8600000f80 tx=0x7f8600011890 comp rx=0 tx=0).stop 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 shutdown_connections 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.490+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56503d45c400 0x56503d4c96b0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56503d447ca0 0x56503d448e20 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56503d43f0c0 0x56503d4488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 --2- 172.21.15.122:0/1037970980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56503d43e7a0 0x56503d4503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 >> 172.21.15.122:0/1037970980 conn(0x56503d438ed0 msgr2=0x56503d4376a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f85e7fff640 0 client.21396 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 shutdown_connections 2024-04-02T13:56:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 1 -- 172.21.15.122:0/1037970980 wait complete. 2024-04-02T13:56:09.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.491+0000 7f8611c39740 20 client.21396 trim_cache size 0 max 16384 2024-04-02T13:56:09.497 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 msgr2=0x7f85f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 0x7f85f80a61e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f85f4005ab0 tx=0x7f85f4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 msgr2=0x7f86081983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00d770 tx=0x7f85fc00dc30 comp rx=0 tx=0).stop 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 shutdown_connections 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f85f80a3d20 0x7f85f80a61e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86081069a0 0x7f860819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8608101350 0x7f8608198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 --2- 172.21.15.122:0/4241702387 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8608100990 0x7f86081983e0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 >> 172.21.15.122:0/4241702387 conn(0x7f86080777b0 msgr2=0x7f860806a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 shutdown_connections 2024-04-02T13:56:09.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:09.492+0000 7f860f055640 1 -- 172.21.15.122:0/4241702387 wait complete. 2024-04-02T13:56:09.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:09 smithi187 ceph-mon[36064]: pgmap v661: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-04-02T13:56:09.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:09 smithi188 ceph-mon[36117]: pgmap v661: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-04-02T13:56:09.820 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:09 smithi122 ceph-mon[32114]: pgmap v661: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 633 op/s 2024-04-02T13:56:10.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 226016 26.20 MB/sec execute 330 sec latency 120.912 ms 2024-04-02T13:56:10.513 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:56:10.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:56:10.513 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- 172.21.15.122:0/3745120960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c101330 msgr2=0x7f423c10d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 --2- 172.21.15.122:0/3745120960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c101330 0x7f423c10d840 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f4234009930 tx=0x7f423402bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- 172.21.15.122:0/3745120960 shutdown_connections 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 --2- 172.21.15.122:0/3745120960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c101330 0x7f423c10d840 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 --2- 172.21.15.122:0/3745120960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c100970 0x7f423c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 --2- 172.21.15.122:0/3745120960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c106980 0x7f423c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- 172.21.15.122:0/3745120960 >> 172.21.15.122:0/3745120960 conn(0x7f423c077b00 msgr2=0x7f423c077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- 172.21.15.122:0/3745120960 shutdown_connections 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- 172.21.15.122:0/3745120960 wait complete. 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 Processor -- start 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.636+0000 7f424464a640 1 -- start start 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c100970 0x7f423c1961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c101330 0x7f423c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f423c10fc80 con 0x7f423c106980 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f423c10fb00 con 0x7f423c101330 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f424464a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f423c10fe00 con 0x7f423c100970 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37094/0 (socket says 172.21.15.122:37094) 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 -- 172.21.15.122:0/705477601 learned_addr learned my addr 172.21.15.122:0/705477601 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4241be5640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c101330 0x7f423c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f42423e6640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c100970 0x7f423c1961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 -- 172.21.15.122:0/705477601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c100970 msgr2=0x7f423c1961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c100970 0x7f423c1961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 -- 172.21.15.122:0/705477601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c101330 msgr2=0x7f423c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c101330 0x7f423c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 -- 172.21.15.122:0/705477601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c19b140 con 0x7f423c106980 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f4242be7640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4234005940 tx=0x7f423402cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.637+0000 7f42337fe640 1 -- 172.21.15.122:0/705477601 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234041070 con 0x7f423c106980 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.638+0000 7f42337fe640 1 -- 172.21.15.122:0/705477601 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4234004450 con 0x7f423c106980 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.638+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c19b3d0 con 0x7f423c106980 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.638+0000 7f42337fe640 1 -- 172.21.15.122:0/705477601 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234003d30 con 0x7f423c106980 2024-04-02T13:56:10.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.638+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1a2cc0 con 0x7f423c106980 2024-04-02T13:56:10.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f42337fe640 1 -- 172.21.15.122:0/705477601 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f4234044070 con 0x7f423c106980 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 Processor -- start 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 -- start start 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e02c0 0x56227d6e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e0be0 0x56227d6e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56227d644770 con 0x56227d6e97c0 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56227d5ec960 con 0x56227d6e0be0 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4247232740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56227d6408c0 con 0x56227d6e02c0 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f42423e6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e0be0 0x56227d6e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37102/0 (socket says 172.21.15.122:37102) 2024-04-02T13:56:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.641+0000 7f4241be5640 1 -- 172.21.15.122:0/519275555 learned_addr learned my addr 172.21.15.122:0/519275555 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e02c0 0x56227d6e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f42337fe640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 0x7f42280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 -- 172.21.15.122:0/519275555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e02c0 msgr2=0x56227d6e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e02c0 0x56227d6e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 -- 172.21.15.122:0/519275555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e0be0 msgr2=0x56227d6e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4242be7640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 0x7f42280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e0be0 0x56227d6e9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 -- 172.21.15.122:0/519275555 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227d4207e0 con 0x56227d6e97c0 2024-04-02T13:56:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f42337fe640 1 -- 172.21.15.122:0/705477601 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f42340f24c0 con 0x7f423c106980 2024-04-02T13:56:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4241be5640 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f422c009690 tx=0x7f422c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f5dccabf10cc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4230bfe640 1 -- 172.21.15.122:0/519275555 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c004030 con 0x56227d6e97c0 2024-04-02T13:56:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4230bfe640 1 -- 172.21.15.122:0/519275555 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c005500 con 0x56227d6e97c0 2024-04-02T13:56:10.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.642+0000 7f4230bfe640 1 -- 172.21.15.122:0/519275555 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c004d50 con 0x56227d6e97c0 2024-04-02T13:56:10.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 -- 172.21.15.122:0/519275555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 msgr2=0x56227d6ebbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4242be7640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 0x7f42280a6260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f423402c820 tx=0x7f423404bbd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f422c009690 tx=0x7f422c005850 comp rx=0 tx=0).stop 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 -- 172.21.15.122:0/519275555 shutdown_connections 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6ebbb0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e0be0 0x56227d6e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 --2- 172.21.15.122:0/519275555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e02c0 0x56227d6e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 -- 172.21.15.122:0/519275555 >> 172.21.15.122:0/519275555 conn(0x56227d6da9f0 msgr2=0x56227d6dadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 -- 172.21.15.122:0/519275555 shutdown_connections 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.643+0000 7f4247232740 1 -- 172.21.15.122:0/519275555 wait complete. 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 -- start start 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 Processor -- start 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6e7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56227d6ec5b0 con 0x56227d6e02c0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56227d5ffd20 con 0x56227d6e97c0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56227d6ec730 con 0x56227d6e0be0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4247232740 10 client.0 _ll_register_callbacks cb 0x56227d5cca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37118/0 (socket says 172.21.15.122:37118) 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f4241be5640 1 -- 172.21.15.122:0/3243017636 learned_addr learned my addr 172.21.15.122:0/3243017636 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.644+0000 7f42423e6640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6e7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 -- 172.21.15.122:0/3243017636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 msgr2=0x56227d6e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 -- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 msgr2=0x56227d6e7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6e7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 -- 172.21.15.122:0/3243017636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227d480f70 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4241be5640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f422c005940 tx=0x7f422c004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481a8993ec13d157 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c004cf0 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f422c01d400 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422c005500 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f422c033050 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4247232740 10 client.21408 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.645+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56227d318170 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f422c027040 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f422c01c040 con 0x56227d6e02c0 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 10 client.21408 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 10 client.21408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:10.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 20 client.21408 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d6f9aa0 0x56227d6fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 4 client.21408 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f422c005500 con 0x56227d6f9aa0 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f42423e6640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d6f9aa0 0x56227d6fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f4225ffb640 10 client.21408 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.646+0000 7f42423e6640 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d6f9aa0 0x56227d6fbe90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb820b08df402cd7 server_cookie=884818b9c9806974 in_seq=0 out_seq=0 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4225ffb640 1 -- 172.21.15.122:0/3243017636 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f422c005500 con 0x56227d6f9aa0 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4225ffb640 10 client.21408 handle_command_reply: tid=0 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 client.21408 shutdown 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 msgr2=0x56227d6e67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f422c005940 tx=0x7f422c004950 comp rx=0 tx=0).stop 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 shutdown_connections 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d6f9aa0 0x56227d6fbe90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6e7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 --2- 172.21.15.122:0/3243017636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e02c0 0x56227d6e67e0 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 >> 172.21.15.122:0/3243017636 conn(0x56227d6da9f0 msgr2=0x56227d6dc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4225ffb640 0 client.21408 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 shutdown_connections 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.648+0000 7f4247232740 1 -- 172.21.15.122:0/3243017636 wait complete. 2024-04-02T13:56:10.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.649+0000 7f4247232740 20 client.21408 trim_cache size 0 max 16384 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 Processor -- start 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 -- start start 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e5940 0x56227d6d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1acf90 con 0x56227d6e97c0 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1aead0 con 0x56227d6e5940 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4247232740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ad660 con 0x56227d6e0be0 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f42423e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f42423e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37126/0 (socket says 172.21.15.122:37126) 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f42423e6640 1 -- 172.21.15.122:0/1479857637 learned_addr learned my addr 172.21.15.122:0/1479857637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.670+0000 7f4241be5640 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 -- 172.21.15.122:0/1479857637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 msgr2=0x56227d6e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 -- 172.21.15.122:0/1479857637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e5940 msgr2=0x56227d6d9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e5940 0x56227d6d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 -- 172.21.15.122:0/1479857637 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56227d6fc360 con 0x56227d6e97c0 2024-04-02T13:56:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f4241be5640 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f42423e6640 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f4238005fd0 tx=0x7f4238011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e5e74811d06fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f4227fff640 1 -- 172.21.15.122:0/1479857637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238026dd0 con 0x56227d6e97c0 2024-04-02T13:56:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f4227fff640 1 -- 172.21.15.122:0/1479857637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238004520 con 0x56227d6e97c0 2024-04-02T13:56:10.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.671+0000 7f4227fff640 1 -- 172.21.15.122:0/1479857637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238026880 con 0x56227d6e97c0 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- 172.21.15.122:0/1479857637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 msgr2=0x56227d6f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f4238005fd0 tx=0x7f4238011040 comp rx=0 tx=0).stop 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- 172.21.15.122:0/1479857637 shutdown_connections 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e5940 0x56227d6d9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e97c0 0x56227d6f54a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 --2- 172.21.15.122:0/1479857637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e0be0 0x56227d6e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- 172.21.15.122:0/1479857637 >> 172.21.15.122:0/1479857637 conn(0x56227d6da9f0 msgr2=0x56227d6d9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- 172.21.15.122:0/1479857637 shutdown_connections 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- 172.21.15.122:0/1479857637 wait complete. 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.672+0000 7f4247232740 1 -- start start 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 Processor -- start 2024-04-02T13:56:10.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e5940 0x56227d6ed270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1ae400 con 0x56227d6e0be0 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1add30 con 0x56227d6e97c0 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1aaf90 con 0x56227d6e5940 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4247232740 10 client.0 _ll_register_callbacks cb 0x7f4238009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37132/0 (socket says 172.21.15.122:37132) 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 -- 172.21.15.122:0/160059684 learned_addr learned my addr 172.21.15.122:0/160059684 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 -- 172.21.15.122:0/160059684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e5940 msgr2=0x56227d6ed270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e5940 0x56227d6ed270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 -- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 msgr2=0x56227d6ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.673+0000 7f4241be5640 1 -- 172.21.15.122:0/160059684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56227d75ee90 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4241be5640 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x56227d6e6070 tx=0x7f422c01b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df32e3be0f43c601 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c00da00 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f422c004a10 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4238004520 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f422c046050 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 10 client.21414 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56227d760070 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238004520 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f422c041030 con 0x56227d6e0be0 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 10 client.21414 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 10 client.21414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 20 client.21414 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d760670 0x56227d762a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 4 client.21414 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.674+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56227d651cb0 con 0x56227d760670 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.675+0000 7f42423e6640 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d760670 0x56227d762a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.675+0000 7f4225ffb640 10 client.21414 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.675+0000 7f42423e6640 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d760670 0x56227d762a90 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f987a18a890857a server_cookie=ebd095d766fc5ad1 in_seq=0 out_seq=0 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4225ffb640 1 -- 172.21.15.122:0/160059684 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56227d651cb0 con 0x56227d760670 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4225ffb640 10 client.21414 handle_command_reply: tid=0 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 client.21414 shutdown 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 msgr2=0x56227d6f1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x56227d6e6070 tx=0x7f422c01b780 comp rx=0 tx=0).stop 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 shutdown_connections 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56227d760670 0x56227d762a90 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56227d6e97c0 0x56227d6ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56227d6e5940 0x56227d6ed270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 --2- 172.21.15.122:0/160059684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56227d6e0be0 0x56227d6f1f00 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 >> 172.21.15.122:0/160059684 conn(0x56227d6da9f0 msgr2=0x56227d6e7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4225ffb640 0 client.21414 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 shutdown_connections 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 1 -- 172.21.15.122:0/160059684 wait complete. 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.678+0000 7f4247232740 20 client.21414 trim_cache size 0 max 16384 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:56:10.689 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "3c04e89c-7738-40b5-96d2-96e34719a79e", 2024-04-02T13:56:10.690 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:56:10.690 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:10.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 msgr2=0x7f42280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 0x7f42280a6260 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f423402c820 tx=0x7f423404bbd0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 msgr2=0x7f423c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f4234005940 tx=0x7f423402cb30 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 shutdown_connections 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f42280a3da0 0x7f42280a6260 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f423c106980 0x7f423c19aa10 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f423c101330 0x7f423c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 --2- 172.21.15.122:0/705477601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c100970 0x7f423c1961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 >> 172.21.15.122:0/705477601 conn(0x7f423c077b00 msgr2=0x7f423c103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 shutdown_connections 2024-04-02T13:56:10.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.679+0000 7f424464a640 1 -- 172.21.15.122:0/705477601 wait complete. 2024-04-02T13:56:10.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:10.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 -- 172.21.15.122:0/1173960452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d84074850 msgr2=0x7f7d8410dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/1173960452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d84074850 0x7f7d8410dd90 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009900 tx=0x7f7d7802bcd0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 -- 172.21.15.122:0/1173960452 shutdown_connections 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/1173960452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d8410e2d0 0x7f7d84110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/1173960452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d84074850 0x7f7d8410dd90 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/1173960452 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d84074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 -- 172.21.15.122:0/1173960452 >> 172.21.15.122:0/1173960452 conn(0x7f7d84078ec0 msgr2=0x7f7d840792d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 -- 172.21.15.122:0/1173960452 shutdown_connections 2024-04-02T13:56:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.838+0000 7f7d8a31b640 1 -- 172.21.15.122:0/1173960452 wait complete. 2024-04-02T13:56:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.839+0000 7f7d8a31b640 1 Processor -- start 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 -- start start 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 0x7f7d8419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8410e2d0 0x7f7d8419ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84114030 con 0x7f7d8410e2d0 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84113eb0 con 0x7f7d84073f30 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d8a31b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841141b0 con 0x7f7d84074850 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 0x7f7d8419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:37572/0 (socket says 172.21.15.122:37572) 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.840+0000 7f7d83fff640 1 -- 172.21.15.122:0/3817467898 learned_addr learned my addr 172.21.15.122:0/3817467898 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d888b8640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8410e2d0 0x7f7d8419ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 -- 172.21.15.122:0/3817467898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 msgr2=0x7f7d8419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 -- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8410e2d0 msgr2=0x7f7d8419ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8410e2d0 0x7f7d8419ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 -- 172.21.15.122:0/3817467898 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8419f550 con 0x7f7d84074850 2024-04-02T13:56:10.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d83fff640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.841+0000 7f7d837fe640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 0x7f7d8419aaf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000c00 tx=0x7f7d7802cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.842+0000 7f7d817fa640 1 -- 172.21.15.122:0/3817467898 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78041070 con 0x7f7d84074850 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.842+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8419f7e0 con 0x7f7d84074850 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.842+0000 7f7d817fa640 1 -- 172.21.15.122:0/3817467898 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d78004450 con 0x7f7d84074850 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.842+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a70d0 con 0x7f7d84074850 2024-04-02T13:56:10.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.842+0000 7f7d817fa640 1 -- 172.21.15.122:0/3817467898 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78003e50 con 0x7f7d84074850 2024-04-02T13:56:10.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.844+0000 7f7d8cfd6740 1 Processor -- start 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.844+0000 7f7d817fa640 1 -- 172.21.15.122:0/3817467898 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7d78044070 con 0x7f7d84074850 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.844+0000 7f7d8cfd6740 1 -- start start 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603f50a5830 con 0x5603f51421f0 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603f504d820 con 0x5603f514add0 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603f50a17e0 con 0x5603f51418d0 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37164/0 (socket says 172.21.15.122:37164) 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 -- 172.21.15.122:0/3996503309 learned_addr learned my addr 172.21.15.122:0/3996503309 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d888b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d837fe640 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d817fa640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 0x7f7d6c0a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 -- 172.21.15.122:0/3996503309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 msgr2=0x5603f5141cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5141cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 -- 172.21.15.122:0/3996503309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 msgr2=0x5603f514d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 -- 172.21.15.122:0/3996503309 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f4e837d0 con 0x5603f51421f0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d888b8640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 0x7f7d6c0a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d817fa640 1 -- 172.21.15.122:0/3817467898 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f7d780f2800 con 0x7f7d84074850 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.845+0000 7f7d83fff640 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7d74011650 tx=0x7f7d74009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefeae3788c8dbe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d6a7fc640 1 -- 172.21.15.122:0/3996503309 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7401e070 con 0x5603f51421f0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d6a7fc640 1 -- 172.21.15.122:0/3996503309 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7400fba0 con 0x5603f51421f0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d888b8640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 0x7f7d6c0a6400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005fd0 tx=0x7f7d7c004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d6a7fc640 1 -- 172.21.15.122:0/3996503309 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74019d00 con 0x5603f51421f0 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/3996503309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 msgr2=0x5603f514a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f7d74011650 tx=0x7f7d74009f90 comp rx=0 tx=0).stop 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.846+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/3996503309 shutdown_connections 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514a890 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/3996503309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5141cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/3996503309 >> 172.21.15.122:0/3996503309 conn(0x5603f513c000 msgr2=0x5603f513c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/3996503309 shutdown_connections 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/3996503309 wait complete. 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- start start 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 Processor -- start 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 0x5603f5147a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 0x5603f5147f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5603f514dbc0 con 0x5603f51418d0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5603f5060d30 con 0x5603f51421f0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5603f514dd40 con 0x5603f514add0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.847+0000 7f7d8cfd6740 10 client.0 _ll_register_callbacks cb 0x5603f5060720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d888b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 0x5603f5147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37174/0 (socket says 172.21.15.122:37174) 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 -- 172.21.15.122:0/2445560589 learned_addr learned my addr 172.21.15.122:0/2445560589 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d837fe640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 0x5603f5147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 -- 172.21.15.122:0/2445560589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 msgr2=0x5603f5147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 0x5603f5147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 -- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 msgr2=0x5603f5147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 0x5603f5147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 -- 172.21.15.122:0/2445560589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f4db93f0 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d837fe640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 0x5603f5147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d888b8640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 0x5603f5147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d83fff640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7d740097f0 tx=0x7f7d740056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9edcf124fbd35f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7401e040 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f7d74009cc0 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7400fba0 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7d74030050 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d8cfd6740 10 client.21426 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.848+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5603f4d790f0 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7400fba0 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f7d74021070 con 0x5603f51418d0 2024-04-02T13:56:10.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 10 client.21426 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 10 client.21426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 20 client.21426 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f515b160 0x5603f515d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 4 client.21426 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f50b2d70 con 0x5603f515b160 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.849+0000 7f7d837fe640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f515b160 0x5603f515d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.850+0000 7f7d637fe640 10 client.21426 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.850+0000 7f7d837fe640 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f515b160 0x5603f515d5b0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71115c2a91302002 server_cookie=fbfc7cbe0ad51d88 in_seq=0 out_seq=0 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.854+0000 7f7d637fe640 1 -- 172.21.15.122:0/2445560589 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603f50b2d70 con 0x5603f515b160 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.854+0000 7f7d637fe640 10 client.21426 handle_command_reply: tid=0 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 client.21426 shutdown 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 msgr2=0x5603f514a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f7d740097f0 tx=0x7f7d740056d0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 shutdown_connections 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f515b160 0x5603f515d5b0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f514add0 0x5603f5147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f51421f0 0x5603f5147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/2445560589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51418d0 0x5603f514a680 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d637fe640 0 client.21426 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 >> 172.21.15.122:0/2445560589 conn(0x5603f513c000 msgr2=0x5603f513dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 shutdown_connections 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/2445560589 wait complete. 2024-04-02T13:56:10.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.855+0000 7f7d8cfd6740 20 client.21426 trim_cache size 0 max 16384 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 Processor -- start 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 -- start start 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5156a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841b13a0 con 0x5603f51421f0 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841b2ee0 con 0x5603f514add0 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841b1a70 con 0x5603f51418d0 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37186/0 (socket says 172.21.15.122:37186) 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 -- 172.21.15.122:0/1846992534 learned_addr learned my addr 172.21.15.122:0/1846992534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d888b8640 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d83fff640 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 -- 172.21.15.122:0/1846992534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 msgr2=0x5603f5156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5156a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 -- 172.21.15.122:0/1846992534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 msgr2=0x5603f514a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.877+0000 7f7d837fe640 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.878+0000 7f7d837fe640 1 -- 172.21.15.122:0/1846992534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f513be30 con 0x5603f51421f0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.878+0000 7f7d837fe640 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f7d780efcd0 tx=0x7f7d780f4d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbde2f5e207a2cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.878+0000 7f7d69ffb640 1 -- 172.21.15.122:0/1846992534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7804b040 con 0x5603f51421f0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.878+0000 7f7d69ffb640 1 -- 172.21.15.122:0/1846992534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d780be520 con 0x5603f51421f0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.878+0000 7f7d69ffb640 1 -- 172.21.15.122:0/1846992534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780f5e60 con 0x5603f51421f0 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/1846992534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 msgr2=0x5603f5146950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f7d780efcd0 tx=0x7f7d780f4d10 comp rx=0 tx=0).stop 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/1846992534 shutdown_connections 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f514a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f5146950 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/1846992534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f5156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/1846992534 >> 172.21.15.122:0/1846992534 conn(0x5603f513c000 msgr2=0x5603f513b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/1846992534 shutdown_connections 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/1846992534 wait complete. 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- start start 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 Processor -- start 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f514e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f51bf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841b2810 con 0x5603f51421f0 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841b2140 con 0x5603f514add0 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37194/0 (socket says 172.21.15.122:37194) 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.879+0000 7f7d8cfd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841af3a0 con 0x5603f51418d0 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 -- 172.21.15.122:0/562168960 learned_addr learned my addr 172.21.15.122:0/562168960 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:10.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d83fff640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f514e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d8cfd6740 10 client.0 _ll_register_callbacks cb 0x7f7d740114d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d888b8640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f51bf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 -- 172.21.15.122:0/562168960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 msgr2=0x5603f514e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f514e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 -- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 msgr2=0x5603f51bf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f51bf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 -- 172.21.15.122:0/562168960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f51412d0 con 0x5603f51421f0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d83fff640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f514e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d837fe640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7d7802c8c0 tx=0x7f7d78003920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e4d89038faf198 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7804b040 con 0x5603f51421f0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9613+0+0 (secure 0 0 0) 0x7f7d780035f0 con 0x5603f51421f0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d780be520 con 0x5603f51421f0 2024-04-02T13:56:10.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7d780bde80 con 0x5603f51421f0 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d8cfd6740 10 client.21432 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.880+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5603f51c0d00 con 0x5603f51421f0 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d780be520 con 0x5603f51421f0 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f7d780be780 con 0x5603f51421f0 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 10 client.21432 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 10 client.21432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 20 client.21432 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f51c1430 0x5603f51c3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 4 client.21432 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603f50ad0c0 con 0x5603f51c1430 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.881+0000 7f7d83fff640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f51c1430 0x5603f51c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.882+0000 7f7d637fe640 10 client.21432 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.882+0000 7f7d83fff640 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f51c1430 0x5603f51c3850 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0d1810cf1b0cef server_cookie=7b009088d821a904 in_seq=0 out_seq=0 2024-04-02T13:56:10.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d637fe640 1 -- 172.21.15.122:0/562168960 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x5603f50ad0c0 con 0x5603f51c1430 2024-04-02T13:56:10.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d637fe640 10 client.21432 handle_command_reply: tid=0 2024-04-02T13:56:10.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 client.21432 shutdown 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 msgr2=0x5603f514e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7d7802c8c0 tx=0x7f7d78003920 comp rx=0 tx=0).stop 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 shutdown_connections 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5603f51c1430 0x5603f51c3850 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5603f514add0 0x5603f51bf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5603f51421f0 0x5603f514e680 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d637fe640 0 client.21432 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 --2- 172.21.15.122:0/562168960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5603f51418d0 0x5603f514e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 >> 172.21.15.122:0/562168960 conn(0x5603f513c000 msgr2=0x5603f51425d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 shutdown_connections 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.884+0000 7f7d8cfd6740 1 -- 172.21.15.122:0/562168960 wait complete. 2024-04-02T13:56:10.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8cfd6740 20 client.21432 trim_cache size 0 max 16384 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "scrub active (29 inodes in the stack)", 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "3c04e89c-7738-40b5-96d2-96e34719a79e": { 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "3c04e89c-7738-40b5-96d2-96e34719a79e", 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:56:10.893 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 msgr2=0x7f7d6c0a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 0x7f7d6c0a6400 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c005fd0 tx=0x7f7d7c004a40 comp rx=0 tx=0).stop 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 msgr2=0x7f7d8419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 0x7f7d8419aaf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d78000c00 tx=0x7f7d7802cc90 comp rx=0 tx=0).stop 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 shutdown_connections 2024-04-02T13:56:10.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7d6c0a3f40 0x7f7d6c0a6400 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d8410e2d0 0x7f7d8419ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d84074850 0x7f7d8419aaf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 --2- 172.21.15.122:0/3817467898 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d84073f30 0x7f7d8419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 >> 172.21.15.122:0/3817467898 conn(0x7f7d84078ec0 msgr2=0x7f7d840fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 shutdown_connections 2024-04-02T13:56:10.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:10.885+0000 7f7d8a31b640 1 -- 172.21.15.122:0/3817467898 wait complete. 2024-04-02T13:56:10.903 INFO:tasks.cephfs.filesystem:scrub status for tag:3c04e89c-7738-40b5-96d2-96e34719a79e - {'path': '/', 'tag': '3c04e89c-7738-40b5-96d2-96e34719a79e', 'options': 'recursive,force'} 2024-04-02T13:56:11.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 226504 26.18 MB/sec execute 331 sec latency 128.672 ms 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: pgmap v662: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:11.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:11 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:11.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: pgmap v662: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-04-02T13:56:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:11.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:11 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: pgmap v662: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 12 MiB/s wr, 637 op/s 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:11.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:11 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 227079 26.20 MB/sec execute 332 sec latency 117.761 ms 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:12.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.737 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:12 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:12 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:12.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:12 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:13.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 227722 26.23 MB/sec execute 333 sec latency 78.158 ms 2024-04-02T13:56:13.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:13 smithi187 ceph-mon[36064]: pgmap v663: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 548 op/s 2024-04-02T13:56:13.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:13 smithi188 ceph-mon[36117]: pgmap v663: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 548 op/s 2024-04-02T13:56:13.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:13 smithi122 ceph-mon[32114]: pgmap v663: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.1 MiB/s wr, 548 op/s 2024-04-02T13:56:14.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 228196 26.21 MB/sec execute 334 sec latency 110.413 ms 2024-04-02T13:56:15.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 228845 26.29 MB/sec execute 335 sec latency 77.666 ms 2024-04-02T13:56:15.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:15 smithi187 ceph-mon[36064]: pgmap v664: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-04-02T13:56:15.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:15 smithi188 ceph-mon[36117]: pgmap v664: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-04-02T13:56:15.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:15 smithi122 ceph-mon[32114]: pgmap v664: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.2 MiB/s wr, 546 op/s 2024-04-02T13:56:16.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 229303 26.26 MB/sec execute 336 sec latency 137.365 ms 2024-04-02T13:56:17.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 229752 26.24 MB/sec execute 337 sec latency 90.370 ms 2024-04-02T13:56:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:17 smithi187 ceph-mon[36064]: pgmap v665: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-02T13:56:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:17 smithi187 ceph-mon[36064]: osdmap e125: 12 total, 12 up, 12 in 2024-04-02T13:56:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:17 smithi188 ceph-mon[36117]: pgmap v665: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-02T13:56:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:17 smithi188 ceph-mon[36117]: osdmap e125: 12 total, 12 up, 12 in 2024-04-02T13:56:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:17 smithi122 ceph-mon[32114]: pgmap v665: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.2 MiB/s wr, 545 op/s 2024-04-02T13:56:17.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:17 smithi122 ceph-mon[32114]: osdmap e125: 12 total, 12 up, 12 in 2024-04-02T13:56:18.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 230329 26.27 MB/sec execute 338 sec latency 89.649 ms 2024-04-02T13:56:19.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 230837 26.25 MB/sec execute 339 sec latency 95.265 ms 2024-04-02T13:56:19.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:19 smithi187 ceph-mon[36064]: pgmap v667: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 518 op/s 2024-04-02T13:56:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:19 smithi188 ceph-mon[36117]: pgmap v667: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 518 op/s 2024-04-02T13:56:19.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:19 smithi122 ceph-mon[32114]: pgmap v667: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 518 op/s 2024-04-02T13:56:20.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231090 26.21 MB/sec execute 340 sec latency 675.762 ms 2024-04-02T13:56:21.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231194 26.14 MB/sec execute 341 sec latency 1393.258 ms 2024-04-02T13:56:21.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:21 smithi187 ceph-mon[36064]: pgmap v668: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 467 op/s 2024-04-02T13:56:21.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:21 smithi188 ceph-mon[36117]: pgmap v668: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 467 op/s 2024-04-02T13:56:21.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:21 smithi122 ceph-mon[32114]: pgmap v668: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 467 op/s 2024-04-02T13:56:22.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231388 26.09 MB/sec execute 342 sec latency 325.699 ms 2024-04-02T13:56:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:22.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:22 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:22.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:22.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:22.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:22.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:22 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:22.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:22 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:23.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 231627 26.06 MB/sec execute 343 sec latency 184.193 ms 2024-04-02T13:56:23.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:23 smithi187 ceph-mon[36064]: pgmap v669: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 541 op/s 2024-04-02T13:56:23.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:23 smithi188 ceph-mon[36117]: pgmap v669: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 541 op/s 2024-04-02T13:56:23.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:23 smithi122 ceph-mon[32114]: pgmap v669: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 541 op/s 2024-04-02T13:56:24.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 232136 26.09 MB/sec execute 344 sec latency 222.095 ms 2024-04-02T13:56:25.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 232639 26.10 MB/sec execute 345 sec latency 113.802 ms 2024-04-02T13:56:25.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:25 smithi187 ceph-mon[36064]: pgmap v670: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-04-02T13:56:25.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:25 smithi188 ceph-mon[36117]: pgmap v670: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-04-02T13:56:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:25 smithi122 ceph-mon[32114]: pgmap v670: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 554 op/s 2024-04-02T13:56:26.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 233110 26.07 MB/sec execute 346 sec latency 108.538 ms 2024-04-02T13:56:27.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 233638 26.10 MB/sec execute 347 sec latency 83.186 ms 2024-04-02T13:56:27.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:27 smithi187 ceph-mon[36064]: pgmap v671: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-04-02T13:56:27.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:27 smithi188 ceph-mon[36117]: pgmap v671: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-04-02T13:56:27.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:27 smithi122 ceph-mon[32114]: pgmap v671: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 559 op/s 2024-04-02T13:56:28.340 INFO:tasks.workunit.client.0.smithi122.stdout: 10 234208 26.12 MB/sec execute 348 sec latency 69.595 ms 2024-04-02T13:56:29.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 234683 26.07 MB/sec execute 349 sec latency 277.228 ms 2024-04-02T13:56:29.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:29 smithi187 ceph-mon[36064]: pgmap v672: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-04-02T13:56:29.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:29 smithi188 ceph-mon[36117]: pgmap v672: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-04-02T13:56:29.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:29 smithi122 ceph-mon[32114]: pgmap v672: 129 pgs: 129 active+clean; 2.5 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 618 op/s 2024-04-02T13:56:30.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 235211 26.08 MB/sec execute 350 sec latency 162.456 ms 2024-04-02T13:56:31.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 235858 26.15 MB/sec execute 351 sec latency 77.683 ms 2024-04-02T13:56:31.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:31 smithi187 ceph-mon[36064]: pgmap v673: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-04-02T13:56:31.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:31 smithi188 ceph-mon[36117]: pgmap v673: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-04-02T13:56:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:31 smithi122 ceph-mon[32114]: pgmap v673: 129 pgs: 129 active+clean; 2.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 538 op/s 2024-04-02T13:56:32.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 236320 26.14 MB/sec execute 352 sec latency 154.250 ms 2024-04-02T13:56:33.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 236704 26.11 MB/sec execute 353 sec latency 197.587 ms 2024-04-02T13:56:33.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:33 smithi187 ceph-mon[36064]: pgmap v674: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-04-02T13:56:33.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:33 smithi188 ceph-mon[36117]: pgmap v674: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-04-02T13:56:33.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:33 smithi122 ceph-mon[32114]: pgmap v674: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 646 op/s 2024-04-02T13:56:34.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 237123 26.11 MB/sec execute 354 sec latency 149.852 ms 2024-04-02T13:56:35.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 237508 26.10 MB/sec execute 355 sec latency 339.770 ms 2024-04-02T13:56:35.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:35 smithi187 ceph-mon[36064]: pgmap v675: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-04-02T13:56:35.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:35 smithi188 ceph-mon[36117]: pgmap v675: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-04-02T13:56:35.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:35 smithi122 ceph-mon[32114]: pgmap v675: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 508 op/s 2024-04-02T13:56:36.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 238051 26.09 MB/sec execute 356 sec latency 125.062 ms 2024-04-02T13:56:37.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 238736 26.11 MB/sec execute 357 sec latency 68.445 ms 2024-04-02T13:56:37.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:37 smithi187 ceph-mon[36064]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-04-02T13:56:37.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:37 smithi188 ceph-mon[36117]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-04-02T13:56:37.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:37 smithi122 ceph-mon[32114]: pgmap v676: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-04-02T13:56:38.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 239215 26.12 MB/sec execute 358 sec latency 159.904 ms 2024-04-02T13:56:39.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 239748 26.15 MB/sec execute 359 sec latency 160.101 ms 2024-04-02T13:56:39.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:39 smithi187 ceph-mon[36064]: pgmap v677: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-04-02T13:56:39.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:39 smithi188 ceph-mon[36117]: pgmap v677: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-04-02T13:56:39.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:39 smithi122 ceph-mon[32114]: pgmap v677: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.5 MiB/s wr, 610 op/s 2024-04-02T13:56:40.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 240172 26.13 MB/sec execute 360 sec latency 160.745 ms 2024-04-02T13:56:40.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:40.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.059+0000 7fa9c471c640 1 -- 172.21.15.122:0/1509183395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc1020f0 msgr2=0x7fa9bc105690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.059+0000 7fa9c471c640 1 --2- 172.21.15.122:0/1509183395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc1020f0 0x7fa9bc105690 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009900 tx=0x7fa9b802bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.060+0000 7fa9c471c640 1 -- 172.21.15.122:0/1509183395 shutdown_connections 2024-04-02T13:56:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.060+0000 7fa9c471c640 1 --2- 172.21.15.122:0/1509183395 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc105dc0 0x7fa9bc10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.060+0000 7fa9c471c640 1 --2- 172.21.15.122:0/1509183395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc1020f0 0x7fa9bc105690 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.060+0000 7fa9c471c640 1 --2- 172.21.15.122:0/1509183395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc101740 0x7fa9bc101b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.060+0000 7fa9c471c640 1 -- 172.21.15.122:0/1509183395 >> 172.21.15.122:0/1509183395 conn(0x7fa9bc075d40 msgr2=0x7fa9bc076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.063+0000 7fa9c471c640 1 -- 172.21.15.122:0/1509183395 shutdown_connections 2024-04-02T13:56:41.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.063+0000 7fa9c471c640 1 -- 172.21.15.122:0/1509183395 wait complete. 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 Processor -- start 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 -- start start 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc101740 0x7fa9bc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc105dc0 0x7fa9bc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10fd20 con 0x7fa9bc19ee40 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10fba0 con 0x7fa9bc105dc0 2024-04-02T13:56:41.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c471c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10fea0 con 0x7fa9bc101740 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c2cb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c2cb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38792/0 (socket says 172.21.15.122:38792) 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/3976913349 learned_addr learned my addr 172.21.15.122:0/3976913349 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc101740 0x7fa9bc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.064+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc105dc0 0x7fa9bc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/3976913349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc101740 msgr2=0x7fa9bc19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc101740 0x7fa9bc19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/3976913349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc105dc0 msgr2=0x7fa9bc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc105dc0 0x7fa9bc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/3976913349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc19f930 con 0x7fa9bc19ee40 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e970 tx=0x7fa9ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9b37fe640 1 -- 172.21.15.122:0/3976913349 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac00ccc0 con 0x7fa9bc19ee40 2024-04-02T13:56:41.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc19fc20 con 0x7fa9bc19ee40 2024-04-02T13:56:41.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9b37fe640 1 -- 172.21.15.122:0/3976913349 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac00ce60 con 0x7fa9bc19ee40 2024-04-02T13:56:41.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1a73e0 con 0x7fa9bc19ee40 2024-04-02T13:56:41.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.065+0000 7fa9b37fe640 1 -- 172.21.15.122:0/3976913349 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac010450 con 0x7fa9bc19ee40 2024-04-02T13:56:41.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.069+0000 7fa9c73e9740 1 Processor -- start 2024-04-02T13:56:41.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.069+0000 7fa9b37fe640 1 -- 172.21.15.122:0/3976913349 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa9ac005020 con 0x7fa9bc19ee40 2024-04-02T13:56:41.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.070+0000 7fa9b37fe640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 0x7fa9a80a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.070+0000 7fa9b37fe640 1 -- 172.21.15.122:0/3976913349 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa9ac004dd0 con 0x7fa9bc19ee40 2024-04-02T13:56:41.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.070+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 0x7fa9a80a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.070+0000 7fa9c73e9740 1 -- start start 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e1acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 0x7fa9a80a63c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802c170 tx=0x7fa9b802df90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 0x562864e261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562864d7e830 con 0x562864e1b1f0 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562864d26820 con 0x562864e23dd0 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562864d7a7e0 con 0x562864e1a8d0 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e1acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e1acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39114/0 (socket says 172.21.15.122:39114) 2024-04-02T13:56:41.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c24b8640 1 -- 172.21.15.122:0/318508922 learned_addr learned my addr 172.21.15.122:0/318508922 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 0x562864e261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/318508922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 msgr2=0x562864e1acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e1acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/318508922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 msgr2=0x562864e23890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e23890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/318508922 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562864b5c7d0 con 0x562864e23dd0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.071+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.072+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 0x562864e261c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0ce190 tx=0x7fa9ac010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e607fc88963855 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.072+0000 7fa9b0ff9640 1 -- 172.21.15.122:0/318508922 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac09c9e0 con 0x562864e23dd0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.072+0000 7fa9b0ff9640 1 -- 172.21.15.122:0/318508922 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac0a0d90 con 0x562864e23dd0 2024-04-02T13:56:41.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.072+0000 7fa9b0ff9640 1 -- 172.21.15.122:0/318508922 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac005310 con 0x562864e23dd0 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- 172.21.15.122:0/318508922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 msgr2=0x562864e261c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 0x562864e261c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0ce190 tx=0x7fa9ac010430 comp rx=0 tx=0).stop 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- 172.21.15.122:0/318508922 shutdown_connections 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e23dd0 0x562864e261c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e23890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/318508922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e1acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- 172.21.15.122:0/318508922 >> 172.21.15.122:0/318508922 conn(0x562864e15000 msgr2=0x562864e153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- 172.21.15.122:0/318508922 shutdown_connections 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- 172.21.15.122:0/318508922 wait complete. 2024-04-02T13:56:41.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 -- start start 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.073+0000 7fa9c73e9740 1 Processor -- start 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e20a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 0x562864e20fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562864e26c10 con 0x562864e1b1f0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562864d39d30 con 0x562864e1a8d0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562864e26d90 con 0x562864e23dd0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c73e9740 10 client.0 _ll_register_callbacks cb 0x562864d39720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:53914/0 (socket says 172.21.15.122:53914) 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c24b8640 1 -- 172.21.15.122:0/1732876537 learned_addr learned my addr 172.21.15.122:0/1732876537 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e20a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 0x562864e20fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 msgr2=0x562864e236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/1732876537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 msgr2=0x562864e20a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e20a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c2cb9640 1 -- 172.21.15.122:0/1732876537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562864a923f0 con 0x562864e23dd0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e20a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.074+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 0x562864e20fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac09ac90 tx=0x7fa9ac0bd190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c17755057d03bb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac09c9e0 con 0x562864e23dd0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa9ac005310 con 0x562864e23dd0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ac0a0d90 con 0x562864e23dd0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa9ac0a51c0 con 0x562864e23dd0 2024-04-02T13:56:41.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9c73e9740 10 client.30094 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.075+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562864a520f0 con 0x562864e23dd0 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ac0a0d90 con 0x562864e23dd0 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa9ac09bf50 con 0x562864e23dd0 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 10 client.30094 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 10 client.30094 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 20 client.30094 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e33c90 0x562864e36080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 4 client.30094 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562864d8bd70 con 0x562864e33c90 2024-04-02T13:56:41.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.076+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e33c90 0x562864e36080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.078+0000 7fa9a5ffb640 10 client.30094 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:41.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.078+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e33c90 0x562864e36080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ca4cfa45e18bb0 server_cookie=2541ad8fe0cd437e in_seq=0 out_seq=0 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.080+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/1732876537 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562864d8bd70 con 0x562864e33c90 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.080+0000 7fa9a5ffb640 10 client.30094 handle_command_reply: tid=0 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.080+0000 7fa9c73e9740 1 client.30094 shutdown 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.080+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 msgr2=0x562864e20fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.080+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 0x562864e20fc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac09ac90 tx=0x7fa9ac0bd190 comp rx=0 tx=0).stop 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 shutdown_connections 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e33c90 0x562864e36080 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e23dd0 0x562864e20fc0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1b1f0 0x562864e20a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/1732876537 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1a8d0 0x562864e236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 >> 172.21.15.122:0/1732876537 conn(0x562864e15000 msgr2=0x562864e16d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9a5ffb640 0 client.30094 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 shutdown_connections 2024-04-02T13:56:41.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 1 -- 172.21.15.122:0/1732876537 wait complete. 2024-04-02T13:56:41.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.081+0000 7fa9c73e9740 20 client.30094 trim_cache size 0 max 16384 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 Processor -- start 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 -- start start 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1b1f0 0x562864e236f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1b1330 con 0x562864e15000 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1b2e70 con 0x562864e1b1f0 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1b1a00 con 0x562864e1a8d0 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.104+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38830/0 (socket says 172.21.15.122:38830) 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 -- 172.21.15.122:0/3298886085 learned_addr learned my addr 172.21.15.122:0/3298886085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1b1f0 0x562864e236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 -- 172.21.15.122:0/3298886085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 msgr2=0x562864e231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 -- 172.21.15.122:0/3298886085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1b1f0 msgr2=0x562864e236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1b1f0 0x562864e236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 -- 172.21.15.122:0/3298886085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562864e1b020 con 0x562864e15000 2024-04-02T13:56:41.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009f40 tx=0x7fa9b4014d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6929c741c70bd18c server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9a7fff640 1 -- 172.21.15.122:0/3298886085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4011b40 con 0x562864e15000 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9a7fff640 1 -- 172.21.15.122:0/3298886085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b4011ce0 con 0x562864e15000 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.105+0000 7fa9a7fff640 1 -- 172.21.15.122:0/3298886085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b401f890 con 0x562864e15000 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.106+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3298886085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 msgr2=0x562864e1f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.106+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009f40 tx=0x7fa9b4014d50 comp rx=0 tx=0).stop 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3298886085 shutdown_connections 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e1b1f0 0x562864e236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1a8d0 0x562864e231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3298886085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e15000 0x562864e1f350 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3298886085 >> 172.21.15.122:0/3298886085 conn(0x562864e21120 msgr2=0x562864e16ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3298886085 shutdown_connections 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3298886085 wait complete. 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 -- start start 2024-04-02T13:56:41.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 Processor -- start 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 0x562864e39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.107+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 0x562864e36490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c73e9740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1b1f0 0x562864e369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1b27a0 con 0x562864e1a8d0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1b20d0 con 0x562864e15000 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c73e9740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc1af330 con 0x562864e1b1f0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c73e9740 10 client.0 _ll_register_callbacks cb 0x7fa9b4009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 0x562864e39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 0x562864e36490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c24b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 0x562864e39100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:53930/0 (socket says 172.21.15.122:53930) 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c24b8640 1 -- 172.21.15.122:0/3880625223 learned_addr learned my addr 172.21.15.122:0/3880625223 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c2cb9640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1b1f0 0x562864e369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 -- 172.21.15.122:0/3880625223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1b1f0 msgr2=0x562864e369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1b1f0 0x562864e369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 -- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 msgr2=0x562864e39100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 0x562864e39100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 -- 172.21.15.122:0/3880625223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562864e37220 con 0x562864e1a8d0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9c1cb7640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 0x562864e36490 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009230 tx=0x7fa9b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2317a839bfecb46a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.108+0000 7fa9a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8004e90 con 0x562864e1a8d0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fa9b803a040 con 0x562864e1a8d0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9b4011ce0 con 0x562864e1a8d0 2024-04-02T13:56:41.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa9b8005030 con 0x562864e1a8d0 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9c73e9740 10 client.21450 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562864ea2720 con 0x562864e1a8d0 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.109+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9b4011ce0 con 0x562864e1a8d0 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa9b8003d10 con 0x562864e1a8d0 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 10 client.21450 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 10 client.21450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 20 client.21450 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e23dd0 0x562864ea4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 4 client.21450 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562864d860c0 con 0x562864e23dd0 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.110+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e23dd0 0x562864ea4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.111+0000 7fa9c24b8640 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e23dd0 0x562864ea4e80 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eef25a64965355c9 server_cookie=53bb6ddc1d66b214 in_seq=0 out_seq=0 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.111+0000 7fa9a5ffb640 10 client.21450 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.112+0000 7fa9a5ffb640 1 -- 172.21.15.122:0/3880625223 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562864d860c0 con 0x562864e23dd0 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.112+0000 7fa9a5ffb640 10 client.21450 handle_command_reply: tid=0 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.112+0000 7fa9c73e9740 1 client.21450 shutdown 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 msgr2=0x562864e36490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 0x562864e36490 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009230 tx=0x7fa9b8009260 comp rx=0 tx=0).stop 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 shutdown_connections 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x562864e23dd0 0x562864ea4e80 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562864e1b1f0 0x562864e369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9a5ffb640 0 client.21450 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562864e1a8d0 0x562864e36490 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 --2- 172.21.15.122:0/3880625223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562864e15000 0x562864e39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.113+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 >> 172.21.15.122:0/3880625223 conn(0x562864e21120 msgr2=0x562864e163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.114+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 shutdown_connections 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.114+0000 7fa9c73e9740 1 -- 172.21.15.122:0/3880625223 wait complete. 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.114+0000 7fa9c73e9740 20 client.21450 trim_cache size 0 max 16384 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:56:41.120 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.115+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 msgr2=0x7fa9a80a63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.115+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 0x7fa9a80a63c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa9b802c170 tx=0x7fa9b802df90 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.115+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 msgr2=0x7fa9bc19f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.115+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00e970 tx=0x7fa9ac00ed70 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 shutdown_connections 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9a80a3f00 0x7fa9a80a63c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9bc19ee40 0x7fa9bc19f2f0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9bc105dc0 0x7fa9bc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 --2- 172.21.15.122:0/3976913349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9bc101740 0x7fa9bc19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.116+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 >> 172.21.15.122:0/3976913349 conn(0x7fa9bc075d40 msgr2=0x7fa9bc0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.117+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 shutdown_connections 2024-04-02T13:56:41.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.117+0000 7fa9c471c640 1 -- 172.21.15.122:0/3976913349 wait complete. 2024-04-02T13:56:41.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:56:41.136 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 -- 172.21.15.122:0/784588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde20103f30 msgr2=0x7fde20104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 --2- 172.21.15.122:0/784588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde20103f30 0x7fde20104310 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fde1c009900 tx=0x7fde1c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 -- 172.21.15.122:0/784588096 shutdown_connections 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 --2- 172.21.15.122:0/784588096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 0x7fde20110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 --2- 172.21.15.122:0/784588096 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde201048e0 0x7fde20108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 --2- 172.21.15.122:0/784588096 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde20103f30 0x7fde20104310 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 -- 172.21.15.122:0/784588096 >> 172.21.15.122:0/784588096 conn(0x7fde200ff7b0 msgr2=0x7fde20101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 -- 172.21.15.122:0/784588096 shutdown_connections 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.276+0000 7fde283e2640 1 -- 172.21.15.122:0/784588096 wait complete. 2024-04-02T13:56:41.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.277+0000 7fde283e2640 1 Processor -- start 2024-04-02T13:56:41.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.277+0000 7fde283e2640 1 -- start start 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.277+0000 7fde283e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.277+0000 7fde283e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 0x7fde2019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.277+0000 7fde283e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 0x7fde2019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde283e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fde201140a0 con 0x7fde2019eed0 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde283e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde20113f20 con 0x7fde20103f30 2024-04-02T13:56:41.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde283e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde20114220 con 0x7fde20109000 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:53958/0 (socket says 172.21.15.122:53958) 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 -- 172.21.15.122:0/1196700344 learned_addr learned my addr 172.21.15.122:0/1196700344 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2597d640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 0x7fde2019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2697f640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 0x7fde2019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 -- 172.21.15.122:0/1196700344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 msgr2=0x7fde2019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 0x7fde2019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 -- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 msgr2=0x7fde2019f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 0x7fde2019f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 -- 172.21.15.122:0/1196700344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2019f9c0 con 0x7fde20103f30 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2697f640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 0x7fde2019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.278+0000 7fde2617e640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006010 tx=0x7fde1c02cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.279+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c041070 con 0x7fde20103f30 2024-04-02T13:56:41.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.279+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde1c004450 con 0x7fde20103f30 2024-04-02T13:56:41.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.279+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2019fc50 con 0x7fde20103f30 2024-04-02T13:56:41.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.279+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde20076230 con 0x7fde20103f30 2024-04-02T13:56:41.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.279+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c003e00 con 0x7fde20103f30 2024-04-02T13:56:41.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.281+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde200688f0 con 0x7fde20103f30 2024-04-02T13:56:41.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.283+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fde1c044070 con 0x7fde20103f30 2024-04-02T13:56:41.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.284+0000 7fde177fe640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.284+0000 7fde2597d640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.284+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fde1c0f2780 con 0x7fde20103f30 2024-04-02T13:56:41.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.285+0000 7fde2597d640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 0x7fde0c0a6260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fde200ffcd0 tx=0x7fde10005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.291+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fde1c0be490 con 0x7fde20103f30 2024-04-02T13:56:41.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 240682 26.14 MB/sec execute 361 sec latency 65.735 ms 2024-04-02T13:56:41.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.616+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde2010c230 con 0x7fde20103f30 2024-04-02T13:56:41.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.617+0000 7fde177fe640 1 -- 172.21.15.122:0/1196700344 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11942 (secure 0 0 0) 0x7fde1c0c2790 con 0x7fde20103f30 2024-04-02T13:56:41.622 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:56:41.622 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:55:07.726900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:56:41.623 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-04-02T13:56:41.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.619+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 msgr2=0x7fde0c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.619+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 0x7fde0c0a6260 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fde200ffcd0 tx=0x7fde10005f90 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 msgr2=0x7fde2019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fde1c006010 tx=0x7fde1c02cb30 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 shutdown_connections 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fde0c0a3da0 0x7fde0c0a6260 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fde2019eed0 0x7fde2019f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde20109000 0x7fde2019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 --2- 172.21.15.122:0/1196700344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde20103f30 0x7fde2019a660 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 >> 172.21.15.122:0/1196700344 conn(0x7fde200ff7b0 msgr2=0x7fde20100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 shutdown_connections 2024-04-02T13:56:41.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.620+0000 7fde283e2640 1 -- 172.21.15.122:0/1196700344 wait complete. 2024-04-02T13:56:41.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:56:41.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:41 smithi187 ceph-mon[36064]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 475 op/s 2024-04-02T13:56:41.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:41 smithi188 ceph-mon[36117]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 475 op/s 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.791+0000 7fd67e760640 1 -- 172.21.15.122:0/428978884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678105ec0 msgr2=0x7fd678106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.791+0000 7fd67e760640 1 --2- 172.21.15.122:0/428978884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678105ec0 0x7fd678106350 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fd670009ca0 tx=0x7fd67002c1f0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.792+0000 7fd67e760640 1 -- 172.21.15.122:0/428978884 shutdown_connections 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.792+0000 7fd67e760640 1 --2- 172.21.15.122:0/428978884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678105ec0 0x7fd678106350 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.792+0000 7fd67e760640 1 --2- 172.21.15.122:0/428978884 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6781013e0 0x7fd6781058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.792+0000 7fd67e760640 1 --2- 172.21.15.122:0/428978884 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd678100ac0 0x7fd678100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.792+0000 7fd67e760640 1 -- 172.21.15.122:0/428978884 >> 172.21.15.122:0/428978884 conn(0x7fd6780fc4d0 msgr2=0x7fd6780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.795+0000 7fd67e760640 1 -- 172.21.15.122:0/428978884 shutdown_connections 2024-04-02T13:56:41.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.795+0000 7fd67e760640 1 -- 172.21.15.122:0/428978884 wait complete. 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 Processor -- start 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 -- start start 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6781013e0 0x7fd6781a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd678105ec0 0x7fd67819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6781185b0 con 0x7fd678100ac0 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd678118430 con 0x7fd6781013e0 2024-04-02T13:56:41.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd67e760640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd678118730 con 0x7fd678105ec0 2024-04-02T13:56:41.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.796+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.797+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38868/0 (socket says 172.21.15.122:38868) 2024-04-02T13:56:41.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.797+0000 7fd677fff640 1 -- 172.21.15.122:0/1589636077 learned_addr learned my addr 172.21.15.122:0/1589636077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.798+0000 7fd677fff640 1 -- 172.21.15.122:0/1589636077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd678105ec0 msgr2=0x7fd67819e870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:56:41.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.798+0000 7fd677fff640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd678105ec0 0x7fd67819e870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.798+0000 7fd677fff640 1 -- 172.21.15.122:0/1589636077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6781013e0 msgr2=0x7fd6781a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.798+0000 7fd677fff640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6781013e0 0x7fd6781a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.798+0000 7fd677fff640 1 -- 172.21.15.122:0/1589636077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67819efc0 con 0x7fd678100ac0 2024-04-02T13:56:41.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.799+0000 7fd677fff640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b450 tx=0x7fd66800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.800+0000 7fd6757fa640 1 -- 172.21.15.122:0/1589636077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668015020 con 0x7fd678100ac0 2024-04-02T13:56:41.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.800+0000 7fd6757fa640 1 -- 172.21.15.122:0/1589636077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd668011d70 con 0x7fd678100ac0 2024-04-02T13:56:41.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.800+0000 7fd6757fa640 1 -- 172.21.15.122:0/1589636077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66800f340 con 0x7fd678100ac0 2024-04-02T13:56:41.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.800+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67819f2b0 con 0x7fd678100ac0 2024-04-02T13:56:41.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.801+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd67806b520 con 0x7fd678100ac0 2024-04-02T13:56:41.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.804+0000 7fd681425740 1 Processor -- start 2024-04-02T13:56:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.808+0000 7fd6757fa640 1 -- 172.21.15.122:0/1589636077 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd668018020 con 0x7fd678100ac0 2024-04-02T13:56:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.808+0000 7fd6757fa640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 0x7fd6600a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd6757fa640 1 -- 172.21.15.122:0/1589636077 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fd6680d03e0 con 0x7fd678100ac0 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 -- start start 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbc0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 0x563b8cbcc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563b8cb247b0 con 0x563b8cbc07a0 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563b8cac3570 con 0x563b8cbc9ca0 2024-04-02T13:56:41.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.809+0000 7fd681425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563b8cb20760 con 0x563b8cbc10c0 2024-04-02T13:56:41.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd67ccfd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 0x563b8cbcc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd67ccfd640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 0x7fd6600a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd67ccfd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 0x563b8cbcc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:53988/0 (socket says 172.21.15.122:53988) 2024-04-02T13:56:41.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd67ccfd640 1 -- 172.21.15.122:0/455518954 learned_addr learned my addr 172.21.15.122:0/455518954 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbc0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 -- 172.21.15.122:0/455518954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc10c0 msgr2=0x563b8cbc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 -- 172.21.15.122:0/455518954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 msgr2=0x563b8cbcc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 0x563b8cbcc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 -- 172.21.15.122:0/455518954 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8c8383f0 con 0x563b8cbc07a0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.811+0000 7fd677fff640 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbc0b80 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cd7f0 tx=0x7fd66809c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b7465d65fc96d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.812+0000 7fd65e7fc640 1 -- 172.21.15.122:0/455518954 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680a03f0 con 0x563b8cbc07a0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.812+0000 7fd65e7fc640 1 -- 172.21.15.122:0/455518954 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680a4cb0 con 0x563b8cbc07a0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.812+0000 7fd65e7fc640 1 -- 172.21.15.122:0/455518954 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680cd960 con 0x563b8cbc07a0 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.813+0000 7fd681425740 1 -- 172.21.15.122:0/455518954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 msgr2=0x563b8cbc0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.813+0000 7fd681425740 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbc0b80 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fd6680cd7f0 tx=0x7fd66809c740 comp rx=0 tx=0).stop 2024-04-02T13:56:41.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.813+0000 7fd67ccfd640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 0x7fd6600a63f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd670032f40 tx=0x7fd670005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 -- 172.21.15.122:0/455518954 shutdown_connections 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc9ca0 0x563b8cbcc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 --2- 172.21.15.122:0/455518954 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbc0b80 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 -- 172.21.15.122:0/455518954 >> 172.21.15.122:0/455518954 conn(0x563b8cbbaed0 msgr2=0x563b8cbbb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 -- 172.21.15.122:0/455518954 shutdown_connections 2024-04-02T13:56:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 -- 172.21.15.122:0/455518954 wait complete. 2024-04-02T13:56:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.814+0000 7fd681425740 1 -- start start 2024-04-02T13:56:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 Processor -- start 2024-04-02T13:56:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc07a0 0x563b8cbc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563b8cbcca80 con 0x563b8cbc10c0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x563b8cadfe80 con 0x563b8cbc07a0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563b8cbccc00 con 0x563b8cbc9ca0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd681425740 10 client.0 _ll_register_callbacks cb 0x563b8cb1baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38894/0 (socket says 172.21.15.122:38894) 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd6777fe640 1 -- 172.21.15.122:0/1387116642 learned_addr learned my addr 172.21.15.122:0/1387116642 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.815+0000 7fd677fff640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc07a0 0x563b8cbc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 -- 172.21.15.122:0/1387116642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 msgr2=0x563b8cbc7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 -- 172.21.15.122:0/1387116642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc07a0 msgr2=0x563b8cbc9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc07a0 0x563b8cbc9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 -- 172.21.15.122:0/1387116642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8c9027d0 con 0x563b8cbc10c0 2024-04-02T13:56:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.816+0000 7fd6777fe640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00bbe0 tx=0x7fd66c00d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab997be724f254a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.817+0000 7fd6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:41.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:41 smithi122 ceph-mon[32114]: pgmap v678: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 475 op/s 2024-04-02T13:56:41.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.817+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66c016030 con 0x563b8cbc10c0 2024-04-02T13:56:41.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.818+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6680a4cb0 con 0x563b8cbc10c0 2024-04-02T13:56:41.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.818+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fd66c005020 con 0x563b8cbc10c0 2024-04-02T13:56:41.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.819+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd66c005220 con 0x563b8cbc10c0 2024-04-02T13:56:41.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.819+0000 7fd681425740 10 client.21468 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:41.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.819+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563b8cbd8b90 con 0x563b8cbc10c0 2024-04-02T13:56:41.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.821+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680a4cb0 con 0x563b8cbc10c0 2024-04-02T13:56:41.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.821+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd66c010590 con 0x563b8cbc10c0 2024-04-02T13:56:41.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 10 client.21468 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:41.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 10 client.21468 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:56:41.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 20 client.21468 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cbda050 0x563b8cbdc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 4 client.21468 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.822+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b8cb31cf0 con 0x563b8cbda050 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.823+0000 7fd677fff640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cbda050 0x563b8cbdc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.823+0000 7fd677fff640 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cbda050 0x563b8cbdc470 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f7591426151314d6 server_cookie=b7291d9efc0db607 in_seq=0 out_seq=0 2024-04-02T13:56:41.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.823+0000 7fd6577fe640 10 client.21468 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:41.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.824+0000 7fd6577fe640 1 -- 172.21.15.122:0/1387116642 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b8cb31cf0 con 0x563b8cbda050 2024-04-02T13:56:41.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.824+0000 7fd6577fe640 10 client.21468 handle_command_reply: tid=0 2024-04-02T13:56:41.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.824+0000 7fd681425740 1 client.21468 shutdown 2024-04-02T13:56:41.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.825+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 msgr2=0x563b8cbc6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.825+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00bbe0 tx=0x7fd66c00d6b0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.826+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 shutdown_connections 2024-04-02T13:56:41.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cbda050 0x563b8cbdc470 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc10c0 0x563b8cbc6af0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd681425740 1 --2- 172.21.15.122:0/1387116642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc07a0 0x563b8cbc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 >> 172.21.15.122:0/1387116642 conn(0x563b8cbbaed0 msgr2=0x563b8cbbcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.827+0000 7fd6577fe640 0 client.21468 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:41.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.828+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 shutdown_connections 2024-04-02T13:56:41.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.828+0000 7fd681425740 1 -- 172.21.15.122:0/1387116642 wait complete. 2024-04-02T13:56:41.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.828+0000 7fd681425740 20 client.21468 trim_cache size 0 max 16384 2024-04-02T13:56:41.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.856+0000 7fd681425740 1 Processor -- start 2024-04-02T13:56:41.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.856+0000 7fd681425740 1 -- start start 2024-04-02T13:56:41.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.856+0000 7fd681425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd681425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd681425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd681425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd678109b60 con 0x563b8cbc07a0 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd681425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd67810b6a0 con 0x563b8cbc10c0 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd681425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd67810a230 con 0x563b8cbc9ca0 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38896/0 (socket says 172.21.15.122:38896) 2024-04-02T13:56:41.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.857+0000 7fd677fff640 1 -- 172.21.15.122:0/810961497 learned_addr learned my addr 172.21.15.122:0/810961497 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 -- 172.21.15.122:0/810961497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 msgr2=0x563b8cbc9760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:41.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 -- 172.21.15.122:0/810961497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 msgr2=0x563b8cbc9220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:41.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 -- 172.21.15.122:0/810961497 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8cbc0eb0 con 0x563b8cbc07a0 2024-04-02T13:56:41.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd677fff640 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd6680103e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5beb7ce192fb2437 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd65dffb640 1 -- 172.21.15.122:0/810961497 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66809a340 con 0x563b8cbc07a0 2024-04-02T13:56:41.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd65dffb640 1 -- 172.21.15.122:0/810961497 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680a00b0 con 0x563b8cbc07a0 2024-04-02T13:56:41.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.859+0000 7fd65dffb640 1 -- 172.21.15.122:0/810961497 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668096bb0 con 0x563b8cbc07a0 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 -- 172.21.15.122:0/810961497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 msgr2=0x563b8cbd5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fd668009fd0 tx=0x7fd6680103e0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 -- 172.21.15.122:0/810961497 shutdown_connections 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbc9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbc9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 --2- 172.21.15.122:0/810961497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbd5940 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.861+0000 7fd681425740 1 -- 172.21.15.122:0/810961497 >> 172.21.15.122:0/810961497 conn(0x563b8cbbaed0 msgr2=0x563b8cbc1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.864+0000 7fd681425740 1 -- 172.21.15.122:0/810961497 shutdown_connections 2024-04-02T13:56:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.865+0000 7fd681425740 1 -- 172.21.15.122:0/810961497 wait complete. 2024-04-02T13:56:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.865+0000 7fd681425740 1 -- start start 2024-04-02T13:56:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.865+0000 7fd681425740 1 Processor -- start 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbcad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbd0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd67810afd0 con 0x563b8cbc07a0 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd67810a900 con 0x563b8cbc10c0 2024-04-02T13:56:41.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd678068d40 con 0x563b8cbc9ca0 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd681425740 10 client.0 _ll_register_callbacks cb 0x7fd67806b0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd677fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38900/0 (socket says 172.21.15.122:38900) 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd677fff640 1 -- 172.21.15.122:0/919130677 learned_addr learned my addr 172.21.15.122:0/919130677 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.866+0000 7fd6777fe640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbcad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd67ccfd640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbd0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 -- 172.21.15.122:0/919130677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 msgr2=0x563b8cbd0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbd0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 -- 172.21.15.122:0/919130677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 msgr2=0x563b8cbcad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:41.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbcad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 -- 172.21.15.122:0/919130677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8cbd13c0 con 0x563b8cbc07a0 2024-04-02T13:56:41.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.867+0000 7fd677fff640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd66809bd40 tx=0x7fd6680d0220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cc29924e4d19af server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.868+0000 7fd6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.868+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6680a03f0 con 0x563b8cbc07a0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.868+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6680a00b0 con 0x563b8cbc07a0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.868+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7fd6680cd960 con 0x563b8cbc07a0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.869+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd668096bb0 con 0x563b8cbc07a0 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.869+0000 7fd681425740 10 client.21474 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:41.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.869+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563b8cbd13c0 con 0x563b8cbc07a0 2024-04-02T13:56:41.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6680a00b0 con 0x563b8cbc07a0 2024-04-02T13:56:41.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fd668022060 con 0x563b8cbc07a0 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 10 client.21474 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 10 client.21474 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 20 client.21474 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cc40600 0x563b8cc429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 4 client.21474 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.870+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b8cb2c040 con 0x563b8cc40600 2024-04-02T13:56:41.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.871+0000 7fd6777fe640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cc40600 0x563b8cc429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:41.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.872+0000 7fd6777fe640 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cc40600 0x563b8cc429f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69abc9fe740753b4 server_cookie=3bae4d82f20d3162 in_seq=0 out_seq=0 2024-04-02T13:56:41.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.872+0000 7fd6577fe640 10 client.21474 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:41.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.878+0000 7fd6577fe640 1 -- 172.21.15.122:0/919130677 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b8cb2c040 con 0x563b8cc40600 2024-04-02T13:56:41.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.878+0000 7fd6577fe640 10 client.21474 handle_command_reply: tid=0 2024-04-02T13:56:41.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.879+0000 7fd681425740 1 client.21474 shutdown 2024-04-02T13:56:41.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.881+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 msgr2=0x563b8cbca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.881+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fd66809bd40 tx=0x7fd6680d0220 comp rx=0 tx=0).stop 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 shutdown_connections 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x563b8cc40600 0x563b8cc429f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563b8cbc9ca0 0x563b8cbd0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x563b8cbc10c0 0x563b8cbcad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 --2- 172.21.15.122:0/919130677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563b8cbc07a0 0x563b8cbca850 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 >> 172.21.15.122:0/919130677 conn(0x563b8cbbaed0 msgr2=0x563b8cbc6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.883+0000 7fd6577fe640 0 client.21474 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:56:41.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.884+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 shutdown_connections 2024-04-02T13:56:41.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.884+0000 7fd681425740 1 -- 172.21.15.122:0/919130677 wait complete. 2024-04-02T13:56:41.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.885+0000 7fd681425740 20 client.21474 trim_cache size 0 max 16384 2024-04-02T13:56:41.889 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 msgr2=0x7fd6600a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 0x7fd6600a63f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd670032f40 tx=0x7fd670005d80 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 msgr2=0x7fd6781a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fd66800b450 tx=0x7fd66800b910 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 shutdown_connections 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6600a3f30 0x7fd6600a63f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd678105ec0 0x7fd67819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6781013e0 0x7fd6781a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 --2- 172.21.15.122:0/1589636077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd678100ac0 0x7fd6781a46c0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 >> 172.21.15.122:0/1589636077 conn(0x7fd6780fc4d0 msgr2=0x7fd6780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 shutdown_connections 2024-04-02T13:56:41.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:41.886+0000 7fd67e760640 1 -- 172.21.15.122:0/1589636077 wait complete. 2024-04-02T13:56:41.908 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:56:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.066+0000 7f2c4adff640 1 -- 172.21.15.122:0/348400134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c4411b2e0 msgr2=0x7f2c4411d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.066+0000 7f2c4adff640 1 --2- 172.21.15.122:0/348400134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c4411b2e0 0x7f2c4411d6d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c009be0 tx=0x7f2c3c02bef0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 -- 172.21.15.122:0/348400134 shutdown_connections 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 --2- 172.21.15.122:0/348400134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c4411b2e0 0x7f2c4411d6d0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 --2- 172.21.15.122:0/348400134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c4410e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 --2- 172.21.15.122:0/348400134 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c44110320 0x7f2c4410de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 -- 172.21.15.122:0/348400134 >> 172.21.15.122:0/348400134 conn(0x7f2c4406cbe0 msgr2=0x7f2c4406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 -- 172.21.15.122:0/348400134 shutdown_connections 2024-04-02T13:56:42.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.067+0000 7f2c4adff640 1 -- 172.21.15.122:0/348400134 wait complete. 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 Processor -- start 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 -- start start 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c441a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c4411b2e0 0x7f2c441ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44120c90 con 0x7f2c44110320 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44120b10 con 0x7f2c4411b2e0 2024-04-02T13:56:42.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4adff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c44120e10 con 0x7f2c4410e350 2024-04-02T13:56:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38912/0 (socket says 172.21.15.122:38912) 2024-04-02T13:56:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c43fff640 1 -- 172.21.15.122:0/4134739762 learned_addr learned my addr 172.21.15.122:0/4134739762 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c441a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.068+0000 7f2c4939c640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c4411b2e0 0x7f2c441ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 -- 172.21.15.122:0/4134739762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 msgr2=0x7f2c441a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c441a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 -- 172.21.15.122:0/4134739762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c4411b2e0 msgr2=0x7f2c441ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c4411b2e0 0x7f2c441ab950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 -- 172.21.15.122:0/4134739762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c441ac080 con 0x7f2c44110320 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c441a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.069+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800e9c0 tx=0x7f2c3800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.070+0000 7f2c41ffb640 1 -- 172.21.15.122:0/4134739762 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3800ccc0 con 0x7f2c44110320 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.070+0000 7f2c41ffb640 1 -- 172.21.15.122:0/4134739762 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38005500 con 0x7f2c44110320 2024-04-02T13:56:42.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.070+0000 7f2c41ffb640 1 -- 172.21.15.122:0/4134739762 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38010430 con 0x7f2c44110320 2024-04-02T13:56:42.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.071+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c441ac370 con 0x7f2c44110320 2024-04-02T13:56:42.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.071+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c441b3c00 con 0x7f2c44110320 2024-04-02T13:56:42.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.075+0000 7f2c41ffb640 1 -- 172.21.15.122:0/4134739762 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2c380046c0 con 0x7f2c44110320 2024-04-02T13:56:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.077+0000 7f2c41ffb640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 0x7f2c2c0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.077+0000 7f2c41ffb640 1 -- 172.21.15.122:0/4134739762 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2c380d0ed0 con 0x7f2c44110320 2024-04-02T13:56:42.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.078+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 0x7f2c2c0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.078+0000 7f2c4d9e6740 1 Processor -- start 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.078+0000 7f2c4d9e6740 1 -- start start 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 0x7f2c2c0a63e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400bf50 tx=0x7f2c3400b390 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa490c0 0x56299aa51760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa54090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56299a9ac7b0 con 0x56299aa51ca0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56299a94b570 con 0x56299aa490c0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4d9e6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56299a9a8760 con 0x56299aa487a0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39252/0 (socket says 172.21.15.122:39252) 2024-04-02T13:56:42.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 -- 172.21.15.122:0/1597312591 learned_addr learned my addr 172.21.15.122:0/1597312591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c43fff640 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa490c0 0x56299aa51760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c4939c640 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa54090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 -- 172.21.15.122:0/1597312591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa490c0 msgr2=0x56299aa51760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa490c0 0x56299aa51760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 -- 172.21.15.122:0/1597312591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 msgr2=0x56299aa54090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa54090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.079+0000 7f2c48b9b640 1 -- 172.21.15.122:0/1597312591 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299a6c03f0 con 0x56299aa487a0 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.080+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c340058c0 tx=0x7f2c34013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b79dd3e8d8da39 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.080+0000 7f2c2affd640 1 -- 172.21.15.122:0/1597312591 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34004220 con 0x56299aa487a0 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.080+0000 7f2c2affd640 1 -- 172.21.15.122:0/1597312591 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c340043c0 con 0x56299aa487a0 2024-04-02T13:56:42.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.080+0000 7f2c2affd640 1 -- 172.21.15.122:0/1597312591 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401cc00 con 0x56299aa487a0 2024-04-02T13:56:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.081+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/1597312591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 msgr2=0x56299aa48b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.081+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2c340058c0 tx=0x7f2c34013000 comp rx=0 tx=0).stop 2024-04-02T13:56:42.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.084+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/1597312591 shutdown_connections 2024-04-02T13:56:42.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.084+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa54090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.084+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa490c0 0x56299aa51760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.084+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/1597312591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa48b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.084+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/1597312591 >> 172.21.15.122:0/1597312591 conn(0x56299aa42ed0 msgr2=0x56299aa432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.085+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/1597312591 shutdown_connections 2024-04-02T13:56:42.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.085+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/1597312591 wait complete. 2024-04-02T13:56:42.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.086+0000 7f2c4d9e6740 1 -- start start 2024-04-02T13:56:42.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.086+0000 7f2c4d9e6740 1 Processor -- start 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.086+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa51720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.086+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.086+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa4f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c4d9e6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56299aa54ae0 con 0x56299aa51ca0 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39266/0 (socket says 172.21.15.122:39266) 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c43fff640 1 -- 172.21.15.122:0/4148944489 learned_addr learned my addr 172.21.15.122:0/4148944489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa51720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56299a967e80 con 0x56299aa487a0 2024-04-02T13:56:42.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56299aa54c60 con 0x56299aa490c0 2024-04-02T13:56:42.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c4939c640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa4f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.087+0000 7f2c4d9e6740 10 client.0 _ll_register_callbacks cb 0x56299a9a3aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 -- 172.21.15.122:0/4148944489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 msgr2=0x56299aa51720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa51720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 -- 172.21.15.122:0/4148944489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 msgr2=0x56299aa4f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa4f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299a78a7d0 con 0x56299aa490c0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c4939c640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa4f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c43fff640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2c380ce230 tx=0x7f2c3809d1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b06d1ca4d6dc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c380b5030 con 0x56299aa490c0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2c380a5200 con 0x56299aa490c0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c340043c0 con 0x56299aa490c0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2c380ad050 con 0x56299aa490c0 2024-04-02T13:56:42.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c4d9e6740 10 client.30121 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.088+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56299aa609a0 con 0x56299aa490c0 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c340043c0 con 0x56299aa490c0 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2c380ba070 con 0x56299aa490c0 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 10 client.30121 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 10 client.30121 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 20 client.30121 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aa61e60 0x56299aa64290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 4 client.30121 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:42.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.089+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56299a9b9cf0 con 0x56299aa61e60 2024-04-02T13:56:42.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.090+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aa61e60 0x56299aa64290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.091+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aa61e60 0x56299aa64290 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6395579a3533d70 server_cookie=d75a00494e2d1f22 in_seq=0 out_seq=0 2024-04-02T13:56:42.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.091+0000 7f2c23fff640 10 client.30121 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:42.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c23fff640 1 -- 172.21.15.122:0/4148944489 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56299a9b9cf0 con 0x56299aa61e60 2024-04-02T13:56:42.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c23fff640 10 client.30121 handle_command_reply: tid=0 2024-04-02T13:56:42.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 client.30121 shutdown 2024-04-02T13:56:42.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 msgr2=0x56299aa4eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2c380ce230 tx=0x7f2c3809d1e0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 shutdown_connections 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aa61e60 0x56299aa64290 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa51ca0 0x56299aa4f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c23fff640 0 client.30121 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa490c0 0x56299aa4eab0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/4148944489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa51720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 >> 172.21.15.122:0/4148944489 conn(0x56299aa42ed0 msgr2=0x56299aa44c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 shutdown_connections 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/4148944489 wait complete. 2024-04-02T13:56:42.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.093+0000 7f2c4d9e6740 20 client.30121 trim_cache size 0 max 16384 2024-04-02T13:56:42.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.111+0000 7f2c4d9e6740 1 Processor -- start 2024-04-02T13:56:42.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.111+0000 7f2c4d9e6740 1 -- start start 2024-04-02T13:56:42.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.111+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.111+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39270/0 (socket says 172.21.15.122:39270) 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38942/0 (socket says 172.21.15.122:38942) 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 -- 172.21.15.122:0/720072570 learned_addr learned my addr 172.21.15.122:0/720072570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 0x56299aa511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441bddf0 con 0x56299aa490c0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441bf930 con 0x56299aa51ca0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441be4c0 con 0x56299aa487a0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4939c640 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 0x56299aa511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 -- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 msgr2=0x56299aa511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 0x56299aa511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 -- 172.21.15.122:0/720072570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 msgr2=0x56299aa4da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 -- 172.21.15.122:0/720072570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56299aa48ef0 con 0x56299aa487a0 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c4939c640 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 0x56299aa511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.112+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401c340 tx=0x7f2c3401c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31f6ac062b558248 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.113+0000 7f2c2a7fc640 1 -- 172.21.15.122:0/720072570 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34039020 con 0x56299aa487a0 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.113+0000 7f2c2a7fc640 1 -- 172.21.15.122:0/720072570 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3401c5b0 con 0x56299aa487a0 2024-04-02T13:56:42.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.113+0000 7f2c2a7fc640 1 -- 172.21.15.122:0/720072570 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3401cb50 con 0x56299aa487a0 2024-04-02T13:56:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 msgr2=0x56299aa5d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2c3401c340 tx=0x7f2c3401c760 comp rx=0 tx=0).stop 2024-04-02T13:56:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 shutdown_connections 2024-04-02T13:56:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa51ca0 0x56299aa511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/720072570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa487a0 0x56299aa5d9a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.114+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 >> 172.21.15.122:0/720072570 conn(0x56299aa42ed0 msgr2=0x56299aa4d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.115+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 shutdown_connections 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.115+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/720072570 wait complete. 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.115+0000 7f2c4d9e6740 1 -- start start 2024-04-02T13:56:42.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 Processor -- start 2024-04-02T13:56:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa5a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 0x56299aa4c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38946/0 (socket says 172.21.15.122:38946) 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c43fff640 1 -- 172.21.15.122:0/3608161453 learned_addr learned my addr 172.21.15.122:0/3608161453 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c48b9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa5a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441bf260 con 0x56299aa490c0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441beb90 con 0x56299aa487a0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2c441bbdf0 con 0x56299aa51ca0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4d9e6740 10 client.0 _ll_register_callbacks cb 0x7f2c3809c880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.116+0000 7f2c4939c640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 0x56299aa4c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 -- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 msgr2=0x56299aa4c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 0x56299aa4c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 -- 172.21.15.122:0/3608161453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 msgr2=0x56299aa4c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56299aa4aed0 con 0x56299aa487a0 2024-04-02T13:56:42.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c43fff640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c4939c640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 0x56299aa4c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c48b9b640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa5a340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2c34010c30 tx=0x7f2c340052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4efe0c4ed78e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c23fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34042060 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2c3401cad0 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.117+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c3401c5b0 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2c34039070 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 10 client.30359 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56299aac7830 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c3401c5b0 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2c34004c60 con 0x56299aa487a0 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 10 client.30359 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 10 client.30359 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:56:42.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 20 client.30359 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aac7cf0 0x56299aaca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 4 client.30359 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:42.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.118+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56299a9b4040 con 0x56299aac7cf0 2024-04-02T13:56:42.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.119+0000 7f2c43fff640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aac7cf0 0x56299aaca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.119+0000 7f2c23fff640 10 client.30359 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.119+0000 7f2c43fff640 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aac7cf0 0x56299aaca110 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fccece01e7366808 server_cookie=8e55063ca6c72087 in_seq=0 out_seq=0 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c23fff640 1 -- 172.21.15.122:0/3608161453 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56299a9b4040 con 0x56299aac7cf0 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c23fff640 10 client.30359 handle_command_reply: tid=0 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 client.30359 shutdown 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 msgr2=0x56299aa5a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa5a340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2c34010c30 tx=0x7f2c340052f0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 shutdown_connections 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56299aa51ca0 0x56299aa4c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x56299aac7cf0 0x56299aaca110 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56299aa490c0 0x56299aa4c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 --2- 172.21.15.122:0/3608161453 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x56299aa487a0 0x56299aa5a340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c23fff640 0 client.30359 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.120+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 >> 172.21.15.122:0/3608161453 conn(0x56299aa42ed0 msgr2=0x56299aa442d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 shutdown_connections 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4d9e6740 1 -- 172.21.15.122:0/3608161453 wait complete. 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4d9e6740 20 client.30359 trim_cache size 0 max 16384 2024-04-02T13:56:42.127 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 msgr2=0x7f2c2c0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 0x7f2c2c0a63e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2c3400bf50 tx=0x7f2c3400b390 comp rx=0 tx=0).stop 2024-04-02T13:56:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 msgr2=0x7f2c441a75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800e9c0 tx=0x7f2c3800ee80 comp rx=0 tx=0).stop 2024-04-02T13:56:42.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 shutdown_connections 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2c2c0a3f20 0x7f2c2c0a63e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2c4411b2e0 0x7f2c441ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.121+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c44110320 0x7f2c441a75d0 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.122+0000 7f2c4adff640 1 --2- 172.21.15.122:0/4134739762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c4410e350 0x7f2c441a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.122+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 >> 172.21.15.122:0/4134739762 conn(0x7f2c4406cbe0 msgr2=0x7f2c4411c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.122+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 shutdown_connections 2024-04-02T13:56:42.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.122+0000 7f2c4adff640 1 -- 172.21.15.122:0/4134739762 wait complete. 2024-04-02T13:56:42.139 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:56:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 -- 172.21.15.122:0/1665646383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a01006a0 msgr2=0x7f15a0100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1665646383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a01006a0 0x7f15a0100a80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1594009860 tx=0x7f159402bb70 comp rx=0 tx=0).stop 2024-04-02T13:56:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 -- 172.21.15.122:0/1665646383 shutdown_connections 2024-04-02T13:56:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1665646383 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a010ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1665646383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a0101050 0x7f15a01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1665646383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a01006a0 0x7f15a0100a80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 -- 172.21.15.122:0/1665646383 >> 172.21.15.122:0/1665646383 conn(0x7f15a0075db0 msgr2=0x7f15a00761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 -- 172.21.15.122:0/1665646383 shutdown_connections 2024-04-02T13:56:42.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.293+0000 7f15a6a55640 1 -- 172.21.15.122:0/1665646383 wait complete. 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 Processor -- start 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 -- start start 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a01006a0 0x7f15a0117300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a0111570 0x7f15a0111a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a010fd80 con 0x7f15a0106f20 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15a010fc00 con 0x7f15a01006a0 2024-04-02T13:56:42.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.294+0000 7f15a6a55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15a010ff00 con 0x7f15a0111570 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38974/0 (socket says 172.21.15.122:38974) 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 -- 172.21.15.122:0/1337647702 learned_addr learned my addr 172.21.15.122:0/1337647702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159ffff640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a01006a0 0x7f15a0117300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a0111570 0x7f15a0111a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 -- 172.21.15.122:0/1337647702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a0111570 msgr2=0x7f15a0111a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a0111570 0x7f15a0111a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 -- 172.21.15.122:0/1337647702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a01006a0 msgr2=0x7f15a0117300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a01006a0 0x7f15a0117300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 -- 172.21.15.122:0/1337647702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a01b3b10 con 0x7f15a0106f20 2024-04-02T13:56:42.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.295+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f159000e7c0 tx=0x7f159000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.296+0000 7f159d7fa640 1 -- 172.21.15.122:0/1337647702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159000ccc0 con 0x7f15a0106f20 2024-04-02T13:56:42.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.296+0000 7f159d7fa640 1 -- 172.21.15.122:0/1337647702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1590010910 con 0x7f15a0106f20 2024-04-02T13:56:42.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.296+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a01b3ce0 con 0x7f15a0106f20 2024-04-02T13:56:42.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.296+0000 7f159d7fa640 1 -- 172.21.15.122:0/1337647702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590018800 con 0x7f15a0106f20 2024-04-02T13:56:42.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.296+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01b41a0 con 0x7f15a0106f20 2024-04-02T13:56:42.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.299+0000 7f159d7fa640 1 -- 172.21.15.122:0/1337647702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f1590010ac0 con 0x7f15a0106f20 2024-04-02T13:56:42.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.300+0000 7f159d7fa640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 0x7f15880a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.300+0000 7f159ffff640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 0x7f15880a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 Processor -- start 2024-04-02T13:56:42.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 -- start start 2024-04-02T13:56:42.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd739247b0 con 0x55cd739c10c0 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd738c3570 con 0x55cd739c07a0 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a9639740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd73920760 con 0x55cd739c9ca0 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38976/0 (socket says 172.21.15.122:38976) 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f159f7fe640 1 -- 172.21.15.122:0/247028060 learned_addr learned my addr 172.21.15.122:0/247028060 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f159ffff640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 0x7f15880a63f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594009560 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f159ffff640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.301+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159d7fa640 1 -- 172.21.15.122:0/1337647702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f15900d0ec0 con 0x7f15a0106f20 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 -- 172.21.15.122:0/247028060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 msgr2=0x55cd739cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 -- 172.21.15.122:0/247028060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 msgr2=0x55cd739c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 -- 172.21.15.122:0/247028060 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd736383f0 con 0x55cd739c07a0 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159ffff640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c0b80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1594009830 tx=0x7f159403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad6ca51139e228fe server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f159f7fe640 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.302+0000 7f15867fc640 1 -- 172.21.15.122:0/247028060 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594045050 con 0x55cd739c07a0 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.303+0000 7f15867fc640 1 -- 172.21.15.122:0/247028060 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159404f070 con 0x55cd739c07a0 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.303+0000 7f15867fc640 1 -- 172.21.15.122:0/247028060 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594039040 con 0x55cd739c07a0 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.304+0000 7f15a9639740 1 -- 172.21.15.122:0/247028060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 msgr2=0x55cd739c0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.304+0000 7f15a9639740 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c0b80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1594009830 tx=0x7f159403d000 comp rx=0 tx=0).stop 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 -- 172.21.15.122:0/247028060 shutdown_connections 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 --2- 172.21.15.122:0/247028060 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c0b80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 -- 172.21.15.122:0/247028060 >> 172.21.15.122:0/247028060 conn(0x55cd739baed0 msgr2=0x55cd739bb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 -- 172.21.15.122:0/247028060 shutdown_connections 2024-04-02T13:56:42.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.305+0000 7f15a9639740 1 -- 172.21.15.122:0/247028060 wait complete. 2024-04-02T13:56:42.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.306+0000 7f15a9639740 1 -- start start 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.306+0000 7f15a9639740 1 Processor -- start 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd739cca30 con 0x55cd739c10c0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f159ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f159ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54086/0 (socket says 172.21.15.122:54086) 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f159ffff640 1 -- 172.21.15.122:0/351142562 learned_addr learned my addr 172.21.15.122:0/351142562 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55cd738dfe80 con 0x55cd739c07a0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.307+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cd739ccbb0 con 0x55cd739c9ca0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f159f7fe640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 -- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 msgr2=0x55cd739c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 -- 172.21.15.122:0/351142562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 msgr2=0x55cd739c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd737027d0 con 0x55cd739c9ca0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a9639740 10 client.0 _ll_register_callbacks cb 0x55cd7391baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739c71c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1598005fd0 tx=0x7f1598012cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e91581f361bf89d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.308+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159404f070 con 0x55cd739c9ca0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.309+0000 7f157f7fe640 10 client.30139 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.309+0000 7f157f7fe640 1 -- 172.21.15.122:0/351142562 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159800cd20 con 0x55cd739c9ca0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.309+0000 7f157f7fe640 1 -- 172.21.15.122:0/351142562 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f159801b440 con 0x55cd739c9ca0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.309+0000 7f157f7fe640 1 -- 172.21.15.122:0/351142562 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f159801b6a0 con 0x55cd739c9ca0 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 10 client.30139 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({fsmap=76}) -- 0x55cd739d89c0 con 0x55cd739c9ca0 2024-04-02T13:56:42.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f157f7fe640 1 -- 172.21.15.122:0/351142562 <== mon.1 v2:172.21.15.188:3300/0 4 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f1598019070 con 0x55cd739c9ca0 2024-04-02T13:56:42.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 10 client.30139 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 10 client.30139 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 20 client.30139 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.310+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d9d70 0x55cd739dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.311+0000 7f15a9639740 4 client.30139 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.311+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f159404f070 con 0x55cd739d9d70 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.311+0000 7f159ffff640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d9d70 0x55cd739dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.312+0000 7f159ffff640 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d9d70 0x55cd739dc200 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11ebe807e5ed7e59 server_cookie=d545883dbad4872d in_seq=0 out_seq=0 2024-04-02T13:56:42.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.312+0000 7f157f7fe640 10 client.30139 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:42.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.315+0000 7f157f7fe640 1 -- 172.21.15.122:0/351142562 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f159404f070 con 0x55cd739d9d70 2024-04-02T13:56:42.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.315+0000 7f157f7fe640 10 client.30139 handle_command_reply: tid=0 2024-04-02T13:56:42.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.315+0000 7f15a9639740 1 client.30139 shutdown 2024-04-02T13:56:42.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 msgr2=0x55cd739c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739c71c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1598005fd0 tx=0x7f1598012cb0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 shutdown_connections 2024-04-02T13:56:42.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c9ca0 0x55cd739c71c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d9d70 0x55cd739dc200 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 --2- 172.21.15.122:0/351142562 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c07a0 0x55cd739c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 >> 172.21.15.122:0/351142562 conn(0x55cd739baed0 msgr2=0x55cd739bcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f157f7fe640 0 client.30139 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 shutdown_connections 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 1 -- 172.21.15.122:0/351142562 wait complete. 2024-04-02T13:56:42.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.316+0000 7f15a9639740 20 client.30139 trim_cache size 0 max 16384 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.336+0000 7f15a9639740 1 Processor -- start 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 -- start start 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c10c0 0x55cd739c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38998/0 (socket says 172.21.15.122:38998) 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 -- 172.21.15.122:0/3498330056 learned_addr learned my addr 172.21.15.122:0/3498330056 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01bdfd0 con 0x55cd739c07a0 2024-04-02T13:56:42.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01bfb10 con 0x55cd739c9ca0 2024-04-02T13:56:42.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01be6a0 con 0x55cd739c10c0 2024-04-02T13:56:42.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159f7fe640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c10c0 0x55cd739c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 241183 26.12 MB/sec execute 362 sec latency 143.112 ms 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 -- 172.21.15.122:0/3498330056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c10c0 msgr2=0x55cd739c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c10c0 0x55cd739c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 -- 172.21.15.122:0/3498330056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 msgr2=0x55cd739c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f159ffff640 1 -- 172.21.15.122:0/3498330056 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd739c7530 con 0x55cd739c07a0 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.337+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.338+0000 7f159ffff640 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1594009780 tx=0x7f159402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60414105375583a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.338+0000 7f1585ffb640 1 -- 172.21.15.122:0/3498330056 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594045030 con 0x55cd739c07a0 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.338+0000 7f1585ffb640 1 -- 172.21.15.122:0/3498330056 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159405d070 con 0x55cd739c07a0 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.338+0000 7f1585ffb640 1 -- 172.21.15.122:0/3498330056 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594039040 con 0x55cd739c07a0 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 msgr2=0x55cd739d58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f1594009780 tx=0x7f159402e000 comp rx=0 tx=0).stop 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 shutdown_connections 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c10c0 0x55cd739c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- 172.21.15.122:0/3498330056 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c07a0 0x55cd739d58f0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 >> 172.21.15.122:0/3498330056 conn(0x55cd739baed0 msgr2=0x55cd739c1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 shutdown_connections 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- 172.21.15.122:0/3498330056 wait complete. 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 -- start start 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 Processor -- start 2024-04-02T13:56:42.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.339+0000 7f15a9639740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a9639740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739cad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a9639740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01bf440 con 0x55cd739c10c0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a9639740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01bed70 con 0x55cd739c9ca0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a9639740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01bbfd0 con 0x55cd739c07a0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a9639740 10 client.0 _ll_register_callbacks cb 0x7f1598009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39330/0 (socket says 172.21.15.122:39330) 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 -- 172.21.15.122:0/1234409473 learned_addr learned my addr 172.21.15.122:0/1234409473 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f15a4ff2640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 -- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 msgr2=0x55cd739cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739cad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 -- 172.21.15.122:0/1234409473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 msgr2=0x55cd739ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 -- 172.21.15.122:0/1234409473 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd739c68a0 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f159ffff640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1594035c80 tx=0x7f1594005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c23116f14107dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.340+0000 7f157f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594045040 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159405d070 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f1594034440 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1594039040 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 10 client.30145 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55cd739d21b0 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159405d070 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f15940035f0 con 0x55cd739c07a0 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 10 client.30145 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 10 client.30145 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:56:42.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 20 client.30145 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d24a0 0x55cd73a42370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 4 client.30145 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.341+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd73931cf0 con 0x55cd739d24a0 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.342+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d24a0 0x55cd73a42370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.342+0000 7f157f7fe640 10 client.30145 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:42.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.342+0000 7f159f7fe640 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d24a0 0x55cd73a42370 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1bd4197f945c3d33 server_cookie=d429ddc3c098d0c6 in_seq=0 out_seq=0 2024-04-02T13:56:42.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f157f7fe640 1 -- 172.21.15.122:0/1234409473 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd73931cf0 con 0x55cd739d24a0 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f157f7fe640 10 client.30145 handle_command_reply: tid=0 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 client.30145 shutdown 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 msgr2=0x55cd739ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1594035c80 tx=0x7f1594005c90 comp rx=0 tx=0).stop 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 shutdown_connections 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x55cd739d24a0 0x55cd73a42370 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55cd739c9ca0 0x55cd739cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f157f7fe640 0 client.30145 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd739c10c0 0x55cd739ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 --2- 172.21.15.122:0/1234409473 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cd739c07a0 0x55cd739ca280 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.345+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 >> 172.21.15.122:0/1234409473 conn(0x55cd739baed0 msgr2=0x55cd739c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.346+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 shutdown_connections 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.346+0000 7f15a9639740 1 -- 172.21.15.122:0/1234409473 wait complete. 2024-04-02T13:56:42.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.346+0000 7f15a9639740 20 client.30145 trim_cache size 0 max 16384 2024-04-02T13:56:42.353 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:42.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 msgr2=0x7f15880a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 0x7f15880a63f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594009560 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 msgr2=0x7f15a0117840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f159000e7c0 tx=0x7f159000ec80 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 shutdown_connections 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f15880a3f30 0x7f15880a63f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a0111570 0x7f15a0111a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15a0106f20 0x7f15a0117840 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 --2- 172.21.15.122:0/1337647702 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f15a01006a0 0x7f15a0117300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 >> 172.21.15.122:0/1337647702 conn(0x7f15a0075db0 msgr2=0x7f15a0108030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 shutdown_connections 2024-04-02T13:56:42.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.347+0000 7f15a6a55640 1 -- 172.21.15.122:0/1337647702 wait complete. 2024-04-02T13:56:42.367 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- 172.21.15.122:0/226118462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af81048c0 msgr2=0x7f2af8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 --2- 172.21.15.122:0/226118462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af81048c0 0x7f2af8108900 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009900 tx=0x7f2ae802bc30 comp rx=0 tx=0).stop 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- 172.21.15.122:0/226118462 shutdown_connections 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 --2- 172.21.15.122:0/226118462 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af8108fe0 0x7f2af8110bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 --2- 172.21.15.122:0/226118462 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af81048c0 0x7f2af8108900 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 --2- 172.21.15.122:0/226118462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8103f10 0x7f2af81042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- 172.21.15.122:0/226118462 >> 172.21.15.122:0/226118462 conn(0x7f2af80ff750 msgr2=0x7f2af8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- 172.21.15.122:0/226118462 shutdown_connections 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- 172.21.15.122:0/226118462 wait complete. 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 Processor -- start 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.519+0000 7f2affd96640 1 -- start start 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 0x7f2af819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af819ee80 0x7f2af819f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81140a0 con 0x7f2af8103f10 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8113f20 con 0x7f2af819ee80 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2affd96640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2af8114220 con 0x7f2af8108fe0 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afdb32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 0x7f2af819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afdb32640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 0x7f2af819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39010/0 (socket says 172.21.15.122:39010) 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39340/0 (socket says 172.21.15.122:39340) 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afdb32640 1 -- 172.21.15.122:0/3769436961 learned_addr learned my addr 172.21.15.122:0/3769436961 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afe333640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af819ee80 0x7f2af819f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 -- 172.21.15.122:0/3769436961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af819ee80 msgr2=0x7f2af819f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af819ee80 0x7f2af819f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 -- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 msgr2=0x7f2af819a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 0x7f2af819a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.520+0000 7f2afd331640 1 -- 172.21.15.122:0/3769436961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af819f970 con 0x7f2af8108fe0 2024-04-02T13:56:42.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2afd331640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8036040 tx=0x7f2ae8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2aeeffd640 1 -- 172.21.15.122:0/3769436961 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8043020 con 0x7f2af8108fe0 2024-04-02T13:56:42.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2aeeffd640 1 -- 172.21.15.122:0/3769436961 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae8005000 con 0x7f2af8108fe0 2024-04-02T13:56:42.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af819fc00 con 0x7f2af8108fe0 2024-04-02T13:56:42.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2aeeffd640 1 -- 172.21.15.122:0/3769436961 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003af0 con 0x7f2af8108fe0 2024-04-02T13:56:42.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.521+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af81a73c0 con 0x7f2af8108fe0 2024-04-02T13:56:42.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.523+0000 7f2aeeffd640 1 -- 172.21.15.122:0/3769436961 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f2ae8046020 con 0x7f2af8108fe0 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.524+0000 7f2aeeffd640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 0x7f2ae40a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.524+0000 7f2aeeffd640 1 -- 172.21.15.122:0/3769436961 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2ae80f29a0 con 0x7f2af8108fe0 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.524+0000 7f2b02977740 1 Processor -- start 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.524+0000 7f2b02977740 1 -- start start 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afdb32640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 0x7f2ae40a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bd8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555a65b3c7b0 con 0x555a65bd90c0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555a65adb570 con 0x555a65bd87a0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2b02977740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555a65b38760 con 0x555a65be1ca0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39014/0 (socket says 172.21.15.122:39014) 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 -- 172.21.15.122:0/106279355 learned_addr learned my addr 172.21.15.122:0/106279355 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afdb32640 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bd8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afe333640 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 -- 172.21.15.122:0/106279355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 msgr2=0x555a65be4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 -- 172.21.15.122:0/106279355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 msgr2=0x555a65bd8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bd8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afd331640 1 -- 172.21.15.122:0/106279355 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a658503f0 con 0x555a65bd90c0 2024-04-02T13:56:42.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.525+0000 7f2afdb32640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 0x7f2ae40a62e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2af4011630 tx=0x7f2af4011a30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.526+0000 7f2afd331640 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80baa60 tx=0x7f2ae80be140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedd8d802b76cbae server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.526+0000 7f2ae3fff640 1 -- 172.21.15.122:0/106279355 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80ba040 con 0x555a65bd90c0 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.526+0000 7f2ae3fff640 1 -- 172.21.15.122:0/106279355 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae80be600 con 0x555a65bd90c0 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.526+0000 7f2ae3fff640 1 -- 172.21.15.122:0/106279355 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003c90 con 0x555a65bd90c0 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- 172.21.15.122:0/106279355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 msgr2=0x555a65be1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80baa60 tx=0x7f2ae80be140 comp rx=0 tx=0).stop 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- 172.21.15.122:0/106279355 shutdown_connections 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be1760 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 --2- 172.21.15.122:0/106279355 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bd8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- 172.21.15.122:0/106279355 >> 172.21.15.122:0/106279355 conn(0x555a65bd2ed0 msgr2=0x555a65bd32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- 172.21.15.122:0/106279355 shutdown_connections 2024-04-02T13:56:42.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- 172.21.15.122:0/106279355 wait complete. 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.527+0000 7f2b02977740 1 -- start start 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 Processor -- start 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65be1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65bdeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x555a65be4850 con 0x555a65bd90c0 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x555a65af7e80 con 0x555a65be1ca0 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555a65be49d0 con 0x555a65bd87a0 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2b02977740 10 client.0 _ll_register_callbacks cb 0x555a65b33aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2afe333640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2afe333640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54144/0 (socket says 172.21.15.122:54144) 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.528+0000 7f2afe333640 1 -- 172.21.15.122:0/3268147984 learned_addr learned my addr 172.21.15.122:0/3268147984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 -- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 msgr2=0x555a65be1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65be1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 -- 172.21.15.122:0/3268147984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 msgr2=0x555a65bdeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65bdeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 -- 172.21.15.122:0/3268147984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a6591a7d0 con 0x555a65be1ca0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2afe333640 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2af00129e0 tx=0x7f2af0012ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c209094cb6622d22 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af0003970 con 0x555a65be1ca0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae80be600 con 0x555a65be1ca0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.529+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2af0003b10 con 0x555a65be1ca0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2af0003d10 con 0x555a65be1ca0 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 10 client.30389 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x555a65bf9110 con 0x555a65be1ca0 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae80be600 con 0x555a65be1ca0 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2af0010430 con 0x555a65be1ca0 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 10 client.30389 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 10 client.30389 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 20 client.30389 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65bfa620 0x555a65bfcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 4 client.30389 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.530+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a65b49cf0 con 0x555a65bfa620 2024-04-02T13:56:42.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.531+0000 7f2afdb32640 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65bfa620 0x555a65bfcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.533+0000 7f2afdb32640 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65bfa620 0x555a65bfcab0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47a6aeb38195aa7b server_cookie=87a9b6f18faecd02 in_seq=0 out_seq=0 2024-04-02T13:56:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.533+0000 7f2ae17fa640 10 client.30389 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.534+0000 7f2ae17fa640 1 -- 172.21.15.122:0/3268147984 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a65b49cf0 con 0x555a65bfa620 2024-04-02T13:56:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.534+0000 7f2ae17fa640 10 client.30389 handle_command_reply: tid=0 2024-04-02T13:56:42.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.534+0000 7f2b02977740 1 client.30389 shutdown 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 msgr2=0x555a65bdf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2af00129e0 tx=0x7f2af0012ea0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 shutdown_connections 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65bfa620 0x555a65bfcab0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65be1ca0 0x555a65bdf1f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65bdeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 --2- 172.21.15.122:0/3268147984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65be1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 >> 172.21.15.122:0/3268147984 conn(0x555a65bd2ed0 msgr2=0x555a65bd4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2ae17fa640 0 client.30389 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:42.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 shutdown_connections 2024-04-02T13:56:42.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 1 -- 172.21.15.122:0/3268147984 wait complete. 2024-04-02T13:56:42.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.536+0000 7f2b02977740 20 client.30389 trim_cache size 0 max 16384 2024-04-02T13:56:42.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 Processor -- start 2024-04-02T13:56:42.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 -- start start 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2afdb32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81acd80 con 0x555a65bd90c0 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ae8c0 con 0x555a65bd87a0 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2b02977740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ad450 con 0x555a65be1ca0 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.566+0000 7f2afdb32640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54156/0 (socket says 172.21.15.122:54156) 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.567+0000 7f2afdb32640 1 -- 172.21.15.122:0/4027208467 learned_addr learned my addr 172.21.15.122:0/4027208467 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.567+0000 7f2afe333640 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.567+0000 7f2afd331640 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.568+0000 7f2afdb32640 1 -- 172.21.15.122:0/4027208467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 msgr2=0x555a65be1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.568+0000 7f2afdb32640 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.568+0000 7f2afdb32640 1 -- 172.21.15.122:0/4027208467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 msgr2=0x555a65be11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.568+0000 7f2afdb32640 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.568+0000 7f2afdb32640 1 -- 172.21.15.122:0/4027208467 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a65bd8ef0 con 0x555a65bd87a0 2024-04-02T13:56:42.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.570+0000 7f2afdb32640 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2af4013010 tx=0x7f2af401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cf059487b2ba072 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.571+0000 7f2ae37fe640 1 -- 172.21.15.122:0/4027208467 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af401a040 con 0x555a65bd87a0 2024-04-02T13:56:42.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.571+0000 7f2ae37fe640 1 -- 172.21.15.122:0/4027208467 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4018040 con 0x555a65bd87a0 2024-04-02T13:56:42.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.571+0000 7f2ae37fe640 1 -- 172.21.15.122:0/4027208467 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af4020d70 con 0x555a65bd87a0 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 -- 172.21.15.122:0/4027208467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 msgr2=0x555a65bdda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2af4013010 tx=0x7f2af401f000 comp rx=0 tx=0).stop 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 -- 172.21.15.122:0/4027208467 shutdown_connections 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65be1ca0 0x555a65be1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65bd90c0 0x555a65be11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 --2- 172.21.15.122:0/4027208467 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd87a0 0x555a65bdda90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 -- 172.21.15.122:0/4027208467 >> 172.21.15.122:0/4027208467 conn(0x555a65bd2ed0 msgr2=0x555a65bdd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 -- 172.21.15.122:0/4027208467 shutdown_connections 2024-04-02T13:56:42.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.572+0000 7f2b02977740 1 -- 172.21.15.122:0/4027208467 wait complete. 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 -- start start 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 Processor -- start 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65bdb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd90c0 0x555a65bdbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81ae1f0 con 0x555a65be1ca0 2024-04-02T13:56:42.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81adb20 con 0x555a65bd90c0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.573+0000 7f2b02977740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2af81aad80 con 0x555a65bd87a0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2b02977740 10 client.0 _ll_register_callbacks cb 0x7f2af0009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39040/0 (socket says 172.21.15.122:39040) 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 -- 172.21.15.122:0/2548712991 learned_addr learned my addr 172.21.15.122:0/2548712991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 -- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 msgr2=0x555a65bdb570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65bdb570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 -- 172.21.15.122:0/2548712991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd90c0 msgr2=0x555a65bdbab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd90c0 0x555a65bdbab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 -- 172.21.15.122:0/2548712991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a65beb220 con 0x555a65be1ca0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2afe333640 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f2af0000f80 tx=0x7f2af0010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55640cc66176a311 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.574+0000 7f2ae17fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.575+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af002f060 con 0x555a65be1ca0 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.575+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f2af0018040 con 0x555a65be1ca0 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.575+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2af4018040 con 0x555a65be1ca0 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.575+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2af0010cd0 con 0x555a65be1ca0 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.575+0000 7f2b02977740 10 client.21525 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.576+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x555a65bf25c0 con 0x555a65be1ca0 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af4018040 con 0x555a65be1ca0 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f2af001ca30 con 0x555a65be1ca0 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 10 client.21525 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 10 client.21525 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 20 client.21525 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65c69c00 0x555a65c6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 4 client.21525 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:42.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.577+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a65b44040 con 0x555a65c69c00 2024-04-02T13:56:42.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.580+0000 7f2afdb32640 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65c69c00 0x555a65c6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.582+0000 7f2afdb32640 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65c69c00 0x555a65c6bff0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9aa7aa428099a8e7 server_cookie=5723e6108f1c5240 in_seq=0 out_seq=0 2024-04-02T13:56:42.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2ae17fa640 10 client.21525 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2ae17fa640 1 -- 172.21.15.122:0/2548712991 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a65b44040 con 0x555a65c69c00 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2ae17fa640 10 client.21525 handle_command_reply: tid=0 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 client.21525 shutdown 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 msgr2=0x555a65beac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f2af0000f80 tx=0x7f2af0010660 comp rx=0 tx=0).stop 2024-04-02T13:56:42.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 shutdown_connections 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x555a65c69c00 0x555a65c6bff0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x555a65be1ca0 0x555a65beac30 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.583+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x555a65bd90c0 0x555a65bdbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2b02977740 1 --2- 172.21.15.122:0/2548712991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555a65bd87a0 0x555a65bdb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2ae17fa640 0 client.21525 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 >> 172.21.15.122:0/2548712991 conn(0x555a65bd2ed0 msgr2=0x555a65bd16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 shutdown_connections 2024-04-02T13:56:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2b02977740 1 -- 172.21.15.122:0/2548712991 wait complete. 2024-04-02T13:56:42.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.584+0000 7f2b02977740 20 client.21525 trim_cache size 0 max 16384 2024-04-02T13:56:42.589 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 msgr2=0x7f2ae40a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 0x7f2ae40a62e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2af4011630 tx=0x7f2af4011a30 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 msgr2=0x7f2af819ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8036040 tx=0x7f2ae8005e80 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 shutdown_connections 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f2ae40a3e20 0x7f2ae40a62e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2af819ee80 0x7f2af819f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af8108fe0 0x7f2af819ab00 secure :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8036040 tx=0x7f2ae8005e80 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 --2- 172.21.15.122:0/3769436961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2af8103f10 0x7f2af819a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.585+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 >> 172.21.15.122:0/3769436961 conn(0x7f2af80ff750 msgr2=0x7f2af8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.586+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 shutdown_connections 2024-04-02T13:56:42.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.586+0000 7f2affd96640 1 -- 172.21.15.122:0/3769436961 wait complete. 2024-04-02T13:56:42.616 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:56:42.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:42 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1196700344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- 172.21.15.122:0/110873273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7138101360 msgr2=0x7f713810d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 --2- 172.21.15.122:0/110873273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7138101360 0x7f713810d820 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7130009860 tx=0x7f713002bc20 comp rx=0 tx=0).stop 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- 172.21.15.122:0/110873273 shutdown_connections 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 --2- 172.21.15.122:0/110873273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7138101360 0x7f713810d820 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 --2- 172.21.15.122:0/110873273 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71381009c0 0x7f7138100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 --2- 172.21.15.122:0/110873273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f7138105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- 172.21.15.122:0/110873273 >> 172.21.15.122:0/110873273 conn(0x7f7138077a90 msgr2=0x7f7138077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- 172.21.15.122:0/110873273 shutdown_connections 2024-04-02T13:56:42.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- 172.21.15.122:0/110873273 wait complete. 2024-04-02T13:56:42.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 Processor -- start 2024-04-02T13:56:42.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.752+0000 7f713ee4e640 1 -- start start 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 0x7f7138198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101360 0x7f7138198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7138111ec0 con 0x7f71381058a0 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7138111d40 con 0x7f7138101360 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713ee4e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112040 con 0x7f71381009c0 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713cbea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 0x7f7138198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713cbea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 0x7f7138198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39422/0 (socket says 172.21.15.122:39422) 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39064/0 (socket says 172.21.15.122:39064) 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713cbea640 1 -- 172.21.15.122:0/2888609655 learned_addr learned my addr 172.21.15.122:0/2888609655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f7137fff640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101360 0x7f7138198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.759 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 -- 172.21.15.122:0/2888609655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 msgr2=0x7f7138198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 0x7f7138198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 -- 172.21.15.122:0/2888609655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101360 msgr2=0x7f7138198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101360 0x7f7138198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.753+0000 7f713d3eb640 1 -- 172.21.15.122:0/2888609655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713819d330 con 0x7f71381058a0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f713d3eb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f713002c160 tx=0x7f71300344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f7135ffb640 1 -- 172.21.15.122:0/2888609655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130041020 con 0x7f71381058a0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713819d5c0 con 0x7f71381058a0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f7135ffb640 1 -- 172.21.15.122:0/2888609655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f713002ce50 con 0x7f71381058a0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f7135ffb640 1 -- 172.21.15.122:0/2888609655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130039040 con 0x7f71381058a0 2024-04-02T13:56:42.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.754+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381a4eb0 con 0x7f71381058a0 2024-04-02T13:56:42.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.757+0000 7f7141a3f740 1 Processor -- start 2024-04-02T13:56:42.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.757+0000 7f7135ffb640 1 -- 172.21.15.122:0/2888609655 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7130044020 con 0x7f71381058a0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.757+0000 7f7141a3f740 1 -- start start 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 0x55d3793cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 0x55d3793d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793db4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d37932f7b0 con 0x55d3793d90b0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d3792ce570 con 0x55d3793cbdf0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7141a3f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d37932b760 con 0x55d3793cb4d0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793db4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f713cbea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 0x55d3793d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 0x55d3793cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f713cbea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 0x55d3793d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54196/0 (socket says 172.21.15.122:54196) 2024-04-02T13:56:42.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f713cbea640 1 -- 172.21.15.122:0/2810313676 learned_addr learned my addr 172.21.15.122:0/2810313676 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7135ffb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 0x7f71240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 -- 172.21.15.122:0/2810313676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 msgr2=0x55d3793d8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 0x55d3793d8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 -- 172.21.15.122:0/2810313676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 msgr2=0x55d3793db4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793db4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7137fff640 1 -- 172.21.15.122:0/2810313676 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3790433f0 con 0x55d3793cb4d0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.758+0000 7f7135ffb640 1 -- 172.21.15.122:0/2888609655 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f71300f1f60 con 0x7f71381058a0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.759+0000 7f7137fff640 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 0x55d3793cb8b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7128009fd0 tx=0x7f7128009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdfaf74d571bfb8f server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.759+0000 7f713d3eb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 0x7f71240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.759+0000 7f712effd640 1 -- 172.21.15.122:0/2810313676 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128003970 con 0x55d3793cb4d0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.759+0000 7f712effd640 1 -- 172.21.15.122:0/2810313676 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712800f730 con 0x55d3793cb4d0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.759+0000 7f712effd640 1 -- 172.21.15.122:0/2810313676 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128004320 con 0x55d3793cb4d0 2024-04-02T13:56:42.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 -- 172.21.15.122:0/2810313676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 msgr2=0x55d3793cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f713d3eb640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 0x7f71240a61e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7130005c40 tx=0x7f71300de040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 0x55d3793cb8b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7128009fd0 tx=0x7f7128009680 comp rx=0 tx=0).stop 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 -- 172.21.15.122:0/2810313676 shutdown_connections 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793db4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cbdf0 0x55d3793d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2810313676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cb4d0 0x55d3793cb8b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 -- 172.21.15.122:0/2810313676 >> 172.21.15.122:0/2810313676 conn(0x55d3793c5ed0 msgr2=0x55d3793c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 -- 172.21.15.122:0/2810313676 shutdown_connections 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.760+0000 7f7141a3f740 1 -- 172.21.15.122:0/2810313676 wait complete. 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 -- start start 2024-04-02T13:56:42.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 Processor -- start 2024-04-02T13:56:42.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 0x55d3793d8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.768 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d90b0 0x55d3793d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d3793dbe70 con 0x55d3793cb4d0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55d3792eae80 con 0x55d3793d90b0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d3793dbff0 con 0x55d3793cbdf0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f713cbea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f7137fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d90b0 0x55d3793d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39436/0 (socket says 172.21.15.122:39436) 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 -- 172.21.15.122:0/1193623719 learned_addr learned my addr 172.21.15.122:0/1193623719 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f7137fff640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 0x55d3793d8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.761+0000 7f7141a3f740 10 client.0 _ll_register_callbacks cb 0x55d379326aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 -- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d90b0 msgr2=0x55d3793d6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d90b0 0x55d3793d6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 -- 172.21.15.122:0/1193623719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 msgr2=0x55d3793d8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 0x55d3793d8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 -- 172.21.15.122:0/1193623719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d37910d7d0 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f7137fff640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 0x55d3793d8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.762+0000 7f713cbea640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7120011800 tx=0x7f7120011cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bfeeaa4e9c49dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7117fff640 10 client.0 ms_handle_connect on v2:172.21.15.188:3300/0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712002f020 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f712800f730 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mon.1 v2:172.21.15.188:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f712000fbe0 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7120037050 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7141a3f740 10 client.30169 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.763+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d3793ec420 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mon.1 v2:172.21.15.188:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f712000fde0 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mon.1 v2:172.21.15.188:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f712002b400 con 0x55d3793cbdf0 2024-04-02T13:56:42.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 10 client.30169 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 10 client.30169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 20 client.30169 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d3793edcb0 0x55d3793f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 4 client.30169 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f712800f730 con 0x55d3793edcb0 2024-04-02T13:56:42.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.764+0000 7f7137fff640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d3793edcb0 0x55d3793f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.772 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:42 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1196700344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.765+0000 7f7137fff640 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d3793edcb0 0x55d3793f0130 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3c66d0be136a73 server_cookie=70a8029a89e78bf6 in_seq=0 out_seq=0 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.765+0000 7f7117fff640 10 client.30169 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.767+0000 7f7117fff640 1 -- 172.21.15.122:0/1193623719 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f712800f730 con 0x55d3793edcb0 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.767+0000 7f7117fff640 10 client.30169 handle_command_reply: tid=0 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.767+0000 7f7141a3f740 1 client.30169 shutdown 2024-04-02T13:56:42.773 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.767+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 msgr2=0x55d3793d5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.767+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7120011800 tx=0x7f7120011cc0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 shutdown_connections 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d3793edcb0 0x55d3793f0130 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d90b0 0x55d3793d6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793d5f50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 --2- 172.21.15.122:0/1193623719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cb4d0 0x55d3793d8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7117fff640 0 client.30169 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 >> 172.21.15.122:0/1193623719 conn(0x55d3793c5ed0 msgr2=0x55d3793c7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 shutdown_connections 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 1 -- 172.21.15.122:0/1193623719 wait complete. 2024-04-02T13:56:42.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.768+0000 7f7141a3f740 20 client.30169 trim_cache size 0 max 16384 2024-04-02T13:56:42.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.795+0000 7f7141a3f740 1 Processor -- start 2024-04-02T13:56:42.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 -- start start 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cbdf0 0x55d3793f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793d90b0 0x55d3793f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7138073490 con 0x55d3793cbdf0 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a6380 con 0x55d3793cb4d0 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.796+0000 7f7141a3f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7138073b60 con 0x55d3793d90b0 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54220/0 (socket says 172.21.15.122:54220) 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 -- 172.21.15.122:0/2568721791 learned_addr learned my addr 172.21.15.122:0/2568721791 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793d90b0 0x55d3793f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f713cbea640 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cbdf0 0x55d3793f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 -- 172.21.15.122:0/2568721791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793d90b0 msgr2=0x55d3793f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793d90b0 0x55d3793f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 -- 172.21.15.122:0/2568721791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cbdf0 msgr2=0x55d3793f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cbdf0 0x55d3793f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.797+0000 7f7137fff640 1 -- 172.21.15.122:0/2568721791 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3793d69b0 con 0x55d3793cb4d0 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.798+0000 7f7137fff640 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7128012a70 tx=0x7f7128021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6194b0344b27375e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.798+0000 7f712e7fc640 1 -- 172.21.15.122:0/2568721791 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7128029040 con 0x55d3793cb4d0 2024-04-02T13:56:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.798+0000 7f712e7fc640 1 -- 172.21.15.122:0/2568721791 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128003e50 con 0x55d3793cb4d0 2024-04-02T13:56:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.798+0000 7f712e7fc640 1 -- 172.21.15.122:0/2568721791 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71280053d0 con 0x55d3793cb4d0 2024-04-02T13:56:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 -- 172.21.15.122:0/2568721791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 msgr2=0x55d3793f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7128012a70 tx=0x7f7128021040 comp rx=0 tx=0).stop 2024-04-02T13:56:42.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 -- 172.21.15.122:0/2568721791 shutdown_connections 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793d90b0 0x55d3793f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793cbdf0 0x55d3793f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 --2- 172.21.15.122:0/2568721791 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793cb4d0 0x55d3793f00a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 -- 172.21.15.122:0/2568721791 >> 172.21.15.122:0/2568721791 conn(0x55d3793c5ed0 msgr2=0x55d3793d5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 -- 172.21.15.122:0/2568721791 shutdown_connections 2024-04-02T13:56:42.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.799+0000 7f7141a3f740 1 -- 172.21.15.122:0/2568721791 wait complete. 2024-04-02T13:56:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.800+0000 7f7141a3f740 1 -- start start 2024-04-02T13:56:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.800+0000 7f7141a3f740 1 Processor -- start 2024-04-02T13:56:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.800+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d25f0 0x55d3793d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a5cb0 con 0x55d3793d90b0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f71381a55e0 con 0x55d3793d25f0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7138071490 con 0x55d3793cbdf0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7141a3f740 10 client.0 _ll_register_callbacks cb 0x7f7120009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7137fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39078/0 (socket says 172.21.15.122:39078) 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 -- 172.21.15.122:0/491173641 learned_addr learned my addr 172.21.15.122:0/491173641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f7137fff640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d25f0 0x55d3793d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 -- 172.21.15.122:0/491173641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 msgr2=0x55d3793e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793e9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 -- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d25f0 msgr2=0x55d3793d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:42.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d25f0 0x55d3793d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.801+0000 7f713cbea640 1 -- 172.21.15.122:0/491173641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3793d2fb0 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f713cbea640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f71200059d0 tx=0x7f712002bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5aac954e86c16d server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7117fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712002f040 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f712000f9e0 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7128003e50 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f712004a050 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7141a3f740 10 client.21543 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.802+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d379453e00 con 0x55d3793d90b0 2024-04-02T13:56:42.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.803+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7128003e50 con 0x55d3793d90b0 2024-04-02T13:56:42.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f712000fc70 con 0x55d3793d90b0 2024-04-02T13:56:42.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 10 client.21543 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:42.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 10 client.21543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 20 client.21543 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d379454600 0x55d379456a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 4 client.21543 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d37933ccf0 con 0x55d379454600 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7137fff640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d379454600 0x55d379456a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7117fff640 10 client.21543 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.804+0000 7f7137fff640 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d379454600 0x55d379456a20 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318421b26d70cb30 server_cookie=f3865d58a69f1103 in_seq=0 out_seq=0 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7117fff640 1 -- 172.21.15.122:0/491173641 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d37933ccf0 con 0x55d379454600 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7117fff640 10 client.21543 handle_command_reply: tid=0 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 client.21543 shutdown 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 msgr2=0x55d3793ea270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f71200059d0 tx=0x7f712002bc70 comp rx=0 tx=0).stop 2024-04-02T13:56:42.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 shutdown_connections 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55d379454600 0x55d379456a20 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55d3793d25f0 0x55d3793d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d3793d90b0 0x55d3793ea270 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7117fff640 0 client.21543 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 --2- 172.21.15.122:0/491173641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d3793cbdf0 0x55d3793e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 >> 172.21.15.122:0/491173641 conn(0x55d3793c5ed0 msgr2=0x55d3793d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 shutdown_connections 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 1 -- 172.21.15.122:0/491173641 wait complete. 2024-04-02T13:56:42.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.805+0000 7f7141a3f740 20 client.21543 trim_cache size 0 max 16384 2024-04-02T13:56:42.812 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:56:42.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 msgr2=0x7f71240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 0x7f71240a61e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7130005c40 tx=0x7f71300de040 comp rx=0 tx=0).stop 2024-04-02T13:56:42.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 msgr2=0x7f713819cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f713002c160 tx=0x7f71300344c0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 shutdown_connections 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f71240a3d20 0x7f71240a61e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71381058a0 0x7f713819cc00 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7138101360 0x7f7138198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 --2- 172.21.15.122:0/2888609655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71381009c0 0x7f7138198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 >> 172.21.15.122:0/2888609655 conn(0x7f7138077a90 msgr2=0x7f71380fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.807+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 shutdown_connections 2024-04-02T13:56:42.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:42.808+0000 7f713ee4e640 1 -- 172.21.15.122:0/2888609655 wait complete. 2024-04-02T13:56:42.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:42 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1196700344' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:56:43.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 241844 26.16 MB/sec execute 363 sec latency 107.495 ms 2024-04-02T13:56:43.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:43 smithi187 ceph-mon[36064]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-04-02T13:56:43.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:43 smithi188 ceph-mon[36117]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-04-02T13:56:43.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:43 smithi122 ceph-mon[32114]: pgmap v679: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.5 MiB/s wr, 586 op/s 2024-04-02T13:56:43.830 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-02T13:56:43.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:56:43.830 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/2400897610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f481009c0 msgr2=0x7f0f48100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/2400897610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f481009c0 0x7f0f48100e20 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009900 tx=0x7f0f3c02bcc0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/2400897610 shutdown_connections 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/2400897610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f48101360 0x7f0f4810d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/2400897610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f481009c0 0x7f0f48100e20 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/2400897610 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 0x7f0f48105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/2400897610 >> 172.21.15.122:0/2400897610 conn(0x7f0f48077a90 msgr2=0x7f0f48077ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:43.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/2400897610 shutdown_connections 2024-04-02T13:56:43.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.973+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/2400897610 wait complete. 2024-04-02T13:56:43.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.974+0000 7f0f4e1ba640 1 Processor -- start 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.974+0000 7f0f4e1ba640 1 -- start start 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 0x7f0f4819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39460/0 (socket says 172.21.15.122:39460) 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39088/0 (socket says 172.21.15.122:39088) 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f477fe640 1 -- 172.21.15.122:0/1251683868 learned_addr learned my addr 172.21.15.122:0/1251683868 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48111e80 con 0x7f0f48101360 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48111d00 con 0x7f0f481058a0 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48112000 con 0x7f0f481009c0 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.975+0000 7f0f47fff640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 0x7f0f4819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f477fe640 1 -- 172.21.15.122:0/1251683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 msgr2=0x7f0f4819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f477fe640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 0x7f0f4819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f477fe640 1 -- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 msgr2=0x7f0f481988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f477fe640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f477fe640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819d350 con 0x7f0f481009c0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.976+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f477fe640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9f0 tx=0x7f0f3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f44ff9640 1 -- 172.21.15.122:0/1251683868 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800cd80 con 0x7f0f481009c0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f44ff9640 1 -- 172.21.15.122:0/1251683868 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38010910 con 0x7f0f481009c0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4819d640 con 0x7f0f481009c0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f44ff9640 1 -- 172.21.15.122:0/1251683868 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38018990 con 0x7f0f481009c0 2024-04-02T13:56:43.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.977+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4810e470 con 0x7f0f481009c0 2024-04-02T13:56:43.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.980+0000 7f0f44ff9640 1 -- 172.21.15.122:0/1251683868 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0f38010ac0 con 0x7f0f481009c0 2024-04-02T13:56:43.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.980+0000 7f0f50da4740 1 Processor -- start 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.980+0000 7f0f50da4740 1 -- start start 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.980+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b8710280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87107c0 0x5592b8712bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f50da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592b866b770 con 0x5592b8707be0 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f50da4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592b8613960 con 0x5592b87107c0 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f50da4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592b86678c0 con 0x5592b87072c0 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39470/0 (socket says 172.21.15.122:39470) 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 -- 172.21.15.122:0/3206156664 learned_addr learned my addr 172.21.15.122:0/3206156664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f477fe640 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b8710280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 -- 172.21.15.122:0/3206156664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87107c0 msgr2=0x5592b8712bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:43.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87107c0 0x5592b8712bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87107c0 0x5592b8712bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 -- 172.21.15.122:0/3206156664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 msgr2=0x5592b8710280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:43.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b8710280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.981+0000 7f0f47fff640 1 -- 172.21.15.122:0/3206156664 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b84477e0 con 0x5592b87072c0 2024-04-02T13:56:43.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f44ff9640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 0x7f0f300a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009fd0 tx=0x7f0f400118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ed36d9b9a34e45e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:43.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f44ff9640 1 -- 172.21.15.122:0/1251683868 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f0f380d0db0 con 0x7f0f481009c0 2024-04-02T13:56:43.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f2dffb640 1 -- 172.21.15.122:0/3206156664 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004d20 con 0x5592b87072c0 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f2dffb640 1 -- 172.21.15.122:0/3206156664 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40004050 con 0x5592b87072c0 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f2dffb640 1 -- 172.21.15.122:0/3206156664 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40005260 con 0x5592b87072c0 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.982+0000 7f0f477fe640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 0x7f0f300a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 -- 172.21.15.122:0/3206156664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 msgr2=0x5592b87076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0f40009fd0 tx=0x7f0f400118d0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 -- 172.21.15.122:0/3206156664 shutdown_connections 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87107c0 0x5592b8712bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b8710280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3206156664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87072c0 0x5592b87076a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 -- 172.21.15.122:0/3206156664 >> 172.21.15.122:0/3206156664 conn(0x5592b87019f0 msgr2=0x5592b8701dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f477fe640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 0x7f0f300a63b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5592b8706d90 tx=0x7f0f38000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.983+0000 7f0f50da4740 1 -- 172.21.15.122:0/3206156664 shutdown_connections 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 -- 172.21.15.122:0/3206156664 wait complete. 2024-04-02T13:56:43.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 -- start start 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 Processor -- start 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b8707be0 0x5592b870de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87107c0 0x5592b870e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5592b87135b0 con 0x5592b87107c0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5592b8626d20 con 0x5592b87072c0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5592b8713730 con 0x5592b8707be0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.984+0000 7f0f50da4740 10 client.0 _ll_register_callbacks cb 0x5592b85f3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b8707be0 0x5592b870de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54238/0 (socket says 172.21.15.122:54238) 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 -- 172.21.15.122:0/186941901 learned_addr learned my addr 172.21.15.122:0/186941901 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:43.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87107c0 0x5592b870e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 -- 172.21.15.122:0/186941901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b8707be0 msgr2=0x5592b870de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b8707be0 0x5592b870de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 -- 172.21.15.122:0/186941901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87107c0 msgr2=0x5592b870e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87107c0 0x5592b870e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.985+0000 7f0f47fff640 1 -- 172.21.15.122:0/186941901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b84a7f70 con 0x5592b87072c0 2024-04-02T13:56:43.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f47fff640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x5592b8701390 tx=0x7f0f40009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26361f20bdc7dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4000f4e0 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f40004050 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f0f4000f680 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0f40036050 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f50da4740 10 client.30419 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.986+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5592b833f170 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.987+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40004050 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.987+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f0f40025040 con 0x5592b87072c0 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 10 client.30419 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 10 client.30419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 20 client.30419 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b8720b40 0x5592b8722f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 4 client.30419 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:43.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b8678cb0 con 0x5592b8720b40 2024-04-02T13:56:43.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.988+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b8720b40 0x5592b8722f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.989+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b8720b40 0x5592b8722f30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd752368f50aba2 server_cookie=9373caaf6032c806 in_seq=0 out_seq=0 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.989+0000 7f0f26ffd640 10 client.30419 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.990+0000 7f0f26ffd640 1 -- 172.21.15.122:0/186941901 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592b8678cb0 con 0x5592b8720b40 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.990+0000 7f0f26ffd640 10 client.30419 handle_command_reply: tid=0 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.990+0000 7f0f50da4740 1 client.30419 shutdown 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 msgr2=0x5592b870d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x5592b8701390 tx=0x7f0f40009be0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 shutdown_connections 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b8720b40 0x5592b8722f30 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87107c0 0x5592b870e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b8707be0 0x5592b870de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 --2- 172.21.15.122:0/186941901 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b870d8f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:43.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 >> 172.21.15.122:0/186941901 conn(0x5592b87019f0 msgr2=0x5592b8703740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:43.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.991+0000 7f0f26ffd640 0 client.30419 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:43.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.992+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 shutdown_connections 2024-04-02T13:56:43.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.992+0000 7f0f50da4740 1 -- 172.21.15.122:0/186941901 wait complete. 2024-04-02T13:56:43.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:43.992+0000 7f0f50da4740 20 client.30419 trim_cache size 0 max 16384 2024-04-02T13:56:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.023+0000 7f0f50da4740 1 Processor -- start 2024-04-02T13:56:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.023+0000 7f0f50da4740 1 -- start start 2024-04-02T13:56:44.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.023+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.023+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b871c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f50da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ace40 con 0x5592b8707be0 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f50da4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ae980 con 0x5592b87072c0 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f50da4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ad510 con 0x5592b87107c0 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b871c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54250/0 (socket says 172.21.15.122:54250) 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b871c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39112/0 (socket says 172.21.15.122:39112) 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 -- 172.21.15.122:0/3251264087 learned_addr learned my addr 172.21.15.122:0/3251264087 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 -- 172.21.15.122:0/3251264087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 msgr2=0x5592b870dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 -- 172.21.15.122:0/3251264087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 msgr2=0x5592b871c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b871c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 -- 172.21.15.122:0/3251264087 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b8712f20 con 0x5592b87072c0 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.024+0000 7f0f47fff640 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0f40011910 tx=0x7f0f400096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396b054e5ca77263 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f2d7fa640 1 -- 172.21.15.122:0/3251264087 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40003ee0 con 0x5592b87072c0 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f2d7fa640 1 -- 172.21.15.122:0/3251264087 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4002d070 con 0x5592b87072c0 2024-04-02T13:56:44.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f2d7fa640 1 -- 172.21.15.122:0/3251264087 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f400194b0 con 0x5592b87072c0 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 -- 172.21.15.122:0/3251264087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 msgr2=0x5592b8710020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0f40011910 tx=0x7f0f400096c0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 -- 172.21.15.122:0/3251264087 shutdown_connections 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b8707be0 0x5592b871c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 --2- 172.21.15.122:0/3251264087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b87072c0 0x5592b8710020 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.025+0000 7f0f50da4740 1 -- 172.21.15.122:0/3251264087 >> 172.21.15.122:0/3251264087 conn(0x5592b87019f0 msgr2=0x5592b870d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- 172.21.15.122:0/3251264087 shutdown_connections 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- 172.21.15.122:0/3251264087 wait complete. 2024-04-02T13:56:44.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- start start 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 Processor -- start 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 0x5592b870aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ae2b0 con 0x5592b87072c0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481adbe0 con 0x5592b8707be0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aae40 con 0x5592b87107c0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.026+0000 7f0f50da4740 10 client.0 _ll_register_callbacks cb 0x7f0f40009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 0x5592b870aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39122/0 (socket says 172.21.15.122:39122) 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f47fff640 1 -- 172.21.15.122:0/1475572529 learned_addr learned my addr 172.21.15.122:0/1475572529 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f47fff640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 -- 172.21.15.122:0/1475572529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 msgr2=0x5592b870af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 -- 172.21.15.122:0/1475572529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 msgr2=0x5592b870a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 -- 172.21.15.122:0/1475572529 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b878e720 con 0x5592b8707be0 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f47fff640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f46ffd640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 0x5592b870aa40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c004c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b131696f813f4bad server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c004450 con 0x5592b8707be0 2024-04-02T13:56:44.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4002d070 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.027+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f0f3c02c910 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c02cb10 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 10 client.30425 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5592b878f950 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4002d070 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f0f3c046040 con 0x5592b8707be0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 10 client.30425 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 10 client.30425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 20 client.30425 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b87900d0 0x5592b87924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 4 client.30425 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5592b8673000 con 0x5592b87900d0 2024-04-02T13:56:44.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.028+0000 7f0f47fff640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b87900d0 0x5592b87924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.029+0000 7f0f47fff640 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b87900d0 0x5592b87924f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=177735e4c90a1ceb server_cookie=3eb6846986854845 in_seq=0 out_seq=0 2024-04-02T13:56:44.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.029+0000 7f0f26ffd640 10 client.30425 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f26ffd640 1 -- 172.21.15.122:0/1475572529 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5592b8673000 con 0x5592b87900d0 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f26ffd640 10 client.30425 handle_command_reply: tid=0 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 client.30425 shutdown 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 msgr2=0x5592b870aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 0x5592b870aa40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c000c00 tx=0x7f0f3c004c20 comp rx=0 tx=0).stop 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 shutdown_connections 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5592b87900d0 0x5592b87924f0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f26ffd640 0 client.30425 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5592b87107c0 0x5592b870af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5592b8707be0 0x5592b870aa40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 --2- 172.21.15.122:0/1475572529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5592b87072c0 0x5592b870a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 >> 172.21.15.122:0/1475572529 conn(0x5592b87019f0 msgr2=0x5592b870e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.031+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 shutdown_connections 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f50da4740 1 -- 172.21.15.122:0/1475572529 wait complete. 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f50da4740 20 client.30425 trim_cache size 0 max 16384 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stdout: "return_code": 0, 2024-04-02T13:56:44.040 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_tag": "5c1e2e79-5fdb-4cb0-8f4e-b6e0352bb192", 2024-04-02T13:56:44.041 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": "asynchronous" 2024-04-02T13:56:44.041 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 msgr2=0x7f0f300a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 0x7f0f300a63b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x5592b8706d90 tx=0x7f0f38000bc0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 msgr2=0x7f0f48198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.032+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800e9f0 tx=0x7f0f3800eeb0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 shutdown_connections 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0f300a3ef0 0x7f0f300a63b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f481058a0 0x7f0f4819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f48101360 0x7f0f481988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 --2- 172.21.15.122:0/1251683868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f481009c0 0x7f0f48198360 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 >> 172.21.15.122:0/1251683868 conn(0x7f0f48077a90 msgr2=0x7f0f480fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 shutdown_connections 2024-04-02T13:56:44.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.033+0000 7f0f4e1ba640 1 -- 172.21.15.122:0/1251683868 wait complete. 2024-04-02T13:56:44.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:56:44.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:56:44.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.227+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/3469438984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e281045c0 msgr2=0x7f6e28104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.227+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/3469438984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e281045c0 0x7f6e28104a40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009ca0 tx=0x7f6e2002c120 comp rx=0 tx=0).stop 2024-04-02T13:56:44.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.229+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/3469438984 shutdown_connections 2024-04-02T13:56:44.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.229+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/3469438984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e281045c0 0x7f6e28104a40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.229+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/3469438984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e28069240 0x7f6e280696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.229+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/3469438984 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e28106a50 0x7f6e28068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.229+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/3469438984 >> 172.21.15.122:0/3469438984 conn(0x7f6e28077b00 msgr2=0x7f6e28077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.232+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/3469438984 shutdown_connections 2024-04-02T13:56:44.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.233+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/3469438984 wait complete. 2024-04-02T13:56:44.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.234+0000 7f6e2ef1c640 1 Processor -- start 2024-04-02T13:56:44.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.237+0000 7f6e2ef1c640 1 -- start start 2024-04-02T13:56:44.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e28069240 0x7f6e2810b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e28106a50 0x7f6e28072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28107960 con 0x7f6e28069240 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e281077e0 con 0x7f6e281045c0 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e2ef1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e28107ae0 con 0x7f6e28106a50 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:54264/0 (socket says 172.21.15.122:54264) 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.238+0000 7f6e27fff640 1 -- 172.21.15.122:0/15566888 learned_addr learned my addr 172.21.15.122:0/15566888 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.239+0000 7f6e27fff640 1 -- 172.21.15.122:0/15566888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e28106a50 msgr2=0x7f6e28072350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.239+0000 7f6e27fff640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e28106a50 0x7f6e28072350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.239+0000 7f6e27fff640 1 -- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e28069240 msgr2=0x7f6e2810b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.239+0000 7f6e27fff640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e28069240 0x7f6e2810b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.239+0000 7f6e27fff640 1 -- 172.21.15.122:0/15566888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e28072c10 con 0x7f6e281045c0 2024-04-02T13:56:44.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.240+0000 7f6e27fff640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b4a0 tx=0x7f6e1800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.240+0000 7f6e25ffb640 1 -- 172.21.15.122:0/15566888 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e18011830 con 0x7f6e281045c0 2024-04-02T13:56:44.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.240+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e28072f00 con 0x7f6e281045c0 2024-04-02T13:56:44.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.240+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e28113690 con 0x7f6e281045c0 2024-04-02T13:56:44.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.241+0000 7f6e25ffb640 1 -- 172.21.15.122:0/15566888 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e18016070 con 0x7f6e281045c0 2024-04-02T13:56:44.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.242+0000 7f6e25ffb640 1 -- 172.21.15.122:0/15566888 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1800f2e0 con 0x7f6e281045c0 2024-04-02T13:56:44.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.244+0000 7f6e25ffb640 1 -- 172.21.15.122:0/15566888 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f6e1800f480 con 0x7f6e281045c0 2024-04-02T13:56:44.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.245+0000 7f6e25ffb640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 0x7f6e140a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.245+0000 7f6e25ffb640 1 -- 172.21.15.122:0/15566888 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f6e180d1220 con 0x7f6e281045c0 2024-04-02T13:56:44.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.245+0000 7f6e31be8740 1 Processor -- start 2024-04-02T13:56:44.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 0x7f6e140a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 -- start start 2024-04-02T13:56:44.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513321f0 0x55675133a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556751295830 con 0x5567513318d0 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55675123d820 con 0x55675133add0 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e31be8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5567512917e0 con 0x5567513321f0 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.248+0000 7f6e2ccb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.249+0000 7f6e2ccb8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39150/0 (socket says 172.21.15.122:39150) 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.249+0000 7f6e2ccb8640 1 -- 172.21.15.122:0/1531887575 learned_addr learned my addr 172.21.15.122:0/1531887575 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.249+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.249+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 0x7f6e140a6320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6e100098d0 tx=0x7f6e10009cd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.249+0000 7f6e2ccb8640 1 -- 172.21.15.122:0/1531887575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513321f0 msgr2=0x55675133a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.250+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513321f0 0x55675133a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.250+0000 7f6e2ccb8640 1 -- 172.21.15.122:0/1531887575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 msgr2=0x55675133d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.250+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.250+0000 7f6e2ccb8640 1 -- 172.21.15.122:0/1531887575 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567510737d0 con 0x5567513318d0 2024-04-02T13:56:44.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.251+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000fb70 tx=0x7f6e10017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5591e4266413bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.251+0000 7f6e1effd640 1 -- 172.21.15.122:0/1531887575 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1001e450 con 0x5567513318d0 2024-04-02T13:56:44.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.251+0000 7f6e1effd640 1 -- 172.21.15.122:0/1531887575 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e10023070 con 0x5567513318d0 2024-04-02T13:56:44.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.252+0000 7f6e1effd640 1 -- 172.21.15.122:0/1531887575 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10014990 con 0x5567513318d0 2024-04-02T13:56:44.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.252+0000 7f6e31be8740 1 -- 172.21.15.122:0/1531887575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 msgr2=0x556751331cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.252+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000fb70 tx=0x7f6e10017000 comp rx=0 tx=0).stop 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 -- 172.21.15.122:0/1531887575 shutdown_connections 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513321f0 0x55675133a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1531887575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751331cb0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 -- 172.21.15.122:0/1531887575 >> 172.21.15.122:0/1531887575 conn(0x55675132c000 msgr2=0x55675132c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 -- 172.21.15.122:0/1531887575 shutdown_connections 2024-04-02T13:56:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 -- 172.21.15.122:0/1531887575 wait complete. 2024-04-02T13:56:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.253+0000 7f6e31be8740 1 -- start start 2024-04-02T13:56:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 Processor -- start 2024-04-02T13:56:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x55675133a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x556751337f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55675133dbb0 con 0x5567513321f0 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x556751250d30 con 0x55675133add0 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55675133dd30 con 0x5567513318d0 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e31be8740 10 client.0 _ll_register_callbacks cb 0x556751250720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e27fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:39154/0 (socket says 172.21.15.122:39154) 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e27fff640 1 -- 172.21.15.122:0/2449873390 learned_addr learned my addr 172.21.15.122:0/2449873390 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.254+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x556751337f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 -- 172.21.15.122:0/2449873390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 msgr2=0x55675133a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x55675133a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 -- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 msgr2=0x556751337f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x556751337f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 -- 172.21.15.122:0/2449873390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556750fa93f0 con 0x5567513321f0 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e27fff640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e1809cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95105712737c4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.255+0000 7f6e07fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.256+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e180ce9a0 con 0x5567513321f0 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.256+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f6e180a5660 con 0x5567513321f0 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.256+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e10023070 con 0x5567513321f0 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.256+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6e18010e80 con 0x5567513321f0 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.257+0000 7f6e31be8740 10 client.21567 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:44.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.257+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x556750f690f0 con 0x5567513321f0 2024-04-02T13:56:44.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.258+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e180a5860 con 0x5567513321f0 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.258+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f6e1809bd40 con 0x5567513321f0 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 10 client.21567 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 10 client.21567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 20 client.21567 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55675134acd0 0x55675134d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 4 client.21567 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.259+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6e10023070 con 0x55675134acd0 2024-04-02T13:56:44.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.260+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55675134acd0 0x55675134d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.261+0000 7f6e07fff640 10 client.21567 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.261+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55675134acd0 0x55675134d0f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9dbccb7934dcef7 server_cookie=418e03a21eea3cab in_seq=0 out_seq=0 2024-04-02T13:56:44.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.263+0000 7f6e07fff640 1 -- 172.21.15.122:0/2449873390 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6e10023070 con 0x55675134acd0 2024-04-02T13:56:44.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.263+0000 7f6e07fff640 10 client.21567 handle_command_reply: tid=0 2024-04-02T13:56:44.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.263+0000 7f6e31be8740 1 client.21567 shutdown 2024-04-02T13:56:44.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.263+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 msgr2=0x556751337a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.263+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f6e18000c00 tx=0x7f6e1809cf40 comp rx=0 tx=0).stop 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 shutdown_connections 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55675134acd0 0x55675134d0f0 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x556751337f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x556751337a50 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 --2- 172.21.15.122:0/2449873390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x55675133a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 >> 172.21.15.122:0/2449873390 conn(0x55675132c000 msgr2=0x55675132dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e07fff640 0 client.21567 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 shutdown_connections 2024-04-02T13:56:44.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 1 -- 172.21.15.122:0/2449873390 wait complete. 2024-04-02T13:56:44.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.264+0000 7f6e31be8740 20 client.21567 trim_cache size 0 max 16384 2024-04-02T13:56:44.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.304+0000 7f6e31be8740 1 Processor -- start 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.306+0000 7f6e31be8740 1 -- start start 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751346b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567513321f0 0x55675132b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08010040 con 0x5567513318d0 2024-04-02T13:56:44.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08011b80 con 0x5567513321f0 2024-04-02T13:56:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e31be8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08010710 con 0x55675133add0 2024-04-02T13:56:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.307+0000 7f6e2d4b9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.308+0000 7f6e2d4b9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39542/0 (socket says 172.21.15.122:39542) 2024-04-02T13:56:44.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.308+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/1405219585 learned_addr learned my addr 172.21.15.122:0/1405219585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.308+0000 7f6e27fff640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567513321f0 0x55675132b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.308+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751346b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.308+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/1405219585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567513321f0 msgr2=0x55675132b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.309+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567513321f0 0x55675132b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.309+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/1405219585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 msgr2=0x556751346b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.309+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751346b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.309+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/1405219585 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675132be30 con 0x55675133add0 2024-04-02T13:56:44.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.309+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751346b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:56:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.310+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009ca0 tx=0x7f6e20009850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3bf7295ecfca6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.311+0000 7f6e1e7fc640 1 -- 172.21.15.122:0/1405219585 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20043030 con 0x55675133add0 2024-04-02T13:56:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.311+0000 7f6e1e7fc640 1 -- 172.21.15.122:0/1405219585 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20032d00 con 0x55675133add0 2024-04-02T13:56:44.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.311+0000 7f6e1e7fc640 1 -- 172.21.15.122:0/1405219585 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20039040 con 0x55675133add0 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 -- 172.21.15.122:0/1405219585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 msgr2=0x55675132daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009ca0 tx=0x7f6e20009850 comp rx=0 tx=0).stop 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 -- 172.21.15.122:0/1405219585 shutdown_connections 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55675133add0 0x55675132daf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5567513321f0 0x55675132b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 --2- 172.21.15.122:0/1405219585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513318d0 0x556751346b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.312+0000 7f6e31be8740 1 -- 172.21.15.122:0/1405219585 >> 172.21.15.122:0/1405219585 conn(0x55675132c000 msgr2=0x55675132d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.313+0000 7f6e31be8740 1 -- 172.21.15.122:0/1405219585 shutdown_connections 2024-04-02T13:56:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.313+0000 7f6e31be8740 1 -- 172.21.15.122:0/1405219585 wait complete. 2024-04-02T13:56:44.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.314+0000 7f6e31be8740 1 -- start start 2024-04-02T13:56:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.314+0000 7f6e31be8740 1 Processor -- start 2024-04-02T13:56:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x556751343140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x55675133e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e2ccb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x556751343140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e2ccb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x556751343140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:39552/0 (socket says 172.21.15.122:39552) 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e2ccb8640 1 -- 172.21.15.122:0/3795645645 learned_addr learned my addr 172.21.15.122:0/3795645645 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e080114b0 con 0x5567513321f0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08010de0 con 0x55675133add0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.315+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0800e040 con 0x5567513318d0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e31be8740 10 client.0 _ll_register_callbacks cb 0x7f6e1809d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:56:44.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e27fff640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x55675133e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/3795645645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 msgr2=0x556751343140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x556751343140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/3795645645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 msgr2=0x55675133e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x55675133e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.316+0000 7f6e2d4b9640 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675133f060 con 0x55675133add0 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.317+0000 7f6e2d4b9640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133eaf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e2002c730 tx=0x7f6e20032830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64202d9368a11470 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.317+0000 7f6e07fff640 10 client.0 ms_handle_connect on v2:172.21.15.187:3300/0 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.317+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20043070 con 0x55675133add0 2024-04-02T13:56:44.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.318+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mon.2 v2:172.21.15.187:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9525+0+0 (secure 0 0 0) 0x7f6e20004200 con 0x55675133add0 2024-04-02T13:56:44.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.318+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e20032d00 con 0x55675133add0 2024-04-02T13:56:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.319+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f6e20039040 con 0x55675133add0 2024-04-02T13:56:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.319+0000 7f6e31be8740 10 client.30437 fetch_fsmap learned FSMap version 76 2024-04-02T13:56:44.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.319+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5567513b0eb0 con 0x55675133add0 2024-04-02T13:56:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mon.2 v2:172.21.15.187:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2003f450 con 0x55675133add0 2024-04-02T13:56:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mon.2 v2:172.21.15.187:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f6e2002dce0 con 0x55675133add0 2024-04-02T13:56:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e31be8740 10 client.30437 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:56:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e31be8740 10 client.30437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:56:44.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e31be8740 20 client.30437 populate_metadata read hostname 'smithi122' 2024-04-02T13:56:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567513b1430 0x5567513b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:56:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.320+0000 7f6e31be8740 4 client.30437 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:56:44.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.321+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6e20032d00 con 0x5567513b1430 2024-04-02T13:56:44.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.323+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567513b1430 0x5567513b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:56:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.327+0000 7f6e2ccb8640 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567513b1430 0x5567513b3850 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d650d78fd1b06bf3 server_cookie=3408fd8f245a3102 in_seq=0 out_seq=0 2024-04-02T13:56:44.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.327+0000 7f6e07fff640 10 client.30437 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.328+0000 7f6e07fff640 1 -- 172.21.15.122:0/3795645645 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f6e20032d00 con 0x5567513b1430 2024-04-02T13:56:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.328+0000 7f6e07fff640 10 client.30437 handle_command_reply: tid=0 2024-04-02T13:56:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.328+0000 7f6e31be8740 1 client.30437 shutdown 2024-04-02T13:56:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.329+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 msgr2=0x55675133eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.329+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133eaf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6e2002c730 tx=0x7f6e20032830 comp rx=0 tx=0).stop 2024-04-02T13:56:44.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 shutdown_connections 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5567513b1430 0x5567513b3850 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55675133add0 0x55675133eaf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5567513321f0 0x55675133e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e07fff640 0 client.30437 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 --2- 172.21.15.122:0/3795645645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567513318d0 0x556751343140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.330+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 >> 172.21.15.122:0/3795645645 conn(0x55675132c000 msgr2=0x55675132b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.332+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 shutdown_connections 2024-04-02T13:56:44.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.332+0000 7f6e31be8740 1 -- 172.21.15.122:0/3795645645 wait complete. 2024-04-02T13:56:44.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.333+0000 7f6e31be8740 20 client.30437 trim_cache size 0 max 16384 2024-04-02T13:56:44.337 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:56:44.337 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": { 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: "5c1e2e79-5fdb-4cb0-8f4e-b6e0352bb192": { 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: "path": "/", 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: "tag": "5c1e2e79-5fdb-4cb0-8f4e-b6e0352bb192", 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: "options": "recursive,force" 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:56:44.338 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.334+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 msgr2=0x7f6e140a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.334+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 0x7f6e140a6320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6e100098d0 tx=0x7f6e10009cd0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.334+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 msgr2=0x7f6e2810ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e1800b4a0 tx=0x7f6e1800b960 comp rx=0 tx=0).stop 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 shutdown_connections 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f6e140a3e60 0x7f6e140a6320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e28106a50 0x7f6e28072350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e281045c0 0x7f6e2810ba50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 --2- 172.21.15.122:0/15566888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e28069240 0x7f6e2810b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:56:44.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.335+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 >> 172.21.15.122:0/15566888 conn(0x7f6e28077b00 msgr2=0x7f6e28101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:56:44.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.336+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 shutdown_connections 2024-04-02T13:56:44.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:56:44.336+0000 7f6e2ef1c640 1 -- 172.21.15.122:0/15566888 wait complete. 2024-04-02T13:56:44.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 242410 26.18 MB/sec execute 364 sec latency 107.978 ms 2024-04-02T13:56:44.357 INFO:tasks.cephfs.filesystem:scrub status for tag:5c1e2e79-5fdb-4cb0-8f4e-b6e0352bb192 - {'path': '/', 'tag': '5c1e2e79-5fdb-4cb0-8f4e-b6e0352bb192', 'options': 'recursive,force'} 2024-04-02T13:56:44.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:44 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:44.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:44 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:44.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:44 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 242896 26.17 MB/sec execute 365 sec latency 113.741 ms 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub queued for path: / 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: active paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 506 op/s 2024-04-02T13:56:45.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.769 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:45 smithi188 ceph-mon[36117]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub queued for path: / 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: active paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 506 op/s 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.822 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:45 smithi122 ceph-mon[32114]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub queued for path: / 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: active paths [/] 2024-04-02T13:56:45.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: pgmap v680: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 506 op/s 2024-04-02T13:56:45.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:45.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:45 smithi187 ceph-mon[36064]: scrub summary: idle+waiting paths [/] 2024-04-02T13:56:46.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 242930 26.10 MB/sec execute 366 sec latency 1099.823 ms 2024-04-02T13:56:47.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243050 26.05 MB/sec execute 367 sec latency 2099.865 ms 2024-04-02T13:56:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:47 smithi188 ceph-mon[36117]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 452 op/s 2024-04-02T13:56:47.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:47 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:47.787 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:47 smithi187 ceph-mon[36064]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 452 op/s 2024-04-02T13:56:47.787 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:47 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:47 smithi122 ceph-mon[32114]: pgmap v681: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 452 op/s 2024-04-02T13:56:47.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:47 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:56:48.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243179 26.00 MB/sec execute 368 sec latency 2198.473 ms 2024-04-02T13:56:49.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243340 25.96 MB/sec execute 369 sec latency 257.670 ms 2024-04-02T13:56:49.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:49 smithi187 ceph-mon[36064]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:56:49.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:49 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:49.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:49 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:49.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:49 smithi188 ceph-mon[36117]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:56:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:49 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:49 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:49 smithi122 ceph-mon[32114]: pgmap v682: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 680 op/s 2024-04-02T13:56:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:49 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:49.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:49 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:50.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243529 25.91 MB/sec execute 370 sec latency 636.392 ms 2024-04-02T13:56:50.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:50.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:50.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:50.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:50.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:50.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:56:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:56:50.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:56:51.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 243771 25.89 MB/sec execute 371 sec latency 241.986 ms 2024-04-02T13:56:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:51 smithi122 ceph-mon[32114]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 581 op/s 2024-04-02T13:56:52.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:51 smithi187 ceph-mon[36064]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 581 op/s 2024-04-02T13:56:52.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:51 smithi188 ceph-mon[36117]: pgmap v683: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 581 op/s 2024-04-02T13:56:52.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 244119 25.87 MB/sec execute 372 sec latency 326.690 ms 2024-04-02T13:56:53.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 244507 25.82 MB/sec execute 373 sec latency 296.322 ms 2024-04-02T13:56:54.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:53 smithi122 ceph-mon[32114]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-04-02T13:56:54.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:53 smithi187 ceph-mon[36064]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-04-02T13:56:54.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:53 smithi188 ceph-mon[36117]: pgmap v684: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 16 MiB/s wr, 691 op/s 2024-04-02T13:56:54.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 245131 25.85 MB/sec execute 374 sec latency 128.170 ms 2024-04-02T13:56:55.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 245600 25.86 MB/sec execute 375 sec latency 319.356 ms 2024-04-02T13:56:56.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:55 smithi122 ceph-mon[32114]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 615 op/s 2024-04-02T13:56:56.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:55 smithi187 ceph-mon[36064]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 615 op/s 2024-04-02T13:56:56.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:55 smithi188 ceph-mon[36117]: pgmap v685: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 615 op/s 2024-04-02T13:56:56.341 INFO:tasks.workunit.client.0.smithi122.stdout: 10 246252 25.90 MB/sec execute 376 sec latency 66.425 ms 2024-04-02T13:56:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:56 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:57.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:56 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:57.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:56 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:57.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:56 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:57.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:56 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:57.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:56 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:57.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 246673 25.88 MB/sec execute 377 sec latency 152.417 ms 2024-04-02T13:56:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:57 smithi122 ceph-mon[32114]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-04-02T13:56:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:57 smithi122 ceph-mon[32114]: scrub summary: idle 2024-04-02T13:56:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:57 smithi187 ceph-mon[36064]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-04-02T13:56:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:58.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:57 smithi187 ceph-mon[36064]: scrub summary: idle 2024-04-02T13:56:58.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:57 smithi188 ceph-mon[36117]: pgmap v686: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 13 MiB/s wr, 567 op/s 2024-04-02T13:56:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:58.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:57 smithi188 ceph-mon[36117]: scrub summary: idle 2024-04-02T13:56:58.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 247146 25.88 MB/sec execute 378 sec latency 120.440 ms 2024-04-02T13:56:59.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 247649 25.88 MB/sec execute 379 sec latency 108.515 ms 2024-04-02T13:57:00.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:56:59 smithi122 ceph-mon[32114]: pgmap v687: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 731 op/s 2024-04-02T13:57:00.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:56:59 smithi187 ceph-mon[36064]: pgmap v687: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 731 op/s 2024-04-02T13:57:00.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:56:59 smithi188 ceph-mon[36117]: pgmap v687: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 731 op/s 2024-04-02T13:57:00.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248117 25.87 MB/sec execute 380 sec latency 323.505 ms 2024-04-02T13:57:01.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248117 25.80 MB/sec execute 381 sec latency 1323.592 ms 2024-04-02T13:57:02.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:01 smithi122 ceph-mon[32114]: pgmap v688: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-04-02T13:57:02.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:01 smithi187 ceph-mon[36064]: pgmap v688: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-04-02T13:57:02.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:01 smithi188 ceph-mon[36117]: pgmap v688: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.6 MiB/s wr, 524 op/s 2024-04-02T13:57:02.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248385 25.76 MB/sec execute 382 sec latency 1518.296 ms 2024-04-02T13:57:03.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248482 25.70 MB/sec execute 383 sec latency 388.090 ms 2024-04-02T13:57:04.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:03 smithi122 ceph-mon[32114]: pgmap v689: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-04-02T13:57:04.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:03 smithi187 ceph-mon[36064]: pgmap v689: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-04-02T13:57:04.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:03 smithi188 ceph-mon[36117]: pgmap v689: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-04-02T13:57:04.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248495 25.64 MB/sec execute 384 sec latency 422.305 ms 2024-04-02T13:57:05.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248500 25.58 MB/sec execute 385 sec latency 528.968 ms 2024-04-02T13:57:06.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:05 smithi122 ceph-mon[32114]: pgmap v690: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 460 op/s 2024-04-02T13:57:06.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:05 smithi187 ceph-mon[36064]: pgmap v690: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 460 op/s 2024-04-02T13:57:06.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:05 smithi188 ceph-mon[36117]: pgmap v690: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.2 MiB/s wr, 460 op/s 2024-04-02T13:57:06.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248664 25.55 MB/sec execute 386 sec latency 533.328 ms 2024-04-02T13:57:07.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 248941 25.53 MB/sec execute 387 sec latency 588.454 ms 2024-04-02T13:57:08.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:07 smithi122 ceph-mon[32114]: pgmap v691: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 385 op/s 2024-04-02T13:57:08.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:07 smithi122 ceph-mon[32114]: osdmap e126: 12 total, 12 up, 12 in 2024-04-02T13:57:08.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:07 smithi187 ceph-mon[36064]: pgmap v691: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 385 op/s 2024-04-02T13:57:08.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:07 smithi187 ceph-mon[36064]: osdmap e126: 12 total, 12 up, 12 in 2024-04-02T13:57:08.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:07 smithi188 ceph-mon[36117]: pgmap v691: 129 pgs: 129 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.9 MiB/s wr, 385 op/s 2024-04-02T13:57:08.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:07 smithi188 ceph-mon[36117]: osdmap e126: 12 total, 12 up, 12 in 2024-04-02T13:57:08.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 249358 25.52 MB/sec execute 388 sec latency 669.663 ms 2024-04-02T13:57:09.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 249804 25.52 MB/sec execute 389 sec latency 172.480 ms 2024-04-02T13:57:10.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:09 smithi187 ceph-mon[36064]: pgmap v693: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-04-02T13:57:10.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:09 smithi188 ceph-mon[36117]: pgmap v693: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-04-02T13:57:10.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:09 smithi122 ceph-mon[32114]: pgmap v693: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-04-02T13:57:10.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 250255 25.52 MB/sec execute 390 sec latency 294.057 ms 2024-04-02T13:57:11.342 INFO:tasks.workunit.client.0.smithi122.stdout: 10 250719 25.51 MB/sec execute 391 sec latency 86.454 ms 2024-04-02T13:57:11.457 INFO:tasks.workunit.client.0.smithi122.stdout:[249854] write failed on handle 14006 (Resource temporarily unavailable) 2024-04-02T13:57:11.458 INFO:tasks.workunit.client.0.smithi122.stdout:Child failed with status 1 2024-04-02T13:57:11.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T13:57:11.475 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-02T13:57:11.475 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-02T13:57:11.795 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-02T13:57:12.040 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9baed45056694728bee1f572a597f1ba Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_191d963ac9de7322da8aff20b567b3fa458e2ef6/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-02T13:57:12.042 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-02T13:57:12.053 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-02T13:57:12.053 INFO:teuthology.task.exec:Executing custom commands... 2024-04-02T13:57:12.054 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:57:12.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-02T13:57:12.070 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:11 smithi122 ceph-mon[32114]: pgmap v694: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-02T13:57:12.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-02T13:57:12.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:11 smithi187 ceph-mon[36064]: pgmap v694: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-02T13:57:12.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:11 smithi188 ceph-mon[36117]: pgmap v694: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 2.6 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 450 op/s 2024-04-02T13:57:12.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-02T13:57:12.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 -- 172.21.15.122:0/1286660237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad941059a0 msgr2=0x7fad94105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 --2- 172.21.15.122:0/1286660237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad941059a0 0x7fad94105d80 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009ca0 tx=0x7fad8c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 -- 172.21.15.122:0/1286660237 shutdown_connections 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 --2- 172.21.15.122:0/1286660237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad9410d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 --2- 172.21.15.122:0/1286660237 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad94100c50 0x7fad941010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 --2- 172.21.15.122:0/1286660237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad941059a0 0x7fad94105d80 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 -- 172.21.15.122:0/1286660237 >> 172.21.15.122:0/1286660237 conn(0x7fad94077860 msgr2=0x7fad94077c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 -- 172.21.15.122:0/1286660237 shutdown_connections 2024-04-02T13:57:12.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.446+0000 7fad9c681640 1 -- 172.21.15.122:0/1286660237 wait complete. 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 Processor -- start 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 -- start start 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad941987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 0x7fad9419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad94110db0 con 0x7fad94100c50 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fad94110c30 con 0x7fad941059a0 2024-04-02T13:57:12.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9c681640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad94110f30 con 0x7fad94101610 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54500/0 (socket says 172.21.15.122:54500) 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 -- 172.21.15.122:0/859618123 learned_addr learned my addr 172.21.15.122:0/859618123 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9ac1e640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 0x7fad9419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad99c1c640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad941987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 -- 172.21.15.122:0/859618123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 msgr2=0x7fad941987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad941987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 -- 172.21.15.122:0/859618123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 msgr2=0x7fad9419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 0x7fad9419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 -- 172.21.15.122:0/859618123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9419d250 con 0x7fad94100c50 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad99c1c640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad941987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9ac1e640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 0x7fad9419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.447+0000 7fad9a41d640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0098d0 tx=0x7fad8c0344c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.448+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c042020 con 0x7fad94100c50 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.448+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad8c034770 con 0x7fad94100c50 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.448+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9419d4e0 con 0x7fad94100c50 2024-04-02T13:57:12.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.448+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad8c039040 con 0x7fad94100c50 2024-04-02T13:57:12.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.448+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad941a4dd0 con 0x7fad94100c50 2024-04-02T13:57:12.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.449+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad94102110 con 0x7fad94100c50 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.450+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fad8c045020 con 0x7fad94100c50 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.450+0000 7fad8b7fe640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 0x7fad800a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.451+0000 7fad99c1c640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 0x7fad800a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.451+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fad8c0f34b0 con 0x7fad94100c50 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.451+0000 7fad99c1c640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 0x7fad800a6300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fad9406b400 tx=0x7fad84005ec0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:12.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.455+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fad8c04c020 con 0x7fad94100c50 2024-04-02T13:57:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.659+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fad9406a320 con 0x7fad800a3e40 2024-04-02T13:57:12.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.672+0000 7fad8b7fe640 1 -- 172.21.15.122:0/859618123 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7fad9406a320 con 0x7fad800a3e40 2024-04-02T13:57:12.675 INFO:teuthology.orchestra.run.smithi122.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-02T00:00:00", "created": "2024-04-02T13:35:15", "first": "2024-04-02T13:36:00", "last": "2024-04-02T13:57:00", "last_pruned": "2024-04-02T13:57:01", "created_count": 22, "pruned_count": 16, "active": true} 2024-04-02T13:57:12.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 msgr2=0x7fad800a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 0x7fad800a6300 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fad9406b400 tx=0x7fad84005ec0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 msgr2=0x7fad94198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0098d0 tx=0x7fad8c0344c0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 shutdown_connections 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fad800a3e40 0x7fad800a6300 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fad941059a0 0x7fad9419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad94101610 0x7fad941987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 --2- 172.21.15.122:0/859618123 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad94100c50 0x7fad94198260 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 >> 172.21.15.122:0/859618123 conn(0x7fad94077860 msgr2=0x7fad94104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:12.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 shutdown_connections 2024-04-02T13:57:12.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.674+0000 7fad9c681640 1 -- 172.21.15.122:0/859618123 wait complete. 2024-04-02T13:57:12.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-02T13:57:12.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 -- 172.21.15.122:0/1830647977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c107ac0 msgr2=0x7f839c107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 --2- 172.21.15.122:0/1830647977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c107ac0 0x7f839c107ea0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f838c009930 tx=0x7f838c02bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:12.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 -- 172.21.15.122:0/1830647977 shutdown_connections 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 --2- 172.21.15.122:0/1830647977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c102470 0x7f839c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 --2- 172.21.15.122:0/1830647977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 --2- 172.21.15.122:0/1830647977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c107ac0 0x7f839c107ea0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 -- 172.21.15.122:0/1830647977 >> 172.21.15.122:0/1830647977 conn(0x7f839c0fd5c0 msgr2=0x7f839c0ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 -- 172.21.15.122:0/1830647977 shutdown_connections 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.789+0000 7f83a10f2640 1 -- 172.21.15.122:0/1830647977 wait complete. 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 Processor -- start 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 -- start start 2024-04-02T13:57:12.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c107ac0 0x7f839c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111ed0 con 0x7f839c102470 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111d50 con 0x7f839c101ab0 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f83a10f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f839c112050 con 0x7f839c107ac0 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54516/0 (socket says 172.21.15.122:54516) 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 -- 172.21.15.122:0/394842530 learned_addr learned my addr 172.21.15.122:0/394842530 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:12.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f839ad9d640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f839b59e640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c107ac0 0x7f839c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 -- 172.21.15.122:0/394842530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c107ac0 msgr2=0x7f839c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c107ac0 0x7f839c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 -- 172.21.15.122:0/394842530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 msgr2=0x7f839c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 -- 172.21.15.122:0/394842530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c19d360 con 0x7f839c102470 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f839ad9d640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8393fff640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f838400d950 tx=0x7f838400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:12.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.790+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384017070 con 0x7f839c102470 2024-04-02T13:57:12.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.791+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c19d650 con 0x7f839c102470 2024-04-02T13:57:12.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.791+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8384009c30 con 0x7f839c102470 2024-04-02T13:57:12.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.791+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838400f6b0 con 0x7f839c102470 2024-04-02T13:57:12.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.791+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1a4ee0 con 0x7f839c102470 2024-04-02T13:57:12.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.792+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c04f860 con 0x7f839c102470 2024-04-02T13:57:12.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.793+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f83840109d0 con 0x7f839c102470 2024-04-02T13:57:12.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.793+0000 7f8398d99640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 0x7f83880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:12.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.793+0000 7f839ad9d640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 0x7f83880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:12.799 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.793+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f83840d0900 con 0x7f839c102470 2024-04-02T13:57:12.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.794+0000 7f839ad9d640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 0x7f83880a6280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f838c036040 tx=0x7f838c000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:12.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:12.798+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f8384009de0 con 0x7f839c102470 2024-04-02T13:57:13.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.006+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 --> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f839c061f40 con 0x7f83880a3dc0 2024-04-02T13:57:13.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.035+0000 7f8398d99640 1 -- 172.21.15.122:0/394842530 <== mgr.15528 v2:172.21.15.122:6800/692246399 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f839c061f40 con 0x7f83880a3dc0 2024-04-02T13:57:13.039 INFO:teuthology.orchestra.run.smithi122.stdout:/ 1m 6m3h 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 msgr2=0x7f83880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 0x7f83880a6280 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f838c036040 tx=0x7f838c000bc0 comp rx=0 tx=0).stop 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 msgr2=0x7f839c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f838400d950 tx=0x7f838400de10 comp rx=0 tx=0).stop 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 shutdown_connections 2024-04-02T13:57:13.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f83880a3dc0 0x7f83880a6280 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f839c107ac0 0x7f839c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f839c102470 0x7f839c1988b0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 --2- 172.21.15.122:0/394842530 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f839c101ab0 0x7f839c198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 >> 172.21.15.122:0/394842530 conn(0x7f839c0fd5c0 msgr2=0x7f839c0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 shutdown_connections 2024-04-02T13:57:13.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:13.038+0000 7f83a10f2640 1 -- 172.21.15.122:0/394842530 wait complete. 2024-04-02T13:57:13.052 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-02T13:57:13.063 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-02T13:57:14.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:13 smithi187 ceph-mon[36064]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-04-02T13:57:14.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:13 smithi187 ceph-mon[36064]: from='client.21576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:14.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:13 smithi188 ceph-mon[36117]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-04-02T13:57:14.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:13 smithi188 ceph-mon[36117]: from='client.21576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:14.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:13 smithi122 ceph-mon[32114]: pgmap v695: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.0 MiB/s wr, 508 op/s 2024-04-02T13:57:14.321 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:13 smithi122 ceph-mon[32114]: from='client.21576 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:14.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-02T13:57:14.358 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-02T13:57:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/955049962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0103690 msgr2=0x7fa9f010bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/955049962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0103690 0x7fa9f010bb70 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009900 tx=0x7fa9e402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/955049962 shutdown_connections 2024-04-02T13:57:14.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/955049962 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f010c140 0x7fa9f010e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/955049962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0103690 0x7fa9f010bb70 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/955049962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f0102d70 0x7fa9f0103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/955049962 >> 172.21.15.122:0/955049962 conn(0x7fa9f0077b60 msgr2=0x7fa9f0077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/955049962 shutdown_connections 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.470+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/955049962 wait complete. 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 Processor -- start 2024-04-02T13:57:14.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 -- start start 2024-04-02T13:57:14.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 0x7fa9f0100f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 0x7fa9f0101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111f10 con 0x7fa9f0102d70 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111d90 con 0x7fa9f0103690 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.471+0000 7fa9f5e1b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0112090 con 0x7fa9f010c140 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54536/0 (socket says 172.21.15.122:54536) 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 -- 172.21.15.122:0/3540628813 learned_addr learned my addr 172.21.15.122:0/3540628813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 0x7fa9f0100f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9effff640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 0x7fa9f0101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 -- 172.21.15.122:0/3540628813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 msgr2=0x7fa9f0101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 0x7fa9f0101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 -- 172.21.15.122:0/3540628813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 msgr2=0x7fa9f0100f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 0x7fa9f0100f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 -- 172.21.15.122:0/3540628813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f01a68f0 con 0x7fa9f0102d70 2024-04-02T13:57:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 0x7fa9f0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9effff640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 0x7fa9f0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000d770 tx=0x7fa9e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ecff9640 1 -- 172.21.15.122:0/3540628813 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0017070 con 0x7fa9f0102d70 2024-04-02T13:57:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ecff9640 1 -- 172.21.15.122:0/3540628813 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e0005500 con 0x7fa9f0102d70 2024-04-02T13:57:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f01a6be0 con 0x7fa9f0102d70 2024-04-02T13:57:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9ecff9640 1 -- 172.21.15.122:0/3540628813 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0004e00 con 0x7fa9f0102d70 2024-04-02T13:57:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.472+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f01a70e0 con 0x7fa9f0102d70 2024-04-02T13:57:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.474+0000 7fa9ecff9640 1 -- 172.21.15.122:0/3540628813 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa9e0004fa0 con 0x7fa9f0102d70 2024-04-02T13:57:14.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 Processor -- start 2024-04-02T13:57:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 -- start start 2024-04-02T13:57:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x557897522cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975231f0 0x55789752b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557897486830 con 0x55789752bdd0 2024-04-02T13:57:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55789742e820 con 0x5578975228d0 2024-04-02T13:57:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578974827e0 con 0x5578975231f0 2024-04-02T13:57:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9ecff9640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 0x7fa9d80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54552/0 (socket says 172.21.15.122:54552) 2024-04-02T13:57:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 -- 172.21.15.122:0/786914410 learned_addr learned my addr 172.21.15.122:0/786914410 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9effff640 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x557897522cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9effff640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 0x7fa9d80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9ecff9640 1 -- 172.21.15.122:0/3540628813 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa9e00d0710 con 0x7fa9f0102d70 2024-04-02T13:57:14.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975231f0 0x55789752b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 -- 172.21.15.122:0/786914410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975231f0 msgr2=0x55789752b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975231f0 0x55789752b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 -- 172.21.15.122:0/786914410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 msgr2=0x557897522cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x557897522cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.475+0000 7fa9eeffd640 1 -- 172.21.15.122:0/786914410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578972647d0 con 0x55789752bdd0 2024-04-02T13:57:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fa9e402c210 tx=0x7fa9e4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc5875d0560764b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9effff640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 0x7fa9d80a6210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8005c40 tx=0x7fa9e8009280 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9d5ffb640 1 -- 172.21.15.122:0/786914410 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4042070 con 0x55789752bdd0 2024-04-02T13:57:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9d5ffb640 1 -- 172.21.15.122:0/786914410 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e402cb20 con 0x55789752bdd0 2024-04-02T13:57:14.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9d5ffb640 1 -- 172.21.15.122:0/786914410 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40046c0 con 0x55789752bdd0 2024-04-02T13:57:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9f8a04740 1 -- 172.21.15.122:0/786914410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 msgr2=0x55789752e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.476+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fa9e402c210 tx=0x7fa9e4005e80 comp rx=0 tx=0).stop 2024-04-02T13:57:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- 172.21.15.122:0/786914410 shutdown_connections 2024-04-02T13:57:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55789752bdd0 0x55789752e1c0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975231f0 0x55789752b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/786914410 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x557897522cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- 172.21.15.122:0/786914410 >> 172.21.15.122:0/786914410 conn(0x55789751d000 msgr2=0x55789751d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- 172.21.15.122:0/786914410 shutdown_connections 2024-04-02T13:57:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- 172.21.15.122:0/786914410 wait complete. 2024-04-02T13:57:14.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- start start 2024-04-02T13:57:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 Processor -- start 2024-04-02T13:57:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x55789752b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55789752bdd0 0x557897528ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55789752eb60 con 0x5578975231f0 2024-04-02T13:57:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x557897441d30 con 0x5578975228d0 2024-04-02T13:57:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55789752ece0 con 0x55789752bdd0 2024-04-02T13:57:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9f8a04740 10 client.0 _ll_register_callbacks cb 0x557897441720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:14.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.477+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54554/0 (socket says 172.21.15.122:54554) 2024-04-02T13:57:14.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 -- 172.21.15.122:0/1075077419 learned_addr learned my addr 172.21.15.122:0/1075077419 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9effff640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55789752bdd0 0x557897528ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x55789752b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 -- 172.21.15.122:0/1075077419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55789752bdd0 msgr2=0x557897528ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55789752bdd0 0x557897528ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 -- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 msgr2=0x55789752b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x55789752b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.500 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 -- 172.21.15.122:0/1075077419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789719a3f0 con 0x5578975231f0 2024-04-02T13:57:14.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e402c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd991484c401f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:14.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4042050 con 0x5578975231f0 2024-04-02T13:57:14.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa9e4004bd0 con 0x5578975231f0 2024-04-02T13:57:14.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e402cb20 con 0x5578975231f0 2024-04-02T13:57:14.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4004dd0 con 0x5578975231f0 2024-04-02T13:57:14.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9f8a04740 10 client.21594 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:14.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55789715a0f0 con 0x5578975231f0 2024-04-02T13:57:14.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.478+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e402cb20 con 0x5578975231f0 2024-04-02T13:57:14.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e4034440 con 0x5578975231f0 2024-04-02T13:57:14.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 10 client.21594 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:14.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 10 client.21594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:14.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 20 client.21594 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:14.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55789753ba80 0x55789753de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 4 client.21594 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:14.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557897493d70 con 0x55789753ba80 2024-04-02T13:57:14.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55789753ba80 0x55789753de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9ceffd640 10 client.21594 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:14.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.479+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55789753ba80 0x55789753de70 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e43d063b33f32ab server_cookie=f3d185aab1faaf1 in_seq=0 out_seq=0 2024-04-02T13:57:14.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9ceffd640 1 -- 172.21.15.122:0/1075077419 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557897493d70 con 0x55789753ba80 2024-04-02T13:57:14.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9ceffd640 10 client.21594 handle_command_reply: tid=0 2024-04-02T13:57:14.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 client.21594 shutdown 2024-04-02T13:57:14.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 msgr2=0x5578975289a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4006010 tx=0x7fa9e402c9f0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 shutdown_connections 2024-04-02T13:57:14.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x55789753ba80 0x55789753de70 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55789752bdd0 0x557897528ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x5578975289a0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/1075077419 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5578975228d0 0x55789752b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9ceffd640 0 client.21594 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:14.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 >> 172.21.15.122:0/1075077419 conn(0x55789751d000 msgr2=0x55789751ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 shutdown_connections 2024-04-02T13:57:14.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 1 -- 172.21.15.122:0/1075077419 wait complete. 2024-04-02T13:57:14.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.480+0000 7fa9f8a04740 20 client.21594 trim_cache size 0 max 16384 2024-04-02T13:57:14.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.497+0000 7fa9f8a04740 1 Processor -- start 2024-04-02T13:57:14.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.497+0000 7fa9f8a04740 1 -- start start 2024-04-02T13:57:14.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789751c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x55789752b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b1280 con 0x5578975231f0 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b2dc0 con 0x55789752bdd0 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b1950 con 0x5578975228d0 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54558/0 (socket says 172.21.15.122:54558) 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 -- 172.21.15.122:0/74019054 learned_addr learned my addr 172.21.15.122:0/74019054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9effff640 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x55789752b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 -- 172.21.15.122:0/74019054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 msgr2=0x55789751c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789751c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 -- 172.21.15.122:0/74019054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 msgr2=0x55789752b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x55789752b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 -- 172.21.15.122:0/74019054 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789751ce30 con 0x5578975231f0 2024-04-02T13:57:14.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9effff640 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x55789752b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4000c00 tx=0x7fa9e40058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28169a512780bad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9d57fa640 1 -- 172.21.15.122:0/74019054 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4042030 con 0x5578975231f0 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9d57fa640 1 -- 172.21.15.122:0/74019054 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e402ca60 con 0x5578975231f0 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.498+0000 7fa9d57fa640 1 -- 172.21.15.122:0/74019054 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40049d0 con 0x5578975231f0 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- 172.21.15.122:0/74019054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 msgr2=0x55789752b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4000c00 tx=0x7fa9e40058b0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- 172.21.15.122:0/74019054 shutdown_connections 2024-04-02T13:57:14.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x55789752b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x55789752b0d0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/74019054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789751c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- 172.21.15.122:0/74019054 >> 172.21.15.122:0/74019054 conn(0x55789751d000 msgr2=0x55789751b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- 172.21.15.122:0/74019054 shutdown_connections 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- 172.21.15.122:0/74019054 wait complete. 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 -- start start 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.499+0000 7fa9f8a04740 1 Processor -- start 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789752fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x5578975305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b26f0 con 0x5578975231f0 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b2020 con 0x55789752bdd0 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01af280 con 0x5578975228d0 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 10 client.0 _ll_register_callbacks cb 0x7fa9e402c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54562/0 (socket says 172.21.15.122:54562) 2024-04-02T13:57:14.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 -- 172.21.15.122:0/3414903461 learned_addr learned my addr 172.21.15.122:0/3414903461 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9effff640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x5578975305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 -- 172.21.15.122:0/3414903461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 msgr2=0x55789752fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789752fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789752fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 -- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 msgr2=0x5578975305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x5578975305b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 -- 172.21.15.122:0/3414903461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578975a0ee0 con 0x5578975231f0 2024-04-02T13:57:14.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789752fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9eeffd640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005ec0 tx=0x7fa9e4005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc57b5edd3b0e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4042040 con 0x5578975231f0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa9e40049d0 con 0x5578975231f0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e402ca60 con 0x5578975231f0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa9e406b050 con 0x5578975231f0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 10 client.21600 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.500+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5578975a2140 con 0x5578975231f0 2024-04-02T13:57:14.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e402ca60 con 0x5578975231f0 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa9e4047450 con 0x5578975231f0 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 10 client.21600 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 10 client.21600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 20 client.21600 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578975a2660 0x5578975a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 4 client.21600 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55789748e0c0 con 0x5578975a2660 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.501+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578975a2660 0x5578975a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9ceffd640 10 client.21600 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9ef7fe640 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578975a2660 0x5578975a4a80 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02e109e5fd55fc2 server_cookie=ef3114c3c1d3d27f in_seq=0 out_seq=0 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9ceffd640 1 -- 172.21.15.122:0/3414903461 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55789748e0c0 con 0x5578975a2660 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9ceffd640 10 client.21600 handle_command_reply: tid=0 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 client.21600 shutdown 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 msgr2=0x557897530070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005ec0 tx=0x7fa9e4005be0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 shutdown_connections 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x5578975a2660 0x5578975a4a80 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55789752bdd0 0x5578975305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9ceffd640 0 client.21600 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5578975231f0 0x557897530070 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 --2- 172.21.15.122:0/3414903461 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578975228d0 0x55789752fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 >> 172.21.15.122:0/3414903461 conn(0x55789751d000 msgr2=0x55789751c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 shutdown_connections 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 1 -- 172.21.15.122:0/3414903461 wait complete. 2024-04-02T13:57:14.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f8a04740 20 client.21600 trim_cache size 0 max 16384 2024-04-02T13:57:14.528 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:14.528 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "no active scrubs running", 2024-04-02T13:57:14.528 INFO:teuthology.orchestra.run.smithi122.stdout: "scrubs": {} 2024-04-02T13:57:14.528 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 msgr2=0x7fa9d80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.502+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 0x7fa9d80a6210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8005c40 tx=0x7fa9e8009280 comp rx=0 tx=0).stop 2024-04-02T13:57:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 msgr2=0x7fa9f01009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000d770 tx=0x7fa9e000dc30 comp rx=0 tx=0).stop 2024-04-02T13:57:14.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 shutdown_connections 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa9d80a3d50 0x7fa9d80a6210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9f010c140 0x7fa9f0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa9f0103690 0x7fa9f0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 --2- 172.21.15.122:0/3540628813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9f0102d70 0x7fa9f01009c0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 >> 172.21.15.122:0/3540628813 conn(0x7fa9f0077b60 msgr2=0x7fa9f00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 shutdown_connections 2024-04-02T13:57:14.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.503+0000 7fa9f5e1b640 1 -- 172.21.15.122:0/3540628813 wait complete. 2024-04-02T13:57:14.532 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-02T13:57:14.532 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 -- 172.21.15.122:0/1768429150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 msgr2=0x7fdfc4100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 --2- 172.21.15.122:0/1768429150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4100e60 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009900 tx=0x7fdfb002bcd0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 -- 172.21.15.122:0/1768429150 shutdown_connections 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 --2- 172.21.15.122:0/1768429150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41013a0 0x7fdfc410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 --2- 172.21.15.122:0/1768429150 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4100e60 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 --2- 172.21.15.122:0/1768429150 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41058e0 0x7fdfc4105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 -- 172.21.15.122:0/1768429150 >> 172.21.15.122:0/1768429150 conn(0x7fdfc4077b30 msgr2=0x7fdfc4077f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.634+0000 7fdfcae33640 1 -- 172.21.15.122:0/1768429150 shutdown_connections 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 -- 172.21.15.122:0/1768429150 wait complete. 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 Processor -- start 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 -- start start 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 0x7fdfc4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 0x7fdfc419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110dd0 con 0x7fdfc4100a00 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110c50 con 0x7fdfc41013a0 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfcae33640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110f50 con 0x7fdfc41058e0 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfc8bcf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfc8bcf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54582/0 (socket says 172.21.15.122:54582) 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.635+0000 7fdfc8bcf640 1 -- 172.21.15.122:0/3002231069 learned_addr learned my addr 172.21.15.122:0/3002231069 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc93d0640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 0x7fdfc419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc3fff640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 0x7fdfc4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 -- 172.21.15.122:0/3002231069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 msgr2=0x7fdfc419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 0x7fdfc419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 -- 172.21.15.122:0/3002231069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 msgr2=0x7fdfc4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 0x7fdfc4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 -- 172.21.15.122:0/3002231069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc419c1f0 con 0x7fdfc4100a00 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc93d0640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 0x7fdfc419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc3fff640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 0x7fdfc4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:14.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc8bcf640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009970 tx=0x7fdfb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800d020 con 0x7fdfc4100a00 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb8007d40 con 0x7fdfc4100a00 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb800ccc0 con 0x7fdfc4100a00 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc419c4e0 con 0x7fdfc4100a00 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.636+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc4069e00 con 0x7fdfc4100a00 2024-04-02T13:57:14.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.637+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4068060 con 0x7fdfc4100a00 2024-04-02T13:57:14.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.638+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fdfb800c460 con 0x7fdfc4100a00 2024-04-02T13:57:14.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.639+0000 7fdfc1ffb640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 0x7fdfb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.639+0000 7fdfc3fff640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 0x7fdfb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.639+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fdfb80d2570 con 0x7fdfc4100a00 2024-04-02T13:57:14.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.639+0000 7fdfc3fff640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 0x7fdfb40a61e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdfb002efd0 tx=0x7fdfb0000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.642+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fdfb809e7d0 con 0x7fdfc4100a00 2024-04-02T13:57:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.863+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfc406a8f0 con 0x7fdfc4100a00 2024-04-02T13:57:14.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.864+0000 7fdfc1ffb640 1 -- 172.21.15.122:0/3002231069 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11942 (secure 0 0 0) 0x7fdfb809e0d0 con 0x7fdfc4100a00 2024-04-02T13:57:14.867 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:57:14.867 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:55:07.726900+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:57:14.867 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 76 2024-04-02T13:57:14.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 msgr2=0x7fdfb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 0x7fdfb40a61e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fdfb002efd0 tx=0x7fdfb0000bc0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 msgr2=0x7fdfc4197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009970 tx=0x7fdfb8009e30 comp rx=0 tx=0).stop 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 shutdown_connections 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fdfb40a3d20 0x7fdfb40a61e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfc41058e0 0x7fdfc419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdfc41013a0 0x7fdfc4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 --2- 172.21.15.122:0/3002231069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdfc4100a00 0x7fdfc4197250 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 >> 172.21.15.122:0/3002231069 conn(0x7fdfc4077b30 msgr2=0x7fdfc4103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 shutdown_connections 2024-04-02T13:57:14.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.866+0000 7fdfcae33640 1 -- 172.21.15.122:0/3002231069 wait complete. 2024-04-02T13:57:14.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-02T13:57:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 -- 172.21.15.122:0/2538536242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d8410e4a0 msgr2=0x7f5d84110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/2538536242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d8410e4a0 0x7f5d84110890 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009ca0 tx=0x7f5d7c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 -- 172.21.15.122:0/2538536242 shutdown_connections 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/2538536242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d8410e4a0 0x7f5d84110890 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/2538536242 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d841012a0 0x7f5d8410df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/2538536242 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841008f0 0x7f5d84100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 -- 172.21.15.122:0/2538536242 >> 172.21.15.122:0/2538536242 conn(0x7f5d84077b60 msgr2=0x7f5d84077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 -- 172.21.15.122:0/2538536242 shutdown_connections 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.983+0000 7f5d8a9db640 1 -- 172.21.15.122:0/2538536242 wait complete. 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 Processor -- start 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 -- start start 2024-04-02T13:57:14.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 0x7f5d8419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d8410e4a0 0x7f5d8419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84114160 con 0x7f5d841008f0 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d84113fe0 con 0x7f5d841012a0 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d8a9db640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841142e0 con 0x7f5d8410e4a0 2024-04-02T13:57:14.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54600/0 (socket says 172.21.15.122:54600) 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 -- 172.21.15.122:0/1087801201 learned_addr learned my addr 172.21.15.122:0/1087801201 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d837fe640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 0x7f5d8419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 -- 172.21.15.122:0/1087801201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d8410e4a0 msgr2=0x7f5d8419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d8410e4a0 0x7f5d8419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 -- 172.21.15.122:0/1087801201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 msgr2=0x7f5d8419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:14.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 0x7f5d8419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d83fff640 1 -- 172.21.15.122:0/1087801201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8419f6c0 con 0x7f5d841008f0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.984+0000 7f5d837fe640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 0x7f5d8419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d83fff640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400d950 tx=0x7f5d7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d817fa640 1 -- 172.21.15.122:0/1087801201 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74017070 con 0x7f5d841008f0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d817fa640 1 -- 172.21.15.122:0/1087801201 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d74009c30 con 0x7f5d841008f0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8419f9b0 con 0x7f5d841008f0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d817fa640 1 -- 172.21.15.122:0/1087801201 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400f6b0 con 0x7f5d841008f0 2024-04-02T13:57:14.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.985+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d841a7240 con 0x7f5d841008f0 2024-04-02T13:57:14.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d817fa640 1 -- 172.21.15.122:0/1087801201 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5d740109d0 con 0x7f5d841008f0 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 Processor -- start 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 -- start start 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 0x55593ae9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55593adf47b0 con 0x55593ae910c0 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55593ad93570 con 0x55593ae99ca0 2024-04-02T13:57:15.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.987+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55593adf0760 con 0x55593ae907a0 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54614/0 (socket says 172.21.15.122:54614) 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 -- 172.21.15.122:0/4131729734 learned_addr learned my addr 172.21.15.122:0/4131729734 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d88f78640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d817fa640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 0x7f5d6c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d837fe640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 0x55593ae9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 -- 172.21.15.122:0/4131729734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 msgr2=0x55593ae90b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae90b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 -- 172.21.15.122:0/4131729734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 msgr2=0x55593ae9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 0x55593ae9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 -- 172.21.15.122:0/4131729734 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593ab083f0 con 0x55593ae910c0 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d88f78640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d837fe640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 0x55593ae9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d88f78640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 0x7f5d6c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d817fa640 1 -- 172.21.15.122:0/1087801201 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f5d740d0980 con 0x7f5d841008f0 2024-04-02T13:57:15.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d83fff640 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f5d7409c180 tx=0x7f5d74022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4626a53d8dbb8ad5 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d6a7fc640 1 -- 172.21.15.122:0/4131729734 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74007420 con 0x55593ae910c0 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d6a7fc640 1 -- 172.21.15.122:0/4131729734 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d74009de0 con 0x55593ae910c0 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d88f78640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 0x7f5d6c0a6260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c700 tx=0x7f5d7c004a60 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.988+0000 7f5d6a7fc640 1 -- 172.21.15.122:0/4131729734 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7409c480 con 0x55593ae910c0 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/4131729734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 msgr2=0x55593ae99760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f5d7409c180 tx=0x7f5d74022000 comp rx=0 tx=0).stop 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/4131729734 shutdown_connections 2024-04-02T13:57:15.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae99ca0 0x55593ae9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae99760 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/4131729734 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae90b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/4131729734 >> 172.21.15.122:0/4131729734 conn(0x55593ae8aed0 msgr2=0x55593ae8b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/4131729734 shutdown_connections 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/4131729734 wait complete. 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.989+0000 7f5d8d5c2740 1 -- start start 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 Processor -- start 2024-04-02T13:57:15.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593ae996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 0x55593ae97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55593ae9c9d0 con 0x55593ae910c0 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55593adafe80 con 0x55593ae907a0 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55593ae9cb50 con 0x55593ae99ca0 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d8d5c2740 10 client.0 _ll_register_callbacks cb 0x55593adebaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54624/0 (socket says 172.21.15.122:54624) 2024-04-02T13:57:15.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 -- 172.21.15.122:0/550737585 learned_addr learned my addr 172.21.15.122:0/550737585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d83fff640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593ae996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d88f78640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 0x55593ae97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 -- 172.21.15.122:0/550737585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 msgr2=0x55593ae97050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 0x55593ae97050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 -- 172.21.15.122:0/550737585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 msgr2=0x55593ae996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593ae996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 -- 172.21.15.122:0/550737585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593abd27d0 con 0x55593ae910c0 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d83fff640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593ae996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d88f78640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 0x55593ae97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d837fe640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007fd0 tx=0x7f5d7800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d4e40248785e server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.990+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d78018020 con 0x55593ae910c0 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f5d78014870 con 0x55593ae910c0 2024-04-02T13:57:15.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d74009de0 con 0x55593ae910c0 2024-04-02T13:57:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5d78023050 con 0x55593ae910c0 2024-04-02T13:57:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 10 client.21618 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55593aea8a90 con 0x55593ae910c0 2024-04-02T13:57:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d74009de0 con 0x55593ae910c0 2024-04-02T13:57:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f5d78013b20 con 0x55593ae910c0 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 10 client.21618 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 10 client.21618 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 20 client.21618 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593aea9ff0 0x55593aeac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 4 client.21618 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55593ae01cf0 con 0x55593aea9ff0 2024-04-02T13:57:15.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d83fff640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593aea9ff0 0x55593aeac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d637fe640 10 client.21618 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.991+0000 7f5d83fff640 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593aea9ff0 0x55593aeac430 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d32ee2b809951a4c server_cookie=948c2d0eab98e97 in_seq=0 out_seq=0 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d637fe640 1 -- 172.21.15.122:0/550737585 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55593ae01cf0 con 0x55593aea9ff0 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d637fe640 10 client.21618 handle_command_reply: tid=0 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 client.21618 shutdown 2024-04-02T13:57:15.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 msgr2=0x55593ae96a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f5d78007fd0 tx=0x7f5d7800f670 comp rx=0 tx=0).stop 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 shutdown_connections 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593aea9ff0 0x55593aeac430 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae99ca0 0x55593ae97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae910c0 0x55593ae96a40 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/550737585 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593ae996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d637fe640 0 client.21618 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:57:15.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 >> 172.21.15.122:0/550737585 conn(0x55593ae8aed0 msgr2=0x55593ae8cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 shutdown_connections 2024-04-02T13:57:15.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/550737585 wait complete. 2024-04-02T13:57:15.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:14.992+0000 7f5d8d5c2740 20 client.21618 trim_cache size 0 max 16384 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 Processor -- start 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 -- start start 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593aea58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae910c0 0x55593ae96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b1490 con 0x55593ae99ca0 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b2fd0 con 0x55593ae907a0 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b1b60 con 0x55593ae910c0 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54638/0 (socket says 172.21.15.122:54638) 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 -- 172.21.15.122:0/3394769835 learned_addr learned my addr 172.21.15.122:0/3394769835 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d83fff640 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593aea58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 -- 172.21.15.122:0/3394769835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae910c0 msgr2=0x55593ae96a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae910c0 0x55593ae96a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 -- 172.21.15.122:0/3394769835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 msgr2=0x55593aea58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593aea58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d88f78640 1 -- 172.21.15.122:0/3394769835 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593aeacab0 con 0x55593ae99ca0 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.006+0000 7f5d83fff640 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593aea58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c004af0 tx=0x7f5d7c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea6f1c089f96ef3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d69ffb640 1 -- 172.21.15.122:0/3394769835 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c045040 con 0x55593ae99ca0 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d69ffb640 1 -- 172.21.15.122:0/3394769835 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c04f070 con 0x55593ae99ca0 2024-04-02T13:57:15.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d69ffb640 1 -- 172.21.15.122:0/3394769835 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c039050 con 0x55593ae99ca0 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3394769835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 msgr2=0x55593ae99170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c004af0 tx=0x7f5d7c03d000 comp rx=0 tx=0).stop 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3394769835 shutdown_connections 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593ae99170 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae910c0 0x55593ae96a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3394769835 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae907a0 0x55593aea58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3394769835 >> 172.21.15.122:0/3394769835 conn(0x55593ae8aed0 msgr2=0x55593ae95310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3394769835 shutdown_connections 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.007+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3394769835 wait complete. 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 -- start start 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 Processor -- start 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae93480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 0x55593ae939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b2900 con 0x55593ae99ca0 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841b2230 con 0x55593ae910c0 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d841af490 con 0x55593ae907a0 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d8d5c2740 10 client.0 _ll_register_callbacks cb 0x7f5d780072c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54644/0 (socket says 172.21.15.122:54644) 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 -- 172.21.15.122:0/3000252766 learned_addr learned my addr 172.21.15.122:0/3000252766 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d83fff640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae93480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 -- 172.21.15.122:0/3000252766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 msgr2=0x55593ae93480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d837fe640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 0x55593ae939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae93480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 -- 172.21.15.122:0/3000252766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 msgr2=0x55593ae939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 0x55593ae939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d88f78640 1 -- 172.21.15.122:0/3000252766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593aea1310 con 0x55593ae99ca0 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.008+0000 7f5d837fe640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 0x55593ae939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d88f78640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c032530 tx=0x7f5d7c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849da8e80303dea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c039040 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f5d7c004410 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c04f070 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5d7c03f470 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 10 client.21624 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55593aea2540 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d7c04f070 con 0x55593ae99ca0 2024-04-02T13:57:15.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f5d7c0347a0 con 0x55593ae99ca0 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 10 client.21624 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 10 client.21624 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.a 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 20 client.21624 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593af10940 0x55593af12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 4 client.21624 mds_command: new command op to 14628 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 --> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55593adfc040 con 0x55593af10940 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.009+0000 7f5d83fff640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593af10940 0x55593af12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d637fe640 10 client.21624 ms_handle_connect on v2:172.21.15.122:6840/385777794 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d83fff640 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593af10940 0x55593af12d60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=97c2627456947a63 server_cookie=dc584a3bdf859bda in_seq=0 out_seq=0 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d637fe640 1 -- 172.21.15.122:0/3000252766 <== mds.4 v2:172.21.15.122:6840/385777794 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55593adfc040 con 0x55593af10940 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d637fe640 10 client.21624 handle_command_reply: tid=0 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 client.21624 shutdown 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 msgr2=0x55593aea0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c032530 tx=0x7f5d7c005da0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 shutdown_connections 2024-04-02T13:57:15.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:6840/385777794,v1:172.21.15.122:6841/385777794] conn(0x55593af10940 0x55593af12d60 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55593ae99ca0 0x55593aea0dd0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55593ae910c0 0x55593ae939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d637fe640 0 client.21624 ms_handle_reset on v2:172.21.15.122:6840/385777794 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 --2- 172.21.15.122:0/3000252766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55593ae907a0 0x55593ae93480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 >> 172.21.15.122:0/3000252766 conn(0x55593ae8aed0 msgr2=0x55593ae896a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 shutdown_connections 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 1 -- 172.21.15.122:0/3000252766 wait complete. 2024-04-02T13:57:15.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.010+0000 7f5d8d5c2740 20 client.21624 trim_cache size 0 max 16384 2024-04-02T13:57:15.030 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:57:15.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 msgr2=0x7f5d6c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 0x7f5d6c0a6260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c02c700 tx=0x7f5d7c004a60 comp rx=0 tx=0).stop 2024-04-02T13:57:15.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 msgr2=0x7f5d8419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400d950 tx=0x7f5d7400de10 comp rx=0 tx=0).stop 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 shutdown_connections 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d6c0a3da0 0x7f5d6c0a6260 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d8410e4a0 0x7f5d8419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d841012a0 0x7f5d8419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 --2- 172.21.15.122:0/1087801201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d841008f0 0x7f5d8419a720 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 >> 172.21.15.122:0/1087801201 conn(0x7f5d84077b60 msgr2=0x7f5d8410f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 shutdown_connections 2024-04-02T13:57:15.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.011+0000 7f5d8a9db640 1 -- 172.21.15.122:0/1087801201 wait complete. 2024-04-02T13:57:15.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-02T13:57:15.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 -- 172.21.15.122:0/1536515201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c1069c0 msgr2=0x7f766c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 --2- 172.21.15.122:0/1536515201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c1069c0 0x7f766c106da0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f7660009930 tx=0x7f766002bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 -- 172.21.15.122:0/1536515201 shutdown_connections 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 --2- 172.21.15.122:0/1536515201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c101370 0x7f766c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 --2- 172.21.15.122:0/1536515201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1009b0 0x7f766c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 --2- 172.21.15.122:0/1536515201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c1069c0 0x7f766c106da0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 -- 172.21.15.122:0/1536515201 >> 172.21.15.122:0/1536515201 conn(0x7f766c076d80 msgr2=0x7f766c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 -- 172.21.15.122:0/1536515201 shutdown_connections 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.135+0000 7f7670f67640 1 -- 172.21.15.122:0/1536515201 wait complete. 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 Processor -- start 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 -- start start 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c1009b0 0x7f766c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1069c0 0x7f766c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f766c111e90 con 0x7f766c101370 2024-04-02T13:57:15.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f766c111d10 con 0x7f766c1069c0 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7670f67640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f766c112010 con 0x7f766c1009b0 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54660/0 (socket says 172.21.15.122:54660) 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 -- 172.21.15.122:0/2667937645 learned_addr learned my addr 172.21.15.122:0/2667937645 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1069c0 0x7f766c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f766a59c640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c1009b0 0x7f766c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 -- 172.21.15.122:0/2667937645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c1009b0 msgr2=0x7f766c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c1009b0 0x7f766c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 -- 172.21.15.122:0/2667937645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1069c0 msgr2=0x7f766c19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1069c0 0x7f766c19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.136+0000 7f7669d9b640 1 -- 172.21.15.122:0/2667937645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c19d3a0 con 0x7f766c101370 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e9f0 tx=0x7f765c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f765b7fe640 1 -- 172.21.15.122:0/2667937645 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c00cd80 con 0x7f766c101370 2024-04-02T13:57:15.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f765b7fe640 1 -- 172.21.15.122:0/2667937645 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c010910 con 0x7f766c101370 2024-04-02T13:57:15.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766c19d690 con 0x7f766c101370 2024-04-02T13:57:15.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f765b7fe640 1 -- 172.21.15.122:0/2667937645 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c018990 con 0x7f766c101370 2024-04-02T13:57:15.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.137+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766c1a4f20 con 0x7f766c101370 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f765b7fe640 1 -- 172.21.15.122:0/2667937645 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f765c010ac0 con 0x7f766c101370 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 Processor -- start 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 -- start start 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca2036eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca202d27b0 con 0x55ca20377ca0 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ca20271570 con 0x55ca2036e7a0 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7673c2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca202ce760 con 0x55ca2036f0c0 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54672/0 (socket says 172.21.15.122:54672) 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 -- 172.21.15.122:0/1720853773 learned_addr learned my addr 172.21.15.122:0/1720853773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f766ad9d640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca2036eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f765b7fe640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 0x7f76540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f766a59c640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 -- 172.21.15.122:0/1720853773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 msgr2=0x55ca20377760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 -- 172.21.15.122:0/1720853773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 msgr2=0x55ca2036eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.140+0000 7f7669d9b640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca2036eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f7669d9b640 1 -- 172.21.15.122:0/1720853773 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca1ffe63f0 con 0x55ca20377ca0 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f766ad9d640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca2036eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f766a59c640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 0x7f76540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f765b7fe640 1 -- 172.21.15.122:0/2667937645 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f765c0d0f60 con 0x7f766c101370 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f7669d9b640 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ce1e0 tx=0x7f765c0672e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db90dbc198573dca server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f7658ff9640 1 -- 172.21.15.122:0/1720853773 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c09c6b0 con 0x55ca20377ca0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 0x7f76540a61e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7664005fd0 tx=0x7f7664004a40 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f7658ff9640 1 -- 172.21.15.122:0/1720853773 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c0bd110 con 0x55ca20377ca0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.141+0000 7f7658ff9640 1 -- 172.21.15.122:0/1720853773 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c09c4d0 con 0x55ca20377ca0 2024-04-02T13:57:15.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- 172.21.15.122:0/1720853773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 msgr2=0x55ca2037a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ce1e0 tx=0x7f765c0672e0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- 172.21.15.122:0/1720853773 shutdown_connections 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca2037a090 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- 172.21.15.122:0/1720853773 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca2036eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- 172.21.15.122:0/1720853773 >> 172.21.15.122:0/1720853773 conn(0x55ca20368ed0 msgr2=0x55ca203692b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- 172.21.15.122:0/1720853773 shutdown_connections 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- 172.21.15.122:0/1720853773 wait complete. 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- start start 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 Processor -- start 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca20377670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20374a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ca2037aa30 con 0x55ca20377ca0 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55ca2028de80 con 0x55ca2036e7a0 2024-04-02T13:57:15.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca2037abb0 con 0x55ca2036f0c0 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.142+0000 7f7673c2e740 10 client.0 _ll_register_callbacks cb 0x55ca202c9aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54684/0 (socket says 172.21.15.122:54684) 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 -- 172.21.15.122:0/2078081214 learned_addr learned my addr 172.21.15.122:0/2078081214 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766a59c640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca20377670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20374a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 -- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 msgr2=0x55ca20374a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20374a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 -- 172.21.15.122:0/2078081214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 msgr2=0x55ca20377670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca20377670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 -- 172.21.15.122:0/2078081214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca200b07d0 con 0x55ca20377ca0 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ed0 tx=0x7f7664004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa48bd1391cb7500 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766401ed70 con 0x55ca20377ca0 2024-04-02T13:57:15.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f7664028470 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f765c0bd110 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7664013e50 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7673c2e740 10 client.21636 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ca203869d0 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.143+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c0bd110 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f766400f040 con 0x55ca20377ca0 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 10 client.21636 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 10 client.21636 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 20 client.21636 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca20387e90 0x55ca2038a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 4 client.21636 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca202dfcf0 con 0x55ca20387e90 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f766a59c640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca20387e90 0x55ca2038a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f7651ffb640 10 client.21636 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.144+0000 7f766a59c640 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca20387e90 0x55ca2038a2c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2869da16d9333ac1 server_cookie=69914eafdf468368 in_seq=0 out_seq=0 2024-04-02T13:57:15.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7651ffb640 1 -- 172.21.15.122:0/2078081214 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca202dfcf0 con 0x55ca20387e90 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7651ffb640 10 client.21636 handle_command_reply: tid=0 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 client.21636 shutdown 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 msgr2=0x55ca20375010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ed0 tx=0x7f7664004ab0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 shutdown_connections 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca20387e90 0x55ca2038a2c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20375010 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.162 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.145+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20374a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7651ffb640 0 client.21636 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2078081214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca20377670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 >> 172.21.15.122:0/2078081214 conn(0x55ca20368ed0 msgr2=0x55ca2036ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 shutdown_connections 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7673c2e740 1 -- 172.21.15.122:0/2078081214 wait complete. 2024-04-02T13:57:15.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.146+0000 7f7673c2e740 20 client.21636 trim_cache size 0 max 16384 2024-04-02T13:57:15.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 Processor -- start 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 -- start start 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca203838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1af2c0 con 0x55ca20377ca0 2024-04-02T13:57:15.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1b0e00 con 0x55ca2036e7a0 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7673c2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1af990 con 0x55ca2036f0c0 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54686/0 (socket says 172.21.15.122:54686) 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 -- 172.21.15.122:0/2115485783 learned_addr learned my addr 172.21.15.122:0/2115485783 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766a59c640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca203838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 -- 172.21.15.122:0/2115485783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 msgr2=0x55ca20377130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 -- 172.21.15.122:0/2115485783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 msgr2=0x55ca203838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca203838f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 -- 172.21.15.122:0/2115485783 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2036eef0 con 0x55ca20377ca0 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7664000f80 tx=0x7f7664009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1738856ca40a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.159+0000 7f7653fff640 1 -- 172.21.15.122:0/2115485783 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664039070 con 0x55ca20377ca0 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7653fff640 1 -- 172.21.15.122:0/2115485783 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664013e50 con 0x55ca20377ca0 2024-04-02T13:57:15.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7653fff640 1 -- 172.21.15.122:0/2115485783 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664028d90 con 0x55ca20377ca0 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- 172.21.15.122:0/2115485783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 msgr2=0x55ca20377670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7664000f80 tx=0x7f7664009fa0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- 172.21.15.122:0/2115485783 shutdown_connections 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca20377670 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036f0c0 0x55ca20377130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2115485783 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036e7a0 0x55ca203838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- 172.21.15.122:0/2115485783 >> 172.21.15.122:0/2115485783 conn(0x55ca20368ed0 msgr2=0x55ca203676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- 172.21.15.122:0/2115485783 shutdown_connections 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- 172.21.15.122:0/2115485783 wait complete. 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 -- start start 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.160+0000 7f7673c2e740 1 Processor -- start 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036e7a0 0x55ca203800d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036f0c0 0x55ca20371d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1b0730 con 0x55ca20377ca0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1b0060 con 0x55ca2036f0c0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f766c1ad2c0 con 0x55ca2036e7a0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 10 client.0 _ll_register_callbacks cb 0x7f7664005bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54688/0 (socket says 172.21.15.122:54688) 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 -- 172.21.15.122:0/2328414804 learned_addr learned my addr 172.21.15.122:0/2328414804 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766a59c640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036e7a0 0x55ca203800d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7669d9b640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036f0c0 0x55ca20371d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 -- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036e7a0 msgr2=0x55ca203800d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036e7a0 0x55ca203800d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 -- 172.21.15.122:0/2328414804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036f0c0 msgr2=0x55ca20371d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036f0c0 0x55ca20371d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 -- 172.21.15.122:0/2328414804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca203729e0 con 0x55ca20377ca0 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f766ad9d640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ef0 tx=0x7f7664028720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612e7449dcf0e5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664039040 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f7664028ba0 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7664013e50 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7664028da0 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.161+0000 7f7673c2e740 10 client.21642 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ca203ee400 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7664013e50 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f7664035570 con 0x55ca20377ca0 2024-04-02T13:57:15.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 10 client.21642 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 10 client.21642 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 20 client.21642 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca203eea60 0x55ca203f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 4 client.21642 mds_command: new command op to 24460 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 --> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ca202da040 con 0x55ca203eea60 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f766a59c640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca203eea60 0x55ca203f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f766a59c640 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca203eea60 0x55ca203f0e50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2dd0b7198c07ac2a server_cookie=1e699aa0d4a77663 in_seq=0 out_seq=0 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.162+0000 7f7651ffb640 10 client.21642 ms_handle_connect on v2:172.21.15.188:6836/391935088 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7651ffb640 1 -- 172.21.15.122:0/2328414804 <== mds.3 v2:172.21.15.188:6836/391935088 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ca202da040 con 0x55ca203eea60 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7651ffb640 10 client.21642 handle_command_reply: tid=0 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 client.21642 shutdown 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 msgr2=0x55ca203722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f7664009ef0 tx=0x7f7664028720 comp rx=0 tx=0).stop 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 shutdown_connections 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:6836/391935088,v1:172.21.15.188:6837/391935088] conn(0x55ca203eea60 0x55ca203f0e50 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ca20377ca0 0x55ca203722a0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55ca2036f0c0 0x55ca20371d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 --2- 172.21.15.122:0/2328414804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca2036e7a0 0x55ca203800d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 >> 172.21.15.122:0/2328414804 conn(0x55ca20368ed0 msgr2=0x55ca20367f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7651ffb640 0 client.21642 ms_handle_reset on v2:172.21.15.188:6836/391935088 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 shutdown_connections 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 1 -- 172.21.15.122:0/2328414804 wait complete. 2024-04-02T13:57:15.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7673c2e740 20 client.21642 trim_cache size 0 max 16384 2024-04-02T13:57:15.181 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 msgr2=0x7f76540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 0x7f76540a61e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7664005fd0 tx=0x7f7664004a40 comp rx=0 tx=0).stop 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 msgr2=0x7f766c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f765c00e9f0 tx=0x7f765c00eeb0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 shutdown_connections 2024-04-02T13:57:15.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f76540a3d20 0x7f76540a61e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f766c1069c0 0x7f766c19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f766c101370 0x7f766c1988f0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 --2- 172.21.15.122:0/2667937645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f766c1009b0 0x7f766c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 >> 172.21.15.122:0/2667937645 conn(0x7f766c076d80 msgr2=0x7f766c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.163+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 shutdown_connections 2024-04-02T13:57:15.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.164+0000 7f7670f67640 1 -- 172.21.15.122:0/2667937645 wait complete. 2024-04-02T13:57:15.183 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:14 smithi122 ceph-mon[32114]: from='client.21582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:15.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-02T13:57:15.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:14 smithi187 ceph-mon[36064]: from='client.21582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:15.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:14 smithi188 ceph-mon[36117]: from='client.21582 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-02T13:57:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 -- 172.21.15.122:0/2479111353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00101330 msgr2=0x7f4a0010d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2479111353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00101330 0x7f4a0010d840 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009cd0 tx=0x7f49f802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 -- 172.21.15.122:0/2479111353 shutdown_connections 2024-04-02T13:57:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2479111353 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00101330 0x7f4a0010d840 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2479111353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00100970 0x7f4a00100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2479111353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00106980 0x7f4a00106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 -- 172.21.15.122:0/2479111353 >> 172.21.15.122:0/2479111353 conn(0x7f4a00077b00 msgr2=0x7f4a00077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 -- 172.21.15.122:0/2479111353 shutdown_connections 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.285+0000 7f4a06e55640 1 -- 172.21.15.122:0/2479111353 wait complete. 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 Processor -- start 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 -- start start 2024-04-02T13:57:15.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 0x7f4a001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00101330 0x7f4a00198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00110d90 con 0x7f4a00106980 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00110c10 con 0x7f4a00100970 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a06e55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a00110f10 con 0x7f4a00101330 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a053f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a053f2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54702/0 (socket says 172.21.15.122:54702) 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a053f2640 1 -- 172.21.15.122:0/2281605274 learned_addr learned my addr 172.21.15.122:0/2281605274 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 0x7f4a001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.286+0000 7f49fffff640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00101330 0x7f4a00198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 -- 172.21.15.122:0/2281605274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00101330 msgr2=0x7f4a00198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00101330 0x7f4a00198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 -- 172.21.15.122:0/2281605274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 msgr2=0x7f4a001983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 0x7f4a001983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 -- 172.21.15.122:0/2281605274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0019d3d0 con 0x7f4a00106980 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 0x7f4a001983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a053f2640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f49f80348f0 tx=0x7f49f8032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f49fdffb640 1 -- 172.21.15.122:0/2281605274 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8041020 con 0x7f4a00106980 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f49fdffb640 1 -- 172.21.15.122:0/2281605274 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f8032e80 con 0x7f4a00106980 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0019d660 con 0x7f4a00106980 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f49fdffb640 1 -- 172.21.15.122:0/2281605274 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8039040 con 0x7f4a00106980 2024-04-02T13:57:15.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.287+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a001a4f50 con 0x7f4a00106980 2024-04-02T13:57:15.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.289+0000 7f49fdffb640 1 -- 172.21.15.122:0/2281605274 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f49f8044020 con 0x7f4a00106980 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.289+0000 7f4a09a41740 1 Processor -- start 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 -- start start 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b66b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b670c0 0x562e25b6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562e25aca7b0 con 0x562e25b6fca0 2024-04-02T13:57:15.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e25a69570 con 0x562e25b667a0 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a09a41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e25ac6760 con 0x562e25b670c0 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fdffb640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 0x7f49e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54710/0 (socket says 172.21.15.122:54710) 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 -- 172.21.15.122:0/1148460659 learned_addr learned my addr 172.21.15.122:0/1148460659 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a053f2640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 0x7f49e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fdffb640 1 -- 172.21.15.122:0/2281605274 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f49f80f2840 con 0x7f4a00106980 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b670c0 0x562e25b6f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b66b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 -- 172.21.15.122:0/1148460659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b670c0 msgr2=0x562e25b6f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b670c0 0x562e25b6f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 -- 172.21.15.122:0/1148460659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 msgr2=0x562e25b66b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b66b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 -- 172.21.15.122:0/1148460659 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e257de3f0 con 0x562e25b6fca0 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f4a053f2640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 0x7f49e80a61e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49f80efd10 tx=0x7f49f8009900 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.290+0000 7f49fffff640 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009fd0 tx=0x7f49f40096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a616907591e1e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f49e6ffd640 1 -- 172.21.15.122:0/1148460659 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004030 con 0x562e25b6fca0 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f49e6ffd640 1 -- 172.21.15.122:0/1148460659 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f4005500 con 0x562e25b6fca0 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f49e6ffd640 1 -- 172.21.15.122:0/1148460659 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4004d70 con 0x562e25b6fca0 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 -- 172.21.15.122:0/1148460659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 msgr2=0x562e25b72090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009fd0 tx=0x7f49f40096e0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 -- 172.21.15.122:0/1148460659 shutdown_connections 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b6fca0 0x562e25b72090 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b670c0 0x562e25b6f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 --2- 172.21.15.122:0/1148460659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b66b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 -- 172.21.15.122:0/1148460659 >> 172.21.15.122:0/1148460659 conn(0x562e25b60ed0 msgr2=0x562e25b612b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 -- 172.21.15.122:0/1148460659 shutdown_connections 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.291+0000 7f4a09a41740 1 -- 172.21.15.122:0/1148460659 wait complete. 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 -- start start 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 Processor -- start 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562e25b72a90 con 0x562e25b670c0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x562e25a85e80 con 0x562e25b667a0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562e25b72c10 con 0x562e25b6fca0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a09a41740 10 client.0 _ll_register_callbacks cb 0x562e25ac1aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54726/0 (socket says 172.21.15.122:54726) 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 -- 172.21.15.122:0/4257983198 learned_addr learned my addr 172.21.15.122:0/4257983198 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f49fffff640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 -- 172.21.15.122:0/4257983198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 msgr2=0x562e25b6d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 -- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 msgr2=0x562e25b6f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b6f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f4a04bf1640 1 -- 172.21.15.122:0/4257983198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e258a87d0 con 0x562e25b670c0 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.292+0000 7f49fffff640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f49f0012700 tx=0x7f49f0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab29bd0655cdb424 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f0004260 con 0x562e25b670c0 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f49f0004400 con 0x562e25b670c0 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f4005500 con 0x562e25b670c0 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f49f00119a0 con 0x562e25b670c0 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 10 client.21654 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562e25b7ebd0 con 0x562e25b670c0 2024-04-02T13:57:15.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f4005500 con 0x562e25b670c0 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f49f000f440 con 0x562e25b670c0 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 10 client.21654 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 10 client.21654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 20 client.21654 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25b800e0 0x562e25b824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 4 client.21654 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.293+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e25ad7cf0 con 0x562e25b800e0 2024-04-02T13:57:15.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.294+0000 7f49fffff640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25b800e0 0x562e25b824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.294+0000 7f49dffff640 10 client.21654 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.294+0000 7f49fffff640 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25b800e0 0x562e25b824d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed81cc3f930d4e9d server_cookie=37ee413f1642d5f in_seq=0 out_seq=0 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f49dffff640 1 -- 172.21.15.122:0/4257983198 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562e25ad7cf0 con 0x562e25b800e0 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f49dffff640 10 client.21654 handle_command_reply: tid=0 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 client.21654 shutdown 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 msgr2=0x562e25b6cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f49f0012700 tx=0x7f49f0012bc0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 shutdown_connections 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b6cae0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25b800e0 0x562e25b824d0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4257983198 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b667a0 0x562e25b6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f49dffff640 0 client.21654 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 >> 172.21.15.122:0/4257983198 conn(0x562e25b60ed0 msgr2=0x562e25b62c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 shutdown_connections 2024-04-02T13:57:15.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 1 -- 172.21.15.122:0/4257983198 wait complete. 2024-04-02T13:57:15.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.295+0000 7f4a09a41740 20 client.21654 trim_cache size 0 max 16384 2024-04-02T13:57:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 Processor -- start 2024-04-02T13:57:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 -- start start 2024-04-02T13:57:15.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b670c0 0x562e25b6f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001af190 con 0x562e25b667a0 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b0cd0 con 0x562e25b670c0 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f4a09a41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001af860 con 0x562e25b6fca0 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54740/0 (socket says 172.21.15.122:54740) 2024-04-02T13:57:15.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 -- 172.21.15.122:0/4062498175 learned_addr learned my addr 172.21.15.122:0/4062498175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 -- 172.21.15.122:0/4062498175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 msgr2=0x562e25b6f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 -- 172.21.15.122:0/4062498175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b670c0 msgr2=0x562e25b6f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b670c0 0x562e25b6f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 -- 172.21.15.122:0/4062498175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e25b6b030 con 0x562e25b667a0 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49fffff640 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f49f4013e00 tx=0x7f49f40048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef9749e1aca3956 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49e67fc640 1 -- 172.21.15.122:0/4062498175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4030070 con 0x562e25b667a0 2024-04-02T13:57:15.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49e67fc640 1 -- 172.21.15.122:0/4062498175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f402b8d0 con 0x562e25b667a0 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.309+0000 7f49e67fc640 1 -- 172.21.15.122:0/4062498175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f401fb90 con 0x562e25b667a0 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- 172.21.15.122:0/4062498175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 msgr2=0x562e25b7b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f49f4013e00 tx=0x7f49f40048d0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- 172.21.15.122:0/4062498175 shutdown_connections 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b6fca0 0x562e25b6f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b670c0 0x562e25b6f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 --2- 172.21.15.122:0/4062498175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b667a0 0x562e25b7b980 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- 172.21.15.122:0/4062498175 >> 172.21.15.122:0/4062498175 conn(0x562e25b60ed0 msgr2=0x562e25b6b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- 172.21.15.122:0/4062498175 shutdown_connections 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- 172.21.15.122:0/4062498175 wait complete. 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 -- start start 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.310+0000 7f4a09a41740 1 Processor -- start 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b667a0 0x562e25b69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b6fca0 0x562e25b6a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001b0600 con 0x562e25b670c0 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001aff30 con 0x562e25b6fca0 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a001ad190 con 0x562e25b667a0 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 10 client.0 _ll_register_callbacks cb 0x7f49f00097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54742/0 (socket says 172.21.15.122:54742) 2024-04-02T13:57:15.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 -- 172.21.15.122:0/318626932 learned_addr learned my addr 172.21.15.122:0/318626932 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f49fffff640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b6fca0 0x562e25b6a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f49fffff640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b667a0 0x562e25b69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 -- 172.21.15.122:0/318626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b667a0 msgr2=0x562e25b69710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b667a0 0x562e25b69710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 -- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b6fca0 msgr2=0x562e25b6a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b6fca0 0x562e25b6a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 -- 172.21.15.122:0/318626932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e25b75fd0 con 0x562e25b670c0 2024-04-02T13:57:15.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a04bf1640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f49f000f8e0 tx=0x7f49f0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4fed0e1f6aba7a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f49dffff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f002d070 con 0x562e25b670c0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f402b8d0 con 0x562e25b670c0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.311+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7f49f00052d0 con 0x562e25b670c0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f49f0039050 con 0x562e25b670c0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 10 client.21660 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x562e25be68b0 con 0x562e25b670c0 2024-04-02T13:57:15.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49f402b8d0 con 0x562e25b670c0 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f49f0003ae0 con 0x562e25b670c0 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 10 client.21660 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 10 client.21660 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 20 client.21660 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25be7030 0x562e25be9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 4 client.21660 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:15.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 --> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e25ad2040 con 0x562e25be7030 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.312+0000 7f49fffff640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25be7030 0x562e25be9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f49dffff640 10 client.21660 ms_handle_connect on v2:172.21.15.187:6836/453335780 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f49fffff640 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25be7030 0x562e25be9450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=345dde9d2c0d6965 server_cookie=752ae2e55082c745 in_seq=0 out_seq=0 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f49dffff640 1 -- 172.21.15.122:0/318626932 <== mds.2 v2:172.21.15.187:6836/453335780 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e25ad2040 con 0x562e25be7030 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f49dffff640 10 client.21660 handle_command_reply: tid=0 2024-04-02T13:57:15.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 client.21660 shutdown 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 msgr2=0x562e25b69c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f49f000f8e0 tx=0x7f49f0009f80 comp rx=0 tx=0).stop 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 shutdown_connections 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:6836/453335780,v1:172.21.15.187:6837/453335780] conn(0x562e25be7030 0x562e25be9450 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x562e25b6fca0 0x562e25b6a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f49dffff640 0 client.21660 ms_handle_reset on v2:172.21.15.187:6836/453335780 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562e25b670c0 0x562e25b69c50 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 --2- 172.21.15.122:0/318626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e25b667a0 0x562e25b69710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 >> 172.21.15.122:0/318626932 conn(0x562e25b60ed0 msgr2=0x562e25b62300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 shutdown_connections 2024-04-02T13:57:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 1 -- 172.21.15.122:0/318626932 wait complete. 2024-04-02T13:57:15.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.313+0000 7f4a09a41740 20 client.21660 trim_cache size 0 max 16384 2024-04-02T13:57:15.329 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 msgr2=0x7f49e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 0x7f49e80a61e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f49f80efd10 tx=0x7f49f8009900 comp rx=0 tx=0).stop 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 msgr2=0x7f4a0019cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f49f80348f0 tx=0x7f49f8032c90 comp rx=0 tx=0).stop 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 shutdown_connections 2024-04-02T13:57:15.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f49e80a3d20 0x7f49e80a61e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a00106980 0x7f4a0019cca0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a00101330 0x7f4a00198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 --2- 172.21.15.122:0/2281605274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a00100970 0x7f4a001983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 >> 172.21.15.122:0/2281605274 conn(0x7f4a00077b00 msgr2=0x7f4a00069000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 shutdown_connections 2024-04-02T13:57:15.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.314+0000 7f4a06e55640 1 -- 172.21.15.122:0/2281605274 wait complete. 2024-04-02T13:57:15.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-02T13:57:15.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 -- 172.21.15.122:0/3431029980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf4810c110 msgr2=0x7fcf4810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 --2- 172.21.15.122:0/3431029980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf4810c110 0x7fcf4810e540 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fcf40009ca0 tx=0x7fcf4002c180 comp rx=0 tx=0).stop 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 -- 172.21.15.122:0/3431029980 shutdown_connections 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 --2- 172.21.15.122:0/3431029980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf4810c110 0x7fcf4810e540 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 --2- 172.21.15.122:0/3431029980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48103660 0x7fcf4810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 --2- 172.21.15.122:0/3431029980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf48102d40 0x7fcf48103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 -- 172.21.15.122:0/3431029980 >> 172.21.15.122:0/3431029980 conn(0x7fcf48076d80 msgr2=0x7fcf48077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 -- 172.21.15.122:0/3431029980 shutdown_connections 2024-04-02T13:57:15.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.433+0000 7fcf4e560640 1 -- 172.21.15.122:0/3431029980 wait complete. 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 Processor -- start 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 -- start start 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48102d40 0x7fcf481983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4810c110 0x7fcf4819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48111ee0 con 0x7fcf48103660 2024-04-02T13:57:15.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48111d60 con 0x7fcf48102d40 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf4e560640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48112060 con 0x7fcf4810c110 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54758/0 (socket says 172.21.15.122:54758) 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf477fe640 1 -- 172.21.15.122:0/2597341011 learned_addr learned my addr 172.21.15.122:0/2597341011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.434+0000 7fcf47fff640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48102d40 0x7fcf481983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4810c110 0x7fcf4819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 -- 172.21.15.122:0/2597341011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4810c110 msgr2=0x7fcf4819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4810c110 0x7fcf4819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 -- 172.21.15.122:0/2597341011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48102d40 msgr2=0x7fcf481983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48102d40 0x7fcf481983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 -- 172.21.15.122:0/2597341011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4819d3b0 con 0x7fcf48103660 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf477fe640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c890 tx=0x7fcf3800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf457fa640 1 -- 172.21.15.122:0/2597341011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38004220 con 0x7fcf48103660 2024-04-02T13:57:15.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4819d6a0 con 0x7fcf48103660 2024-04-02T13:57:15.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf457fa640 1 -- 172.21.15.122:0/2597341011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf380043c0 con 0x7fcf48103660 2024-04-02T13:57:15.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf457fa640 1 -- 172.21.15.122:0/2597341011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38004a00 con 0x7fcf48103660 2024-04-02T13:57:15.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.435+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf481a4f30 con 0x7fcf48103660 2024-04-02T13:57:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.437+0000 7fcf457fa640 1 -- 172.21.15.122:0/2597341011 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcf380079b0 con 0x7fcf48103660 2024-04-02T13:57:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.437+0000 7fcf51228740 1 Processor -- start 2024-04-02T13:57:15.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.437+0000 7fcf51228740 1 -- start start 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce7fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce8b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e65cde37b0 con 0x55e65ce800c0 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e65cd82570 con 0x55e65ce88ca0 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf51228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e65cddf760 con 0x55e65ce7f7a0 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54772/0 (socket says 172.21.15.122:54772) 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 -- 172.21.15.122:0/1385958111 learned_addr learned my addr 172.21.15.122:0/1385958111 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf477fe640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce8b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce7fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf457fa640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 -- 172.21.15.122:0/1385958111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 msgr2=0x55e65ce7fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce7fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 -- 172.21.15.122:0/1385958111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 msgr2=0x55e65ce8b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce8b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 -- 172.21.15.122:0/1385958111 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65caf73f0 con 0x55e65ce800c0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf477fe640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce8b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf457fa640 1 -- 172.21.15.122:0/2597341011 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fcf380d08e0 con 0x7fcf48103660 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf47fff640 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a70 tx=0x7fcf3c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3182265de978c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf2e7fc640 1 -- 172.21.15.122:0/1385958111 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c017070 con 0x55e65ce800c0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf2e7fc640 1 -- 172.21.15.122:0/1385958111 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c013470 con 0x55e65ce800c0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.438+0000 7fcf2e7fc640 1 -- 172.21.15.122:0/1385958111 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c012630 con 0x55e65ce800c0 2024-04-02T13:57:15.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- 172.21.15.122:0/1385958111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 msgr2=0x55e65ce88760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 0x7fcf300a61e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006010 tx=0x7fcf400095c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a70 tx=0x7fcf3c009f30 comp rx=0 tx=0).stop 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- 172.21.15.122:0/1385958111 shutdown_connections 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce8b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce88760 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 --2- 172.21.15.122:0/1385958111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce7fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- 172.21.15.122:0/1385958111 >> 172.21.15.122:0/1385958111 conn(0x55e65ce79ed0 msgr2=0x55e65ce7a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- 172.21.15.122:0/1385958111 shutdown_connections 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- 172.21.15.122:0/1385958111 wait complete. 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.439+0000 7fcf51228740 1 -- start start 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 Processor -- start 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce85a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e65ce8ba90 con 0x55e65ce88ca0 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e65cd9ee80 con 0x55e65ce800c0 2024-04-02T13:57:15.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e65ce8bc10 con 0x55e65ce7f7a0 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf51228740 10 client.0 _ll_register_callbacks cb 0x55e65cddaaa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54780/0 (socket says 172.21.15.122:54780) 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 -- 172.21.15.122:0/3649253498 learned_addr learned my addr 172.21.15.122:0/3649253498 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf477fe640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce85a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 -- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 msgr2=0x55e65ce886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 -- 172.21.15.122:0/3649253498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 msgr2=0x55e65ce85a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce85a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 -- 172.21.15.122:0/3649253498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65cbc17d0 con 0x55e65ce88ca0 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf477fe640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce85a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fcf40036000 tx=0x7fcf4002c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3456880ae8a1ad2a server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.440+0000 7fcf277fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40039040 con 0x55e65ce88ca0 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fcf4003f990 con 0x55e65ce88ca0 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c013470 con 0x55e65ce88ca0 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fcf40059050 con 0x55e65ce88ca0 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 10 client.21672 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e65ce97ac0 con 0x55e65ce88ca0 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c013470 con 0x55e65ce88ca0 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fcf40049070 con 0x55e65ce88ca0 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 10 client.21672 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 10 client.21672 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 20 client.21672 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ce98f80 0x55e65ce9b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 4 client.21672 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:15.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e65cdf0cf0 con 0x55e65ce98f80 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.441+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ce98f80 0x55e65ce9b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.442+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ce98f80 0x55e65ce9b3f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=931413edca0a60 server_cookie=e513eb34e92197c9 in_seq=0 out_seq=0 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.442+0000 7fcf277fe640 10 client.21672 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.442+0000 7fcf277fe640 1 -- 172.21.15.122:0/3649253498 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e65cdf0cf0 con 0x55e65ce98f80 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.442+0000 7fcf277fe640 10 client.21672 handle_command_reply: tid=0 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 client.21672 shutdown 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 msgr2=0x55e65ce86050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fcf40036000 tx=0x7fcf4002c180 comp rx=0 tx=0).stop 2024-04-02T13:57:15.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 shutdown_connections 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ce98f80 0x55e65ce9b3f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce88ca0 0x55e65ce86050 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce85a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf277fe640 0 client.21672 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 --2- 172.21.15.122:0/3649253498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 >> 172.21.15.122:0/3649253498 conn(0x55e65ce79ed0 msgr2=0x55e65ce7bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 shutdown_connections 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 1 -- 172.21.15.122:0/3649253498 wait complete. 2024-04-02T13:57:15.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.443+0000 7fcf51228740 20 client.21672 trim_cache size 0 max 16384 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 Processor -- start 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 -- start start 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce88170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce88ca0 0x55e65ce886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481af2b0 con 0x55e65ce7f7a0 2024-04-02T13:57:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481b0df0 con 0x55e65ce800c0 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf51228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481af980 con 0x55e65ce88ca0 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54794/0 (socket says 172.21.15.122:54794) 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 -- 172.21.15.122:0/3996093583 learned_addr learned my addr 172.21.15.122:0/3996093583 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf477fe640 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce88170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 -- 172.21.15.122:0/3996093583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce88ca0 msgr2=0x55e65ce886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce88ca0 0x55e65ce886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 -- 172.21.15.122:0/3996093583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 msgr2=0x55e65ce88170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce88170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf47fff640 1 -- 172.21.15.122:0/3996093583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e65ce7fef0 con 0x55e65ce7f7a0 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.456+0000 7fcf477fe640 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce88170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf47fff640 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099c0 tx=0x7fcf3c009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a98ebe1f0ea38db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf2dffb640 1 -- 172.21.15.122:0/3996093583 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c017040 con 0x55e65ce7f7a0 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf2dffb640 1 -- 172.21.15.122:0/3996093583 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c00ed40 con 0x55e65ce7f7a0 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf2dffb640 1 -- 172.21.15.122:0/3996093583 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c027550 con 0x55e65ce7f7a0 2024-04-02T13:57:15.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 -- 172.21.15.122:0/3996093583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 msgr2=0x55e65ce94950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0099c0 tx=0x7fcf3c009110 comp rx=0 tx=0).stop 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 -- 172.21.15.122:0/3996093583 shutdown_connections 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce88ca0 0x55e65ce886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce800c0 0x55e65ce88170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 --2- 172.21.15.122:0/3996093583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce7f7a0 0x55e65ce94950 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 -- 172.21.15.122:0/3996093583 >> 172.21.15.122:0/3996093583 conn(0x55e65ce79ed0 msgr2=0x55e65ce85a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 -- 172.21.15.122:0/3996093583 shutdown_connections 2024-04-02T13:57:15.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.457+0000 7fcf51228740 1 -- 172.21.15.122:0/3996093583 wait complete. 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 -- start start 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 Processor -- start 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce81d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481b0720 con 0x55e65ce800c0 2024-04-02T13:57:15.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481b0050 con 0x55e65ce88ca0 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481ad2b0 con 0x55e65ce7f7a0 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf51228740 10 client.0 _ll_register_callbacks cb 0x7fcf40037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54808/0 (socket says 172.21.15.122:54808) 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 -- 172.21.15.122:0/4230843278 learned_addr learned my addr 172.21.15.122:0/4230843278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 -- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 msgr2=0x55e65ce81d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf47fff640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce81d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce81d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 -- 172.21.15.122:0/4230843278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 msgr2=0x55e65ce827d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce827d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf477fe640 1 -- 172.21.15.122:0/4230843278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e65ce8ff60 con 0x55e65ce800c0 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf4cafd640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.458+0000 7fcf47fff640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf477fe640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fcf380ce1f0 tx=0x7fcf38005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cb408e76cd0495 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf380d6040 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fcf380d0460 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf3c00ed40 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fcf3801d040 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 10 client.21678 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55e65ce91230 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf3c00ed40 con 0x55e65ce800c0 2024-04-02T13:57:15.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fcf380b5bc0 con 0x55e65ce800c0 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 10 client.21678 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 10 client.21678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 20 client.21678 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ceff690 0x55e65cf01ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 4 client.21678 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 --> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e65cdeb040 con 0x55e65ceff690 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.459+0000 7fcf47fff640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ceff690 0x55e65cf01ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf277fe640 10 client.21678 ms_handle_connect on v2:172.21.15.188:6838/180395175 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf47fff640 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ceff690 0x55e65cf01ab0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee8dd62089964dba server_cookie=237b046734d0a277 in_seq=0 out_seq=0 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf277fe640 1 -- 172.21.15.122:0/4230843278 <== mds.1 v2:172.21.15.188:6838/180395175 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e65cdeb040 con 0x55e65ceff690 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf277fe640 10 client.21678 handle_command_reply: tid=0 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 client.21678 shutdown 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 msgr2=0x55e65ce82290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fcf380ce1f0 tx=0x7fcf38005700 comp rx=0 tx=0).stop 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 shutdown_connections 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:6838/180395175,v1:172.21.15.188:6839/180395175] conn(0x55e65ceff690 0x55e65cf01ab0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e65ce88ca0 0x55e65ce827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e65ce800c0 0x55e65ce82290 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 --2- 172.21.15.122:0/4230843278 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e65ce7f7a0 0x55e65ce81d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf277fe640 0 client.21678 ms_handle_reset on v2:172.21.15.188:6838/180395175 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 >> 172.21.15.122:0/4230843278 conn(0x55e65ce79ed0 msgr2=0x55e65ce84250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 shutdown_connections 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.460+0000 7fcf51228740 1 -- 172.21.15.122:0/4230843278 wait complete. 2024-04-02T13:57:15.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf51228740 20 client.21678 trim_cache size 0 max 16384 2024-04-02T13:57:15.479 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:57:15.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 msgr2=0x7fcf300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 0x7fcf300a61e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf40006010 tx=0x7fcf400095c0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 msgr2=0x7fcf48198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcf3800c890 tx=0x7fcf3800cc90 comp rx=0 tx=0).stop 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 shutdown_connections 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcf300a3d20 0x7fcf300a61e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4810c110 0x7fcf4819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf48103660 0x7fcf48198900 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 --2- 172.21.15.122:0/2597341011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf48102d40 0x7fcf481983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 >> 172.21.15.122:0/2597341011 conn(0x7fcf48076d80 msgr2=0x7fcf480fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 shutdown_connections 2024-04-02T13:57:15.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.461+0000 7fcf4e560640 1 -- 172.21.15.122:0/2597341011 wait complete. 2024-04-02T13:57:15.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-02T13:57:15.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/993915427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4107b50 msgr2=0x7fa7c4107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/993915427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4107b50 0x7fa7c4107f30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009cd0 tx=0x7fa7bc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/993915427 shutdown_connections 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/993915427 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c40696c0 0x7fa7c410e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/993915427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c4068d00 0x7fa7c4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/993915427 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4107b50 0x7fa7c4107f30 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/993915427 >> 172.21.15.122:0/993915427 conn(0x7fa7c4077b00 msgr2=0x7fa7c4077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.583+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/993915427 shutdown_connections 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/993915427 wait complete. 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 Processor -- start 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 -- start start 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c40696c0 0x7fa7c4198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 0x7fa7c419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4111ea0 con 0x7fa7c4068d00 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4111d20 con 0x7fa7c4107b50 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cc5c8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c4112020 con 0x7fa7c40696c0 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54826/0 (socket says 172.21.15.122:54826) 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7ca364640 1 -- 172.21.15.122:0/2402753312 learned_addr learned my addr 172.21.15.122:0/2402753312 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.584+0000 7fa7cab65640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 0x7fa7c419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c40696c0 0x7fa7c4198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 -- 172.21.15.122:0/2402753312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c40696c0 msgr2=0x7fa7c4198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c40696c0 0x7fa7c4198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 -- 172.21.15.122:0/2402753312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 msgr2=0x7fa7c419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 0x7fa7c419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 -- 172.21.15.122:0/2402753312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c419d310 con 0x7fa7c4068d00 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7cab65640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 0x7fa7c419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02c700 tx=0x7fa7bc0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7bb7fe640 1 -- 172.21.15.122:0/2402753312 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc042020 con 0x7fa7c4068d00 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7bb7fe640 1 -- 172.21.15.122:0/2402753312 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7bc0346c0 con 0x7fa7c4068d00 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c419d5a0 con 0x7fa7c4068d00 2024-04-02T13:57:15.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7bb7fe640 1 -- 172.21.15.122:0/2402753312 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc039040 con 0x7fa7c4068d00 2024-04-02T13:57:15.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.585+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c41a4e90 con 0x7fa7c4068d00 2024-04-02T13:57:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.587+0000 7fa7bb7fe640 1 -- 172.21.15.122:0/2402753312 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa7bc045020 con 0x7fa7c4068d00 2024-04-02T13:57:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 Processor -- start 2024-04-02T13:57:15.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 -- start start 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7bb7fe640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 0x7fa7b00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x5645528edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645528517b0 con 0x5645528f6ca0 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x5645527f0570 con 0x5645528ee0c0 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56455284d760 con 0x5645528ed7a0 2024-04-02T13:57:15.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cab65640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 0x7fa7b00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7bb7fe640 1 -- 172.21.15.122:0/2402753312 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa7bc0f2bf0 con 0x7fa7c4068d00 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cab65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cab65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54838/0 (socket says 172.21.15.122:54838) 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7cab65640 1 -- 172.21.15.122:0/3174127597 learned_addr learned my addr 172.21.15.122:0/3174127597 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.588+0000 7fa7ca364640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x5645528edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3174127597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 msgr2=0x5645528edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x5645528edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3174127597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 msgr2=0x5645528f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3174127597 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645525653f0 con 0x5645528ee0c0 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7cab65640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7cab65640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 0x7fa7b00a6260 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00139d0 tx=0x7fa7c000bbb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f6760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a90 tx=0x7fa7b4009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2682dc7e98a922de server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7b8bfe640 1 -- 172.21.15.122:0/3174127597 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4004230 con 0x5645528ee0c0 2024-04-02T13:57:15.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7b8bfe640 1 -- 172.21.15.122:0/3174127597 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b40043d0 con 0x5645528ee0c0 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.589+0000 7fa7b8bfe640 1 -- 172.21.15.122:0/3174127597 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4011470 con 0x5645528ee0c0 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3174127597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 msgr2=0x5645528f6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f6760 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a90 tx=0x7fa7b4009e90 comp rx=0 tx=0).stop 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3174127597 shutdown_connections 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528f6ca0 0x5645528f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f6760 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3174127597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x5645528edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3174127597 >> 172.21.15.122:0/3174127597 conn(0x5645528e7ed0 msgr2=0x5645528e82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3174127597 shutdown_connections 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3174127597 wait complete. 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 -- start start 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.590+0000 7fa7cf1ad740 1 Processor -- start 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ee0c0 0x5645528f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5645528fdea0 con 0x5645528ed7a0 2024-04-02T13:57:15.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x56455280ce80 con 0x5645528f6ca0 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645528fe020 con 0x5645528ee0c0 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 10 client.0 _ll_register_callbacks cb 0x564552848aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54850/0 (socket says 172.21.15.122:54850) 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 -- 172.21.15.122:0/708872175 learned_addr learned my addr 172.21.15.122:0/708872175 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cab65640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 -- 172.21.15.122:0/708872175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ee0c0 msgr2=0x5645528f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ee0c0 0x5645528f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 -- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 msgr2=0x5645528f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 -- 172.21.15.122:0/708872175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56455262f7d0 con 0x5645528ed7a0 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cab65640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7ca364640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0be560 tx=0x7fa7bc0be460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d06ccb25eb6e286 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0c2c10 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa7bc0e0210 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.591+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b40043d0 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc0c74d0 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 10 client.21690 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x564552909e80 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b40043d0 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa7bc0c7040 con 0x5645528ed7a0 2024-04-02T13:57:15.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 10 client.21690 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 10 client.21690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 20 client.21690 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56455290b3c0 0x56455290d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 4 client.21690 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56455285ecf0 con 0x56455290b3c0 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.592+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56455290b3c0 0x56455290d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7adffb640 10 client.21690 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:15.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56455290b3c0 0x56455290d850 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ff17af52dcaeda server_cookie=aafcb51f3dbdf48a in_seq=0 out_seq=0 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7adffb640 1 -- 172.21.15.122:0/708872175 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56455285ecf0 con 0x56455290b3c0 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7adffb640 10 client.21690 handle_command_reply: tid=0 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7cf1ad740 1 client.21690 shutdown 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 msgr2=0x5645528f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0be560 tx=0x7fa7bc0be460 comp rx=0 tx=0).stop 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 shutdown_connections 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.593+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x56455290b3c0 0x56455290d850 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7adffb640 0 client.21690 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ee0c0 0x5645528f39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/708872175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f6650 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 >> 172.21.15.122:0/708872175 conn(0x5645528e7ed0 msgr2=0x5645528e9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 shutdown_connections 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/708872175 wait complete. 2024-04-02T13:57:15.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.594+0000 7fa7cf1ad740 20 client.21690 trim_cache size 0 max 16384 2024-04-02T13:57:15.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 Processor -- start 2024-04-02T13:57:15.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 -- start start 2024-04-02T13:57:15.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x564552906d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ace20 con 0x5645528ee0c0 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ae960 con 0x5645528f6ca0 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ad4f0 con 0x5645528ed7a0 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54862/0 (socket says 172.21.15.122:54862) 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3023533212 learned_addr learned my addr 172.21.15.122:0/3023533212 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3023533212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 msgr2=0x564552906d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x564552906d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3023533212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 msgr2=0x5645528f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 -- 172.21.15.122:0/3023533212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645528f42f0 con 0x5645528ee0c0 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.607+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x5645528ed270 tx=0x7fa7b4003b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d884828465d3db server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7affff640 1 -- 172.21.15.122:0/3023533212 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40035f0 con 0x5645528ee0c0 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7affff640 1 -- 172.21.15.122:0/3023533212 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4003790 con 0x5645528ee0c0 2024-04-02T13:57:15.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7affff640 1 -- 172.21.15.122:0/3023533212 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40052a0 con 0x5645528ee0c0 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3023533212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 msgr2=0x5645528f6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x5645528ed270 tx=0x7fa7b4003b00 comp rx=0 tx=0).stop 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3023533212 shutdown_connections 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528f6ca0 0x5645528f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ee0c0 0x5645528f6110 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/3023533212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528ed7a0 0x564552906d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3023533212 >> 172.21.15.122:0/3023533212 conn(0x5645528e7ed0 msgr2=0x5645528f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3023533212 shutdown_connections 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/3023533212 wait complete. 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 -- start start 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.608+0000 7fa7cf1ad740 1 Processor -- start 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528f6ca0 0x5645528f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41ae290 con 0x5645528ed7a0 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41adbc0 con 0x5645528ee0c0 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c41aae20 con 0x5645528f6ca0 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 10 client.0 _ll_register_callbacks cb 0x7fa7bc0be630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54878/0 (socket says 172.21.15.122:54878) 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 -- 172.21.15.122:0/2110720586 learned_addr learned my addr 172.21.15.122:0/2110720586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:15.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 -- 172.21.15.122:0/2110720586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528f6ca0 msgr2=0x5645528f8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528f6ca0 0x5645528f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 -- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 msgr2=0x5645528f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 -- 172.21.15.122:0/2110720586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564552901740 con 0x5645528ed7a0 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7ca364640 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0be7f0 tx=0x7fa7bc0be4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2d6488f3939865 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7adffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0db070 con 0x5645528ed7a0 2024-04-02T13:57:15.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9613+0+0 (secure 0 0 0) 0x7fa7bc0a90b0 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b4003790 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa7bc0c2c10 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 10 client.21696 fetch_fsmap learned FSMap version 76 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.609+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x564552901740 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4003790 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7fa7bc004b40 con 0x5645528ed7a0 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 10 client.21696 fetch_fsmap finished waiting for FSMap version 76 2024-04-02T13:57:15.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 10 client.21696 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 20 client.21696 populate_metadata read hostname 'smithi122' 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564552972250 0x5645529746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 4 client.21696 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 --> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564552859040 con 0x564552972250 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564552972250 0x5645529746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7c9b63640 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564552972250 0x5645529746b0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3746928397606082 server_cookie=d720f2b00bb9a408 in_seq=0 out_seq=0 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.610+0000 7fa7adffb640 10 client.21696 ms_handle_connect on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7adffb640 1 -- 172.21.15.122:0/2110720586 <== mds.0 v2:172.21.15.187:6838/2503909279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564552859040 con 0x564552972250 2024-04-02T13:57:15.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7adffb640 10 client.21696 handle_command_reply: tid=0 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 client.21696 shutdown 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 msgr2=0x5645528f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0be7f0 tx=0x7fa7bc0be4d0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 shutdown_connections 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:6838/2503909279,v1:172.21.15.187:6839/2503909279] conn(0x564552972250 0x5645529746b0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645528f6ca0 0x5645528f8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7adffb640 0 client.21696 ms_handle_reset on v2:172.21.15.187:6838/2503909279 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x5645528ee0c0 0x5645528f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 --2- 172.21.15.122:0/2110720586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5645528ed7a0 0x5645528f7f00 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 >> 172.21.15.122:0/2110720586 conn(0x5645528e7ed0 msgr2=0x5645528e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 shutdown_connections 2024-04-02T13:57:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 1 -- 172.21.15.122:0/2110720586 wait complete. 2024-04-02T13:57:15.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cf1ad740 20 client.21696 trim_cache size 0 max 16384 2024-04-02T13:57:15.627 INFO:teuthology.orchestra.run.smithi122.stdout:[] 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 msgr2=0x7fa7b00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 0x7fa7b00a6260 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa7c00139d0 tx=0x7fa7c000bbb0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 msgr2=0x7fa7c4198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc02c700 tx=0x7fa7bc0324f0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 shutdown_connections 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa7b00a3da0 0x7fa7b00a6260 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa7c4107b50 0x7fa7c419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7c40696c0 0x7fa7c4198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 --2- 172.21.15.122:0/2402753312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7c4068d00 0x7fa7c4198320 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:15.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 >> 172.21.15.122:0/2402753312 conn(0x7fa7c4077b00 msgr2=0x7fa7c40fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:15.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 shutdown_connections 2024-04-02T13:57:15.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:15.611+0000 7fa7cc5c8640 1 -- 172.21.15.122:0/2402753312 wait complete. 2024-04-02T13:57:16.235 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:15 smithi187 ceph-mon[36064]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-04-02T13:57:16.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:15 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3002231069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:16.267 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:15 smithi188 ceph-mon[36117]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-04-02T13:57:16.268 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:15 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3002231069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:16.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:15 smithi122 ceph-mon[32114]: pgmap v696: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.5 MiB/s wr, 427 op/s 2024-04-02T13:57:16.322 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:15 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3002231069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:57:16.631 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-02T13:57:16.631 INFO:tasks.fwd_scrub:done joining 2024-04-02T13:57:16.631 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-02T13:57:16.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:57:16.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.843+0000 7f93f5e35640 1 -- 172.21.15.122:0/395514565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 msgr2=0x7f93f0115e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:16.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.843+0000 7f93f5e35640 1 --2- 172.21.15.122:0/395514565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 0x7f93f0115e40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009ca0 tx=0x7f93e802c120 comp rx=0 tx=0).stop 2024-04-02T13:57:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.847+0000 7f93f5e35640 1 -- 172.21.15.122:0/395514565 shutdown_connections 2024-04-02T13:57:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.847+0000 7f93f5e35640 1 --2- 172.21.15.122:0/395514565 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 0x7f93f0115e40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:16.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.847+0000 7f93f5e35640 1 --2- 172.21.15.122:0/395514565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f0113510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.847+0000 7f93f5e35640 1 --2- 172.21.15.122:0/395514565 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f010a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:16.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.848+0000 7f93f5e35640 1 -- 172.21.15.122:0/395514565 >> 172.21.15.122:0/395514565 conn(0x7f93f006cbf0 msgr2=0x7f93f006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:16.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.851+0000 7f93f5e35640 1 -- 172.21.15.122:0/395514565 shutdown_connections 2024-04-02T13:57:16.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.855+0000 7f93f5e35640 1 -- 172.21.15.122:0/395514565 wait complete. 2024-04-02T13:57:16.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.856+0000 7f93f5e35640 1 Processor -- start 2024-04-02T13:57:16.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.860+0000 7f93f5e35640 1 -- start start 2024-04-02T13:57:16.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.860+0000 7f93f5e35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93f5e35640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f01a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93f5e35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 0x7f93f019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93f5e35640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93f01184c0 con 0x7f93f010adc0 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93f5e35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f93f0118340 con 0x7f93f0113a50 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93f5e35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93f0118640 con 0x7f93f010a4a0 2024-04-02T13:57:16.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93ef7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93ef7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.122:52484/0 (socket says 172.21.15.122:52484) 2024-04-02T13:57:16.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.861+0000 7f93ef7fe640 1 -- 172.21.15.122:0/1687968279 learned_addr learned my addr 172.21.15.122:0/1687968279 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.862+0000 7f93eeffd640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f01a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.862+0000 7f93ef7fe640 1 -- 172.21.15.122:0/1687968279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 msgr2=0x7f93f019e7e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:57:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.862+0000 7f93ef7fe640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 0x7f93f019e7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:16.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.862+0000 7f93ef7fe640 1 -- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 msgr2=0x7f93f01a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.864+0000 7f93ef7fe640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f01a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.864+0000 7f93ef7fe640 1 -- 172.21.15.122:0/1687968279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f019efb0 con 0x7f93f010a4a0 2024-04-02T13:57:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.864+0000 7f93eeffd640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:57:16.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.864+0000 7f93ef7fe640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b3d0 tx=0x7f93e400b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:16.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.867+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4015020 con 0x7f93f010a4a0 2024-04-02T13:57:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.868+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f019f2a0 con 0x7f93f010a4a0 2024-04-02T13:57:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.868+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f01ab4f0 con 0x7f93f010a4a0 2024-04-02T13:57:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.868+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93e4011d10 con 0x7f93f010a4a0 2024-04-02T13:57:16.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.868+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400f2e0 con 0x7f93f010a4a0 2024-04-02T13:57:16.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.876+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f93e4018020 con 0x7f93f010a4a0 2024-04-02T13:57:16.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.876+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93dc005290 con 0x7f93f010a4a0 2024-04-02T13:57:16.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.877+0000 7f93ecff9640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 0x7f93d80aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:16.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.877+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f93e40d0250 con 0x7f93f010a4a0 2024-04-02T13:57:16.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.884+0000 7f93eeffd640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 0x7f93d80aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:16.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.886+0000 7f93eeffd640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 0x7f93d80aa7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93e00058c0 tx=0x7f93e0005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:16.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:16.889+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f93e401f020 con 0x7f93f010a4a0 2024-04-02T13:57:17.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.265+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93dc005850 con 0x7f93f010a4a0 2024-04-02T13:57:17.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.267+0000 7f93ecff9640 1 -- 172.21.15.122:0/1687968279 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+25415 (secure 0 0 0) 0x7f93e409c5b0 con 0x7f93f010a4a0 2024-04-02T13:57:17.271 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:57:17.271 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":127,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:57:16.447119+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]},{"pool":4,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 msgr2=0x7f93d80aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 0x7f93d80aa7f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f93e00058c0 tx=0x7f93e0005850 comp rx=0 tx=0).stop 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 msgr2=0x7f93f01a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f93e400b3d0 tx=0x7f93e400b890 comp rx=0 tx=0).stop 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 shutdown_connections 2024-04-02T13:57:17.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f93d80a8330 0x7f93d80aa7f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f93f0113a50 0x7f93f019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93f010adc0 0x7f93f01a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 --2- 172.21.15.122:0/1687968279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f010a4a0 0x7f93f01a4620 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 >> 172.21.15.122:0/1687968279 conn(0x7f93f006cbf0 msgr2=0x7f93f00707b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 shutdown_connections 2024-04-02T13:57:17.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.271+0000 7f93f5e35640 1 -- 172.21.15.122:0/1687968279 wait complete. 2024-04-02T13:57:17.294 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:57:17.295 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:57:17.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.490+0000 7fd7d4847640 1 -- 172.21.15.122:0/949728092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0100970 msgr2=0x7fd7d0100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.490+0000 7fd7d4847640 1 --2- 172.21.15.122:0/949728092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0100970 0x7fd7d0100df0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009900 tx=0x7fd7c402bc40 comp rx=0 tx=0).stop 2024-04-02T13:57:17.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.494+0000 7fd7d4847640 1 -- 172.21.15.122:0/949728092 shutdown_connections 2024-04-02T13:57:17.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.494+0000 7fd7d4847640 1 --2- 172.21.15.122:0/949728092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d0101330 0x7fd7d010d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.494+0000 7fd7d4847640 1 --2- 172.21.15.122:0/949728092 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0100970 0x7fd7d0100df0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.494+0000 7fd7d4847640 1 --2- 172.21.15.122:0/949728092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0106980 0x7fd7d0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.494+0000 7fd7d4847640 1 -- 172.21.15.122:0/949728092 >> 172.21.15.122:0/949728092 conn(0x7fd7d0077b00 msgr2=0x7fd7d0077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:17.501 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.499+0000 7fd7d4847640 1 -- 172.21.15.122:0/949728092 shutdown_connections 2024-04-02T13:57:17.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.502+0000 7fd7d4847640 1 -- 172.21.15.122:0/949728092 wait complete. 2024-04-02T13:57:17.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.503+0000 7fd7d4847640 1 Processor -- start 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 -- start start 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0100970 0x7fd7d01972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d0106980 0x7fd7d019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0110de0 con 0x7fd7d0106980 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0110c60 con 0x7fd7d0101330 2024-04-02T13:57:17.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.508+0000 7fd7d4847640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0110f60 con 0x7fd7d0100970 2024-04-02T13:57:17.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.509+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:17.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.509+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:56852/0 (socket says 172.21.15.122:56852) 2024-04-02T13:57:17.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.509+0000 7fd7cdd9b640 1 -- 172.21.15.122:0/2475804403 learned_addr learned my addr 172.21.15.122:0/2475804403 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:17.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 -- 172.21.15.122:0/2475804403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0100970 msgr2=0x7fd7d01972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0100970 0x7fd7d01972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 -- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d0106980 msgr2=0x7fd7d019bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-02T13:57:17.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d0106980 0x7fd7d019bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 -- 172.21.15.122:0/2475804403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019c230 con 0x7fd7d0101330 2024-04-02T13:57:17.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.510+0000 7fd7cdd9b640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd7c402c250 tx=0x7fd7c40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:17.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.512+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4041070 con 0x7fd7d0101330 2024-04-02T13:57:17.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.512+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019c460 con 0x7fd7d0101330 2024-04-02T13:57:17.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.512+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d0068c60 con 0x7fd7d0101330 2024-04-02T13:57:17.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.513+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c4004c20 con 0x7fd7d0101330 2024-04-02T13:57:17.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.514+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4003970 con 0x7fd7d0101330 2024-04-02T13:57:17.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.515+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b0005290 con 0x7fd7d0101330 2024-04-02T13:57:17.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.516+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd7c4044070 con 0x7fd7d0101330 2024-04-02T13:57:17.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.517+0000 7fd7bf7fe640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 0x7fd7b80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.518+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fd7c40f2950 con 0x7fd7d0101330 2024-04-02T13:57:17.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.518+0000 7fd7ce59c640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 0x7fd7b80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:17.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.520+0000 7fd7ce59c640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 0x7fd7b80a6180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0005d80 tx=0x7fd7c0005c90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:17.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.521+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd7c40becb0 con 0x7fd7d0101330 2024-04-02T13:57:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:17 smithi122 ceph-mon[32114]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-04-02T13:57:17.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:17 smithi122 ceph-mon[32114]: osdmap e127: 12 total, 12 up, 12 in 2024-04-02T13:57:17.572 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:17 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1687968279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:57:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:17 smithi187 ceph-mon[36064]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-04-02T13:57:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:17 smithi187 ceph-mon[36064]: osdmap e127: 12 total, 12 up, 12 in 2024-04-02T13:57:17.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:17 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1687968279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:57:17.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:17 smithi188 ceph-mon[36117]: pgmap v697: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-04-02T13:57:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:17 smithi188 ceph-mon[36117]: osdmap e127: 12 total, 12 up, 12 in 2024-04-02T13:57:17.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:17 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1687968279' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:57:17.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.875+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd7b0005850 con 0x7fd7d0101330 2024-04-02T13:57:17.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.879+0000 7fd7bf7fe640 1 -- 172.21.15.122:0/2475804403 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7fd7c40be5b0 con 0x7fd7d0101330 2024-04-02T13:57:17.882 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:57:17.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.882+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 msgr2=0x7fd7b80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 0x7fd7b80a6180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0005d80 tx=0x7fd7c0005c90 comp rx=0 tx=0).stop 2024-04-02T13:57:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 msgr2=0x7fd7d0197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd7c402c250 tx=0x7fd7c40045e0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 shutdown_connections 2024-04-02T13:57:17.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd7b80a3cc0 0x7fd7b80a6180 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7d0106980 0x7fd7d019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd7d0101330 0x7fd7d0197800 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 --2- 172.21.15.122:0/2475804403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7d0100970 0x7fd7d01972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.883+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 >> 172.21.15.122:0/2475804403 conn(0x7fd7d0077b00 msgr2=0x7fd7d01030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.884+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 shutdown_connections 2024-04-02T13:57:17.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:17.884+0000 7fd7d4847640 1 -- 172.21.15.122:0/2475804403 wait complete. 2024-04-02T13:57:17.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:57:18.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.096+0000 7fed0c270640 1 -- 172.21.15.122:0/919431599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04106980 msgr2=0x7fed04106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.096+0000 7fed0c270640 1 --2- 172.21.15.122:0/919431599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04106980 0x7fed04106d60 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fed0000d940 tx=0x7fed0002dbd0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.098+0000 7fed0c270640 1 -- 172.21.15.122:0/919431599 shutdown_connections 2024-04-02T13:57:18.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.098+0000 7fed0c270640 1 --2- 172.21.15.122:0/919431599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed04101330 0x7fed0410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.098+0000 7fed0c270640 1 --2- 172.21.15.122:0/919431599 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed04100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.098+0000 7fed0c270640 1 --2- 172.21.15.122:0/919431599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04106980 0x7fed04106d60 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.098+0000 7fed0c270640 1 -- 172.21.15.122:0/919431599 >> 172.21.15.122:0/919431599 conn(0x7fed04077b00 msgr2=0x7fed04077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:18.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.105+0000 7fed0c270640 1 -- 172.21.15.122:0/919431599 shutdown_connections 2024-04-02T13:57:18.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.106+0000 7fed0c270640 1 -- 172.21.15.122:0/919431599 wait complete. 2024-04-02T13:57:18.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.107+0000 7fed0c270640 1 Processor -- start 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 -- start start 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed041a7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 0x7fed041a7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed04106980 0x7fed041abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410fcf0 con 0x7fed04101330 2024-04-02T13:57:18.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410fb70 con 0x7fed04100970 2024-04-02T13:57:18.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.109+0000 7fed0c270640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed0410fe70 con 0x7fed04106980 2024-04-02T13:57:18.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.110+0000 7fed0a00c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed041a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.110+0000 7fed0a00c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed041a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.122:56862/0 (socket says 172.21.15.122:56862) 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.110+0000 7fed0a00c640 1 -- 172.21.15.122:0/3813545275 learned_addr learned my addr 172.21.15.122:0/3813545275 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.110+0000 7fed0980b640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 0x7fed041a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 -- 172.21.15.122:0/3813545275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed04106980 msgr2=0x7fed041abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed04106980 0x7fed041abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 -- 172.21.15.122:0/3813545275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 msgr2=0x7fed041a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed041a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 -- 172.21.15.122:0/3813545275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed041ac230 con 0x7fed04101330 2024-04-02T13:57:18.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.111+0000 7fed0980b640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 0x7fed041a7780 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:18.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.112+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf4015020 con 0x7fed04101330 2024-04-02T13:57:18.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.112+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecf4011d10 con 0x7fed04101330 2024-04-02T13:57:18.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.112+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf400f2e0 con 0x7fed04101330 2024-04-02T13:57:18.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.113+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041ac520 con 0x7fed04101330 2024-04-02T13:57:18.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.113+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041b3db0 con 0x7fed04101330 2024-04-02T13:57:18.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.116+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fece8005290 con 0x7fed04101330 2024-04-02T13:57:18.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.118+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fecf4018020 con 0x7fed04101330 2024-04-02T13:57:18.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.119+0000 7fecfb7fe640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 0x7fecf00a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.119+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fecf40d00d0 con 0x7fed04101330 2024-04-02T13:57:18.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.119+0000 7fed0a00c640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 0x7fecf00a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.128+0000 7fed0a00c640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 0x7fecf00a63e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed00006010 tx=0x7fed0000d2f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:18.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.128+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fecf401f020 con 0x7fed04101330 2024-04-02T13:57:18.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.476+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fece8005850 con 0x7fed04101330 2024-04-02T13:57:18.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.478+0000 7fecfb7fe640 1 -- 172.21.15.122:0/3813545275 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) ==== 104+0+11 (secure 0 0 0) 0x7fecf409c430 con 0x7fed04101330 2024-04-02T13:57:18.481 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:57:18.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 msgr2=0x7fecf00a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 0x7fecf00a63e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fed00006010 tx=0x7fed0000d2f0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 msgr2=0x7fed041a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 0x7fed041a7780 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fd0 tx=0x7fecf400b870 comp rx=0 tx=0).stop 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 shutdown_connections 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fecf00a3f20 0x7fecf00a63e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed04106980 0x7fed041abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed04101330 0x7fed041a7780 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 --2- 172.21.15.122:0/3813545275 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fed04100970 0x7fed041a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.481+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 >> 172.21.15.122:0/3813545275 conn(0x7fed04077b00 msgr2=0x7fed041026b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:18.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.482+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 shutdown_connections 2024-04-02T13:57:18.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.483+0000 7fed0c270640 1 -- 172.21.15.122:0/3813545275 wait complete. 2024-04-02T13:57:18.503 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:57:18.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.677+0000 7fcf086a5640 1 -- 172.21.15.122:0/3648745548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001094c0 msgr2=0x7fcf00115350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.677+0000 7fcf086a5640 1 --2- 172.21.15.122:0/3648745548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001094c0 0x7fcf00115350 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fcef8005fd0 tx=0x7fcef80301f0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.678+0000 7fcf086a5640 1 -- 172.21.15.122:0/3648745548 shutdown_connections 2024-04-02T13:57:18.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.678+0000 7fcf086a5640 1 --2- 172.21.15.122:0/3648745548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf001094c0 0x7fcf00115350 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.678+0000 7fcf086a5640 1 --2- 172.21.15.122:0/3648745548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf00108b00 0x7fcf00108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.678+0000 7fcf086a5640 1 --2- 172.21.15.122:0/3648745548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf0010e590 0x7fcf0010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.678+0000 7fcf086a5640 1 -- 172.21.15.122:0/3648745548 >> 172.21.15.122:0/3648745548 conn(0x7fcf0006d040 msgr2=0x7fcf0006d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:18.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.680+0000 7fcf086a5640 1 -- 172.21.15.122:0/3648745548 shutdown_connections 2024-04-02T13:57:18.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.681+0000 7fcf086a5640 1 -- 172.21.15.122:0/3648745548 wait complete. 2024-04-02T13:57:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 Processor -- start 2024-04-02T13:57:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 -- start start 2024-04-02T13:57:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 0x7fcf001a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 0x7fcf001a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0010e590 0x7fcf001abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001184e0 con 0x7fcf00108b00 2024-04-02T13:57:18.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00118360 con 0x7fcf001094c0 2024-04-02T13:57:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.682+0000 7fcf086a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf00118660 con 0x7fcf0010e590 2024-04-02T13:57:18.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.683+0000 7fcf06441640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 0x7fcf001a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.683+0000 7fcf06441640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 0x7fcf001a71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56482/0 (socket says 172.21.15.122:56482) 2024-04-02T13:57:18.691 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.683+0000 7fcf06441640 1 -- 172.21.15.122:0/2442584644 learned_addr learned my addr 172.21.15.122:0/2442584644 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.683+0000 7fcf05c40640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 0x7fcf001a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.683+0000 7fcf06c42640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0010e590 0x7fcf001abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 -- 172.21.15.122:0/2442584644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0010e590 msgr2=0x7fcf001abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0010e590 0x7fcf001abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 -- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 msgr2=0x7fcf001a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.692 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 0x7fcf001a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 -- 172.21.15.122:0/2442584644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf001ac1e0 con 0x7fcf001094c0 2024-04-02T13:57:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.684+0000 7fcf05c40640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 0x7fcf001a7730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009fd0 tx=0x7fcefc00b7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.685+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc026020 con 0x7fcf001094c0 2024-04-02T13:57:18.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.685+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf001ac4d0 con 0x7fcf001094c0 2024-04-02T13:57:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.685+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0019d9b0 con 0x7fcf001094c0 2024-04-02T13:57:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.685+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc022b20 con 0x7fcf001094c0 2024-04-02T13:57:18.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.685+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc021c60 con 0x7fcf001094c0 2024-04-02T13:57:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.687+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf000688f0 con 0x7fcf001094c0 2024-04-02T13:57:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.688+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcefc029020 con 0x7fcf001094c0 2024-04-02T13:57:18.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.689+0000 7fcef77fe640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 0x7fceec0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.689+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fcefc0e2080 con 0x7fcf001094c0 2024-04-02T13:57:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.690+0000 7fcf06441640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 0x7fceec0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:18.696 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.690+0000 7fcf06441640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 0x7fceec0a63e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009a30 tx=0x7fcef0009190 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:18.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.696+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcefc030020 con 0x7fcf001094c0 2024-04-02T13:57:18.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:18 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2475804403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:57:18.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:18 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2475804403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:57:18.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:18 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2475804403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:57:18.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.958+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcf001103f0 con 0x7fcf001094c0 2024-04-02T13:57:18.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.959+0000 7fcef77fe640 1 -- 172.21.15.122:0/2442584644 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) ==== 100+0+11 (secure 0 0 0) 0x7fcefc0ae3e0 con 0x7fcf001094c0 2024-04-02T13:57:18.963 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:57:18.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 msgr2=0x7fceec0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 0x7fceec0a63e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009a30 tx=0x7fcef0009190 comp rx=0 tx=0).stop 2024-04-02T13:57:18.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 msgr2=0x7fcf001a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:18.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 0x7fcf001a7730 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009fd0 tx=0x7fcefc00b7f0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 shutdown_connections 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fceec0a3f20 0x7fceec0a63e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf0010e590 0x7fcf001abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf001094c0 0x7fcf001a7730 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 --2- 172.21.15.122:0/2442584644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf00108b00 0x7fcf001a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 >> 172.21.15.122:0/2442584644 conn(0x7fcf0006d040 msgr2=0x7fcf00072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 shutdown_connections 2024-04-02T13:57:18.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:18.961+0000 7fcf086a5640 1 -- 172.21.15.122:0/2442584644 wait complete. 2024-04-02T13:57:18.975 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:57:19.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.108+0000 7fcdcfab1640 1 -- 172.21.15.122:0/1879920005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806bcb0 msgr2=0x7fcdc811adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.108+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/1879920005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806bcb0 0x7fcdc811adc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009890 tx=0x7fcdc402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:19.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.110+0000 7fcdcfab1640 1 -- 172.21.15.122:0/1879920005 shutdown_connections 2024-04-02T13:57:19.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.110+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/1879920005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 0x7fcdc811d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.110+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/1879920005 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806bcb0 0x7fcdc811adc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.110+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/1879920005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806b390 0x7fcdc806b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.110+0000 7fcdcfab1640 1 -- 172.21.15.122:0/1879920005 >> 172.21.15.122:0/1879920005 conn(0x7fcdc8073090 msgr2=0x7fcdc80734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:19.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.111+0000 7fcdcfab1640 1 -- 172.21.15.122:0/1879920005 shutdown_connections 2024-04-02T13:57:19.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.111+0000 7fcdcfab1640 1 -- 172.21.15.122:0/1879920005 wait complete. 2024-04-02T13:57:19.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 Processor -- start 2024-04-02T13:57:19.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 -- start start 2024-04-02T13:57:19.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806b390 0x7fcdc811aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 0x7fcdc81170a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8110fa0 con 0x7fcdc806bcb0 2024-04-02T13:57:19.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8110e20 con 0x7fcdc806b390 2024-04-02T13:57:19.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.112+0000 7fcdcfab1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc8111120 con 0x7fcdc811b300 2024-04-02T13:57:19.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.116+0000 7fcdce04e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 0x7fcdc81170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.117+0000 7fcdcd04c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.117+0000 7fcdcd04c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56498/0 (socket says 172.21.15.122:56498) 2024-04-02T13:57:19.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 -- 172.21.15.122:0/3318466845 learned_addr learned my addr 172.21.15.122:0/3318466845 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:19.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 -- 172.21.15.122:0/3318466845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 msgr2=0x7fcdc81170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 0x7fcdc81170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 -- 172.21.15.122:0/3318466845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806b390 msgr2=0x7fcdc811aaf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-02T13:57:19.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806b390 0x7fcdc811aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.118+0000 7fcdcd04c640 1 -- 172.21.15.122:0/3318466845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc8117820 con 0x7fcdc806bcb0 2024-04-02T13:57:19.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.119+0000 7fcdcd04c640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4000c00 tx=0x7fcdc4034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.119+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4039040 con 0x7fcdc806bcb0 2024-04-02T13:57:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.119+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc81ab0a0 con 0x7fcdc806bcb0 2024-04-02T13:57:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.120+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc81ab500 con 0x7fcdc806bcb0 2024-04-02T13:57:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.120+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc4042070 con 0x7fcdc806bcb0 2024-04-02T13:57:19.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.120+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc402ce60 con 0x7fcdc806bcb0 2024-04-02T13:57:19.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.121+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdac005290 con 0x7fcdc806bcb0 2024-04-02T13:57:19.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.123+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fcdc4005050 con 0x7fcdc806bcb0 2024-04-02T13:57:19.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.124+0000 7fcdbeffd640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 0x7fcdb40a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.124+0000 7fcdcd84d640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 0x7fcdb40a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.124+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fcdc40f1f80 con 0x7fcdc806bcb0 2024-04-02T13:57:19.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.125+0000 7fcdcd84d640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 0x7fcdb40a6360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005c40 tx=0x7fcdc0005bd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:19.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.132+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fcdc40be2e0 con 0x7fcdc806bcb0 2024-04-02T13:57:19.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.468+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcdac005850 con 0x7fcdc806bcb0 2024-04-02T13:57:19.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.470+0000 7fcdbeffd640 1 -- 172.21.15.122:0/3318466845 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v127) ==== 103+0+11 (secure 0 0 0) 0x7fcdc40bdbe0 con 0x7fcdc806bcb0 2024-04-02T13:57:19.475 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:57:19.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 msgr2=0x7fcdb40a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 0x7fcdb40a6360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0005c40 tx=0x7fcdc0005bd0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 msgr2=0x7fcdc8116b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4000c00 tx=0x7fcdc4034670 comp rx=0 tx=0).stop 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 shutdown_connections 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fcdb40a3ea0 0x7fcdb40a6360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc811b300 0x7fcdc81170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcdc806bcb0 0x7fcdc8116b60 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 --2- 172.21.15.122:0/3318466845 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcdc806b390 0x7fcdc811aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 >> 172.21.15.122:0/3318466845 conn(0x7fcdc8073090 msgr2=0x7fcdc811b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 shutdown_connections 2024-04-02T13:57:19.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.474+0000 7fcdcfab1640 1 -- 172.21.15.122:0/3318466845 wait complete. 2024-04-02T13:57:19.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-02T13:57:19.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 -- 172.21.15.122:0/2911631108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 msgr2=0x7f03741011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 --2- 172.21.15.122:0/2911631108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f03741011d0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f0364009930 tx=0x7f036402bc30 comp rx=0 tx=0).stop 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 -- 172.21.15.122:0/2911631108 shutdown_connections 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 --2- 172.21.15.122:0/2911631108 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0374101710 0x7f037410d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 --2- 172.21.15.122:0/2911631108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f03741011d0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 --2- 172.21.15.122:0/2911631108 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03741059e0 0x7f0374105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 -- 172.21.15.122:0/2911631108 >> 172.21.15.122:0/2911631108 conn(0x7f0374076800 msgr2=0x7f0374076c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.593+0000 7f037c488640 1 -- 172.21.15.122:0/2911631108 shutdown_connections 2024-04-02T13:57:19.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 -- 172.21.15.122:0/2911631108 wait complete. 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 Processor -- start 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 -- start start 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0374101710 0x7f03741997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 0x7f03741a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0374110e10 con 0x7f0374100d50 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0374110c90 con 0x7f03741059e0 2024-04-02T13:57:19.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037c488640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0374110f90 con 0x7f0374101710 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037a224640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037a224640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:56518/0 (socket says 172.21.15.122:56518) 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f037a224640 1 -- 172.21.15.122:0/1095979930 learned_addr learned my addr 172.21.15.122:0/1095979930 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.594+0000 7f0379a23640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0374101710 0x7f03741997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037aa25640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 0x7f03741a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 -- 172.21.15.122:0/1095979930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0374101710 msgr2=0x7f03741997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0374101710 0x7f03741997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 -- 172.21.15.122:0/1095979930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 msgr2=0x7f03741a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 0x7f03741a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 -- 172.21.15.122:0/1095979930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03741a4830 con 0x7f0374100d50 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037aa25640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 0x7f03741a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037a224640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f037000e9f0 tx=0x7f037000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037000cd80 con 0x7f0374100d50 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03741a4a00 con 0x7f0374100d50 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0370005500 con 0x7f0374100d50 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03741a4ed0 con 0x7f0374100d50 2024-04-02T13:57:19.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.595+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0370018630 con 0x7f0374100d50 2024-04-02T13:57:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.596+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0374069980 con 0x7f0374100d50 2024-04-02T13:57:19.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.597+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f03700046c0 con 0x7f0374100d50 2024-04-02T13:57:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.597+0000 7f036b7fe640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 0x7f03600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:57:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.598+0000 7f0379a23640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 0x7f03600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:57:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.598+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f03700d1130 con 0x7f0374100d50 2024-04-02T13:57:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.598+0000 7f0379a23640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 0x7f03600a61e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0364005ab0 tx=0x7f0364000bc0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:57:19.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.602+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f037009d3e0 con 0x7f0374100d50 2024-04-02T13:57:19.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:19 smithi187 ceph-mon[36064]: pgmap v699: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-04-02T13:57:19.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3813545275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2442584644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:19 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3318466845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:19 smithi188 ceph-mon[36117]: pgmap v699: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-04-02T13:57:19.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3813545275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2442584644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:19 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3318466845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:19 smithi122 ceph-mon[32114]: pgmap v699: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-04-02T13:57:19.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3813545275' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2442584644' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:19 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3318466845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:57:19.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.836+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f0374102120 con 0x7f0374100d50 2024-04-02T13:57:19.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.843+0000 7f036b7fe640 1 -- 172.21.15.122:0/1095979930 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746068 (secure 0 0 0) 0x7f037009cce0 con 0x7f0374100d50 2024-04-02T13:57:19.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:57:19.857 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:19.857 INFO:teuthology.orchestra.run.smithi122.stdout: "epoch": 25, 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "active_gid": 15528, 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "active_name": "x", 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "active_addrs": { 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:19.858 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:6800", 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 692246399 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:6801", 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 692246399 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:19.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "active_addr": "172.21.15.122:6801/692246399", 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "active_change": "2024-04-02T13:35:09.813283+0000", 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "active_mgr_features": 4540701547738038271, 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "available": true, 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "standbys": [ 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 25018, 2024-04-02T13:57:19.860 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "z", 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr_features": 4540701547738038271, 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:19.861 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.862 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-04-02T13:57:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-04-02T13:57:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-04-02T13:57:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-04-02T13:57:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-04-02T13:57:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:57:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-04-02T13:57:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-04-02T13:57:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:57:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-04-02T13:57:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-04-02T13:57:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:57:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-04-02T13:57:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:57:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:57:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-04-02T13:57:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:57:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.955 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:57:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-04-02T13:57:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:57:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-04-02T13:57:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:57:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:57:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-04-02T13:57:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:57:19.986 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.987 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.988 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.989 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:19.990 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.991 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.992 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.994 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.995 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.996 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.997 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:19.998 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-04-02T13:57:19.999 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.000 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.001 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.002 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:57:20.003 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.006 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-04-02T13:57:20.007 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:57:20.008 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.009 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.010 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.011 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.013 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-04-02T13:57:20.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:57:20.015 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.016 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.018 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-04-02T13:57:20.019 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:57:20.020 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.021 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.022 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.023 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.024 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.025 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.026 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:57:20.027 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.028 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.029 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.030 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.031 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.032 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.033 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.035 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.036 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.038 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.040 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-04-02T13:57:20.041 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:57:20.043 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.044 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:57:20.046 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.047 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.048 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.049 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.050 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:57:20.051 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.052 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.053 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.054 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.055 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:57:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.057 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.058 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.060 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.061 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.062 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.063 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:57:20.064 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.065 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.068 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:57:20.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.070 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.071 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.072 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.073 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.075 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.076 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:57:20.077 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.078 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.079 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "[]", 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.080 INFO:teuthology.orchestra.run.smithi122.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "{}", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.081 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:57:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.083 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.084 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.085 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.086 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:57:20.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.089 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.090 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.091 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.092 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.093 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.094 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:57:20.095 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.096 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.097 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.098 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.099 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.100 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:57:20.101 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.102 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.103 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.105 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:57:20.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.107 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.108 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:57:20.109 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.110 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.111 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:57:20.112 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.114 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.115 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.116 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.117 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.118 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.119 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.120 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.121 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-04-02T13:57:20.122 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.123 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.124 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.125 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.126 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.127 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.128 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.130 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.132 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.133 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.134 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.135 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:20.136 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.137 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.138 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.140 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.141 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.142 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.143 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.144 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.145 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.147 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.149 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.150 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.151 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.152 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.153 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.154 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.155 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.156 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.157 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.158 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.159 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.161 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.162 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.163 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.164 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.166 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:57:20.168 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.169 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.171 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.172 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.173 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.175 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.176 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.177 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.178 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.180 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-04-02T13:57:20.181 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.182 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.183 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.184 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:57:20.185 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.186 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-04-02T13:57:20.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:57:20.188 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.189 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.190 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-04-02T13:57:20.191 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.192 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.193 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.194 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.195 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.196 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.197 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.198 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.199 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.200 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:57:20.201 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.203 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-04-02T13:57:20.204 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-04-02T13:57:20.205 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.206 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:57:20.207 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.207 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.207 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.209 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-04-02T13:57:20.209 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-04-02T13:57:20.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:57:20.210 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.211 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.213 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.214 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.215 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.216 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.217 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.218 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.219 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.220 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.221 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.222 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.223 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.224 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.225 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.226 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.227 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.228 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.230 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.231 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.232 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-04-02T13:57:20.233 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.234 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.236 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.237 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.238 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.239 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.240 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.244 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.246 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.247 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.248 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.249 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.250 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.251 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.252 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.253 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.254 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.255 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-04-02T13:57:20.256 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-04-02T13:57:20.257 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.258 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.259 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.261 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.263 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.264 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.266 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.267 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.268 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.269 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.270 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.273 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.274 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.275 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.276 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.277 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.278 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.279 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.280 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.281 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.282 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.283 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.284 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.285 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.286 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.287 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.288 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.290 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.291 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.292 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.293 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.294 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.295 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.296 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.297 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.298 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.299 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.301 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.302 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.303 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.304 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.305 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.306 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.307 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.308 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.309 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.311 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.312 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.313 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.314 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.315 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.317 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.318 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.319 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.320 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.321 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.323 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.325 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.326 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.327 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.329 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.330 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:20.332 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-04-02T13:57:20.334 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:57:20.335 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.337 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.338 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.339 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.341 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.342 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.343 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.345 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-04-02T13:57:20.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:57:20.347 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.348 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.349 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.350 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.351 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-04-02T13:57:20.352 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:57:20.353 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.354 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.355 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.356 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.357 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.358 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.359 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.361 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-04-02T13:57:20.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.363 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-04-02T13:57:20.366 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.367 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.368 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.369 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.370 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-04-02T13:57:20.371 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:20.372 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.373 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.374 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.375 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.376 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.377 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.378 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.379 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.380 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.381 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.382 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.384 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:57:20.385 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.386 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.387 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.389 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-04-02T13:57:20.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.392 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.393 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.394 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.395 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.396 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.397 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.398 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.400 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.401 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.402 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.403 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.404 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.405 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.408 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.409 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.410 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.411 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.412 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.413 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.417 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.417 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-04-02T13:57:20.417 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.418 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.419 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.420 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.421 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.423 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.424 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.425 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.426 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.427 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.428 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.429 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.430 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.431 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.432 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.433 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.434 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.435 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.436 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.437 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-04-02T13:57:20.439 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.440 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.441 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-04-02T13:57:20.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.443 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.444 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.445 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.446 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-04-02T13:57:20.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.448 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.449 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:20.451 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-04-02T13:57:20.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.453 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.456 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-04-02T13:57:20.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:57:20.459 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.460 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.462 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.463 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.464 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.465 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.466 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.467 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.468 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.469 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-04-02T13:57:20.470 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.471 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.472 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.473 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.474 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.475 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.476 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.478 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.479 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.480 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.481 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.482 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.483 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.484 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.485 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.487 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.488 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.490 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.491 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.492 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.493 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.494 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.495 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.497 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.498 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.499 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.500 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-04-02T13:57:20.501 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.502 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.504 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-04-02T13:57:20.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:57:20.506 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.508 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.509 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.510 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.511 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.512 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.513 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:57:20.514 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:20.515 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.516 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.517 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.519 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.520 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.521 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.522 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.523 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.524 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.525 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.528 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.529 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.530 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.531 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.534 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-04-02T13:57:20.535 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.536 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.537 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.538 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.539 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.541 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.542 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.543 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.544 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.545 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.547 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.548 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.549 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.550 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.552 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.553 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.554 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.556 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.558 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.559 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:57:20.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:57:20.561 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-04-02T13:57:20.562 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.563 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.564 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.565 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.566 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.568 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.569 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.570 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.571 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.572 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.573 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.575 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.576 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.577 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-04-02T13:57:20.578 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.579 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.580 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:20.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "gid": 25067, 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "y", 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr_features": 4540701547738038271, 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-04-02T13:57:20.582 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.583 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.585 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.586 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.587 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.588 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.589 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.590 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.591 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.592 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.593 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.595 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-04-02T13:57:20.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.597 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.598 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.599 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.600 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-04-02T13:57:20.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.602 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.603 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.605 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.607 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.608 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.609 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-04-02T13:57:20.610 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.611 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.614 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-04-02T13:57:20.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:57:20.616 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.617 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.618 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-04-02T13:57:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-04-02T13:57:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.619 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:20.621 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.622 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.623 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.625 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.626 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.627 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.628 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.629 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.631 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:20.632 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-04-02T13:57:20.633 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.635 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.637 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.638 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.639 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.640 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.641 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.642 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.643 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.644 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-04-02T13:57:20.645 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.646 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.647 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.648 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.649 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.650 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.651 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.652 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-04-02T13:57:20.653 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-04-02T13:57:20.654 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.656 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.657 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.658 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-04-02T13:57:20.659 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.660 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:57:20.661 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.663 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-04-02T13:57:20.664 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:20.665 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.667 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.668 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-04-02T13:57:20.669 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.670 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.672 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.673 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-04-02T13:57:20.674 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.675 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.676 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.678 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-04-02T13:57:20.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:20.680 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.681 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.683 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-04-02T13:57:20.684 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:57:20.685 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.686 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.688 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:57:20.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:57:20.690 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.691 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.693 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-04-02T13:57:20.694 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:57:20.695 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.696 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.698 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-04-02T13:57:20.699 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.700 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.701 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.702 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.703 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.704 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.705 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.706 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.707 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.708 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:57:20.709 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.710 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.711 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.712 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.713 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.714 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.715 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-04-02T13:57:20.716 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-04-02T13:57:20.717 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.718 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.721 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:20.722 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-04-02T13:57:20.723 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-04-02T13:57:20.724 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.725 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.726 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.727 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.728 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-04-02T13:57:20.729 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:57:20.730 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.731 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.733 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:57:20.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:57:20.735 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.736 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.738 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-04-02T13:57:20.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:57:20.740 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.741 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.743 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:57:20.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:57:20.745 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.746 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.748 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.749 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.750 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.751 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.753 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.754 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.756 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.757 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-04-02T13:57:20.758 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.759 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-04-02T13:57:20.760 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.761 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.762 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.763 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.764 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.766 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.767 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.768 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:57:20.769 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.770 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.771 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.772 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.773 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:57:20.774 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.775 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.778 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:57:20.779 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.780 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.781 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.783 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:57:20.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.785 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.786 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.788 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:57:20.789 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.790 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.791 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.792 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.793 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:57:20.794 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.795 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.796 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "[]", 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.798 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:57:20.799 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "{}", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.800 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.801 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.803 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:57:20.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.805 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.806 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.808 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.809 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.810 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.811 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.812 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.813 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.814 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.815 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.816 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.817 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.818 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.819 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:57:20.820 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.821 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.822 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.824 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.825 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.826 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.829 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.829 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.830 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.831 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.832 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.833 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.834 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.835 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.836 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.837 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.838 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.839 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-04-02T13:57:20.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.841 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.842 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.843 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-04-02T13:57:20.844 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:20.845 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.846 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.847 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.848 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.850 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.851 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.852 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.853 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.854 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.855 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.856 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.857 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:20.859 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:20.860 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.861 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.862 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.863 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.864 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.865 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.866 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.867 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.868 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.869 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.870 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.871 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.872 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.873 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.874 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.875 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.876 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.877 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.878 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.881 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-04-02T13:57:20.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:57:20.883 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.885 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:20.886 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-04-02T13:57:20.887 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.888 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.889 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.890 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-04-02T13:57:20.891 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.892 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.893 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.894 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.895 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.896 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.897 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.898 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.899 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.900 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.901 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.902 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.903 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:20.905 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.906 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-04-02T13:57:20.907 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:57:20.908 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.909 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.910 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.911 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:20.912 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.913 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.914 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.915 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.916 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.917 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.918 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.919 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.920 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.921 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-04-02T13:57:20.922 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.923 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.924 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.925 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.926 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.927 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.928 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.929 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.930 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.931 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.933 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.934 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.936 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.938 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.939 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.940 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.942 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.943 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.945 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.947 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.948 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.949 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.951 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.952 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:20.953 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.954 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.955 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.956 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.957 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.959 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.960 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.961 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.962 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.963 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.964 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.965 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.966 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.967 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.968 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.970 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.971 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.972 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.974 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-04-02T13:57:20.975 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:57:20.976 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.977 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.979 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.980 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:57:20.981 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.982 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.983 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.984 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.985 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.986 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.987 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.988 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.989 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.990 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:20.991 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.992 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.993 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.994 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.995 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:20.996 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:20.997 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:20.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:20.999 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.000 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.001 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.002 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.003 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.004 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.005 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.007 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.008 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.009 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.010 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.012 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.013 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.014 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.015 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.016 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.017 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.018 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.019 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-04-02T13:57:21.021 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.023 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.024 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.025 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.026 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.027 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.028 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.030 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.032 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.033 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.034 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.035 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.036 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.037 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.038 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.039 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.040 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.041 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.042 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.043 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.045 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.046 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.047 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.048 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.049 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.050 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.051 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.052 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.053 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.054 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.056 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.057 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.058 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.059 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.060 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.061 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.062 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.063 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.066 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.067 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.068 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.069 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.070 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-04-02T13:57:21.071 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:57:21.072 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.073 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.074 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.075 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.076 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.077 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.078 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.080 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-04-02T13:57:21.081 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.082 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.083 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.084 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.085 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:21.086 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.087 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.088 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.089 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.090 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:21.091 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-04-02T13:57:21.092 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:21.093 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.094 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-04-02T13:57:21.095 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.096 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.097 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.099 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.100 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.102 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.103 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.106 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-04-02T13:57:21.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.108 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.109 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.110 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:21.111 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.112 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.113 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.114 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.116 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.117 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.118 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.119 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.120 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.121 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.122 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.123 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.126 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.127 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.128 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.129 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.130 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.131 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.132 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.134 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.135 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.136 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.137 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.139 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.140 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.141 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.142 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.145 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.146 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.148 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.150 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.151 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.152 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.154 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-04-02T13:57:21.155 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.156 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.157 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.158 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.159 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-04-02T13:57:21.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.161 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.162 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.164 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-04-02T13:57:21.165 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.166 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.167 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.168 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.169 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.170 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.172 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.173 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.174 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.176 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.177 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.178 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.179 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.180 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.181 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.182 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.183 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.184 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.186 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.187 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.189 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.190 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.191 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.193 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.196 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.197 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.198 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.199 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.200 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.201 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.202 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-04-02T13:57:21.203 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:57:21.204 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.205 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.206 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.207 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.208 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.209 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.210 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.211 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.212 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.213 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.214 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.215 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.217 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.218 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.219 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.220 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.221 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.222 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.223 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-04-02T13:57:21.224 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.225 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.226 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.227 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.228 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.229 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.230 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-04-02T13:57:21.231 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.232 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.233 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.234 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.235 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.236 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-04-02T13:57:21.236 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-04-02T13:57:21.237 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.237 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.238 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.239 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.240 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.241 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.242 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.244 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.245 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.246 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.247 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.248 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.249 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.250 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.251 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-04-02T13:57:21.252 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.253 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.254 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.255 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.256 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.257 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.258 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.260 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.261 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.262 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.263 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.264 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.265 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.267 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.268 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.269 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-04-02T13:57:21.270 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:57:21.271 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.272 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.274 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:57:21.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:57:21.276 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.277 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-04-02T13:57:21.278 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-04-02T13:57:21.279 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.281 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.283 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.284 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.285 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.286 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.287 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.288 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.289 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.290 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.291 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-04-02T13:57:21.292 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.293 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.294 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.295 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: "modules": [ 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: "dashboard", 2024-04-02T13:57:21.296 INFO:teuthology.orchestra.run.smithi122.stdout: "iostat", 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "nfs", 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "restful", 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "snap_schedule" 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "available_modules": [ 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "alerts", 2024-04-02T13:57:21.297 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.298 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How frequently to reexamine health status", 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.299 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.300 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.301 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.302 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.303 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.304 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.305 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.307 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_destination": { 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_destination", 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.308 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email address to send alerts to", 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.309 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_from_name": { 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_from_name", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "Ceph", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.310 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Email From: name", 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_host": { 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_host", 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.311 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP server", 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.312 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_password": { 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_password", 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.313 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Password to authenticate with", 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_port": { 2024-04-02T13:57:21.314 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_port", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "465", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP port", 2024-04-02T13:57:21.315 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_sender": { 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_sender", 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.316 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SMTP envelope sender", 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.317 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_ssl": { 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_ssl", 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.318 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "smtp_user": { 2024-04-02T13:57:21.319 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "smtp_user", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.320 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "User to authenticate as", 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "balancer", 2024-04-02T13:57:21.321 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "active": { 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "active", 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.322 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically balance PGs across cluster", 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.323 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_time": { 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_time", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0000", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.324 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "beginning time of day to automatically balance", 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "begin_weekday": { 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "begin_weekday", 2024-04-02T13:57:21.325 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-02T13:57:21.326 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_max_iterations": { 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_max_iterations", 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.327 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "25", 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "250", 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_metrics": { 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_metrics", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "pgs,objects,bytes", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.329 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "crush_compat_step": { 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crush_compat_step", 2024-04-02T13:57:21.330 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.5", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0.001", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "0.999", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "aggressiveness of optimization", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-02T13:57:21.331 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "end_time": { 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_time", 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.332 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2359", 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ending time of day to automatically balance", 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "end_weekday": { 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "end_weekday", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "0", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "6", 2024-04-02T13:57:21.334 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.335 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.336 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.337 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.338 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.339 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.340 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.341 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.342 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.343 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "min_score": { 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_score", 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.344 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:21.345 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "upmap", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.346 INFO:teuthology.orchestra.run.smithi122.stdout: "crush-compat", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "none", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "read", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap-read" 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Balancer mode", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.347 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_ids": { 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_ids", 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.348 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.349 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-02T13:57:21.350 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_deviation": { 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_deviation", 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.351 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.352 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "upmap_max_optimizations": { 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "upmap_max_optimizations", 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.353 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.354 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm", 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_down_multiplier": { 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_down_multiplier", 2024-04-02T13:57:21.355 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-02T13:57:21.356 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_refresh_rate": { 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_refresh_rate", 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.357 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "20", 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "agent_starting_port": { 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "agent_starting_port", 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4721", 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.359 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_ptrace": { 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_ptrace", 2024-04-02T13:57:21.360 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-02T13:57:21.361 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_interval": { 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_interval", 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.362 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.363 INFO:teuthology.orchestra.run.smithi122.stdout: "autotune_memory_target_ratio": { 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "autotune_memory_target_ratio", 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0.7", 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.364 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm_log_destination": { 2024-04-02T13:57:21.365 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cephadm_log_destination", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.366 INFO:teuthology.orchestra.run.smithi122.stdout: "file", 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "file,syslog", 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "syslog" 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.367 INFO:teuthology.orchestra.run.smithi122.stdout: "cgroups_split": { 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cgroups_split", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.368 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "config_checks_enabled": { 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_checks_enabled", 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.369 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.370 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "config_dashboard": { 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_dashboard", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.371 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_alertmanager": { 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_alertmanager", 2024-04-02T13:57:21.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.373 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_base": { 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_base", 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.374 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Container image name, without the tag", 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_elasticsearch": { 2024-04-02T13:57:21.375 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_elasticsearch", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "elasticsearch container image", 2024-04-02T13:57:21.376 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_grafana": { 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_grafana", 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.378 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_haproxy": { 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_haproxy", 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "HAproxy container image", 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_agent": { 2024-04-02T13:57:21.380 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_agent", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.381 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger agent container image", 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_collector": { 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_collector", 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.382 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger collector container image", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.383 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_jaeger_query": { 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_jaeger_query", 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-02T13:57:21.384 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Jaeger query container image", 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_keepalived": { 2024-04-02T13:57:21.385 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_keepalived", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.386 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Keepalived container image", 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_loki": { 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_loki", 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.387 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Loki container image", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.388 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_node_exporter": { 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_node_exporter", 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-02T13:57:21.389 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.390 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_nvmeof": { 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_nvmeof", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.391 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Nvme-of container image", 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_prometheus": { 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_prometheus", 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.392 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Prometheus container image", 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_promtail": { 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_promtail", 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.394 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Promtail container image", 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_samba": { 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_samba", 2024-04-02T13:57:21.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Samba/SMB container image", 2024-04-02T13:57:21.396 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "container_image_snmp_gateway": { 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_image_snmp_gateway", 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.397 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "SNMP Gateway container image", 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "container_init": { 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "container_init", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.399 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Run podman/docker with `--init`", 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "daemon_cache_timeout": { 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "daemon_cache_timeout", 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.400 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.401 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "default_cephadm_command_timeout": { 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_cephadm_command_timeout", 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "900", 2024-04-02T13:57:21.402 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.403 INFO:teuthology.orchestra.run.smithi122.stdout: "default_registry": { 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "default_registry", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "docker.io", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.404 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "device_cache_timeout": { 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_cache_timeout", 2024-04-02T13:57:21.405 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1800", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache device inventory", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "device_enhanced_scan": { 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_enhanced_scan", 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.407 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.408 INFO:teuthology.orchestra.run.smithi122.stdout: "facts_cache_timeout": { 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "facts_cache_timeout", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.409 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "seconds to cache host facts data", 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "grafana_dashboards_path": { 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "grafana_dashboards_path", 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.410 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.411 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "host_check_interval": { 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "host_check_interval", 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.412 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to perform a host check", 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "hw_monitoring": { 2024-04-02T13:57:21.413 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hw_monitoring", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.414 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "inventory_list_all": { 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "inventory_list_all", 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.415 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.416 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.417 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.418 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "log_refresh_metadata": { 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_refresh_metadata", 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.419 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.420 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.421 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.423 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.424 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.425 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-02T13:57:21.426 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-02T13:57:21.427 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "*", 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "max_count_per_host": { 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_count_per_host", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.430 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of daemons per service per host", 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "max_osd_draining_count": { 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_osd_draining_count", 2024-04-02T13:57:21.431 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.432 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "migration_current": { 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "migration_current", 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.433 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "internal - do not modify", 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.434 INFO:teuthology.orchestra.run.smithi122.stdout: "mode": { 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mode", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "root", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.435 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm-package", 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "root" 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "mode for remote execution of cephadm", 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.436 INFO:teuthology.orchestra.run.smithi122.stdout: "oob_default_addr": { 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "oob_default_addr", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "169.254.1.1", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.437 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "prometheus_alerts_path": { 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus_alerts_path", 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.438 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-02T13:57:21.441 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.441 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.441 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_insecure": { 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_insecure", 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.442 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_password": { 2024-04-02T13:57:21.443 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_password", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-02T13:57:21.444 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_url": { 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_url", 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "registry_username": { 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "registry_username", 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.447 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "secure_monitoring_stack": { 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "secure_monitoring_stack", 2024-04-02T13:57:21.448 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.449 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "service_discovery_port": { 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "service_discovery_port", 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.450 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8765", 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "cephadm service discovery port", 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "ssh_config_file": { 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssh_config_file", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.452 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "use_agent": { 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_agent", 2024-04-02T13:57:21.453 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.454 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "use_repo_digest": { 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "use_repo_digest", 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.455 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.456 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_failed_host_check": { 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_failed_host_check", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.457 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if the host check fails", 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_daemons": { 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_daemons", 2024-04-02T13:57:21.458 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.459 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_on_stray_hosts": { 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_on_stray_hosts", 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.460 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.461 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crash", 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.462 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.463 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.464 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.465 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.466 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.467 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.468 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.469 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "retain_interval": { 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retain_interval", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "31536000", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.471 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_recent_interval": { 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_recent_interval", 2024-04-02T13:57:21.472 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1209600", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.473 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dashboard", 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.474 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.475 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_HOST": { 2024-04-02T13:57:21.476 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.477 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.478 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_ENABLED": { 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_ENABLED", 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.480 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-02T13:57:21.481 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.482 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "ENABLE_BROWSABLE_API": { 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.484 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.485 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-02T13:57:21.486 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.487 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.488 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.490 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-02T13:57:21.491 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.492 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.493 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.495 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-02T13:57:21.496 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.497 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_PASSWORD": { 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.498 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.499 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.500 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_URL": { 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_URL", 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.502 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_API_USERNAME": { 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:21.503 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-02T13:57:21.504 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.505 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.506 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.507 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.508 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-02T13:57:21.509 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.510 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.511 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "[]", 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "MULTICLUSTER_CONFIG": { 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "{}", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.513 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_HOST": { 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.514 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.515 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.516 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.517 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.518 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-02T13:57:21.519 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.520 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.522 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.523 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-02T13:57:21.524 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.525 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.526 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.528 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_ENABLED": { 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-02T13:57:21.529 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.530 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-02T13:57:21.531 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.532 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.533 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.534 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.535 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "45", 2024-04-02T13:57:21.536 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ACCESS_KEY": { 2024-04-02T13:57:21.537 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.538 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.539 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "admin", 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.540 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SECRET_KEY": { 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.541 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "RGW_API_SSL_VERIFY": { 2024-04-02T13:57:21.542 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.543 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "UNSAFE_TLS_v1_2": { 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.545 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.546 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-02T13:57:21.547 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.548 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.549 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.550 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "cross_origin_url": { 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cross_origin_url", 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.551 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "crt_file": { 2024-04-02T13:57:21.552 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "crt_file", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.553 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "debug": { 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "debug", 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Enable/disable debug options", 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.555 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "jwt_token_ttl": { 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "jwt_token_ttl", 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "28800", 2024-04-02T13:57:21.556 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:21.557 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.558 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.560 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.561 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.562 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.563 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.564 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.566 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "motd": { 2024-04-02T13:57:21.567 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "motd", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "The message of the day", 2024-04-02T13:57:21.568 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect_resolve_ip_addr": { 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "redirect_resolve_ip_addr", 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.569 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.571 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:21.572 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8080", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.573 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.574 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl_server_port": { 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl_server_port", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8443", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.576 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:21.577 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "redirect", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.578 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "redirect" 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:21.579 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.580 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "url_prefix": { 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url_prefix", 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.582 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.583 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_monitoring": { 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_monitoring", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.584 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "monitor device health metrics", 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.585 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.586 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.587 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.588 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.589 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.590 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.591 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.592 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "mark_out_threshold": { 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mark_out_threshold", 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.593 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "2419200", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.594 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "pool_name": { 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pool_name", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "device_health_metrics", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.595 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "retention_period": { 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "retention_period", 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.596 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15552000", 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long to retain device health metrics", 2024-04-02T13:57:21.597 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_frequency": { 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_frequency", 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "self_heal": { 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "self_heal", 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.600 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.602 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how frequently to wake up and check device health", 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "warn_threshold": { 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "warn_threshold", 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.603 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7257600", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.604 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "diskprediction_local", 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.605 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.606 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.607 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.608 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.609 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.610 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.611 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.613 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "predict_interval": { 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predict_interval", 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "86400", 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.614 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "predictor_model": { 2024-04-02T13:57:21.615 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "predictor_model", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "prophetstor", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.616 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "600", 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "influx", 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": false, 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "influxdb python module not found", 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "batch_size": { 2024-04-02T13:57:21.619 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "batch_size", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5000", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.620 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "database": { 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "database", 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.621 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "ceph", 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.622 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": { 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "hostname", 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.623 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server hostname", 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.624 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "30", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "5", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.625 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.626 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.627 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.628 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.629 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.630 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.631 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.633 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "password": { 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "password", 2024-04-02T13:57:21.634 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "password of InfluxDB server user", 2024-04-02T13:57:21.635 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "port": { 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "port", 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.636 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "8086", 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "InfluxDB server port", 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "ssl": { 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ssl", 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "false", 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.638 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "threads": { 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threads", 2024-04-02T13:57:21.639 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "32", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.640 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "username": { 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "username", 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.641 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "username of InfluxDB server user", 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.642 INFO:teuthology.orchestra.run.smithi122.stdout: "verify_ssl": { 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "verify_ssl", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "true", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.643 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.644 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "insights", 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.645 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.646 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.647 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.648 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.649 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.650 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.651 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.652 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.653 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.654 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "iostat", 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.655 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.656 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.657 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.658 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.660 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.661 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.662 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.663 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "k8sevents", 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.664 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_event_retention_days": { 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "ceph_event_retention_days", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "7", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.665 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "config_check_secs": { 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "config_check_secs", 2024-04-02T13:57:21.666 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "10", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-02T13:57:21.667 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.668 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.669 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.670 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.671 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.672 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.673 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.674 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.675 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.676 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "localpool", 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "failure_domain": { 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "failure_domain", 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.677 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "host", 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "failure domain for any created local pool", 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.679 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.680 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.681 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.682 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.683 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.684 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.685 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.686 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "min_size": { 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "min_size", 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.687 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default min_size for any created local pool", 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "num_rep": { 2024-04-02T13:57:21.688 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "num_rep", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.689 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default replica count for any created local pool", 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_num": { 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_num", 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "128", 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "default pg_num for any created local pool", 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.691 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "prefix": { 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prefix", 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.692 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "name prefix for any created local pool", 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "subtree": { 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "subtree", 2024-04-02T13:57:21.693 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "rack", 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.694 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mds_autoscaler", 2024-04-02T13:57:21.695 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.696 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.697 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.698 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.699 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.700 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.701 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.702 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.703 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirroring", 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.704 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.705 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.706 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.707 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.708 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.709 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.710 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.711 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.712 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "nfs", 2024-04-02T13:57:21.713 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.714 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.715 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.716 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.718 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.719 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.720 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.721 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.722 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "fail_fs": { 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "fail_fs", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.723 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.724 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.725 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.726 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.727 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.728 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.729 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.730 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.731 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator": { 2024-04-02T13:57:21.732 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "orchestrator", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "cephadm", 2024-04-02T13:57:21.733 INFO:teuthology.orchestra.run.smithi122.stdout: "rook", 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: "test_orchestrator" 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Orchestrator backend", 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_perf_query", 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.735 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.736 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.737 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.738 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.739 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.740 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.741 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.742 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "osd_support", 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.744 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.745 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.747 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.750 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.751 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "pg_autoscaler", 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.754 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.755 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.756 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.757 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.758 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.760 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.761 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "threshold": { 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "threshold", 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "3.0", 2024-04-02T13:57:21.763 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1.0", 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "scaling threshold", 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "progress", 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_pg_recovery_event": { 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_pg_recovery_event", 2024-04-02T13:57:21.765 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-02T13:57:21.766 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.767 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.769 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.770 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.771 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.772 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.773 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.774 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.775 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.776 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "max_completed_events": { 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_completed_events", 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "50", 2024-04-02T13:57:21.777 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "number of past completed events to remember", 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.778 INFO:teuthology.orchestra.run.smithi122.stdout: "sleep_interval": { 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "sleep_interval", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "5", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.779 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "how long the module is going to sleep", 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.780 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "prometheus", 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "cache": { 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "cache", 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.782 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "exclude_perf_counters": { 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "exclude_perf_counters", 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.783 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.784 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.785 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.786 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.789 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.790 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools": { 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools", 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.793 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "300", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.795 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "scrape_interval": { 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "scrape_interval", 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.796 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15.0", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.797 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "::", 2024-04-02T13:57:21.798 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:21.799 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "9283", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-02T13:57:21.800 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "stale_cache_strategy": { 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stale_cache_strategy", 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.801 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "log", 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_behaviour": { 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_behaviour", 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "default", 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.803 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "default", 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "error" 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.804 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "standby_error_status_code": { 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "standby_error_status_code", 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "500", 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "400", 2024-04-02T13:57:21.805 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "599", 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.808 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.809 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.810 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.811 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.813 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.814 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.815 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_snap_create": { 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_snap_create", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.816 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "mirror_snapshot_schedule": { 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "mirror_snapshot_schedule", 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.817 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.818 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "trash_purge_schedule": { 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "trash_purge_schedule", 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.819 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.820 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "restful", 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "enable_auth": { 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enable_auth", 2024-04-02T13:57:21.821 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.822 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "key_file": { 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "key_file", 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.823 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.825 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.826 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.827 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.828 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.829 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.830 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.831 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.832 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "server_addr": { 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_addr", 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.833 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.834 INFO:teuthology.orchestra.run.smithi122.stdout: "server_port": { 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "server_port", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.835 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.836 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rgw", 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.838 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.839 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.840 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.841 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.842 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.844 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rook", 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.846 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.847 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.848 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.849 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.850 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.851 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.851 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.853 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.853 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.853 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.854 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.855 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "storage_class": { 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "storage_class", 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "local", 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.857 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "selftest", 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.859 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.860 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.861 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.862 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.863 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.864 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.866 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "roption1": { 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption1", 2024-04-02T13:57:21.867 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.868 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "roption2": { 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "roption2", 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.869 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "xyz", 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption1": { 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption1", 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.871 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption2": { 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption2", 2024-04-02T13:57:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.873 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption3": { 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption3", 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "float", 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.874 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption4": { 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption4", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.876 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption5": { 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption5", 2024-04-02T13:57:21.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.878 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption6": { 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption6", 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.879 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "rwoption7": { 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rwoption7", 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "1", 2024-04-02T13:57:21.881 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "42", 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "testkey": { 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testkey", 2024-04-02T13:57:21.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.883 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "testlkey": { 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testlkey", 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.885 INFO:teuthology.orchestra.run.smithi122.stdout: "testnewline": { 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "testnewline", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.886 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.887 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "allow_m_granularity": { 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "allow_m_granularity", 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.888 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "allow minute scheduled snapshots", 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "dump_on_update": { 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "dump_on_update", 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.890 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "dump database to debug log on update", 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.891 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.892 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.893 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.894 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.895 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.896 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.897 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.898 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "stats", 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.901 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.902 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.905 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.906 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.907 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "status", 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.909 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.910 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.912 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.914 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.915 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telegraf", 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "address": { 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "address", 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.918 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.919 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "15", 2024-04-02T13:57:21.920 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.921 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.922 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.923 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.924 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.926 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.927 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.928 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.929 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "telemetry", 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_basic": { 2024-04-02T13:57:21.930 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_basic", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-02T13:57:21.931 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_crash": { 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_crash", 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.932 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_device": { 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_device", 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.934 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_ident": { 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_ident", 2024-04-02T13:57:21.935 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.936 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "channel_perf": { 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "channel_perf", 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.937 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.938 INFO:teuthology.orchestra.run.smithi122.stdout: "contact": { 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "contact", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.939 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "description": { 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "description", 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.940 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.941 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "device_url": { 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "device_url", 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-02T13:57:21.942 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "enabled": { 2024-04-02T13:57:21.943 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "enabled", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.944 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.945 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "24", 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "8", 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "last_opt_revision": { 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "last_opt_revision", 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "1", 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.947 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard": { 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard", 2024-04-02T13:57:21.948 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.949 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "leaderboard_description": { 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "leaderboard_description", 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.950 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.951 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.953 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.954 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.956 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.957 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.958 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "organization": { 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "organization", 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.960 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "proxy": { 2024-04-02T13:57:21.961 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "proxy", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.962 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "url": { 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "url", 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.964 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "test_orchestrator", 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.965 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.966 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.967 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.968 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.970 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.971 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.972 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.973 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.974 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.975 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.976 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.977 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.979 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.980 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.981 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.982 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "max_concurrent_clones": { 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "max_concurrent_clones", 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "4", 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.983 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "periodic_async_work": { 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "periodic_async_work", 2024-04-02T13:57:21.984 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Periodically check for async work", 2024-04-02T13:57:21.985 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_delay": { 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_delay", 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.986 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "0", 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.987 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "snapshot_clone_no_wait": { 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snapshot_clone_no_wait", 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "True", 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.988 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.989 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix", 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "can_run": true, 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "error_string": "", 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "module_options": { 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "discovery_interval": { 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "discovery_interval", 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "uint", 2024-04-02T13:57:21.990 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "100", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.991 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "identifier": { 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "identifier", 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.992 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.993 INFO:teuthology.orchestra.run.smithi122.stdout: "interval": { 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "interval", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "secs", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "60", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.994 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "log_level": { 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_level", 2024-04-02T13:57:21.995 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:21.996 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.997 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster": { 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster", 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:21.998 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:21.999 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_cluster_level": { 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_cluster_level", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "info", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:22.000 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [ 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "critical", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "debug", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "error", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "info", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "warning" 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:22.001 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "log_to_file": { 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "log_to_file", 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "bool", 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:22.002 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 1, 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "False", 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:22.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_host": { 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_host", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:22.004 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_port": { 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_port", 2024-04-02T13:57:22.005 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "int", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "10051", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:22.006 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "zabbix_sender": { 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "zabbix_sender", 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "str", 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "level": "advanced", 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": 0, 2024-04-02T13:57:22.007 INFO:teuthology.orchestra.run.smithi122.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "min": "", 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "max": "", 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "enum_allowed": [], 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "desc": "", 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "long_desc": "", 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "tags": [], 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: "see_also": [] 2024-04-02T13:57:22.008 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "services": { 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "dashboard": "https://172.21.15.122:8443/" 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "always_on_modules": { 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "octopus": [ 2024-04-02T13:57:22.009 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-04-02T13:57:22.010 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "pacific": [ 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-04-02T13:57:22.011 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "quincy": [ 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-04-02T13:57:22.012 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-04-02T13:57:22.013 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "reef": [ 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-04-02T13:57:22.014 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "squid": [ 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "balancer", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "crash", 2024-04-02T13:57:22.015 INFO:teuthology.orchestra.run.smithi122.stdout: "devicehealth", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "orchestrator", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "pg_autoscaler", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "progress", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "rbd_support", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "status", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "telemetry", 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: "volumes" 2024-04-02T13:57:22.016 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "last_failure_osd_epoch": 95, 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "active_clients": [ 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "devicehealth", 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.017 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 2416269565 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "libcephsqlite", 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3983667063 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "rbd_support", 2024-04-02T13:57:22.019 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3517947500 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.020 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 1048105311 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "snap_schedule", 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 32222157 2024-04-02T13:57:22.022 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "name": "volumes", 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "addrvec": [ 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v2", 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "172.21.15.122:0", 2024-04-02T13:57:22.023 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 2184595336 2024-04-02T13:57:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:22.024 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-04-02T13:57:22.024 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:22.026 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:20 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1095979930' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:57:22.026 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:21 smithi187 ceph-mon[36064]: pgmap v700: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 346 op/s 2024-04-02T13:57:22.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:20 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1095979930' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:57:22.026 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:21 smithi188 ceph-mon[36117]: pgmap v700: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 346 op/s 2024-04-02T13:57:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 msgr2=0x7f03600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 0x7f03600a61e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0364005ab0 tx=0x7f0364000bc0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 msgr2=0x7f0374199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:57:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f037000e9f0 tx=0x7f037000edf0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 shutdown_connections 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f03600a3d20 0x7f03600a61e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03741059e0 0x7f03741a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0374101710 0x7f03741997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 --2- 172.21.15.122:0/1095979930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0374100d50 0x7f0374199270 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 >> 172.21.15.122:0/1095979930 conn(0x7f0374076800 msgr2=0x7f03740feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.848+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 shutdown_connections 2024-04-02T13:57:22.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:57:19.849+0000 7f037c488640 1 -- 172.21.15.122:0/1095979930 wait complete. 2024-04-02T13:57:22.035 DEBUG:tasks.check_counter:Getting stats from a 2024-04-02T13:57:22.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.a perf dump 2024-04-02T13:57:22.036 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:20 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1095979930' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-02T13:57:22.036 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:21 smithi122 ceph-mon[32114]: pgmap v700: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 9.0 MiB/s wr, 346 op/s 2024-04-02T13:57:22.304 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:57:23.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:23 smithi122 ceph-mon[32114]: pgmap v701: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 284 op/s 2024-04-02T13:57:23.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:23 smithi188 ceph-mon[36117]: pgmap v701: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 284 op/s 2024-04-02T13:57:23.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:23 smithi187 ceph-mon[36064]: pgmap v701: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 284 op/s 2024-04-02T13:57:24.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:24 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:24.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:24 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:24.988 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 545532, 2024-04-02T13:57:24.988 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 502509, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 172970323, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 312011826, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 10, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 29.149269582, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 11.694336512, 2024-04-02T13:57:24.989 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 20.508442824, 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 1.679639855, 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 502509, 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 39.495584820, 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000078596 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:24.990 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 973889, 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.172953117, 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000177 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:24.991 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 246507, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 345813, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 70002380, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 404260837, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 29, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 6, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 30.989559866, 2024-04-02T13:57:24.992 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 22.363995822, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 8.905065722, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 1.628531989, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 345808, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 31.211381790, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000090256 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:24.993 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 303169, 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.081173437, 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000267 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 697392 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:24.994 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 155814, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 157955, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 46549429, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 275225737, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 80, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 9, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 11.459079953, 2024-04-02T13:57:24.995 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 4.322455230, 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 5.974208333, 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 1.948826725, 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 157955, 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 15.519805265, 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000098254 2024-04-02T13:57:24.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 237560, 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.047084591, 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000198 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1921856, 2024-04-02T13:57:24.997 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 304528 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-02T13:57:24.998 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 75135, 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 91.240084765, 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001214348 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-02T13:57:24.999 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 43554, 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 5.580498715, 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000128128 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 150012, 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 147202, 2024-04-02T13:57:25.000 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 147202, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 730.878968047, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004965142 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 2808, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:25.001 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 48, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 2, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 1359, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 540, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 802, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 17, 2024-04-02T13:57:25.002 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 577, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 209, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 134, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 134, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 39, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 281820, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 158562, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 2799, 2024-04-02T13:57:25.003 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 219, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 4146, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 183229, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 1, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 3, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 101, 2024-04-02T13:57:25.004 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 11, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 408, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 2225, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 721, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 211896219, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 6, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 2225, 2024-04-02T13:57:25.005 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 280, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 280, 2024-04-02T13:57:25.006 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 1945, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 25, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 8969, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 75136, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 46591, 2024-04-02T13:57:25.007 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 22516, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 6729, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 117569 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 1542, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 3608, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 1865, 2024-04-02T13:57:25.008 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 3644, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 2965, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 1, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 22348, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 22347, 2024-04-02T13:57:25.009 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-02T13:57:25.010 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 11, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 2, 2024-04-02T13:57:25.011 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 122698, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 38148, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 36871, 2024-04-02T13:57:25.012 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 85827, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 1277, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 177, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 49, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 46, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 131, 2024-04-02T13:57:25.013 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 7, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 3, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 74936271, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 553045762, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 0, 2024-04-02T13:57:25.014 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 39311, 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 306.532382495, 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.007797623 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 0 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.015 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1168, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 23671, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 22503, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 233, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 1960, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 1727, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 1362, 2024-04-02T13:57:25.016 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 37573, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 36211, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 134, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 24245, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 24111, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 82356, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 223608 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-02T13:57:25.017 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 251291, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 4767, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 150012, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 4593, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 25476, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 22369, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 146.835817038, 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.006564254 2024-04-02T13:57:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 9679, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 158.222277238, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.016346965 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.019 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-02T13:57:25.020 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 40979, 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 95.519716430, 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.002330943 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.022 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.023 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:25.024 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:24 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.025 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.026 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.027 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2, 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.003345469, 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001672734 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-02T13:57:25.028 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 45176, 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 192.210967613, 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004254714 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 5193, 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 34.417759996, 2024-04-02T13:57:25.029 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.006627721 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.030 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.031 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 52, 2024-04-02T13:57:25.033 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 1.463723889, 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.028148536 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.034 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1404, 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 1.667514234, 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.001187688 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-02T13:57:25.035 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 22348, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 100.537846140, 2024-04-02T13:57:25.037 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004498740 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.038 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 2, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 1, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 1, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-02T13:57:25.039 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 20127, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 20127, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 2427, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:25.040 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:25.041 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:25.042 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:25.043 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:25.044 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 353376, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 810, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 440, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 5, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:25.045 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 184, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:25.046 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 4485736, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 36130, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-02T13:57:25.047 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 90522, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 554427295, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 33, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 90522, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 90522, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 1023.459034617, 2024-04-02T13:57:25.048 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.011306191 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 90522, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 92504 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 90522, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 3365, 2024-04-02T13:57:25.049 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 87157, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 0, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 213, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 1140, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 40451, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 635, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-02T13:57:25.050 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 2, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 44852, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 2225, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 340, 2024-04-02T13:57:25.051 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:25.052 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-02T13:57:25.053 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-02T13:57:25.054 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 127, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 37, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 12, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 12, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 1927, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 0, 2024-04-02T13:57:25.055 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 719 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 1, 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-02T13:57:25.056 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 2 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 5, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 44694, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 22347, 2024-04-02T13:57:25.057 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 947849, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 207615948, 2024-04-02T13:57:25.058 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 947849, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 947849, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 207615948, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.059 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:25.060 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 90522, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 556865822, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 90522, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 556865822, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.061 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:25.062 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 90522, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 90522, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 90522, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 90522, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.063 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 22347, 2024-04-02T13:57:25.064 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 2438527, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 22347, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 1140, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 2438527, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.065 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x55cb5fc1c500": { 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 5929, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 122698, 2024-04-02T13:57:25.066 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 548851458, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 122698, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 39311, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 548845529, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:25.067 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:25.068 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:25.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:25 smithi122 ceph-mon[32114]: pgmap v702: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:25.955 INFO:tasks.check_counter:Daemon mds.a mds.exported=3 2024-04-02T13:57:25.955 INFO:tasks.check_counter:Daemon mds.a mds.imported=11 2024-04-02T13:57:25.955 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=1542 2024-04-02T13:57:25.955 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=3608 2024-04-02T13:57:25.955 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-04-02T13:57:25.956 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:25.956 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:25.956 DEBUG:tasks.check_counter:Getting stats from d 2024-04-02T13:57:25.956 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.d perf dump 2024-04-02T13:57:25.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:25 smithi187 ceph-mon[36064]: pgmap v702: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:26.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:25 smithi188 ceph-mon[36117]: pgmap v702: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:26.208 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:57:27.789 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:27 smithi122 ceph-mon[32114]: pgmap v703: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:27.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:27 smithi187 ceph-mon[36064]: pgmap v703: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:28.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:27 smithi188 ceph-mon[36117]: pgmap v703: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 268 op/s 2024-04-02T13:57:28.348 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-04-02T13:57:28.349 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.001131483, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000247379, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:28.350 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 414, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 402, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1508240, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 247904, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-04-02T13:57:28.352 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.062086119, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.022553106, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.037902021, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.007774449, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 402, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.016784664, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000041752 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.353 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1508240, 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 247904 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.354 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 3, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 306, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 480, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 287520, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.022714691, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.016717251, 2024-04-02T13:57:28.355 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.002467763, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 303, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.021622318, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000071360 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:28.356 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 287520 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-04-02T13:57:28.357 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:28.358 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:28.359 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:28.360 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:28.361 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148794, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 32, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:28.362 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-04-02T13:57:28.363 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:28.364 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:28.365 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 413, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1447330, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 413, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 413, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1447330, 2024-04-02T13:57:28.366 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:28.367 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:29.093 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-02T13:57:29.094 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-02T13:57:29.094 DEBUG:tasks.check_counter:Getting stats from g 2024-04-02T13:57:29.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.g perf dump 2024-04-02T13:57:29.338 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:57:29.750 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:29 smithi122 ceph-mon[32114]: pgmap v704: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 224 op/s 2024-04-02T13:57:29.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:29 smithi187 ceph-mon[36064]: pgmap v704: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 224 op/s 2024-04-02T13:57:30.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:29 smithi188 ceph-mon[36117]: pgmap v704: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.8 MiB/s wr, 224 op/s 2024-04-02T13:57:31.138 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:31.138 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:31.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2609, 2024-04-02T13:57:31.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2610, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 193719019, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 829345, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 9, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.341538993, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.079315682, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.171882154, 2024-04-02T13:57:31.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.071175943, 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2608, 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.108474532, 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000041592 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:31.140 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:31.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 2466, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2406, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 161781693, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 878121, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 5, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.303530396, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.060402990, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.187013704, 2024-04-02T13:57:31.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.054046478, 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 2405, 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.077484926, 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000032218 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.143 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1890640, 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 252240 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.144 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 1054, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 1353, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 196924351, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 1031467, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 6, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.226568246, 2024-04-02T13:57:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.035497816, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.154464107, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.027603484, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1351, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.065100461, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000048186 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.146 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 698192 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.147 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 9267, 2024-04-02T13:57:31.148 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.226507932, 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000024442 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.149 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-04-02T13:57:31.150 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:31.151 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 23555, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 3944, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 19611, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 17920, 2024-04-02T13:57:31.152 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 21, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-04-02T13:57:31.153 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-04-02T13:57:31.154 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:57:31.155 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:57:31.156 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-04-02T13:57:31.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-04-02T13:57:31.158 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:31.159 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:31.160 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 122712, 2024-04-02T13:57:31.161 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 131, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 11, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-04-02T13:57:31.162 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 74936271, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 553152329, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 553141826, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.163 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 122712 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 1083, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 23544, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 22461, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 71, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 82, 2024-04-02T13:57:31.164 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 11, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 23555, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 23660, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 105, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 80972, 2024-04-02T13:57:31.165 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215416 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-04-02T13:57:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.167 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.168 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-04-02T13:57:31.169 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.170 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.170 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.171 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:31.171 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.171 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.171 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.172 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.173 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.174 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.175 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.175 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-04-02T13:57:31.176 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.176 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.176 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.177 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.178 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.179 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.180 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.181 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.183 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-04-02T13:57:31.184 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.185 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.186 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 2, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 1, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 1, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-04-02T13:57:31.188 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 2434, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:31.189 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:31.190 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:31.191 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:31.192 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:31.193 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 285109, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 1294, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:31.194 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 184, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-02T13:57:31.195 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 19491877, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 123231, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-04-02T13:57:31.196 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 5624, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 21, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 3, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 5624, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 5624, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 25.535401949, 2024-04-02T13:57:31.197 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.004540434 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 5624, 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 5625 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 5624, 2024-04-02T13:57:31.198 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 5624, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 3195, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 2427, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-04-02T13:57:31.199 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 1, 2024-04-02T13:57:31.200 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-04-02T13:57:31.201 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-04-02T13:57:31.202 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-04-02T13:57:31.203 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 127, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 37, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 11, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 12, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 1, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-04-02T13:57:31.204 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-04-02T13:57:31.205 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-04-02T13:57:31.206 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 6125, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 551845460, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:31.207 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 6125, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 6125, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 551845460, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:31.209 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 5624, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 548980095, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 5624, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 548980095, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.210 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:31.211 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 5624, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 5624, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 5624, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 5624, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.212 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:31.213 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:31.214 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x55b60daf06e0": { 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:31.215 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:31.216 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:31.217 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:31.217 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:31.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:31.217 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:31.821 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:31 smithi122 ceph-mon[32114]: pgmap v705: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 167 op/s 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:31.944 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:31.944 DEBUG:tasks.check_counter:Getting stats from j 2024-04-02T13:57:31.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.j perf dump 2024-04-02T13:57:31.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:31 smithi187 ceph-mon[36064]: pgmap v705: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 167 op/s 2024-04-02T13:57:32.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:31 smithi188 ceph-mon[36117]: pgmap v705: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 167 op/s 2024-04-02T13:57:32.187 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.a/config 2024-04-02T13:57:33.686 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:33 smithi122 ceph-mon[32114]: pgmap v706: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-04-02T13:57:33.686 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:33 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:33.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:33 smithi187 ceph-mon[36064]: pgmap v706: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-04-02T13:57:33.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:33 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:34.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:33 smithi188 ceph-mon[36117]: pgmap v706: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 118 op/s 2024-04-02T13:57:34.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:33 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:34.111 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-04-02T13:57:34.111 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 413, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 403, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 1506096, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 248896, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.064961772, 2024-04-02T13:57:34.112 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.022178319, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.038912557, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.008102082, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 402, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.016174900, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000040236 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.113 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 1506096, 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 248896 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.114 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 7, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 310, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3504, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 288320, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.023186383, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.016978452, 2024-04-02T13:57:34.115 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.002755472, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 308, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019907603, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000064635 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:34.116 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 288320 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-04-02T13:57:34.117 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000371208, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000291750, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:34.118 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:34.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-04-02T13:57:34.120 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:34.121 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:34.122 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:34.123 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-04-02T13:57:34.124 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 149745, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 31, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:34.125 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:34.126 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:34.127 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 416, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 1447784, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:34.128 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 416, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 416, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1447784, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-04-02T13:57:34.129 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:34.130 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-04-02T13:57:34.130 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-04-02T13:57:34.751 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:34 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:34.778 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-02T13:57:34.778 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-02T13:57:34.778 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-02T13:57:34.779 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-02T13:57:34.779 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-02T13:57:34.779 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-02T13:57:34.779 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-02T13:57:34.779 DEBUG:tasks.check_counter:Getting stats from b 2024-04-02T13:57:34.779 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.b perf dump 2024-04-02T13:57:34.857 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:34 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:35.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:34 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:35.093 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:57:35.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:35 smithi187 ceph-mon[36064]: pgmap v707: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 128 KiB/s wr, 39 op/s 2024-04-02T13:57:36.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:35 smithi188 ceph-mon[36117]: pgmap v707: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 128 KiB/s wr, 39 op/s 2024-04-02T13:57:36.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:35 smithi122 ceph-mon[32114]: pgmap v707: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 128 KiB/s wr, 39 op/s 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 466102, 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 517333, 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 135503583, 2024-04-02T13:57:37.232 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 405327946, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 10, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 27.406804928, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 11.805265900, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 17.589450533, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.117746770, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 517335, 2024-04-02T13:57:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 24.374657461, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000047115 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 911771, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.147269045, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000161 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1969328, 2024-04-02T13:57:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 897152 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 371707, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 480564, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 106939026, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 646665593, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 31, 2024-04-02T13:57:37.235 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 7, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 20.434750817, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 9.274302384, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 11.568426558, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.830516844, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 480558, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 34.453073234, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000071693 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.236 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 415650, 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.113617059, 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000273 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 786080, 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 394976 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.237 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 427909, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 504721, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 118543741, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 515696348, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 327, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 8, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 24.318825039, 2024-04-02T13:57:37.238 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 9.751523168, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 15.160731014, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.664531245, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 504720, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 25.675748776, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000050871 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.239 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 843482, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.132171290, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000156 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 45248, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 1424 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-02T13:57:37.240 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 111165, 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 112.704716479, 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001013850 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.241 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 39790, 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 6.498646813, 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000163323 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.242 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 228868, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 191541, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 191541, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1264.301958053, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.006600685 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-02T13:57:37.243 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 37327, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 12, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 2, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 148, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 13, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 9, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 2361, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 1067, 2024-04-02T13:57:37.244 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 1251, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 43, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 836, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 3320, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 181, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 181, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 33, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 358491, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 207765, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 37280, 2024-04-02T13:57:37.245 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 347, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 2947, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 25372, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 23, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 2128, 2024-04-02T13:57:37.246 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 14, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 638, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 4237, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 707, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 222186924, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 6, 2024-04-02T13:57:37.247 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 4237, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 1033, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 41, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 992, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 3204, 2024-04-02T13:57:37.248 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 21, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 42948, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 92574, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 80660, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 19651, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 8503, 2024-04-02T13:57:37.249 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 133310 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 2808, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 1737, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 906, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 1940, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 6249, 2024-04-02T13:57:37.250 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 21084, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 21084, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:37.251 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 41, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 24, 2024-04-02T13:57:37.252 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 22, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 151095, 2024-04-02T13:57:37.253 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 58260, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 49161, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 101934, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 9099, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 259, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 131, 2024-04-02T13:57:37.254 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 118, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 141, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 9, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 13, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 440060997, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 917099382, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-02T13:57:37.255 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 56843, 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 348.796703710, 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.006136141 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 1254, 2024-04-02T13:57:37.256 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 24294, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 23040, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 212, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 1221, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 1009, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 2366, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 45633, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 43267, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 181, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 24966, 2024-04-02T13:57:37.257 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 24785, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 90724, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 231800 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 331232, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 38599, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 228868, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 2853, 2024-04-02T13:57:37.258 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 86558, 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 21836, 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 168.002270279, 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.007693820 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-02T13:57:37.259 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 27439, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 157.386832480, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005735880 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-02T13:57:37.260 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 44280, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 478.511271459, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.010806487 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.261 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1, 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000320869, 2024-04-02T13:57:37.262 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000320869 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.263 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.264 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.264 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.264 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:37.264 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 22, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.014434151, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000656097 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.265 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 119, 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.911737760, 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.007661661 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.266 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 22, 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 12.582814434, 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.571946110 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 10, 2024-04-02T13:57:37.267 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.013426867, 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001342686 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 71607, 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 260.394368111, 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003636437 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-02T13:57:37.268 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4297, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 36.078822769, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.008396281 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:37.269 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000761070, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000761070 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.270 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 16, 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 14.117554646, 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.882347165 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.271 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 35, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 4.684855781, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.133853022 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.272 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 764, 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.712823656, 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002241915 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.273 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 8, 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.020204451, 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002525556 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.274 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 21084, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 129.869459270, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.006159621 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.275 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-04-02T13:57:37.276 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 6, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 4, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 2, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 18297, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 9148, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1322, 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:37.277 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:37.278 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:37.279 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:37.280 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:37.281 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 338440, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 788, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 176, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 2, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-02T13:57:37.282 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 184, 2024-04-02T13:57:37.283 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 5669157, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 53750, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-02T13:57:37.284 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 107445, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 919382266, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 37, 2024-04-02T13:57:37.285 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 107445, 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 107445, 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1027.240872586, 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.009560620 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-02T13:57:37.286 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 107445, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 109233 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 107445, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 5079, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 102366, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2024-04-02T13:57:37.287 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 172, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 787, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 57589, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 438, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 13, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 43072, 2024-04-02T13:57:37.288 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 4285, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 284, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-02T13:57:37.289 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-02T13:57:37.290 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-02T13:57:37.291 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 127, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 42, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2024-04-02T13:57:37.292 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 2066, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 28, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 499 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2024-04-02T13:57:37.293 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 4 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:37.294 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 5, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 42168, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 21084, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:37.295 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1265714, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 252227085, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:37.296 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1265714, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1265714, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 252227085, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.297 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:37.298 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 107445, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 921682838, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 107445, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 921682838, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.299 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:37.300 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 107445, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 107445, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 107445, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 107445, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.301 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 21084, 2024-04-02T13:57:37.302 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 2300772, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 21084, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 746, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2300772, 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:37.303 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x557b2d27c500": { 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:37.304 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 151095, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 912905078, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 151095, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 56843, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 912905078, 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:37.305 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.306 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:37.306 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-02T13:57:37.963 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:37 smithi187 ceph-mon[36064]: pgmap v708: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-04-02T13:57:38.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:37 smithi188 ceph-mon[36117]: pgmap v708: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-04-02T13:57:38.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:37 smithi122 ceph-mon[32114]: pgmap v708: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-04-02T13:57:38.720 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:38 smithi187 ceph-mon[36064]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds.exported=23 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds.imported=14 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=2808 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=1737 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-02T13:57:38.749 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=22 2024-04-02T13:57:38.750 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=16 2024-04-02T13:57:38.750 DEBUG:tasks.check_counter:Getting stats from e 2024-04-02T13:57:38.750 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.e perf dump 2024-04-02T13:57:38.855 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:38 smithi122 ceph-mon[32114]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:39.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:38 smithi188 ceph-mon[36117]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-02T13:57:39.102 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:57:39.942 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:39 smithi187 ceph-mon[36064]: pgmap v709: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 29 op/s 2024-04-02T13:57:40.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:39 smithi188 ceph-mon[36117]: pgmap v709: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 29 op/s 2024-04-02T13:57:40.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:39 smithi122 ceph-mon[32114]: pgmap v709: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 187 KiB/s wr, 29 op/s 2024-04-02T13:57:41.440 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-02T13:57:41.440 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:41.440 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 3333, 2024-04-02T13:57:41.440 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 3335, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 322568777, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1059904, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.401434343, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.089575393, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.207134306, 2024-04-02T13:57:41.441 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.081910621, 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3333, 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.107209342, 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000032166 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.442 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1991, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1925, 2024-04-02T13:57:41.443 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 300112049, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 726788, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.338094128, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.049850637, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.241850552, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.045926344, 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:41.444 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1924, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.075099237, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000039032 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.445 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1930896, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 256400 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1546, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1850, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 294002274, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1193641, 2024-04-02T13:57:41.446 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.302622764, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.039021175, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.210978316, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.038862084, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1848, 2024-04-02T13:57:41.447 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.066949895, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000036228 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T13:57:41.448 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 703920 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:41.449 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 10265, 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.232632153, 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000022662 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:41.450 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-04-02T13:57:41.451 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-04-02T13:57:41.452 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 22353, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 3315, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 19038, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-04-02T13:57:41.453 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 17618, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 30, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-04-02T13:57:41.454 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-04-02T13:57:41.455 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 707, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 222229154, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 6, 2024-04-02T13:57:41.456 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-04-02T13:57:41.457 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:57:41.458 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-04-02T13:57:41.459 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:41.460 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-04-02T13:57:41.461 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-04-02T13:57:41.462 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 151096, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 141, 2024-04-02T13:57:41.463 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 14, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 440060997, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 917180286, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 917099382, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.464 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 151096 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 1134, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 23610, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 22476, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 70, 2024-04-02T13:57:41.465 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 252, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 182, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 22353, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 24394, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 2041, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-04-02T13:57:41.466 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 79048, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 215416 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-04-02T13:57:41.467 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.468 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.469 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.470 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.471 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.472 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-02T13:57:41.473 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.474 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.474 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.474 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.475 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.476 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.477 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.478 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.479 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.481 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.482 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.483 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-02T13:57:41.484 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.485 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 6, 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 4, 2024-04-02T13:57:41.486 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 4, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 3970, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:41.487 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:41.488 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:41.489 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:41.490 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 291924, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 1459, 2024-04-02T13:57:41.491 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:41.492 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 184, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 18946705, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 121897, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-02T13:57:41.493 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 6352, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 26, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 4, 2024-04-02T13:57:41.494 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 6352, 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6352, 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 21.590661523, 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003399033 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-02T13:57:41.495 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 6352, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 6353 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 6352, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 6352, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-02T13:57:41.496 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 3587, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 2762, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-04-02T13:57:41.497 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 2, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:41.498 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:41.499 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-02T13:57:41.500 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-02T13:57:41.501 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 127, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 37, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 11, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-02T13:57:41.502 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-02T13:57:41.503 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-02T13:57:41.504 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:41.505 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 6866, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 916036127, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 6866, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 6866, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 916036127, 2024-04-02T13:57:41.506 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:41.507 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 6352, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 913008706, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 6352, 2024-04-02T13:57:41.508 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 913008706, 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:41.509 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 6352, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 6352, 2024-04-02T13:57:41.510 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 6352, 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 6352, 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.511 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:41.512 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.513 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x5565cd0a46e0": { 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:41.514 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:41.515 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:41.516 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.516 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:41.516 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-02T13:57:41.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:41 smithi187 ceph-mon[36064]: pgmap v710: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 19 op/s 2024-04-02T13:57:42.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:41 smithi188 ceph-mon[36117]: pgmap v710: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 19 op/s 2024-04-02T13:57:42.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:41 smithi122 ceph-mon[32114]: pgmap v710: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 124 KiB/s wr, 19 op/s 2024-04-02T13:57:42.159 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-02T13:57:42.159 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-02T13:57:42.160 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-02T13:57:42.160 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-02T13:57:42.160 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-02T13:57:42.160 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:42.160 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:42.160 DEBUG:tasks.check_counter:Getting stats from h 2024-04-02T13:57:42.160 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.h perf dump 2024-04-02T13:57:42.406 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:57:43.922 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:43 smithi187 ceph-mon[36064]: pgmap v711: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 25 op/s 2024-04-02T13:57:44.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:43 smithi188 ceph-mon[36117]: pgmap v711: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 25 op/s 2024-04-02T13:57:44.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:43 smithi122 ceph-mon[32114]: pgmap v711: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 154 KiB/s wr, 25 op/s 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 431139, 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 412822, 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 132661605, 2024-04-02T13:57:44.163 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 309242890, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 85, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 8, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 27.165595703, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 11.989316552, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 16.814314718, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.562962863, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 412825, 2024-04-02T13:57:44.164 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 29.572150873, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000071633 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 755357, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.124638779, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000165 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-02T13:57:44.165 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 448 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 177059, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 184202, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 51252371, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 293160944, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 11, 2024-04-02T13:57:44.166 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 7, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 12.265764083, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 4.365876300, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 6.910074217, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.911235726, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 184201, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 19.864219712, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000107839 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.167 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 280189, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.056943753, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000203 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1959824, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 318800 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 447181, 2024-04-02T13:57:44.168 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 592473, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 131471949, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 529372836, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 24, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 8, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 25.913427542, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 11.599123835, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 15.775923071, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 1.453007263, 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:44.169 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 592466, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 55.042795684, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000092904 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 652831, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.148372736, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000227 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.170 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 706720 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 72581, 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 109.564893485, 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.001509553 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:44.172 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 51062, 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 7.814038385, 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000153030 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 206854, 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 173903, 2024-04-02T13:57:44.173 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 173903, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1011.343228530, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005815559 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 32950, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:44.174 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 74, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 4, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 2, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 1639, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 676, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 938, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 25, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 674, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 755, 2024-04-02T13:57:44.175 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 136, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 136, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 35, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 344420, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 169351, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 32918, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 288, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:44.176 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 3915, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 8235, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 9, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 360, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 8, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 620, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 2629, 2024-04-02T13:57:44.177 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 721, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 211896219, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 6, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 2629, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:44.178 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 420, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 420, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 2209, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 21, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 28494, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 101351, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:44.179 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 59733, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 16874, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 6378, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 129264 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 442, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 3563, 2024-04-02T13:57:44.180 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 1869, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 3515, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 2908, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 26507, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 26507, 2024-04-02T13:57:44.181 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:44.182 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 9, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 6, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-02T13:57:44.183 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 127027, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 48365, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 36871, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 90156, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 11494, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 195, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 67, 2024-04-02T13:57:44.184 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 50, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 145, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 8, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 17, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 84580856, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 615635664, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 0, 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-02T13:57:44.185 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 39009, 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 361.132585270, 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.009257673 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 0 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 1209, 2024-04-02T13:57:44.186 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 28221, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 27012, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 220, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 1925, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 1705, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 1638, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 51047, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 49409, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 136, 2024-04-02T13:57:44.187 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 28947, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 28811, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 82040, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 223608 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 307214, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 7729, 2024-04-02T13:57:44.188 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 206854, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 2733, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 28143, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 26519, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 154.558504754, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005828217 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.189 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 11167, 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 226.349824872, 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.020269528 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.190 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 53121, 2024-04-02T13:57:44.191 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 212.336337124, 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003997220 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-02T13:57:44.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.193 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.194 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:44.194 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.194 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.194 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.194 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-02T13:57:44.195 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.196 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.197 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-02T13:57:44.198 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 51238, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 272.348310724, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.005315357 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 4707, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 39.401620504, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.008370856 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.199 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.200 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.201 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-02T13:57:44.202 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 84, 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 2.414876069, 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.028748524 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.203 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 560, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1.251085762, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.002234081 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.204 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.205 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 26507, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 102.682668721, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003873794 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:44.206 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-04-02T13:57:44.207 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 2, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 22070, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 11035, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 1279, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-02T13:57:44.208 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:44.209 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:44.210 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:44.211 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 394068, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 742, 2024-04-02T13:57:44.212 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 352, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 4, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:44.213 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 184, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 4801474, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 40446, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:44.214 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 98992, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 617959639, 2024-04-02T13:57:44.215 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 27, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 98992, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 98992, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 1142.313181598, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.011539449 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-02T13:57:44.216 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 98992, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 101002 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 98992, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 3756, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 95236, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-02T13:57:44.217 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 0, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 279, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 1127, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 40136, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 633, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-02T13:57:44.218 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 4, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 53206, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 2629, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 444, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:44.219 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-02T13:57:44.220 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-02T13:57:44.221 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 127, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-02T13:57:44.222 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 37, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 12, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 0, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 1841, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 0, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 703 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.223 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 1, 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.224 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 5, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 53014, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 26507, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.225 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 1055375, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 224911099, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 1055375, 2024-04-02T13:57:44.226 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1055375, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 224911099, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.227 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:44.228 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 98992, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 620851038, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 98992, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 620851038, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.229 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:44.230 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 98992, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 98992, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 98992, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 98992, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.231 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 26507, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 2891399, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:44.232 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 26507, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 1127, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 2891399, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.233 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x563c880d0500": { 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 127027, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 611441360, 2024-04-02T13:57:44.234 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 127027, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 39009, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 611441360, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:44.235 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:44.236 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.236 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:44.236 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds.exported=9 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds.imported=8 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=442 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3563 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-02T13:57:45.051 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:45.052 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:45.052 DEBUG:tasks.check_counter:Getting stats from k 2024-04-02T13:57:45.052 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.k perf dump 2024-04-02T13:57:45.292 INFO:teuthology.orchestra.run.smithi187.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.b/config 2024-04-02T13:57:45.707 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:45 smithi187 ceph-mon[36064]: pgmap v712: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 16 op/s 2024-04-02T13:57:46.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:45 smithi188 ceph-mon[36117]: pgmap v712: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 16 op/s 2024-04-02T13:57:46.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:45 smithi122 ceph-mon[32114]: pgmap v712: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 98 KiB/s wr, 16 op/s 2024-04-02T13:57:47.112 INFO:teuthology.orchestra.run.smithi187.stdout:{ 2024-04-02T13:57:47.112 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:47.112 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1463, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1465, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 184933863, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 465298, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 9, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.209795891, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.036600645, 2024-04-02T13:57:47.113 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.124133398, 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.036811914, 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1463, 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.057551697, 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000039338 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:47.114 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 1495, 2024-04-02T13:57:47.115 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 1698, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 165763946, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1143929, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 5, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.267072581, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.049384975, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.182287806, 2024-04-02T13:57:47.116 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.035362079, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 1698, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.084577039, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000049809 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.117 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 1935008, 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 834192 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_messages": 3221, 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages": 3257, 2024-04-02T13:57:47.118 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_bytes": 264270900, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_bytes": 1152840, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_active_connections": 6, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_total_time": 0.425581035, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_send_time": 0.075519158, 2024-04-02T13:57:47.119 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_recv_time": 0.234789839, 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_running_fast_dispatch_time": 0.090828483, 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 3254, 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.112678935, 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000034627 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:47.120 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "msgr_send_encrypted_bytes": 128880 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "cct": { 2024-04-02T13:57:47.121 INFO:teuthology.orchestra.run.smithi187.stdout: "total_workers": 1, 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "unhealthy_workers": 0 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-MDSRank": { 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 9243, 2024-04-02T13:57:47.122 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.235961339, 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000025528 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "queue_len": 0, 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "complete_latency": { 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.123 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "mds": { 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "request": 0, 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "reply": 0, 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "reply_latency": { 2024-04-02T13:57:47.124 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_reply": 0, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "forward": 0, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:47.125 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_commit": 0, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_split": 0, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_merge": 0, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes": 27817, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_top": 5693, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_bottom": 22124, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pin_tail": 0, 2024-04-02T13:57:47.126 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_pinned": 19685, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_expired": 0, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "inodes_with_caps": 0, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "caps": 0, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "subtrees": 23, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse": 0, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_hit": 0, 2024-04-02T13:57:47.127 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_forward": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_discover": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "traverse_lock": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "load_cent": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "q": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "exported": 0, 2024-04-02T13:57:47.128 INFO:teuthology.orchestra.run.smithi187.stdout: "exported_inodes": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "imported": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "imported_inodes": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rfiles": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rbytes": 0, 2024-04-02T13:57:47.129 INFO:teuthology.orchestra.run.smithi187.stdout: "root_rsnaps": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:47.130 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "scrub_file_inodes": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:47.131 INFO:teuthology.orchestra.run.smithi187.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps": 0, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_cap_release": 0, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "process_request_cap_release": 0 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_cache": { 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update": 0, 2024-04-02T13:57:47.132 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_update_receipt": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_try_discover": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_send_discover": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "dir_handle_discover": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_created": 0, 2024-04-02T13:57:47.133 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_enqueued": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "strays_migrated": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_started": 0, 2024-04-02T13:57:47.134 INFO:teuthology.orchestra.run.smithi187.stdout: "recovery_completed": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_exportdir": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_flush": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:47.135 INFO:teuthology.orchestra.run.smithi187.stdout: "ireq_inodestats": 0 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_log": { 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "evlrg": 0, 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "evadd": 0, 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "evex": 0, 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "evtrm": 0, 2024-04-02T13:57:47.136 INFO:teuthology.orchestra.run.smithi187.stdout: "ev": 127029, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "evexg": 0, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "evexd": 0, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "segadd": 0, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "segex": 0, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "segtrm": 0, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "seg": 145, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "segmjr": 12, 2024-04-02T13:57:47.137 INFO:teuthology.orchestra.run.smithi187.stdout: "segexg": 0, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "segexd": 0, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "expos": 84580856, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "wrpos": 615656250, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "rdpos": 615649443, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "jlat": { 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.138 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "replayed": 127029 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_mem": { 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "ino": 1097, 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "ino+": 27854, 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "ino-": 26757, 2024-04-02T13:57:47.139 INFO:teuthology.orchestra.run.smithi187.stdout: "dir": 80, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "dir+": 111, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "dir-": 31, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "dn": 27818, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "dn+": 28177, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "dn-": 359, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "cap": 0, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "cap+": 0, 2024-04-02T13:57:47.140 INFO:teuthology.orchestra.run.smithi187.stdout: "cap-": 0, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "rss": 80828, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "heap": 215416 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_server": { 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_client_request": 0, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "dispatch_server_request": 0, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_request": 0, 2024-04-02T13:57:47.141 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_client_session": 0, 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: "handle_peer_request": 0, 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: "req_create_latency": { 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.142 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getattr_latency": { 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.143 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: "req_link_latency": { 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.144 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.145 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookup_latency": { 2024-04-02T13:57:47.145 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.145 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.145 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.145 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupino_latency": { 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.146 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupname_latency": { 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.147 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.148 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: "req_lssnap_latency": { 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.149 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.150 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mkdir_latency": { 2024-04-02T13:57:47.150 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.150 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.150 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mknod_latency": { 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "req_mksnap_latency": { 2024-04-02T13:57:47.151 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "req_open_latency": { 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.152 INFO:teuthology.orchestra.run.smithi187.stdout: "req_readdir_latency": { 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rename_latency": { 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.153 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:47.154 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmdir_latency": { 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.155 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.156 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setattr_latency": { 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.157 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.158 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setlayout_latency": { 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "req_setxattr_latency": { 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.159 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "req_symlink_latency": { 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "req_unlink_latency": { 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.160 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.161 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_sessions": { 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "session_count": 2, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "session_add": 3, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "session_remove": 1, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_open": 3, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "sessions_stale": 0, 2024-04-02T13:57:47.162 INFO:teuthology.orchestra.run.smithi187.stdout: "total_load": 0, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "average_load": 0, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "avg_session_uptime": 3316, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "mempool": { 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:47.163 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:47.164 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:47.165 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:47.166 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_items": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:47.167 INFO:teuthology.orchestra.run.smithi187.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_bytes": 291390, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_anon_items": 1426, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_bytes": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_items": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:47.168 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_items": 184, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "pgmap_items": 0, 2024-04-02T13:57:47.169 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_bytes": 22617769, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "mds_co_items": 142684, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_1_items": 0, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "unittest_2_items": 0 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "objecter": { 2024-04-02T13:57:47.170 INFO:teuthology.orchestra.run.smithi187.stdout: "op_active": 0, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_laggy": 0, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send": 5657, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_send_bytes": 21, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_resend": 3, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_reply": 5657, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "op_latency": { 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5657, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 21.100670661, 2024-04-02T13:57:47.171 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.003730010 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "op_inflight": 0, 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "oplen_avg": { 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 5657, 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 5658 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "op": 5657, 2024-04-02T13:57:47.172 INFO:teuthology.orchestra.run.smithi187.stdout: "op_r": 5657, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "op_w": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "op_rmw": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "op_pg": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_stat": 3279, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_create": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_read": 2376, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_write": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writefull": 0, 2024-04-02T13:57:47.173 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_writesame": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_append": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_zero": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_truncate": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_delete": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_mapext": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:47.174 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_getxattr": 1, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_setxattr": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_call": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_watch": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_notify": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:47.175 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "osdop_other": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_active": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_send": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_resend": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "linger_ping": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_active": 0, 2024-04-02T13:57:47.176 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_send": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "poolop_resend": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_active": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_send": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "poolstat_resend": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_active": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_send": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "statfs_resend": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "command_active": 0, 2024-04-02T13:57:47.177 INFO:teuthology.orchestra.run.smithi187.stdout: "command_send": 0, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "command_resend": 0, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "map_epoch": 127, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "map_full": 0, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "map_inc": 37, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_sessions": 11, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_open": 12, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_session_close": 1, 2024-04-02T13:57:47.178 INFO:teuthology.orchestra.run.smithi187.stdout: "osd_laggy": 0, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_wr": 0, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_rd": 2, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_del": 0 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "oft": { 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_objs": 0, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:47.179 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_updates": 0, 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "omap_total_removes": 0 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "osd-slow-ops": { 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "slow_ops_count": 0 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "purge_queue": { 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:47.180 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executing_high_water": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed_ops": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_executed": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:47.181 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 6175, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 614383764, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 6175, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:47.182 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 6175, 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 614383764, 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.183 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 104857600, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 5657, 2024-04-02T13:57:47.184 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 611471807, 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 5657, 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 611471807, 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.185 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 1024, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:47.186 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 5657, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 5657, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 5657, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 5657, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.187 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:47.188 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.189 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: }, 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "throttle-write_buf_throttle-0x55fdba9386e0": { 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "val": 0, 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "max": 3758096384, 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "get_started": 0, 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "get": 0, 2024-04-02T13:57:47.190 INFO:teuthology.orchestra.run.smithi187.stdout: "get_sum": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "take": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "take_sum": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "put": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "put_sum": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "wait": { 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "avgcount": 0, 2024-04-02T13:57:47.191 INFO:teuthology.orchestra.run.smithi187.stdout: "sum": 0.000000000, 2024-04-02T13:57:47.192 INFO:teuthology.orchestra.run.smithi187.stdout: "avgtime": 0.000000000 2024-04-02T13:57:47.192 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.192 INFO:teuthology.orchestra.run.smithi187.stdout: } 2024-04-02T13:57:47.192 INFO:teuthology.orchestra.run.smithi187.stdout:} 2024-04-02T13:57:47.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:47 smithi187 ceph-mon[36064]: pgmap v713: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 15 op/s 2024-04-02T13:57:47.925 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-02T13:57:47.925 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-02T13:57:47.926 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-02T13:57:47.926 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-02T13:57:47.926 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-02T13:57:47.926 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:47.926 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:47.926 DEBUG:tasks.check_counter:Getting stats from c 2024-04-02T13:57:47.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.c perf dump 2024-04-02T13:57:48.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:47 smithi188 ceph-mon[36117]: pgmap v713: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 15 op/s 2024-04-02T13:57:48.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:47 smithi122 ceph-mon[32114]: pgmap v713: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 97 KiB/s wr, 15 op/s 2024-04-02T13:57:48.311 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:57:49.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:49 smithi188 ceph-mon[36117]: pgmap v714: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 22 op/s 2024-04-02T13:57:49.873 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:49 smithi122 ceph-mon[32114]: pgmap v714: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 22 op/s 2024-04-02T13:57:49.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:49 smithi187 ceph-mon[36064]: pgmap v714: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 133 KiB/s wr, 22 op/s 2024-04-02T13:57:50.254 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-02T13:57:50.254 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 384849, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 370719, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 118165461, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 252106378, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 16, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 24.052575261, 2024-04-02T13:57:50.255 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 10.356075408, 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 15.085526180, 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.380047344, 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 370718, 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 27.531144810, 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000074264 2024-04-02T13:57:50.256 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 691746, 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.116107688, 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000167 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1951184, 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 299344 2024-04-02T13:57:50.257 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 187451, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 182713, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 54553883, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 318564389, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 18, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 6, 2024-04-02T13:57:50.258 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 12.452132316, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 4.669276407, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 6.784264906, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.828385717, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 182711, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 19.834732484, 2024-04-02T13:57:50.259 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000108557 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 282427, 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.056911539, 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000201 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 6256, 2024-04-02T13:57:50.260 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 134592 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 422343, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 540627, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 121426947, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 486176836, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 86, 2024-04-02T13:57:50.261 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 23.668732075, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 10.399529933, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 15.070683744, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.124459874, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 540625, 2024-04-02T13:57:50.262 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 53.736718599, 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000099397 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 613058, 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.152784735, 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000249 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.263 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 577536 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.264 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 64414, 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 90.106727977, 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001398868 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.265 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 40664, 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 5.565648160, 2024-04-02T13:57:50.266 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000136869 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 191906, 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 158326, 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-02T13:57:50.267 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 158326, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 708.280147788, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004473555 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 33578, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 2, 2024-04-02T13:57:50.268 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 65, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 4, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 1, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 1384, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 541, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 817, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 26, 2024-04-02T13:57:50.269 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 609, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 577, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 121, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 121, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 24, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 315735, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 159615, 2024-04-02T13:57:50.270 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 33575, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 214, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 3889, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 8715, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 2, 2024-04-02T13:57:50.271 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 9, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 402, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 9, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 521, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 2433, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:50.272 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 721, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 211896219, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 6, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 2433, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:50.273 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 280, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 280, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 2153, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 9, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 27036, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 94217, 2024-04-02T13:57:50.274 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 61652, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 18524, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 8316, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 121350 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.275 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 1259, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 4100, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 2127, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 4035, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 1669, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-02T13:57:50.276 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 21206, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 21206, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:50.277 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 12, 2024-04-02T13:57:50.278 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 5, 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-02T13:57:50.279 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 116548, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 36871, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 36871, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 79677, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 12290, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 175, 2024-04-02T13:57:50.280 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 47, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 47, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 128, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 7, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 17, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 78006737, 2024-04-02T13:57:50.281 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 549707032, 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 33538, 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 282.206838578, 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.008414539 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.282 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 1130, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 22793, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 21663, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 206, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 2166, 2024-04-02T13:57:50.283 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1960, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 1394, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 44070, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 42676, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 97, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 23864, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 23767, 2024-04-02T13:57:50.284 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 80488, 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223608 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 284155, 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 3462, 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 191906, 2024-04-02T13:57:50.285 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 3067, 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 19253, 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 21191, 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 113.336660221, 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005348339 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-02T13:57:50.286 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5759, 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 93.552160761, 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.016244514 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.287 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.288 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 48290, 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 95.015660988, 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001967605 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:50.289 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1, 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000414403, 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000414403 2024-04-02T13:57:50.290 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.291 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:50.292 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.292 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.292 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.292 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.293 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:50.293 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.293 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.293 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.293 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-02T13:57:50.294 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.295 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-02T13:57:50.296 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 56403, 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 288.610395722, 2024-04-02T13:57:50.297 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005116933 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5097, 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 30.129517341, 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005911225 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.298 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.299 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.300 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.301 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 67, 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.844711709, 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.027533010 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.302 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 312, 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.578981077, 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001855708 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.303 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.304 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.305 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 21206, 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 85.211645566, 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.004018279 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:50.306 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-04-02T13:57:50.307 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 20989, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 20989, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 1359, 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:50.308 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 240, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 240, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:50.309 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:50.310 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:50.311 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:50.312 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 337505, 2024-04-02T13:57:50.313 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 731, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 264, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 3, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:50.314 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 184, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 4339371, 2024-04-02T13:57:50.315 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 35262, 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-02T13:57:50.316 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 82380, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 551063125, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 33, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 82380, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 82380, 2024-04-02T13:57:50.317 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1030.227321739, 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.012505794 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 82380, 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 84393 2024-04-02T13:57:50.318 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 82380, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3418, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 78962, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 0, 2024-04-02T13:57:50.319 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 259, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 983, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 34521, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 546, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-02T13:57:50.320 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 4, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 42579, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2433, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 409, 2024-04-02T13:57:50.321 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:50.322 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-02T13:57:50.323 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-02T13:57:50.324 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 127, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 37, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-04-02T13:57:50.325 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 2038, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 4, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 617 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.326 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-02T13:57:50.327 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 5, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 42412, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 21206, 2024-04-02T13:57:50.328 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 994639, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 208916166, 2024-04-02T13:57:50.329 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 994639, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 994639, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 208916166, 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:50.330 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:50.331 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 82380, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 553376803, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 82380, 2024-04-02T13:57:50.332 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 553376803, 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:50.333 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 82380, 2024-04-02T13:57:50.334 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 82380, 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 82380, 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 82380, 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.335 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 21206, 2024-04-02T13:57:50.336 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 2313710, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 21206, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 983, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 2313710, 2024-04-02T13:57:50.337 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x5626c1dac500": { 2024-04-02T13:57:50.338 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 81636, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 116548, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 545512728, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 116548, 2024-04-02T13:57:50.339 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 33538, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 545431092, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:50.340 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:50.341 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.341 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:50.341 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-02T13:57:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:50.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:50 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:50 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-02T13:57:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:50 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds.imported=9 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1259 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=4100 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:51.260 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:51.260 DEBUG:tasks.check_counter:Getting stats from f 2024-04-02T13:57:51.261 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.f perf dump 2024-04-02T13:57:51.521 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:57:51.921 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: pgmap v715: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-04-02T13:57:51.921 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.921 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.921 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:51.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:51.922 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:51 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: pgmap v715: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-04-02T13:57:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:51.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:51.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:51 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: pgmap v715: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 69 KiB/s wr, 14 op/s 2024-04-02T13:57:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:52.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-02T13:57:52.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-02T13:57:52.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:51 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' 2024-04-02T13:57:53.698 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:53 smithi188 ceph-mon[36117]: pgmap v716: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 4110, 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 4112, 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 218424700, 2024-04-02T13:57:53.699 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1306987, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.432227461, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.098058231, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.200123283, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.108350060, 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:53.700 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 4110, 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.137504945, 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000033456 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.701 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 873, 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1178, 2024-04-02T13:57:53.702 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 168003836, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 981480, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.195744069, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.027659640, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.136040791, 2024-04-02T13:57:53.703 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.023161193, 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1176, 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.052387249, 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000044546 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.704 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 705712 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1341, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1275, 2024-04-02T13:57:53.705 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 199275608, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 521146, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 6, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.246193163, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.038930255, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.178378923, 2024-04-02T13:57:53.706 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.031135638, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1274, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.063792720, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000050072 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.707 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1931552, 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 258160 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-02T13:57:53.708 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9574, 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.229766399, 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000023998 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.709 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.711 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-02T13:57:53.712 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 19330, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 3107, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 16223, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 14892, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-04-02T13:57:53.713 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 16, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-04-02T13:57:53.714 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:57:53.715 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:57:53.716 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:53.717 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-04-02T13:57:53.718 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-04-02T13:57:53.719 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-02T13:57:53.720 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-02T13:57:53.721 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 101363, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-02T13:57:53.722 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 130, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 10, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 81897580, 2024-04-02T13:57:53.723 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 586354562, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 586347755, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 101363 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.724 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 1700, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 19383, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 17683, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 89, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 113, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 24, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 19330, 2024-04-02T13:57:53.725 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 19634, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 304, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 75824, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215416 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-02T13:57:53.726 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.727 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:53.728 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.729 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.731 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.732 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.733 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.734 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-02T13:57:53.734 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.735 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.736 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-02T13:57:53.737 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.738 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.739 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.741 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.742 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.743 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-02T13:57:53.744 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.745 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-04-02T13:57:53.746 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 3, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 5302, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-02T13:57:53.747 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:53.748 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:53.749 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:53.750 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:53.751 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 309698, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 1940, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-02T13:57:53.752 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 184, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:53.753 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 17977029, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 120862, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-02T13:57:53.754 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 5804, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 5804, 2024-04-02T13:57:53.755 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5804, 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 15.722714134, 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002708944 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5804, 2024-04-02T13:57:53.756 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 5805 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 5804, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 5804, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 3290, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-04-02T13:57:53.757 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 2512, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-02T13:57:53.758 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-02T13:57:53.759 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-02T13:57:53.760 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-02T13:57:53.761 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 127, 2024-04-02T13:57:53.762 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 37, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 11, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 1, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-04-02T13:57:53.763 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-02T13:57:53.764 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-04-02T13:57:53.765 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 6320, 2024-04-02T13:57:53.766 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 585106201, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 6320, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6320, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 585106201, 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:53.767 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:53.768 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 5804, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 582171718, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 5804, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 582171718, 2024-04-02T13:57:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:53.770 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 5804, 2024-04-02T13:57:53.771 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 5804, 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 5804, 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 5804, 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.772 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:53.773 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.774 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55718e1566e0": { 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-02T13:57:53.776 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:53.777 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-02T13:57:53.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:53 smithi187 ceph-mon[36064]: pgmap v716: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-02T13:57:54.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:53 smithi122 ceph-mon[32114]: pgmap v716: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 104 KiB/s wr, 20 op/s 2024-04-02T13:57:54.506 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:54.507 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:54.507 DEBUG:tasks.check_counter:Getting stats from i 2024-04-02T13:57:54.508 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.i perf dump 2024-04-02T13:57:54.753 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:57:55.899 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:55 smithi188 ceph-mon[36117]: pgmap v717: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-04-02T13:57:55.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:55 smithi187 ceph-mon[36064]: pgmap v717: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-04-02T13:57:56.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:55 smithi122 ceph-mon[32114]: pgmap v717: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 15 op/s 2024-04-02T13:57:56.538 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 391823, 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 338634, 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 122858112, 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 195603287, 2024-04-02T13:57:56.539 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 10, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 20.171166699, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 7.746240071, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 14.824123239, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.707134306, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 338633, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 29.226101013, 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000086306 2024-04-02T13:57:56.540 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 676395, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.116054588, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000171 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1959584, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 298704 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.541 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 270020, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 360909, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 76015984, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 488806618, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 100, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 12, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 16.086980381, 2024-04-02T13:57:56.542 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 7.357138285, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 9.105879330, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.180926817, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 360903, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 35.034520111, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000097074 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.543 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 311461, 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.084830093, 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000272 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 5040, 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 130576 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:56.544 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 289976, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 274168, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 94588394, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 339465722, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 9, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 5, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 18.645615787, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 7.997556593, 2024-04-02T13:57:56.545 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 11.150351395, 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 1.273753953, 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 274170, 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 23.421286096, 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000085426 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:56.546 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 493292, 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.086269418, 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000174 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 581056 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.547 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:56.548 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 56934, 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 77.391568994, 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001359320 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:56.549 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 34138, 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 4.505286895, 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000131972 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 182060, 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 145111, 2024-04-02T13:57:56.550 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 145111, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 747.533724586, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.005151461 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 36949, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 1, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:56.551 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 71, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 4, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 2041, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 922, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 1098, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 21, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 724, 2024-04-02T13:57:56.552 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 417, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 77, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 77, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 37, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 285807, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 154313, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 36942, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 245, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:56.553 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 3334, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 5779, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 9, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 300, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 16, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 1108, 2024-04-02T13:57:56.554 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 2730, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 721, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 211896219, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 6, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 2730, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:56.555 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 280, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 280, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 2450, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 53, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 21345, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 72804, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:56.556 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 47863, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 13641, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 6026, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 117948 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 1689, 2024-04-02T13:57:56.557 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 3155, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 1656, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 3124, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 2467, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 17440, 2024-04-02T13:57:56.558 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 17440, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:56.559 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 12, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 4, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.560 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 101364, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 25622, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 24581, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 76783, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 1041, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 165, 2024-04-02T13:57:56.561 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 37, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 35, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 130, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 7, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 2, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 81897580, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 586367907, 2024-04-02T13:57:56.562 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 0, 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 29677, 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 265.854530601, 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.008958268 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 0 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-02T13:57:56.563 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 1818, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 19592, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 17774, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 225, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 1704, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 1479, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 2041, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 32657, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 30616, 2024-04-02T13:57:56.564 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 77, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 20045, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 19968, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 79328, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 223608 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 260217, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 7300, 2024-04-02T13:57:56.565 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 182060, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 1766, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 24407, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 17377, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 133.383306019, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.007675853 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.566 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 12657, 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 193.485233270, 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.015286816 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.567 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 32092, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 89.558487214, 2024-04-02T13:57:56.568 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002790679 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.569 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.570 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:56.571 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.571 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.571 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.571 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-02T13:57:56.572 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.573 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.574 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 19, 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.033818986, 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.001779946 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 61525, 2024-04-02T13:57:56.575 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 180.676322953, 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.002936632 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3518, 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 32.349390853, 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.009195392 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:56.576 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.577 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.578 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-02T13:57:56.579 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 23, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.488055062, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.021219785 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.580 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 460, 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 1.495263974, 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003250573 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.581 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.582 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 17440, 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 116.063846255, 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.006655037 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.583 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-02T13:57:56.584 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 1, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 19954, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 19954, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 1273, 2024-04-02T13:57:56.585 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:56.586 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:56.587 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:56.588 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-02T13:57:56.589 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 1002005, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 1513, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 88, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 1, 2024-04-02T13:57:56.590 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 184, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:56.591 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 6514036, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 54993, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:56.592 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 71179, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 587236097, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 32, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 71179, 2024-04-02T13:57:56.593 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 71179, 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 923.763259764, 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.012978030 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 71179, 2024-04-02T13:57:56.594 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 73142 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 71179, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 3686, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 67493, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 8, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 249, 2024-04-02T13:57:56.595 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 955, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 30632, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 538, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 4, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 35048, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-02T13:57:56.596 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 2730, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 394, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-02T13:57:56.597 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-02T13:57:56.598 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-02T13:57:56.599 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 127, 2024-04-02T13:57:56.600 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 37, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 12, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 12, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 1959, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-04-02T13:57:56.601 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 623 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 1, 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.602 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 2 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 10, 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 5, 2024-04-02T13:57:56.603 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 34880, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 17440, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:56.604 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 951815, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 211699070, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 951815, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 951815, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 211699070, 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:56.605 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:56.606 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 71179, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 589140545, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 71179, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 589140545, 2024-04-02T13:57:56.607 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:56.608 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 71179, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 71179, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 71179, 2024-04-02T13:57:56.609 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 71179, 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:56.610 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 17440, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 1904464, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 17440, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:56.611 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 955, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 1904464, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:56.612 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x562b15554500": { 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 101364, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 582173603, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 101364, 2024-04-02T13:57:56.613 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 29677, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 582173603, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:56.614 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.615 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:56.615 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-02T13:57:57.294 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-04-02T13:57:57.294 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2024-04-02T13:57:57.294 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=1689 2024-04-02T13:57:57.294 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=3155 2024-04-02T13:57:57.295 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-02T13:57:57.295 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:57:57.295 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:57:57.295 DEBUG:tasks.check_counter:Getting stats from l 2024-04-02T13:57:57.295 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6 shell --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 -- ceph daemon mds.l perf dump 2024-04-02T13:57:57.536 INFO:teuthology.orchestra.run.smithi188.stderr:Inferring config /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/mon.c/config 2024-04-02T13:57:57.977 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:57 smithi188 ceph-mon[36117]: pgmap v718: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-04-02T13:57:57.985 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:57 smithi187 ceph-mon[36064]: pgmap v718: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-04-02T13:57:58.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:57 smithi122 ceph-mon[32114]: pgmap v718: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 71 KiB/s wr, 14 op/s 2024-04-02T13:57:59.471 INFO:teuthology.orchestra.run.smithi188.stdout:{ 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-0": { 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 3146, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 3148, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 219314593, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1000429, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 8, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:59.472 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.394582878, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.081607630, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.198387667, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.090044755, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 3146, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.123013025, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000039101 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.473 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 0 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.474 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-1": { 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1486, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1797, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 201726595, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 1266766, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 10, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 7, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.220584129, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.043761628, 2024-04-02T13:57:59.475 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.128544946, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.037001426, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1794, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.086422127, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000048172 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.476 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 795104 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "AsyncMessenger::Worker-2": { 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_messages": 1639, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages": 1571, 2024-04-02T13:57:59.477 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_bytes": 128051686, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_bytes": 616696, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_created_connections": 5, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_active_connections": 4, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_total_time": 0.481145152, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_send_time": 0.050080046, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_recv_time": 0.146381262, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_running_fast_dispatch_time": 0.284605837, 2024-04-02T13:57:59.478 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_messages_queue_lat": { 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 1571, 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.056642209, 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000036054 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_handle_ack_lat": { 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.479 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_recv_encrypted_bytes": 1935936, 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "msgr_send_encrypted_bytes": 261808 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "cct": { 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "total_workers": 1, 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "unhealthy_workers": 0 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-MDSRank": { 2024-04-02T13:57:59.480 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 9476, 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.231372145, 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000024416 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "finisher-PurgeQueue": { 2024-04-02T13:57:59.481 INFO:teuthology.orchestra.run.smithi188.stdout: "queue_len": 0, 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "complete_latency": { 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "mds": { 2024-04-02T13:57:59.482 INFO:teuthology.orchestra.run.smithi188.stdout: "request": 0, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "reply": 0, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "reply_latency": { 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_reply": 0, 2024-04-02T13:57:59.483 INFO:teuthology.orchestra.run.smithi188.stdout: "forward": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_complete": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_fetch_keys": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_commit": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_split": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_merge": 0, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes": 22371, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_top": 3849, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_bottom": 18522, 2024-04-02T13:57:59.484 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pin_tail": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_pinned": 16872, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_expired": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "inodes_with_caps": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "caps": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "subtrees": 6, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_hit": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_forward": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_discover": 0, 2024-04-02T13:57:59.485 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_dir_fetch": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_remote_ino": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "traverse_lock": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "load_cent": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "q": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "exported": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "exported_inodes": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "imported": 0, 2024-04-02T13:57:59.486 INFO:teuthology.orchestra.run.smithi188.stdout: "imported_inodes": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_dir_fetch": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_backtrace_fetch": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "openino_peer_discover": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rfiles": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rbytes": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "root_rsnaps": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_fetch": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_set_tag": 0, 2024-04-02T13:57:59.487 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_backtrace_repaired": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_inotable_repaired": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_inodes": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dir_base_inodes": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_dirfrag_rstats": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "scrub_file_inodes": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_inode_file_caps": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_revoke": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_grant": 0, 2024-04-02T13:57:59.488 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_trunc": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_caps_dirty": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_cap_release": 0, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "process_request_cap_release": 0 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_cache": { 2024-04-02T13:57:59.489 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_update_receipt": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_try_discover": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_send_discover": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "dir_handle_discover": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_delayed": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "num_strays_enqueuing": 0, 2024-04-02T13:57:59.490 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_created": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_enqueued": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_reintegrated": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "strays_migrated": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_processing": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_enqueued": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "num_recovering_prioritized": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_started": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "recovery_completed": 0, 2024-04-02T13:57:59.491 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_path": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_quiesce_inode": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_enqueue_scrub": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_exportdir": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_flush": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragmentdir": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_fragstats": 0, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "ireq_inodestats": 0 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_log": { 2024-04-02T13:57:59.492 INFO:teuthology.orchestra.run.smithi188.stdout: "evlrg": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "evadd": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "evex": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "evtrm": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "ev": 116552, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "evexg": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "evexd": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "segadd": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "segex": 0, 2024-04-02T13:57:59.493 INFO:teuthology.orchestra.run.smithi188.stdout: "segtrm": 0, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "seg": 128, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "segmjr": 11, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "segexg": 0, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "segexd": 0, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "expos": 78006737, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "wrpos": 549729358, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "rdpos": 549718769, 2024-04-02T13:57:59.494 INFO:teuthology.orchestra.run.smithi188.stdout: "jlat": { 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "replayed": 116552 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_mem": { 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "ino": 1045, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "ino+": 22579, 2024-04-02T13:57:59.495 INFO:teuthology.orchestra.run.smithi188.stdout: "ino-": 21534, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dir": 62, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dir+": 84, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dir-": 22, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dn": 22371, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dn+": 22767, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "dn-": 396, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "cap": 0, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "cap+": 0, 2024-04-02T13:57:59.496 INFO:teuthology.orchestra.run.smithi188.stdout: "cap-": 0, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "rss": 76500, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "heap": 215416 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_server": { 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_client_request": 0, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "dispatch_server_request": 0, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_request": 0, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_client_session": 0, 2024-04-02T13:57:59.497 INFO:teuthology.orchestra.run.smithi188.stdout: "handle_peer_request": 0, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "req_create_latency": { 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getattr_latency": { 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.498 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getfilelock_latency": { 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "req_link_latency": { 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.499 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookup_latency": { 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookuphash_latency": { 2024-04-02T13:57:59.500 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupino_latency": { 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.501 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupname_latency": { 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupparent_latency": { 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.502 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lookupsnap_latency": { 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "req_lssnap_latency": { 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.503 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.504 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.504 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.504 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mkdir_latency": { 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mknod_latency": { 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.505 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "req_mksnap_latency": { 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "req_open_latency": { 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.506 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "req_readdir_latency": { 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rename_latency": { 2024-04-02T13:57:59.507 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "req_renamesnap_latency": { 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.508 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "req_snapdiff_latency": { 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmdir_latency": { 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.509 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmsnap_latency": { 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.510 INFO:teuthology.orchestra.run.smithi188.stdout: "req_rmxattr_latency": { 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setattr_latency": { 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.511 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setdirlayout_latency": { 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setfilelock_latency": { 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.512 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setlayout_latency": { 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "req_setxattr_latency": { 2024-04-02T13:57:59.513 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "req_symlink_latency": { 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.514 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "req_unlink_latency": { 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_revoke_eviction": 0, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "cap_acquisition_throttle": 0, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "req_getvxattr_latency": { 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.515 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_sessions": { 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "session_count": 1, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "session_add": 3, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "session_remove": 2, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_open": 3, 2024-04-02T13:57:59.516 INFO:teuthology.orchestra.run.smithi188.stdout: "sessions_stale": 0, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "total_load": 0, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "average_load": 0, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "avg_session_uptime": 5307, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "mdthresh_evicted": 0 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "mempool": { 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_bytes": 0, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bloom_filter_items": 0, 2024-04-02T13:57:59.517 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_bytes": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_alloc_items": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_bytes": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_data_items": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_onode_items": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_meta_items": 0, 2024-04-02T13:57:59.518 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_bytes": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_other_items": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_cache_buffer_items": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_bytes": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_extent_items": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_bytes": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_blob_items": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-02T13:57:59.519 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_shared_blob_items": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_inline_bl_items": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_bytes": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_fsck_items": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_bytes": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_txc_items": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_deferred_items": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_bytes": 0, 2024-04-02T13:57:59.520 INFO:teuthology.orchestra.run.smithi188.stdout: "bluestore_writing_items": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_bytes": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_items": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_bytes": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_reader_items": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_bytes": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "bluefs_file_writer_items": 0, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_bytes": 284464, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_anon_items": 1258, 2024-04-02T13:57:59.521 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_bytes": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "buffer_meta_items": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_bytes": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_items": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_bytes": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_mapbl_items": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_bytes": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_pglog_items": 0, 2024-04-02T13:57:59.522 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_bytes": 9608, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_items": 184, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_bytes": 0, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "osdmap_mapping_items": 0, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_bytes": 0, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "pgmap_items": 0, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_bytes": 18544603, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "mds_co_items": 117492, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_bytes": 0, 2024-04-02T13:57:59.523 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_1_items": 0, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_bytes": 0, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "unittest_2_items": 0 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "objecter": { 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "op_active": 0, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "op_laggy": 0, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send": 5745, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "op_send_bytes": 21, 2024-04-02T13:57:59.524 INFO:teuthology.orchestra.run.smithi188.stdout: "op_resend": 3, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "op_reply": 5745, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "op_latency": { 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5745, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 18.031672052, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.003138672 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "op_inflight": 0, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "oplen_avg": { 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 5745, 2024-04-02T13:57:59.525 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 5746 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "op": 5745, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "op_r": 5745, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "op_w": 0, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "op_rmw": 0, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "op_pg": 0, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_stat": 3219, 2024-04-02T13:57:59.526 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_create": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_read": 2524, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_write": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writefull": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_writesame": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_append": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_zero": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_truncate": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_delete": 0, 2024-04-02T13:57:59.527 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_mapext": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_sparse_read": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_clonerange": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_getxattr": 1, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_setxattr": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_cmpxattr": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_rmxattr": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_resetxattrs": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_call": 0, 2024-04-02T13:57:59.528 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_watch": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_notify": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_src_cmpxattr": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_pgls_filter": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "osdop_other": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_active": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_send": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_resend": 0, 2024-04-02T13:57:59.529 INFO:teuthology.orchestra.run.smithi188.stdout: "linger_ping": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_active": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_send": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolop_resend": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_active": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_send": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "poolstat_resend": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_active": 0, 2024-04-02T13:57:59.530 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_send": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "statfs_resend": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "command_active": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "command_send": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "command_resend": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "map_epoch": 127, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "map_full": 0, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "map_inc": 37, 2024-04-02T13:57:59.531 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_sessions": 11, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_open": 11, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_session_close": 0, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "osd_laggy": 0, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_wr": 0, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_rd": 2, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_del": 0 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "oft": { 2024-04-02T13:57:59.532 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_objs": 0, 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_kv_pairs": 0, 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_updates": 0, 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "omap_total_removes": 0 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "osd-slow-ops": { 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "slow_ops_count": 0 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.533 INFO:teuthology.orchestra.run.smithi188.stdout: "purge_queue": { 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_ops_high_water": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executing_high_water": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed_ops": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_executed": 0, 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: "pq_item_in_journal": 0 2024-04-02T13:57:59.534 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 6267, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 548499398, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 6267, 2024-04-02T13:57:59.535 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 6267, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 548499398, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.536 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_bytes": { 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 104857600, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:59.537 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 5745, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 545574068, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 5745, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 545574068, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.538 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-objecter_ops": { 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 1024, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:59.539 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 5745, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 5745, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 5745, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 5745, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.540 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle": { 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:59.541 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.542 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: }, 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "throttle-write_buf_throttle-0x55b1107846e0": { 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "val": 0, 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "max": 3758096384, 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "get_started": 0, 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "get": 0, 2024-04-02T13:57:59.543 INFO:teuthology.orchestra.run.smithi188.stdout: "get_sum": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_fail": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "get_or_fail_success": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "take": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "take_sum": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "put": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "put_sum": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "wait": { 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "avgcount": 0, 2024-04-02T13:57:59.544 INFO:teuthology.orchestra.run.smithi188.stdout: "sum": 0.000000000, 2024-04-02T13:57:59.545 INFO:teuthology.orchestra.run.smithi188.stdout: "avgtime": 0.000000000 2024-04-02T13:57:59.545 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.545 INFO:teuthology.orchestra.run.smithi188.stdout: } 2024-04-02T13:57:59.545 INFO:teuthology.orchestra.run.smithi188.stdout:} 2024-04-02T13:57:59.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:57:59 smithi188 ceph-mon[36117]: pgmap v719: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 20 op/s 2024-04-02T13:57:59.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:57:59 smithi187 ceph-mon[36064]: pgmap v719: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 20 op/s 2024-04-02T13:58:00.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:57:59 smithi122 ceph-mon[32114]: pgmap v719: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 20 op/s 2024-04-02T13:58:00.274 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-02T13:58:00.275 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-02T13:58:00.276 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-02T13:58:00.286 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-02T13:58:00.287 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:58:00.287 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:58:00.315 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-04-02T13:58:00.315 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-02T13:58:00.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:58:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- 172.21.15.122:0/3582665251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c107aa0 msgr2=0x7f5d9c107e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/3582665251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c107e80 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009930 tx=0x7f5d9802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- 172.21.15.122:0/3582665251 shutdown_connections 2024-04-02T13:58:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/3582665251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/3582665251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c101a90 0x7f5d9c101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/3582665251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c107e80 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- 172.21.15.122:0/3582665251 >> 172.21.15.122:0/3582665251 conn(0x7f5d9c078f00 msgr2=0x7f5d9c079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- 172.21.15.122:0/3582665251 shutdown_connections 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- 172.21.15.122:0/3582665251 wait complete. 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 Processor -- start 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.528+0000 7f5da3b4d640 1 -- start start 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.534 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c111e80 con 0x7f5d9c101a90 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c111d00 con 0x7f5d9c102450 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c112000 con 0x7f5d9c107aa0 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51270/0 (socket says 172.21.15.122:51270) 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 -- 172.21.15.122:0/4219214289 learned_addr learned my addr 172.21.15.122:0/4219214289 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da10e8640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.535 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 -- 172.21.15.122:0/4219214289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c107aa0 msgr2=0x7f5d9c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da20ea640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 -- 172.21.15.122:0/4219214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 msgr2=0x7f5d9c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 -- 172.21.15.122:0/4219214289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c19d340 con 0x7f5d9c101a90 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da10e8640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da18e9640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009230 tx=0x7f5d9802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98041070 con 0x7f5d9c101a90 2024-04-02T13:58:00.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c19d5d0 con 0x7f5d9c101a90 2024-04-02T13:58:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d98004450 con 0x7f5d9c101a90 2024-04-02T13:58:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.529+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98034440 con 0x7f5d9c101a90 2024-04-02T13:58:00.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.530+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1a4ec0 con 0x7f5d9c101a90 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.531+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c068d40 con 0x7f5d9c101a90 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.532+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5d98044070 con 0x7f5d9c101a90 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.532+0000 7f5d92ffd640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.532+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f5d980f2650 con 0x7f5d9c101a90 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.532+0000 7f5da10e8640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.533+0000 7f5da10e8640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 0x7f5d880a61e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c103d20 tx=0x7f5d8c00a5c0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:00.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.537+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5d980bea10 con 0x7f5d9c101a90 2024-04-02T13:58:00.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.722+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d9c10a000 con 0x7f5d9c101a90 2024-04-02T13:58:00.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.724+0000 7f5d92ffd640 1 -- 172.21.15.122:0/4219214289 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+25415 (secure 0 0 0) 0x7f5d980be310 con 0x7f5d9c101a90 2024-04-02T13:58:00.727 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:58:00.727 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":127,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:57:16.447119+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]},{"pool":4,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:58:00.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 msgr2=0x7f5d880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 0x7f5d880a61e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c103d20 tx=0x7f5d8c00a5c0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 msgr2=0x7f5d9c1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009230 tx=0x7f5d9802cc70 comp rx=0 tx=0).stop 2024-04-02T13:58:00.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 shutdown_connections 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5d880a3d20 0x7f5d880a61e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d9c107aa0 0x7f5d9c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d9c102450 0x7f5d9c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 --2- 172.21.15.122:0/4219214289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d9c101a90 0x7f5d9c1983a0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 >> 172.21.15.122:0/4219214289 conn(0x7f5d9c078f00 msgr2=0x7f5d9c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 shutdown_connections 2024-04-02T13:58:00.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.726+0000 7f5da3b4d640 1 -- 172.21.15.122:0/4219214289 wait complete. 2024-04-02T13:58:00.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-02T13:28:14.467087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-02T13:32:47.895085+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-02T13:32:48.813983+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-02T13:32:52.540451+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-02T13:58:00.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-02T13:58:00.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- 172.21.15.122:0/3962998028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d88068ce0 msgr2=0x7f0d88069160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 --2- 172.21.15.122:0/3962998028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d88068ce0 0x7f0d88069160 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009930 tx=0x7f0d7802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:00.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- 172.21.15.122:0/3962998028 shutdown_connections 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 --2- 172.21.15.122:0/3962998028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d880696a0 0x7f0d8810e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 --2- 172.21.15.122:0/3962998028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d88068ce0 0x7f0d88069160 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 --2- 172.21.15.122:0/3962998028 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88107a80 0x7f0d88107e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- 172.21.15.122:0/3962998028 >> 172.21.15.122:0/3962998028 conn(0x7f0d88077b00 msgr2=0x7f0d88077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- 172.21.15.122:0/3962998028 shutdown_connections 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- 172.21.15.122:0/3962998028 wait complete. 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 Processor -- start 2024-04-02T13:58:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.842+0000 7f0d8e738640 1 -- start start 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 0x7f0d88198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 0x7f0d8819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111e60 con 0x7f0d880696a0 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111ce0 con 0x7f0d88068ce0 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d88111fe0 con 0x7f0d88107a80 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51296/0 (socket says 172.21.15.122:51296) 2024-04-02T13:58:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 -- 172.21.15.122:0/1156216675 learned_addr learned my addr 172.21.15.122:0/1156216675 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8ccd5640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 0x7f0d8819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d87fff640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 0x7f0d88198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 -- 172.21.15.122:0/1156216675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 msgr2=0x7f0d8819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 0x7f0d8819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 -- 172.21.15.122:0/1156216675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 msgr2=0x7f0d88198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 0x7f0d88198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 -- 172.21.15.122:0/1156216675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8819d370 con 0x7f0d880696a0 2024-04-02T13:58:00.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8ccd5640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 0x7f0d8819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d87fff640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 0x7f0d88198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d877fe640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009230 tx=0x7f0d7802cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78041070 con 0x7f0d880696a0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8819d600 con 0x7f0d880696a0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d78004450 con 0x7f0d880696a0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78034440 con 0x7f0d880696a0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.843+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d881a4ef0 con 0x7f0d880696a0 2024-04-02T13:58:00.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.844+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d880688f0 con 0x7f0d880696a0 2024-04-02T13:58:00.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.845+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f0d78044070 con 0x7f0d880696a0 2024-04-02T13:58:00.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.846+0000 7f0d857fa640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:00.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.846+0000 7f0d87fff640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:00.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.846+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f0d780f2650 con 0x7f0d880696a0 2024-04-02T13:58:00.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.846+0000 7f0d87fff640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 0x7f0d700a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00baa0 tx=0x7f0d7c005d20 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:00.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:00.850+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f0d780bea10 con 0x7f0d880696a0 2024-04-02T13:58:01.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.040+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d88109a70 con 0x7f0d880696a0 2024-04-02T13:58:01.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.040+0000 7f0d857fa640 1 -- 172.21.15.122:0/1156216675 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) ==== 93+0+10 (secure 0 0 0) 0x7f0d780be310 con 0x7f0d880696a0 2024-04-02T13:58:01.043 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-04-02T13:58:01.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 msgr2=0x7f0d700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 0x7f0d700a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00baa0 tx=0x7f0d7c005d20 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 msgr2=0x7f0d881988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009230 tx=0x7f0d7802cc70 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 shutdown_connections 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f0d700a3d20 0x7f0d700a61e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d88107a80 0x7f0d8819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d880696a0 0x7f0d881988c0 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 --2- 172.21.15.122:0/1156216675 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0d88068ce0 0x7f0d88198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 >> 172.21.15.122:0/1156216675 conn(0x7f0d88077b00 msgr2=0x7f0d880fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 shutdown_connections 2024-04-02T13:58:01.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.042+0000 7f0d8e738640 1 -- 172.21.15.122:0/1156216675 wait complete. 2024-04-02T13:58:01.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-02T13:58:01.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 -- 172.21.15.122:0/4142441238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108100970 msgr2=0x7fa108100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 --2- 172.21.15.122:0/4142441238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108100970 0x7fa108100df0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009930 tx=0x7fa0f802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:01.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 -- 172.21.15.122:0/4142441238 shutdown_connections 2024-04-02T13:58:01.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 --2- 172.21.15.122:0/4142441238 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108101330 0x7fa10810d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 --2- 172.21.15.122:0/4142441238 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108100970 0x7fa108100df0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.178 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 --2- 172.21.15.122:0/4142441238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108106980 0x7fa108106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 -- 172.21.15.122:0/4142441238 >> 172.21.15.122:0/4142441238 conn(0x7fa108077b00 msgr2=0x7fa108077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 -- 172.21.15.122:0/4142441238 shutdown_connections 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.173+0000 7fa10f80d640 1 -- 172.21.15.122:0/4142441238 wait complete. 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 Processor -- start 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 -- start start 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 0x7fa1081972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108106980 0x7fa10819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110d90 con 0x7fa108101330 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110c10 con 0x7fa108106980 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10f80d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa108110f10 con 0x7fa108100970 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51310/0 (socket says 172.21.15.122:51310) 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 -- 172.21.15.122:0/479527011 learned_addr learned my addr 172.21.15.122:0/479527011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10d5a9640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 0x7fa1081972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10ddaa640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108106980 0x7fa10819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 -- 172.21.15.122:0/479527011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 msgr2=0x7fa1081972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 0x7fa1081972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 -- 172.21.15.122:0/479527011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108106980 msgr2=0x7fa10819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108106980 0x7fa10819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 -- 172.21.15.122:0/479527011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10819c2a0 con 0x7fa108101330 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10d5a9640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 0x7fa1081972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.174+0000 7fa10cda8640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009230 tx=0x7fa0f802cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.175+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8041070 con 0x7fa108101330 2024-04-02T13:58:01.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.175+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa10819c530 con 0x7fa108101330 2024-04-02T13:58:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.175+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8004450 con 0x7fa108101330 2024-04-02T13:58:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.175+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8034440 con 0x7fa108101330 2024-04-02T13:58:01.182 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.175+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa108068c60 con 0x7fa108101330 2024-04-02T13:58:01.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.176+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1080688f0 con 0x7fa108101330 2024-04-02T13:58:01.185 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.177+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa0f8044070 con 0x7fa108101330 2024-04-02T13:58:01.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.178+0000 7fa0fe7fc640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 0x7fa0f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.178+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fa0f80f2480 con 0x7fa108101330 2024-04-02T13:58:01.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.178+0000 7fa10d5a9640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 0x7fa0f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.178+0000 7fa10d5a9640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 0x7fa0f40a61e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa100009f40 tx=0x7fa100010040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.183+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa0f80be1f0 con 0x7fa108101330 2024-04-02T13:58:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.364+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa108069680 con 0x7fa108101330 2024-04-02T13:58:01.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.364+0000 7fa0fe7fc640 1 -- 172.21.15.122:0/479527011 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) ==== 104+0+11 (secure 0 0 0) 0x7fa0f80c24f0 con 0x7fa108101330 2024-04-02T13:58:01.367 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:58:01.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 msgr2=0x7fa0f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 0x7fa0f40a61e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa100009f40 tx=0x7fa100010040 comp rx=0 tx=0).stop 2024-04-02T13:58:01.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 msgr2=0x7fa1081977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009230 tx=0x7fa0f802cba0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 shutdown_connections 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa0f40a3d20 0x7fa0f40a61e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108106980 0x7fa10819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa108101330 0x7fa1081977f0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 --2- 172.21.15.122:0/479527011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108100970 0x7fa1081972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 >> 172.21.15.122:0/479527011 conn(0x7fa108077b00 msgr2=0x7fa108103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 shutdown_connections 2024-04-02T13:58:01.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.366+0000 7fa10f80d640 1 -- 172.21.15.122:0/479527011 wait complete. 2024-04-02T13:58:01.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-02T13:58:01.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 -- 172.21.15.122:0/2167157258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 msgr2=0x7fb028103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 --2- 172.21.15.122:0/2167157258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb028103120 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fb024009900 tx=0x7fb02402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 -- 172.21.15.122:0/2167157258 shutdown_connections 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 --2- 172.21.15.122:0/2167157258 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 --2- 172.21.15.122:0/2167157258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 0x7fb02810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 --2- 172.21.15.122:0/2167157258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb028103120 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 -- 172.21.15.122:0/2167157258 >> 172.21.15.122:0/2167157258 conn(0x7fb028076d80 msgr2=0x7fb028077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 -- 172.21.15.122:0/2167157258 shutdown_connections 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.483+0000 7fb0304ba640 1 -- 172.21.15.122:0/2167157258 wait complete. 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 Processor -- start 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 -- start start 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 0x7fb0281988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111ea0 con 0x7fb028102d40 2024-04-02T13:58:01.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111d20 con 0x7fb02810c110 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb0304ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb028112020 con 0x7fb028103660 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51332/0 (socket says 172.21.15.122:51332) 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 -- 172.21.15.122:0/4105084184 learned_addr learned my addr 172.21.15.122:0/4105084184 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02ea57640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02da55640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 0x7fb0281988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 -- 172.21.15.122:0/4105084184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 msgr2=0x7fb0281988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 0x7fb0281988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.491 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 -- 172.21.15.122:0/4105084184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 msgr2=0x7fb02819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02e256640 1 -- 172.21.15.122:0/4105084184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02819d390 con 0x7fb028102d40 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.484+0000 7fb02ea57640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb02e256640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fb024005940 tx=0x7fb02402cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024041070 con 0x7fb028102d40 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb024004450 con 0x7fb028102d40 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02819d620 con 0x7fb028102d40 2024-04-02T13:58:01.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024003df0 con 0x7fb028102d40 2024-04-02T13:58:01.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.485+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281a4f10 con 0x7fb028102d40 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.486+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02806a8f0 con 0x7fb028102d40 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.487+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fb024044070 con 0x7fb028102d40 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.488+0000 7fb01f7fe640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 0x7fb0140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.488+0000 7fb02da55640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 0x7fb0140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.488+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fb0240f28b0 con 0x7fb028102d40 2024-04-02T13:58:01.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.488+0000 7fb02da55640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 0x7fb0140a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb0281001d0 tx=0x7fb018005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.492+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fb0240bec70 con 0x7fb028102d40 2024-04-02T13:58:01.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.674+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb028107200 con 0x7fb028102d40 2024-04-02T13:58:01.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.675+0000 7fb01f7fe640 1 -- 172.21.15.122:0/4105084184 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) ==== 100+0+11 (secure 0 0 0) 0x7fb0240be570 con 0x7fb028102d40 2024-04-02T13:58:01.678 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-04-02T13:58:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 msgr2=0x7fb0140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 0x7fb0140a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb0281001d0 tx=0x7fb018005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 msgr2=0x7fb0281983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7fb024005940 tx=0x7fb02402cc40 comp rx=0 tx=0).stop 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 shutdown_connections 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fb0140a3d20 0x7fb0140a61e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb02810c110 0x7fb02819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb028103660 0x7fb0281988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 --2- 172.21.15.122:0/4105084184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb028102d40 0x7fb0281983a0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 >> 172.21.15.122:0/4105084184 conn(0x7fb028076d80 msgr2=0x7fb0280fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 shutdown_connections 2024-04-02T13:58:01.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.677+0000 7fb0304ba640 1 -- 172.21.15.122:0/4105084184 wait complete. 2024-04-02T13:58:01.691 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-02T13:58:01.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 -- 172.21.15.122:0/4013696996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58102d40 msgr2=0x7fae58103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/4013696996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58102d40 0x7fae58103120 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fae54009900 tx=0x7fae5402bcd0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 -- 172.21.15.122:0/4013696996 shutdown_connections 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/4013696996 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae5810c110 0x7fae5810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/4013696996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae58103660 0x7fae5810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/4013696996 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58102d40 0x7fae58103120 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 -- 172.21.15.122:0/4013696996 >> 172.21.15.122:0/4013696996 conn(0x7fae58076d80 msgr2=0x7fae58077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 -- 172.21.15.122:0/4013696996 shutdown_connections 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.792+0000 7fae5fa8f640 1 -- 172.21.15.122:0/4013696996 wait complete. 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 Processor -- start 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 -- start start 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae58102d40 0x7fae58100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5810c110 0x7fae58101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae58111ea0 con 0x7fae58103660 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae58111d20 con 0x7fae58102d40 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5fa8f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae58112020 con 0x7fae5810c110 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5d02a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5d02a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51354/0 (socket says 172.21.15.122:51354) 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5d02a640 1 -- 172.21.15.122:0/3828580094 learned_addr learned my addr 172.21.15.122:0/3828580094 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:01.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5d82b640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae58102d40 0x7fae58100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.793+0000 7fae5e02c640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5810c110 0x7fae58101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 -- 172.21.15.122:0/3828580094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5810c110 msgr2=0x7fae58101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5810c110 0x7fae58101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 -- 172.21.15.122:0/3828580094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae58102d40 msgr2=0x7fae58100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae58102d40 0x7fae58100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 -- 172.21.15.122:0/3828580094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae581a6800 con 0x7fae58103660 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5d02a640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c920 tx=0x7fae4800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48007c70 con 0x7fae58103660 2024-04-02T13:58:01.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae48007e10 con 0x7fae58103660 2024-04-02T13:58:01.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae581a6af0 con 0x7fae58103660 2024-04-02T13:58:01.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae48018890 con 0x7fae58103660 2024-04-02T13:58:01.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.794+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a7090 con 0x7fae58103660 2024-04-02T13:58:01.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.795+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae58068d40 con 0x7fae58103660 2024-04-02T13:58:01.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.796+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fae48018a30 con 0x7fae58103660 2024-04-02T13:58:01.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.797+0000 7fae4effd640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 0x7fae440a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.797+0000 7fae5d82b640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 0x7fae440a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.797+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fae480d0e80 con 0x7fae58103660 2024-04-02T13:58:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.797+0000 7fae5d82b640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 0x7fae440a6290 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fae5402efd0 tx=0x7fae5402cf90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:01.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.800+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae48014030 con 0x7fae58103660 2024-04-02T13:58:01.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.979+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fae58106c70 con 0x7fae58103660 2024-04-02T13:58:01.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.979+0000 7fae4effd640 1 -- 172.21.15.122:0/3828580094 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v127) ==== 103+0+11 (secure 0 0 0) 0x7fae4809d240 con 0x7fae58103660 2024-04-02T13:58:01.984 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-04-02T13:58:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:01 smithi187 ceph-mon[36064]: pgmap v720: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-04-02T13:58:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4219214289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1156216675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:58:01.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:01 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/479527011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:58:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 msgr2=0x7fae440a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 0x7fae440a6290 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fae5402efd0 tx=0x7fae5402cf90 comp rx=0 tx=0).stop 2024-04-02T13:58:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 msgr2=0x7fae58100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fae4800c920 tx=0x7fae4800cde0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 shutdown_connections 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae440a3dd0 0x7fae440a6290 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae5810c110 0x7fae58101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae58103660 0x7fae58100ed0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 --2- 172.21.15.122:0/3828580094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae58102d40 0x7fae58100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.981+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 >> 172.21.15.122:0/3828580094 conn(0x7fae58076d80 msgr2=0x7fae580fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.982+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 shutdown_connections 2024-04-02T13:58:01.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:01.982+0000 7fae5fa8f640 1 -- 172.21.15.122:0/3828580094 wait complete. 2024-04-02T13:58:01.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:01 smithi122 ceph-mon[32114]: pgmap v720: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-04-02T13:58:01.995 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4219214289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:01.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1156216675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:58:01.996 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:01 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/479527011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:58:01.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-02T13:58:02.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:01 smithi188 ceph-mon[36117]: pgmap v720: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 113 KiB/s wr, 13 op/s 2024-04-02T13:58:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4219214289' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1156216675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:01 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/479527011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/2309973119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4107ab0 msgr2=0x7fbfa4107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/2309973119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4107ab0 0x7fbfa4107e90 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c009cd0 tx=0x7fbf9c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/2309973119 shutdown_connections 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/2309973119 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4102460 0x7fbfa410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/2309973119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4101aa0 0x7fbfa4101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/2309973119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4107ab0 0x7fbfa4107e90 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/2309973119 >> 172.21.15.122:0/2309973119 conn(0x7fbfa40fd530 msgr2=0x7fbfa40ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:02.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/2309973119 shutdown_connections 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.095+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/2309973119 wait complete. 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 Processor -- start 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 -- start start 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4102460 0x7fbfa419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 0x7fbfa419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4111ec0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4111d40 con 0x7fbfa4107ab0 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfaa1f6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa4112040 con 0x7fbfa4102460 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.096+0000 7fbfa37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51374/0 (socket says 172.21.15.122:51374) 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 -- 172.21.15.122:0/569912787 learned_addr learned my addr 172.21.15.122:0/569912787 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa2ffd640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4102460 0x7fbfa419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa3fff640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 0x7fbfa419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 -- 172.21.15.122:0/569912787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4102460 msgr2=0x7fbfa419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4102460 0x7fbfa419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 -- 172.21.15.122:0/569912787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 msgr2=0x7fbfa419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 0x7fbfa419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 -- 172.21.15.122:0/569912787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa419f630 con 0x7fbfa4101aa0 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa3fff640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 0x7fbfa419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa37fe640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c004af0 tx=0x7fbf9c0324f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c039040 con 0x7fbfa4101aa0 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa419f8c0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf9c035b60 con 0x7fbfa4101aa0 2024-04-02T13:58:02.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c0042a0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.097+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa41a71b0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.098+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa41031e0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.099+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fbf9c046030 con 0x7fbfa4101aa0 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.100+0000 7fbfa0ff9640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 0x7fbf900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.100+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fbf9c0f2440 con 0x7fbfa4101aa0 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.100+0000 7fbfa2ffd640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 0x7fbf900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.100+0000 7fbfa2ffd640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 0x7fbf900a61e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005fd0 tx=0x7fbf8c005850 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:02.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.104+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fbf9c04d060 con 0x7fbfa4101aa0 2024-04-02T13:58:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.295+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbfa40688f0 con 0x7fbfa4101aa0 2024-04-02T13:58:02.299 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.295+0000 7fbfa0ff9640 1 -- 172.21.15.122:0/569912787 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v127) ==== 81+0+960 (secure 0 0 0) 0x7fbf9c0be890 con 0x7fbfa4101aa0 2024-04-02T13:58:02.299 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/3450525377 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.299 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/3450525377 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3229862468 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3421882822 2024-04-03T13:25:54.372176+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6801/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/684561021 2024-04-03T13:25:27.690608+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/3032683818 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/776403907 2024-04-03T13:25:54.372176+0000 2024-04-02T13:58:02.300 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/4063470519 2024-04-03T13:25:54.372176+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1783168866 2024-04-03T13:25:54.372176+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1773721972 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:6800/1645681773 2024-04-03T13:25:27.690608+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1474593903 2024-04-03T13:25:27.690608+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/2213768328 2024-04-03T13:35:09.812961+0000 2024-04-02T13:58:02.301 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.15.122:0/1325797276 2024-04-03T13:25:27.690608+0000 2024-04-02T13:58:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:listed 16 entries 2024-04-02T13:58:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 msgr2=0x7fbf900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 0x7fbf900a61e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c005fd0 tx=0x7fbf8c005850 comp rx=0 tx=0).stop 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 msgr2=0x7fbfa419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c004af0 tx=0x7fbf9c0324f0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 shutdown_connections 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fbf900a3d20 0x7fbf900a61e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfa4107ab0 0x7fbfa419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa4102460 0x7fbfa419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.303 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 --2- 172.21.15.122:0/569912787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbfa4101aa0 0x7fbfa419a640 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 >> 172.21.15.122:0/569912787 conn(0x7fbfa40fd530 msgr2=0x7fbfa410cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:02.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 shutdown_connections 2024-04-02T13:58:02.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.297+0000 7fbfaa1f6640 1 -- 172.21.15.122:0/569912787 wait complete. 2024-04-02T13:58:02.312 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-02T13:58:02.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:02.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-02T13:58:02.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T13:58:02.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:02.603 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-02T13:58:02.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:02.639 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-04-02T13:58:02.744 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-02T13:58:02.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:02.745 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-02T13:58:02.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-02T13:58:02.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-04-02T13:58:02.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:02 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/4105084184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:02 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3828580094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.853 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:02 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/569912787' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:58:02.873 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-02T13:58:02.883 INFO:tasks.cephadm:Teardown begin 2024-04-02T13:58:02.884 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-02T13:58:02.884 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:58:02.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:02 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/4105084184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:02 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3828580094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:58:02.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:02 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/569912787' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:58:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- 172.21.15.122:0/3053940751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 msgr2=0x7f5f80106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3053940751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f80106df0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009930 tx=0x7f5f7002bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- 172.21.15.122:0/3053940751 shutdown_connections 2024-04-02T13:58:02.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3053940751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 0x7f5f8010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3053940751 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3053940751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f80106df0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- 172.21.15.122:0/3053940751 >> 172.21.15.122:0/3053940751 conn(0x7f5f80077b00 msgr2=0x7f5f80077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- 172.21.15.122:0/3053940751 shutdown_connections 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- 172.21.15.122:0/3053940751 wait complete. 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 Processor -- start 2024-04-02T13:58:02.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.986+0000 7f5f860a5640 1 -- start start 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 0x7f5f80198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f80110e20 con 0x7f5f80106a10 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5f80110ca0 con 0x7f5f80100a00 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f80110fa0 con 0x7f5f801013c0 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51396/0 (socket says 172.21.15.122:51396) 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 -- 172.21.15.122:0/3872005352 learned_addr learned my addr 172.21.15.122:0/3872005352 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7effd640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 0x7f5f80198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7f7fe640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 -- 172.21.15.122:0/3872005352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 msgr2=0x7f5f80198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 0x7f5f80198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 -- 172.21.15.122:0/3872005352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 msgr2=0x7f5f80198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:02.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 -- 172.21.15.122:0/3872005352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8019d3f0 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7f7fe640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7ffff640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800eaa0 tx=0x7f5f7800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7800cdb0 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8019d6e0 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f78005020 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f780116d0 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.987+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f801a4f70 con 0x7f5f80106a10 2024-04-02T13:58:02.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.988+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f800688f0 con 0x7f5f80106a10 2024-04-02T13:58:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.989+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f5f780054d0 con 0x7f5f80106a10 2024-04-02T13:58:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.990+0000 7f5f7cff9640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 0x7f5f680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:02.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.990+0000 7f5f7f7fe640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 0x7f5f680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.990+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f5f780d0dc0 con 0x7f5f80106a10 2024-04-02T13:58:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.991+0000 7f5f7f7fe640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 0x7f5f680a61e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006010 tx=0x7f5f7002c6f0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:02.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:02.995+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f5f78016030 con 0x7f5f80106a10 2024-04-02T13:58:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:02 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/4105084184' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-02T13:58:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:02 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3828580094' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-02T13:58:03.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:02 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/569912787' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-02T13:58:03.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.212+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f80109fe0 con 0x7f5f80106a10 2024-04-02T13:58:03.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.212+0000 7f5f7cff9640 1 -- 172.21.15.122:0/3872005352 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11954 (secure 0 0 0) 0x7f5f7809d070 con 0x7f5f80106a10 2024-04-02T13:58:03.217 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:58:03.217 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:57:36.793988+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:58:03.217 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 80 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.214+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 msgr2=0x7f5f680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.214+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 0x7f5f680a61e0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5f70006010 tx=0x7f5f7002c6f0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.214+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 msgr2=0x7f5f8019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.214+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f5f7800eaa0 tx=0x7f5f7800ef60 comp rx=0 tx=0).stop 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 shutdown_connections 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f5f680a3d20 0x7f5f680a61e0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f80106a10 0x7f5f8019ccc0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f801013c0 0x7f5f80198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 --2- 172.21.15.122:0/3872005352 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5f80100a00 0x7f5f80198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 >> 172.21.15.122:0/3872005352 conn(0x7f5f80077b00 msgr2=0x7f5f80068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 shutdown_connections 2024-04-02T13:58:03.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.215+0000 7f5f860a5640 1 -- 172.21.15.122:0/3872005352 wait complete. 2024-04-02T13:58:03.230 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-02T13:58:03.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 -- 172.21.15.122:0/19140185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 msgr2=0x7f36bc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 --2- 172.21.15.122:0/19140185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc100f00 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009930 tx=0x7f36a402bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:03.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 -- 172.21.15.122:0/19140185 shutdown_connections 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 --2- 172.21.15.122:0/19140185 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc101440 0x7f36bc10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 --2- 172.21.15.122:0/19140185 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc100f00 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 --2- 172.21.15.122:0/19140185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc106a40 0x7f36bc106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 -- 172.21.15.122:0/19140185 >> 172.21.15.122:0/19140185 conn(0x7f36bc077b60 msgr2=0x7f36bc077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 -- 172.21.15.122:0/19140185 shutdown_connections 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 -- 172.21.15.122:0/19140185 wait complete. 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.329+0000 7f36c3055640 1 Processor -- start 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 -- start start 2024-04-02T13:58:03.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc101440 0x7f36bc1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc106a40 0x7f36bc1a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc110e50 con 0x7f36bc100a80 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc110cd0 con 0x7f36bc106a40 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc110fd0 con 0x7f36bc101440 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51416/0 (socket says 172.21.15.122:51416) 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 -- 172.21.15.122:0/3931888710 learned_addr learned my addr 172.21.15.122:0/3931888710 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:03.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36bbfff640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc101440 0x7f36bc1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c15f2640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc106a40 0x7f36bc1a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 -- 172.21.15.122:0/3931888710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc101440 msgr2=0x7f36bc1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc101440 0x7f36bc1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 -- 172.21.15.122:0/3931888710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc106a40 msgr2=0x7f36bc1a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc106a40 0x7f36bc1a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 -- 172.21.15.122:0/3931888710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc1a4870 con 0x7f36bc100a80 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c0df1640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f36b400eab0 tx=0x7f36b400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400ce40 con 0x7f36bc100a80 2024-04-02T13:58:03.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b4011950 con 0x7f36bc100a80 2024-04-02T13:58:03.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4010ae0 con 0x7f36bc100a80 2024-04-02T13:58:03.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.330+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc1a4a40 con 0x7f36bc100a80 2024-04-02T13:58:03.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.331+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a4f10 con 0x7f36bc100a80 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.332+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc068d40 con 0x7f36bc100a80 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.333+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f36b4011b00 con 0x7f36bc100a80 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.333+0000 7f36b9ffb640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 0x7f36a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.334+0000 7f36bbfff640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 0x7f36a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.334+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f36b40d1190 con 0x7f36bc100a80 2024-04-02T13:58:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.334+0000 7f36bbfff640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 0x7f36a80a61e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f36a4005ab0 tx=0x7f36a4005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.338+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f36b4016030 con 0x7f36bc100a80 2024-04-02T13:58:03.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.558+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36bc108fa0 con 0x7f36bc100a80 2024-04-02T13:58:03.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.559+0000 7f36b9ffb640 1 -- 172.21.15.122:0/3931888710 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11954 (secure 0 0 0) 0x7f36b409d440 con 0x7f36bc100a80 2024-04-02T13:58:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:58:03.564 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/2838975565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":2838975565},{"type":"v1","addr":"172.21.15.122:6835","nonce":2838975565}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1110044990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1110044990},{"type":"v1","addr":"172.21.15.122:6837","nonce":1110044990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-02T13:32:49.884932+0000","modified":"2024-04-02T13:57:36.793988+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24512,"mds_1":24487,"mds_2":24485,"mds_3":24460,"mds_4":14628},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1672540384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1672540384},{"type":"v1","addr":"172.21.15.122:6839","nonce":1672540384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14628":{"gid":14628,"name":"a","rank":4,"incarnation":25,"state":"up:active","state_seq":8,"addr":"172.21.15.122:6841/385777794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6840","nonce":385777794},{"type":"v1","addr":"172.21.15.122:6841","nonce":385777794}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6833/546075479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":546075479},{"type":"v1","addr":"172.21.15.188:6833","nonce":546075479}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6833/1223173792","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":1223173792},{"type":"v1","addr":"172.21.15.187:6833","nonce":1223173792}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.188:6835/179772221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":179772221},{"type":"v1","addr":"172.21.15.188:6835","nonce":179772221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.187:6835/4231455845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6834","nonce":4231455845},{"type":"v1","addr":"172.21.15.187:6835","nonce":4231455845}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6837/391935088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":391935088},{"type":"v1","addr":"172.21.15.188:6837","nonce":391935088}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.187:6837/453335780","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6836","nonce":453335780},{"type":"v1","addr":"172.21.15.187:6837","nonce":453335780}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.188:6839/180395175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6838","nonce":180395175},{"type":"v1","addr":"172.21.15.188:6839","nonce":180395175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6839/2503909279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6838","nonce":2503909279},{"type":"v1","addr":"172.21.15.187:6839","nonce":2503909279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24512,"qdb_cluster":[14628,24512,24460,24485,24487]},"id":1}]} 2024-04-02T13:58:03.564 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 80 2024-04-02T13:58:03.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 msgr2=0x7f36a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 0x7f36a80a61e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f36a4005ab0 tx=0x7f36a4005fd0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 msgr2=0x7f36bc1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f36b400eab0 tx=0x7f36b400ef70 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 shutdown_connections 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f36a80a3d20 0x7f36a80a61e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc106a40 0x7f36bc1a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc101440 0x7f36bc1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 --2- 172.21.15.122:0/3931888710 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36bc100a80 0x7f36bc1992b0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 >> 172.21.15.122:0/3931888710 conn(0x7f36bc077b60 msgr2=0x7f36bc103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 shutdown_connections 2024-04-02T13:58:03.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.561+0000 7f36c3055640 1 -- 172.21.15.122:0/3931888710 wait complete. 2024-04-02T13:58:03.576 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-02T13:58:03.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 -- 172.21.15.122:0/2460150481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28102450 msgr2=0x7fae2810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 --2- 172.21.15.122:0/2460150481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28102450 0x7fae2810e960 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fae18009930 tx=0x7fae1802bc30 comp rx=0 tx=0).stop 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 -- 172.21.15.122:0/2460150481 shutdown_connections 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 --2- 172.21.15.122:0/2460150481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28102450 0x7fae2810e960 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 --2- 172.21.15.122:0/2460150481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 0x7fae28101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 --2- 172.21.15.122:0/2460150481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28107aa0 0x7fae28107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 -- 172.21.15.122:0/2460150481 >> 172.21.15.122:0/2460150481 conn(0x7fae280fd5a0 msgr2=0x7fae280ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 -- 172.21.15.122:0/2460150481 shutdown_connections 2024-04-02T13:58:03.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.676+0000 7fae2f660640 1 -- 172.21.15.122:0/2460150481 wait complete. 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 Processor -- start 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 -- start start 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 0x7fae28073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 0x7fae28074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fae28111ed0 con 0x7fae28107aa0 2024-04-02T13:58:03.684 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae28111d50 con 0x7fae28102450 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2f660640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae28112050 con 0x7fae28101a90 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51420/0 (socket says 172.21.15.122:51420) 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 -- 172.21.15.122:0/676575105 learned_addr learned my addr 172.21.15.122:0/676575105 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2d3fc640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 0x7fae28073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2cbfb640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 0x7fae28074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 -- 172.21.15.122:0/676575105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 msgr2=0x7fae28073e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 0x7fae28073e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 -- 172.21.15.122:0/676575105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 msgr2=0x7fae28074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 0x7fae28074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 -- 172.21.15.122:0/676575105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae281a68d0 con 0x7fae28107aa0 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2cbfb640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 0x7fae28074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae2dbfd640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fae18004f50 tx=0x7fae1802ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.677+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18041070 con 0x7fae28107aa0 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.678+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae281a6b60 con 0x7fae28107aa0 2024-04-02T13:58:03.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.678+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae180044d0 con 0x7fae28107aa0 2024-04-02T13:58:03.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.678+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae18034440 con 0x7fae28107aa0 2024-04-02T13:58:03.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.678+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae281a7050 con 0x7fae28107aa0 2024-04-02T13:58:03.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.678+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae28068d40 con 0x7fae28107aa0 2024-04-02T13:58:03.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.680+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fae18044070 con 0x7fae28107aa0 2024-04-02T13:58:03.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.680+0000 7fae267fc640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 0x7fae100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.680+0000 7fae2d3fc640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 0x7fae100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.680+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7fae180f2a80 con 0x7fae28107aa0 2024-04-02T13:58:03.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.681+0000 7fae2d3fc640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 0x7fae100a61e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae2000a400 tx=0x7fae2000d040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.684+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fae180bee40 con 0x7fae28107aa0 2024-04-02T13:58:03.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.865+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae2810a020 con 0x7fae28107aa0 2024-04-02T13:58:03.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.866+0000 7fae267fc640 1 -- 172.21.15.122:0/676575105 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+25415 (secure 0 0 0) 0x7fae180be740 con 0x7fae28107aa0 2024-04-02T13:58:03.871 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-04-02T13:58:03.871 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":127,"fsid":"3c818384-f0f4-11ee-b647-cb9ed24678a4","created":"2024-04-02T13:25:00.910314+0000","modified":"2024-04-02T13:57:16.447119+0000","last_up_change":"2024-04-02T13:32:00.163177+0000","last_in_change":"2024-04-02T13:31:39.501719+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-02T13:28:14.467087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-02T13:32:47.895085+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-02T13:32:48.813983+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-02T13:32:52.540451+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"94d678b8-fbb0-485b-b98a-133e7a6de516","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6803","nonce":933440597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6805","nonce":933440597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6809","nonce":933440597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":933440597},{"type":"v1","addr":"172.21.15.122:6807","nonce":933440597}]},"public_addr":"172.21.15.122:6803/933440597","cluster_addr":"172.21.15.122:6805/933440597","heartbeat_back_addr":"172.21.15.122:6809/933440597","heartbeat_front_addr":"172.21.15.122:6807/933440597","state":["exists","up"]},{"osd":1,"uuid":"349c8fa1-e818-4c5e-862d-e30ff9fa14bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6801","nonce":2690363076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6803","nonce":2690363076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6807","nonce":2690363076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2690363076},{"type":"v1","addr":"172.21.15.187:6805","nonce":2690363076}]},"public_addr":"172.21.15.187:6801/2690363076","cluster_addr":"172.21.15.187:6803/2690363076","heartbeat_back_addr":"172.21.15.187:6807/2690363076","heartbeat_front_addr":"172.21.15.187:6805/2690363076","state":["exists","up"]},{"osd":2,"uuid":"e9bc5736-0e61-468b-b9a8-f988e2d36bf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6801","nonce":2840236580}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6803","nonce":2840236580}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6807","nonce":2840236580}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2840236580},{"type":"v1","addr":"172.21.15.188:6805","nonce":2840236580}]},"public_addr":"172.21.15.188:6801/2840236580","cluster_addr":"172.21.15.188:6803/2840236580","heartbeat_back_addr":"172.21.15.188:6807/2840236580","heartbeat_front_addr":"172.21.15.188:6805/2840236580","state":["exists","up"]},{"osd":3,"uuid":"e7608c20-4f88-434c-99ae-225f08577c91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6811","nonce":2685457026}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6813","nonce":2685457026}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6817","nonce":2685457026}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2685457026},{"type":"v1","addr":"172.21.15.122:6815","nonce":2685457026}]},"public_addr":"172.21.15.122:6811/2685457026","cluster_addr":"172.21.15.122:6813/2685457026","heartbeat_back_addr":"172.21.15.122:6817/2685457026","heartbeat_front_addr":"172.21.15.122:6815/2685457026","state":["exists","up"]},{"osd":4,"uuid":"038779c8-ba3f-4165-a72d-61c31b3c3cbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6809","nonce":101695590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6811","nonce":101695590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6815","nonce":101695590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":101695590},{"type":"v1","addr":"172.21.15.187:6813","nonce":101695590}]},"public_addr":"172.21.15.187:6809/101695590","cluster_addr":"172.21.15.187:6811/101695590","heartbeat_back_addr":"172.21.15.187:6815/101695590","heartbeat_front_addr":"172.21.15.187:6813/101695590","state":["exists","up"]},{"osd":5,"uuid":"266e5146-9b00-4cbe-8ced-85833fa9e959","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6809","nonce":1528823096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6811","nonce":1528823096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6815","nonce":1528823096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1528823096},{"type":"v1","addr":"172.21.15.188:6813","nonce":1528823096}]},"public_addr":"172.21.15.188:6809/1528823096","cluster_addr":"172.21.15.188:6811/1528823096","heartbeat_back_addr":"172.21.15.188:6815/1528823096","heartbeat_front_addr":"172.21.15.188:6813/1528823096","state":["exists","up"]},{"osd":6,"uuid":"2961ced1-b269-44f8-ad13-dc4435028869","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6819","nonce":2482601616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6821","nonce":2482601616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6825","nonce":2482601616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2482601616},{"type":"v1","addr":"172.21.15.122:6823","nonce":2482601616}]},"public_addr":"172.21.15.122:6819/2482601616","cluster_addr":"172.21.15.122:6821/2482601616","heartbeat_back_addr":"172.21.15.122:6825/2482601616","heartbeat_front_addr":"172.21.15.122:6823/2482601616","state":["exists","up"]},{"osd":7,"uuid":"1fd83ac7-db42-40ec-aee5-c69f77239ce9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6817","nonce":3961503181}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6819","nonce":3961503181}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6823","nonce":3961503181}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":3961503181},{"type":"v1","addr":"172.21.15.187:6821","nonce":3961503181}]},"public_addr":"172.21.15.187:6817/3961503181","cluster_addr":"172.21.15.187:6819/3961503181","heartbeat_back_addr":"172.21.15.187:6823/3961503181","heartbeat_front_addr":"172.21.15.187:6821/3961503181","state":["exists","up"]},{"osd":8,"uuid":"5093b075-74f5-494e-afef-c42bda5e04b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6817","nonce":2378797977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6819","nonce":2378797977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6823","nonce":2378797977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2378797977},{"type":"v1","addr":"172.21.15.188:6821","nonce":2378797977}]},"public_addr":"172.21.15.188:6817/2378797977","cluster_addr":"172.21.15.188:6819/2378797977","heartbeat_back_addr":"172.21.15.188:6823/2378797977","heartbeat_front_addr":"172.21.15.188:6821/2378797977","state":["exists","up"]},{"osd":9,"uuid":"a26c2ba8-71db-4d24-8add-1e8ec85b3b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6827","nonce":3872606263}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6829","nonce":3872606263}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6833","nonce":3872606263}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":3872606263},{"type":"v1","addr":"172.21.15.122:6831","nonce":3872606263}]},"public_addr":"172.21.15.122:6827/3872606263","cluster_addr":"172.21.15.122:6829/3872606263","heartbeat_back_addr":"172.21.15.122:6833/3872606263","heartbeat_front_addr":"172.21.15.122:6831/3872606263","state":["exists","up"]},{"osd":10,"uuid":"9ee33545-53b3-428d-9b14-69e4bba42144","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6825","nonce":442100564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6827","nonce":442100564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6831","nonce":442100564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":442100564},{"type":"v1","addr":"172.21.15.187:6829","nonce":442100564}]},"public_addr":"172.21.15.187:6825/442100564","cluster_addr":"172.21.15.187:6827/442100564","heartbeat_back_addr":"172.21.15.187:6831/442100564","heartbeat_front_addr":"172.21.15.187:6829/442100564","state":["exists","up"]},{"osd":11,"uuid":"14fa986c-a900-464f-a1d3-4bec844c1256","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6825","nonce":993879353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6827","nonce":993879353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6831","nonce":993879353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":993879353},{"type":"v1","addr":"172.21.15.188:6829","nonce":993879353}]},"public_addr":"172.21.15.188:6825/993879353","cluster_addr":"172.21.15.188:6827/993879353","heartbeat_back_addr":"172.21.15.188:6831/993879353","heartbeat_front_addr":"172.21.15.188:6829/993879353","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:25.049449+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:27:48.852966+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:11.791634+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:28:36.080767+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:00.622541+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:24.714045+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:29:49.858478+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:15.257018+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:30:39.826190+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:06.240674+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:32.079071+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-02T13:31:57.602331+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.3b","mappings":[{"from":3,"to":9}]},{"pgid":"4.b","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.1d","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6801/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/3450525377":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3229862468":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/3421882822":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6801/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/684561021":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/3032683818":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/776403907":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/4063470519":"2024-04-03T13:25:54.372176+0000","172.21.15.122:6800/1783168866":"2024-04-03T13:25:54.372176+0000","172.21.15.122:0/1773721972":"2024-04-03T13:35:09.812961+0000","172.21.15.122:6800/1645681773":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/1474593903":"2024-04-03T13:25:27.690608+0000","172.21.15.122:0/2213768328":"2024-04-03T13:35:09.812961+0000","172.21.15.122:0/1325797276":"2024-04-03T13:25:27.690608+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]},{"pool":4,"snaps":[{"begin":26,"length":1},{"begin":39,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.868+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 msgr2=0x7fae100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.868+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 0x7fae100a61e0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fae2000a400 tx=0x7fae2000d040 comp rx=0 tx=0).stop 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.868+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 msgr2=0x7fae28074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.868+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fae18004f50 tx=0x7fae1802ce20 comp rx=0 tx=0).stop 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 shutdown_connections 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fae100a3d20 0x7fae100a61e0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae28107aa0 0x7fae28074880 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae28102450 0x7fae28074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 --2- 172.21.15.122:0/676575105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae28101a90 0x7fae28073e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 >> 172.21.15.122:0/676575105 conn(0x7fae280fd5a0 msgr2=0x7fae280fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 shutdown_connections 2024-04-02T13:58:03.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.869+0000 7fae2f660640 1 -- 172.21.15.122:0/676575105 wait complete. 2024-04-02T13:58:03.882 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:03 smithi122 ceph-mon[32114]: pgmap v721: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-04-02T13:58:03.882 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3872005352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:03.883 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:03 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3931888710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:03.884 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-02T13:58:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:03 smithi187 ceph-mon[36064]: pgmap v721: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-04-02T13:58:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3872005352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:03.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:03 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3931888710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 -- 172.21.15.122:0/3232613251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04106980 msgr2=0x7f7d04106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3232613251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04106980 0x7f7d04106d60 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc009cd0 tx=0x7f7cfc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 -- 172.21.15.122:0/3232613251 shutdown_connections 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3232613251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04101330 0x7f7d0410d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3232613251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04100970 0x7f7d04100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3232613251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04106980 0x7f7d04106d60 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 -- 172.21.15.122:0/3232613251 >> 172.21.15.122:0/3232613251 conn(0x7f7d04077b00 msgr2=0x7f7d04077f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:03.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 -- 172.21.15.122:0/3232613251 shutdown_connections 2024-04-02T13:58:03.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 -- 172.21.15.122:0/3232613251 wait complete. 2024-04-02T13:58:03.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.984+0000 7f7d088a7640 1 Processor -- start 2024-04-02T13:58:03.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- start start 2024-04-02T13:58:03.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 0x7f7d04198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04106980 0x7f7d0419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04110d60 con 0x7f7d04101330 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04110be0 con 0x7f7d04106980 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04110ee0 con 0x7f7d04100970 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51438/0 (socket says 172.21.15.122:51438) 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 -- 172.21.15.122:0/3693046930 learned_addr learned my addr 172.21.15.122:0/3693046930 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:03.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d02d9d640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04106980 0x7f7d0419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d0259c640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 0x7f7d04198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 -- 172.21.15.122:0/3693046930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 msgr2=0x7f7d04198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 0x7f7d04198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 -- 172.21.15.122:0/3693046930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04106980 msgr2=0x7f7d0419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04106980 0x7f7d0419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 -- 172.21.15.122:0/3693046930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0419d280 con 0x7f7d04101330 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d0259c640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 0x7f7d04198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:03.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d01d9b640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c920 tx=0x7f7cec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec007c70 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cec007e10 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0419d570 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cec018890 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.985+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a4e00 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.987+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d040688f0 con 0x7f7d04101330 2024-04-02T13:58:03.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.987+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7f7cec018a70 con 0x7f7d04101330 2024-04-02T13:58:03.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.988+0000 7f7cfb7fe640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 0x7f7cf00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:03.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.988+0000 7f7d0259c640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 0x7f7cf00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:03.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.988+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9525+0+0 (secure 0 0 0) 0x7f7cec0d0e20 con 0x7f7d04101330 2024-04-02T13:58:03.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.988+0000 7f7d0259c640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 0x7f7cf00a61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc037040 tx=0x7f7cfc005d80 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:03.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:03.992+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7f7cec014030 con 0x7f7d04101330 2024-04-02T13:58:04.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:03 smithi188 ceph-mon[36117]: pgmap v721: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 21 op/s 2024-04-02T13:58:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3872005352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:04.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:03 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3931888710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-02T13:58:04.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.206+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7d041a50e0 con 0x7f7d04101330 2024-04-02T13:58:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.209+0000 7f7cfb7fe640 1 -- 172.21.15.122:0/3693046930 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) ==== 161+0+0 (secure 0 0 0) 0x7f7cec00de20 con 0x7f7d04101330 2024-04-02T13:58:04.214 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-02T13:58:04.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.212+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 msgr2=0x7f7cf00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:04.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.212+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 0x7f7cf00a61e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc037040 tx=0x7f7cfc005d80 comp rx=0 tx=0).stop 2024-04-02T13:58:04.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.212+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 msgr2=0x7f7d041987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:04.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.212+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00c920 tx=0x7f7cec00cde0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 shutdown_connections 2024-04-02T13:58:04.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7f7cf00a3d20 0x7f7cf00a61e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d04106980 0x7f7d0419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d04101330 0x7f7d041987d0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 --2- 172.21.15.122:0/3693046930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d04100970 0x7f7d04198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 >> 172.21.15.122:0/3693046930 conn(0x7f7d04077b00 msgr2=0x7f7d040ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:04.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 shutdown_connections 2024-04-02T13:58:04.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.213+0000 7f7d088a7640 1 -- 172.21.15.122:0/3693046930 wait complete. 2024-04-02T13:58:04.233 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-02T13:58:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2893413568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c41013e0 msgr2=0x7fc3c410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2893413568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c41013e0 0x7fc3c410d8f0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009cd0 tx=0x7fc3bc02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2893413568 shutdown_connections 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2893413568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c41013e0 0x7fc3c410d8f0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2893413568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c4100a20 0x7fc3c4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2893413568 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2893413568 >> 172.21.15.122:0/2893413568 conn(0x7fc3c4077b60 msgr2=0x7fc3c4077f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2893413568 shutdown_connections 2024-04-02T13:58:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.341+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2893413568 wait complete. 2024-04-02T13:58:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 Processor -- start 2024-04-02T13:58:04.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 -- start start 2024-04-02T13:58:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c41013e0 0x7fc3c41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4111f40 con 0x7fc3c4100a20 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c4111dc0 con 0x7fc3c4106a30 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c8e0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c41120c0 con 0x7fc3c41013e0 2024-04-02T13:58:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51482/0 (socket says 172.21.15.122:51482) 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 -- 172.21.15.122:0/2259388458 learned_addr learned my addr 172.21.15.122:0/2259388458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c2d9d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3b9d9b640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c41013e0 0x7fc3c41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 -- 172.21.15.122:0/2259388458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c41013e0 msgr2=0x7fc3c41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c41013e0 0x7fc3c41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 -- 172.21.15.122:0/2259388458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 msgr2=0x7fc3c419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:04.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 -- 172.21.15.122:0/2259388458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c419d420 con 0x7fc3c4100a20 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c2d9d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.342+0000 7fc3c259c640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400ea60 tx=0x7fc3b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.343+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b400cde0 con 0x7fc3c4100a20 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.343+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b40035f0 con 0x7fc3c4100a20 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.343+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c419d710 con 0x7fc3c4100a20 2024-04-02T13:58:04.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.343+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4010670 con 0x7fc3c4100a20 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.343+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c41a4fa0 con 0x7fc3c4100a20 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.344+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c4068d40 con 0x7fc3c4100a20 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.345+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fc3b40037a0 con 0x7fc3c4100a20 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.346+0000 7fc3bbfff640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 0x7fc3b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.346+0000 7fc3b9d9b640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 0x7fc3b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.346+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9867+0+0 (secure 0 0 0) 0x7fc3b40d11d0 con 0x7fc3c4100a20 2024-04-02T13:58:04.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.346+0000 7fc3b9d9b640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 0x7fc3b00a61e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40781a0 tx=0x7fc3ac005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:04.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.349+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fc3b409d3c0 con 0x7fc3c4100a20 2024-04-02T13:58:04.571 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a[32110]: 2024-04-02T13:58:04.207+0000 7f22d4ced700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:58:04.571 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.572 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.572 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mds[71098]: -- 172.21.15.122:0/1009826835 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7b6d0fa40 con 0x55c7b6ce2800 2024-04-02T13:58:04.572 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: starting mds.g at 2024-04-02T13:58:04.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:04.567+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc3c4108fe0 con 0x7fc3c4100a20 2024-04-02T13:58:04.573 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a[71451]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.573 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a[71451]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.573 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mds[71457]: -- 172.21.15.122:0/811917230 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56460146fa40 con 0x564601442c00 2024-04-02T13:58:04.573 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a[71451]: starting mds.a at 2024-04-02T13:58:04.650 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.650 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.651 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: starting mds.e at 2024-04-02T13:58:04.651 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.651 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.651 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mds[66033]: -- 172.21.15.187:0/3957027628 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dad036fa40 con 0x55dad0342800 2024-04-02T13:58:04.652 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: starting mds.k at 2024-04-02T13:58:04.652 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.652 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.652 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mds[66976]: -- 172.21.15.187:0/2851050961 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562203815a40 con 0x5622037e8800 2024-04-02T13:58:04.652 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: starting mds.b at 2024-04-02T13:58:04.653 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.653 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.653 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: starting mds.h at 2024-04-02T13:58:04.654 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.654 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.654 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mds[66873]: -- 172.21.15.188:0/3439052074 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5565bee97a40 con 0x5565bee6a800 2024-04-02T13:58:04.654 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: starting mds.f at 2024-04-02T13:58:04.655 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.655 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.655 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mds[67181]: -- 172.21.15.188:0/1403299038 <== mon.0 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56116ae87a40 con 0x56116ae5ac00 2024-04-02T13:58:04.655 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: starting mds.c at 2024-04-02T13:58:04.655 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.656 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.656 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mds[67504]: -- 172.21.15.188:0/3622878531 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f5dab9a40 con 0x561f5da8cc00 2024-04-02T13:58:04.656 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: starting mds.i at 2024-04-02T13:58:04.656 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: ignoring --setuser ceph since I am not root 2024-04-02T13:58:04.656 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: ignoring --setgroup ceph since I am not root 2024-04-02T13:58:04.657 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mds[66560]: -- 172.21.15.188:0/3854664402 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56281e6f64e0 con 0x56281e6cc800 2024-04-02T13:58:04.657 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mds[66560]: -- 172.21.15.188:0/3854664402 <== mon.1 v2:172.21.15.188:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56281e6f9a40 con 0x56281e6cc800 2024-04-02T13:58:04.657 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: starting mds.l at 2024-04-02T13:58:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/676575105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:58:04.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:58:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:58:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:58:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: osdmap e128: 12 total, 12 up, 12 in 2024-04-02T13:58:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T13:58:04.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:04 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:58:05.017 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/676575105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: osdmap e128: 12 total, 12 up, 12 in 2024-04-02T13:58:05.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T13:58:05.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:04 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:58:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/676575105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-02T13:58:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-02T13:58:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-02T13:58:05.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-02T13:58:05.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/3693046930' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-02T13:58:05.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: osdmap e128: 12 total, 12 up, 12 in 2024-04-02T13:58:05.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-02T13:58:05.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:04 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-02T13:58:05.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.214+0000 7fc3bbfff640 1 -- 172.21.15.122:0/2259388458 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) ==== 104+0+0 (secure 0 0 0) 0x7fc3b409ccc0 con 0x7fc3c4100a20 2024-04-02T13:58:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 msgr2=0x7fc3b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 0x7fc3b00a61e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40781a0 tx=0x7fc3ac005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 msgr2=0x7fc3c4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fc3b400ea60 tx=0x7fc3b400ef20 comp rx=0 tx=0).stop 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 shutdown_connections 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fc3b00a3d20 0x7fc3b00a61e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc3c4106a30 0x7fc3c419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c41013e0 0x7fc3c41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 --2- 172.21.15.122:0/2259388458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3c4100a20 0x7fc3c4198480 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.217+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 >> 172.21.15.122:0/2259388458 conn(0x7fc3c4077b60 msgr2=0x7fc3c4103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.218+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 shutdown_connections 2024-04-02T13:58:05.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.218+0000 7fc3c8e0d640 1 -- 172.21.15.122:0/2259388458 wait complete. 2024-04-02T13:58:05.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-02T13:58:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 -- 172.21.15.122:0/395649518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234102450 msgr2=0x7fa23410e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 --2- 172.21.15.122:0/395649518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234102450 0x7fa23410e960 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fa22c009cd0 tx=0x7fa22c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 -- 172.21.15.122:0/395649518 shutdown_connections 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 --2- 172.21.15.122:0/395649518 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234102450 0x7fa23410e960 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 --2- 172.21.15.122:0/395649518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa234101a90 0x7fa234101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 --2- 172.21.15.122:0/395649518 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234107aa0 0x7fa234107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 -- 172.21.15.122:0/395649518 >> 172.21.15.122:0/395649518 conn(0x7fa234078f00 msgr2=0x7fa234079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 -- 172.21.15.122:0/395649518 shutdown_connections 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 -- 172.21.15.122:0/395649518 wait complete. 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.348+0000 7fa23b463640 1 Processor -- start 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 -- start start 2024-04-02T13:58:05.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 0x7fa234198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa234107aa0 0x7fa23419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa234111eb0 con 0x7fa234101a90 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa234111d30 con 0x7fa234102450 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa234112030 con 0x7fa234107aa0 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51496/0 (socket says 172.21.15.122:51496) 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 -- 172.21.15.122:0/28967314 learned_addr learned my addr 172.21.15.122:0/28967314 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:05.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2389fe640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 0x7fa234198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 -- 172.21.15.122:0/28967314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa234107aa0 msgr2=0x7fa23419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa234107aa0 0x7fa23419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 -- 172.21.15.122:0/28967314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 msgr2=0x7fa234198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 0x7fa234198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 -- 172.21.15.122:0/28967314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23419d340 con 0x7fa234101a90 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2389fe640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 0x7fa234198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa2391ff640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa23000c890 tx=0x7fa23000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230004220 con 0x7fa234101a90 2024-04-02T13:58:05.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23419d630 con 0x7fa234101a90 2024-04-02T13:58:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2300043c0 con 0x7fa234101a90 2024-04-02T13:58:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.349+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230004a00 con 0x7fa234101a90 2024-04-02T13:58:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.350+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2341a4ec0 con 0x7fa234101a90 2024-04-02T13:58:05.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.350+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa234068d40 con 0x7fa234101a90 2024-04-02T13:58:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.351+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fa2300079b0 con 0x7fa234101a90 2024-04-02T13:58:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.352+0000 7fa22a7fc640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 0x7fa2200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.352+0000 7fa2389fe640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 0x7fa2200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.352+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9867+0+0 (secure 0 0 0) 0x7fa2300d0a50 con 0x7fa234101a90 2024-04-02T13:58:05.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.353+0000 7fa2389fe640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 0x7fa2200a61e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa2341060b0 tx=0x7fa224005d30 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:05.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.357+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fa230014030 con 0x7fa234101a90 2024-04-02T13:58:05.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:05.535+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa23402d370 con 0x7fa234101a90 2024-04-02T13:58:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: pgmap v723: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 206 KiB/s wr, 37 op/s 2024-04-02T13:58:05.986 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: Cluster is now healthy 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6838/3243033203,v1:172.21.15.122:6840/3243033203] up:boot 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.122:6839/841431954,v1:172.21.15.122:6841/841431954] up:boot 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6834/1048941663,v1:172.21.15.187:6838/1048941663] up:boot 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6835/3125680182,v1:172.21.15.187:6839/3125680182] up:boot 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6835/4221831665,v1:172.21.15.188:6839/4221831665] up:boot 2024-04-02T13:58:05.987 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6834/13463830,v1:172.21.15.188:6838/13463830] up:boot 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6833/3436056559,v1:172.21.15.188:6837/3436056559] up:boot 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6832/3606856157,v1:172.21.15.187:6836/3606856157] up:boot 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.187:6833/3011608739,v1:172.21.15.187:6837/3011608739] up:boot 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: mds.? [v2:172.21.15.188:6832/758069012,v1:172.21.15.188:6836/758069012] up:boot 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: fsmap 12 up:standby 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:58:05.988 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:58:05.989 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:05 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: pgmap v723: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 206 KiB/s wr, 37 op/s 2024-04-02T13:58:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:58:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:58:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: Cluster is now healthy 2024-04-02T13:58:06.018 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6838/3243033203,v1:172.21.15.122:6840/3243033203] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.122:6839/841431954,v1:172.21.15.122:6841/841431954] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6834/1048941663,v1:172.21.15.187:6838/1048941663] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6835/3125680182,v1:172.21.15.187:6839/3125680182] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6835/4221831665,v1:172.21.15.188:6839/4221831665] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6834/13463830,v1:172.21.15.188:6838/13463830] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6833/3436056559,v1:172.21.15.188:6837/3436056559] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6832/3606856157,v1:172.21.15.187:6836/3606856157] up:boot 2024-04-02T13:58:06.019 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.187:6833/3011608739,v1:172.21.15.187:6837/3011608739] up:boot 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: mds.? [v2:172.21.15.188:6832/758069012,v1:172.21.15.188:6836/758069012] up:boot 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: fsmap 12 up:standby 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:58:06.020 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:58:06.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:58:06.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:58:06.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:58:06.021 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:05 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:06.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: pgmap v723: 129 pgs: 129 active+clean; 2.7 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 206 KiB/s wr, 37 op/s 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: Cluster is now healthy 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/2259388458' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6838/3243033203,v1:172.21.15.122:6840/3243033203] up:boot 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.122:6839/841431954,v1:172.21.15.122:6841/841431954] up:boot 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6834/1048941663,v1:172.21.15.187:6838/1048941663] up:boot 2024-04-02T13:58:06.072 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6835/3125680182,v1:172.21.15.187:6839/3125680182] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6835/4221831665,v1:172.21.15.188:6839/4221831665] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6834/13463830,v1:172.21.15.188:6838/13463830] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6833/3436056559,v1:172.21.15.188:6837/3436056559] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6832/3606856157,v1:172.21.15.187:6836/3606856157] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.187:6833/3011608739,v1:172.21.15.187:6837/3011608739] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: mds.? [v2:172.21.15.188:6832/758069012,v1:172.21.15.188:6836/758069012] up:boot 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: fsmap 12 up:standby 2024-04-02T13:58:06.073 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-02T13:58:06.074 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-02T13:58:06.075 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='mgr.15528 172.21.15.122:0/698896198' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-02T13:58:06.075 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:05 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.216+0000 7fa22a7fc640 1 -- 172.21.15.122:0/28967314 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) ==== 181+0+0 (secure 0 0 0) 0x7fa23009cbb0 con 0x7fa234101a90 2024-04-02T13:58:06.220 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-04-02T13:58:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.219+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 msgr2=0x7fa2200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.219+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 0x7fa2200a61e0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa2341060b0 tx=0x7fa224005d30 comp rx=0 tx=0).stop 2024-04-02T13:58:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.219+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 msgr2=0x7fa234198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:06.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.219+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa23000c890 tx=0x7fa23000cd50 comp rx=0 tx=0).stop 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 shutdown_connections 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fa2200a3d20 0x7fa2200a61e0 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa2341060b0 tx=0x7fa224005d30 comp rx=0 tx=0).stop 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa234107aa0 0x7fa23419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa234102450 0x7fa234198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 --2- 172.21.15.122:0/28967314 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa234101a90 0x7fa234198350 secure :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fa23000c890 tx=0x7fa23000cd50 comp rx=0 tx=0).stop 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 >> 172.21.15.122:0/28967314 conn(0x7fa234078f00 msgr2=0x7fa234104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:06.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.220+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 shutdown_connections 2024-04-02T13:58:06.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.221+0000 7fa23b463640 1 -- 172.21.15.122:0/28967314 wait complete. 2024-04-02T13:58:06.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-02T13:58:06.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 -- 172.21.15.122:0/3406452749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834114c50 msgr2=0x7ff834117040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:06.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 --2- 172.21.15.122:0/3406452749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834114c50 0x7ff834117040 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009ca0 tx=0x7ff82c02c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 -- 172.21.15.122:0/3406452749 shutdown_connections 2024-04-02T13:58:06.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 --2- 172.21.15.122:0/3406452749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834114c50 0x7ff834117040 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 --2- 172.21.15.122:0/3406452749 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834109600 0x7ff834114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 --2- 172.21.15.122:0/3406452749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 0x7ff8341090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.393+0000 7ff839d52640 1 -- 172.21.15.122:0/3406452749 >> 172.21.15.122:0/3406452749 conn(0x7ff83406cbe0 msgr2=0x7ff83406cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:06.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.394+0000 7ff839d52640 1 -- 172.21.15.122:0/3406452749 shutdown_connections 2024-04-02T13:58:06.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.394+0000 7ff839d52640 1 -- 172.21.15.122:0/3406452749 wait complete. 2024-04-02T13:58:06.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.395+0000 7ff839d52640 1 Processor -- start 2024-04-02T13:58:06.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.396+0000 7ff839d52640 1 -- start start 2024-04-02T13:58:06.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 0x7ff834113b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:06.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:06.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 0x7ff8341abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:06.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff83411a630 con 0x7ff834109600 2024-04-02T13:58:06.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff83411a4b0 con 0x7ff834114c50 2024-04-02T13:58:06.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff839d52640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff83411a7b0 con 0x7ff834108ce0 2024-04-02T13:58:06.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff833fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 0x7ff8341abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:06.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:06.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff832ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51512/0 (socket says 172.21.15.122:51512) 2024-04-02T13:58:06.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff832ffd640 1 -- 172.21.15.122:0/1287158495 learned_addr learned my addr 172.21.15.122:0/1287158495 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:06.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.397+0000 7ff8337fe640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 0x7ff834113b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:06.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 -- 172.21.15.122:0/1287158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 msgr2=0x7ff834113b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:06.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 0x7ff834113b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 -- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 msgr2=0x7ff834114080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 -- 172.21.15.122:0/1287158495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8341ac230 con 0x7ff834114c50 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff832ffd640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff833fff640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 0x7ff8341abb00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff82c037040 tx=0x7ff82c032ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c041020 con 0x7ff834114c50 2024-04-02T13:58:06.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff839d52640 1 -- 172.21.15.122:0/1287158495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8341ac4c0 con 0x7ff834114c50 2024-04-02T13:58:06.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.398+0000 7ff839d52640 1 -- 172.21.15.122:0/1287158495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8341b3db0 con 0x7ff834114c50 2024-04-02T13:58:06.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.400+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff82c034cd0 con 0x7ff834114c50 2024-04-02T13:58:06.409 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.400+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c039040 con 0x7ff834114c50 2024-04-02T13:58:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.401+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7ff82c044020 con 0x7ff834114c50 2024-04-02T13:58:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.401+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff820005290 con 0x7ff834114c50 2024-04-02T13:58:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.402+0000 7ff830ff9640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 0x7ff81c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.402+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9313+0+0 (secure 0 0 0) 0x7ff82c0f2500 con 0x7ff834114c50 2024-04-02T13:58:06.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.403+0000 7ff8337fe640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 0x7ff81c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:06.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.404+0000 7ff8337fe640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 0x7ff81c0a62e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff834114bc0 tx=0x7ff824005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:06.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.408+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7ff82c04b020 con 0x7ff834114c50 2024-04-02T13:58:06.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:06.722+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff820005580 con 0x7ff834114c50 2024-04-02T13:58:07.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.469+0000 7ff830ff9640 1 -- 172.21.15.122:0/1287158495 <== mon.2 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) ==== 169+0+0 (secure 0 0 0) 0x7ff82c0bfa00 con 0x7ff834114c50 2024-04-02T13:58:07.473 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 msgr2=0x7ff81c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 0x7ff81c0a62e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff834114bc0 tx=0x7ff824005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 msgr2=0x7ff8341abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 0x7ff8341abb00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff82c037040 tx=0x7ff82c032ea0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 shutdown_connections 2024-04-02T13:58:07.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7ff81c0a3e20 0x7ff81c0a62e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff834114c50 0x7ff8341abb00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff834109600 0x7ff834114080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 --2- 172.21.15.122:0/1287158495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff834108ce0 0x7ff834113b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 >> 172.21.15.122:0/1287158495 conn(0x7ff83406cbe0 msgr2=0x7ff834071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 shutdown_connections 2024-04-02T13:58:07.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.473+0000 7ff81a7fc640 1 -- 172.21.15.122:0/1287158495 wait complete. 2024-04-02T13:58:07.485 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:07.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:07 smithi187 ceph-mon[36064]: osdmap e129: 12 total, 12 up, 12 in 2024-04-02T13:58:07.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:07 smithi187 ceph-mon[36064]: pgmap v725: 97 pgs: 97 active+clean; 345 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:58:07.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:07 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/1287158495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:07 smithi187 ceph-mon[36064]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.496 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:07.497 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:07 smithi122 ceph-mon[32114]: osdmap e129: 12 total, 12 up, 12 in 2024-04-02T13:58:07.497 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:07 smithi122 ceph-mon[32114]: pgmap v725: 97 pgs: 97 active+clean; 345 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:58:07.497 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:07 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/1287158495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.497 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:07 smithi122 ceph-mon[32114]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.498 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-02T13:58:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/28967314' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:07.517 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:07 smithi188 ceph-mon[36117]: osdmap e129: 12 total, 12 up, 12 in 2024-04-02T13:58:07.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:07 smithi188 ceph-mon[36117]: pgmap v725: 97 pgs: 97 active+clean; 345 MiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-02T13:58:07.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:07 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/1287158495' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.518 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:07 smithi188 ceph-mon[36117]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:07.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 -- 172.21.15.122:0/2341622758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 msgr2=0x7fd6c010e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:07.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/2341622758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c010e980 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009cd0 tx=0x7fd6b802c0f0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 -- 172.21.15.122:0/2341622758 shutdown_connections 2024-04-02T13:58:07.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/2341622758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c010e980 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/2341622758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 0x7fd6c0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/2341622758 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 0x7fd6c01080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 -- 172.21.15.122:0/2341622758 >> 172.21.15.122:0/2341622758 conn(0x7fd6c0075d40 msgr2=0x7fd6c0076150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 -- 172.21.15.122:0/2341622758 shutdown_connections 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 -- 172.21.15.122:0/2341622758 wait complete. 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.607+0000 7fd6c68d5640 1 Processor -- start 2024-04-02T13:58:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 -- start start 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 0x7fd6c0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 0x7fd6c019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111f60 con 0x7fd6c0100d80 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0111de0 con 0x7fd6c0107d00 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c68d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01120e0 con 0x7fd6c01003c0 2024-04-02T13:58:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51522/0 (socket says 172.21.15.122:51522) 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 -- 172.21.15.122:0/394562511 learned_addr learned my addr 172.21.15.122:0/394562511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bffff640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 0x7fd6c0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6c4e72640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 0x7fd6c019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 -- 172.21.15.122:0/394562511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 msgr2=0x7fd6c0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 0x7fd6c0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 -- 172.21.15.122:0/394562511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 msgr2=0x7fd6c019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 0x7fd6c019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.608+0000 7fd6bf7fe640 1 -- 172.21.15.122:0/394562511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c019d410 con 0x7fd6c0100d80 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6bf7fe640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c970 tx=0x7fd6b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4007bb0 con 0x7fd6c0100d80 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c019d700 con 0x7fd6c0100d80 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6b4007d50 con 0x7fd6c0100d80 2024-04-02T13:58:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b400f450 con 0x7fd6c0100d80 2024-04-02T13:58:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.609+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c01a4f90 con 0x7fd6c0100d80 2024-04-02T13:58:07.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.610+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c0068d40 con 0x7fd6c0100d80 2024-04-02T13:58:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.611+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 25) ==== 138778+0+0 (secure 0 0 0) 0x7fd6b4005020 con 0x7fd6c0100d80 2024-04-02T13:58:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.611+0000 7fd6bd7fa640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 0x7fd6a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-02T13:58:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.611+0000 7fd6bffff640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 0x7fd6a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-02T13:58:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.611+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 8754+0+0 (secure 0 0 0) 0x7fd6b40d0700 con 0x7fd6c0100d80 2024-04-02T13:58:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.612+0000 7fd6bffff640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 0x7fd6a80a61e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009b50 tx=0x7fd6b0005f90 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-02T13:58:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.616+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188173 (secure 0 0 0) 0x7fd6b4014030 con 0x7fd6c0100d80 2024-04-02T13:58:07.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:07.796+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd6c010a0a0 con 0x7fd6c0100d80 2024-04-02T13:58:08.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.467+0000 7fd6bd7fa640 1 -- 172.21.15.122:0/394562511 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v131) ==== 178+0+0 (secure 0 0 0) 0x7fd6b409ccb0 con 0x7fd6c0100d80 2024-04-02T13:58:08.472 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data_ec' removed 2024-04-02T13:58:08.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 msgr2=0x7fd6a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:08.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 0x7fd6a80a61e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0009b50 tx=0x7fd6b0005f90 comp rx=0 tx=0).stop 2024-04-02T13:58:08.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 msgr2=0x7fd6c0198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c970 tx=0x7fd6b400ce30 comp rx=0 tx=0).stop 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 shutdown_connections 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:6800/692246399,v1:172.21.15.122:6801/692246399] conn(0x7fd6a80a3d20 0x7fd6a80a61e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd6c0107d00 0x7fd6c019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6c0100d80 0x7fd6c0198960 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 --2- 172.21.15.122:0/394562511 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6c01003c0 0x7fd6c0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 >> 172.21.15.122:0/394562511 conn(0x7fd6c0075d40 msgr2=0x7fd6c00fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-02T13:58:08.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 shutdown_connections 2024-04-02T13:58:08.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-04-02T13:58:08.470+0000 7fd6c68d5640 1 -- 172.21.15.122:0/394562511 wait complete. 2024-04-02T13:58:08.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:58:08.520 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:58:08.549 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T13:58:08.628 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-02T13:58:08.628 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:58:08.650 DEBUG:teuthology.orchestra.run.smithi187:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:58:08.665 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-02T13:58:08.681 INFO:tasks.cephadm:Stopping all daemons... 2024-04-02T13:58:08.681 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-02T13:58:08.681 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a 2024-04-02T13:58:08.735 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:08 smithi187 ceph-mon[36064]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:08.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:08 smithi187 ceph-mon[36064]: osdmap e130: 12 total, 12 up, 12 in 2024-04-02T13:58:08.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:08 smithi187 ceph-mon[36064]: from='client.? 172.21.15.122:0/394562511' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:08.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:08 smithi122 ceph-mon[32114]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:08.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:08 smithi122 ceph-mon[32114]: osdmap e130: 12 total, 12 up, 12 in 2024-04-02T13:58:08.744 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:08 smithi122 ceph-mon[32114]: from='client.? 172.21.15.122:0/394562511' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:08.767 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:08 smithi188 ceph-mon[36117]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-02T13:58:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:08 smithi188 ceph-mon[36117]: osdmap e130: 12 total, 12 up, 12 in 2024-04-02T13:58:08.768 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:08 smithi188 ceph-mon[36117]: from='client.? 172.21.15.122:0/394562511' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-02T13:58:09.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:08 smithi122 systemd[1]: Stopping Ceph mon.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a[32110]: 2024-04-02T13:58:09.205+0000 7f22da4f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:09.484 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a[32110]: 2024-04-02T13:58:09.205+0000 7f22da4f8700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-02T13:58:09.793 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:09 smithi122 podman[116350]: 2024-04-02 13:58:09.481887504 +0000 UTC m=+0.401509327 container died ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:09 smithi122 podman[116350]: 2024-04-02 13:58:09.889604453 +0000 UTC m=+0.809226290 container cleanup ce047d83baa79e9aba52547d379de064d2ae98e0686e35a89ec5ea599bf80612 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=) 2024-04-02T13:58:10.071 INFO:journalctl@ceph.mon.a.smithi122.stdout:Apr 02 13:58:09 smithi122 bash[116350]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-a 2024-04-02T13:58:10.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.a.service' 2024-04-02T13:58:10.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:10.399 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-02T13:58:10.399 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-02T13:58:10.399 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.b 2024-04-02T13:58:10.736 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:10 smithi187 systemd[1]: Stopping Ceph mon.b for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b[36060]: 2024-04-02T13:58:10.742+0000 7f972c76e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b[36060]: 2024-04-02T13:58:10.742+0000 7f972c76e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-02T13:58:11.236 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:10 smithi187 podman[76292]: 2024-04-02 13:58:10.967774343 +0000 UTC m=+0.328649527 container died 2d596404099a55f2602781b2030082928ee96b6dbd2eb02739ece9b9a90e77ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-02T13:58:12.234 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:11 smithi187 podman[76292]: 2024-04-02 13:58:11.995831859 +0000 UTC m=+1.356707050 container cleanup 2d596404099a55f2602781b2030082928ee96b6dbd2eb02739ece9b9a90e77ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T13:58:12.234 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:11 smithi187 bash[76292]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b 2024-04-02T13:58:12.470 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.b.service' 2024-04-02T13:58:12.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:12 smithi187 podman[76304]: 2024-04-02 13:58:12.230949756 +0000 UTC m=+1.258837336 container remove 2d596404099a55f2602781b2030082928ee96b6dbd2eb02739ece9b9a90e77ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-b, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T13:58:12.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:12 smithi187 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.b.service: Deactivated successfully. 2024-04-02T13:58:12.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:12 smithi187 systemd[1]: Stopped Ceph mon.b for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T13:58:12.486 INFO:journalctl@ceph.mon.b.smithi187.stdout:Apr 02 13:58:12 smithi187 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.b.service: Consumed 31.211s CPU time. 2024-04-02T13:58:12.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:12.528 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-02T13:58:12.528 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-02T13:58:12.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.c 2024-04-02T13:58:12.939 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:12 smithi188 systemd[1]: Stopping Ceph mon.c for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:13.269 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-c[36105]: 2024-04-02T13:58:12.935+0000 7f691eaaf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:13.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-c[36105]: 2024-04-02T13:58:12.935+0000 7f691eaaf700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-02T13:58:13.270 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:13 smithi188 podman[76845]: 2024-04-02 13:58:13.130212913 +0000 UTC m=+0.335247796 container died 0f49511e3d89640a5aa4f519e93ccdac509c933021a0d85e7ffe8ac365ea6f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True) 2024-04-02T13:58:13.527 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:13 smithi188 podman[76845]: 2024-04-02 13:58:13.51177744 +0000 UTC m=+0.716812325 container cleanup 0f49511e3d89640a5aa4f519e93ccdac509c933021a0d85e7ffe8ac365ea6f8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-02T13:58:13.528 INFO:journalctl@ceph.mon.c.smithi188.stdout:Apr 02 13:58:13 smithi188 bash[76845]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mon-c 2024-04-02T13:58:14.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mon.c.service' 2024-04-02T13:58:14.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:14.266 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-02T13:58:14.266 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-02T13:58:14.266 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x 2024-04-02T13:58:14.571 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:58:14 smithi122 systemd[1]: Stopping Ceph mgr.x for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:15.071 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:58:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x[32344]: 2024-04-02T13:58:14.643+0000 7f57343a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:15.071 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:58:14 smithi122 podman[116471]: 2024-04-02 13:58:14.675825885 +0000 UTC m=+0.163502863 container died b8451ced65223f205dca42db07ed74640f5ab4fc18d95f8725e9202386372837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-02T13:58:15.425 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:58:15 smithi122 podman[116471]: 2024-04-02 13:58:15.179039826 +0000 UTC m=+0.666716782 container cleanup b8451ced65223f205dca42db07ed74640f5ab4fc18d95f8725e9202386372837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:191d963ac9de7322da8aff20b567b3fa458e2ef6, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-02T13:58:15.425 INFO:journalctl@ceph.mgr.x.smithi122.stdout:Apr 02 13:58:15 smithi122 bash[116471]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-x 2024-04-02T13:58:15.586 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.x.service' 2024-04-02T13:58:15.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:15.624 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-02T13:58:15.624 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-02T13:58:15.624 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.y 2024-04-02T13:58:15.949 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:58:15 smithi187 systemd[1]: Stopping Ceph mgr.y for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:16.236 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:58:15 smithi187 podman[76413]: 2024-04-02 13:58:15.946257941 +0000 UTC m=+0.127944791 container died 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T13:58:16.736 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:58:16 smithi187 podman[76413]: 2024-04-02 13:58:16.334340587 +0000 UTC m=+0.516027432 container cleanup 740bc047a017d4d3cac25891a248fe8821de5a06f564b9e68a8a04ad1b5afddf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:16.736 INFO:journalctl@ceph.mgr.y.smithi187.stdout:Apr 02 13:58:16 smithi187 bash[76413]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-y 2024-04-02T13:58:17.017 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.y.service' 2024-04-02T13:58:17.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:17.053 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-02T13:58:17.053 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-02T13:58:17.054 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.z 2024-04-02T13:58:17.399 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:58:17 smithi188 systemd[1]: Stopping Ceph mgr.z for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:17.684 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:58:17 smithi188 podman[76978]: 2024-04-02 13:58:17.396352745 +0000 UTC m=+0.155502786 container died 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T13:58:18.018 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:58:17 smithi188 podman[76978]: 2024-04-02 13:58:17.767712853 +0000 UTC m=+0.526862913 container cleanup 60c096db2232ba7efb67e361b8f6969cec4aaaa2a2ac3d0bc831e5ea216cf854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T13:58:18.018 INFO:journalctl@ceph.mgr.z.smithi188.stdout:Apr 02 13:58:17 smithi188 bash[76978]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mgr-z 2024-04-02T13:58:18.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mgr.z.service' 2024-04-02T13:58:18.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:18.226 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-02T13:58:18.226 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-02T13:58:18.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.0 2024-04-02T13:58:18.571 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:18 smithi122 systemd[1]: Stopping Ceph osd.0 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:19.071 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0[44413]: 2024-04-02T13:58:18.616+0000 7fd476ca5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:19.071 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0[44413]: 2024-04-02T13:58:18.616+0000 7fd476ca5700 -1 osd.0 131 *** Got signal Terminated *** 2024-04-02T13:58:19.071 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0[44413]: 2024-04-02T13:58:18.616+0000 7fd476ca5700 -1 osd.0 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:58:24.389 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:24 smithi122 podman[116592]: 2024-04-02 13:58:24.129867177 +0000 UTC m=+5.713414239 container died 287f9322fd0e1be4067aad7fe8354380033e64736e3f8760a47b4e92c92baaee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326) 2024-04-02T13:58:24.689 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:24 smithi122 podman[116592]: 2024-04-02 13:58:24.476315775 +0000 UTC m=+6.059862838 container cleanup 287f9322fd0e1be4067aad7fe8354380033e64736e3f8760a47b4e92c92baaee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T13:58:24.689 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:24 smithi122 bash[116592]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0 2024-04-02T13:58:25.071 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:24 smithi122 podman[116620]: 2024-04-02 13:58:24.686375126 +0000 UTC m=+0.554599272 container remove 287f9322fd0e1be4067aad7fe8354380033e64736e3f8760a47b4e92c92baaee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-02T13:58:25.393 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:24.994230235 +0000 UTC m=+0.018122757 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:58:25.393 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:25.129514241 +0000 UTC m=+0.153406789 container create 028ce1482243c0aa45000b0309c788c1f736e4bbf7f31246e6b21b932d29fb56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-02T13:58:25.393 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:25.390394149 +0000 UTC m=+0.414286666 container init 028ce1482243c0aa45000b0309c788c1f736e4bbf7f31246e6b21b932d29fb56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T13:58:25.821 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:25.393502522 +0000 UTC m=+0.417395025 container start 028ce1482243c0aa45000b0309c788c1f736e4bbf7f31246e6b21b932d29fb56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T13:58:25.821 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:25.515647154 +0000 UTC m=+0.539539673 container attach 028ce1482243c0aa45000b0309c788c1f736e4bbf7f31246e6b21b932d29fb56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:58:26.230 INFO:journalctl@ceph.osd.0.smithi122.stdout:Apr 02 13:58:25 smithi122 podman[116712]: 2024-04-02 13:58:25.923177985 +0000 UTC m=+0.947070495 container died 028ce1482243c0aa45000b0309c788c1f736e4bbf7f31246e6b21b932d29fb56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T13:58:26.527 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.0.service' 2024-04-02T13:58:26.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:26.570 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-02T13:58:26.571 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-02T13:58:26.571 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.1 2024-04-02T13:58:26.901 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:26 smithi187 systemd[1]: Stopping Ceph osd.1 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:27.236 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1[40671]: 2024-04-02T13:58:26.896+0000 7f68ba34d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:27.236 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1[40671]: 2024-04-02T13:58:26.897+0000 7f68ba34d700 -1 osd.1 131 *** Got signal Terminated *** 2024-04-02T13:58:27.236 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1[40671]: 2024-04-02T13:58:26.897+0000 7f68ba34d700 -1 osd.1 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:58:32.633 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:32 smithi187 podman[76551]: 2024-04-02 13:58:32.31748802 +0000 UTC m=+5.552388729 container died bfec25de3dd0efee35091a06b42ba0357088c2c70018e611adfd127dccaccba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-02T13:58:32.885 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:32 smithi187 podman[76551]: 2024-04-02 13:58:32.630160046 +0000 UTC m=+5.865060731 container cleanup bfec25de3dd0efee35091a06b42ba0357088c2c70018e611adfd127dccaccba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-02T13:58:32.885 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:32 smithi187 bash[76551]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1 2024-04-02T13:58:33.236 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:32 smithi187 podman[76568]: 2024-04-02 13:58:32.882039108 +0000 UTC m=+0.563168088 container remove bfec25de3dd0efee35091a06b42ba0357088c2c70018e611adfd127dccaccba4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=) 2024-04-02T13:58:33.572 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:33 smithi187 podman[76663]: 2024-04-02 13:58:33.191528545 +0000 UTC m=+0.017623999 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:58:33.572 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:33 smithi187 podman[76663]: 2024-04-02 13:58:33.324881707 +0000 UTC m=+0.150977143 container create 7364afa8269693cf300928227bafad70d24de882f105836dfce81c7044319e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-02T13:58:33.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:33 smithi187 podman[76663]: 2024-04-02 13:58:33.569343229 +0000 UTC m=+0.395438672 container init 7364afa8269693cf300928227bafad70d24de882f105836dfce81c7044319e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=) 2024-04-02T13:58:33.986 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:33 smithi187 podman[76663]: 2024-04-02 13:58:33.572534946 +0000 UTC m=+0.398630380 container start 7364afa8269693cf300928227bafad70d24de882f105836dfce81c7044319e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:33.987 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:33 smithi187 podman[76663]: 2024-04-02 13:58:33.677816127 +0000 UTC m=+0.503911575 container attach 7364afa8269693cf300928227bafad70d24de882f105836dfce81c7044319e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2) 2024-04-02T13:58:34.381 INFO:journalctl@ceph.osd.1.smithi187.stdout:Apr 02 13:58:34 smithi187 podman[76663]: 2024-04-02 13:58:34.110851563 +0000 UTC m=+0.936947009 container died 7364afa8269693cf300928227bafad70d24de882f105836dfce81c7044319e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T13:58:34.774 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.1.service' 2024-04-02T13:58:34.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:34.816 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-02T13:58:34.817 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-02T13:58:34.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.2 2024-04-02T13:58:35.208 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:34 smithi188 systemd[1]: Stopping Ceph osd.2 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:35.517 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:58:35.203+0000 7f657720b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:35.518 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:58:35.203+0000 7f657720b700 -1 osd.2 131 *** Got signal Terminated *** 2024-04-02T13:58:35.518 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2[41278]: 2024-04-02T13:58:35.203+0000 7f657720b700 -1 osd.2 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:58:41.049 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:40 smithi188 podman[77100]: 2024-04-02 13:58:40.769876767 +0000 UTC m=+5.752850981 container died 2439b0877a5d928470f2cdc1835fbc42bfa3ae383af8eaee705163258f590ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-02T13:58:41.417 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77100]: 2024-04-02 13:58:41.137294751 +0000 UTC m=+6.120268958 container cleanup 2439b0877a5d928470f2cdc1835fbc42bfa3ae383af8eaee705163258f590ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD) 2024-04-02T13:58:41.417 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 bash[77100]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2 2024-04-02T13:58:41.744 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77116]: 2024-04-02 13:58:41.41395356 +0000 UTC m=+0.642443251 container remove 2439b0877a5d928470f2cdc1835fbc42bfa3ae383af8eaee705163258f590ef2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-02T13:58:41.996 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77208]: 2024-04-02 13:58:41.644627362 +0000 UTC m=+0.017827338 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:58:41.996 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77208]: 2024-04-02 13:58:41.765295239 +0000 UTC m=+0.138495208 container create efe06b68adbe7482d88d5fa32118d9dd0bf5f14901c7c5735b856f4e3a238309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-02T13:58:41.996 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77208]: 2024-04-02 13:58:41.99337549 +0000 UTC m=+0.366575480 container init efe06b68adbe7482d88d5fa32118d9dd0bf5f14901c7c5735b856f4e3a238309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T13:58:42.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:41 smithi188 podman[77208]: 2024-04-02 13:58:41.996963441 +0000 UTC m=+0.370163417 container start efe06b68adbe7482d88d5fa32118d9dd0bf5f14901c7c5735b856f4e3a238309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-04-02T13:58:42.268 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:42 smithi188 podman[77208]: 2024-04-02 13:58:42.109739774 +0000 UTC m=+0.482939765 container attach efe06b68adbe7482d88d5fa32118d9dd0bf5f14901c7c5735b856f4e3a238309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:42.813 INFO:journalctl@ceph.osd.2.smithi188.stdout:Apr 02 13:58:42 smithi188 podman[77208]: 2024-04-02 13:58:42.541426113 +0000 UTC m=+0.914626092 container died efe06b68adbe7482d88d5fa32118d9dd0bf5f14901c7c5735b856f4e3a238309 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=) 2024-04-02T13:58:43.199 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.2.service' 2024-04-02T13:58:43.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:43.241 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-02T13:58:43.241 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-02T13:58:43.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.3 2024-04-02T13:58:43.559 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:43 smithi122 systemd[1]: Stopping Ceph osd.3 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:43.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:43 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:43.556+0000 7f597f1bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:43.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:43 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:43.556+0000 7f597f1bf700 -1 osd.3 131 *** Got signal Terminated *** 2024-04-02T13:58:43.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:43 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:43.556+0000 7f597f1bf700 -1 osd.3 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:58:44.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:44 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:44.422+0000 7f59777ab700 -1 osd.3 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:19.109253+0000 front 2024-04-02T13:58:19.109228+0000 (oldest deadline 2024-04-02T13:58:44.409384+0000) 2024-04-02T13:58:45.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:45 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:45.469+0000 7f59777ab700 -1 osd.3 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:19.109253+0000 front 2024-04-02T13:58:19.109228+0000 (oldest deadline 2024-04-02T13:58:44.409384+0000) 2024-04-02T13:58:46.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:45.611+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:46.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:46 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:46.143+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:46.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:46 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:46.490+0000 7f59777ab700 -1 osd.3 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:19.109253+0000 front 2024-04-02T13:58:19.109228+0000 (oldest deadline 2024-04-02T13:58:44.409384+0000) 2024-04-02T13:58:47.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:46.600+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:47.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:47 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:47.177+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:47.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:47 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:47.463+0000 7f59777ab700 -1 osd.3 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:19.109253+0000 front 2024-04-02T13:58:19.109228+0000 (oldest deadline 2024-04-02T13:58:44.409384+0000) 2024-04-02T13:58:48.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:47.577+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:48.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:47 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:47.939+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:48.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:48 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:48.216+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:48.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:48.596+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:48.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:48.405+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:48.821 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3[51206]: 2024-04-02T13:58:48.474+0000 7f59777ab700 -1 osd.3 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:19.109253+0000 front 2024-04-02T13:58:19.109228+0000 (oldest deadline 2024-04-02T13:58:44.409384+0000) 2024-04-02T13:58:49.294 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:48 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:48.930+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:49.294 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:48 smithi122 podman[116816]: 2024-04-02 13:58:48.980659398 +0000 UTC m=+5.548884786 container died 7052ec9c460828f99464a585b71f85eedeed2ceaf2115a1e4125bea2fe5f0552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:58:49.571 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:49 smithi122 podman[116816]: 2024-04-02 13:58:49.38156957 +0000 UTC m=+5.949794958 container cleanup 7052ec9c460828f99464a585b71f85eedeed2ceaf2115a1e4125bea2fe5f0552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:49.571 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:49 smithi122 bash[116816]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3 2024-04-02T13:58:49.705 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:49.392+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:49.706 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:49.618+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:49.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:49 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:49.263+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:49.932 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:49 smithi122 podman[116833]: 2024-04-02 13:58:49.634117265 +0000 UTC m=+0.651432864 container remove 7052ec9c460828f99464a585b71f85eedeed2ceaf2115a1e4125bea2fe5f0552 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:58:50.017 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:49.702+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:50.207 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:49 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:49.928+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:50.208 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:49 smithi122 podman[116923]: 2024-04-02 13:58:49.839577448 +0000 UTC m=+0.017417191 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:58:50.208 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:49 smithi122 podman[116923]: 2024-04-02 13:58:49.943344229 +0000 UTC m=+0.121183970 container create ce2b35271200187f4bd4d08d4a5f9bec31e0b1d4021a40774985941b4c0b30f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-02T13:58:50.208 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:50 smithi122 podman[116923]: 2024-04-02 13:58:50.137367232 +0000 UTC m=+0.315206990 container init ce2b35271200187f4bd4d08d4a5f9bec31e0b1d4021a40774985941b4c0b30f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-04-02T13:58:50.208 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:50 smithi122 podman[116923]: 2024-04-02 13:58:50.140758966 +0000 UTC m=+0.318598709 container start ce2b35271200187f4bd4d08d4a5f9bec31e0b1d4021a40774985941b4c0b30f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T13:58:50.208 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:50 smithi122 podman[116923]: 2024-04-02 13:58:50.204040369 +0000 UTC m=+0.381880140 container attach ce2b35271200187f4bd4d08d4a5f9bec31e0b1d4021a40774985941b4c0b30f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T13:58:50.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:50 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:50.222+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:50.571 INFO:journalctl@ceph.osd.3.smithi122.stdout:Apr 02 13:58:50 smithi122 podman[116923]: 2024-04-02 13:58:50.432187966 +0000 UTC m=+0.610027715 container died ce2b35271200187f4bd4d08d4a5f9bec31e0b1d4021a40774985941b4c0b30f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-3-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:58:50.655 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:50.392+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:50.957 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.3.service' 2024-04-02T13:58:50.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:50.999 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-02T13:58:50.999 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-02T13:58:50.999 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.4 2024-04-02T13:58:51.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:50.652+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:51.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:50.735+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:51.290 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:50 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:50.960+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:51.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:51 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:51.248+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:51.486 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:51 smithi187 systemd[1]: Stopping Ceph osd.4 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:51.673 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:51.403+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:51.986 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:51 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:51.584+0000 7fa54a978700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:58:51.986 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:51 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:51.584+0000 7fa54a978700 -1 osd.4 131 *** Got signal Terminated *** 2024-04-02T13:58:51.986 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:51 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:51.584+0000 7fa54a978700 -1 osd.4 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:58:52.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:51.669+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:52.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:51.731+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:52.209 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:51 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:51.954+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:52.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:52 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:52.242+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:52.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:52 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:52.620+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:52.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:52 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:52.382+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:53.224 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:52 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:52.880+0000 7fa546f6c700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:29.114388+0000 front 2024-04-02T13:58:29.114461+0000 (oldest deadline 2024-04-02T13:58:52.614588+0000) 2024-04-02T13:58:53.241 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:52 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:52.963+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:53.246 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:52 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:52.774+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:53.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:53 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:53.221+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:53.517 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:53 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:53.340+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:53.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:53 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:53.237+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:54.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:53 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:53.649+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:54.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:53 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:53.795+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:54.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:53 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:53.795+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:54.198 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:53 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:53.836+0000 7fa546f6c700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:29.114388+0000 front 2024-04-02T13:58:29.114461+0000 (oldest deadline 2024-04-02T13:58:52.614588+0000) 2024-04-02T13:58:54.232 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:53 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:53.937+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:54.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:54 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:54.194+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:54.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:54 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:54.194+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:58:54.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:54 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:54.228+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:54.641 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:54 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:54.370+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:54.642 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:54 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:54.370+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:55.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:54 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:54.636+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:55.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:54 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:54.809+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:55.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:54 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:54.809+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:55.217 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:54 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:54.916+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:55.225 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:54 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:54.867+0000 7fa546f6c700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:29.114388+0000 front 2024-04-02T13:58:29.114461+0000 (oldest deadline 2024-04-02T13:58:52.614588+0000) 2024-04-02T13:58:55.486 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:58:55 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:58:55.423+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:58:55.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:55 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:55.222+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:55.487 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:55 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:55.222+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:58:55.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:55 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:55.213+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:55.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:55.393+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:55.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:55.393+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:55.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:55.590+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:55.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:55.591+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:58:56.183 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:55 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:55.869+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:56.183 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:55 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:55.869+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:58:56.236 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:55 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4[47671]: 2024-04-02T13:58:55.844+0000 7fa546f6c700 -1 osd.4 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:29.114388+0000 front 2024-04-02T13:58:29.114461+0000 (oldest deadline 2024-04-02T13:58:52.614588+0000) 2024-04-02T13:58:56.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:55.838+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:56.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:55 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:55.838+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:56.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:56 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:56.179+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:56.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:58:56 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:58:56.414+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:58:56.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:56 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:56.234+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:56.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:56 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:56.234+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:58:56.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:56.439+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:56.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:56.439+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:56.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:56.568+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:56.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:56.568+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:58:57.185 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:56 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:56.848+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:57.185 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:56 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:56.848+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:58:57.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:56.820+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:57.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:56 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:56.820+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:57.371 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:57 smithi187 podman[76781]: 2024-04-02 13:58:57.049841988 +0000 UTC m=+5.856815804 container died 29c9a4704ace39561bb74936454720b2f603197b0753e0b337cc5dfe9f3da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-02T13:58:57.371 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:57 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:57.236+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:57.371 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:57 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:57.236+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:58:57.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:57 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:57.181+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:57.723 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:58:57 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:58:57.451+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:58:57.723 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:57 smithi187 podman[76781]: 2024-04-02 13:58:57.485225756 +0000 UTC m=+6.292199517 container cleanup 29c9a4704ace39561bb74936454720b2f603197b0753e0b337cc5dfe9f3da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, ceph=True) 2024-04-02T13:58:57.723 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:57 smithi187 bash[76781]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4 2024-04-02T13:58:57.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:57.525+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:57.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:57.525+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:58:57.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:57.427+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:57.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:57.427+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:57.986 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:57 smithi187 podman[76799]: 2024-04-02 13:58:57.720281811 +0000 UTC m=+0.663676139 container remove 29c9a4704ace39561bb74936454720b2f603197b0753e0b337cc5dfe9f3da197 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:58:58.183 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:57 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:57.873+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:58.184 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:57 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:57.873+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:58:58.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:57.773+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:58.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:57 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:57.773+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:58.425 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:58 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:58.240+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:58.425 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:58 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:58.240+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:58:58.425 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.034889482 +0000 UTC m=+0.017814478 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:58:58.426 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.138070038 +0000 UTC m=+0.120995068 container create eda63c9a9e74569867461d6a819231ea8b350e5aec8440263ce688a45a686997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-02T13:58:58.426 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.340562516 +0000 UTC m=+0.323487509 container init eda63c9a9e74569867461d6a819231ea8b350e5aec8440263ce688a45a686997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T13:58:58.426 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.343757522 +0000 UTC m=+0.326682515 container start eda63c9a9e74569867461d6a819231ea8b350e5aec8440263ce688a45a686997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T13:58:58.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:58 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:58.180+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:58.738 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:58:58 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:58:58.421+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:58:58.738 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.440429691 +0000 UTC m=+0.423354706 container attach eda63c9a9e74569867461d6a819231ea8b350e5aec8440263ce688a45a686997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:58:58.738 INFO:journalctl@ceph.osd.4.smithi187.stdout:Apr 02 13:58:58 smithi187 podman[76894]: 2024-04-02 13:58:58.632641142 +0000 UTC m=+0.615566129 container died eda63c9a9e74569867461d6a819231ea8b350e5aec8440263ce688a45a686997 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True) 2024-04-02T13:58:58.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:58.429+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:58.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:58.429+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:58.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:58.536+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:58.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:58.536+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:58:59.206 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:58 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:58.894+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:58:59.206 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:58 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:58.894+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:58:59.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:58.788+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:58:59.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:58 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:58.788+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:58:59.345 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.4.service' 2024-04-02T13:58:59.386 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:58:59.386 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-02T13:58:59.387 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-02T13:58:59.387 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.5 2024-04-02T13:58:59.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:58:59 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:58:59.203+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:58:59.702 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:59.392+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:58:59.702 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:59.392+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:58:59.703 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 systemd[1]: Stopping Ceph osd.5 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:58:59.703 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:59.545+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:58:59.703 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:58:59.545+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:58:59.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:58:59 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:58:59.390+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:58:59.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:59 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:59.256+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:58:59.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:58:59 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:58:59.256+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:00.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:59.780+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:00.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:58:59.780+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:00.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:59.698+0000 7f5e6350d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:00.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:59.698+0000 7f5e6350d700 -1 osd.5 131 *** Got signal Terminated *** 2024-04-02T13:59:00.019 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:58:59 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:58:59.698+0000 7f5e6350d700 -1 osd.5 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:00.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:00.155+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:00.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:59 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:59.931+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:00.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:58:59 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:58:59.931+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:00.486 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:00 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:00.404+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:00.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:00 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:00.224+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:00.487 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:00 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:00.224+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:00.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:00.583+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:00.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:00.583+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:00.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:00.388+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:59:00.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:00.388+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:59:01.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:00.810+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:01.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:00 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:00.810+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:01.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:01 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:01.120+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:01.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:00.960+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:01.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:00.960+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:01.486 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:01 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:01.356+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:01.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:01 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:01.199+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:01.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:01 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:01.199+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:01.767 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:01.385+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:59:01.768 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:01.385+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:59:01.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:01.592+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:01.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:01.592+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:02.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:01.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:02.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:01 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:01.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:02.285 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:02 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:02.110+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:02.286 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:01 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:01.987+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:02.286 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:01 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:01.987+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:02.613 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:02.346+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:59:02.614 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:02.346+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:59:02.614 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:02.346+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:35.567493+0000 front 2024-04-02T13:58:35.567475+0000 (oldest deadline 2024-04-02T13:59:01.467672+0000) 2024-04-02T13:59:02.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:02 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:02.334+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:02.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:02 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:02.245+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:02.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:02 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:02.245+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:02.869 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:02.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:02.869 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:02.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:02.869 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:02.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:03.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:02.865+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:03.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:02.865+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:03.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:02 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:02.865+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:03.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:03.125+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:03.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:03.125+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:03.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.001+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:03.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.001+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:03.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.001+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:03.608 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:03.330+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:59:03.608 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:03.330+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:59:03.608 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:03.330+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:35.567493+0000 front 2024-04-02T13:58:35.567475+0000 (oldest deadline 2024-04-02T13:59:01.467672+0000) 2024-04-02T13:59:03.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:03.288+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:03.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:03.282+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:03.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:03 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:03.282+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:03.875 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:03.604+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:03.875 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:03.604+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:03.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:03.604+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:04.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:03.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:04.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:03.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:04.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:03 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:03.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:04.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:04.174+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:04.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:04.174+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:04.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.987+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:04.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.987+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:04.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:03 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:03.987+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:04.638 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:04.312+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.763728+0000 front 2024-04-02T13:58:21.763656+0000 (oldest deadline 2024-04-02T13:58:47.663672+0000) 2024-04-02T13:59:04.639 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:04.312+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.065602+0000 front 2024-04-02T13:58:31.065642+0000 (oldest deadline 2024-04-02T13:58:53.365695+0000) 2024-04-02T13:59:04.639 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5[48140]: 2024-04-02T13:59:04.312+0000 7f5e5baf9700 -1 osd.5 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:35.567493+0000 front 2024-04-02T13:58:35.567475+0000 (oldest deadline 2024-04-02T13:59:01.467672+0000) 2024-04-02T13:59:04.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:04.307+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:04.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:04.307+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:04.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:04.298+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:04.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:04.298+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:04.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:04 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:04.298+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:05.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:04.635+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:05.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:04.635+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:05.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:04.635+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:05.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:04.838+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:05.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:04.838+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:05.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:04 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:04.838+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:05.219 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:04.938+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:05.219 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:04.938+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:05.219 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:04.938+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:05.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:05.215+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:05.578 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:05.215+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:05.643 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:05 smithi188 podman[77328]: 2024-04-02 13:59:05.334292549 +0000 UTC m=+5.753344298 container died 093aa9886f1b4eb49110989e31f344601bf1a939d3721e917107b4289549b2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:59:05.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:05.311+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:05.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:05.311+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:05.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:05.317+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:05.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:05.317+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:05.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:05 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:05.317+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:06.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:05 smithi188 podman[77328]: 2024-04-02 13:59:05.664899432 +0000 UTC m=+6.083951178 container cleanup 093aa9886f1b4eb49110989e31f344601bf1a939d3721e917107b4289549b2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:59:06.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:05 smithi188 bash[77328]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5 2024-04-02T13:59:06.018 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:05 smithi188 podman[77345]: 2024-04-02 13:59:05.875015985 +0000 UTC m=+0.532131459 container remove 093aa9886f1b4eb49110989e31f344601bf1a939d3721e917107b4289549b2ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:05.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:05.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:05.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:05.638+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:05.638+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:06.019 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:05 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:05.638+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:06.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:06.172+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:06.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:06.172+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:06.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:05.952+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:06.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:05.952+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:06.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:05 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:05.952+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:06.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.259167071 +0000 UTC m=+0.099507331 container create abc5f67c5d93d9e1d728fb317044f03f9391e721c805e13b9e22f9f1befda536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-02T13:59:06.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.177431762 +0000 UTC m=+0.017772028 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:06.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.504418626 +0000 UTC m=+0.344758911 container init abc5f67c5d93d9e1d728fb317044f03f9391e721c805e13b9e22f9f1befda536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T13:59:06.518 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.507816356 +0000 UTC m=+0.348156617 container start abc5f67c5d93d9e1d728fb317044f03f9391e721c805e13b9e22f9f1befda536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-02T13:59:06.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:06.338+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:06.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:06.338+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:06.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:06.303+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:06.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:06.303+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:06.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:06 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:06.303+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:06.918 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.604543462 +0000 UTC m=+0.444883743 container attach abc5f67c5d93d9e1d728fb317044f03f9391e721c805e13b9e22f9f1befda536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-02T13:59:06.919 INFO:journalctl@ceph.osd.5.smithi188.stdout:Apr 02 13:59:06 smithi188 podman[77436]: 2024-04-02 13:59:06.800482103 +0000 UTC m=+0.640822365 container died abc5f67c5d93d9e1d728fb317044f03f9391e721c805e13b9e22f9f1befda536 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-02T13:59:06.919 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:06.630+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:06.919 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:06.630+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:06.919 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:06.630+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:07.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:06.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:07.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:06.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:07.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:06 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:06.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:07.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:07.161+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:07.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:07.161+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:07.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:06.984+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:07.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:06.984+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:07.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:06 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:06.984+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:07.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.5.service' 2024-04-02T13:59:07.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:07.501 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-02T13:59:07.502 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-02T13:59:07.502 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.6 2024-04-02T13:59:07.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:07.364+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:07.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:07.364+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:07.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:07.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:07.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:07.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:07.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:07 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:07.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:07.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:07.609+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:07.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:07.609+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:07.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:07.609+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:07.821 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 systemd[1]: Stopping Ceph osd.6 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:08.156 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:08.012+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:08.157 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:08.012+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:08.157 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:08.012+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:08.158 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:07.818+0000 7f71114d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:08.158 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:07.818+0000 7f71114d8700 -1 osd.6 131 *** Got signal Terminated *** 2024-04-02T13:59:08.158 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:07 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:07.818+0000 7f71114d8700 -1 osd.6 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:08.158 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:08.152+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:08.158 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:08 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:08.152+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:08.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:07.906+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:08.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:07.906+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:08.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:07 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:07.906+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:08.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:08.328+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:08.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:08.328+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:08.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:08.289+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:08.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:08.289+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:08.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:08 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:08.289+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:08.958 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:08.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:08.958 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:08.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:08.958 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:08.610+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:09.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:08.954+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:09.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:08.954+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:09.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:08 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:08.954+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:09.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:09.183+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:09.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:09.183+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:09.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:09.027+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:09.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:09.027+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:09.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:09.027+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:09.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:09.365+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:09.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:09.365+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:09.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:09.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:09.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:09.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:09.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:09 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:09.327+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:09.957 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:09.616+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:09.958 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:09.616+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:09.958 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:09.616+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:10.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:09.953+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:10.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:09.953+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:10.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:09 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:09.953+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:10.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:10.211+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:10.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:10.211+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:10.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:10.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:10.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:10.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:10.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:10 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:10.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:10.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:10.339+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:10.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:10.339+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:10.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:10.367+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:10.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:10.367+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:10.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:10.367+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:10.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:10 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:10.367+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:10.953 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:10.568+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:10.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:10.568+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:10.954 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:10.568+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:11.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:10.950+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:11.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:10.950+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:11.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:10 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:10.950+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:11.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:11.236+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:11.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:11.236+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:11.321 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:11.236+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.779416+0000 front 2024-04-02T13:58:46.779471+0000 (oldest deadline 2024-04-02T13:59:10.279590+0000) 2024-04-02T13:59:11.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.025+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:11.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.025+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:11.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.025+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:11.322 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.025+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:11.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:11.370+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:11.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:11.370+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:11.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:11.370+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:11.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:11.389+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:11.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:11.389+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:11.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:11.389+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:11.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:11 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:11.389+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:11.972 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:11.542+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:11.973 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:11.542+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:11.973 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:11.542+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:12.264 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.997+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:12.264 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.997+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:12.264 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.997+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:12.264 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:11 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:11.997+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:12.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:11.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:12.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:11.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:12.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:11 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:11.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:12.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:12.260+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:27.575431+0000 front 2024-04-02T13:58:27.575397+0000 (oldest deadline 2024-04-02T13:58:52.875449+0000) 2024-04-02T13:59:12.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:12.260+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.778015+0000 front 2024-04-02T13:58:39.778030+0000 (oldest deadline 2024-04-02T13:59:02.678040+0000) 2024-04-02T13:59:12.572 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:12 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6[57891]: 2024-04-02T13:59:12.260+0000 7f710dacc700 -1 osd.6 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.779416+0000 front 2024-04-02T13:58:46.779471+0000 (oldest deadline 2024-04-02T13:59:10.279590+0000) 2024-04-02T13:59:12.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:12.341+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:12.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:12.341+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:12.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:12.341+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:12.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:12.344+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:12.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:12.344+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:12.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:12.344+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:12.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:12 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:12.344+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:13.017 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:12.514+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:13.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:12.514+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:13.018 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:12 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:12.514+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:13.291 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.017+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:13.292 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.017+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:13.292 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.017+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:13.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:13.038+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:13.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:13.038+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:13.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:13.038+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:13.321 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:13 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:13.038+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:13.639 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:13 smithi122 podman[117039]: 2024-04-02 13:59:13.352934998 +0000 UTC m=+5.656614957 container died 844c21a7ed1171202db62f3d03e435b89aa029f285709c2aa454391aa75a0157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T13:59:13.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:13.297+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:13.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:13.297+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:13.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:13.297+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:13.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:13.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:13.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:13.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:13.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:13.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:13.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:13.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:13.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:13.523+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:13.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:13.523+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:13.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:13.523+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:14.006 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:13 smithi122 podman[117039]: 2024-04-02 13:59:13.745205313 +0000 UTC m=+6.048885269 container cleanup 844c21a7ed1171202db62f3d03e435b89aa029f285709c2aa454391aa75a0157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6, org.label-schema.build-date=20240326, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-02T13:59:14.006 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:13 smithi122 bash[117039]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6 2024-04-02T13:59:14.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.973+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:14.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.973+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:14.228 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:13 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:13.973+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:14.310 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117055]: 2024-04-02 13:59:14.00355439 +0000 UTC m=+0.644769131 container remove 844c21a7ed1171202db62f3d03e435b89aa029f285709c2aa454391aa75a0157 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T13:59:14.313 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:14.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:14.313 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:14.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:14.313 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:14.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:14.314 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:14 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:14.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:14.571 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.209984904 +0000 UTC m=+0.018540261 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:14.572 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.337697878 +0000 UTC m=+0.146253222 container create 6ee10d083205d7886abaecfa4558ee187cfbd2f9fc5c406bf016e9e9f370e09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326) 2024-04-02T13:59:14.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:14.321+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:14.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:14.321+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:14.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:14.321+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:14.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:14.371+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:14.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:14.371+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:14.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:14.371+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:14.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:14 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:14.371+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:14.868 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.574395051 +0000 UTC m=+0.382950398 container init 6ee10d083205d7886abaecfa4558ee187cfbd2f9fc5c406bf016e9e9f370e09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326) 2024-04-02T13:59:14.868 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.577810409 +0000 UTC m=+0.386365751 container start 6ee10d083205d7886abaecfa4558ee187cfbd2f9fc5c406bf016e9e9f370e09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T13:59:14.868 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.657212497 +0000 UTC m=+0.465767847 container attach 6ee10d083205d7886abaecfa4558ee187cfbd2f9fc5c406bf016e9e9f370e09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:59:14.935 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:14.525+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:14.935 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:14.525+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:14.935 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:14.525+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:15.154 INFO:journalctl@ceph.osd.6.smithi122.stdout:Apr 02 13:59:14 smithi122 podman[117150]: 2024-04-02 13:59:14.865896401 +0000 UTC m=+0.674451759 container died 6ee10d083205d7886abaecfa4558ee187cfbd2f9fc5c406bf016e9e9f370e09c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:59:15.155 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:15 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:15.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:15.155 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:15 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:15.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:15.155 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:15 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:15.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:15.155 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:15 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:15.059+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:15.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:14.931+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:15.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:14.931+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:15.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:14 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:14.931+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:15.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.6.service' 2024-04-02T13:59:15.544 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:15.544 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-02T13:59:15.544 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-02T13:59:15.545 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.7 2024-04-02T13:59:15.573 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.298+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:15.573 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.298+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:15.573 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.298+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:15.574 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:15.374+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:15.574 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:15.374+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:15.574 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:15.374+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:15.574 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:15.374+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:15.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:15.482+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:15.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:15.482+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:15.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:15.482+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:15.833 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 systemd[1]: Stopping Ceph osd.7 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:16.236 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.829+0000 7fc549cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:16.236 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.829+0000 7fc549cf0700 -1 osd.7 131 *** Got signal Terminated *** 2024-04-02T13:59:16.236 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:15 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:15.829+0000 7fc549cf0700 -1 osd.7 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:16.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:15.972+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:16.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:15.972+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:16.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:15 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:15.972+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:16.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:16 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:16.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:16.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:16 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:16.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:16.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:16 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:16.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:16.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:16 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:16.087+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:16.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:16.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:16.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:16.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:16.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:16.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:16.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:16.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:16.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:16.322+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:16.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:16.322+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:16.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:16 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:16.322+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:16.770 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:16.492+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:16.770 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:16.492+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:16.770 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:16.492+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:17.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:16.985+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:17.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:16.985+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:17.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:16 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:16.985+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:17.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:17 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:17.095+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:17.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:17 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:17.095+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:17.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:17 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:17.095+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:17.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:17 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:17.095+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:17.638 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:17.304+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:17.638 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:17.304+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:17.638 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:17.304+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:17.638 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:17.346+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:17.639 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:17.346+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:17.639 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:17.346+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:17.639 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:17.346+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:17.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:17.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:17.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:17.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:17.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:17.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:18.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:17.967+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:18.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:17.967+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:18.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:17 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:17.967+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:18.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:18.080+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:18.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:18.080+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:18.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:18.080+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:18.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:18 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:18.080+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:18.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:18.329+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:18.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:18.329+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:18.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:18.329+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:18.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:18.329+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:18.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:18.329+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:18.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:18.334+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:18.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:18.334+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:18.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:18 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:18.334+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:18.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:18.485+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:18.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:18.485+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:18.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:18.485+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:18.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:18.485+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:19.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:18.983+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:19.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:18.983+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:19.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:18 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:18.983+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:19.472 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:19 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:19.112+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:19.473 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:19 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:19.112+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:19.473 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:19 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:19.112+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:19.473 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:19 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:19.112+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:19.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:19.349+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:19.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:19.349+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:19.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:19.349+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:19.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:19.349+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:19.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:19.349+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:19.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:19.348+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:19.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:19.348+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:19.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:19 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:19.348+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:19.939 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:19.519+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:19.939 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:19.519+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:19.939 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:19.519+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:19.939 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:19.519+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:20.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:19.935+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:20.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:19.935+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:20.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:19.935+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:20.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:19 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:19.935+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:20.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:20.081+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:20.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:20.081+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:20.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:20.081+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:20.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:20 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:20.081+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:20.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:20.332+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:31.847251+0000 front 2024-04-02T13:58:31.847279+0000 (oldest deadline 2024-04-02T13:58:54.747396+0000) 2024-04-02T13:59:20.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:20.332+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.849698+0000 front 2024-04-02T13:58:39.849619+0000 (oldest deadline 2024-04-02T13:59:03.949789+0000) 2024-04-02T13:59:20.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7[54224]: 2024-04-02T13:59:20.332+0000 7fc5462e4700 -1 osd.7 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:46.851056+0000 front 2024-04-02T13:58:46.851377+0000 (oldest deadline 2024-04-02T13:59:10.951259+0000) 2024-04-02T13:59:20.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:20.385+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:20.738 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:20.385+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:20.738 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:20.385+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:20.738 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:20.385+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:20.739 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:20 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:20.385+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:20.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:20.504+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:20.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:20.504+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:20.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:20.504+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:20.769 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:20.504+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:21.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:20.888+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:21.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:20.888+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:21.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:20.888+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:21.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:20 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:20.888+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:21.551 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:21 smithi187 podman[76998]: 2024-04-02 13:59:21.245687126 +0000 UTC m=+5.514874098 container died db582118161f3904ee1f6baca41047776eb9e85be706002191b81387e54d799b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:59:21.551 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:21.391+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:21.551 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:21.391+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:21.552 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:21.391+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:21.552 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:21.391+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:21.552 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:21.391+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:21.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:21.069+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:21.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:21.069+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:21.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:21.069+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:21.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:21 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:21.069+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:21.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:21.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:21.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:21.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:21.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:21.893 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:21 smithi187 podman[76998]: 2024-04-02 13:59:21.647365535 +0000 UTC m=+5.916552498 container cleanup db582118161f3904ee1f6baca41047776eb9e85be706002191b81387e54d799b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T13:59:21.893 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:21 smithi187 bash[76998]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7 2024-04-02T13:59:22.218 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:21 smithi187 podman[77027]: 2024-04-02 13:59:21.890347289 +0000 UTC m=+0.635111783 container remove db582118161f3904ee1f6baca41047776eb9e85be706002191b81387e54d799b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T13:59:22.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:21.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:22.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:21.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:22.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:21.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:22.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:21 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:21.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:22.480 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.118245911 +0000 UTC m=+0.017035824 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:22.481 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.224727401 +0000 UTC m=+0.123517300 container create 1120d80e84baffc965202845a61a8a68b8f6bd30343248006b59f7939581f6e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True) 2024-04-02T13:59:22.481 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:22 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:22.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:22.481 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:22 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:22.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:22.481 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:22 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:22.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:22.482 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:22 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:22.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:22.482 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:22 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:22.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:22.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:22.091+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:22.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:22.091+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:22.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:22.091+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:22.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:22.091+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:22.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:22 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:22.091+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:22.736 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.477429635 +0000 UTC m=+0.376219534 container init 1120d80e84baffc965202845a61a8a68b8f6bd30343248006b59f7939581f6e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-04-02T13:59:22.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.480616839 +0000 UTC m=+0.379406741 container start 1120d80e84baffc965202845a61a8a68b8f6bd30343248006b59f7939581f6e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True) 2024-04-02T13:59:22.737 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.577374513 +0000 UTC m=+0.476164430 container attach 1120d80e84baffc965202845a61a8a68b8f6bd30343248006b59f7939581f6e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T13:59:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:22.467+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:22.467+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:22.467+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:22.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:22.467+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:23.066 INFO:journalctl@ceph.osd.7.smithi187.stdout:Apr 02 13:59:22 smithi187 podman[77116]: 2024-04-02 13:59:22.764706935 +0000 UTC m=+0.663496837 container died 1120d80e84baffc965202845a61a8a68b8f6bd30343248006b59f7939581f6e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T13:59:23.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:22.824+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:23.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:22.824+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:23.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:22.824+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:23.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:22 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:22.824+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:23.375 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.7.service' 2024-04-02T13:59:23.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:23.460 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-02T13:59:23.460 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-02T13:59:23.460 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.8 2024-04-02T13:59:23.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:23.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:23.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:23.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:23.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:23.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:23.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:23.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:23.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:23 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:23.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:23.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:23 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:23.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:23.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:23 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:23.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:23.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:23 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:23.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:23.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:23 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:23.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:23.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:23 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:23.339+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:23.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:23.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:23.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:23.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.479+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:23.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 systemd[1]: Stopping Ceph osd.8 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.767+0000 7fc6c67ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.767+0000 7fc6c67ea700 -1 osd.8 131 *** Got signal Terminated *** 2024-04-02T13:59:24.268 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:23.767+0000 7fc6c67ea700 -1 osd.8 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:24.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:23.827+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:24.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:23.827+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:24.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:23.827+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:24.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:23 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:23.827+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:24.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:24.133+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:24.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:24.133+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:24.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:24.133+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:24.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:24.133+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:24.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:24 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:24.133+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:24.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:24 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:24.358+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:24.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:24 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:24.358+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:24.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:24 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:24.358+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:24.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:24 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:24.358+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:24.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:24 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:24.358+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:24.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:24.529+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:24.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:24.529+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:24.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:24.529+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:24.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:24.529+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:25.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:24.834+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:25.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:24.834+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:25.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:24.834+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:25.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:24 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:24.834+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:25.574 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:25.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:25.574 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:25.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:25.575 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:25.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:25.575 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:25.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:25.575 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:25 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:25.141+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:25.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:25.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:25.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:25.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:25.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:25.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:25.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:25.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:25.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:25.373+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:25.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:25.511+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:25.767 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:25.511+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:25.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:25.511+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:25.768 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:25.511+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:26.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:25.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:26.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:25.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:26.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:25.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:26.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:25 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:25.867+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:26.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:26.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:26.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:26.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:26.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:26.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:26 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:26.138+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:26.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:26.413+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:26.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:26.413+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:26.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:26.413+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:26.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:26.413+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:26.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:26 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:26.413+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:26.858 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:26.528+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:26.858 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:26.528+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:26.858 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:26.528+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:26.858 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:26.528+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:27.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:26.854+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:27.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:26.854+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:27.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:26.854+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:27.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:26.854+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:27.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:26 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:26.854+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:27.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:27.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:27.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:27.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:27.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:27.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:27 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:27.188+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:27.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:27 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:27.451+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:27.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:27 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:27.451+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:27.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:27 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:27.451+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:27.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:27 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:27.451+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:27.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:27 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:27.451+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:27.832 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:27.560+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:27.832 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:27.560+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:27.832 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:27.560+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:27.832 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:27.560+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:28.267 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:27.828+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:28.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:27.828+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:28.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:27.828+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:28.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:27.828+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:28.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:27 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:27.828+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:28.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:28.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:28.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:28.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:28.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:28.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:28 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:28.145+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:28.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:28 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:28.414+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:28.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:28 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:28.414+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:28.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:28 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:28.414+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:28.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:28 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:28.414+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:28.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:28 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:28.414+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:28.875 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:28.520+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.476491+0000 front 2024-04-02T13:58:21.476568+0000 (oldest deadline 2024-04-02T13:58:45.477206+0000) 2024-04-02T13:59:28.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:28.520+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.178613+0000 front 2024-04-02T13:58:30.178743+0000 (oldest deadline 2024-04-02T13:58:54.878643+0000) 2024-04-02T13:59:28.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:28.520+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.880445+0000 front 2024-04-02T13:58:38.880472+0000 (oldest deadline 2024-04-02T13:59:01.780618+0000) 2024-04-02T13:59:28.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:28.520+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:52.982672+0000 front 2024-04-02T13:58:52.982582+0000 (oldest deadline 2024-04-02T13:59:17.682597+0000) 2024-04-02T13:59:28.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8[55163]: 2024-04-02T13:59:28.520+0000 7fc6c2dde700 -1 osd.8 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:04.683968+0000 front 2024-04-02T13:59:04.683946+0000 (oldest deadline 2024-04-02T13:59:28.184177+0000) 2024-04-02T13:59:29.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:28.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:29.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:28.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:29.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:28.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:29.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:28.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:29.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:28 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:28.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:29.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:29.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:29.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:29.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:29.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:29.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:29 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:29.135+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:29.628 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:29 smithi188 podman[77552]: 2024-04-02 13:59:29.34790841 +0000 UTC m=+5.691727123 container died 91c2996826846deaf5dd7894faee99083b88c291e7b7d3597dfda366a0b6edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-02T13:59:29.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:29.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:29.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:29.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:29.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:29.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:29 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:29.439+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:29.910 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:29 smithi188 podman[77552]: 2024-04-02 13:59:29.718107128 +0000 UTC m=+6.061925841 container cleanup 91c2996826846deaf5dd7894faee99083b88c291e7b7d3597dfda366a0b6edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T13:59:29.910 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:29 smithi188 bash[77552]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8 2024-04-02T13:59:30.267 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:29 smithi188 podman[77569]: 2024-04-02 13:59:29.986185023 +0000 UTC m=+0.635938063 container remove 91c2996826846deaf5dd7894faee99083b88c291e7b7d3597dfda366a0b6edaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8, GIT_CLEAN=True, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T13:59:30.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:29.905+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:30.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:29.905+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:30.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:29.905+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:30.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:29.905+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:30.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:29.905+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:30.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:30.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:30.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:30.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:30.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:30.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:30 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:30.144+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:30.584 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.232333894 +0000 UTC m=+0.017638041 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:30.585 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.337174742 +0000 UTC m=+0.122478865 container create 9552b012bc094cfed4df9181c3193be94686f862794158af34e6492902835df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-02T13:59:30.585 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.582101445 +0000 UTC m=+0.367405586 container init 9552b012bc094cfed4df9181c3193be94686f862794158af34e6492902835df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-02T13:59:30.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:30.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:30.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:30.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:30.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:30.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:30 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:30.452+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:30.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.585591881 +0000 UTC m=+0.370896012 container start 9552b012bc094cfed4df9181c3193be94686f862794158af34e6492902835df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-02T13:59:30.876 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.681818271 +0000 UTC m=+0.467122418 container attach 9552b012bc094cfed4df9181c3193be94686f862794158af34e6492902835df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-02T13:59:31.255 INFO:journalctl@ceph.osd.8.smithi188.stdout:Apr 02 13:59:30 smithi188 podman[77663]: 2024-04-02 13:59:30.873606831 +0000 UTC m=+0.658910972 container died 9552b012bc094cfed4df9181c3193be94686f862794158af34e6492902835df5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-02T13:59:31.255 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:30 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:30.874+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:31.256 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:30 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:30.874+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:31.256 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:30 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:30.874+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:31.256 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:30 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:30.874+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:31.257 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:30 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:30.874+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:31.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.8.service' 2024-04-02T13:59:31.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:31.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:31.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:31.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:31.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:31.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.108+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:31.578 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:31.578 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-02T13:59:31.578 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-02T13:59:31.578 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.9 2024-04-02T13:59:31.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:31.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:31.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:31.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:31.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:31.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:31 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:31.481+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:31.892 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 systemd[1]: Stopping Ceph osd.9 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:32.159 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.887+0000 7f240bd74700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:32.159 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.887+0000 7f240bd74700 -1 osd.9 131 *** Got signal Terminated *** 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:31 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:31.887+0000 7f240bd74700 -1 osd.9 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:32.160 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:32 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:32.155+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:32.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:31.831+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:32.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:31.831+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:32.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:31.831+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:32.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:31.831+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:32.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:31 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:31.831+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:32.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:32.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:32.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:32.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:32.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:32.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:32 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:32.489+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:33.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:32 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:32.819+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:33.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:32 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:32.819+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:33.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:32 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:32.819+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:33.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:32 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:32.819+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:33.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:32 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:32.819+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:33.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:33.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:33.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:33.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:33.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:33.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:33 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:33.172+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:33.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:33.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:33.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:33.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:33.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:33.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:33 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:33.523+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:34.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:33.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:34.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:33.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:34.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:33.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:34.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:33.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:34.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:33.852+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:34.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:34.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:34.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:34.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:34.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:34.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:34 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:34.189+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:34.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:34.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:34.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:34.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:34.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:34.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:34 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:34.557+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:35.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:35.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:35.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:35.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:35.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:35.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:34 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:34.826+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:35.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:35.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:35.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:35.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:35.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:35.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:35 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:35.198+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:35.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:35.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:35.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:35.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:35.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:35.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:35 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:35.529+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:36.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:36.295 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:36.295 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:36.295 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:36.295 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:36.296 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:35 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:35.875+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:36.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:21.872920+0000 front 2024-04-02T13:58:21.872971+0000 (oldest deadline 2024-04-02T13:58:47.173122+0000) 2024-04-02T13:59:36.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:30.074378+0000 front 2024-04-02T13:58:30.074403+0000 (oldest deadline 2024-04-02T13:58:55.374496+0000) 2024-04-02T13:59:36.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:39.475794+0000 front 2024-04-02T13:58:39.475725+0000 (oldest deadline 2024-04-02T13:59:02.276481+0000) 2024-04-02T13:59:36.571 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.577529+0000 front 2024-04-02T13:58:44.577509+0000 (oldest deadline 2024-04-02T13:59:10.477681+0000) 2024-04-02T13:59:36.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.080211+0000 front 2024-04-02T13:58:56.080143+0000 (oldest deadline 2024-04-02T13:59:21.380063+0000) 2024-04-02T13:59:36.572 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:36 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9[64491]: 2024-04-02T13:59:36.236+0000 7f2408368700 -1 osd.9 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:01.380878+0000 front 2024-04-02T13:59:01.380967+0000 (oldest deadline 2024-04-02T13:59:26.080737+0000) 2024-04-02T13:59:36.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:36.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:36.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:36.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:36.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:36.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:36.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:36 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:36.543+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:37.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:37.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:37.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:37.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:36 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:36.900+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:37.632 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:37 smithi122 podman[117267]: 2024-04-02 13:59:37.33810344 +0000 UTC m=+5.560485673 container died 31c40174ba354d660be58e844a4b7792b4387aa1dbc1082bf4377b1dbb3749dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:59:37.884 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:37 smithi122 podman[117267]: 2024-04-02 13:59:37.629881372 +0000 UTC m=+5.852263606 container cleanup 31c40174ba354d660be58e844a4b7792b4387aa1dbc1082bf4377b1dbb3749dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:59:37.884 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:37 smithi122 bash[117267]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9 2024-04-02T13:59:37.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:37.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:37.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:37.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:37.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:37.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:37.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:37 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:37.514+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:38.182 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:37 smithi122 podman[117284]: 2024-04-02 13:59:37.881201217 +0000 UTC m=+0.538568253 container remove 31c40174ba354d660be58e844a4b7792b4387aa1dbc1082bf4377b1dbb3749dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-02T13:59:38.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:38.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:38.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:38.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:38.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:38.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:37.914+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:38.432 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.081841168 +0000 UTC m=+0.017866189 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:38.432 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.190305985 +0000 UTC m=+0.126330993 container create b0ea4af9b056b1d5a1d3b89086d48fa0622eed950acdee15991e91b4707b5250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:59:38.432 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.426331901 +0000 UTC m=+0.362356911 container init b0ea4af9b056b1d5a1d3b89086d48fa0622eed950acdee15991e91b4707b5250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:59:38.433 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.429626083 +0000 UTC m=+0.365651086 container start b0ea4af9b056b1d5a1d3b89086d48fa0622eed950acdee15991e91b4707b5250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD) 2024-04-02T13:59:38.718 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.526167603 +0000 UTC m=+0.462192612 container attach b0ea4af9b056b1d5a1d3b89086d48fa0622eed950acdee15991e91b4707b5250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-02T13:59:38.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:38.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:38.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:38.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:38.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:38.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:38.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:38 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:38.503+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:38.998 INFO:journalctl@ceph.osd.9.smithi122.stdout:Apr 02 13:59:38 smithi122 podman[117373]: 2024-04-02 13:59:38.715710162 +0000 UTC m=+0.651735173 container died b0ea4af9b056b1d5a1d3b89086d48fa0622eed950acdee15991e91b4707b5250 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-9-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, ceph=True) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:39.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:38 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:38.948+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:39.294 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.9.service' 2024-04-02T13:59:39.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:39.338 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-02T13:59:39.338 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-02T13:59:39.339 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.10 2024-04-02T13:59:39.644 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 systemd[1]: Stopping Ceph osd.10 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:39.644 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.471+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:39.645 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.640+0000 7f6fb4a2b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:39.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.640+0000 7f6fb4a2b700 -1 osd.10 131 *** Got signal Terminated *** 2024-04-02T13:59:39.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:39 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:39.640+0000 7f6fb4a2b700 -1 osd.10 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.919+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.920+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.920+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.920+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.920+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:40.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:39 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:39.920+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:40.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:40.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:40.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:40.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:40.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:40.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:40.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:40 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:40.472+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:41.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:41.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:41.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:41.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:41.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:41.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:40 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:40.934+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:41.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:41.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:41.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:41.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:41.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:41.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:41.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:41 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:41.447+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:42.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:41.979+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:42.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:42.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:42.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:42.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:42.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:42.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:42.987 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:42 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:42.483+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:43.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:43.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:43.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:43.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:43.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:43.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:42 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:42.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:43.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:43.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:43.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:43.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:43.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:43.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:43.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:43 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:43.479+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:44.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:43 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:43.987+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:44.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:20.630435+0000 front 2024-04-02T13:58:20.630634+0000 (oldest deadline 2024-04-02T13:58:45.930572+0000) 2024-04-02T13:59:44.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.231778+0000 front 2024-04-02T13:58:28.231803+0000 (oldest deadline 2024-04-02T13:58:53.532014+0000) 2024-04-02T13:59:44.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.133746+0000 front 2024-04-02T13:58:38.133893+0000 (oldest deadline 2024-04-02T13:59:04.033855+0000) 2024-04-02T13:59:44.736 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6814 osd.3 since back 2024-04-02T13:58:44.034432+0000 front 2024-04-02T13:58:44.034400+0000 (oldest deadline 2024-04-02T13:59:09.334473+0000) 2024-04-02T13:59:44.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:54.035441+0000 front 2024-04-02T13:58:54.035477+0000 (oldest deadline 2024-04-02T13:59:17.535585+0000) 2024-04-02T13:59:44.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.337050+0000 front 2024-04-02T13:59:03.337181+0000 (oldest deadline 2024-04-02T13:59:28.637038+0000) 2024-04-02T13:59:44.737 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:44 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10[61281]: 2024-04-02T13:59:44.461+0000 7f6fad017700 -1 osd.10 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:10.739639+0000 front 2024-04-02T13:59:10.739726+0000 (oldest deadline 2024-04-02T13:59:36.039547+0000) 2024-04-02T13:59:45.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:45 smithi187 podman[77221]: 2024-04-02 13:59:45.178497491 +0000 UTC m=+5.651094704 container died f7ed1bcae85f848643c8ab34b37ee4dbb17d4a866a67fbd9050550c205045769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:45.518 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.035+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:45.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:45 smithi187 podman[77221]: 2024-04-02 13:59:45.657267299 +0000 UTC m=+6.129864511 container cleanup f7ed1bcae85f848643c8ab34b37ee4dbb17d4a866a67fbd9050550c205045769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T13:59:45.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:45 smithi187 bash[77221]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10 2024-04-02T13:59:45.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:45 smithi187 podman[77238]: 2024-04-02 13:59:45.816640803 +0000 UTC m=+0.637722865 container remove f7ed1bcae85f848643c8ab34b37ee4dbb17d4a866a67fbd9050550c205045769 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T13:59:46.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:46.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:46.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:46.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:46.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:46.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:45 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:45.986+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:46.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.22589583 +0000 UTC m=+0.114532177 container create 62b74b63bf0979e83e8c083bb28a01a3bb9eb3e489cbe11fd9f650865a5a7192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326) 2024-04-02T13:59:46.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.129328697 +0000 UTC m=+0.017965011 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:46.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.444845037 +0000 UTC m=+0.333481373 container init 62b74b63bf0979e83e8c083bb28a01a3bb9eb3e489cbe11fd9f650865a5a7192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T13:59:46.486 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.447964141 +0000 UTC m=+0.336600453 container start 62b74b63bf0979e83e8c083bb28a01a3bb9eb3e489cbe11fd9f650865a5a7192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T13:59:46.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.54486196 +0000 UTC m=+0.433498301 container attach 62b74b63bf0979e83e8c083bb28a01a3bb9eb3e489cbe11fd9f650865a5a7192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-02T13:59:46.986 INFO:journalctl@ceph.osd.10.smithi187.stdout:Apr 02 13:59:46 smithi187 podman[77328]: 2024-04-02 13:59:46.740716424 +0000 UTC m=+0.629352754 container died 62b74b63bf0979e83e8c083bb28a01a3bb9eb3e489cbe11fd9f650865a5a7192 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-02T13:59:47.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:47.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:47.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:47.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:47.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:47.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:47.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:46 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:46.969+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:47.333 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.10.service' 2024-04-02T13:59:47.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:47.372 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-02T13:59:47.373 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-02T13:59:47.373 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.11 2024-04-02T13:59:47.682 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 systemd[1]: Stopping Ceph osd.11 for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:47.980 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.678+0000 7f7351e11700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:47.981 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.678+0000 7f7351e11700 -1 osd.11 131 *** Got signal Terminated *** 2024-04-02T13:59:47.981 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.678+0000 7f7351e11700 -1 osd.11 131 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:48.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:47 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:47.977+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:49.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:49.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:48 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:48.947+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:50.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:49 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:49.915+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:51.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:51.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:50 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:50.871+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6806 osd.0 since back 2024-04-02T13:58:23.414626+0000 front 2024-04-02T13:58:23.414651+0000 (oldest deadline 2024-04-02T13:58:48.714612+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6804 osd.1 since back 2024-04-02T13:58:28.715303+0000 front 2024-04-02T13:58:28.715287+0000 (oldest deadline 2024-04-02T13:58:53.415363+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6804 osd.2 since back 2024-04-02T13:58:38.116536+0000 front 2024-04-02T13:58:38.116565+0000 (oldest deadline 2024-04-02T13:59:02.216685+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6812 osd.4 since back 2024-04-02T13:58:56.120119+0000 front 2024-04-02T13:58:56.120200+0000 (oldest deadline 2024-04-02T13:59:19.020122+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6812 osd.5 since back 2024-04-02T13:59:03.021717+0000 front 2024-04-02T13:59:03.021677+0000 (oldest deadline 2024-04-02T13:59:25.921766+0000) 2024-04-02T13:59:52.268 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.122:6822 osd.6 since back 2024-04-02T13:59:09.323748+0000 front 2024-04-02T13:59:09.323710+0000 (oldest deadline 2024-04-02T13:59:34.623834+0000) 2024-04-02T13:59:52.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.187:6820 osd.7 since back 2024-04-02T13:59:18.124845+0000 front 2024-04-02T13:59:18.124874+0000 (oldest deadline 2024-04-02T13:59:46.325396+0000) 2024-04-02T13:59:52.269 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:51 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11[61799]: 2024-04-02T13:59:51.845+0000 7f734a3fd700 -1 osd.11 131 heartbeat_check: no reply from 172.21.15.188:6820 osd.8 since back 2024-04-02T13:59:26.826161+0000 front 2024-04-02T13:59:26.826175+0000 (oldest deadline 2024-04-02T13:59:51.526318+0000) 2024-04-02T13:59:53.356 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:53 smithi188 podman[77781]: 2024-04-02 13:59:53.085323844 +0000 UTC m=+5.519516235 container died 49d58663acd94178d184017edaa14e7a3c9e618b169e7df0f6a1d5e2187192a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:59:53.723 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:53 smithi188 podman[77781]: 2024-04-02 13:59:53.452755604 +0000 UTC m=+5.886947986 container cleanup 49d58663acd94178d184017edaa14e7a3c9e618b169e7df0f6a1d5e2187192a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240326) 2024-04-02T13:59:53.723 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:53 smithi188 bash[77781]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11 2024-04-02T13:59:54.018 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:53 smithi188 podman[77797]: 2024-04-02 13:59:53.720692311 +0000 UTC m=+0.633306194 container remove 49d58663acd94178d184017edaa14e7a3c9e618b169e7df0f6a1d5e2187192a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, io.buildah.version=1.33.5) 2024-04-02T13:59:54.302 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:53.927824329 +0000 UTC m=+0.017941859 image pull f7301e070545fc23d8e5b7b052a2c15e832a0e57102f1acad8e4f3583709d23f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534 2024-04-02T13:59:54.303 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:54.029762207 +0000 UTC m=+0.119879722 container create 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True) 2024-04-02T13:59:54.303 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:54.232130724 +0000 UTC m=+0.322248243 container init 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-02T13:59:54.303 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:54.235122744 +0000 UTC m=+0.325240260 container start 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:59:54.303 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:54.29872557 +0000 UTC m=+0.388843103 container attach 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T13:59:54.768 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:54 smithi188 podman[77887]: 2024-04-02 13:59:54.539083234 +0000 UTC m=+0.629200745 container died 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-02T13:59:55.122 INFO:journalctl@ceph.osd.11.smithi188.stdout:Apr 02 13:59:55 smithi188 podman[77914]: 2024-04-02 13:59:55.102232377 +0000 UTC m=+0.557867255 container remove 07133c9c4bd136eb3313347cc8e0954d8772db9bce1baf4cfbae544daa57a2b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T13:59:55.127 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@osd.11.service' 2024-04-02T13:59:55.169 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T13:59:55.169 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-02T13:59:55.169 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-02T13:59:55.169 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.a 2024-04-02T13:59:55.544 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:55 smithi122 systemd[1]: Stopping Ceph mds.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T13:59:55.821 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:55 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a[71451]: 2024-04-02T13:59:55.539+0000 7fd14b476700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T13:59:55.821 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:55 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a[71451]: 2024-04-02T13:59:55.539+0000 7fd14b476700 -1 mds.a *** got signal Terminated *** 2024-04-02T13:59:59.810 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:59 smithi122 podman[117494]: 2024-04-02 13:59:59.54715182 +0000 UTC m=+4.181647449 container died 9c52f184c1e202b79a2a8727e3bfe580f2866755473aeb5708d8b9cd2e1a44f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:00:00.071 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:59 smithi122 podman[117494]: 2024-04-02 13:59:59.898442247 +0000 UTC m=+4.532937883 container cleanup 9c52f184c1e202b79a2a8727e3bfe580f2866755473aeb5708d8b9cd2e1a44f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-02T14:00:00.071 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 13:59:59 smithi122 bash[117494]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a 2024-04-02T14:00:00.334 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 14:00:00 smithi122 podman[117508]: 2024-04-02 14:00:00.191407573 +0000 UTC m=+0.640779435 container remove 9c52f184c1e202b79a2a8727e3bfe580f2866755473aeb5708d8b9cd2e1a44f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-02T14:00:00.334 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 14:00:00 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.a.service: Deactivated successfully. 2024-04-02T14:00:00.335 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 14:00:00 smithi122 systemd[1]: Stopped Ceph mds.a for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T14:00:00.335 INFO:journalctl@ceph.mds.a.smithi122.stdout:Apr 02 14:00:00 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.a.service: Consumed 8min 54.027s CPU time. 2024-04-02T14:00:00.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.a.service' 2024-04-02T14:00:00.375 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:00.375 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-02T14:00:00.375 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-02T14:00:00.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.d 2024-04-02T14:00:00.618 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:00 smithi122 systemd[1]: Stopping Ceph mds.d for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:01.071 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d[70501]: 2024-04-02T14:00:00.702+0000 7f04ed68f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:01.071 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:00 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d[70501]: 2024-04-02T14:00:00.702+0000 7f04ed68f700 -1 mds.d *** got signal Terminated *** 2024-04-02T14:00:03.779 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:03 smithi122 podman[117615]: 2024-04-02 14:00:03.490098243 +0000 UTC m=+2.898714043 container died 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-02T14:00:04.056 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:03 smithi122 podman[117615]: 2024-04-02 14:00:03.835866203 +0000 UTC m=+3.244481971 container cleanup 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5) 2024-04-02T14:00:04.057 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:03 smithi122 bash[117615]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d 2024-04-02T14:00:04.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.d.service' 2024-04-02T14:00:04.307 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:04 smithi122 podman[117629]: 2024-04-02 14:00:04.053710964 +0000 UTC m=+0.557722386 container remove 80318e11bbb17298acfdd6e195ccfe86e13212303e5ef1e04f3f9330321afb1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.build-date=20240326, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, maintainer=Guillaume Abrioux ) 2024-04-02T14:00:04.307 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:04 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.d.service: Deactivated successfully. 2024-04-02T14:00:04.307 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:04 smithi122 systemd[1]: Stopped Ceph mds.d for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T14:00:04.307 INFO:journalctl@ceph.mds.d.smithi122.stdout:Apr 02 14:00:04 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.d.service: Consumed 1.162s CPU time. 2024-04-02T14:00:04.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:04.319 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-02T14:00:04.319 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-02T14:00:04.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.g 2024-04-02T14:00:04.571 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:04 smithi122 systemd[1]: Stopping Ceph mds.g for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:05.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: 2024-04-02T14:00:04.648+0000 7fd79a6b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:05.071 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:04 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g[71094]: 2024-04-02T14:00:04.648+0000 7fd79a6b9700 -1 mds.g *** got signal Terminated *** 2024-04-02T14:00:08.552 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:08 smithi122 podman[117737]: 2024-04-02 14:00:08.284692152 +0000 UTC m=+3.746886418 container died 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0) 2024-04-02T14:00:08.821 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:08 smithi122 podman[117737]: 2024-04-02 14:00:08.631400759 +0000 UTC m=+4.093595032 container cleanup 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-02T14:00:08.821 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:08 smithi122 bash[117737]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g 2024-04-02T14:00:09.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.g.service' 2024-04-02T14:00:09.092 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:08 smithi122 podman[117751]: 2024-04-02 14:00:08.824109484 +0000 UTC m=+0.531731846 container remove 9a85fd395f00b21d9a9e33b81f8ce3cf23cf056e0d1855dd610ee1072d3a3ab8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-g, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-02T14:00:09.093 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:09 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.g.service: Deactivated successfully. 2024-04-02T14:00:09.093 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:09 smithi122 systemd[1]: Stopped Ceph mds.g for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T14:00:09.093 INFO:journalctl@ceph.mds.g.smithi122.stdout:Apr 02 14:00:09 smithi122 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.g.service: Consumed 28.114s CPU time. 2024-04-02T14:00:09.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:09.103 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-02T14:00:09.103 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-02T14:00:09.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.j 2024-04-02T14:00:09.421 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:09 smithi122 systemd[1]: Stopping Ceph mds.j for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:09.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j[70149]: 2024-04-02T14:00:09.418+0000 7f8f847e8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:09.821 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:09 smithi122 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j[70149]: 2024-04-02T14:00:09.418+0000 7f8f847e8700 -1 mds.j *** got signal Terminated *** 2024-04-02T14:00:12.929 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:12 smithi122 podman[117857]: 2024-04-02 14:00:12.608719717 +0000 UTC m=+3.292436509 container died 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240326, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS) 2024-04-02T14:00:12.930 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:12 smithi122 podman[117857]: 2024-04-02 14:00:12.926605859 +0000 UTC m=+3.610322642 container cleanup 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-02T14:00:13.321 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:12 smithi122 bash[117857]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j 2024-04-02T14:00:13.321 INFO:journalctl@ceph.mds.j.smithi122.stdout:Apr 02 14:00:13 smithi122 podman[117871]: 2024-04-02 14:00:13.144616609 +0000 UTC m=+0.536511549 container remove 9dfc2b0c1cfd91f7395a24ad31fbd31bcc7e474a2593ec84daf916f7a7025922 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-02T14:00:13.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.j.service' 2024-04-02T14:00:13.382 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:13.383 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-02T14:00:13.383 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-02T14:00:13.383 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.b 2024-04-02T14:00:13.693 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:13 smithi187 systemd[1]: Stopping Ceph mds.b for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:13.986 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: 2024-04-02T14:00:13.689+0000 7fdd4337e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:13.986 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:13 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b[66972]: 2024-04-02T14:00:13.689+0000 7fdd4337e700 -1 mds.b *** got signal Terminated *** 2024-04-02T14:00:16.569 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:16 smithi187 podman[77447]: 2024-04-02 14:00:16.301299326 +0000 UTC m=+2.721275624 container died 7db0b3aa574771c30562779d753b86a92ac087ae176ab883c400a7b87e52daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-02T14:00:16.935 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:16 smithi187 podman[77447]: 2024-04-02 14:00:16.664142001 +0000 UTC m=+3.084118262 container cleanup 7db0b3aa574771c30562779d753b86a92ac087ae176ab883c400a7b87e52daa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963) 2024-04-02T14:00:16.935 INFO:journalctl@ceph.mds.b.smithi187.stdout:Apr 02 14:00:16 smithi187 bash[77447]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-b 2024-04-02T14:00:17.086 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.b.service' 2024-04-02T14:00:17.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:17.124 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-02T14:00:17.125 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-02T14:00:17.125 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.e 2024-04-02T14:00:17.454 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:17 smithi187 systemd[1]: Stopping Ceph mds.e for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:17.736 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: 2024-04-02T14:00:17.450+0000 7fb9c28dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:17.736 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:17 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e[66345]: 2024-04-02T14:00:17.450+0000 7fb9c28dc700 -1 mds.e *** got signal Terminated *** 2024-04-02T14:00:20.629 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:20 smithi187 podman[77566]: 2024-04-02 14:00:20.302043792 +0000 UTC m=+2.952165681 container died 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:00:20.629 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:20 smithi187 podman[77566]: 2024-04-02 14:00:20.626276026 +0000 UTC m=+3.276397898 container cleanup 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, org.label-schema.build-date=20240326, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-02T14:00:20.986 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:20 smithi187 bash[77566]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e 2024-04-02T14:00:20.986 INFO:journalctl@ceph.mds.e.smithi187.stdout:Apr 02 14:00:20 smithi187 podman[77580]: 2024-04-02 14:00:20.844184905 +0000 UTC m=+0.537441313 container remove 88c84384d0db46f02ff4e3d11987327ae1e0325c219a94aaef6d06881d944387 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0) 2024-04-02T14:00:21.053 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.e.service' 2024-04-02T14:00:21.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:21.089 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-02T14:00:21.089 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-02T14:00:21.089 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.h 2024-04-02T14:00:21.408 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:21 smithi187 systemd[1]: Stopping Ceph mds.h for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:21.736 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: 2024-04-02T14:00:21.404+0000 7fefef060700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:21.736 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:21 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h[66654]: 2024-04-02T14:00:21.404+0000 7fefef060700 -1 mds.h *** got signal Terminated *** 2024-04-02T14:00:24.651 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:24 smithi187 podman[77688]: 2024-04-02 14:00:24.30510486 +0000 UTC m=+3.005601611 container died 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-02T14:00:24.984 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:24 smithi187 podman[77688]: 2024-04-02 14:00:24.72968457 +0000 UTC m=+3.430181311 container cleanup 891ac58885d0c2650a11bb239203da6c42d2d1e19ba741a708a19cae213ea1f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.schema-version=1.0) 2024-04-02T14:00:24.984 INFO:journalctl@ceph.mds.h.smithi187.stdout:Apr 02 14:00:24 smithi187 bash[77688]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-h 2024-04-02T14:00:25.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.h.service' 2024-04-02T14:00:25.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:25.138 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-02T14:00:25.138 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-02T14:00:25.139 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.k 2024-04-02T14:00:25.236 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:25 smithi187 systemd[1]: Stopping Ceph mds.k for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:25.736 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: 2024-04-02T14:00:25.450+0000 7f0f6adef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:25.736 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:25 smithi187 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k[66029]: 2024-04-02T14:00:25.450+0000 7f0f6adef700 -1 mds.k *** got signal Terminated *** 2024-04-02T14:00:28.637 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:28 smithi187 podman[77809]: 2024-04-02 14:00:28.300165967 +0000 UTC m=+2.951893347 container died 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, RELEASE=wip-batrick-testing-20240402.004512-191d963, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240326, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T14:00:28.920 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:28 smithi187 podman[77809]: 2024-04-02 14:00:28.708647222 +0000 UTC m=+3.360374600 container cleanup 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-02T14:00:28.921 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:28 smithi187 bash[77809]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k 2024-04-02T14:00:29.146 DEBUG:teuthology.orchestra.run.smithi187:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.k.service' 2024-04-02T14:00:29.171 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:28 smithi187 podman[77824]: 2024-04-02 14:00:28.917996414 +0000 UTC m=+0.524402235 container remove 179cdf46346402b7412146775c7199c7a5b3ee983c5ccf5e0ab1c4650f6a23bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-k, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-02T14:00:29.171 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:29 smithi187 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.k.service: Deactivated successfully. 2024-04-02T14:00:29.171 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:29 smithi187 systemd[1]: Stopped Ceph mds.k for 3c818384-f0f4-11ee-b647-cb9ed24678a4. 2024-04-02T14:00:29.171 INFO:journalctl@ceph.mds.k.smithi187.stdout:Apr 02 14:00:29 smithi187 systemd[1]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.k.service: Consumed 30.987s CPU time. 2024-04-02T14:00:29.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:29.180 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-02T14:00:29.180 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-02T14:00:29.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.c 2024-04-02T14:00:29.477 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:29 smithi188 systemd[1]: Stopping Ceph mds.c for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:29.744 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: 2024-04-02T14:00:29.473+0000 7fbc1309d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:29.745 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:29 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c[67177]: 2024-04-02T14:00:29.473+0000 7fbc1309d700 -1 mds.c *** got signal Terminated *** 2024-04-02T14:00:32.626 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:32 smithi188 podman[77994]: 2024-04-02 14:00:32.300181721 +0000 UTC m=+2.924424989 container died 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326) 2024-04-02T14:00:32.627 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:32 smithi188 podman[77994]: 2024-04-02 14:00:32.623567643 +0000 UTC m=+3.247810897 container cleanup 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.license=GPLv2) 2024-04-02T14:00:33.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:32 smithi188 bash[77994]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c 2024-04-02T14:00:33.018 INFO:journalctl@ceph.mds.c.smithi188.stdout:Apr 02 14:00:32 smithi188 podman[78020]: 2024-04-02 14:00:32.816890205 +0000 UTC m=+0.511389403 container remove 1d13ad6701b02bf5f4880b23fdeb9e418da81ed3b2944a7aeb58fba203b19262 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, org.label-schema.build-date=20240326, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-02T14:00:33.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.c.service' 2024-04-02T14:00:33.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:33.090 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-02T14:00:33.090 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-02T14:00:33.090 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.f 2024-04-02T14:00:33.397 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:33 smithi188 systemd[1]: Stopping Ceph mds.f for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:33.767 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: 2024-04-02T14:00:33.393+0000 7fb05dfef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:33.768 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:33 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f[66869]: 2024-04-02T14:00:33.393+0000 7fb05dfef700 -1 mds.f *** got signal Terminated *** 2024-04-02T14:00:36.564 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:36 smithi188 podman[78127]: 2024-04-02 14:00:36.304089439 +0000 UTC m=+3.000064629 container died 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326) 2024-04-02T14:00:36.839 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:36 smithi188 podman[78127]: 2024-04-02 14:00:36.635319031 +0000 UTC m=+3.331294203 container cleanup 0b7cff888dff40af314bec6cb08b070e044416f72d761032ac5b7cbc8a3e974e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240326, GIT_CLEAN=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-02T14:00:36.839 INFO:journalctl@ceph.mds.f.smithi188.stdout:Apr 02 14:00:36 smithi188 bash[78127]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-f 2024-04-02T14:00:37.036 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.f.service' 2024-04-02T14:00:37.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:37.073 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-02T14:00:37.074 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-02T14:00:37.074 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.i 2024-04-02T14:00:37.392 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:37 smithi188 systemd[1]: Stopping Ceph mds.i for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: 2024-04-02T14:00:37.388+0000 7f0a3f3e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:37.768 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:37 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i[67500]: 2024-04-02T14:00:37.388+0000 7f0a3f3e0700 -1 mds.i *** got signal Terminated *** 2024-04-02T14:00:40.518 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:40 smithi188 podman[78248]: 2024-04-02 14:00:40.307412203 +0000 UTC m=+3.015973002 container died c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.build-date=20240326, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-02T14:00:40.884 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:40 smithi188 podman[78248]: 2024-04-02 14:00:40.655220385 +0000 UTC m=+3.363781160 container cleanup c1376978952443c119fb2a7507854638102caccb51847c71ed10ca059bda4246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240326, ceph=True, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-02T14:00:40.884 INFO:journalctl@ceph.mds.i.smithi188.stdout:Apr 02 14:00:40 smithi188 bash[78248]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-i 2024-04-02T14:00:41.024 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.i.service' 2024-04-02T14:00:41.059 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:41.059 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-02T14:00:41.060 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-02T14:00:41.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.l 2024-04-02T14:00:41.517 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:41 smithi188 systemd[1]: Stopping Ceph mds.l for 3c818384-f0f4-11ee-b647-cb9ed24678a4... 2024-04-02T14:00:41.518 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: 2024-04-02T14:00:41.358+0000 7f99f428c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-02T14:00:41.518 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:41 smithi188 ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l[66556]: 2024-04-02T14:00:41.358+0000 7f99f428c700 -1 mds.l *** got signal Terminated *** 2024-04-02T14:00:44.646 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:44 smithi188 podman[78370]: 2024-04-02 14:00:44.30277028 +0000 UTC m=+3.030560558 container died e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240402.004512-191d963, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240326) 2024-04-02T14:00:45.013 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:44 smithi188 podman[78370]: 2024-04-02 14:00:44.742355018 +0000 UTC m=+3.470145251 container cleanup e44ef441bb2312165bea5ab07264994233c4caa9044a599664ea3ae4ff12ffd2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:453f87f941362961d3118868553a25727c89e5b7c0629cbb7db5be9ea34ce534, name=ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240326, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240402.004512-191d963, GIT_BRANCH=HEAD, GIT_COMMIT=12717c0777377369ea674892da98b0d85250f5b0, org.label-schema.vendor=CentOS) 2024-04-02T14:00:45.013 INFO:journalctl@ceph.mds.l.smithi188.stdout:Apr 02 14:00:44 smithi188 bash[78370]: ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4-mds-l 2024-04-02T14:00:45.175 DEBUG:teuthology.orchestra.run.smithi188:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3c818384-f0f4-11ee-b647-cb9ed24678a4@mds.l.service' 2024-04-02T14:00:45.209 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-02T14:00:45.209 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-02T14:00:45.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:00:45.411 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:00:47.935 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:00:48.132 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:00:50.729 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force --keep-logs 2024-04-02T14:00:50.921 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:00:53.352 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:00:53.387 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:00:53.420 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-02T14:00:53.450 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-02T14:00:53.453 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi122/crash 2024-04-02T14:00:53.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:00:53.493 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:00:53.494 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:00:53.495 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi187/crash 2024-04-02T14:00:53.496 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:00:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:tar: /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:00:53.538 INFO:teuthology.orchestra.run.smithi187.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:00:53.540 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi188/crash 2024-04-02T14:00:53.541 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash -- . 2024-04-02T14:00:53.618 INFO:teuthology.orchestra.run.smithi188.stderr:tar: /var/lib/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/crash: Cannot open: No such file or directory 2024-04-02T14:00:53.618 INFO:teuthology.orchestra.run.smithi188.stderr:tar: Error is not recoverable: exiting now 2024-04-02T14:00:53.619 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-02T14:00:53.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-02T14:00:53.702 INFO:tasks.cephadm:Compressing logs... 2024-04-02T14:00:53.702 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:00:53.704 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:00:53.706 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-02T14:00:53.736 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T14:00:53.744 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:00:53.747 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-02T14:00:53.747 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log 2024-04-02T14:00:53.747 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log 2024-04-02T14:00:53.747 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log 2024-04-02T14:00:53.748 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:00:53.750 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:00:53.751 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:00:53.751 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log 2024-04-02T14:00:53.751 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log 2024-04-02T14:00:53.751 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log 2024-04-02T14:00:53.752 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log: 87.7% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:00:53.754 INFO:teuthology.orchestra.run.smithi187.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T14:00:53.755 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:00:53.755 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log 2024-04-02T14:00:53.761 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:gzip/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log: -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log 2024-04-02T14:00:53.762 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:00:53.763 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log 2024-04-02T14:00:53.764 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log 2024-04-02T14:00:53.765 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log: gzip -5 --verbose -- /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log 2024-04-02T14:00:53.766 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log: 92.5% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.z.log.gz 2024-04-02T14:00:53.766 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log: /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log: 92.5% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.y.log.gz 2024-04-02T14:00:53.816 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:00:53.857 INFO:teuthology.orchestra.run.smithi187.stderr: 84.4% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:00:53.866 INFO:teuthology.orchestra.run.smithi122.stderr: 84.7% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:00:53.922 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:00:53.922 INFO:teuthology.orchestra.run.smithi188.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:00:53.934 INFO:teuthology.orchestra.run.smithi187.stderr: 90.4% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:00:53.956 INFO:teuthology.orchestra.run.smithi188.stderr: 84.3% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.cephadm.log.gz 2024-04-02T14:00:53.977 INFO:teuthology.orchestra.run.smithi188.stderr: 90.5% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.audit.log.gz 2024-04-02T14:00:54.005 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:00:54.058 INFO:teuthology.orchestra.run.smithi188.stderr: 87.7% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:00:54.076 INFO:teuthology.orchestra.run.smithi122.stderr: 94.6% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.j.log.gz 2024-04-02T14:00:54.077 INFO:teuthology.orchestra.run.smithi122.stderr: 94.6% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.d.log.gz 2024-04-02T14:00:54.107 INFO:teuthology.orchestra.run.smithi122.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:00:54.114 INFO:teuthology.orchestra.run.smithi187.stderr: 90.9% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-volume.log.gz 2024-04-02T14:00:54.580 INFO:teuthology.orchestra.run.smithi187.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-02T14:00:54.818 INFO:teuthology.orchestra.run.smithi187.stderr: 87.7% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph.log.gz 2024-04-02T14:00:55.955 INFO:teuthology.orchestra.run.smithi122.stderr: 90.4% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mgr.x.log.gz 2024-04-02T14:01:01.286 INFO:teuthology.orchestra.run.smithi188.stderr: 91.9% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.c.log.gz 2024-04-02T14:01:04.887 INFO:teuthology.orchestra.run.smithi122.stderr: 90.6% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.a.log.gz 2024-04-02T14:02:02.554 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mon.b.log.gz 2024-04-02T14:02:11.634 INFO:teuthology.orchestra.run.smithi122.stderr: 95.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.g.log.gz 2024-04-02T14:03:23.321 INFO:teuthology.orchestra.run.smithi188.stderr: 94.9% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.f.log.gz 2024-04-02T14:03:41.275 INFO:teuthology.orchestra.run.smithi188.stderr: 95.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.l.log.gz 2024-04-02T14:03:48.586 INFO:teuthology.orchestra.run.smithi122.stderr: 91.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.6.log.gz 2024-04-02T14:04:15.196 INFO:teuthology.orchestra.run.smithi187.stderr: 95.2% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.e.log.gz 2024-04-02T14:04:20.111 INFO:teuthology.orchestra.run.smithi187.stderr: 95.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.k.log.gz 2024-04-02T14:07:48.467 INFO:teuthology.orchestra.run.smithi187.stderr: 91.3% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.1.log.gz 2024-04-02T14:08:18.890 INFO:teuthology.orchestra.run.smithi187.stderr: 91.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.10.log.gz 2024-04-02T14:09:25.755 INFO:teuthology.orchestra.run.smithi122.stderr: 91.3% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.0.log.gz 2024-04-02T14:09:35.382 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.3.log.gz 2024-04-02T14:09:38.631 INFO:teuthology.orchestra.run.smithi122.stderr: 90.8% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.9.log.gz 2024-04-02T14:09:53.263 INFO:teuthology.orchestra.run.smithi188.stderr: 91.3% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.11.log.gz 2024-04-02T14:10:54.066 INFO:teuthology.orchestra.run.smithi188.stderr: 90.8% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.8.log.gz 2024-04-02T14:11:14.841 INFO:teuthology.orchestra.run.smithi122.stderr: 93.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.a.log.gz 2024-04-02T14:11:14.843 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-04-02T14:11:14.843 INFO:teuthology.orchestra.run.smithi122.stderr:real 10m21.127s 2024-04-02T14:11:14.843 INFO:teuthology.orchestra.run.smithi122.stderr:user 6m4.694s 2024-04-02T14:11:14.843 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m15.299s 2024-04-02T14:11:34.300 INFO:teuthology.orchestra.run.smithi187.stderr: 91.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.7.log.gz 2024-04-02T14:11:44.976 INFO:teuthology.orchestra.run.smithi187.stderr: 91.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.4.log.gz 2024-04-02T14:13:29.774 INFO:teuthology.orchestra.run.smithi188.stderr: 91.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.2.log.gz 2024-04-02T14:14:28.809 INFO:teuthology.orchestra.run.smithi188.stderr: 90.9% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-osd.5.log.gz 2024-04-02T14:19:16.116 INFO:teuthology.orchestra.run.smithi188.stderr: 93.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.i.log.gz 2024-04-02T14:19:38.466 INFO:teuthology.orchestra.run.smithi188.stderr: 93.1% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.c.log.gz 2024-04-02T14:19:38.479 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-04-02T14:19:38.480 INFO:teuthology.orchestra.run.smithi188.stderr:real 18m44.760s 2024-04-02T14:19:38.480 INFO:teuthology.orchestra.run.smithi188.stderr:user 8m29.642s 2024-04-02T14:19:38.480 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m22.163s 2024-04-02T14:22:15.701 INFO:teuthology.orchestra.run.smithi187.stderr: 93.0% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.h.log.gz 2024-04-02T14:22:47.205 INFO:teuthology.orchestra.run.smithi187.stderr: 93.2% -- replaced with /var/log/ceph/3c818384-f0f4-11ee-b647-cb9ed24678a4/ceph-mds.b.log.gz 2024-04-02T14:22:47.234 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-02T14:22:47.234 INFO:teuthology.orchestra.run.smithi187.stderr:real 21m53.504s 2024-04-02T14:22:47.235 INFO:teuthology.orchestra.run.smithi187.stderr:user 9m17.633s 2024-04-02T14:22:47.235 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m25.325s 2024-04-02T14:22:47.235 INFO:tasks.cephadm:Archiving logs... 2024-04-02T14:22:47.236 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi122/log 2024-04-02T14:22:47.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:23:47.854 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi187/log 2024-04-02T14:23:47.855 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:25:15.185 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi188/log 2024-04-02T14:25:15.194 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-02T14:26:38.134 INFO:tasks.cephadm:Removing cluster... 2024-04-02T14:26:38.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:26:38.373 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:26:39.127 DEBUG:teuthology.orchestra.run.smithi187:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:26:39.566 INFO:teuthology.orchestra.run.smithi187.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:26:40.302 DEBUG:teuthology.orchestra.run.smithi188:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3c818384-f0f4-11ee-b647-cb9ed24678a4 --force 2024-04-02T14:26:40.697 INFO:teuthology.orchestra.run.smithi188.stdout:Deleting cluster with fsid: 3c818384-f0f4-11ee-b647-cb9ed24678a4 2024-04-02T14:26:41.537 INFO:tasks.cephadm:Removing cephadm ... 2024-04-02T14:26:41.538 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:26:41.557 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:26:41.576 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-02T14:26:41.591 INFO:tasks.cephadm:Teardown complete 2024-04-02T14:26:41.591 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-02T14:26:41.609 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-02T14:26:41.609 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:26:41.611 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:26:41.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-02T14:26:41.806 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-02T14:26:41.819 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-02T14:26:41.820 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:26:41.822 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:26:41.825 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-02T14:26:41.836 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:26:41.838 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:26:41.839 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-04-02T14:26:41.872 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:26:41.873 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-04-02T14:26:41.873 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 62 -443us[ -443us] +/- 85ms 2024-04-02T14:26:41.873 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv02.front.sepia.ceph.com 3 7 377 63 -1382us[-1374us] +/- 52ms 2024-04-02T14:26:41.873 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +2250us[+2250us] +/- 74ms 2024-04-02T14:26:41.873 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 63 -428us[ -396us] +/- 85ms 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 58 -1443us[-1410us] +/- 52ms 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +2224us[+2256us] +/- 74ms 2024-04-02T14:26:41.963 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:26:41.979 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-02T14:26:41.979 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-04-02T14:26:41.979 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 60 -487us[ -477us] +/- 85ms 2024-04-02T14:26:41.980 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -1450us[-1439us] +/- 52ms 2024-04-02T14:26:41.980 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +2211us[+2221us] +/- 74ms 2024-04-02T14:26:41.980 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-02T14:26:41.981 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-02T14:26:41.993 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-02T14:26:41.994 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-02T14:26:42.006 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:26:42.161 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:26:42.284 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-02T14:26:42.409 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:26:42.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:42.442 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:26:42.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:42.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-02T14:26:42.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:42.511 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-02T14:26:42.521 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-02T14:26:42.535 INFO:teuthology.task.internal:Duration was 4157.237009 seconds 2024-04-02T14:26:42.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-02T14:26:42.548 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-02T14:26:42.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:26:42.551 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:26:42.554 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-02T14:26:42.604 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:26:42.630 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:26:42.631 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-02T14:26:43.075 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-02T14:26:43.075 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-04-02T14:26:43.076 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:26:43.096 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-02T14:26:43.097 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:26:43.119 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-04-02T14:26:43.119 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-02T14:26:43.145 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-02T14:26:43.145 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:26:43.148 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:26:43.162 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-02T14:26:43.227 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-02T14:26:43.228 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:26:43.230 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:26:43.239 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-02T14:26:43.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-02T14:26:43.678 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-02T14:26:43.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:26:43.713 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:26:43.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-02T14:26:43.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-02T14:26:43.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.787 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.789 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.815 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-04-02T14:26:43.831 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-02T14:26:43.849 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-04-02T14:26:43.880 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:43.894 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:43.909 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-02T14:26:43.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-02T14:26:43.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-02T14:26:43.948 INFO:teuthology.task.internal:Transferring archived files... 2024-04-02T14:26:43.949 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi122 2024-04-02T14:26:43.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:26:43.998 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi187 2024-04-02T14:26:43.999 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:26:44.039 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588/remote/smithi188 2024-04-02T14:26:44.039 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-02T14:26:44.079 INFO:teuthology.task.internal:Removing archive directory... 2024-04-02T14:26:44.079 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:26:44.082 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:26:44.084 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-02T14:26:44.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-02T14:26:44.141 INFO:teuthology.task.internal:Not uploading archives. 2024-04-02T14:26:44.141 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-02T14:26:44.154 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-02T14:26:44.154 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:26:44.156 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:26:44.159 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-02T14:26:44.168 INFO:teuthology.orchestra.run.smithi122.stdout: 265306 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:26 /home/ubuntu/cephtest 2024-04-02T14:26:44.171 INFO:teuthology.orchestra.run.smithi187.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:26 /home/ubuntu/cephtest 2024-04-02T14:26:44.182 INFO:teuthology.orchestra.run.smithi188.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 2 14:26 /home/ubuntu/cephtest 2024-04-02T14:26:44.184 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-02T14:26:44.195 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-02T14:26:44.223 INFO:teuthology.nuke:Checking targets against current locks 2024-04-02T14:26:44.249 DEBUG:teuthology.nuke:shortname: smithi122 2024-04-02T14:26:44.249 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:26:44.270 DEBUG:teuthology.nuke:shortname: smithi187 2024-04-02T14:26:44.270 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:26:44.276 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.157021', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNskywgo/DeubM9W/8LlYkUwcBurw16cuyUumgZiyDTe4tKKKoPWITtOwy6JLTYuyUBBA3+r6L3IhIgt1mbGoMY='} 2024-04-02T14:26:44.294 DEBUG:teuthology.nuke:shortname: smithi188 2024-04-02T14:26:44.294 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-02T14:26:44.300 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.158497', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCEPZCnQtkekO28xEN8unj+D/VAtm8pS8j2A1KvqbTiEUQTZ2OV726p29G0WP0FPq0/zCsUr/8MvXZ2hMTUcvMc='} 2024-04-02T14:26:44.325 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/7635588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-02 13:09:11.155260', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMP/0mwIDEySyDMRr37bT2RE0AEaGbpH6ontMNdo44iFCc0s3vvjzsh0qaea0YZuGciJKQFsQk2Cu1ROpTGvnU='} 2024-04-02T14:26:44.336 INFO:teuthology.orchestra.console.smithi122:Power off 2024-04-02T14:26:44.337 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:44.362 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:44.362 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:44.385 INFO:teuthology.orchestra.console.smithi187:Power off 2024-04-02T14:26:44.385 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:44.399 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-02T14:26:44.413 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:44.414 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:44.431 INFO:teuthology.orchestra.console.smithi188:Power off 2024-04-02T14:26:44.432 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-02T14:26:44.442 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-02T14:26:44.455 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-04-02T14:26:44.456 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:44.482 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-04-02T14:26:48.402 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:48.443 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:48.483 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:48.513 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-02T14:26:48.554 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-02T14:26:48.593 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-04-02T14:26:52.519 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:52.555 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:52.594 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:52.630 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-04-02T14:26:52.665 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-02T14:26:52.705 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-04-02T14:26:52.766 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-04-02T14:26:56.635 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:56.705 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-02T14:26:56.747 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-04-02T14:26:56.817 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-04-02T14:26:56.847 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-04-02T14:26:56.918 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-04-02T14:26:57.025 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 4157.237008810043 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi122 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=191d963ac9de7322da8aff20b567b3fa458e2ef6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9baed45056694728bee1f572a597f1ba status: fail success: false 2024-04-02T14:26:57.026 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-02T14:26:57.108 INFO:teuthology.run:FAIL